2024-09-13T08:21:43.006 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T08:21:43.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T08:21:43.131 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319 branch: wip-athakkar-testing-2024-09-12-1849 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: null first_in_suite: false job_id: '7903319' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-athakkar-testing-2024-09-12-1849 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: d454760ef8a93a9dd7f4e4dedfb085faac216e7e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d454760ef8a93a9dd7f4e4dedfb085faac216e7e kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-athakkar-testing-2024-09-12-1849 sha1: d454760ef8a93a9dd7f4e4dedfb085faac216e7e owner: scheduled_khiremat@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6477 sha1: d454760ef8a93a9dd7f4e4dedfb085faac216e7e sleep_before_teardown: 0 subset: 3/4 suite: fs:volumes suite_branch: wip-athakkar-testing-2024-09-12-1849 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d454760ef8a93a9dd7f4e4dedfb085faac216e7e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d454760ef8a93a9dd7f4e4dedfb085faac216e7e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBDdRvWRbk0VlKlnw7uTrP5D7c3O3liJr+aRepiHJTGT8wNouA4Lk82ueVRyHIzNt5B6KiJlRDd/bTjP9kulgY8= smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEcBbt6vGhvBFz8tsw5W/hccsCmEzHENOa/XgFXwX32N5wRc0056tOEpesdyLvJzcu2CCHZxYiHBsXeJD9FBKSo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_08:07:46 tube: smithi user: khiremat verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T08:21:43.132 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d454760ef8a93a9dd7f4e4dedfb085faac216e7e/qa; will attempt to use it 2024-09-13T08:21:43.132 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d454760ef8a93a9dd7f4e4dedfb085faac216e7e/qa/tasks 2024-09-13T08:21:43.133 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T08:21:43.135 INFO:teuthology.task.internal:Checking packages... 2024-09-13T08:21:43.162 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd454760ef8a93a9dd7f4e4dedfb085faac216e7e' 2024-09-13T08:21:43.162 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T08:21:43.162 INFO:teuthology.packaging:ref: None 2024-09-13T08:21:43.162 INFO:teuthology.packaging:tag: None 2024-09-13T08:21:43.162 INFO:teuthology.packaging:branch: wip-athakkar-testing-2024-09-12-1849 2024-09-13T08:21:43.162 INFO:teuthology.packaging:sha1: d454760ef8a93a9dd7f4e4dedfb085faac216e7e 2024-09-13T08:21:43.162 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-athakkar-testing-2024-09-12-1849 2024-09-13T08:21:43.330 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4881-gd454760e-1jammy 2024-09-13T08:21:43.332 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T08:21:43.338 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T08:21:43.338 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T08:21:43.370 INFO:teuthology.task.internal:Saving configuration 2024-09-13T08:21:43.391 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T08:21:43.440 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T08:21:43.470 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 08:16:02.008820', 'locked_by': 'scheduled_khiremat@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIChlGX8T6jILf4dkg8p+4rbMD+cOm1GA2RB67ExS8fcu'} 2024-09-13T08:21:43.502 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 08:16:02.010563', 'locked_by': 'scheduled_khiremat@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICd5HEN2qyv2eN1Gv1vxabrGy2sM9ubFCkt7OiHxuLBu'} 2024-09-13T08:21:43.502 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T08:21:43.508 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-13T08:21:43.508 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-13T08:21:43.508 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T08:21:43.600 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f339aa1bac0>, signals=[15]) 2024-09-13T08:21:43.600 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T08:21:43.605 INFO:teuthology.task.internal:Opening connections... 2024-09-13T08:21:43.605 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:21:43.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:21:43.717 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:21:43.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:21:43.813 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T08:21:43.819 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-09-13T08:21:43.824 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-09-13T08:21:43.825 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-09-13T08:21:43.872 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T08:21:43.872 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="Ubuntu" 2024-09-13T08:21:43.872 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="22.04" 2024-09-13T08:21:43.872 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T08:21:43.872 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_CODENAME=jammy 2024-09-13T08:21:43.872 INFO:teuthology.orchestra.run.smithi044.stdout:ID=ubuntu 2024-09-13T08:21:43.873 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE=debian 2024-09-13T08:21:43.873 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T08:21:43.873 INFO:teuthology.orchestra.run.smithi044.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T08:21:43.873 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T08:21:43.873 INFO:teuthology.orchestra.run.smithi044.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T08:21:43.873 INFO:teuthology.orchestra.run.smithi044.stdout:UBUNTU_CODENAME=jammy 2024-09-13T08:21:43.873 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-09-13T08:21:43.900 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-09-13T08:21:43.905 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-09-13T08:21:43.906 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-09-13T08:21:43.952 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T08:21:43.952 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="Ubuntu" 2024-09-13T08:21:43.952 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="22.04" 2024-09-13T08:21:43.952 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T08:21:43.952 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_CODENAME=jammy 2024-09-13T08:21:43.952 INFO:teuthology.orchestra.run.smithi129.stdout:ID=ubuntu 2024-09-13T08:21:43.953 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE=debian 2024-09-13T08:21:43.953 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T08:21:43.953 INFO:teuthology.orchestra.run.smithi129.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T08:21:43.953 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T08:21:43.953 INFO:teuthology.orchestra.run.smithi129.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T08:21:43.953 INFO:teuthology.orchestra.run.smithi129.stdout:UBUNTU_CODENAME=jammy 2024-09-13T08:21:43.953 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-09-13T08:21:43.975 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T08:21:43.983 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T08:21:44.004 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T08:21:44.004 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-09-13T08:21:44.007 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-09-13T08:21:44.011 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T08:21:44.028 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T08:21:44.029 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-09-13T08:21:44.053 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-09-13T08:21:44.062 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T08:21:44.125 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T08:21:44.145 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-13T08:21:44.146 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-13T08:21:44.146 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-13T08:21:44.146 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T08:21:44.147 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T08:21:44.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:21:44.152 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-13T08:21:44.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:21:44.163 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-13T08:21:44.324 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T08:21:44.325 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-13T08:21:44.325 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-13T08:21:44.330 INFO:teuthology.orchestra.run.smithi129.stdout:5.15.0-56-generic 2024-09-13T08:21:44.331 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T08:21:44.331 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-13T08:21:44.331 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-13T08:21:44.331 DEBUG:teuthology.orchestra.run.smithi129:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-13T08:21:44.333 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T08:21:44.334 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-13T08:21:44.334 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-09-13T08:21:44.339 INFO:teuthology.orchestra.run.smithi044.stdout:5.15.0-56-generic 2024-09-13T08:21:44.340 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T08:21:44.340 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-13T08:21:44.340 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-13T08:21:44.340 DEBUG:teuthology.orchestra.run.smithi044:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-13T08:21:44.387 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-13T08:21:44.414 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-13T08:21:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-13T08:21:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-13T08:21:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-13T08:21:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:hint: 2024-09-13T08:21:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:hint: git config --global init.defaultBranch 2024-09-13T08:21:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:hint: 2024-09-13T08:21:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-13T08:21:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-13T08:21:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:hint: 2024-09-13T08:21:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:hint: git branch -m 2024-09-13T08:21:44.539 INFO:teuthology.orchestra.run.smithi044.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-13T08:21:44.541 DEBUG:teuthology.orchestra.run.smithi044:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-13T08:21:44.544 INFO:teuthology.orchestra.run.smithi129.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-13T08:21:44.544 INFO:teuthology.orchestra.run.smithi129.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-13T08:21:44.544 INFO:teuthology.orchestra.run.smithi129.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-13T08:21:44.544 INFO:teuthology.orchestra.run.smithi129.stderr:hint: 2024-09-13T08:21:44.544 INFO:teuthology.orchestra.run.smithi129.stderr:hint: git config --global init.defaultBranch 2024-09-13T08:21:44.544 INFO:teuthology.orchestra.run.smithi129.stderr:hint: 2024-09-13T08:21:44.544 INFO:teuthology.orchestra.run.smithi129.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-13T08:21:44.545 INFO:teuthology.orchestra.run.smithi129.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-13T08:21:44.545 INFO:teuthology.orchestra.run.smithi129.stderr:hint: 2024-09-13T08:21:44.545 INFO:teuthology.orchestra.run.smithi129.stderr:hint: git branch -m 2024-09-13T08:21:44.546 INFO:teuthology.orchestra.run.smithi129.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-13T08:21:44.548 DEBUG:teuthology.orchestra.run.smithi129:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-13T08:21:44.611 DEBUG:teuthology.orchestra.run.smithi044:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-13T08:21:44.614 DEBUG:teuthology.orchestra.run.smithi129:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-13T08:21:44.666 DEBUG:teuthology.orchestra.run.smithi044:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-13T08:21:44.668 DEBUG:teuthology.orchestra.run.smithi129:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-13T08:22:20.875 INFO:teuthology.orchestra.run.smithi129.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-13T08:22:20.876 INFO:teuthology.orchestra.run.smithi129.stderr: * [new branch] main -> origin/main 2024-09-13T08:22:20.912 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20190312 -> 20190312 2024-09-13T08:22:20.912 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20190416 -> 20190416 2024-09-13T08:22:20.912 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20190514 -> 20190514 2024-09-13T08:22:20.913 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20190618 -> 20190618 2024-09-13T08:22:20.913 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20190717 -> 20190717 2024-09-13T08:22:20.913 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20190815 -> 20190815 2024-09-13T08:22:20.913 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20190923 -> 20190923 2024-09-13T08:22:20.913 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20191022 -> 20191022 2024-09-13T08:22:20.913 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20191215 -> 20191215 2024-09-13T08:22:20.915 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20200122 -> 20200122 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20200316 -> 20200316 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20200421 -> 20200421 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20200519 -> 20200519 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20200619 -> 20200619 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20200721 -> 20200721 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20200817 -> 20200817 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20200918 -> 20200918 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20201022 -> 20201022 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20201118 -> 20201118 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20201218 -> 20201218 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20210208 -> 20210208 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20210315 -> 20210315 2024-09-13T08:22:20.916 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20210511 -> 20210511 2024-09-13T08:22:20.917 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20210716 -> 20210716 2024-09-13T08:22:20.917 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20210818 -> 20210818 2024-09-13T08:22:20.917 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20210919 -> 20210919 2024-09-13T08:22:20.917 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20211027 -> 20211027 2024-09-13T08:22:20.917 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20211216 -> 20211216 2024-09-13T08:22:20.917 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20220209 -> 20220209 2024-09-13T08:22:20.917 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20220310 -> 20220310 2024-09-13T08:22:20.917 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20220411 -> 20220411 2024-09-13T08:22:20.917 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20220509 -> 20220509 2024-09-13T08:22:20.920 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20220610 -> 20220610 2024-09-13T08:22:20.921 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20220708 -> 20220708 2024-09-13T08:22:20.921 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20220815 -> 20220815 2024-09-13T08:22:20.921 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20220913 -> 20220913 2024-09-13T08:22:20.921 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20221012 -> 20221012 2024-09-13T08:22:20.921 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20221109 -> 20221109 2024-09-13T08:22:20.921 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20221214 -> 20221214 2024-09-13T08:22:20.921 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20230117 -> 20230117 2024-09-13T08:22:20.921 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20230210 -> 20230210 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20230310 -> 20230310 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20230404 -> 20230404 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20230515 -> 20230515 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20230625 -> 20230625 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20230804 -> 20230804 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20230919 -> 20230919 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20231030 -> 20231030 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20231111 -> 20231111 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20231211 -> 20231211 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20240115 -> 20240115 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20240220 -> 20240220 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20240312 -> 20240312 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20240410 -> 20240410 2024-09-13T08:22:20.922 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20240513 -> 20240513 2024-09-13T08:22:20.923 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20240610 -> 20240610 2024-09-13T08:22:20.923 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20240709 -> 20240709 2024-09-13T08:22:20.923 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20240811 -> 20240811 2024-09-13T08:22:20.923 INFO:teuthology.orchestra.run.smithi129.stderr: * [new tag] 20240909 -> 20240909 2024-09-13T08:22:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-13T08:22:21.571 INFO:teuthology.orchestra.run.smithi044.stderr: * [new branch] main -> origin/main 2024-09-13T08:22:21.601 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20190312 -> 20190312 2024-09-13T08:22:21.601 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20190416 -> 20190416 2024-09-13T08:22:21.601 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20190514 -> 20190514 2024-09-13T08:22:21.601 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20190618 -> 20190618 2024-09-13T08:22:21.601 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20190717 -> 20190717 2024-09-13T08:22:21.601 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20190815 -> 20190815 2024-09-13T08:22:21.601 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20190923 -> 20190923 2024-09-13T08:22:21.601 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20191022 -> 20191022 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20191215 -> 20191215 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20200122 -> 20200122 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20200316 -> 20200316 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20200421 -> 20200421 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20200519 -> 20200519 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20200619 -> 20200619 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20200721 -> 20200721 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20200817 -> 20200817 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20200918 -> 20200918 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20201022 -> 20201022 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20201118 -> 20201118 2024-09-13T08:22:21.604 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20201218 -> 20201218 2024-09-13T08:22:21.605 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20210208 -> 20210208 2024-09-13T08:22:21.605 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20210315 -> 20210315 2024-09-13T08:22:21.605 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20210511 -> 20210511 2024-09-13T08:22:21.605 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20210716 -> 20210716 2024-09-13T08:22:21.605 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20210818 -> 20210818 2024-09-13T08:22:21.605 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20210919 -> 20210919 2024-09-13T08:22:21.605 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20211027 -> 20211027 2024-09-13T08:22:21.605 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20211216 -> 20211216 2024-09-13T08:22:21.608 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20220209 -> 20220209 2024-09-13T08:22:21.608 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20220310 -> 20220310 2024-09-13T08:22:21.608 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20220411 -> 20220411 2024-09-13T08:22:21.608 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20220509 -> 20220509 2024-09-13T08:22:21.608 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20220610 -> 20220610 2024-09-13T08:22:21.608 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20220708 -> 20220708 2024-09-13T08:22:21.608 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20220815 -> 20220815 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20220913 -> 20220913 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20221012 -> 20221012 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20221109 -> 20221109 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20221214 -> 20221214 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20230117 -> 20230117 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20230210 -> 20230210 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20230310 -> 20230310 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20230404 -> 20230404 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20230515 -> 20230515 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20230625 -> 20230625 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20230804 -> 20230804 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20230919 -> 20230919 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20231030 -> 20231030 2024-09-13T08:22:21.609 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20231111 -> 20231111 2024-09-13T08:22:21.610 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20231211 -> 20231211 2024-09-13T08:22:21.610 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20240115 -> 20240115 2024-09-13T08:22:21.610 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20240220 -> 20240220 2024-09-13T08:22:21.610 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20240312 -> 20240312 2024-09-13T08:22:21.610 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20240410 -> 20240410 2024-09-13T08:22:21.610 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20240513 -> 20240513 2024-09-13T08:22:21.610 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20240610 -> 20240610 2024-09-13T08:22:21.610 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20240709 -> 20240709 2024-09-13T08:22:21.610 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20240811 -> 20240811 2024-09-13T08:22:21.611 INFO:teuthology.orchestra.run.smithi044.stderr: * [new tag] 20240909 -> 20240909 2024-09-13T08:22:25.493 INFO:teuthology.orchestra.run.smithi129.stderr:Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 59% (2077/3520) Updating files: 59% (2079/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 69% (2429/3520) Updating files: 69% (2439/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 82% (2887/3520) Updating files: 82% (2905/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-13T08:22:25.499 INFO:teuthology.orchestra.run.smithi129.stdout:HEAD is now at b9daf8c3 Merge branch 'amd-staging' into 'main' 2024-09-13T08:22:25.521 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-13T08:22:25.522 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T08:22:25.689 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T08:22:25.689 DEBUG:teuthology.orchestra.run.smithi129:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-13T08:22:25.692 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi129.front.sepia.ceph.com to complete... 2024-09-13T08:22:26.456 INFO:teuthology.orchestra.run.smithi044.stderr:Updating files: 29% (1023/3520) Updating files: 30% (1056/3520) Updating files: 31% (1092/3520) Updating files: 32% (1127/3520) Updating files: 33% (1162/3520) Updating files: 34% (1197/3520) Updating files: 35% (1232/3520) Updating files: 36% (1268/3520) Updating files: 37% (1303/3520) Updating files: 38% (1338/3520) Updating files: 39% (1373/3520) Updating files: 40% (1408/3520) Updating files: 41% (1444/3520) Updating files: 42% (1479/3520) Updating files: 43% (1514/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 58% (2057/3520) Updating files: 59% (2077/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 68% (2427/3520) Updating files: 69% (2429/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 81% (2886/3520) Updating files: 82% (2887/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-13T08:22:26.462 INFO:teuthology.orchestra.run.smithi044.stdout:HEAD is now at b9daf8c3 Merge branch 'amd-staging' into 'main' 2024-09-13T08:22:26.487 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-13T08:22:26.487 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-13T08:22:26.636 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T08:22:26.637 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-13T08:22:26.639 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi044.front.sepia.ceph.com to complete... 2024-09-13T08:22:26.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13 08:22:26 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-13T08:22:26.797 INFO:teuthology.orchestra.run.smithi129.stderr:FINISHED --2024-09-13 08:22:26-- 2024-09-13T08:22:26.797 INFO:teuthology.orchestra.run.smithi129.stderr:Total wall clock time: 1.0s 2024-09-13T08:22:26.797 INFO:teuthology.orchestra.run.smithi129.stderr:Downloaded: 1 files, 43M in 0.8s (51.6 MB/s) 2024-09-13T08:22:26.801 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-13T08:22:26.801 DEBUG:teuthology.orchestra.run.smithi129:> sudo dpkg -i /tmp/linux-image.deb 2024-09-13T08:22:27.037 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-13T08:22:27.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13 08:22:28 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-13T08:22:27.694 INFO:teuthology.orchestra.run.smithi044.stderr:FINISHED --2024-09-13 08:22:28-- 2024-09-13T08:22:27.694 INFO:teuthology.orchestra.run.smithi044.stderr:Total wall clock time: 1.0s 2024-09-13T08:22:27.694 INFO:teuthology.orchestra.run.smithi044.stderr:Downloaded: 1 files, 43M in 0.9s (50.6 MB/s) 2024-09-13T08:22:27.698 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-13T08:22:27.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo dpkg -i /tmp/linux-image.deb 2024-09-13T08:22:27.984 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-13T08:22:28.964 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-13T08:22:28.968 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-13T08:22:29.060 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-13T08:22:30.034 INFO:teuthology.orchestra.run.smithi044.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-13T08:22:30.039 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-13T08:22:30.247 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-13T08:22:32.636 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-13T08:22:32.741 INFO:teuthology.orchestra.run.smithi129.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:34.143 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-13T08:22:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:55.566 INFO:teuthology.orchestra.run.smithi129.stderr:Sourcing file `/etc/default/grub' 2024-09-13T08:22:55.574 INFO:teuthology.orchestra.run.smithi129.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T08:22:55.607 INFO:teuthology.orchestra.run.smithi129.stderr:Generating grub configuration file ... 2024-09-13T08:22:55.912 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:55.926 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:56.203 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T08:22:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T08:22:56.270 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T08:22:56.275 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T08:22:56.423 INFO:teuthology.orchestra.run.smithi129.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T08:22:56.423 INFO:teuthology.orchestra.run.smithi129.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T08:22:56.423 INFO:teuthology.orchestra.run.smithi129.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T08:22:56.463 INFO:teuthology.orchestra.run.smithi129.stderr:done 2024-09-13T08:22:56.660 DEBUG:teuthology.orchestra.run.smithi129:> dpkg-deb -c /tmp/linux-image.deb 2024-09-13T08:22:56.668 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-13T08:22:56.668 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-13T08:22:56.668 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:56.686 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:56.686 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:56.709 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-13T08:22:56.709 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-13T08:22:56.710 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-13T08:22:56.710 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-13T08:22:56.710 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-13T08:22:56.710 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-13T08:22:56.710 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-13T08:22:56.710 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-13T08:22:56.710 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T08:22:56.711 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-13T08:22:56.711 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-13T08:22:56.711 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-13T08:22:56.711 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-13T08:22:56.711 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-13T08:22:56.711 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-13T08:22:56.711 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-13T08:22:56.712 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-13T08:22:56.712 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-13T08:22:56.712 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-13T08:22:56.712 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-13T08:22:56.712 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-13T08:22:56.712 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-13T08:22:56.712 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-13T08:22:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-13T08:22:56.714 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-13T08:22:56.714 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-13T08:22:56.714 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-13T08:22:56.714 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-13T08:22:56.714 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-13T08:22:56.714 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-13T08:22:56.714 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-13T08:22:56.714 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-13T08:22:56.714 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-13T08:22:56.724 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-13T08:22:56.724 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-13T08:22:56.724 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-13T08:22:56.725 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-13T08:22:56.726 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-13T08:22:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-13T08:22:56.728 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-13T08:22:56.729 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-13T08:22:56.730 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-13T08:22:56.731 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-13T08:22:56.732 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-13T08:22:56.733 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-13T08:22:56.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-13T08:22:56.735 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-13T08:22:56.800 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-13T08:22:56.801 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-13T08:22:56.802 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-13T08:22:56.802 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-13T08:22:56.802 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-13T08:22:56.802 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-13T08:22:56.802 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-13T08:22:56.802 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-13T08:22:56.802 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-13T08:22:56.803 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-13T08:22:56.803 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-13T08:22:56.803 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-13T08:22:56.803 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-13T08:22:56.803 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-13T08:22:56.803 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-13T08:22:56.804 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-13T08:22:56.804 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-13T08:22:56.804 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-13T08:22:56.804 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-13T08:22:56.804 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-13T08:22:56.804 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-13T08:22:56.804 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-13T08:22:56.805 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-13T08:22:56.805 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-13T08:22:56.805 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-13T08:22:56.805 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-13T08:22:56.805 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-13T08:22:56.805 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-13T08:22:56.806 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-13T08:22:56.806 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-13T08:22:56.806 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-13T08:22:56.806 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-13T08:22:56.806 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-13T08:22:56.806 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-13T08:22:56.806 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-13T08:22:56.806 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-13T08:22:56.807 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-13T08:22:56.807 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-13T08:22:56.807 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-13T08:22:56.807 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-13T08:22:56.807 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-13T08:22:56.807 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-13T08:22:56.808 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-13T08:22:56.808 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-13T08:22:56.808 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-13T08:22:56.808 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-13T08:22:56.808 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-13T08:22:56.808 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-13T08:22:56.808 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-13T08:22:56.809 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-13T08:22:56.809 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-13T08:22:56.809 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-13T08:22:56.809 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-13T08:22:56.809 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-13T08:22:56.810 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-13T08:22:56.810 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-13T08:22:56.810 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-13T08:22:56.810 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-13T08:22:56.810 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-13T08:22:56.810 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-13T08:22:56.810 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-13T08:22:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-13T08:22:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-13T08:22:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-13T08:22:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-13T08:22:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-13T08:22:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-13T08:22:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-13T08:22:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-13T08:22:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-13T08:22:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-13T08:22:56.812 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-13T08:22:56.813 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-13T08:22:56.814 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-13T08:22:56.815 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-13T08:22:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-13T08:22:56.817 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-13T08:22:56.818 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-13T08:22:56.818 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-13T08:22:56.818 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-13T08:22:56.818 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-13T08:22:56.818 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-13T08:22:56.818 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-13T08:22:56.818 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-13T08:22:56.818 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-13T08:22:56.818 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-13T08:22:56.819 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-13T08:22:56.820 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-13T08:22:56.821 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-13T08:22:56.822 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-13T08:22:56.822 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-13T08:22:56.822 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-13T08:22:56.822 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-13T08:22:56.822 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-13T08:22:56.822 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-13T08:22:56.822 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-13T08:22:56.822 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-13T08:22:56.822 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-13T08:22:56.823 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-13T08:22:56.824 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-13T08:22:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-13T08:22:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-13T08:22:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-13T08:22:56.828 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-13T08:22:56.828 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-13T08:22:56.828 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-13T08:22:56.828 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-13T08:22:56.828 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-13T08:22:56.828 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-13T08:22:56.828 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-13T08:22:56.828 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-13T08:22:56.828 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-13T08:22:56.828 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-13T08:22:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-13T08:22:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-13T08:22:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-13T08:22:56.832 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-13T08:22:56.833 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-13T08:22:56.833 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-13T08:22:56.833 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-13T08:22:56.833 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-13T08:22:56.833 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-13T08:22:56.833 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-13T08:22:56.833 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-13T08:22:56.833 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-13T08:22:56.833 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-13T08:22:56.834 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-13T08:22:56.835 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-13T08:22:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-13T08:22:56.837 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-13T08:22:56.838 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-13T08:22:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-13T08:22:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-13T08:22:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-13T08:22:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-13T08:22:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-13T08:22:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-13T08:22:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-13T08:22:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-13T08:22:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-13T08:22:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-13T08:22:56.840 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-13T08:22:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-13T08:22:56.842 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-13T08:22:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-13T08:22:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-13T08:22:56.845 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-13T08:22:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-13T08:22:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-13T08:22:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-13T08:22:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-13T08:22:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-13T08:22:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-13T08:22:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-13T08:22:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-13T08:22:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-13T08:22:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-13T08:22:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-13T08:22:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-13T08:22:56.849 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-13T08:22:56.849 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-13T08:22:56.849 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-13T08:22:56.849 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-13T08:22:56.849 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-13T08:22:56.849 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-13T08:22:56.849 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-13T08:22:56.849 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-13T08:22:56.849 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-13T08:22:56.849 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-13T08:22:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-13T08:22:56.851 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-13T08:22:56.851 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-13T08:22:56.851 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-13T08:22:56.851 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-13T08:22:56.851 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-13T08:22:56.851 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-13T08:22:56.851 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-13T08:22:56.851 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-13T08:22:56.851 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-13T08:22:56.852 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-13T08:22:56.852 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-13T08:22:56.852 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-13T08:22:56.852 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-13T08:22:56.852 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-13T08:22:56.852 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-13T08:22:56.852 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-13T08:22:56.852 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-13T08:22:56.852 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-13T08:22:56.852 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-13T08:22:56.853 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-13T08:22:56.854 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-13T08:22:56.854 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-13T08:22:56.854 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-13T08:22:56.854 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-13T08:22:56.854 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-13T08:22:56.854 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-13T08:22:56.854 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-13T08:22:56.854 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-13T08:22:56.854 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-13T08:22:56.854 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-13T08:22:56.855 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-13T08:22:56.856 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-13T08:22:56.857 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-13T08:22:56.858 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-13T08:22:56.859 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-13T08:22:56.860 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-13T08:22:56.861 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-13T08:22:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-13T08:22:56.863 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-13T08:22:56.864 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-13T08:22:56.864 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-13T08:22:56.864 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-13T08:22:56.864 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-13T08:22:56.864 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-13T08:22:56.864 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-13T08:22:56.864 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-13T08:22:56.864 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-13T08:22:56.938 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-13T08:22:56.938 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-13T08:22:56.938 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-13T08:22:56.938 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-13T08:22:56.938 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-13T08:22:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-13T08:22:56.940 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-13T08:22:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-13T08:22:56.942 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-13T08:22:56.943 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-13T08:22:56.944 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-13T08:22:56.944 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-13T08:22:56.944 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-13T08:22:56.944 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-13T08:22:56.944 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-13T08:22:56.944 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-13T08:22:56.944 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-13T08:22:56.944 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-13T08:22:56.944 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-13T08:22:56.944 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-13T08:22:56.945 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-13T08:22:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-13T08:22:56.947 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-13T08:22:56.948 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-13T08:22:56.948 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-13T08:22:56.948 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-13T08:22:56.948 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-13T08:22:56.948 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-13T08:22:56.948 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-13T08:22:56.948 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-13T08:22:56.948 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-13T08:22:56.948 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-13T08:22:56.948 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-13T08:22:56.949 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-13T08:22:56.950 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-13T08:22:56.951 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-13T08:22:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-13T08:22:56.953 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-13T08:22:56.954 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-13T08:22:56.954 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-13T08:22:56.954 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-13T08:22:56.954 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-13T08:22:56.954 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-13T08:22:56.954 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-13T08:22:56.954 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-13T08:22:56.954 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-13T08:22:56.954 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-13T08:22:56.954 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-13T08:22:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-13T08:22:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-13T08:22:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-13T08:22:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-13T08:22:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-13T08:22:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-13T08:22:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-13T08:22:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-13T08:22:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-13T08:22:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-13T08:22:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-13T08:22:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-13T08:22:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-13T08:22:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-13T08:22:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-13T08:22:56.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-13T08:22:56.961 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-13T08:22:56.961 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-13T08:22:56.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-13T08:22:56.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-13T08:22:56.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-13T08:22:56.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-13T08:22:56.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-13T08:22:56.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-13T08:22:56.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-13T08:22:56.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-13T08:22:56.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-13T08:22:56.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-13T08:22:56.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-13T08:22:56.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-13T08:22:56.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-13T08:22:56.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-13T08:22:56.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-13T08:22:56.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-13T08:22:56.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-13T08:22:56.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-13T08:22:56.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-13T08:22:56.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-13T08:22:56.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-13T08:22:56.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-13T08:22:56.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-13T08:22:56.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-13T08:22:56.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-13T08:22:56.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-13T08:22:56.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-13T08:22:56.969 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-13T08:22:56.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-13T08:22:56.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-13T08:22:56.969 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-13T08:22:56.969 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-13T08:22:56.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-13T08:22:56.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-13T08:22:56.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-13T08:22:56.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-13T08:22:56.970 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-13T08:22:56.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-13T08:22:56.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-13T08:22:56.971 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-13T08:22:56.971 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-13T08:22:56.971 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-13T08:22:56.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-13T08:22:56.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-13T08:22:56.972 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-13T08:22:56.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-13T08:22:56.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-13T08:22:56.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-13T08:22:56.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-13T08:22:56.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-13T08:22:56.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-13T08:22:56.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-13T08:22:56.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-13T08:22:56.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-13T08:22:56.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-13T08:22:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-13T08:22:56.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-13T08:22:56.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-13T08:22:56.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-13T08:22:56.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-13T08:22:56.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-13T08:22:56.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-13T08:22:56.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-13T08:22:56.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-13T08:22:56.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-13T08:22:56.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-13T08:22:56.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-13T08:22:56.977 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-13T08:22:56.978 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-13T08:22:56.979 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-13T08:22:56.980 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-13T08:22:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-13T08:22:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-13T08:22:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-13T08:22:56.984 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-13T08:22:56.984 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-13T08:22:56.984 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-13T08:22:56.984 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-13T08:22:56.984 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-13T08:22:56.984 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-13T08:22:56.984 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-13T08:22:56.984 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-13T08:22:56.984 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-13T08:22:56.984 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-13T08:22:56.985 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-13T08:22:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-13T08:22:56.987 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-13T08:22:56.988 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-13T08:22:56.989 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-13T08:22:56.990 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-13T08:22:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-13T08:22:56.992 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-13T08:22:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-13T08:22:56.994 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-13T08:22:56.994 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-13T08:22:56.994 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-13T08:22:56.994 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-13T08:22:56.994 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-13T08:22:56.994 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-13T08:22:56.994 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-13T08:22:56.994 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-13T08:22:56.994 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-13T08:22:56.994 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-13T08:22:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-13T08:22:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-13T08:22:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-13T08:22:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-13T08:22:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-13T08:22:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-13T08:22:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-13T08:22:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-13T08:22:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-13T08:22:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-13T08:22:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-13T08:22:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-13T08:22:56.998 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-13T08:22:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-13T08:22:57.000 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-13T08:22:57.000 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-13T08:22:57.000 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-13T08:22:57.000 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-13T08:22:57.000 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-13T08:22:57.000 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-13T08:22:57.000 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-13T08:22:57.000 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-13T08:22:57.000 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-13T08:22:57.001 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-13T08:22:57.001 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-13T08:22:57.001 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-13T08:22:57.001 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-13T08:22:57.001 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-13T08:22:57.001 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-13T08:22:57.001 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-13T08:22:57.002 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-13T08:22:57.002 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-13T08:22:57.002 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-13T08:22:57.002 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-13T08:22:57.002 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-13T08:22:57.002 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-13T08:22:57.003 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-13T08:22:57.003 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-13T08:22:57.003 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-13T08:22:57.003 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-13T08:22:57.003 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-13T08:22:57.003 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-13T08:22:57.003 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-13T08:22:57.004 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-13T08:22:57.004 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-13T08:22:57.004 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-13T08:22:57.004 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-13T08:22:57.004 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-13T08:22:57.004 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-13T08:22:57.005 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-13T08:22:57.005 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-13T08:22:57.005 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-13T08:22:57.005 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-13T08:22:57.005 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-13T08:22:57.005 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-13T08:22:57.005 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-13T08:22:57.006 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-13T08:22:57.006 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-13T08:22:57.006 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-13T08:22:57.006 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-13T08:22:57.006 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-13T08:22:57.006 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-13T08:22:57.006 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-13T08:22:57.006 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-13T08:22:57.006 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-13T08:22:57.006 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-13T08:22:57.007 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-13T08:22:57.008 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-13T08:22:57.009 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-13T08:22:57.010 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-13T08:22:57.011 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-13T08:22:57.012 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-13T08:22:57.012 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-13T08:22:57.012 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-13T08:22:57.012 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-13T08:22:57.012 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-13T08:22:57.012 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-13T08:22:57.012 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-13T08:22:57.012 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-13T08:22:57.012 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-13T08:22:57.012 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-13T08:22:57.013 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-13T08:22:57.014 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-13T08:22:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-13T08:22:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-13T08:22:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-13T08:22:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-13T08:22:57.019 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-13T08:22:57.020 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-13T08:22:57.021 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-13T08:22:57.022 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-13T08:22:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-13T08:22:57.024 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-13T08:22:57.024 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-13T08:22:57.069 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-13T08:22:57.069 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-13T08:22:57.069 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-13T08:22:57.070 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-13T08:22:57.071 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-13T08:22:57.071 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-13T08:22:57.071 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-13T08:22:57.071 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-13T08:22:57.071 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-13T08:22:57.071 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-13T08:22:57.071 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-13T08:22:57.071 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-13T08:22:57.071 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-13T08:22:57.071 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-13T08:22:57.072 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-13T08:22:57.073 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-13T08:22:57.074 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-13T08:22:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-13T08:22:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-13T08:22:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-13T08:22:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-13T08:22:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-13T08:22:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-13T08:22:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-13T08:22:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-13T08:22:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-13T08:22:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-13T08:22:57.076 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-13T08:22:57.077 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-13T08:22:57.077 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-13T08:22:57.077 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-13T08:22:57.077 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-13T08:22:57.077 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-13T08:22:57.077 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-13T08:22:57.077 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-13T08:22:57.077 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-13T08:22:57.077 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-13T08:22:57.078 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-13T08:22:57.079 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-13T08:22:57.079 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-13T08:22:57.079 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-13T08:22:57.079 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-13T08:22:57.079 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-13T08:22:57.079 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-13T08:22:57.079 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-13T08:22:57.079 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-13T08:22:57.079 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-13T08:22:57.080 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-13T08:22:57.081 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-13T08:22:57.082 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-13T08:22:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-13T08:22:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-13T08:22:57.085 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-13T08:22:57.086 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-13T08:22:57.087 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-13T08:22:57.088 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-13T08:22:57.089 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-13T08:22:57.089 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-13T08:22:57.089 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-13T08:22:57.089 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-13T08:22:57.089 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-13T08:22:57.089 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-13T08:22:57.089 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-13T08:22:57.089 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-13T08:22:57.089 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-13T08:22:57.089 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-13T08:22:57.090 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-13T08:22:57.091 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-13T08:22:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-13T08:22:57.093 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-13T08:22:57.094 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-13T08:22:57.095 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-13T08:22:57.096 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-13T08:22:57.097 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-13T08:22:57.098 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-13T08:22:57.098 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-13T08:22:57.098 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-13T08:22:57.098 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-13T08:22:57.098 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-13T08:22:57.098 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-13T08:22:57.098 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-13T08:22:57.098 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-13T08:22:57.098 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-13T08:22:57.098 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-13T08:22:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-13T08:22:57.100 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-13T08:22:57.100 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-13T08:22:57.100 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-13T08:22:57.100 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-13T08:22:57.100 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-13T08:22:57.100 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-13T08:22:57.100 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-13T08:22:57.100 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-13T08:22:57.100 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-13T08:22:57.101 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-13T08:22:57.102 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-13T08:22:57.103 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-13T08:22:57.104 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-13T08:22:57.105 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-13T08:22:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-13T08:22:57.107 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-13T08:22:57.108 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-13T08:22:57.108 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-13T08:22:57.108 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-13T08:22:57.108 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-13T08:22:57.108 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-13T08:22:57.108 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-13T08:22:57.108 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-13T08:22:57.108 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-13T08:22:57.108 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-13T08:22:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-13T08:22:57.110 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-13T08:22:57.110 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-13T08:22:57.110 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-13T08:22:57.110 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-13T08:22:57.110 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-13T08:22:57.110 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-13T08:22:57.110 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-13T08:22:57.110 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-13T08:22:57.110 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-13T08:22:57.110 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-13T08:22:57.111 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-13T08:22:57.112 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-13T08:22:57.113 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-13T08:22:57.113 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-13T08:22:57.113 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-13T08:22:57.113 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-13T08:22:57.113 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-13T08:22:57.113 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-13T08:22:57.113 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-13T08:22:57.114 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-13T08:22:57.114 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-13T08:22:57.114 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-13T08:22:57.114 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-13T08:22:57.114 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-13T08:22:57.114 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-13T08:22:57.115 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-13T08:22:57.115 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-13T08:22:57.115 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-13T08:22:57.115 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-13T08:22:57.115 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-13T08:22:57.115 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-13T08:22:57.116 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-13T08:22:57.116 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-13T08:22:57.116 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-13T08:22:57.116 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-13T08:22:57.116 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-13T08:22:57.116 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-13T08:22:57.117 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-13T08:22:57.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-13T08:22:57.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-13T08:22:57.117 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-13T08:22:57.117 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-13T08:22:57.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-13T08:22:57.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-13T08:22:57.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-13T08:22:57.118 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-13T08:22:57.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-13T08:22:57.118 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-13T08:22:57.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-13T08:22:57.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-13T08:22:57.119 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-13T08:22:57.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-13T08:22:57.119 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-13T08:22:57.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-13T08:22:57.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-13T08:22:57.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-13T08:22:57.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-13T08:22:57.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-13T08:22:57.120 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-13T08:22:57.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-13T08:22:57.121 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-13T08:22:57.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-13T08:22:57.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-13T08:22:57.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-13T08:22:57.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-13T08:22:57.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-13T08:22:57.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-13T08:22:57.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-13T08:22:57.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-13T08:22:57.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-13T08:22:57.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-13T08:22:57.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-13T08:22:57.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-13T08:22:57.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-13T08:22:57.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-13T08:22:57.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-13T08:22:57.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-13T08:22:57.123 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-13T08:22:57.124 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-13T08:22:57.124 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-13T08:22:57.124 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T08:22:57.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-13T08:22:57.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-13T08:22:57.132 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:57.132 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:57.132 DEBUG:teuthology.orchestra.run.smithi129:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-13T08:22:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-13T08:22:57.138 INFO:teuthology.orchestra.run.smithi129.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-13T08:22:57.138 INFO:teuthology.orchestra.run.smithi129.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-13T08:22:57.139 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-13T08:22:57.139 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:57.139 DEBUG:teuthology.orchestra.run.smithi129:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-13T08:22:57.184 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi129.front.sepia.ceph.com to complete... 2024-09-13T08:22:57.665 INFO:teuthology.orchestra.run.smithi129.stderr:Sourcing file `/etc/default/grub' 2024-09-13T08:22:57.666 INFO:teuthology.orchestra.run.smithi129.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T08:22:57.682 INFO:teuthology.orchestra.run.smithi129.stderr:Generating grub configuration file ... 2024-09-13T08:22:57.889 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:57.896 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:58.092 INFO:teuthology.orchestra.run.smithi044.stderr:Sourcing file `/etc/default/grub' 2024-09-13T08:22:58.110 INFO:teuthology.orchestra.run.smithi044.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T08:22:58.132 INFO:teuthology.orchestra.run.smithi044.stderr:Generating grub configuration file ... 2024-09-13T08:22:58.135 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T08:22:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T08:22:58.183 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T08:22:58.188 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T08:22:58.312 INFO:teuthology.orchestra.run.smithi129.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T08:22:58.313 INFO:teuthology.orchestra.run.smithi129.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T08:22:58.313 INFO:teuthology.orchestra.run.smithi129.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T08:22:58.323 INFO:teuthology.orchestra.run.smithi129.stderr:done 2024-09-13T08:22:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:58.813 INFO:teuthology.orchestra.run.smithi044.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T08:22:58.820 INFO:teuthology.orchestra.run.smithi044.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T08:22:58.884 INFO:teuthology.orchestra.run.smithi044.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T08:22:58.890 INFO:teuthology.orchestra.run.smithi044.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T08:22:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T08:22:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T08:22:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T08:22:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:done 2024-09-13T08:22:59.267 DEBUG:teuthology.orchestra.run.smithi044:> dpkg-deb -c /tmp/linux-image.deb 2024-09-13T08:22:59.275 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-13T08:22:59.275 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-13T08:22:59.276 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:59.296 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:59.296 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-13T08:22:59.319 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-13T08:22:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-13T08:22:59.321 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-13T08:22:59.321 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-13T08:22:59.321 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-13T08:22:59.321 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-13T08:22:59.321 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-13T08:22:59.322 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-13T08:22:59.322 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-13T08:22:59.322 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-13T08:22:59.322 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-13T08:22:59.322 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-13T08:22:59.322 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-13T08:22:59.322 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-13T08:22:59.323 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-13T08:22:59.323 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-13T08:22:59.323 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-13T08:22:59.323 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-13T08:22:59.323 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-13T08:22:59.323 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-13T08:22:59.323 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-13T08:22:59.338 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-13T08:22:59.339 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-13T08:22:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-13T08:22:59.341 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-13T08:22:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-13T08:22:59.343 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-13T08:22:59.344 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-13T08:22:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-13T08:22:59.346 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-13T08:22:59.347 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-13T08:22:59.348 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-13T08:22:59.349 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-13T08:22:59.349 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-13T08:22:59.349 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-13T08:22:59.349 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-13T08:22:59.349 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-13T08:22:59.349 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-13T08:22:59.349 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-13T08:22:59.349 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-13T08:22:59.349 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-13T08:22:59.376 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-13T08:22:59.376 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-13T08:22:59.376 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-13T08:22:59.376 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-13T08:22:59.377 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-13T08:22:59.378 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-13T08:22:59.379 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-13T08:22:59.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-13T08:22:59.381 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-13T08:22:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-13T08:22:59.383 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-13T08:22:59.384 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-13T08:22:59.385 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-13T08:22:59.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-13T08:22:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-13T08:22:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-13T08:22:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-13T08:22:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-13T08:22:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-13T08:22:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-13T08:22:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-13T08:22:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-13T08:22:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-13T08:22:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-13T08:22:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-13T08:22:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-13T08:22:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-13T08:22:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-13T08:22:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-13T08:22:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-13T08:22:59.403 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-13T08:22:59.403 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-13T08:22:59.403 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-13T08:22:59.403 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-13T08:22:59.403 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-13T08:22:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-13T08:22:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-13T08:22:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-13T08:22:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-13T08:22:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-13T08:22:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-13T08:22:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-13T08:22:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-13T08:22:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-13T08:22:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-13T08:22:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-13T08:22:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-13T08:22:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-13T08:22:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-13T08:22:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-13T08:22:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-13T08:22:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-13T08:22:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-13T08:22:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-13T08:22:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-13T08:22:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-13T08:22:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-13T08:22:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-13T08:22:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-13T08:22:59.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-13T08:22:59.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-13T08:22:59.449 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-13T08:22:59.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-13T08:22:59.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-13T08:22:59.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-13T08:22:59.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-13T08:22:59.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-13T08:22:59.460 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-13T08:22:59.460 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-13T08:22:59.460 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-13T08:22:59.460 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-13T08:22:59.460 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-13T08:22:59.460 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-13T08:22:59.460 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-13T08:22:59.460 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-13T08:22:59.460 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-13T08:22:59.461 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-13T08:22:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-13T08:22:59.463 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-13T08:22:59.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-13T08:22:59.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-13T08:22:59.466 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-13T08:22:59.466 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-13T08:22:59.466 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-13T08:22:59.468 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-13T08:22:59.468 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-13T08:22:59.469 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-13T08:22:59.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-13T08:22:59.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-13T08:22:59.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-13T08:22:59.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-13T08:22:59.471 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-13T08:22:59.471 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-13T08:22:59.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-13T08:22:59.471 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-13T08:22:59.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-13T08:22:59.473 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-13T08:22:59.474 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-13T08:22:59.474 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-13T08:22:59.474 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-13T08:22:59.474 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-13T08:22:59.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-13T08:22:59.477 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-13T08:22:59.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-13T08:22:59.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-13T08:22:59.477 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-13T08:22:59.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-13T08:22:59.477 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-13T08:22:59.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-13T08:22:59.477 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-13T08:22:59.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-13T08:22:59.482 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-13T08:22:59.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-13T08:22:59.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-13T08:22:59.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-13T08:22:59.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-13T08:22:59.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-13T08:22:59.488 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-13T08:22:59.488 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-13T08:22:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-13T08:22:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-13T08:22:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-13T08:22:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-13T08:22:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-13T08:22:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-13T08:22:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-13T08:22:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-13T08:22:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-13T08:22:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-13T08:22:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-13T08:22:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-13T08:22:59.505 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-13T08:22:59.506 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-13T08:22:59.507 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-13T08:22:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-13T08:22:59.509 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-13T08:22:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-13T08:22:59.511 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-13T08:22:59.512 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-13T08:22:59.513 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-13T08:22:59.514 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-13T08:22:59.515 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-13T08:22:59.516 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-13T08:22:59.516 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-13T08:22:59.516 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-13T08:22:59.516 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-13T08:22:59.519 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-13T08:22:59.519 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-13T08:22:59.519 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-13T08:22:59.519 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-13T08:22:59.519 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-13T08:22:59.519 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-13T08:22:59.520 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-13T08:22:59.521 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-13T08:22:59.522 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-13T08:22:59.523 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-13T08:22:59.523 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-13T08:22:59.523 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-13T08:22:59.523 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-13T08:22:59.524 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-13T08:22:59.524 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-13T08:22:59.524 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-13T08:22:59.524 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-13T08:22:59.524 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-13T08:22:59.525 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-13T08:22:59.525 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-13T08:22:59.525 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-13T08:22:59.525 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-13T08:22:59.525 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-13T08:22:59.525 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-13T08:22:59.526 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-13T08:22:59.526 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-13T08:22:59.526 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-13T08:22:59.526 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-13T08:22:59.527 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-13T08:22:59.527 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-13T08:22:59.527 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-13T08:22:59.532 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-13T08:22:59.532 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-13T08:22:59.532 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-13T08:22:59.533 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-13T08:22:59.533 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-13T08:22:59.533 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-13T08:22:59.534 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-13T08:22:59.534 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-13T08:22:59.535 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-13T08:22:59.535 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-13T08:22:59.535 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-13T08:22:59.535 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-13T08:22:59.535 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-13T08:22:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-13T08:22:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-13T08:22:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-13T08:22:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-13T08:22:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-13T08:22:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-13T08:22:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-13T08:22:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-13T08:22:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-13T08:22:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-13T08:22:59.538 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-13T08:22:59.538 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-13T08:22:59.539 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-13T08:22:59.539 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-13T08:22:59.539 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-13T08:22:59.539 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-13T08:22:59.539 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-13T08:22:59.541 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-13T08:22:59.541 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-13T08:22:59.545 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-13T08:22:59.546 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-13T08:22:59.547 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-13T08:22:59.548 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-13T08:22:59.554 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-13T08:22:59.554 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-13T08:22:59.555 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-13T08:22:59.556 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-13T08:22:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-13T08:22:59.558 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-13T08:22:59.559 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-13T08:22:59.560 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-13T08:22:59.561 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-13T08:22:59.561 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-13T08:22:59.561 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-13T08:22:59.561 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-13T08:22:59.561 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-13T08:22:59.561 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-13T08:22:59.561 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-13T08:22:59.561 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-13T08:22:59.578 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-13T08:22:59.579 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-13T08:22:59.580 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-13T08:22:59.581 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-13T08:22:59.582 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-13T08:22:59.583 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-13T08:22:59.584 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-13T08:22:59.585 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-13T08:22:59.586 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-13T08:22:59.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-13T08:22:59.588 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-13T08:22:59.589 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-13T08:22:59.590 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-13T08:22:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-13T08:22:59.592 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-13T08:22:59.593 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-13T08:22:59.594 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-13T08:22:59.595 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-13T08:22:59.596 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-13T08:22:59.597 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-13T08:22:59.597 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-13T08:22:59.597 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-13T08:22:59.597 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-13T08:22:59.597 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-13T08:22:59.599 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-13T08:22:59.599 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-13T08:22:59.599 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-13T08:22:59.599 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-13T08:22:59.599 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-13T08:22:59.600 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-13T08:22:59.601 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-13T08:22:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-13T08:22:59.603 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-13T08:22:59.603 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-13T08:22:59.603 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-13T08:22:59.603 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-13T08:22:59.603 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-13T08:22:59.603 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-13T08:22:59.606 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-13T08:22:59.606 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-13T08:22:59.606 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-13T08:22:59.606 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-13T08:22:59.606 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-13T08:22:59.610 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-13T08:22:59.610 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-13T08:22:59.613 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-13T08:22:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-13T08:22:59.615 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-13T08:22:59.616 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-13T08:22:59.617 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-13T08:22:59.618 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-13T08:22:59.618 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-13T08:22:59.618 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-13T08:22:59.618 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-13T08:22:59.618 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-13T08:22:59.619 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-13T08:22:59.619 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-13T08:22:59.619 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-13T08:22:59.619 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-13T08:22:59.625 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-13T08:22:59.625 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-13T08:22:59.626 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-13T08:22:59.627 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-13T08:22:59.627 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-13T08:22:59.627 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-13T08:22:59.627 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-13T08:22:59.627 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-13T08:22:59.627 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-13T08:22:59.627 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-13T08:22:59.627 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-13T08:22:59.627 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-13T08:22:59.630 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-13T08:22:59.630 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-13T08:22:59.630 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-13T08:22:59.630 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-13T08:22:59.630 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-13T08:22:59.630 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-13T08:22:59.630 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-13T08:22:59.630 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-13T08:22:59.630 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-13T08:22:59.631 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-13T08:22:59.631 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-13T08:22:59.631 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-13T08:22:59.631 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-13T08:22:59.641 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-13T08:22:59.641 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-13T08:22:59.642 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-13T08:22:59.643 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-13T08:22:59.644 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-13T08:22:59.644 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-13T08:22:59.644 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-13T08:22:59.644 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-13T08:22:59.646 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-13T08:22:59.646 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-13T08:22:59.647 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-13T08:22:59.648 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-13T08:22:59.649 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-13T08:22:59.649 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-13T08:22:59.649 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-13T08:22:59.649 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-13T08:22:59.649 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-13T08:22:59.649 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-13T08:22:59.649 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-13T08:22:59.651 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-13T08:22:59.652 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-13T08:22:59.653 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-13T08:22:59.653 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-13T08:22:59.653 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-13T08:22:59.653 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-13T08:22:59.662 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-13T08:22:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-13T08:22:59.664 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-13T08:22:59.665 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-13T08:22:59.666 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-13T08:22:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-13T08:22:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-13T08:22:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-13T08:22:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-13T08:22:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-13T08:22:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-13T08:22:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-13T08:22:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-13T08:22:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-13T08:22:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-13T08:22:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-13T08:22:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-13T08:22:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-13T08:22:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-13T08:22:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-13T08:22:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-13T08:22:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-13T08:22:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-13T08:22:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-13T08:22:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-13T08:22:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-13T08:22:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-13T08:22:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-13T08:22:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-13T08:22:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-13T08:22:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-13T08:22:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-13T08:22:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-13T08:22:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-13T08:22:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-13T08:22:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-13T08:22:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-13T08:22:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-13T08:22:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-13T08:22:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-13T08:22:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-13T08:22:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-13T08:22:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-13T08:22:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-13T08:22:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-13T08:22:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-13T08:22:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-13T08:22:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-13T08:22:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-13T08:22:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-13T08:22:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-13T08:22:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-13T08:22:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-13T08:22:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-13T08:22:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-13T08:22:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-13T08:22:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-13T08:22:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-13T08:22:59.714 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:59.714 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:59.714 DEBUG:teuthology.orchestra.run.smithi044:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-13T08:22:59.761 INFO:teuthology.orchestra.run.smithi044.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-13T08:22:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-13T08:22:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-13T08:22:59.762 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-13T08:22:59.762 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:22:59.762 DEBUG:teuthology.orchestra.run.smithi044:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-13T08:22:59.805 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi044.front.sepia.ceph.com to complete... 2024-09-13T08:23:00.233 INFO:teuthology.orchestra.run.smithi044.stderr:Sourcing file `/etc/default/grub' 2024-09-13T08:23:00.237 INFO:teuthology.orchestra.run.smithi044.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T08:23:00.243 INFO:teuthology.orchestra.run.smithi044.stderr:Generating grub configuration file ... 2024-09-13T08:23:00.461 INFO:teuthology.orchestra.run.smithi044.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:23:00.467 INFO:teuthology.orchestra.run.smithi044.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:23:00.725 INFO:teuthology.orchestra.run.smithi044.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T08:23:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T08:23:00.771 INFO:teuthology.orchestra.run.smithi044.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T08:23:00.776 INFO:teuthology.orchestra.run.smithi044.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T08:23:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T08:23:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T08:23:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T08:23:00.902 INFO:teuthology.orchestra.run.smithi044.stderr:done 2024-09-13T08:23:29.481 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-13T08:23:29.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T08:23:29.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:23:31.995 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-13T08:23:31.996 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T08:23:31.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:23:48.029 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-13T08:23:50.333 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-09-13T08:23:57.037 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T08:23:57.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:23:59.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T08:23:59.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:24:00.125 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-13T08:24:12.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T08:24:12.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:24:30.525 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-13T08:24:45.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T08:24:45.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:24:46.206 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-13T08:24:47.242 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T08:24:47.242 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-13T08:24:47.243 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-13T08:24:47.250 INFO:teuthology.orchestra.run.smithi129.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:24:47.250 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:24:47.251 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T08:24:47.251 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T08:24:48.252 DEBUG:teuthology.parallel:result is None 2024-09-13T08:24:59.347 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T08:25:11.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T08:25:11.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:25:11.506 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-09-13T08:25:12.081 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T08:25:12.081 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-13T08:25:12.082 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-09-13T08:25:12.134 INFO:teuthology.orchestra.run.smithi044.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:25:12.134 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-13T08:25:12.135 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T08:25:12.135 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T08:25:13.139 DEBUG:teuthology.parallel:result is None 2024-09-13T08:25:13.139 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T08:25:13.148 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T08:25:13.148 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T08:25:13.152 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T08:25:13.161 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T08:25:13.202 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T08:25:13.252 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T08:25:13.252 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T08:25:13.255 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T08:25:13.276 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T08:25:13.302 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T08:25:13.302 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T08:25:13.312 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T08:25:13.418 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T08:25:13.418 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T08:25:13.437 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T08:25:13.440 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T08:25:13.443 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T08:25:13.451 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T08:25:13.451 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T08:25:13.455 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T08:25:13.476 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T08:25:13.491 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T08:25:13.491 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T08:25:13.521 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T08:25:13.529 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T08:25:13.578 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T08:25:13.631 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:25:13.631 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T08:25:13.695 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T08:25:13.704 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T08:25:13.754 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:25:13.755 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T08:25:13.837 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-09-13T08:25:13.841 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-09-13T08:25:13.960 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T08:25:13.969 INFO:teuthology.task.internal:Starting timer... 2024-09-13T08:25:13.969 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T08:25:14.023 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T08:25:14.070 DEBUG:teuthology.task.selinux:Excluding smithi044: OS 'ubuntu' does not support SELinux 2024-09-13T08:25:14.070 DEBUG:teuthology.task.selinux:Excluding smithi129: OS 'ubuntu' does not support SELinux 2024-09-13T08:25:14.070 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T08:25:14.070 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T08:25:14.070 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T08:25:14.070 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T08:25:14.120 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T08:25:14.132 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T08:25:14.133 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi129.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T08:33:34.852 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi129.front.sepia.ceph.com')] 2024-09-13T08:33:34.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T08:33:34.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:33:34.964 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-09-13T08:33:35.096 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T08:33:35.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T08:33:35.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:33:35.207 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-13T08:33:35.339 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T08:33:35.340 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T08:33:35.351 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T08:33:35.352 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T08:33:35.352 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T08:33:35.356 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T08:33:35.356 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T08:33:35.391 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T08:33:35.392 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: Command line: ntpd -gq 2024-09-13T08:33:35.392 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: ---------------------------------------------------- 2024-09-13T08:33:35.392 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T08:33:35.392 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T08:33:35.392 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: corporation. Support and training for ntp-4 are 2024-09-13T08:33:35.392 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: available at https://www.nwtime.org/support 2024-09-13T08:33:35.392 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: ---------------------------------------------------- 2024-09-13T08:33:35.393 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: proto: precision = 0.044 usec (-24) 2024-09-13T08:33:35.393 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: basedate set to 2022-02-04 2024-09-13T08:33:35.393 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: gps base set to 2022-02-06 (week 2196) 2024-09-13T08:33:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T08:33:35.394 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T08:33:35.394 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T08:33:35.394 INFO:teuthology.orchestra.run.smithi044.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T08:33:35.395 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T08:33:35.395 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T08:33:35.395 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T08:33:35.395 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: Listen normally on 3 ens1f0 172.21.15.44:123 2024-09-13T08:33:35.396 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: Listen normally on 4 lo [::1]:123 2024-09-13T08:33:35.396 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1572%3]:123 2024-09-13T08:33:35.396 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:35 ntpd[18413]: Listening on routing socket on fd #22 for interface updates 2024-09-13T08:33:35.421 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T08:33:35.421 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: Command line: ntpd -gq 2024-09-13T08:33:35.421 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: ---------------------------------------------------- 2024-09-13T08:33:35.421 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T08:33:35.422 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T08:33:35.422 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: corporation. Support and training for ntp-4 are 2024-09-13T08:33:35.422 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: available at https://www.nwtime.org/support 2024-09-13T08:33:35.422 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: ---------------------------------------------------- 2024-09-13T08:33:35.422 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: proto: precision = 0.042 usec (-24) 2024-09-13T08:33:35.422 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: basedate set to 2022-02-04 2024-09-13T08:33:35.423 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: gps base set to 2022-02-06 (week 2196) 2024-09-13T08:33:35.423 INFO:teuthology.orchestra.run.smithi129.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T08:33:35.423 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T08:33:35.423 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T08:33:35.424 INFO:teuthology.orchestra.run.smithi129.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T08:33:35.424 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T08:33:35.424 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T08:33:35.425 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T08:33:35.425 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: Listen normally on 3 enp3s0f1 172.21.15.129:123 2024-09-13T08:33:35.425 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: Listen normally on 4 lo [::1]:123 2024-09-13T08:33:35.425 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7261%5]:123 2024-09-13T08:33:35.425 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:35 ntpd[18484]: Listening on routing socket on fd #22 for interface updates 2024-09-13T08:33:36.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:36.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:36.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:36.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:36.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:36.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:36.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:36.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:36.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:36.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:36.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:36.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:36 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:36.421 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:36 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:36.421 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:36 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:36.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:36 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:36.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:36 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:36.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:36 ntpd[18484]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:36.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:36 ntpd[18484]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:37.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:37 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:37.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:37 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:37.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:37 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:37.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:37 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:37.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:37 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:37.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:37 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:37.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:37 ntpd[18484]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:37.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:37 ntpd[18484]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:38 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:38.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:38 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:38.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:38 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:38.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:38 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:38.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:38 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:38.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:38 ntpd[18484]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:38.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:38 ntpd[18484]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:39.392 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:39 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:39.392 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:39 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:39.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:39 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:39.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:39 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:39.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:39 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:39.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:39 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:39.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:39 ntpd[18484]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:39.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:39 ntpd[18484]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:40.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:40.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:40.395 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:40.395 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:40.395 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:40.395 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:40.395 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:40.395 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:40 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:40.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:40.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:40.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:40.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:40.423 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:40.423 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:40.423 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:40.423 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:40.423 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:40.423 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:40.423 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:40.423 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:40 ntpd[18484]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:41.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:41 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:41.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:41 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:41.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:41 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:41.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:41 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:41.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:41 ntpd[18484]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:41.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:41 ntpd[18484]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:42.392 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:42.394 INFO:teuthology.orchestra.run.smithi044.stdout:13 Sep 08:33:42 ntpd[18413]: ntpd: time slew +0.000159 s 2024-09-13T08:33:42.394 INFO:teuthology.orchestra.run.smithi044.stdout:ntpd: time slew +0.000159s 2024-09-13T08:33:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T08:33:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:13 Sep 08:33:42 ntpd[18413]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T08:33:42.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:42 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:42.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:42 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:42.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:42 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:42.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:42 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:42.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:42 ntpd[18484]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T08:33:42.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:42 ntpd[18484]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T08:33:42.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:42 ntpd[18484]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T08:33:42.422 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:42 ntpd[18484]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T08:33:42.423 INFO:teuthology.orchestra.run.smithi129.stdout:13 Sep 08:33:42 ntpd[18484]: ntpd: time slew -0.000492 s 2024-09-13T08:33:42.423 INFO:teuthology.orchestra.run.smithi129.stdout:ntpd: time slew -0.000492s 2024-09-13T08:33:42.423 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:42 ntpd[18484]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T08:33:42.423 INFO:teuthology.orchestra.run.smithi129.stderr:13 Sep 08:33:42 ntpd[18484]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T08:33:42.506 INFO:teuthology.orchestra.run.smithi044.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T08:33:42.506 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================== 2024-09-13T08:33:42.506 INFO:teuthology.orchestra.run.smithi044.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T08:33:42.506 INFO:teuthology.orchestra.run.smithi044.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T08:33:42.506 INFO:teuthology.orchestra.run.smithi044.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T08:33:42.506 INFO:teuthology.orchestra.run.smithi044.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T08:33:42.524 INFO:teuthology.orchestra.run.smithi129.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T08:33:42.524 INFO:teuthology.orchestra.run.smithi129.stdout:============================================================================== 2024-09-13T08:33:42.524 INFO:teuthology.orchestra.run.smithi129.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T08:33:42.524 INFO:teuthology.orchestra.run.smithi129.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T08:33:42.525 INFO:teuthology.orchestra.run.smithi129.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T08:33:42.525 INFO:teuthology.orchestra.run.smithi129.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T08:33:42.526 INFO:teuthology.run_tasks:Running task install... 2024-09-13T08:33:42.533 DEBUG:teuthology.task.install:project ceph 2024-09-13T08:33:42.534 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd454760ef8a93a9dd7f4e4dedfb085faac216e7e'}} 2024-09-13T08:33:42.534 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd454760ef8a93a9dd7f4e4dedfb085faac216e7e'} 2024-09-13T08:33:42.534 INFO:teuthology.task.install:Using flavor: default 2024-09-13T08:33:42.540 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T08:33:42.540 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T08:33:42.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo apt-key list | grep Ceph 2024-09-13T08:33:42.541 DEBUG:teuthology.orchestra.run.smithi129:> sudo apt-key list | grep Ceph 2024-09-13T08:33:42.654 INFO:teuthology.orchestra.run.smithi044.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T08:33:42.665 INFO:teuthology.orchestra.run.smithi129.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T08:33:42.706 INFO:teuthology.orchestra.run.smithi044.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T08:33:42.706 INFO:teuthology.orchestra.run.smithi044.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T08:33:42.707 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T08:33:42.708 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T08:33:42.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d454760ef8a93a9dd7f4e4dedfb085faac216e7e 2024-09-13T08:33:42.725 INFO:teuthology.orchestra.run.smithi129.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T08:33:42.725 INFO:teuthology.orchestra.run.smithi129.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T08:33:42.726 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T08:33:42.726 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T08:33:42.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d454760ef8a93a9dd7f4e4dedfb085faac216e7e 2024-09-13T08:33:42.869 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default/ 2024-09-13T08:33:42.869 INFO:teuthology.task.install.deb:Package version is 19.3.0-4881-gd454760e-1jammy 2024-09-13T08:33:42.880 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default/ 2024-09-13T08:33:42.880 INFO:teuthology.task.install.deb:Package version is 19.3.0-4881-gd454760e-1jammy 2024-09-13T08:33:42.992 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:33:42.992 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T08:33:42.998 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:33:42.998 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T08:33:43.011 DEBUG:teuthology.orchestra.run.smithi044:> sudo apt-get update 2024-09-13T08:33:43.017 DEBUG:teuthology.orchestra.run.smithi129:> sudo apt-get update 2024-09-13T08:33:43.198 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T08:33:43.198 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T08:33:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T08:33:43.221 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T08:33:43.249 INFO:teuthology.orchestra.run.smithi044.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T08:33:43.249 INFO:teuthology.orchestra.run.smithi129.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T08:33:43.270 INFO:teuthology.orchestra.run.smithi129.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-13T08:33:43.271 INFO:teuthology.orchestra.run.smithi044.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-13T08:33:43.293 INFO:teuthology.orchestra.run.smithi129.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T08:33:43.293 INFO:teuthology.orchestra.run.smithi044.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T08:33:43.314 INFO:teuthology.orchestra.run.smithi044.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-09-13T08:33:43.315 INFO:teuthology.orchestra.run.smithi129.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-13T08:33:43.336 INFO:teuthology.orchestra.run.smithi044.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T08:33:43.336 INFO:teuthology.orchestra.run.smithi044.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T08:33:43.351 INFO:teuthology.orchestra.run.smithi129.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-09-13T08:33:43.357 INFO:teuthology.orchestra.run.smithi044.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-13T08:33:43.397 INFO:teuthology.orchestra.run.smithi044.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T08:33:43.479 INFO:teuthology.orchestra.run.smithi044.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T08:33:43.798 INFO:teuthology.orchestra.run.smithi129.stdout:Fetched 28.6 kB in 1s (51.5 kB/s) 2024-09-13T08:33:43.953 INFO:teuthology.orchestra.run.smithi044.stdout:Fetched 28.6 kB in 1s (40.4 kB/s) 2024-09-13T08:33:44.829 INFO:teuthology.orchestra.run.smithi129.stdout:Reading package lists... 2024-09-13T08:33:44.844 DEBUG:teuthology.orchestra.run.smithi129:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4881-gd454760e-1jammy cephadm=19.3.0-4881-gd454760e-1jammy ceph-mds=19.3.0-4881-gd454760e-1jammy ceph-mgr=19.3.0-4881-gd454760e-1jammy ceph-common=19.3.0-4881-gd454760e-1jammy ceph-fuse=19.3.0-4881-gd454760e-1jammy ceph-test=19.3.0-4881-gd454760e-1jammy ceph-volume=19.3.0-4881-gd454760e-1jammy radosgw=19.3.0-4881-gd454760e-1jammy python3-rados=19.3.0-4881-gd454760e-1jammy python3-rgw=19.3.0-4881-gd454760e-1jammy python3-cephfs=19.3.0-4881-gd454760e-1jammy python3-rbd=19.3.0-4881-gd454760e-1jammy libcephfs2=19.3.0-4881-gd454760e-1jammy libcephfs-dev=19.3.0-4881-gd454760e-1jammy librados2=19.3.0-4881-gd454760e-1jammy librbd1=19.3.0-4881-gd454760e-1jammy rbd-fuse=19.3.0-4881-gd454760e-1jammy python3-cephfs=19.3.0-4881-gd454760e-1jammy cephfs-shell=19.3.0-4881-gd454760e-1jammy cephfs-top=19.3.0-4881-gd454760e-1jammy cephfs-mirror=19.3.0-4881-gd454760e-1jammy 2024-09-13T08:33:44.920 INFO:teuthology.orchestra.run.smithi129.stdout:Reading package lists... 2024-09-13T08:33:45.025 INFO:teuthology.orchestra.run.smithi044.stdout:Reading package lists... 2024-09-13T08:33:45.040 DEBUG:teuthology.orchestra.run.smithi044:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4881-gd454760e-1jammy cephadm=19.3.0-4881-gd454760e-1jammy ceph-mds=19.3.0-4881-gd454760e-1jammy ceph-mgr=19.3.0-4881-gd454760e-1jammy ceph-common=19.3.0-4881-gd454760e-1jammy ceph-fuse=19.3.0-4881-gd454760e-1jammy ceph-test=19.3.0-4881-gd454760e-1jammy ceph-volume=19.3.0-4881-gd454760e-1jammy radosgw=19.3.0-4881-gd454760e-1jammy python3-rados=19.3.0-4881-gd454760e-1jammy python3-rgw=19.3.0-4881-gd454760e-1jammy python3-cephfs=19.3.0-4881-gd454760e-1jammy python3-rbd=19.3.0-4881-gd454760e-1jammy libcephfs2=19.3.0-4881-gd454760e-1jammy libcephfs-dev=19.3.0-4881-gd454760e-1jammy librados2=19.3.0-4881-gd454760e-1jammy librbd1=19.3.0-4881-gd454760e-1jammy rbd-fuse=19.3.0-4881-gd454760e-1jammy python3-cephfs=19.3.0-4881-gd454760e-1jammy cephfs-shell=19.3.0-4881-gd454760e-1jammy cephfs-top=19.3.0-4881-gd454760e-1jammy cephfs-mirror=19.3.0-4881-gd454760e-1jammy 2024-09-13T08:33:45.104 INFO:teuthology.orchestra.run.smithi129.stdout:Building dependency tree... 2024-09-13T08:33:45.105 INFO:teuthology.orchestra.run.smithi129.stdout:Reading state information... 2024-09-13T08:33:45.119 INFO:teuthology.orchestra.run.smithi044.stdout:Reading package lists... 2024-09-13T08:33:45.294 INFO:teuthology.orchestra.run.smithi129.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T08:33:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T08:33:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T08:33:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T08:33:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T08:33:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T08:33:45.295 INFO:teuthology.orchestra.run.smithi129.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T08:33:45.295 INFO:teuthology.orchestra.run.smithi129.stdout:The following additional packages will be installed: 2024-09-13T08:33:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T08:33:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T08:33:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-13T08:33:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-13T08:33:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-13T08:33:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-13T08:33:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-13T08:33:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T08:33:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-13T08:33:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-13T08:33:45.299 INFO:teuthology.orchestra.run.smithi129.stdout:Suggested packages: 2024-09-13T08:33:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-13T08:33:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-13T08:33:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-13T08:33:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-13T08:33:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-13T08:33:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T08:33:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-13T08:33:45.299 INFO:teuthology.orchestra.run.smithi129.stdout:Recommended packages: 2024-09-13T08:33:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: btrfs-tools 2024-09-13T08:33:45.314 INFO:teuthology.orchestra.run.smithi044.stdout:Building dependency tree... 2024-09-13T08:33:45.314 INFO:teuthology.orchestra.run.smithi044.stdout:Reading state information... 2024-09-13T08:33:45.452 INFO:teuthology.orchestra.run.smithi129.stdout:The following NEW packages will be installed: 2024-09-13T08:33:45.452 INFO:teuthology.orchestra.run.smithi129.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T08:33:45.452 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T08:33:45.452 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T08:33:45.453 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-13T08:33:45.453 INFO:teuthology.orchestra.run.smithi129.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-13T08:33:45.453 INFO:teuthology.orchestra.run.smithi129.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-13T08:33:45.453 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-13T08:33:45.454 INFO:teuthology.orchestra.run.smithi129.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-13T08:33:45.454 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-13T08:33:45.454 INFO:teuthology.orchestra.run.smithi129.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T08:33:45.454 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T08:33:45.454 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-13T08:33:45.454 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-13T08:33:45.454 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T08:33:45.455 INFO:teuthology.orchestra.run.smithi129.stdout: socat unzip xmlstarlet zip 2024-09-13T08:33:45.515 INFO:teuthology.orchestra.run.smithi044.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T08:33:45.515 INFO:teuthology.orchestra.run.smithi044.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T08:33:45.515 INFO:teuthology.orchestra.run.smithi044.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T08:33:45.516 INFO:teuthology.orchestra.run.smithi044.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T08:33:45.516 INFO:teuthology.orchestra.run.smithi044.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T08:33:45.516 INFO:teuthology.orchestra.run.smithi044.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T08:33:45.516 INFO:teuthology.orchestra.run.smithi044.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T08:33:45.517 INFO:teuthology.orchestra.run.smithi044.stdout:The following additional packages will be installed: 2024-09-13T08:33:45.517 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T08:33:45.519 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T08:33:45.519 INFO:teuthology.orchestra.run.smithi044.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-13T08:33:45.519 INFO:teuthology.orchestra.run.smithi044.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-13T08:33:45.519 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-13T08:33:45.519 INFO:teuthology.orchestra.run.smithi044.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-13T08:33:45.519 INFO:teuthology.orchestra.run.smithi044.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-13T08:33:45.519 INFO:teuthology.orchestra.run.smithi044.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T08:33:45.519 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-13T08:33:45.519 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-13T08:33:45.520 INFO:teuthology.orchestra.run.smithi044.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-13T08:33:45.521 INFO:teuthology.orchestra.run.smithi044.stdout:Suggested packages: 2024-09-13T08:33:45.521 INFO:teuthology.orchestra.run.smithi044.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-13T08:33:45.521 INFO:teuthology.orchestra.run.smithi044.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-13T08:33:45.521 INFO:teuthology.orchestra.run.smithi044.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-13T08:33:45.521 INFO:teuthology.orchestra.run.smithi044.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-13T08:33:45.521 INFO:teuthology.orchestra.run.smithi044.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-13T08:33:45.521 INFO:teuthology.orchestra.run.smithi044.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T08:33:45.521 INFO:teuthology.orchestra.run.smithi044.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-13T08:33:45.521 INFO:teuthology.orchestra.run.smithi044.stdout:Recommended packages: 2024-09-13T08:33:45.521 INFO:teuthology.orchestra.run.smithi044.stdout: btrfs-tools 2024-09-13T08:33:45.603 INFO:teuthology.orchestra.run.smithi129.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-13T08:33:45.604 INFO:teuthology.orchestra.run.smithi129.stdout:Need to get 252 MB of archives. 2024-09-13T08:33:45.604 INFO:teuthology.orchestra.run.smithi129.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-13T08:33:45.604 INFO:teuthology.orchestra.run.smithi129.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4881-gd454760e-1jammy [3,634 kB] 2024-09-13T08:33:45.650 INFO:teuthology.orchestra.run.smithi129.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-13T08:33:45.682 INFO:teuthology.orchestra.run.smithi044.stdout:The following NEW packages will be installed: 2024-09-13T08:33:45.683 INFO:teuthology.orchestra.run.smithi044.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T08:33:45.683 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T08:33:45.683 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T08:33:45.683 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-13T08:33:45.683 INFO:teuthology.orchestra.run.smithi044.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-13T08:33:45.683 INFO:teuthology.orchestra.run.smithi044.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-13T08:33:45.683 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-13T08:33:45.684 INFO:teuthology.orchestra.run.smithi044.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-13T08:33:45.684 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-13T08:33:45.684 INFO:teuthology.orchestra.run.smithi044.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T08:33:45.685 INFO:teuthology.orchestra.run.smithi044.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T08:33:45.686 INFO:teuthology.orchestra.run.smithi044.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T08:33:45.686 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T08:33:45.686 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T08:33:45.686 INFO:teuthology.orchestra.run.smithi044.stdout: socat unzip xmlstarlet zip 2024-09-13T08:33:45.732 INFO:teuthology.orchestra.run.smithi129.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T08:33:45.754 INFO:teuthology.orchestra.run.smithi044.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-13T08:33:45.754 INFO:teuthology.orchestra.run.smithi044.stdout:Need to get 252 MB of archives. 2024-09-13T08:33:45.754 INFO:teuthology.orchestra.run.smithi044.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-13T08:33:45.754 INFO:teuthology.orchestra.run.smithi044.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-13T08:33:45.772 INFO:teuthology.orchestra.run.smithi044.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T08:33:45.776 INFO:teuthology.orchestra.run.smithi129.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4881-gd454760e-1jammy [3,334 kB] 2024-09-13T08:33:45.778 INFO:teuthology.orchestra.run.smithi129.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-13T08:33:45.781 INFO:teuthology.orchestra.run.smithi044.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-13T08:33:45.825 INFO:teuthology.orchestra.run.smithi129.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4881-gd454760e-1jammy [722 kB] 2024-09-13T08:33:45.826 INFO:teuthology.orchestra.run.smithi044.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T08:33:45.834 INFO:teuthology.orchestra.run.smithi129.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4881-gd454760e-1jammy [356 kB] 2024-09-13T08:33:45.838 INFO:teuthology.orchestra.run.smithi044.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4881-gd454760e-1jammy [3,634 kB] 2024-09-13T08:33:45.839 INFO:teuthology.orchestra.run.smithi129.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4881-gd454760e-1jammy [32.8 kB] 2024-09-13T08:33:45.840 INFO:teuthology.orchestra.run.smithi129.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4881-gd454760e-1jammy [180 kB] 2024-09-13T08:33:45.843 INFO:teuthology.orchestra.run.smithi129.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4881-gd454760e-1jammy [72.6 kB] 2024-09-13T08:33:45.844 INFO:teuthology.orchestra.run.smithi044.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T08:33:45.845 INFO:teuthology.orchestra.run.smithi129.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4881-gd454760e-1jammy [338 kB] 2024-09-13T08:33:45.847 INFO:teuthology.orchestra.run.smithi044.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T08:33:45.850 INFO:teuthology.orchestra.run.smithi129.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4881-gd454760e-1jammy [7,987 kB] 2024-09-13T08:33:45.858 INFO:teuthology.orchestra.run.smithi044.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T08:33:45.899 INFO:teuthology.orchestra.run.smithi044.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T08:33:45.901 INFO:teuthology.orchestra.run.smithi044.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T08:33:45.908 INFO:teuthology.orchestra.run.smithi044.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T08:33:45.909 INFO:teuthology.orchestra.run.smithi044.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-13T08:33:45.910 INFO:teuthology.orchestra.run.smithi044.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T08:33:45.911 INFO:teuthology.orchestra.run.smithi044.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T08:33:45.917 INFO:teuthology.orchestra.run.smithi044.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-13T08:33:45.918 INFO:teuthology.orchestra.run.smithi044.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-13T08:33:45.920 INFO:teuthology.orchestra.run.smithi044.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-13T08:33:45.921 INFO:teuthology.orchestra.run.smithi044.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-13T08:33:45.933 INFO:teuthology.orchestra.run.smithi044.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-13T08:33:45.935 INFO:teuthology.orchestra.run.smithi044.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-13T08:33:45.937 INFO:teuthology.orchestra.run.smithi044.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-13T08:33:45.938 INFO:teuthology.orchestra.run.smithi044.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T08:33:45.939 INFO:teuthology.orchestra.run.smithi044.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T08:33:45.950 INFO:teuthology.orchestra.run.smithi044.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T08:33:45.951 INFO:teuthology.orchestra.run.smithi044.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T08:33:45.951 INFO:teuthology.orchestra.run.smithi044.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T08:33:45.952 INFO:teuthology.orchestra.run.smithi044.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T08:33:45.952 INFO:teuthology.orchestra.run.smithi044.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T08:33:45.967 INFO:teuthology.orchestra.run.smithi044.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T08:33:45.967 INFO:teuthology.orchestra.run.smithi044.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T08:33:45.968 INFO:teuthology.orchestra.run.smithi044.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T08:33:45.970 INFO:teuthology.orchestra.run.smithi044.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T08:33:45.971 INFO:teuthology.orchestra.run.smithi044.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T08:33:45.975 INFO:teuthology.orchestra.run.smithi129.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T08:33:45.984 INFO:teuthology.orchestra.run.smithi044.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T08:33:45.985 INFO:teuthology.orchestra.run.smithi044.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T08:33:45.985 INFO:teuthology.orchestra.run.smithi044.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T08:33:45.986 INFO:teuthology.orchestra.run.smithi044.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T08:33:45.986 INFO:teuthology.orchestra.run.smithi044.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T08:33:45.989 INFO:teuthology.orchestra.run.smithi129.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4881-gd454760e-1jammy [112 kB] 2024-09-13T08:33:45.990 INFO:teuthology.orchestra.run.smithi129.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4881-gd454760e-1jammy [473 kB] 2024-09-13T08:33:45.998 INFO:teuthology.orchestra.run.smithi129.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4881-gd454760e-1jammy [28.3 MB] 2024-09-13T08:33:46.001 INFO:teuthology.orchestra.run.smithi044.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T08:33:46.002 INFO:teuthology.orchestra.run.smithi044.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T08:33:46.002 INFO:teuthology.orchestra.run.smithi044.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T08:33:46.007 INFO:teuthology.orchestra.run.smithi044.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T08:33:46.007 INFO:teuthology.orchestra.run.smithi044.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T08:33:46.018 INFO:teuthology.orchestra.run.smithi044.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T08:33:46.019 INFO:teuthology.orchestra.run.smithi044.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T08:33:46.020 INFO:teuthology.orchestra.run.smithi044.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T08:33:46.021 INFO:teuthology.orchestra.run.smithi044.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T08:33:46.022 INFO:teuthology.orchestra.run.smithi044.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T08:33:46.036 INFO:teuthology.orchestra.run.smithi044.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T08:33:46.036 INFO:teuthology.orchestra.run.smithi044.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T08:33:46.038 INFO:teuthology.orchestra.run.smithi044.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T08:33:46.038 INFO:teuthology.orchestra.run.smithi044.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T08:33:46.039 INFO:teuthology.orchestra.run.smithi044.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T08:33:46.052 INFO:teuthology.orchestra.run.smithi044.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T08:33:46.066 INFO:teuthology.orchestra.run.smithi129.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T08:33:46.080 INFO:teuthology.orchestra.run.smithi129.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T08:33:46.097 INFO:teuthology.orchestra.run.smithi044.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T08:33:46.098 INFO:teuthology.orchestra.run.smithi044.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T08:33:46.099 INFO:teuthology.orchestra.run.smithi044.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T08:33:46.100 INFO:teuthology.orchestra.run.smithi044.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T08:33:46.101 INFO:teuthology.orchestra.run.smithi044.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T08:33:46.120 INFO:teuthology.orchestra.run.smithi044.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T08:33:46.123 INFO:teuthology.orchestra.run.smithi044.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T08:33:46.123 INFO:teuthology.orchestra.run.smithi044.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T08:33:46.129 INFO:teuthology.orchestra.run.smithi129.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T08:33:46.140 INFO:teuthology.orchestra.run.smithi044.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T08:33:46.141 INFO:teuthology.orchestra.run.smithi044.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T08:33:46.141 INFO:teuthology.orchestra.run.smithi044.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T08:33:46.142 INFO:teuthology.orchestra.run.smithi044.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T08:33:46.143 INFO:teuthology.orchestra.run.smithi044.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T08:33:46.143 INFO:teuthology.orchestra.run.smithi044.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T08:33:46.150 INFO:teuthology.orchestra.run.smithi044.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T08:33:46.151 INFO:teuthology.orchestra.run.smithi044.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T08:33:46.155 INFO:teuthology.orchestra.run.smithi044.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T08:33:46.167 INFO:teuthology.orchestra.run.smithi044.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T08:33:46.169 INFO:teuthology.orchestra.run.smithi044.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T08:33:46.170 INFO:teuthology.orchestra.run.smithi044.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T08:33:46.173 INFO:teuthology.orchestra.run.smithi044.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T08:33:46.176 INFO:teuthology.orchestra.run.smithi044.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-13T08:33:46.176 INFO:teuthology.orchestra.run.smithi044.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-13T08:33:46.177 INFO:teuthology.orchestra.run.smithi044.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-13T08:33:46.184 INFO:teuthology.orchestra.run.smithi044.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T08:33:46.189 INFO:teuthology.orchestra.run.smithi044.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-13T08:33:46.190 INFO:teuthology.orchestra.run.smithi044.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T08:33:46.202 INFO:teuthology.orchestra.run.smithi044.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T08:33:46.202 INFO:teuthology.orchestra.run.smithi044.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T08:33:46.203 INFO:teuthology.orchestra.run.smithi044.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T08:33:46.204 INFO:teuthology.orchestra.run.smithi044.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T08:33:46.205 INFO:teuthology.orchestra.run.smithi044.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T08:33:46.206 INFO:teuthology.orchestra.run.smithi044.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T08:33:46.213 INFO:teuthology.orchestra.run.smithi044.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T08:33:46.219 INFO:teuthology.orchestra.run.smithi044.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T08:33:46.220 INFO:teuthology.orchestra.run.smithi044.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T08:33:46.222 INFO:teuthology.orchestra.run.smithi044.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T08:33:46.236 INFO:teuthology.orchestra.run.smithi044.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T08:33:46.329 INFO:teuthology.orchestra.run.smithi129.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T08:33:46.337 INFO:teuthology.orchestra.run.smithi129.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T08:33:46.365 INFO:teuthology.orchestra.run.smithi129.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T08:33:46.374 INFO:teuthology.orchestra.run.smithi129.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-13T08:33:46.376 INFO:teuthology.orchestra.run.smithi129.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T08:33:46.377 INFO:teuthology.orchestra.run.smithi129.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T08:33:46.399 INFO:teuthology.orchestra.run.smithi129.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-13T08:33:46.404 INFO:teuthology.orchestra.run.smithi129.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-13T08:33:46.408 INFO:teuthology.orchestra.run.smithi129.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-13T08:33:46.411 INFO:teuthology.orchestra.run.smithi129.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-13T08:33:46.492 INFO:teuthology.orchestra.run.smithi129.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-13T08:33:46.494 INFO:teuthology.orchestra.run.smithi129.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-13T08:33:46.496 INFO:teuthology.orchestra.run.smithi129.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-13T08:33:46.496 INFO:teuthology.orchestra.run.smithi129.stdout:Get:33 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4881-gd454760e-1jammy [5,206 kB] 2024-09-13T08:33:46.498 INFO:teuthology.orchestra.run.smithi129.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T08:33:46.498 INFO:teuthology.orchestra.run.smithi129.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T08:33:46.499 INFO:teuthology.orchestra.run.smithi129.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T08:33:46.500 INFO:teuthology.orchestra.run.smithi129.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T08:33:46.500 INFO:teuthology.orchestra.run.smithi129.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T08:33:46.501 INFO:teuthology.orchestra.run.smithi129.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T08:33:46.574 INFO:teuthology.orchestra.run.smithi129.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T08:33:46.581 INFO:teuthology.orchestra.run.smithi129.stdout:Get:41 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4881-gd454760e-1jammy [252 kB] 2024-09-13T08:33:46.583 INFO:teuthology.orchestra.run.smithi129.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T08:33:46.583 INFO:teuthology.orchestra.run.smithi129.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T08:33:46.583 INFO:teuthology.orchestra.run.smithi129.stdout:Get:44 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4881-gd454760e-1jammy [124 kB] 2024-09-13T08:33:46.584 INFO:teuthology.orchestra.run.smithi129.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T08:33:46.585 INFO:teuthology.orchestra.run.smithi129.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4881-gd454760e-1jammy [1,561 kB] 2024-09-13T08:33:46.586 INFO:teuthology.orchestra.run.smithi129.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T08:33:46.586 INFO:teuthology.orchestra.run.smithi129.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T08:33:46.587 INFO:teuthology.orchestra.run.smithi129.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T08:33:46.587 INFO:teuthology.orchestra.run.smithi129.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T08:33:46.588 INFO:teuthology.orchestra.run.smithi129.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T08:33:46.611 INFO:teuthology.orchestra.run.smithi129.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4881-gd454760e-1jammy [6,309 kB] 2024-09-13T08:33:46.656 INFO:teuthology.orchestra.run.smithi129.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T08:33:46.671 INFO:teuthology.orchestra.run.smithi129.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T08:33:46.672 INFO:teuthology.orchestra.run.smithi129.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T08:33:46.673 INFO:teuthology.orchestra.run.smithi129.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T08:33:46.673 INFO:teuthology.orchestra.run.smithi129.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T08:33:46.678 INFO:teuthology.orchestra.run.smithi129.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T08:33:46.678 INFO:teuthology.orchestra.run.smithi129.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T08:33:46.728 INFO:teuthology.orchestra.run.smithi129.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4881-gd454760e-1jammy [23.1 MB] 2024-09-13T08:33:46.737 INFO:teuthology.orchestra.run.smithi129.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T08:33:46.738 INFO:teuthology.orchestra.run.smithi129.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T08:33:46.739 INFO:teuthology.orchestra.run.smithi129.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T08:33:46.819 INFO:teuthology.orchestra.run.smithi129.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T08:33:46.819 INFO:teuthology.orchestra.run.smithi129.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T08:33:46.821 INFO:teuthology.orchestra.run.smithi129.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T08:33:46.821 INFO:teuthology.orchestra.run.smithi129.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T08:33:46.824 INFO:teuthology.orchestra.run.smithi129.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T08:33:46.824 INFO:teuthology.orchestra.run.smithi129.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T08:33:46.901 INFO:teuthology.orchestra.run.smithi129.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T08:33:46.903 INFO:teuthology.orchestra.run.smithi129.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T08:33:46.909 INFO:teuthology.orchestra.run.smithi044.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4881-gd454760e-1jammy [3,334 kB] 2024-09-13T08:33:47.013 INFO:teuthology.orchestra.run.smithi129.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T08:33:47.014 INFO:teuthology.orchestra.run.smithi129.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T08:33:47.015 INFO:teuthology.orchestra.run.smithi129.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T08:33:47.015 INFO:teuthology.orchestra.run.smithi129.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T08:33:47.017 INFO:teuthology.orchestra.run.smithi129.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T08:33:47.088 INFO:teuthology.orchestra.run.smithi129.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T08:33:47.090 INFO:teuthology.orchestra.run.smithi129.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T08:33:47.090 INFO:teuthology.orchestra.run.smithi129.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T08:33:47.144 INFO:teuthology.orchestra.run.smithi129.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4881-gd454760e-1jammy [14.0 kB] 2024-09-13T08:33:47.144 INFO:teuthology.orchestra.run.smithi129.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4881-gd454760e-1jammy [880 kB] 2024-09-13T08:33:47.152 INFO:teuthology.orchestra.run.smithi129.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4881-gd454760e-1jammy [2,454 kB] 2024-09-13T08:33:47.160 INFO:teuthology.orchestra.run.smithi129.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T08:33:47.160 INFO:teuthology.orchestra.run.smithi129.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T08:33:47.161 INFO:teuthology.orchestra.run.smithi129.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T08:33:47.161 INFO:teuthology.orchestra.run.smithi129.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T08:33:47.163 INFO:teuthology.orchestra.run.smithi129.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T08:33:47.164 INFO:teuthology.orchestra.run.smithi129.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T08:33:47.198 INFO:teuthology.orchestra.run.smithi129.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4881-gd454760e-1jammy [813 kB] 2024-09-13T08:33:47.211 INFO:teuthology.orchestra.run.smithi129.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4881-gd454760e-1jammy [160 kB] 2024-09-13T08:33:47.213 INFO:teuthology.orchestra.run.smithi129.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4881-gd454760e-1jammy [61.4 MB] 2024-09-13T08:33:47.228 INFO:teuthology.orchestra.run.smithi129.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T08:33:47.229 INFO:teuthology.orchestra.run.smithi129.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T08:33:47.233 INFO:teuthology.orchestra.run.smithi129.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T08:33:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T08:33:47.312 INFO:teuthology.orchestra.run.smithi129.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T08:33:47.312 INFO:teuthology.orchestra.run.smithi129.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T08:33:47.317 INFO:teuthology.orchestra.run.smithi129.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T08:33:47.319 INFO:teuthology.orchestra.run.smithi129.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-13T08:33:47.320 INFO:teuthology.orchestra.run.smithi129.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-13T08:33:47.321 INFO:teuthology.orchestra.run.smithi129.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-13T08:33:47.383 INFO:teuthology.orchestra.run.smithi044.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4881-gd454760e-1jammy [722 kB] 2024-09-13T08:33:47.391 INFO:teuthology.orchestra.run.smithi129.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T08:33:47.396 INFO:teuthology.orchestra.run.smithi129.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-13T08:33:47.397 INFO:teuthology.orchestra.run.smithi129.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T08:33:47.473 INFO:teuthology.orchestra.run.smithi129.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T08:33:47.474 INFO:teuthology.orchestra.run.smithi129.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T08:33:47.474 INFO:teuthology.orchestra.run.smithi129.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T08:33:47.474 INFO:teuthology.orchestra.run.smithi129.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T08:33:47.476 INFO:teuthology.orchestra.run.smithi129.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T08:33:47.477 INFO:teuthology.orchestra.run.smithi129.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T08:33:47.484 INFO:teuthology.orchestra.run.smithi129.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T08:33:47.486 INFO:teuthology.orchestra.run.smithi044.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4881-gd454760e-1jammy [356 kB] 2024-09-13T08:33:47.538 INFO:teuthology.orchestra.run.smithi044.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4881-gd454760e-1jammy [32.8 kB] 2024-09-13T08:33:47.539 INFO:teuthology.orchestra.run.smithi044.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4881-gd454760e-1jammy [180 kB] 2024-09-13T08:33:47.541 INFO:teuthology.orchestra.run.smithi044.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4881-gd454760e-1jammy [72.6 kB] 2024-09-13T08:33:47.545 INFO:teuthology.orchestra.run.smithi044.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4881-gd454760e-1jammy [338 kB] 2024-09-13T08:33:47.554 INFO:teuthology.orchestra.run.smithi129.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T08:33:47.555 INFO:teuthology.orchestra.run.smithi129.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T08:33:47.557 INFO:teuthology.orchestra.run.smithi129.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T08:33:47.595 INFO:teuthology.orchestra.run.smithi044.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4881-gd454760e-1jammy [7,987 kB] 2024-09-13T08:33:47.636 INFO:teuthology.orchestra.run.smithi129.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T08:33:48.287 INFO:teuthology.orchestra.run.smithi044.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4881-gd454760e-1jammy [112 kB] 2024-09-13T08:33:48.288 INFO:teuthology.orchestra.run.smithi044.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4881-gd454760e-1jammy [473 kB] 2024-09-13T08:33:48.294 INFO:teuthology.orchestra.run.smithi044.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4881-gd454760e-1jammy [28.3 MB] 2024-09-13T08:33:48.467 INFO:teuthology.orchestra.run.smithi129.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4881-gd454760e-1jammy [8,625 kB] 2024-09-13T08:33:48.646 INFO:teuthology.orchestra.run.smithi129.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4881-gd454760e-1jammy [14.2 kB] 2024-09-13T08:33:48.646 INFO:teuthology.orchestra.run.smithi129.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4881-gd454760e-1jammy [55.8 MB] 2024-09-13T08:33:49.610 INFO:teuthology.orchestra.run.smithi044.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4881-gd454760e-1jammy [5,206 kB] 2024-09-13T08:33:49.758 INFO:teuthology.orchestra.run.smithi044.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4881-gd454760e-1jammy [252 kB] 2024-09-13T08:33:49.760 INFO:teuthology.orchestra.run.smithi044.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4881-gd454760e-1jammy [124 kB] 2024-09-13T08:33:49.761 INFO:teuthology.orchestra.run.smithi044.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4881-gd454760e-1jammy [1,561 kB] 2024-09-13T08:33:49.811 INFO:teuthology.orchestra.run.smithi044.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4881-gd454760e-1jammy [6,309 kB] 2024-09-13T08:33:50.014 INFO:teuthology.orchestra.run.smithi044.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4881-gd454760e-1jammy [23.1 MB] 2024-09-13T08:33:50.308 INFO:teuthology.orchestra.run.smithi129.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4881-gd454760e-1jammy [130 kB] 2024-09-13T08:33:50.309 INFO:teuthology.orchestra.run.smithi129.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4881-gd454760e-1jammy [233 kB] 2024-09-13T08:33:50.312 INFO:teuthology.orchestra.run.smithi129.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4881-gd454760e-1jammy [33.9 kB] 2024-09-13T08:33:50.312 INFO:teuthology.orchestra.run.smithi129.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4881-gd454760e-1jammy [23.4 kB] 2024-09-13T08:33:50.312 INFO:teuthology.orchestra.run.smithi129.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4881-gd454760e-1jammy [39.0 kB] 2024-09-13T08:33:50.313 INFO:teuthology.orchestra.run.smithi129.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4881-gd454760e-1jammy [15.9 MB] 2024-09-13T08:33:50.790 INFO:teuthology.orchestra.run.smithi044.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4881-gd454760e-1jammy [14.0 kB] 2024-09-13T08:33:50.790 INFO:teuthology.orchestra.run.smithi044.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4881-gd454760e-1jammy [880 kB] 2024-09-13T08:33:50.807 INFO:teuthology.orchestra.run.smithi044.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4881-gd454760e-1jammy [2,454 kB] 2024-09-13T08:33:50.835 INFO:teuthology.orchestra.run.smithi129.stdout:Get:125 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4881-gd454760e-1jammy [91.2 kB] 2024-09-13T08:33:50.860 INFO:teuthology.orchestra.run.smithi044.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4881-gd454760e-1jammy [813 kB] 2024-09-13T08:33:50.872 INFO:teuthology.orchestra.run.smithi044.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4881-gd454760e-1jammy [160 kB] 2024-09-13T08:33:50.875 INFO:teuthology.orchestra.run.smithi044.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4881-gd454760e-1jammy [61.4 MB] 2024-09-13T08:33:51.489 INFO:teuthology.orchestra.run.smithi129.stdout:Fetched 252 MB in 5s (46.9 MB/s) 2024-09-13T08:33:51.670 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T08:33:51.735 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-13T08:33:51.739 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-13T08:33:51.793 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-13T08:33:51.902 INFO:teuthology.orchestra.run.smithi044.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4881-gd454760e-1jammy [8,625 kB] 2024-09-13T08:33:52.028 INFO:teuthology.orchestra.run.smithi044.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4881-gd454760e-1jammy [14.2 kB] 2024-09-13T08:33:52.028 INFO:teuthology.orchestra.run.smithi044.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4881-gd454760e-1jammy [55.8 MB] 2024-09-13T08:33:52.113 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T08:33:52.126 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T08:33:52.202 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T08:33:52.591 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T08:33:52.605 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-13T08:33:52.658 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-13T08:33:52.958 INFO:teuthology.orchestra.run.smithi044.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4881-gd454760e-1jammy [130 kB] 2024-09-13T08:33:52.959 INFO:teuthology.orchestra.run.smithi044.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4881-gd454760e-1jammy [233 kB] 2024-09-13T08:33:52.961 INFO:teuthology.orchestra.run.smithi044.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4881-gd454760e-1jammy [33.9 kB] 2024-09-13T08:33:52.962 INFO:teuthology.orchestra.run.smithi044.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4881-gd454760e-1jammy [23.4 kB] 2024-09-13T08:33:52.962 INFO:teuthology.orchestra.run.smithi044.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4881-gd454760e-1jammy [39.0 kB] 2024-09-13T08:33:52.963 INFO:teuthology.orchestra.run.smithi044.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4881-gd454760e-1jammy [15.9 MB] 2024-09-13T08:33:53.103 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T08:33:53.117 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T08:33:53.213 INFO:teuthology.orchestra.run.smithi044.stdout:Get:125 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-12-1849/d454760ef8a93a9dd7f4e4dedfb085faac216e7e/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4881-gd454760e-1jammy [91.2 kB] 2024-09-13T08:33:53.219 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T08:33:53.640 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T08:33:53.655 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T08:33:53.706 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T08:33:53.925 INFO:teuthology.orchestra.run.smithi044.stdout:Fetched 252 MB in 8s (33.5 MB/s) 2024-09-13T08:33:54.060 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T08:33:54.075 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T08:33:54.106 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T08:33:54.118 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T08:33:54.174 INFO:teuthology.orchestra.run.smithi044.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-13T08:33:54.177 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-13T08:33:54.230 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-13T08:33:54.583 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T08:33:54.597 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T08:33:54.649 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T08:33:54.681 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T08:33:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T08:33:54.755 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:33:55.010 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T08:33:55.025 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-13T08:33:55.069 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-13T08:33:55.175 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T08:33:55.190 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T08:33:55.233 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:33:55.480 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T08:33:55.497 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T08:33:55.597 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T08:33:55.628 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T08:33:55.643 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T08:33:55.686 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:33:56.009 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T08:33:56.023 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T08:33:56.024 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T08:33:56.038 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T08:33:56.073 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T08:33:56.076 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T08:33:56.384 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package librados2. 2024-09-13T08:33:56.399 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../010-librados2_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:33:56.442 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking librados2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:33:56.520 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T08:33:56.536 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T08:33:56.587 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T08:33:56.946 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libnbd0. 2024-09-13T08:33:56.960 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-13T08:33:57.003 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-13T08:33:57.065 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T08:33:57.081 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T08:33:57.140 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:33:57.283 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package librbd1. 2024-09-13T08:33:57.298 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../012-librbd1_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:33:57.334 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking librbd1 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:33:57.619 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T08:33:57.634 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T08:33:57.685 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:33:57.786 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T08:33:57.801 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:33:57.844 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libcephfs2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:33:58.117 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T08:33:58.129 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T08:33:58.180 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:33:58.355 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-rados. 2024-09-13T08:33:58.370 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:33:58.413 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-rados (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:33:58.533 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T08:33:58.549 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T08:33:58.600 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T08:33:58.725 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T08:33:58.739 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:33:58.783 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-ceph-argparse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:33:59.011 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package librados2. 2024-09-13T08:33:59.026 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../010-librados2_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:33:59.035 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T08:33:59.050 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:33:59.078 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking librados2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:33:59.085 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-cephfs (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:33:59.346 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T08:33:59.361 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:33:59.405 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-ceph-common (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:33:59.614 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libnbd0. 2024-09-13T08:33:59.629 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-13T08:33:59.681 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-13T08:33:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T08:33:59.756 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T08:33:59.791 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T08:34:00.027 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T08:34:00.042 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:00.050 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package librbd1. 2024-09-13T08:34:00.066 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../012-librbd1_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:00.077 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-rbd (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:00.108 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking librbd1 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:00.455 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T08:34:00.469 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T08:34:00.513 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T08:34:00.603 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T08:34:00.618 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:00.703 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libcephfs2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:00.773 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-13T08:34:00.788 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-13T08:34:00.832 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T08:34:01.014 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-rados. 2024-09-13T08:34:01.029 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:01.080 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-rados (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:01.151 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-13T08:34:01.166 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-13T08:34:01.201 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T08:34:01.475 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T08:34:01.490 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:01.529 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package lua5.1. 2024-09-13T08:34:01.533 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-ceph-argparse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:01.543 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-13T08:34:01.588 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-13T08:34:01.761 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T08:34:01.776 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:01.811 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-cephfs (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:01.941 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package lua-any. 2024-09-13T08:34:01.957 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-13T08:34:01.999 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-13T08:34:02.147 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T08:34:02.162 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:02.205 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-ceph-common (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:02.243 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package zip. 2024-09-13T08:34:02.259 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-13T08:34:02.293 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking zip (3.0-12build2) ... 2024-09-13T08:34:02.533 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T08:34:02.549 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T08:34:02.572 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package unzip. 2024-09-13T08:34:02.586 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-13T08:34:02.591 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T08:34:02.629 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-13T08:34:02.934 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T08:34:02.951 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:03.071 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package luarocks. 2024-09-13T08:34:03.086 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-13T08:34:03.133 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-13T08:34:03.161 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-rbd (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:03.489 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T08:34:03.504 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T08:34:03.532 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package librgw2. 2024-09-13T08:34:03.546 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../028-librgw2_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:03.547 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T08:34:03.590 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking librgw2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:03.825 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-13T08:34:03.840 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-13T08:34:03.883 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T08:34:04.135 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T08:34:04.150 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:04.185 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-rgw (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:04.203 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-13T08:34:04.218 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-13T08:34:04.261 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T08:34:04.538 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T08:34:04.553 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T08:34:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package lua5.1. 2024-09-13T08:34:04.572 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-13T08:34:04.596 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T08:34:04.638 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-13T08:34:04.917 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T08:34:04.926 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package lua-any. 2024-09-13T08:34:04.930 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:04.941 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-13T08:34:04.973 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libradosstriper1 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:04.984 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-13T08:34:05.228 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package zip. 2024-09-13T08:34:05.244 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-13T08:34:05.305 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking zip (3.0-12build2) ... 2024-09-13T08:34:05.452 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-common. 2024-09-13T08:34:05.467 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:05.510 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-common (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:05.567 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package unzip. 2024-09-13T08:34:05.582 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-13T08:34:05.616 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-13T08:34:06.062 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package luarocks. 2024-09-13T08:34:06.077 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-13T08:34:06.120 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-13T08:34:06.541 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package librgw2. 2024-09-13T08:34:06.556 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../028-librgw2_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:06.603 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking librgw2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:06.865 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-base. 2024-09-13T08:34:06.880 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:06.941 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-base (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:07.089 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T08:34:07.103 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:07.139 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-rgw (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:07.484 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T08:34:07.499 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T08:34:07.542 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T08:34:07.602 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T08:34:07.617 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T08:34:07.669 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T08:34:07.878 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T08:34:07.894 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:07.936 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libradosstriper1 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:07.997 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T08:34:08.012 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T08:34:08.055 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T08:34:08.306 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-common. 2024-09-13T08:34:08.322 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:08.356 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-common (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:08.634 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T08:34:08.649 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T08:34:08.700 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T08:34:09.069 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T08:34:09.085 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T08:34:09.136 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T08:34:09.505 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T08:34:09.520 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T08:34:09.536 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-base. 2024-09-13T08:34:09.552 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:09.572 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T08:34:09.604 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-base (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:09.916 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T08:34:09.931 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T08:34:09.975 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T08:34:10.141 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T08:34:10.153 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T08:34:10.190 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T08:34:10.286 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-portend. 2024-09-13T08:34:10.301 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-13T08:34:10.344 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T08:34:10.427 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T08:34:10.439 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T08:34:10.477 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T08:34:10.672 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T08:34:10.687 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T08:34:10.730 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T08:34:10.830 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T08:34:10.844 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T08:34:10.888 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T08:34:10.975 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T08:34:10.990 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T08:34:11.033 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T08:34:11.200 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T08:34:11.214 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T08:34:11.258 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T08:34:11.403 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T08:34:11.419 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T08:34:11.454 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T08:34:11.561 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T08:34:11.575 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T08:34:11.619 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T08:34:11.714 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T08:34:11.730 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T08:34:11.765 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T08:34:11.947 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T08:34:11.961 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T08:34:12.005 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T08:34:12.093 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T08:34:12.108 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T08:34:12.151 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T08:34:12.275 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-portend. 2024-09-13T08:34:12.289 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-13T08:34:12.325 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T08:34:12.453 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-mako. 2024-09-13T08:34:12.469 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T08:34:12.537 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T08:34:12.661 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T08:34:12.675 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T08:34:12.719 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T08:34:12.848 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T08:34:12.863 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T08:34:12.906 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T08:34:13.022 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T08:34:13.036 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T08:34:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T08:34:13.243 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T08:34:13.258 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T08:34:13.301 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T08:34:13.570 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-webob. 2024-09-13T08:34:13.585 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T08:34:13.617 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T08:34:13.628 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T08:34:13.631 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T08:34:13.675 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T08:34:13.965 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T08:34:13.980 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T08:34:14.003 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T08:34:14.017 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T08:34:14.018 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T08:34:14.062 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T08:34:14.293 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T08:34:14.308 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T08:34:14.351 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T08:34:14.407 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T08:34:14.420 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T08:34:14.465 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T08:34:14.679 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-paste. 2024-09-13T08:34:14.694 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T08:34:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T08:34:14.768 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-mako. 2024-09-13T08:34:14.781 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T08:34:14.826 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T08:34:15.024 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T08:34:15.038 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T08:34:15.081 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T08:34:15.171 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T08:34:15.184 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T08:34:15.221 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T08:34:15.418 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T08:34:15.433 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T08:34:15.476 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T08:34:15.532 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T08:34:15.546 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T08:34:15.590 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T08:34:15.779 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T08:34:15.794 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T08:34:15.837 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T08:34:15.918 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-webob. 2024-09-13T08:34:15.932 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T08:34:15.976 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T08:34:16.148 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T08:34:16.164 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T08:34:16.198 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T08:34:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T08:34:16.359 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T08:34:16.407 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T08:34:16.545 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T08:34:16.560 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T08:34:16.621 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T08:34:16.691 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T08:34:16.704 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T08:34:16.749 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T08:34:16.906 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-idna. 2024-09-13T08:34:16.921 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T08:34:16.956 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T08:34:17.052 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-paste. 2024-09-13T08:34:17.065 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T08:34:17.110 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T08:34:17.284 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T08:34:17.299 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T08:34:17.342 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T08:34:17.430 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T08:34:17.443 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T08:34:17.488 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T08:34:17.670 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-requests. 2024-09-13T08:34:17.686 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T08:34:17.728 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T08:34:17.800 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T08:34:17.813 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T08:34:17.858 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T08:34:18.081 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T08:34:18.097 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T08:34:18.144 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T08:34:18.156 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T08:34:18.157 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T08:34:18.194 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T08:34:19.355 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T08:34:19.369 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T08:34:19.413 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T08:34:19.460 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T08:34:19.475 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:19.518 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:19.802 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T08:34:19.815 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T08:34:19.860 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T08:34:19.906 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T08:34:19.921 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:19.956 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:20.171 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-idna. 2024-09-13T08:34:20.188 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T08:34:20.229 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T08:34:20.275 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T08:34:20.290 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T08:34:20.367 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T08:34:20.558 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T08:34:20.571 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T08:34:20.616 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T08:34:20.678 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T08:34:20.693 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T08:34:20.736 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T08:34:20.927 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-requests. 2024-09-13T08:34:20.939 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T08:34:21.010 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T08:34:21.081 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T08:34:21.096 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:21.130 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:21.322 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T08:34:21.334 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T08:34:21.380 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T08:34:21.526 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T08:34:21.541 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:21.583 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mon (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:21.683 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T08:34:21.695 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:21.733 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:22.113 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T08:34:22.124 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:22.163 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:22.212 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T08:34:22.227 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:22.270 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-osd (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:22.466 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T08:34:22.477 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T08:34:22.515 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T08:34:22.819 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T08:34:22.830 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T08:34:22.868 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T08:34:23.124 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph. 2024-09-13T08:34:23.139 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../069-ceph_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:23.173 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:23.221 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T08:34:23.233 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:23.271 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-mgr (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:23.467 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T08:34:23.482 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:23.526 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-fuse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:23.841 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T08:34:23.853 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:23.899 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-mon (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:23.921 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T08:34:23.936 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:23.978 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mds (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:24.307 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package cephadm. 2024-09-13T08:34:24.321 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../072-cephadm_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:24.356 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking cephadm (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:24.503 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T08:34:24.515 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:24.553 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-osd (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:24.709 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T08:34:24.724 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T08:34:24.776 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T08:34:25.129 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T08:34:25.144 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T08:34:25.187 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T08:34:25.339 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph. 2024-09-13T08:34:25.351 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../069-ceph_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:25.389 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:25.684 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T08:34:25.696 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:25.725 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-fuse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:26.095 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T08:34:26.107 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:26.145 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-mds (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:26.266 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-babel. 2024-09-13T08:34:26.280 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T08:34:26.324 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T08:34:26.548 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package cephadm. 2024-09-13T08:34:26.560 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../072-cephadm_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:26.598 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking cephadm (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:26.661 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T08:34:26.675 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T08:34:26.719 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T08:34:27.051 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T08:34:27.063 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T08:34:27.064 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T08:34:27.079 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:27.101 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T08:34:27.122 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:27.403 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T08:34:27.416 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T08:34:27.453 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T08:34:27.475 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T08:34:27.490 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T08:34:27.534 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T08:34:27.820 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-routes. 2024-09-13T08:34:27.835 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T08:34:27.878 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T08:34:28.239 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T08:34:28.254 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:28.298 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:28.458 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-babel. 2024-09-13T08:34:28.469 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T08:34:28.508 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T08:34:28.844 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T08:34:28.855 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T08:34:28.886 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T08:34:29.214 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T08:34:29.227 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:29.272 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:29.634 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T08:34:29.646 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T08:34:29.685 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T08:34:29.954 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-routes. 2024-09-13T08:34:29.966 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T08:34:30.003 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T08:34:30.340 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T08:34:30.353 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:30.381 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:30.861 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T08:34:30.876 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T08:34:30.919 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T08:34:31.389 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T08:34:31.403 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T08:34:31.558 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T08:34:31.953 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T08:34:31.968 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T08:34:32.011 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T08:34:32.356 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T08:34:32.371 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T08:34:32.406 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T08:34:32.705 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T08:34:32.717 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T08:34:32.753 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T08:34:33.140 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T08:34:33.154 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:33.198 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:33.231 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T08:34:33.243 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T08:34:33.384 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T08:34:33.688 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T08:34:33.702 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T08:34:33.738 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T08:34:34.232 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T08:34:34.246 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T08:34:34.290 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T08:34:34.368 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T08:34:34.383 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T08:34:34.426 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T08:34:34.850 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T08:34:34.865 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T08:34:34.912 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T08:34:35.008 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T08:34:35.021 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:35.066 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:35.266 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T08:34:35.282 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T08:34:35.324 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T08:34:35.669 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T08:34:35.685 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-13T08:34:35.727 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T08:34:35.997 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T08:34:36.013 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T08:34:36.047 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T08:34:36.053 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T08:34:36.067 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T08:34:36.111 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T08:34:36.375 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T08:34:36.391 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T08:34:36.414 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T08:34:36.428 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T08:34:36.433 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T08:34:36.472 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T08:34:36.747 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T08:34:36.763 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T08:34:36.805 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T08:34:36.826 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T08:34:36.841 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T08:34:36.884 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T08:34:37.166 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T08:34:37.182 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T08:34:37.195 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T08:34:37.210 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-13T08:34:37.216 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T08:34:37.279 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T08:34:37.599 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T08:34:37.613 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T08:34:37.657 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T08:34:37.938 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T08:34:37.953 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:37.988 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:37.994 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T08:34:38.008 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T08:34:38.043 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T08:34:38.249 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T08:34:38.264 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T08:34:38.332 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T08:34:38.413 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T08:34:38.427 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T08:34:38.463 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T08:34:38.685 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T08:34:38.699 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T08:34:38.743 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T08:34:38.783 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T08:34:38.797 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T08:34:38.833 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T08:34:39.063 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package jq. 2024-09-13T08:34:39.078 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T08:34:39.121 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T08:34:39.349 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package socat. 2024-09-13T08:34:39.363 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T08:34:39.398 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T08:34:39.522 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T08:34:39.535 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:39.571 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:39.710 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T08:34:39.726 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T08:34:39.760 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T08:34:39.899 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T08:34:39.913 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T08:34:39.958 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T08:34:40.277 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T08:34:40.291 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T08:34:40.336 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T08:34:40.412 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-test. 2024-09-13T08:34:40.427 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:40.472 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-test (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:40.605 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package jq. 2024-09-13T08:34:40.619 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T08:34:40.663 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T08:34:40.866 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package socat. 2024-09-13T08:34:40.880 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T08:34:40.924 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T08:34:41.245 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T08:34:41.258 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T08:34:41.302 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T08:34:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-test. 2024-09-13T08:34:41.611 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:41.640 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-test (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:42.726 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T08:34:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:42.784 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-volume (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:43.189 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T08:34:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking cephfs-mirror (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:43.516 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T08:34:43.532 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-13T08:34:43.574 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-13T08:34:43.713 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T08:34:43.727 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:43.771 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ceph-volume (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:43.860 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T08:34:43.876 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:43.918 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking cephfs-shell (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:44.196 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T08:34:44.212 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:44.254 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking cephfs-top (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:44.317 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T08:34:44.332 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:44.367 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking cephfs-mirror (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:44.507 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T08:34:44.523 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:44.556 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libcephfs-dev (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:44.687 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T08:34:44.701 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-13T08:34:44.761 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-13T08:34:44.867 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-13T08:34:44.883 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-13T08:34:44.925 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T08:34:45.006 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T08:34:45.020 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:45.064 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking cephfs-shell (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:45.314 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-13T08:34:45.330 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-13T08:34:45.372 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-13T08:34:45.378 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T08:34:45.393 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4881-gd454760e-1jammy_all.deb ... 2024-09-13T08:34:45.441 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking cephfs-top (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:45.686 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T08:34:45.701 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:45.735 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libcephfs-dev (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:45.750 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T08:34:45.766 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T08:34:45.834 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T08:34:46.047 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-13T08:34:46.065 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-13T08:34:46.105 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T08:34:46.255 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package pkg-config. 2024-09-13T08:34:46.271 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-13T08:34:46.314 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T08:34:46.435 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-13T08:34:46.450 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-13T08:34:46.492 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-13T08:34:46.591 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T08:34:46.607 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T08:34:46.645 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T08:34:46.855 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T08:34:46.871 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T08:34:46.905 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T08:34:47.069 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T08:34:47.085 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T08:34:47.157 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T08:34:47.302 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package pkg-config. 2024-09-13T08:34:47.317 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-13T08:34:47.360 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T08:34:47.439 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T08:34:47.455 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T08:34:47.497 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T08:34:47.614 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T08:34:47.629 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T08:34:47.671 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T08:34:47.842 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T08:34:47.858 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T08:34:47.900 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T08:34:48.100 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T08:34:48.115 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T08:34:48.158 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T08:34:48.211 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T08:34:48.227 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T08:34:48.269 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T08:34:48.461 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T08:34:48.523 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T08:34:48.523 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T08:34:48.647 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-py. 2024-09-13T08:34:48.663 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-13T08:34:48.705 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T08:34:48.873 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T08:34:48.891 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T08:34:48.931 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T08:34:49.050 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T08:34:49.066 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T08:34:49.108 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T08:34:49.267 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T08:34:49.283 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T08:34:49.325 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T08:34:49.587 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T08:34:49.603 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T08:34:49.645 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T08:34:49.678 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-py. 2024-09-13T08:34:49.694 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-13T08:34:49.737 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T08:34:49.957 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-toml. 2024-09-13T08:34:49.973 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-13T08:34:50.006 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T08:34:50.140 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T08:34:50.155 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T08:34:50.198 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T08:34:50.367 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T08:34:50.383 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T08:34:50.434 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T08:34:50.644 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T08:34:50.658 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T08:34:50.694 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T08:34:50.829 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T08:34:50.845 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T08:34:50.888 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T08:34:50.997 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-toml. 2024-09-13T08:34:51.011 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-13T08:34:51.055 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T08:34:51.133 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T08:34:51.148 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T08:34:51.181 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T08:34:51.391 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T08:34:51.406 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T08:34:51.449 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T08:34:51.786 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T08:34:51.801 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T08:34:51.844 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T08:34:51.960 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package radosgw. 2024-09-13T08:34:51.976 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../123-radosgw_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:52.018 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking radosgw (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:52.114 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T08:34:52.129 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T08:34:52.164 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T08:34:52.772 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T08:34:52.788 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:52.830 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking rbd-fuse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:52.860 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package radosgw. 2024-09-13T08:34:52.874 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../123-radosgw_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:52.918 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking radosgw (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:53.150 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T08:34:53.443 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T08:34:53.569 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T08:34:53.613 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T08:34:53.629 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4881-gd454760e-1jammy_amd64.deb ... 2024-09-13T08:34:53.671 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking rbd-fuse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:53.828 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T08:34:54.008 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T08:34:54.237 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T08:34:54.288 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T08:34:54.410 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T08:34:54.711 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T08:34:54.877 INFO:teuthology.orchestra.run.smithi129.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T08:34:54.891 INFO:teuthology.orchestra.run.smithi129.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-13T08:34:54.992 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up cephadm (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:55.133 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T08:34:55.548 INFO:teuthology.orchestra.run.smithi129.stdout:Adding system user cephadm....done 2024-09-13T08:34:55.702 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T08:34:55.777 INFO:teuthology.orchestra.run.smithi044.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T08:34:55.815 INFO:teuthology.orchestra.run.smithi044.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-13T08:34:55.862 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up cephadm (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:56.070 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T08:34:56.330 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T08:34:56.389 INFO:teuthology.orchestra.run.smithi044.stdout:Adding system user cephadm....done 2024-09-13T08:34:56.456 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T08:34:56.569 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T08:34:56.724 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T08:34:56.887 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T08:34:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T08:34:57.109 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T08:34:57.155 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T08:34:57.280 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T08:34:57.419 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T08:34:57.523 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T08:34:57.783 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T08:34:57.787 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T08:34:57.909 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T08:34:58.039 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-13T08:34:58.227 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T08:34:58.307 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T08:34:58.616 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-13T08:34:58.630 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T08:34:58.859 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T08:34:58.906 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-13T08:34:59.127 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T08:34:59.174 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T08:34:59.403 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T08:34:59.518 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-13T08:34:59.662 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-ceph-argparse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:34:59.964 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T08:35:00.077 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T08:35:00.089 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-13T08:35:00.207 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T08:35:00.362 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T08:35:00.392 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-13T08:35:00.663 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T08:35:00.667 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T08:35:00.793 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T08:35:00.939 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-ceph-argparse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:01.199 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T08:35:01.316 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-13T08:35:01.395 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-13T08:35:01.442 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T08:35:01.568 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-13T08:35:01.763 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-13T08:35:01.783 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-13T08:35:01.844 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T08:35:01.848 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T08:35:01.979 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T08:35:01.990 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T08:35:02.247 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-13T08:35:02.367 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T08:35:02.493 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T08:35:02.674 INFO:teuthology.orchestra.run.smithi044.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-13T08:35:02.694 INFO:teuthology.orchestra.run.smithi044.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-13T08:35:02.758 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T08:35:02.760 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T08:35:02.884 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T08:35:03.036 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T08:35:03.277 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T08:35:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up zip (3.0-12build2) ... 2024-09-13T08:35:03.412 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T08:35:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T08:35:03.696 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T08:35:03.981 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T08:35:04.166 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T08:35:04.307 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up zip (3.0-12build2) ... 2024-09-13T08:35:04.416 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T08:35:04.433 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T08:35:04.559 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T08:35:04.710 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T08:35:05.019 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T08:35:05.135 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T08:35:05.404 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T08:35:05.411 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T08:35:05.570 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T08:35:05.672 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T08:35:05.696 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T08:35:06.031 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T08:35:06.073 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T08:35:06.383 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-13T08:35:06.424 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T08:35:06.683 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T08:35:06.717 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T08:35:07.069 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T08:35:07.102 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T08:35:07.345 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T08:35:07.445 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-ceph-common (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:07.495 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-13T08:35:07.764 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T08:35:07.829 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T08:35:08.115 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T08:35:08.198 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T08:35:08.457 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T08:35:08.574 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-ceph-common (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:08.901 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T08:35:09.260 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T08:35:09.328 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T08:35:09.597 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:35:09.731 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T08:35:09.999 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T08:35:10.125 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T08:35:10.226 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T08:35:10.527 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T08:35:10.541 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T08:35:10.786 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T08:35:10.818 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:35:10.952 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T08:35:11.045 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:35:11.172 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T08:35:11.237 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T08:35:11.388 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T08:35:11.490 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T08:35:11.506 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T08:35:11.757 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T08:35:11.815 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T08:35:12.067 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T08:35:12.083 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T08:35:12.318 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T08:35:12.351 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:35:12.477 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T08:35:12.578 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T08:35:12.795 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T08:35:12.853 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T08:35:12.979 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T08:35:13.080 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T08:35:13.272 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T08:35:13.364 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T08:35:13.432 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T08:35:13.640 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T08:35:13.732 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T08:35:13.900 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T08:35:14.042 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T08:35:14.184 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T08:35:14.310 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T08:35:14.318 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T08:35:14.444 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-13T08:35:14.545 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T08:35:14.612 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T08:35:14.616 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T08:35:14.737 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T08:35:14.880 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T08:35:15.006 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T08:35:15.064 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T08:35:15.390 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T08:35:15.407 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T08:35:15.650 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T08:35:15.700 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-13T08:35:15.801 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T08:35:15.809 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-13T08:35:15.935 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T08:35:16.024 INFO:teuthology.orchestra.run.smithi044.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T08:35:16.068 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:35:16.203 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T08:35:16.279 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T08:35:16.404 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T08:35:16.512 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T08:35:16.839 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T08:35:16.840 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T08:35:17.115 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-13T08:35:17.217 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T08:35:17.225 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T08:35:17.342 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T08:35:17.492 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T08:35:17.619 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T08:35:17.619 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T08:35:17.895 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T08:35:17.987 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T08:35:18.155 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T08:35:18.304 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T08:35:18.540 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T08:35:18.707 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T08:35:18.825 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T08:35:18.833 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T08:35:19.102 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T08:35:19.168 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T08:35:19.294 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up librados2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:19.403 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T08:35:19.420 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:19.581 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-13T08:35:19.688 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T08:35:19.908 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libcephfs2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:20.009 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libradosstriper1 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:20.081 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T08:35:20.137 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T08:35:20.358 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T08:35:20.436 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up librbd1 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:20.537 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr-modules-core (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:20.647 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-fuse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:20.719 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T08:35:20.845 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up librados2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:20.854 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T08:35:20.854 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T08:35:21.004 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:21.131 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-13T08:35:21.450 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libcephfs2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:21.523 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libcephfs-dev (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:21.601 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libradosstriper1 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:21.657 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-rados (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:21.727 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T08:35:21.775 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up librgw2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:21.926 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up cephfs-top (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:22.029 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up librbd1 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:22.060 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T08:35:22.155 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-mgr-modules-core (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:22.298 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-fuse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:22.508 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T08:35:22.508 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T08:35:23.232 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libcephfs-dev (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-rados (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:23.386 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-rbd (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:23.476 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up librgw2 (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:23.529 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up rbd-fuse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:23.627 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up cephfs-top (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:23.655 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-rgw (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:23.754 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T08:35:23.781 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-cephfs (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:23.915 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up cephfs-shell (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:24.041 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-common (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:24.552 INFO:teuthology.orchestra.run.smithi129.stdout:Adding group ceph....done 2024-09-13T08:35:25.004 INFO:teuthology.orchestra.run.smithi129.stdout:Adding system user ceph....done 2024-09-13T08:35:25.082 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-rbd (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:25.182 INFO:teuthology.orchestra.run.smithi129.stdout:Setting system user ceph properties....done 2024-09-13T08:35:25.197 INFO:teuthology.orchestra.run.smithi129.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T08:35:25.209 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up rbd-fuse (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:25.348 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T08:35:25.558 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-rgw (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:25.736 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up python3-cephfs (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:25.748 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T08:35:25.871 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up cephfs-shell (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:25.997 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-common (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:26.409 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up cephfs-mirror (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:26.567 INFO:teuthology.orchestra.run.smithi044.stdout:Adding group ceph....done 2024-09-13T08:35:26.618 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T08:35:26.618 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T08:35:26.976 INFO:teuthology.orchestra.run.smithi044.stdout:Adding system user ceph....done 2024-09-13T08:35:27.145 INFO:teuthology.orchestra.run.smithi044.stdout:Setting system user ceph properties....done 2024-09-13T08:35:27.165 INFO:teuthology.orchestra.run.smithi044.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T08:35:27.260 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-test (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:27.313 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T08:35:27.361 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up radosgw (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:27.707 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T08:35:27.857 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T08:35:27.858 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T08:35:28.415 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up cephfs-mirror (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:28.471 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-base (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:28.647 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T08:35:28.648 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T08:35:28.830 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T08:35:29.258 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-test (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:29.384 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up radosgw (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:29.489 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mds (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:29.746 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T08:35:29.746 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T08:35:29.977 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T08:35:29.977 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T08:35:30.375 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:30.604 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T08:35:30.605 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T08:35:30.611 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-base (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:31.234 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-osd (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:31.532 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T08:35:31.618 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T08:35:31.618 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T08:35:32.210 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-mds (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:32.270 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:32.395 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:32.484 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T08:35:32.484 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T08:35:32.538 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mon (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:32.749 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T08:35:32.749 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T08:35:33.162 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-mgr (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:33.381 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr-cephadm (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:33.382 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T08:35:33.382 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T08:35:33.524 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:33.675 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr-dashboard (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:33.817 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-volume (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:34.014 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-osd (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:34.162 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T08:35:34.287 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T08:35:34.334 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T08:35:34.334 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T08:35:34.439 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T08:35:35.057 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:35.183 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:35.389 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T08:35:35.693 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-mon (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:35.913 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T08:35:35.913 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T08:35:36.120 INFO:teuthology.orchestra.run.smithi129.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T08:35:36.125 DEBUG:teuthology.orchestra.run.smithi129:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T08:35:36.202 INFO:teuthology.orchestra.run.smithi129.stdout:Reading package lists... 2024-09-13T08:35:36.386 INFO:teuthology.orchestra.run.smithi129.stdout:Building dependency tree... 2024-09-13T08:35:36.387 INFO:teuthology.orchestra.run.smithi129.stdout:Reading state information... 2024-09-13T08:35:36.545 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-mgr-cephadm (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:36.575 INFO:teuthology.orchestra.run.smithi129.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-13T08:35:36.575 INFO:teuthology.orchestra.run.smithi129.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-13T08:35:36.575 INFO:teuthology.orchestra.run.smithi129.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-13T08:35:36.575 INFO:teuthology.orchestra.run.smithi129.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T08:35:36.575 INFO:teuthology.orchestra.run.smithi129.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T08:35:36.575 INFO:teuthology.orchestra.run.smithi129.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T08:35:36.575 INFO:teuthology.orchestra.run.smithi129.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T08:35:36.576 INFO:teuthology.orchestra.run.smithi129.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T08:35:36.576 INFO:teuthology.orchestra.run.smithi129.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T08:35:36.576 INFO:teuthology.orchestra.run.smithi129.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T08:35:36.581 INFO:teuthology.orchestra.run.smithi129.stdout:The following additional packages will be installed: 2024-09-13T08:35:36.581 INFO:teuthology.orchestra.run.smithi129.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-13T08:35:36.581 INFO:teuthology.orchestra.run.smithi129.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-13T08:35:36.582 INFO:teuthology.orchestra.run.smithi129.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-13T08:35:36.582 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-13T08:35:36.582 INFO:teuthology.orchestra.run.smithi129.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T08:35:36.582 INFO:teuthology.orchestra.run.smithi129.stdout:Suggested packages: 2024-09-13T08:35:36.582 INFO:teuthology.orchestra.run.smithi129.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-13T08:35:36.582 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-13T08:35:36.582 INFO:teuthology.orchestra.run.smithi129.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-13T08:35:36.635 INFO:teuthology.orchestra.run.smithi129.stdout:The following NEW packages will be installed: 2024-09-13T08:35:36.636 INFO:teuthology.orchestra.run.smithi129.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-13T08:35:36.636 INFO:teuthology.orchestra.run.smithi129.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-13T08:35:36.637 INFO:teuthology.orchestra.run.smithi129.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-13T08:35:36.637 INFO:teuthology.orchestra.run.smithi129.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-13T08:35:36.637 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-13T08:35:36.637 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-13T08:35:36.637 INFO:teuthology.orchestra.run.smithi129.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T08:35:36.638 INFO:teuthology.orchestra.run.smithi129.stdout:The following packages will be upgraded: 2024-09-13T08:35:36.639 INFO:teuthology.orchestra.run.smithi129.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-13T08:35:36.687 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:36.813 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-mgr-dashboard (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:36.825 INFO:teuthology.orchestra.run.smithi129.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-13T08:35:36.825 INFO:teuthology.orchestra.run.smithi129.stdout:Need to get 55.8 MB of archives. 2024-09-13T08:35:36.825 INFO:teuthology.orchestra.run.smithi129.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-13T08:35:36.825 INFO:teuthology.orchestra.run.smithi129.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-13T08:35:36.965 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ceph-volume (19.3.0-4881-gd454760e-1jammy) ... 2024-09-13T08:35:37.326 INFO:teuthology.orchestra.run.smithi044.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T08:35:37.426 INFO:teuthology.orchestra.run.smithi044.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T08:35:37.446 INFO:teuthology.orchestra.run.smithi129.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-13T08:35:37.462 INFO:teuthology.orchestra.run.smithi129.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-13T08:35:37.586 INFO:teuthology.orchestra.run.smithi129.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-13T08:35:37.587 INFO:teuthology.orchestra.run.smithi129.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-13T08:35:37.588 INFO:teuthology.orchestra.run.smithi129.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-13T08:35:37.603 INFO:teuthology.orchestra.run.smithi044.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T08:35:37.617 INFO:teuthology.orchestra.run.smithi129.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-13T08:35:37.618 INFO:teuthology.orchestra.run.smithi129.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-13T08:35:37.621 INFO:teuthology.orchestra.run.smithi129.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-13T08:35:37.624 INFO:teuthology.orchestra.run.smithi129.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-13T08:35:37.624 INFO:teuthology.orchestra.run.smithi129.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-13T08:35:37.626 INFO:teuthology.orchestra.run.smithi129.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-13T08:35:37.690 INFO:teuthology.orchestra.run.smithi129.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-13T08:35:37.691 INFO:teuthology.orchestra.run.smithi129.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-13T08:35:37.691 INFO:teuthology.orchestra.run.smithi129.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-13T08:35:37.772 INFO:teuthology.orchestra.run.smithi129.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-13T08:35:38.453 INFO:teuthology.orchestra.run.smithi129.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-13T08:35:38.455 INFO:teuthology.orchestra.run.smithi129.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-13T08:35:38.455 INFO:teuthology.orchestra.run.smithi129.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-13T08:35:38.458 INFO:teuthology.orchestra.run.smithi129.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-13T08:35:38.458 INFO:teuthology.orchestra.run.smithi129.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-13T08:35:38.463 INFO:teuthology.orchestra.run.smithi129.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-13T08:35:38.464 INFO:teuthology.orchestra.run.smithi129.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-13T08:35:38.513 INFO:teuthology.orchestra.run.smithi129.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-13T08:35:38.514 INFO:teuthology.orchestra.run.smithi129.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-13T08:35:38.549 INFO:teuthology.orchestra.run.smithi129.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-13T08:35:38.597 INFO:teuthology.orchestra.run.smithi129.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-13T08:35:38.607 INFO:teuthology.orchestra.run.smithi129.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-13T08:35:38.684 INFO:teuthology.orchestra.run.smithi129.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-13T08:35:38.687 INFO:teuthology.orchestra.run.smithi129.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-13T08:35:38.688 INFO:teuthology.orchestra.run.smithi129.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-13T08:35:38.688 INFO:teuthology.orchestra.run.smithi129.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-13T08:35:38.689 INFO:teuthology.orchestra.run.smithi129.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-13T08:35:38.705 INFO:teuthology.orchestra.run.smithi129.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-13T08:35:38.707 INFO:teuthology.orchestra.run.smithi129.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-13T08:35:38.740 INFO:teuthology.orchestra.run.smithi044.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T08:35:39.199 INFO:teuthology.orchestra.run.smithi129.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-13T08:35:39.199 INFO:teuthology.orchestra.run.smithi129.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-13T08:35:39.199 INFO:teuthology.orchestra.run.smithi129.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-13T08:35:39.199 INFO:teuthology.orchestra.run.smithi129.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-13T08:35:39.199 INFO:teuthology.orchestra.run.smithi129.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-13T08:35:39.200 INFO:teuthology.orchestra.run.smithi129.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-13T08:35:39.213 INFO:teuthology.orchestra.run.smithi129.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-13T08:35:39.213 INFO:teuthology.orchestra.run.smithi129.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-13T08:35:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T08:35:39.491 DEBUG:teuthology.orchestra.run.smithi044:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T08:35:39.569 INFO:teuthology.orchestra.run.smithi044.stdout:Reading package lists... 2024-09-13T08:35:39.597 INFO:teuthology.orchestra.run.smithi129.stdout:Preconfiguring packages ... 2024-09-13T08:35:39.757 INFO:teuthology.orchestra.run.smithi044.stdout:Building dependency tree... 2024-09-13T08:35:39.758 INFO:teuthology.orchestra.run.smithi044.stdout:Reading state information... 2024-09-13T08:35:39.782 INFO:teuthology.orchestra.run.smithi129.stdout:Fetched 55.8 MB in 3s (21.7 MB/s) 2024-09-13T08:35:39.948 INFO:teuthology.orchestra.run.smithi044.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-13T08:35:39.948 INFO:teuthology.orchestra.run.smithi044.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-13T08:35:39.948 INFO:teuthology.orchestra.run.smithi044.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-13T08:35:39.948 INFO:teuthology.orchestra.run.smithi044.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T08:35:39.948 INFO:teuthology.orchestra.run.smithi044.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T08:35:39.948 INFO:teuthology.orchestra.run.smithi044.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T08:35:39.948 INFO:teuthology.orchestra.run.smithi044.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T08:35:39.949 INFO:teuthology.orchestra.run.smithi044.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T08:35:39.949 INFO:teuthology.orchestra.run.smithi044.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T08:35:39.949 INFO:teuthology.orchestra.run.smithi044.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T08:35:39.949 INFO:teuthology.orchestra.run.smithi044.stdout:The following additional packages will be installed: 2024-09-13T08:35:39.950 INFO:teuthology.orchestra.run.smithi044.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-13T08:35:39.950 INFO:teuthology.orchestra.run.smithi044.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-13T08:35:39.950 INFO:teuthology.orchestra.run.smithi044.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-13T08:35:39.950 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-13T08:35:39.950 INFO:teuthology.orchestra.run.smithi044.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T08:35:39.951 INFO:teuthology.orchestra.run.smithi044.stdout:Suggested packages: 2024-09-13T08:35:39.951 INFO:teuthology.orchestra.run.smithi044.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-13T08:35:39.952 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-13T08:35:39.952 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-13T08:35:40.009 INFO:teuthology.orchestra.run.smithi044.stdout:The following NEW packages will be installed: 2024-09-13T08:35:40.010 INFO:teuthology.orchestra.run.smithi044.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-13T08:35:40.010 INFO:teuthology.orchestra.run.smithi044.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-13T08:35:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-13T08:35:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-13T08:35:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-13T08:35:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-13T08:35:40.011 INFO:teuthology.orchestra.run.smithi044.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T08:35:40.012 INFO:teuthology.orchestra.run.smithi044.stdout:The following packages will be upgraded: 2024-09-13T08:35:40.012 INFO:teuthology.orchestra.run.smithi044.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-13T08:35:40.071 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143290 files and directories currently installed.) 2024-09-13T08:35:40.072 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T08:35:40.199 INFO:teuthology.orchestra.run.smithi044.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-13T08:35:40.199 INFO:teuthology.orchestra.run.smithi044.stdout:Need to get 55.8 MB of archives. 2024-09-13T08:35:40.199 INFO:teuthology.orchestra.run.smithi044.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-13T08:35:40.200 INFO:teuthology.orchestra.run.smithi044.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-13T08:35:40.221 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T08:35:40.820 INFO:teuthology.orchestra.run.smithi044.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-13T08:35:40.837 INFO:teuthology.orchestra.run.smithi044.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-13T08:35:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-13T08:35:40.960 INFO:teuthology.orchestra.run.smithi044.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-13T08:35:40.961 INFO:teuthology.orchestra.run.smithi044.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-13T08:35:40.964 INFO:teuthology.orchestra.run.smithi044.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-13T08:35:40.990 INFO:teuthology.orchestra.run.smithi044.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-13T08:35:40.992 INFO:teuthology.orchestra.run.smithi044.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-13T08:35:40.995 INFO:teuthology.orchestra.run.smithi044.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-13T08:35:40.998 INFO:teuthology.orchestra.run.smithi044.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-13T08:35:40.998 INFO:teuthology.orchestra.run.smithi044.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-13T08:35:41.000 INFO:teuthology.orchestra.run.smithi044.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-13T08:35:41.064 INFO:teuthology.orchestra.run.smithi044.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-13T08:35:41.065 INFO:teuthology.orchestra.run.smithi044.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-13T08:35:41.065 INFO:teuthology.orchestra.run.smithi044.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-13T08:35:41.146 INFO:teuthology.orchestra.run.smithi044.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-13T08:35:41.840 INFO:teuthology.orchestra.run.smithi044.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-13T08:35:41.842 INFO:teuthology.orchestra.run.smithi044.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-13T08:35:41.842 INFO:teuthology.orchestra.run.smithi044.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-13T08:35:41.845 INFO:teuthology.orchestra.run.smithi044.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-13T08:35:41.845 INFO:teuthology.orchestra.run.smithi044.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-13T08:35:41.850 INFO:teuthology.orchestra.run.smithi044.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-13T08:35:41.852 INFO:teuthology.orchestra.run.smithi044.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-13T08:35:41.888 INFO:teuthology.orchestra.run.smithi044.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-13T08:35:41.904 INFO:teuthology.orchestra.run.smithi044.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-13T08:35:41.935 INFO:teuthology.orchestra.run.smithi044.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-13T08:35:41.966 INFO:teuthology.orchestra.run.smithi129.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-13T08:35:41.987 INFO:teuthology.orchestra.run.smithi044.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-13T08:35:41.998 INFO:teuthology.orchestra.run.smithi044.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-13T08:35:42.074 INFO:teuthology.orchestra.run.smithi044.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-13T08:35:42.078 INFO:teuthology.orchestra.run.smithi044.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-13T08:35:42.079 INFO:teuthology.orchestra.run.smithi044.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-13T08:35:42.079 INFO:teuthology.orchestra.run.smithi044.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-13T08:35:42.080 INFO:teuthology.orchestra.run.smithi044.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-13T08:35:42.092 INFO:teuthology.orchestra.run.smithi044.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-13T08:35:42.094 INFO:teuthology.orchestra.run.smithi044.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-13T08:35:42.319 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package flex. 2024-09-13T08:35:42.388 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143290 files and directories currently installed.) 2024-09-13T08:35:42.392 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-13T08:35:42.442 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-13T08:35:42.583 INFO:teuthology.orchestra.run.smithi044.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-13T08:35:42.583 INFO:teuthology.orchestra.run.smithi044.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-13T08:35:42.583 INFO:teuthology.orchestra.run.smithi044.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-13T08:35:42.583 INFO:teuthology.orchestra.run.smithi044.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-13T08:35:42.584 INFO:teuthology.orchestra.run.smithi044.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-13T08:35:42.584 INFO:teuthology.orchestra.run.smithi044.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-13T08:35:42.647 INFO:teuthology.orchestra.run.smithi044.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-13T08:35:42.647 INFO:teuthology.orchestra.run.smithi044.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-13T08:35:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-13T08:35:43.051 INFO:teuthology.orchestra.run.smithi044.stdout:Preconfiguring packages ... 2024-09-13T08:35:43.173 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-13T08:35:43.207 INFO:teuthology.orchestra.run.smithi044.stdout:Fetched 55.8 MB in 3s (21.2 MB/s) 2024-09-13T08:35:43.528 INFO:teuthology.orchestra.run.smithi044.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143290 files and directories currently installed.) 2024-09-13T08:35:43.533 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T08:35:43.596 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T08:35:43.679 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T08:35:44.175 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143363 files and directories currently installed.) 2024-09-13T08:35:44.178 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T08:35:44.322 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T08:35:44.330 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-13T08:35:44.826 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T08:35:45.003 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T08:35:45.361 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-13T08:35:45.475 INFO:teuthology.orchestra.run.smithi044.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-13T08:35:45.734 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package flex. 2024-09-13T08:35:45.737 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package bison. 2024-09-13T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143290 files and directories currently installed.) 2024-09-13T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-13T08:35:45.812 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143363 files and directories currently installed.) 2024-09-13T08:35:45.817 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-13T08:35:45.856 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-13T08:35:45.857 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-13T08:35:46.256 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T08:35:46.271 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-13T08:35:46.315 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-13T08:35:46.378 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-13T08:35:46.588 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-13T08:35:46.668 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package dump. 2024-09-13T08:35:46.683 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-13T08:35:46.718 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking dump (0.4b47-3) ... 2024-09-13T08:35:46.963 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T08:35:47.013 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package indent. 2024-09-13T08:35:47.028 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:35:47.097 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-13T08:35:47.416 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T08:35:47.431 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-13T08:35:47.474 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-13T08:35:47.527 INFO:teuthology.orchestra.run.smithi044.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143363 files and directories currently installed.) 2024-09-13T08:35:47.531 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T08:35:47.654 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T08:35:47.795 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T08:35:47.810 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-13T08:35:47.846 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-13T08:35:48.017 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T08:35:48.114 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T08:35:48.130 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-13T08:35:48.151 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T08:35:48.172 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-13T08:35:48.492 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-13T08:35:48.534 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-13T08:35:48.551 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-13T08:35:48.592 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-13T08:35:48.829 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-13T08:35:48.836 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package bison. 2024-09-13T08:35:48.845 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-13T08:35:48.878 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-13T08:35:48.910 INFO:teuthology.orchestra.run.smithi044.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143363 files and directories currently installed.) 2024-09-13T08:35:48.914 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-13T08:35:48.954 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-13T08:35:49.122 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libjson-perl. 2024-09-13T08:35:49.139 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-13T08:35:49.181 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-13T08:35:49.364 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T08:35:49.379 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-13T08:35:49.413 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-13T08:35:49.509 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-13T08:35:49.525 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-13T08:35:49.567 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-13T08:35:49.750 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package dump. 2024-09-13T08:35:49.766 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-13T08:35:49.816 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking dump (0.4b47-3) ... 2024-09-13T08:35:50.170 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package indent. 2024-09-13T08:35:50.186 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:35:50.229 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-13T08:35:50.523 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T08:35:50.540 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-13T08:35:50.581 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-13T08:35:50.910 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T08:35:50.927 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-13T08:35:50.962 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T08:35:50.968 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-13T08:35:50.978 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T08:35:51.020 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T08:35:51.205 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T08:35:51.221 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-13T08:35:51.254 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-13T08:35:51.298 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T08:35:51.314 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T08:35:51.348 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T08:35:51.591 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-13T08:35:51.608 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-13T08:35:51.650 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-13T08:35:51.667 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T08:35:51.683 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-13T08:35:51.725 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T08:35:51.937 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-13T08:35:51.952 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-13T08:35:51.993 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-13T08:35:52.086 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T08:35:52.103 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-13T08:35:52.145 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-13T08:35:52.246 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libjson-perl. 2024-09-13T08:35:52.263 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-13T08:35:52.305 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-13T08:35:52.499 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T08:35:52.515 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-13T08:35:52.549 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-13T08:35:52.633 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-13T08:35:52.650 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-13T08:35:52.691 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-13T08:35:52.926 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T08:35:52.942 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:35:52.985 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:35:53.362 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T08:35:53.378 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:35:53.421 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T08:35:53.723 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T08:35:53.740 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:35:53.778 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T08:35:54.018 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T08:35:54.034 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-13T08:35:54.061 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T08:35:54.076 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T08:35:54.078 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T08:35:54.119 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T08:35:54.405 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T08:35:54.422 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T08:35:54.463 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T08:35:54.571 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T08:35:54.586 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-13T08:35:54.646 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-13T08:35:54.791 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T08:35:54.808 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-13T08:35:54.849 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T08:35:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package modemmanager. 2024-09-13T08:35:54.931 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-13T08:35:54.965 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T08:35:55.211 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T08:35:55.228 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-13T08:35:55.269 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-13T08:35:55.555 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T08:35:55.572 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-13T08:35:55.613 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-13T08:35:55.623 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package network-manager. 2024-09-13T08:35:55.638 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-13T08:35:55.688 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-13T08:35:55.966 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T08:35:55.983 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:35:56.025 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:35:56.288 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ppp. 2024-09-13T08:35:56.304 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-13T08:35:56.394 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T08:35:56.411 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:35:56.453 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T08:35:56.655 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-13T08:35:56.738 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T08:35:56.756 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:35:56.797 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T08:35:57.001 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T08:35:57.015 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-13T08:35:57.058 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T08:35:57.058 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-13T08:35:57.075 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-13T08:35:57.116 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T08:35:57.270 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T08:35:57.284 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-13T08:35:57.345 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-13T08:35:57.686 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T08:35:57.703 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-13T08:35:57.740 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T08:35:57.744 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-13T08:35:57.755 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-13T08:35:57.798 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-client-common (238) ... 2024-09-13T08:35:58.106 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package modemmanager. 2024-09-13T08:35:58.123 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-13T08:35:58.127 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-13T08:35:58.143 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:35:58.164 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T08:35:58.185 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:35:58.698 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T08:35:58.714 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-13T08:35:58.775 INFO:teuthology.orchestra.run.smithi129.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T08:35:58.797 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-common (238) ... 2024-09-13T08:35:58.829 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package network-manager. 2024-09-13T08:35:58.846 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-13T08:35:58.896 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-13T08:35:59.255 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-14. 2024-09-13T08:35:59.271 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:35:59.316 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:35:59.519 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package ppp. 2024-09-13T08:35:59.537 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-13T08:35:59.896 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-13T08:36:00.241 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T08:36:00.257 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-13T08:36:00.299 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-13T08:36:00.466 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql. 2024-09-13T08:36:00.482 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-13T08:36:00.524 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql (14+238) ... 2024-09-13T08:36:00.569 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T08:36:00.584 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-13T08:36:00.619 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-13T08:36:00.810 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T08:36:00.825 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-13T08:36:00.868 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-client (14+238) ... 2024-09-13T08:36:00.880 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T08:36:00.896 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-13T08:36:00.938 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking postgresql-client-common (238) ... 2024-09-13T08:36:01.096 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T08:36:01.111 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-13T08:36:01.148 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-13T08:36:01.275 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-13T08:36:01.291 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:36:01.333 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:36:01.431 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package tcl8.6. 2024-09-13T08:36:01.446 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-13T08:36:01.489 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-13T08:36:01.750 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package tcl. 2024-09-13T08:36:01.766 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-13T08:36:01.821 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-13T08:36:01.938 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T08:36:01.954 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-13T08:36:02.015 INFO:teuthology.orchestra.run.smithi044.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T08:36:02.045 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking postgresql-common (238) ... 2024-09-13T08:36:02.137 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T08:36:02.153 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-13T08:36:02.186 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-13T08:36:02.543 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T08:36:02.561 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-13T08:36:02.929 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package postgresql-14. 2024-09-13T08:36:02.946 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T08:36:02.957 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-13T08:36:02.990 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:36:03.335 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T08:36:03.352 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-13T08:36:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-13T08:36:03.721 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-client-common (238) ... 2024-09-13T08:36:03.907 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-13T08:36:04.008 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-13T08:36:04.160 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:36:04.173 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package postgresql. 2024-09-13T08:36:04.189 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-13T08:36:04.231 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking postgresql (14+238) ... 2024-09-13T08:36:04.294 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-13T08:36:04.437 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-13T08:36:04.509 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T08:36:04.524 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-13T08:36:04.567 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking postgresql-client (14+238) ... 2024-09-13T08:36:04.588 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:36:04.794 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T08:36:04.810 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-13T08:36:04.852 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-13T08:36:05.188 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package tcl8.6. 2024-09-13T08:36:05.204 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-13T08:36:05.247 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-13T08:36:05.525 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package tcl. 2024-09-13T08:36:05.541 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-13T08:36:05.559 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T08:36:05.587 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-13T08:36:05.660 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-13T08:36:05.961 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T08:36:05.978 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-13T08:36:06.019 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-13T08:36:06.062 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T08:36:06.062 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T08:36:06.356 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T08:36:06.373 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-13T08:36:06.439 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-13T08:36:06.788 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-13T08:36:06.793 INFO:teuthology.orchestra.run.smithi044.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T08:36:06.810 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-13T08:36:06.859 INFO:teuthology.orchestra.run.smithi044.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-13T08:36:06.938 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-13T08:36:07.090 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T08:36:07.171 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up postgresql-client-common (238) ... 2024-09-13T08:36:07.240 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-13T08:36:07.381 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-13T08:36:07.500 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-13T08:36:07.516 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-13T08:36:07.626 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:36:07.653 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-13T08:36:07.777 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-13T08:36:07.804 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-13T08:36:07.903 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-13T08:36:08.045 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:36:08.364 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-13T08:36:09.017 INFO:teuthology.orchestra.run.smithi044.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T08:36:09.126 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-13T08:36:09.145 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-13T08:36:09.287 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up dump (0.4b47-3) ... 2024-09-13T08:36:09.399 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T08:36:09.472 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T08:36:09.512 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T08:36:09.512 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T08:36:09.623 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-13T08:36:09.799 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-13T08:36:09.950 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-13T08:36:10.084 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-13T08:36:10.174 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-13T08:36:10.245 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-13T08:36:10.254 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-13T08:36:10.380 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-13T08:36:10.413 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-13T08:36:10.481 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T08:36:10.589 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-13T08:36:10.598 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-13T08:36:10.698 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T08:36:10.733 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-13T08:36:10.799 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-13T08:36:10.859 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-13T08:36:10.942 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-client (14+238) ... 2024-09-13T08:36:11.018 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-13T08:36:11.093 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-13T08:36:11.203 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-13T08:36:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:11.651 INFO:teuthology.orchestra.run.smithi129.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T08:36:11.652 INFO:teuthology.orchestra.run.smithi129.stdout:which means they are currently configured by ifupdown: 2024-09-13T08:36:11.652 INFO:teuthology.orchestra.run.smithi129.stdout:- enp3s0f1 2024-09-13T08:36:11.652 INFO:teuthology.orchestra.run.smithi129.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T08:36:11.652 INFO:teuthology.orchestra.run.smithi129.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T08:36:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:11.784 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T08:36:11.866 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-13T08:36:11.992 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up dump (0.4b47-3) ... 2024-09-13T08:36:12.087 INFO:teuthology.orchestra.run.smithi044.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T08:36:12.152 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T08:36:12.158 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T08:36:12.303 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-13T08:36:12.429 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-13T08:36:12.551 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T08:36:12.555 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-13T08:36:12.673 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-13T08:36:12.745 INFO:teuthology.orchestra.run.smithi044.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-13T08:36:12.816 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-13T08:36:12.975 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-13T08:36:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-13T08:36:13.245 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T08:36:13.277 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T08:36:13.380 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-13T08:36:13.403 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-13T08:36:13.515 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-13T08:36:13.596 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up postgresql-client (14+238) ... 2024-09-13T08:36:13.640 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-common (238) ... 2024-09-13T08:36:13.714 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-13T08:36:14.534 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:14.534 INFO:teuthology.orchestra.run.smithi044.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T08:36:14.534 INFO:teuthology.orchestra.run.smithi044.stdout:which means they are currently configured by ifupdown: 2024-09-13T08:36:14.534 INFO:teuthology.orchestra.run.smithi044.stdout:- ens1f0 2024-09-13T08:36:14.534 INFO:teuthology.orchestra.run.smithi044.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T08:36:14.534 INFO:teuthology.orchestra.run.smithi044.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T08:36:14.535 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:14.610 INFO:teuthology.orchestra.run.smithi129.stdout:Adding user postgres to group ssl-cert 2024-09-13T08:36:14.684 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T08:36:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:14.848 INFO:teuthology.orchestra.run.smithi129.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T08:36:15.069 INFO:teuthology.orchestra.run.smithi129.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T08:36:15.069 INFO:teuthology.orchestra.run.smithi129.stdout:Removing obsolete dictionary files: 2024-09-13T08:36:15.096 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T08:36:15.518 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T08:36:15.793 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T08:36:16.234 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T08:36:16.250 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:36:16.385 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-13T08:36:16.511 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-13T08:36:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-13T08:36:16.602 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-13T08:36:16.637 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up postgresql-common (238) ... 2024-09-13T08:36:16.653 INFO:teuthology.orchestra.run.smithi129.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T08:36:16.653 INFO:teuthology.orchestra.run.smithi129.stdout:This user must also own the server process. 2024-09-13T08:36:16.653 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:16.654 INFO:teuthology.orchestra.run.smithi129.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T08:36:16.654 INFO:teuthology.orchestra.run.smithi129.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T08:36:16.654 INFO:teuthology.orchestra.run.smithi129.stdout:The default text search configuration will be set to "english". 2024-09-13T08:36:16.654 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:16.654 INFO:teuthology.orchestra.run.smithi129.stdout:Data page checksums are disabled. 2024-09-13T08:36:16.654 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:16.654 INFO:teuthology.orchestra.run.smithi129.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-13T08:36:16.654 INFO:teuthology.orchestra.run.smithi129.stdout:creating subdirectories ... ok 2024-09-13T08:36:16.655 INFO:teuthology.orchestra.run.smithi129.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T08:36:16.678 INFO:teuthology.orchestra.run.smithi129.stdout:selecting default max_connections ... 100 2024-09-13T08:36:16.712 INFO:teuthology.orchestra.run.smithi129.stdout:selecting default shared_buffers ... 128MB 2024-09-13T08:36:16.726 INFO:teuthology.orchestra.run.smithi129.stdout:selecting default time zone ... Etc/UTC 2024-09-13T08:36:16.727 INFO:teuthology.orchestra.run.smithi129.stdout:creating configuration files ... ok 2024-09-13T08:36:16.887 INFO:teuthology.orchestra.run.smithi129.stdout:running bootstrap script ... ok 2024-09-13T08:36:17.503 INFO:teuthology.orchestra.run.smithi129.stdout:performing post-bootstrap initialization ... ok 2024-09-13T08:36:17.631 INFO:teuthology.orchestra.run.smithi044.stdout:Adding user postgres to group ssl-cert 2024-09-13T08:36:17.891 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:17.891 INFO:teuthology.orchestra.run.smithi044.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T08:36:18.064 INFO:teuthology.orchestra.run.smithi129.stdout:syncing data to disk ... ok 2024-09-13T08:36:18.153 INFO:teuthology.orchestra.run.smithi044.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T08:36:18.153 INFO:teuthology.orchestra.run.smithi044.stdout:Removing obsolete dictionary files: 2024-09-13T08:36:18.595 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T08:36:18.918 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T08:36:19.388 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T08:36:19.745 INFO:teuthology.orchestra.run.smithi044.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-13T08:36:19.764 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-13T08:36:19.820 INFO:teuthology.orchestra.run.smithi044.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T08:36:19.820 INFO:teuthology.orchestra.run.smithi044.stdout:This user must also own the server process. 2024-09-13T08:36:19.820 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:19.820 INFO:teuthology.orchestra.run.smithi044.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T08:36:19.820 INFO:teuthology.orchestra.run.smithi044.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T08:36:19.820 INFO:teuthology.orchestra.run.smithi044.stdout:The default text search configuration will be set to "english". 2024-09-13T08:36:19.820 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:19.820 INFO:teuthology.orchestra.run.smithi044.stdout:Data page checksums are disabled. 2024-09-13T08:36:19.820 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:19.820 INFO:teuthology.orchestra.run.smithi044.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-13T08:36:19.821 INFO:teuthology.orchestra.run.smithi044.stdout:creating subdirectories ... ok 2024-09-13T08:36:19.821 INFO:teuthology.orchestra.run.smithi044.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T08:36:19.845 INFO:teuthology.orchestra.run.smithi044.stdout:selecting default max_connections ... 100 2024-09-13T08:36:19.880 INFO:teuthology.orchestra.run.smithi044.stdout:selecting default shared_buffers ... 128MB 2024-09-13T08:36:19.894 INFO:teuthology.orchestra.run.smithi044.stdout:selecting default time zone ... Etc/UTC 2024-09-13T08:36:19.895 INFO:teuthology.orchestra.run.smithi044.stdout:creating configuration files ... ok 2024-09-13T08:36:20.064 INFO:teuthology.orchestra.run.smithi044.stdout:running bootstrap script ... ok 2024-09-13T08:36:20.691 INFO:teuthology.orchestra.run.smithi044.stdout:performing post-bootstrap initialization ... ok 2024-09-13T08:36:21.277 INFO:teuthology.orchestra.run.smithi044.stdout:syncing data to disk ... ok 2024-09-13T08:36:21.616 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T08:36:21.742 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-13T08:36:21.809 INFO:teuthology.orchestra.run.smithi044.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T08:36:21.851 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql (14+238) ... 2024-09-13T08:36:21.986 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T08:36:22.087 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T08:36:22.347 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T08:36:22.347 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T08:36:23.064 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T08:36:23.156 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T08:36:23.600 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T08:36:23.769 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T08:36:24.889 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T08:36:25.032 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-13T08:36:25.158 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up postgresql (14+238) ... 2024-09-13T08:36:25.163 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-13T08:36:25.264 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T08:36:25.283 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T08:36:25.409 INFO:teuthology.orchestra.run.smithi044.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T08:36:25.677 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T08:36:25.677 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T08:36:25.844 INFO:teuthology.orchestra.run.smithi129.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T08:36:25.852 DEBUG:teuthology.parallel:result is None 2024-09-13T08:36:26.437 INFO:teuthology.orchestra.run.smithi044.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T08:36:26.529 INFO:teuthology.orchestra.run.smithi044.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T08:36:26.865 INFO:teuthology.orchestra.run.smithi044.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T08:36:27.050 INFO:teuthology.orchestra.run.smithi044.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T08:36:28.536 INFO:teuthology.orchestra.run.smithi044.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-13T08:36:28.636 INFO:teuthology.orchestra.run.smithi044.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T08:36:29.213 INFO:teuthology.orchestra.run.smithi044.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T08:36:29.220 DEBUG:teuthology.parallel:result is None 2024-09-13T08:36:29.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d454760ef8a93a9dd7f4e4dedfb085faac216e7e 2024-09-13T08:36:29.365 DEBUG:teuthology.orchestra.run.smithi044:> dpkg-query -W -f '${Version}' ceph 2024-09-13T08:36:29.390 INFO:teuthology.orchestra.run.smithi044.stdout:19.3.0-4881-gd454760e-1jammy 2024-09-13T08:36:29.390 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4881-gd454760e-1jammy 2024-09-13T08:36:29.391 INFO:teuthology.task.install:The correct ceph version 19.3.0-4881-gd454760e-1jammy is installed. 2024-09-13T08:36:29.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d454760ef8a93a9dd7f4e4dedfb085faac216e7e 2024-09-13T08:36:29.534 DEBUG:teuthology.orchestra.run.smithi129:> dpkg-query -W -f '${Version}' ceph 2024-09-13T08:36:29.558 INFO:teuthology.orchestra.run.smithi129.stdout:19.3.0-4881-gd454760e-1jammy 2024-09-13T08:36:29.559 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4881-gd454760e-1jammy 2024-09-13T08:36:29.559 INFO:teuthology.task.install:The correct ceph version 19.3.0-4881-gd454760e-1jammy is installed. 2024-09-13T08:36:29.561 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T08:36:29.561 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:29.561 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T08:36:29.581 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:36:29.582 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T08:36:29.625 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T08:36:29.626 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:29.626 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T08:36:29.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T08:36:29.708 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:36:29.708 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T08:36:29.728 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T08:36:29.792 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T08:36:29.792 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:29.792 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T08:36:29.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T08:36:29.871 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:36:29.872 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T08:36:29.892 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T08:36:29.956 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T08:36:29.956 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:29.956 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T08:36:29.975 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T08:36:30.036 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:36:30.036 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T08:36:30.056 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T08:36:30.120 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T08:36:30.212 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T08:36:30.213 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /var/log/ceph 2024-09-13T08:36:30.216 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /var/log/ceph 2024-09-13T08:36:30.236 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T08:36:30.236 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T08:36:30.280 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T08:36:30.303 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T08:36:30.303 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T08:36:30.339 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T08:36:30.371 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-13T08:36:30.371 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-13T08:36:30.371 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-13T08:36:30.371 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:30.371 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T08:36:30.409 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T08:36:30.503 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:36:30.503 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T08:36:30.512 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T08:36:30.614 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T08:36:30.614 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T08:36:30.614 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319', 'branch': 'wip-athakkar-testing-2024-09-12-1849', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': None, 'first_in_suite': False, 'job_id': '7903319', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-athakkar-testing-2024-09-12-1849'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'd454760ef8a93a9dd7f4e4dedfb085faac216e7e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd454760ef8a93a9dd7f4e4dedfb085faac216e7e'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-athakkar-testing-2024-09-12-1849', 'sha1': 'd454760ef8a93a9dd7f4e4dedfb085faac216e7e'}}, 'owner': 'scheduled_khiremat@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6477, 'sha1': 'd454760ef8a93a9dd7f4e4dedfb085faac216e7e', 'sleep_before_teardown': 0, 'subset': '3/4', 'suite': 'fs:volumes', 'suite_branch': 'wip-athakkar-testing-2024-09-12-1849', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d454760ef8a93a9dd7f4e4dedfb085faac216e7e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd454760ef8a93a9dd7f4e4dedfb085faac216e7e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi044.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBDdRvWRbk0VlKlnw7uTrP5D7c3O3liJr+aRepiHJTGT8wNouA4Lk82ueVRyHIzNt5B6KiJlRDd/bTjP9kulgY8=', 'smithi129.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEcBbt6vGhvBFz8tsw5W/hccsCmEzHENOa/XgFXwX32N5wRc0056tOEpesdyLvJzcu2CCHZxYiHBsXeJD9FBKSo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd454760ef8a93a9dd7f4e4dedfb085faac216e7e'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'd454760ef8a93a9dd7f4e4dedfb085faac216e7e', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_08:07:46', 'tube': 'smithi', 'user': 'khiremat', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T08:36:30.616 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T08:36:30.620 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T08:36:30.629 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T08:36:30.672 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T08:36:30.696 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:30.696 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T08:36:30.737 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T08:36:30.737 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-09-13T08:36:30.788 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T08:36:30.789 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T08:36:30.789 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 897 Links: 1 2024-09-13T08:36:30.789 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T08:36:30.789 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 08:29:45.568320557 +0000 2024-09-13T08:36:30.789 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 08:29:45.268323238 +0000 2024-09-13T08:36:30.789 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 08:29:45.268323238 +0000 2024-09-13T08:36:30.789 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 08:29:45.268323238 +0000 2024-09-13T08:36:30.789 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T08:36:30.850 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T08:36:30.851 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T08:36:30.851 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000646323 s, 792 kB/s 2024-09-13T08:36:30.852 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T08:36:30.902 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-09-13T08:36:30.956 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T08:36:30.957 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T08:36:30.957 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 914 Links: 1 2024-09-13T08:36:30.957 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T08:36:30.957 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 08:29:46.164315233 +0000 2024-09-13T08:36:30.957 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 08:29:45.868317878 +0000 2024-09-13T08:36:30.957 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 08:29:45.868317878 +0000 2024-09-13T08:36:30.957 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 08:29:45.868317878 +0000 2024-09-13T08:36:30.957 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T08:36:31.018 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T08:36:31.019 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T08:36:31.019 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000344074 s, 1.5 MB/s 2024-09-13T08:36:31.020 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T08:36:31.079 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-09-13T08:36:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T08:36:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T08:36:31.129 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 931 Links: 1 2024-09-13T08:36:31.129 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T08:36:31.129 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 08:29:46.768309839 +0000 2024-09-13T08:36:31.129 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 08:29:46.468312519 +0000 2024-09-13T08:36:31.130 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 08:29:46.468312519 +0000 2024-09-13T08:36:31.130 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 08:29:46.468312519 +0000 2024-09-13T08:36:31.130 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T08:36:31.191 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T08:36:31.191 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T08:36:31.191 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000347202 s, 1.5 MB/s 2024-09-13T08:36:31.193 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T08:36:31.247 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-09-13T08:36:31.297 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T08:36:31.297 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T08:36:31.297 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 951 Links: 1 2024-09-13T08:36:31.297 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T08:36:31.297 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 08:29:47.316304945 +0000 2024-09-13T08:36:31.297 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 08:29:47.060307230 +0000 2024-09-13T08:36:31.297 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 08:29:47.060307230 +0000 2024-09-13T08:36:31.297 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 08:29:47.060307230 +0000 2024-09-13T08:36:31.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T08:36:31.359 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T08:36:31.359 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T08:36:31.359 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000359346 s, 1.4 MB/s 2024-09-13T08:36:31.361 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T08:36:31.419 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T08:36:31.420 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:36:31.420 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T08:36:31.429 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T08:36:31.429 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-09-13T08:36:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T08:36:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T08:36:31.481 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 6h/6d Inode: 893 Links: 1 2024-09-13T08:36:31.481 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T08:36:31.481 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-13 08:29:45.191539323 +0000 2024-09-13T08:36:31.481 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-13 08:29:44.927541681 +0000 2024-09-13T08:36:31.481 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-13 08:29:44.927541681 +0000 2024-09-13T08:36:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-13 08:29:44.927541681 +0000 2024-09-13T08:36:31.482 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T08:36:31.543 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-13T08:36:31.543 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-13T08:36:31.543 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000401339 s, 1.3 MB/s 2024-09-13T08:36:31.545 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T08:36:31.603 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-09-13T08:36:31.657 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T08:36:31.657 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T08:36:31.657 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 6h/6d Inode: 912 Links: 1 2024-09-13T08:36:31.657 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T08:36:31.657 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-13 08:29:45.775534106 +0000 2024-09-13T08:36:31.658 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-13 08:29:45.495536608 +0000 2024-09-13T08:36:31.658 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-13 08:29:45.495536608 +0000 2024-09-13T08:36:31.658 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-13 08:29:45.495536608 +0000 2024-09-13T08:36:31.658 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T08:36:31.719 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-13T08:36:31.719 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-13T08:36:31.719 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000391278 s, 1.3 MB/s 2024-09-13T08:36:31.721 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T08:36:31.774 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-09-13T08:36:31.825 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T08:36:31.825 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T08:36:31.825 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 6h/6d Inode: 927 Links: 1 2024-09-13T08:36:31.825 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T08:36:31.825 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-13 08:29:46.071531463 +0000 2024-09-13T08:36:31.825 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-13 08:29:46.067531498 +0000 2024-09-13T08:36:31.825 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-13 08:29:46.067531498 +0000 2024-09-13T08:36:31.825 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-13 08:29:46.067531498 +0000 2024-09-13T08:36:31.826 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T08:36:31.886 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-13T08:36:31.887 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-13T08:36:31.887 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000616312 s, 831 kB/s 2024-09-13T08:36:31.889 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T08:36:31.942 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-09-13T08:36:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T08:36:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T08:36:31.992 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 6h/6d Inode: 946 Links: 1 2024-09-13T08:36:31.993 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T08:36:31.993 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-13 08:29:46.891524138 +0000 2024-09-13T08:36:31.993 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-13 08:29:46.611526639 +0000 2024-09-13T08:36:31.993 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-13 08:29:46.611526639 +0000 2024-09-13T08:36:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-13 08:29:46.611526639 +0000 2024-09-13T08:36:31.993 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T08:36:32.055 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-13T08:36:32.055 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-13T08:36:32.055 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000388292 s, 1.3 MB/s 2024-09-13T08:36:32.056 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T08:36:32.111 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T08:36:32.111 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T08:36:32.111 INFO:tasks.ceph:Generating config... 2024-09-13T08:36:32.112 INFO:tasks.ceph:[client] pid file = 2024-09-13T08:36:32.112 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-13T08:36:32.112 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-13T08:36:32.112 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T08:36:32.113 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-13T08:36:32.113 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T08:36:32.113 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T08:36:32.113 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T08:36:32.113 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T08:36:32.113 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T08:36:32.113 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-13T08:36:32.113 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T08:36:32.113 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T08:36:32.113 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-13T08:36:32.113 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T08:36:32.113 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T08:36:32.113 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T08:36:32.113 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T08:36:32.113 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T08:36:32.114 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T08:36:32.114 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T08:36:32.114 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T08:36:32.114 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T08:36:32.114 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T08:36:32.114 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T08:36:32.114 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T08:36:32.114 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T08:36:32.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T08:36:32.162 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T08:36:32.180 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T08:36:32.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T08:36:32.292 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.44'), ('mon.b', '172.21.15.129'), ('mon.c', '[v2:172.21.15.129:3301,v1:172.21.15.129:6790]')] 2024-09-13T08:36:32.293 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.44,172.21.15.129,[v2:172.21.15.129:3301,v1:172.21.15.129:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-13T08:36:32.293 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:32.293 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T08:36:32.345 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.44 --add b 172.21.15.129 --addv c '[v2:172.21.15.129:3301,v1:172.21.15.129:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:32.441 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: generated fsid d85be7ac-239d-467b-bce8-dbc7fa81939e 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:setting min_mon_release = quincy 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 0 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:fsid d85be7ac-239d-467b-bce8-dbc7fa81939e 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:last_changed 2024-09-13T08:36:32.442919+0000 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-09-13T08:36:32.442919+0000 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:min_mon_release 17 (quincy) 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:election_strategy: 1 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-09-13T08:36:32.442 INFO:teuthology.orchestra.run.smithi044.stdout:2: [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] mon.c 2024-09-13T08:36:32.443 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T08:36:32.445 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T08:36:32.453 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d85be7ac-239d-467b-bce8-dbc7fa81939e... 2024-09-13T08:36:32.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T08:36:32.503 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T08:36:32.543 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: chdir = "" 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: auth supported = cephx 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: filestore xattr use omap = true 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: mon clock drift allowed = 1.000 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: osd crush chooseleaf type = 0 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: auth debug = true 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on old message = true 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on bug = true 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: mon pg warn max object skew = 0 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default size = 2 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow primary affinity = true 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow pg remap = true 2024-09-13T08:36:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on legacy crush tunables = false 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on crush straw calc version zero = false 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on no sortbitwise = false 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on osd down out interval zero = false 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on too few osds = false 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: mon_allow_pool_size_one = true 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: osd default data pool replay window = 5 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: mon allow pool delete = true 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: mon cluster log file level = debug 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: debug asserts on shutdown = true 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: mon health detail to clog = false 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: mon host = "172.21.15.44,172.21.15.129,[v2:172.21.15.129:3301,v1:172.21.15.129:6790]" 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on skipped message = False 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = d85be7ac-239d-467b-bce8-dbc7fa81939e 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout:[osd] 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: osd journal size = 100 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub load threshold = 5.0 2024-09-13T08:36:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub max interval = 600 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: osd mclock profile = high_recovery_ops 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: osd recover clone overlap = true 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: osd recovery max chunk = 1048576 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug shutdown = true 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug op order = true 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug verify stray on activate = true 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug trim objects = true 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: osd open classes on start = true 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug pg log writeout = true 2024-09-13T08:36:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: osd deep scrub update digest min age = 30 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: osd map max advance = 10 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: journal zero on create = true 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: filestore ondisk finisher threads = 3 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: filestore apply finisher threads = 3 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: bdev debug aio = true 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug misdirected ops = true 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: bdev async discard = True 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: bdev enable discard = True 2024-09-13T08:36:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore allocator = bitmap 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore block size = 96636764160 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore fsck on mount = True 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluefs = 1/20 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluestore = 1/20 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: debug osd = 20 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: debug rocksdb = 4/10 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd full ratio = 0.9 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd nearfull ratio = 0.8 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: osd failsafe full ratio = 0.95 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: osd objectstore = bluestore 2024-09-13T08:36:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr] 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: debug mgr = 20 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: mgr/telemetry/nag = false 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: client mount timeout = 30 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: debug client = 20 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: debug finisher = 20 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout:[mon] 2024-09-13T08:36:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: debug paxos = 20 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: mon data avail warn = 5 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: mon mgr mkfs grace = 240 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd reporter subtree level = osd 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd prime pg temp = true 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T08:36:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: # 1m isn't quite enough 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: mon_down_mkfs_grace = 2m 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_filestore_osds = false 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: mon down mkfs grace = 300 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout:[client] 2024-09-13T08:36:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: rgw cache enabled = true 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable ops log = true 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable usage log = true 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: pid file = "" 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.a] 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.b] 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.c] 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c] 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b] 2024-09-13T08:36:32.554 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d] 2024-09-13T08:36:32.555 INFO:teuthology.orchestra.run.smithi129.stdout:[global] 2024-09-13T08:36:32.555 INFO:teuthology.orchestra.run.smithi129.stdout: chdir = "" 2024-09-13T08:36:32.555 INFO:teuthology.orchestra.run.smithi129.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T08:36:32.555 INFO:teuthology.orchestra.run.smithi129.stdout: auth supported = cephx 2024-09-13T08:36:32.555 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.555 INFO:teuthology.orchestra.run.smithi129.stdout: filestore xattr use omap = true 2024-09-13T08:36:32.555 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.555 INFO:teuthology.orchestra.run.smithi129.stdout: mon clock drift allowed = 1.000 2024-09-13T08:36:32.555 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.555 INFO:teuthology.orchestra.run.smithi129.stdout: osd crush chooseleaf type = 0 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: auth debug = true 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on old message = true 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on bug = true 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: mon pg warn max object skew = 0 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default size = 2 2024-09-13T08:36:32.556 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow primary affinity = true 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow pg remap = true 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on legacy crush tunables = false 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on crush straw calc version zero = false 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on no sortbitwise = false 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on osd down out interval zero = false 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on too few osds = false 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: mon_allow_pool_size_one = true 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T08:36:32.557 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: osd default data pool replay window = 5 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: mon allow pool delete = true 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: mon cluster log file level = debug 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: debug asserts on shutdown = true 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: mon health detail to clog = false 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: mon host = "172.21.15.44,172.21.15.129,[v2:172.21.15.129:3301,v1:172.21.15.129:6790]" 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on skipped message = False 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: fsid = d85be7ac-239d-467b-bce8-dbc7fa81939e 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout:[osd] 2024-09-13T08:36:32.558 INFO:teuthology.orchestra.run.smithi129.stdout: osd journal size = 100 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub load threshold = 5.0 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub max interval = 600 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: osd mclock profile = high_recovery_ops 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: osd recover clone overlap = true 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: osd recovery max chunk = 1048576 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug shutdown = true 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug op order = true 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug verify stray on activate = true 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug trim objects = true 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.559 INFO:teuthology.orchestra.run.smithi129.stdout: osd open classes on start = true 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug pg log writeout = true 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: osd deep scrub update digest min age = 30 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: osd map max advance = 10 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: journal zero on create = true 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: filestore ondisk finisher threads = 3 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: filestore apply finisher threads = 3 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: bdev debug aio = true 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug misdirected ops = true 2024-09-13T08:36:32.560 INFO:teuthology.orchestra.run.smithi129.stdout: bdev async discard = True 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: bdev enable discard = True 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore allocator = bitmap 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore block size = 96636764160 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore fsck on mount = True 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluefs = 1/20 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluestore = 1/20 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: debug osd = 20 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: debug rocksdb = 4/10 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd full ratio = 0.9 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd nearfull ratio = 0.8 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: osd failsafe full ratio = 0.95 2024-09-13T08:36:32.561 INFO:teuthology.orchestra.run.smithi129.stdout: osd objectstore = bluestore 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr] 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: debug mgr = 20 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: mgr/telemetry/nag = false 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: client mount timeout = 30 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: debug client = 20 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: debug finisher = 20 2024-09-13T08:36:32.562 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout:[mon] 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: debug paxos = 20 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: mon data avail warn = 5 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: mon mgr mkfs grace = 240 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd reporter subtree level = osd 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd prime pg temp = true 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.563 INFO:teuthology.orchestra.run.smithi129.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: # 1m isn't quite enough 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: mon_down_mkfs_grace = 2m 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_filestore_osds = false 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: mon down mkfs grace = 300 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T08:36:32.564 INFO:teuthology.orchestra.run.smithi129.stdout:[client] 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout: rgw cache enabled = true 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable ops log = true 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable usage log = true 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout: pid file = "" 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.a] 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.b] 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.c] 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.a] 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.c] 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.b] 2024-09-13T08:36:32.565 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.d] 2024-09-13T08:36:32.567 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T08:36:32.567 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T08:36:32.666 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T08:36:32.666 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:32.667 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T08:36:32.675 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:32.676 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T08:36:32.724 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:36:32.724 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:32.724 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T08:36:32.724 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T08:36:32.800 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:36:32.800 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:32.848 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:36:32.849 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:36:32.849 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T08:36:32.849 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T08:36:32.884 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:36:32.884 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:32.934 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T08:36:32.934 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T08:36:32.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T08:36:32.990 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T08:36:32.995 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T08:36:33.055 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T08:36:33.069 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T08:36:33.070 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T08:36:33.126 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T08:36:33.131 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-13T08:36:33.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T08:36:33.248 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T08:36:33.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-13T08:36:33.274 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T08:36:33.333 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T08:36:33.338 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-13T08:36:33.358 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T08:36:33.454 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T08:36:33.459 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-13T08:36:33.478 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T08:36:33.479 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T08:36:33.522 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T08:36:33.539 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-13T08:36:33.626 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-13T08:36:33.643 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T08:36:33.687 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T08:36:33.703 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-13T08:36:33.786 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-13T08:36:33.803 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T08:36:33.804 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T08:36:33.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T08:36:33.823 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T08:36:33.823 INFO:tasks.ceph:role: osd.0 2024-09-13T08:36:33.824 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:36:33.824 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T08:36:33.892 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T08:36:33.893 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T08:36:33.893 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T08:36:33.893 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T08:36:33.893 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T08:36:33.893 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-09-13T08:36:33.893 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T08:36:33.893 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T08:36:33.893 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T08:36:33.893 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T08:36:34.291 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-09-13T08:36:34.294 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-09-13T08:36:34.294 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T08:36:34.384 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T08:36:34.441 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T08:36:34.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:36:34.442 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T08:36:34.506 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T08:36:34.506 INFO:tasks.ceph:role: osd.1 2024-09-13T08:36:34.506 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:36:34.507 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T08:36:34.580 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T08:36:34.580 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T08:36:34.580 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T08:36:34.580 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T08:36:34.580 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T08:36:34.581 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-09-13T08:36:34.581 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T08:36:34.581 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T08:36:34.581 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T08:36:34.581 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T08:36:34.981 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-09-13T08:36:34.984 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-09-13T08:36:34.984 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T08:36:35.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T08:36:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T08:36:35.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:36:35.131 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T08:36:35.192 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T08:36:35.193 INFO:tasks.ceph:role: osd.2 2024-09-13T08:36:35.193 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:36:35.193 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T08:36:35.264 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T08:36:35.265 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T08:36:35.265 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T08:36:35.265 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T08:36:35.265 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T08:36:35.265 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-09-13T08:36:35.265 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T08:36:35.265 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T08:36:35.265 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T08:36:35.265 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T08:36:35.678 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-09-13T08:36:35.681 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-09-13T08:36:35.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T08:36:35.822 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T08:36:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T08:36:35.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:36:35.841 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T08:36:35.906 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T08:36:35.906 INFO:tasks.ceph:role: osd.3 2024-09-13T08:36:35.906 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:36:35.906 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T08:36:35.977 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T08:36:35.978 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T08:36:35.978 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T08:36:35.978 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T08:36:35.978 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T08:36:35.978 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-09-13T08:36:35.978 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T08:36:35.978 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T08:36:35.978 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T08:36:35.978 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T08:36:36.342 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-09-13T08:36:36.345 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-09-13T08:36:36.346 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T08:36:36.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T08:36:36.460 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T08:36:36.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:36:36.462 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:36.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:36.577+0000 7fa825b207c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T08:36:36.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:36.581+0000 7fa825b207c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T08:36:36.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:36.581+0000 7fa825b207c0 -1 bdev(0x55ba5a9af800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-13T08:36:36.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:36.581+0000 7fa825b207c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T08:36:39.844 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T08:36:39.866 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:39.965+0000 7f9b259507c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T08:36:39.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:39.965+0000 7f9b259507c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T08:36:39.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:39.969+0000 7f9b259507c0 -1 bdev(0x55ff83709800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-13T08:36:39.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:39.969+0000 7f9b259507c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T08:36:43.201 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T08:36:43.223 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:43.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:43.317+0000 7f52f3aba7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T08:36:43.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:43.317+0000 7f52f3aba7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T08:36:43.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:43.317+0000 7f52f3aba7c0 -1 bdev(0x559f4e897800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-13T08:36:43.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:43.317+0000 7f52f3aba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T08:36:46.531 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T08:36:46.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:46.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:46.641+0000 7f94d2eea7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T08:36:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:46.645+0000 7f94d2eea7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T08:36:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:46.645+0000 7f94d2eea7c0 -1 bdev(0x564e064d5800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-13T08:36:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:36:46.645+0000 7f94d2eea7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T08:36:49.935 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T08:36:49.956 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T08:36:49.978 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T08:36:49.978 INFO:tasks.ceph:role: osd.4 2024-09-13T08:36:49.978 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:36:49.978 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T08:36:50.052 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T08:36:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T08:36:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T08:36:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T08:36:50.052 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T08:36:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-13T08:36:50.053 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T08:36:50.053 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T08:36:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T08:36:50.053 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T08:36:50.454 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-13T08:36:50.456 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-13T08:36:50.457 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T08:36:50.559 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T08:36:50.577 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T08:36:50.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:36:50.578 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T08:36:50.640 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T08:36:50.640 INFO:tasks.ceph:role: osd.5 2024-09-13T08:36:50.640 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:36:50.640 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T08:36:50.711 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T08:36:50.712 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T08:36:50.712 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T08:36:50.712 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T08:36:50.712 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T08:36:50.712 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-13T08:36:50.712 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T08:36:50.712 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T08:36:50.712 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T08:36:50.712 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T08:36:51.120 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-13T08:36:51.123 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-13T08:36:51.123 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T08:36:51.262 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T08:36:51.279 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T08:36:51.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:36:51.280 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T08:36:51.345 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T08:36:51.345 INFO:tasks.ceph:role: osd.6 2024-09-13T08:36:51.345 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:36:51.345 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T08:36:51.416 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T08:36:51.416 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T08:36:51.416 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T08:36:51.416 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T08:36:51.416 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T08:36:51.416 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-13T08:36:51.416 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T08:36:51.417 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T08:36:51.417 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T08:36:51.417 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T08:36:51.828 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-13T08:36:51.831 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-13T08:36:51.831 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T08:36:51.957 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T08:36:51.973 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T08:36:51.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:36:51.974 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T08:36:52.040 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T08:36:52.040 INFO:tasks.ceph:role: osd.7 2024-09-13T08:36:52.040 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:36:52.040 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T08:36:52.115 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T08:36:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T08:36:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T08:36:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T08:36:52.115 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T08:36:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-13T08:36:52.115 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T08:36:52.115 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T08:36:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T08:36:52.116 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T08:36:52.483 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-13T08:36:52.486 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-13T08:36:52.486 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T08:36:52.569 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T08:36:52.632 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T08:36:52.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:36:52.634 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:52.767 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:52.764+0000 7f1cbacd67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T08:36:52.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:52.768+0000 7f1cbacd67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T08:36:52.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:52.768+0000 7f1cbacd67c0 -1 bdev(0x561bd12cd800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-13T08:36:52.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:52.768+0000 7f1cbacd67c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T08:36:55.983 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T08:36:56.005 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:56.100+0000 7f497e43d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T08:36:56.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:56.100+0000 7f497e43d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T08:36:56.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:56.100+0000 7f497e43d7c0 -1 bdev(0x561066c7f800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-13T08:36:56.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:56.100+0000 7f497e43d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T08:36:59.323 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T08:36:59.345 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:36:59.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:59.440+0000 7f0916cf17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T08:36:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:59.444+0000 7f0916cf17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T08:36:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:59.444+0000 7f0916cf17c0 -1 bdev(0x556b2ad0f800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-13T08:36:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:36:59.444+0000 7f0916cf17c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T08:37:00.617 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:37:00.621 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:37:02.930 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T08:37:02.951 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:37:03.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:03.040+0000 7f607c5677c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T08:37:03.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:03.044+0000 7f607c5677c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T08:37:03.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:03.044+0000 7f607c5677c0 -1 bdev(0x5562b1aa9800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-13T08:37:03.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:03.044+0000 7f607c5677c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T08:37:06.246 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T08:37:06.266 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T08:37:06.266 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:06.266 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T08:37:06.289 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:06.289 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-13T08:37:06.358 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:06.358 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-13T08:37:06.426 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:06.426 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T08:37:06.490 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:06.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T08:37:06.553 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:06.554 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T08:37:06.617 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:06.618 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T08:37:06.682 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:06.682 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T08:37:06.701 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:06.702 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-13T08:37:06.764 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:06.764 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-13T08:37:06.824 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:06.824 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T08:37:06.887 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:06.888 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T08:37:06.951 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:06.951 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T08:37:07.011 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:07.012 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T08:37:07.076 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:07.076 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T08:37:07.085 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:07.085 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-13T08:37:07.137 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:07.137 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T08:37:07.145 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:07.146 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-13T08:37:07.196 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T08:37:07.196 DEBUG:teuthology.orchestra.run.smithi044:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T08:37:07.199 DEBUG:teuthology.orchestra.run.smithi129:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T08:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.y] 2024-09-13T08:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg== 2024-09-13T08:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2024-09-13T08:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw== 2024-09-13T08:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c] 2024-09-13T08:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA== 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.0] 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg== 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.1] 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg== 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.2] 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw== 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.3] 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig== 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.x] 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ== 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b] 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ== 2024-09-13T08:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d] 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw== 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.4] 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ== 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.5] 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g== 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.6] 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg== 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.7] 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg== 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout:[client.2] 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA== 2024-09-13T08:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout:[client.1] 2024-09-13T08:37:07.247 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w== 2024-09-13T08:37:07.247 INFO:teuthology.orchestra.run.smithi044.stdout:[client.3] 2024-09-13T08:37:07.247 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w== 2024-09-13T08:37:07.257 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.y] 2024-09-13T08:37:07.257 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg== 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.a] 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw== 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.c] 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA== 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.0] 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg== 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.1] 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg== 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.2] 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw== 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.3] 2024-09-13T08:37:07.258 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig== 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.x] 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ== 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.b] 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ== 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.d] 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw== 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.4] 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ== 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.5] 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g== 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.6] 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg== 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.7] 2024-09-13T08:37:07.259 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg== 2024-09-13T08:37:07.260 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2024-09-13T08:37:07.260 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:37:07.260 INFO:teuthology.orchestra.run.smithi129.stdout:[client.2] 2024-09-13T08:37:07.260 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA== 2024-09-13T08:37:07.260 INFO:teuthology.orchestra.run.smithi129.stdout:[client.1] 2024-09-13T08:37:07.260 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w== 2024-09-13T08:37:07.260 INFO:teuthology.orchestra.run.smithi129.stdout:[client.3] 2024-09-13T08:37:07.260 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w== 2024-09-13T08:37:07.261 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T08:37:07.292 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T08:37:07.362 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T08:37:07.366 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T08:37:07.424 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T08:37:07.430 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T08:37:07.486 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.489 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.546 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.602 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.656 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.659 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.713 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T08:37:07.716 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T08:37:07.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T08:37:07.773 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T08:37:07.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T08:37:07.834 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T08:37:07.891 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.894 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.948 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:07.951 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:08.005 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:08.008 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:08.063 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:08.066 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T08:37:08.120 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T08:37:08.124 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T08:37:08.181 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T08:37:08.184 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T08:37:08.241 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T08:37:08.244 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T08:37:08.302 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T08:37:08.305 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T08:37:08.361 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T08:37:08.362 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T08:37:08.382 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T08:37:08.495 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T08:37:08.515 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T08:37:08.534 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T08:37:08.646 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T08:37:08.666 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T08:37:08.730 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T08:37:08.839 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T08:37:08.859 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:37:08.864 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T08:37:08.912 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T08:37:08.913 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T08:37:08.913 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T08:37:08.916 INFO:tasks.ceph.mon.a:Started 2024-09-13T08:37:08.916 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T08:37:08.916 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T08:37:08.959 INFO:tasks.ceph.mon.b:Started 2024-09-13T08:37:08.959 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T08:37:08.959 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T08:37:08.963 INFO:tasks.ceph.mon.c:Started 2024-09-13T08:37:08.963 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T08:37:08.963 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T08:37:08.963 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T08:37:08.966 INFO:tasks.ceph.mgr.y:Started 2024-09-13T08:37:08.967 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T08:37:08.967 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T08:37:08.969 INFO:tasks.ceph.mgr.x:Started 2024-09-13T08:37:08.970 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-13T08:37:08.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-13T08:37:08.974 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-13T08:37:08.974 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-13T08:37:09.020 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-13T08:37:09.020 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-13T08:37:09.072 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-13T08:37:09.072 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-13T08:37:09.119 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-13T08:37:09.119 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-13T08:37:09.168 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-13T08:37:09.168 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-13T08:37:09.220 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-13T08:37:09.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-13T08:37:09.267 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-13T08:37:09.267 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-13T08:37:09.319 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-13T08:37:09.319 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-13T08:37:09.368 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-13T08:37:09.368 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-13T08:37:09.424 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-13T08:37:09.424 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-13T08:37:09.476 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-13T08:37:09.476 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-13T08:37:09.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.496+0000 7fa6b9600640 1 -- 172.21.15.44:0/721266307 <== mon.2 v2:172.21.15.129:3301/0 10 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v4)=0 v4) ==== 109+0+0 (secure 0 0 0) 0x7fa6a8010a40 con 0x7fa6bc159460 2024-09-13T08:37:09.504 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-13T08:37:09.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-13T08:37:09.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.504+0000 7fa6aea00640 1 -- 172.21.15.44:0/721266307 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6bc159460 msgr2=0x7fa6bc0cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.504+0000 7fa6aea00640 1 --2- 172.21.15.44:0/721266307 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6bc159460 0x7fa6bc0cb590 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800d680 tx=0x7fa6a800db40 comp rx=0 tx=0).stop 2024-09-13T08:37:09.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.500+0000 7f0cd4c00640 1 -- 172.21.15.44:0/3289346930 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0cc8018020 con 0x7f0cd80ae4d0 2024-09-13T08:37:09.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.500+0000 7f0cd4c00640 1 -- 172.21.15.44:0/3289346930 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0cc801a570 con 0x7f0cd80ae4d0 2024-09-13T08:37:09.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.504+0000 7f0cde400640 1 -- 172.21.15.44:0/3289346930 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd815eeb0 con 0x7f0cd80ae4d0 2024-09-13T08:37:09.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.508+0000 7fa6aea00640 1 -- 172.21.15.44:0/721266307 shutdown_connections 2024-09-13T08:37:09.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.508+0000 7fa6aea00640 1 --2- 172.21.15.44:0/721266307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6bc0cbad0 0x7fa6bc0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.508+0000 7fa6aea00640 1 --2- 172.21.15.44:0/721266307 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6bc159460 0x7fa6bc0cb590 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.508+0000 7fa6aea00640 1 --2- 172.21.15.44:0/721266307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6a4005320 0x7fa6bc157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.508+0000 7fa6aea00640 1 -- 172.21.15.44:0/721266307 >> 172.21.15.44:0/721266307 conn(0x7fa6bc0d1dd0 msgr2=0x7fa6bc15e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.516 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-13T08:37:09.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-13T08:37:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.512+0000 7f0cd4c00640 1 -- 172.21.15.44:0/3289346930 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0cc8010440 con 0x7f0cd80ae4d0 2024-09-13T08:37:09.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.516+0000 7fa6aea00640 1 -- 172.21.15.44:0/721266307 shutdown_connections 2024-09-13T08:37:09.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.516+0000 7fa6aea00640 1 -- 172.21.15.44:0/721266307 wait complete. 2024-09-13T08:37:09.550 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-13T08:37:09.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-13T08:37:09.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.556+0000 7ff057800640 1 -- 172.21.15.44:0/1630072044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7ff024005300 con 0x7ff05006cc20 2024-09-13T08:37:09.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.560+0000 7f6fec000640 1 -- 172.21.15.44:0/2328138891 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6fdc0e2f10 msgr2=0x7f6fdc034370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.560+0000 7f6fec000640 1 --2- 172.21.15.44:0/2328138891 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6fdc0e2f10 0x7f6fdc034370 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0013f80 tx=0x7f6fd0024e60 comp rx=0 tx=0).stop 2024-09-13T08:37:09.565 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-13T08:37:09.566 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-13T08:37:09.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.564+0000 7fdcefe00640 1 -- 172.21.15.44:0/2126992659 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fdcc4005320 con 0x7fdcf015f760 2024-09-13T08:37:09.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.564+0000 7f6fec000640 1 -- 172.21.15.44:0/2328138891 shutdown_connections 2024-09-13T08:37:09.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.564+0000 7f6fec000640 1 --2- 172.21.15.44:0/2328138891 >> v2:172.21.15.129:3300/0 conn(0x7f6fdc035f40 0x7f6fdc0363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.564+0000 7f6fec000640 1 --2- 172.21.15.44:0/2328138891 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6fdc0e2f10 0x7f6fdc034370 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.564+0000 7f6fec000640 1 -- 172.21.15.44:0/2328138891 >> 172.21.15.44:0/2328138891 conn(0x7f6fdc01d7b0 msgr2=0x7f6fdc0545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.568+0000 7f6fec000640 1 -- 172.21.15.44:0/2328138891 shutdown_connections 2024-09-13T08:37:09.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.568+0000 7f6fec000640 1 -- 172.21.15.44:0/2328138891 wait complete. 2024-09-13T08:37:09.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.572+0000 7f6fec000640 1 Processor -- start 2024-09-13T08:37:09.576 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-13T08:37:09.577 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-13T08:37:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.572+0000 7fdce7400640 1 -- 172.21.15.44:0/2126992659 <== mon.2 v2:172.21.15.129:3301/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fdce8014cf0 con 0x7fdcf015f760 2024-09-13T08:37:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.572+0000 7f0cd4c00640 1 -- 172.21.15.44:0/3289346930 <== mon.2 v2:172.21.15.129:3301/0 7 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f0cc8010730 con 0x7f0cd80ae4d0 2024-09-13T08:37:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.580+0000 7ff04cc00640 1 -- 172.21.15.44:0/1630072044 <== mon.1 v2:172.21.15.129:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7ff0440037b0 con 0x7ff05006cc20 2024-09-13T08:37:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.580+0000 7ff04cc00640 1 -- 172.21.15.44:0/1630072044 <== mon.1 v2:172.21.15.129:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v5)=0 v5) ==== 114+0+0 (secure 0 0 0) 0x7ff044003b20 con 0x7ff05006cc20 2024-09-13T08:37:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.580+0000 7ff04cc00640 1 -- 172.21.15.44:0/1630072044 <== mon.1 v2:172.21.15.129:3300/0 12 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff044003dc0 con 0x7ff05006cc20 2024-09-13T08:37:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.580+0000 7fdce7400640 1 -- 172.21.15.44:0/2126992659 <== mon.2 v2:172.21.15.129:3301/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdce80274e0 con 0x7fdcf015f760 2024-09-13T08:37:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.580+0000 7fdce7400640 1 -- 172.21.15.44:0/2126992659 <== mon.2 v2:172.21.15.129:3301/0 12 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7fdce80145e0 con 0x7fdcf015f760 2024-09-13T08:37:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.580+0000 7f0cd4c00640 1 -- 172.21.15.44:0/3289346930 <== mon.2 v2:172.21.15.129:3301/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cc8010ab0 con 0x7f0cd80ae4d0 2024-09-13T08:37:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fec000640 1 -- start start 2024-09-13T08:37:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fec000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc005a40 0x7f6fdc0dcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fec000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fdc035f40 0x7f6fdc0da170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fec000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6fdc0e2f10 0x7f6fdc0da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fec000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc055490 con 0x7f6fdc005a40 2024-09-13T08:37:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fec000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc055310 con 0x7f6fdc035f40 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fec000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc033930 con 0x7f6fdc0e2f10 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fe8c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fdc035f40 0x7f6fdc0da170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fea000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6fdc0e2f10 0x7f6fdc0da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fe8c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fdc035f40 0x7f6fdc0da170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:49204/0 (socket says 172.21.15.44:49204) 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fe8c00640 1 -- 172.21.15.44:0/3252927735 learned_addr learned my addr 172.21.15.44:0/3252927735 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fe9600640 1 --2- 172.21.15.44:0/3252927735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc005a40 0x7f6fdc0dcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fe8c00640 1 -- 172.21.15.44:0/3252927735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6fdc0e2f10 msgr2=0x7f6fdc0da6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fe8c00640 1 --2- 172.21.15.44:0/3252927735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6fdc0e2f10 0x7f6fdc0da6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fe8c00640 1 -- 172.21.15.44:0/3252927735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc005a40 msgr2=0x7f6fdc0dcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fe8c00640 1 --2- 172.21.15.44:0/3252927735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc005a40 0x7f6fdc0dcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.576+0000 7f6fe8c00640 1 -- 172.21.15.44:0/3252927735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc17c630 con 0x7f6fdc035f40 2024-09-13T08:37:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.580+0000 7f6fe8c00640 1 --2- 172.21.15.44:0/3252927735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fdc035f40 0x7f6fdc0da170 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800b4a0 tx=0x7f6fd800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:09.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.580+0000 7f6fe1600640 1 -- 172.21.15.44:0/3252927735 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8014070 con 0x7f6fdc035f40 2024-09-13T08:37:09.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.580+0000 7f6fe1600640 1 -- 172.21.15.44:0/3252927735 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fd8010610 con 0x7f6fdc035f40 2024-09-13T08:37:09.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.580+0000 7f6fe1600640 1 -- 172.21.15.44:0/3252927735 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8010aa0 con 0x7f6fdc035f40 2024-09-13T08:37:09.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.588+0000 7f6fec000640 1 -- 172.21.15.44:0/3252927735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc0d9110 con 0x7f6fdc035f40 2024-09-13T08:37:09.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.588+0000 7f6fec000640 1 -- 172.21.15.44:0/3252927735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc0d9690 con 0x7f6fdc035f40 2024-09-13T08:37:09.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.588+0000 7f6fe1600640 1 -- 172.21.15.44:0/3252927735 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6fd8017070 con 0x7f6fdc035f40 2024-09-13T08:37:09.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.588+0000 7f6fe1600640 1 -- 172.21.15.44:0/3252927735 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6fd800fa40 con 0x7f6fdc035f40 2024-09-13T08:37:09.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7f6fec000640 1 -- 172.21.15.44:0/3252927735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc0363a0 con 0x7f6fdc035f40 2024-09-13T08:37:09.600 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-13T08:37:09.600 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-13T08:37:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7ff057800640 1 -- 172.21.15.44:0/1630072044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff05006cc20 msgr2=0x7ff050156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7ff057800640 1 --2- 172.21.15.44:0/1630072044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff05006cc20 0x7ff050156e30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff04400f7d0 tx=0x7ff04400fc90 comp rx=0 tx=0).stop 2024-09-13T08:37:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7ff057800640 1 -- 172.21.15.44:0/1630072044 shutdown_connections 2024-09-13T08:37:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7ff057800640 1 --2- 172.21.15.44:0/1630072044 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff0500d0060 0x7ff0500c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7ff057800640 1 --2- 172.21.15.44:0/1630072044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0500cd060 0x7ff0500b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7ff057800640 1 --2- 172.21.15.44:0/1630072044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff05006cc20 0x7ff050156e30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7ff057800640 1 -- 172.21.15.44:0/1630072044 >> 172.21.15.44:0/1630072044 conn(0x7ff0500cd740 msgr2=0x7ff0500c3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7ff057800640 1 -- 172.21.15.44:0/1630072044 shutdown_connections 2024-09-13T08:37:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7ff057800640 1 -- 172.21.15.44:0/1630072044 wait complete. 2024-09-13T08:37:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.592+0000 7f6fe1600640 1 -- 172.21.15.44:0/3252927735 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6fd801e030 con 0x7f6fdc035f40 2024-09-13T08:37:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.604+0000 7fdcefe00640 1 -- 172.21.15.44:0/2126992659 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdcf015f760 msgr2=0x7fdcf00b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.604+0000 7fdcefe00640 1 --2- 172.21.15.44:0/2126992659 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdcf015f760 0x7fdcf00b1b30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdce800b6d0 tx=0x7fdce800bb90 comp rx=0 tx=0).stop 2024-09-13T08:37:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.604+0000 7fdcefe00640 1 -- 172.21.15.44:0/2126992659 shutdown_connections 2024-09-13T08:37:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.604+0000 7fdcefe00640 1 --2- 172.21.15.44:0/2126992659 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdcf015f760 0x7fdcf00b1b30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.604+0000 7fdcefe00640 1 --2- 172.21.15.44:0/2126992659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcf015adc0 0x7fdcf00b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.604+0000 7fdcefe00640 1 --2- 172.21.15.44:0/2126992659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf00d09c0 0x7fdcf0154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.604+0000 7fdcefe00640 1 -- 172.21.15.44:0/2126992659 >> 172.21.15.44:0/2126992659 conn(0x7fdcf00742f0 msgr2=0x7fdcf00ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.612+0000 7fdcefe00640 1 -- 172.21.15.44:0/2126992659 shutdown_connections 2024-09-13T08:37:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.612+0000 7fdcefe00640 1 -- 172.21.15.44:0/2126992659 wait complete. 2024-09-13T08:37:09.617 DEBUG:tasks.ceph:set 18 configs 2024-09-13T08:37:09.618 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi044') 2024-09-13T08:37:09.618 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi044') 2024-09-13T08:37:09.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.612+0000 7fd3b4400640 1 -- 172.21.15.44:0/1538061668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a40337b0 msgr2=0x7fd3a4039870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.612+0000 7fd3b4400640 1 --2- 172.21.15.44:0/1538061668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a40337b0 0x7fd3a4039870 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400d4a0 tx=0x7fd3a001c820 comp rx=0 tx=0).stop 2024-09-13T08:37:09.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.612+0000 7fd3b4400640 1 -- 172.21.15.44:0/1538061668 shutdown_connections 2024-09-13T08:37:09.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.612+0000 7fd3b4400640 1 --2- 172.21.15.44:0/1538061668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a40337b0 0x7fd3a4039870 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.612+0000 7fd3b4400640 1 --2- 172.21.15.44:0/1538061668 >> v2:172.21.15.129:3300/0 conn(0x7fd3a4039490 0x7fd3a40550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.612+0000 7fd3b4400640 1 -- 172.21.15.44:0/1538061668 >> 172.21.15.44:0/1538061668 conn(0x7fd3a401d950 msgr2=0x7fd3a4054740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.616+0000 7fd3b4400640 1 -- 172.21.15.44:0/1538061668 shutdown_connections 2024-09-13T08:37:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.616+0000 7fd3b4400640 1 -- 172.21.15.44:0/1538061668 wait complete. 2024-09-13T08:37:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.616+0000 7fd3b4400640 1 Processor -- start 2024-09-13T08:37:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.620+0000 7fd3b4400640 1 -- start start 2024-09-13T08:37:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.624+0000 7fd3b4400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a40337b0 0x7fd3a40f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.624+0000 7fd3b4400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3a4039490 0x7fd3a40da970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.624+0000 7fd3b4400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3a40e7250 0x7fd3a40e0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.624+0000 7fd3b2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a40337b0 0x7fd3a40f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.624+0000 7fd3b2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a40337b0 0x7fd3a40f0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45120/0 (socket says 172.21.15.44:45120) 2024-09-13T08:37:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.628+0000 7fd3b2e00640 1 -- 172.21.15.44:0/3589171657 learned_addr learned my addr 172.21.15.44:0/3589171657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.628+0000 7fd3b1000640 1 --2- 172.21.15.44:0/3589171657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3a4039490 0x7fd3a40da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.628+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a4007660 con 0x7fd3a40337b0 2024-09-13T08:37:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.628+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a40074e0 con 0x7fd3a4039490 2024-09-13T08:37:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.628+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a40e13b0 con 0x7fd3a40e7250 2024-09-13T08:37:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.632+0000 7fd3b1a00640 1 --2- 172.21.15.44:0/3589171657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3a40e7250 0x7fd3a40e0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.632+0000 7fd3b1a00640 1 -- 172.21.15.44:0/3589171657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3a4039490 msgr2=0x7fd3a40da970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.632+0000 7fd3b1a00640 1 --2- 172.21.15.44:0/3589171657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3a4039490 0x7fd3a40da970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.632+0000 7fd3b1a00640 1 -- 172.21.15.44:0/3589171657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a40337b0 msgr2=0x7fd3a40f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.632+0000 7fd3b1a00640 1 --2- 172.21.15.44:0/3589171657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a40337b0 0x7fd3a40f0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.632+0000 7fd3b1a00640 1 -- 172.21.15.44:0/3589171657 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a40f1590 con 0x7fd3a40e7250 2024-09-13T08:37:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.632+0000 7fd3b2e00640 1 --2- 172.21.15.44:0/3589171657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a40337b0 0x7fd3a40f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.636+0000 7fd3b1a00640 1 --2- 172.21.15.44:0/3589171657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3a40e7250 0x7fd3a40e0b90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c3f0 tx=0x7fd39800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.640+0000 7fd3a9600640 1 -- 172.21.15.44:0/3589171657 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398017070 con 0x7fd3a40e7250 2024-09-13T08:37:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.640+0000 7fd3a9600640 1 -- 172.21.15.44:0/3589171657 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3980053c0 con 0x7fd3a40e7250 2024-09-13T08:37:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.640+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a40f12b0 con 0x7fd3a40e7250 2024-09-13T08:37:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.640+0000 7fd3a9600640 1 -- 172.21.15.44:0/3589171657 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3980118b0 con 0x7fd3a40e7250 2024-09-13T08:37:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.640+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a4048900 con 0x7fd3a40e7250 2024-09-13T08:37:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.644+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a4039870 con 0x7fd3a40e7250 2024-09-13T08:37:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.648+0000 7fd3a9600640 1 -- 172.21.15.44:0/3589171657 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd39801a070 con 0x7fd3a40e7250 2024-09-13T08:37:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.648+0000 7fd3a9600640 1 -- 172.21.15.44:0/3589171657 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd39801c440 con 0x7fd3a40e7250 2024-09-13T08:37:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.648+0000 7fd3a9600640 1 -- 172.21.15.44:0/3589171657 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd398021020 con 0x7fd3a40e7250 2024-09-13T08:37:09.682 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi044') 2024-09-13T08:37:09.682 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-09-13T08:37:09.682 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-09-13T08:37:09.700 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi044') 2024-09-13T08:37:09.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.740+0000 7fcdabc00640 1 -- 172.21.15.44:0/3378269766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcda40b7fc0 msgr2=0x7fcda409bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.740+0000 7fcdabc00640 1 --2- 172.21.15.44:0/3378269766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcda40b7fc0 0x7fcda409bf10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcda40c7b20 tx=0x7fcd9c01cca0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.748+0000 7fcdabc00640 1 -- 172.21.15.44:0/3378269766 shutdown_connections 2024-09-13T08:37:09.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.748+0000 7fcdabc00640 1 --2- 172.21.15.44:0/3378269766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcda40b7fc0 0x7fcda409bf10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.748+0000 7fcdabc00640 1 --2- 172.21.15.44:0/3378269766 >> v2:172.21.15.129:3300/0 conn(0x7fcda40d39f0 0x7fcda40b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.748+0000 7fcdabc00640 1 -- 172.21.15.44:0/3378269766 >> 172.21.15.44:0/3378269766 conn(0x7fcda40d4050 msgr2=0x7fcda40d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.756+0000 7fcdabc00640 1 -- 172.21.15.44:0/3378269766 shutdown_connections 2024-09-13T08:37:09.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.756+0000 7fcdabc00640 1 -- 172.21.15.44:0/3378269766 wait complete. 2024-09-13T08:37:09.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.756+0000 7fcdabc00640 1 Processor -- start 2024-09-13T08:37:09.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcdabc00640 1 -- start start 2024-09-13T08:37:09.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcdabc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcda40b30b0 0x7fcda415b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcdabc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcda40b7fc0 0x7fcda40cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcdabc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcda40d39f0 0x7fcda415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcdabc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcda416f250 con 0x7fcda40b7fc0 2024-09-13T08:37:09.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcdabc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcda416f0d0 con 0x7fcda40b30b0 2024-09-13T08:37:09.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcdabc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcda416f3d0 con 0x7fcda40d39f0 2024-09-13T08:37:09.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcda40b7fc0 0x7fcda40cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcda40b30b0 0x7fcda415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcda40b30b0 0x7fcda415b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:49238/0 (socket says 172.21.15.44:49238) 2024-09-13T08:37:09.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda3e00640 1 -- 172.21.15.44:0/4008436388 learned_addr learned my addr 172.21.15.44:0/4008436388 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:09.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda9200640 1 --2- 172.21.15.44:0/4008436388 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcda40d39f0 0x7fcda415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda3e00640 1 -- 172.21.15.44:0/4008436388 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcda40d39f0 msgr2=0x7fcda415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda3e00640 1 --2- 172.21.15.44:0/4008436388 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcda40d39f0 0x7fcda415bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda3e00640 1 -- 172.21.15.44:0/4008436388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcda40b7fc0 msgr2=0x7fcda40cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda3e00640 1 --2- 172.21.15.44:0/4008436388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcda40b7fc0 0x7fcda40cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda3e00640 1 -- 172.21.15.44:0/4008436388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda40cba90 con 0x7fcda40b30b0 2024-09-13T08:37:09.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.760+0000 7fcda3e00640 1 --2- 172.21.15.44:0/4008436388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcda40b30b0 0x7fcda415b630 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800b3f0 tx=0x7fcd9800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:09.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.764+0000 7fcda0c00640 1 -- 172.21.15.44:0/4008436388 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98015020 con 0x7fcda40b30b0 2024-09-13T08:37:09.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.764+0000 7fcda0c00640 1 -- 172.21.15.44:0/4008436388 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd98011940 con 0x7fcda40b30b0 2024-09-13T08:37:09.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.764+0000 7fcda0c00640 1 -- 172.21.15.44:0/4008436388 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98011d90 con 0x7fcda40b30b0 2024-09-13T08:37:09.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.764+0000 7fcdabc00640 1 -- 172.21.15.44:0/4008436388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda40cbdd0 con 0x7fcda40b30b0 2024-09-13T08:37:09.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.768+0000 7fcda0c00640 1 -- 172.21.15.44:0/4008436388 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd98018020 con 0x7fcda40b30b0 2024-09-13T08:37:09.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.768+0000 7fcdabc00640 1 -- 172.21.15.44:0/4008436388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda40cc610 con 0x7fcda40b30b0 2024-09-13T08:37:09.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.772+0000 7fcda0c00640 1 -- 172.21.15.44:0/4008436388 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcd980193b0 con 0x7fcda40b30b0 2024-09-13T08:37:09.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.776+0000 7fcdabc00640 1 -- 172.21.15.44:0/4008436388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda409bf10 con 0x7fcda40b30b0 2024-09-13T08:37:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.776+0000 7fcda0c00640 1 -- 172.21.15.44:0/4008436388 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcd9801f020 con 0x7fcda40b30b0 2024-09-13T08:37:09.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.784+0000 7f0cde400640 1 -- 172.21.15.44:0/3289346930 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f0cb0005190 con 0x7f0cd80ae4d0 2024-09-13T08:37:09.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.808+0000 7f0cd4c00640 1 -- 172.21.15.44:0/3289346930 <== mon.2 v2:172.21.15.129:3301/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f0cc8024480 con 0x7f0cd80ae4d0 2024-09-13T08:37:09.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.828+0000 7f0cde400640 1 -- 172.21.15.44:0/3289346930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0cd80ae4d0 msgr2=0x7f0cd82046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.828+0000 7f0cde400640 1 --2- 172.21.15.44:0/3289346930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0cd80ae4d0 0x7f0cd82046f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8001200 tx=0x7f0cc80016c0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.832+0000 7f0cde400640 1 -- 172.21.15.44:0/3289346930 shutdown_connections 2024-09-13T08:37:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.832+0000 7f0cde400640 1 --2- 172.21.15.44:0/3289346930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd80cd5d0 0x7f0cd8157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.832+0000 7f0cde400640 1 --2- 172.21.15.44:0/3289346930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cd80ca050 0x7f0cd8157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.832+0000 7f0cde400640 1 --2- 172.21.15.44:0/3289346930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0cd80ae4d0 0x7f0cd82046f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.832+0000 7f0cde400640 1 -- 172.21.15.44:0/3289346930 >> 172.21.15.44:0/3289346930 conn(0x7f0cd80cfa00 msgr2=0x7f0cd80b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.832+0000 7f0cde400640 1 -- 172.21.15.44:0/3289346930 shutdown_connections 2024-09-13T08:37:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.832+0000 7f0cde400640 1 -- 172.21.15.44:0/3289346930 wait complete. 2024-09-13T08:37:09.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.856+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fd3a4032480 con 0x7fd3a40e7250 2024-09-13T08:37:09.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.860+0000 7f6fec000640 1 -- 172.21.15.44:0/3252927735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f6fdc051110 con 0x7f6fdc035f40 2024-09-13T08:37:09.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.872+0000 7f413f200640 1 -- 172.21.15.44:0/733384925 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4138036620 msgr2=0x7f4138034340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.872+0000 7f413f200640 1 --2- 172.21.15.44:0/733384925 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4138036620 0x7f4138034340 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f412c009870 tx=0x7f412c01c830 comp rx=0 tx=0).stop 2024-09-13T08:37:09.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.872+0000 7f413f200640 1 -- 172.21.15.44:0/733384925 shutdown_connections 2024-09-13T08:37:09.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.872+0000 7f413f200640 1 --2- 172.21.15.44:0/733384925 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4138036620 0x7f4138034340 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.872+0000 7f413f200640 1 -- 172.21.15.44:0/733384925 >> 172.21.15.44:0/733384925 conn(0x7f413801d780 msgr2=0x7f4138054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.872+0000 7f413f200640 1 -- 172.21.15.44:0/733384925 shutdown_connections 2024-09-13T08:37:09.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.876+0000 7fd3a9600640 1 -- 172.21.15.44:0/3589171657 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v8)=0 v8) ==== 106+0+0 (secure 0 0 0) 0x7fd398003600 con 0x7fd3a40e7250 2024-09-13T08:37:09.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.872+0000 7f413f200640 1 -- 172.21.15.44:0/733384925 wait complete. 2024-09-13T08:37:09.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413f200640 1 Processor -- start 2024-09-13T08:37:09.883 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi044') 2024-09-13T08:37:09.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413f200640 1 -- start start 2024-09-13T08:37:09.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3a40e7250 msgr2=0x7fd3a40e0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7fd3b4400640 1 --2- 172.21.15.44:0/3589171657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3a40e7250 0x7fd3a40e0b90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c3f0 tx=0x7fd39800c8b0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 shutdown_connections 2024-09-13T08:37:09.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7fd3b4400640 1 --2- 172.21.15.44:0/3589171657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3a40e7250 0x7fd3a40e0b90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7fd3b4400640 1 --2- 172.21.15.44:0/3589171657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3a4039490 0x7fd3a40da970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7fd3b4400640 1 --2- 172.21.15.44:0/3589171657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a40337b0 0x7fd3a40f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 >> 172.21.15.44:0/3589171657 conn(0x7fd3a401d950 msgr2=0x7fd3a400a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413f200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4138036620 0x7f4138188c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 shutdown_connections 2024-09-13T08:37:09.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7fd3b4400640 1 -- 172.21.15.44:0/3589171657 wait complete. 2024-09-13T08:37:09.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4138052a50 0x7f4138048d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41380e2ee0 0x7f4138049280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4138033a80 con 0x7f41380e2ee0 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4138033900 con 0x7f4138052a50 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413f200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f41380392c0 con 0x7f4138036620 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41380e2ee0 0x7f4138049280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413de00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4138036620 0x7f4138188c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41380e2ee0 0x7f4138049280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45154/0 (socket says 172.21.15.44:45154) 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.880+0000 7f413e800640 1 -- 172.21.15.44:0/2129341841 learned_addr learned my addr 172.21.15.44:0/2129341841 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f413d400640 1 --2- 172.21.15.44:0/2129341841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4138052a50 0x7f4138048d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f413de00640 1 -- 172.21.15.44:0/2129341841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4138052a50 msgr2=0x7f4138048d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f413de00640 1 --2- 172.21.15.44:0/2129341841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4138052a50 0x7f4138048d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f413de00640 1 -- 172.21.15.44:0/2129341841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41380e2ee0 msgr2=0x7f4138049280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f413de00640 1 --2- 172.21.15.44:0/2129341841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41380e2ee0 0x7f4138049280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f413de00640 1 -- 172.21.15.44:0/2129341841 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413804add0 con 0x7f4138036620 2024-09-13T08:37:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f413de00640 1 --2- 172.21.15.44:0/2129341841 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4138036620 0x7f4138188c00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f413000c3e0 tx=0x7f413000c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:09.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f4136000640 1 -- 172.21.15.44:0/2129341841 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41300068a0 con 0x7f4138036620 2024-09-13T08:37:09.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f4136000640 1 -- 172.21.15.44:0/2129341841 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4130006a60 con 0x7f4138036620 2024-09-13T08:37:09.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f4136000640 1 -- 172.21.15.44:0/2129341841 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130007020 con 0x7f4138036620 2024-09-13T08:37:09.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f413f200640 1 -- 172.21.15.44:0/2129341841 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4138048890 con 0x7f4138036620 2024-09-13T08:37:09.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f413f200640 1 -- 172.21.15.44:0/2129341841 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4138000da0 con 0x7f4138036620 2024-09-13T08:37:09.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f4136000640 1 -- 172.21.15.44:0/2129341841 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4130019020 con 0x7f4138036620 2024-09-13T08:37:09.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f4136000640 1 -- 172.21.15.44:0/2129341841 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4130007550 con 0x7f4138036620 2024-09-13T08:37:09.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.884+0000 7f412b400640 1 -- 172.21.15.44:0/2129341841 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41180052a0 con 0x7f4138036620 2024-09-13T08:37:09.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.888+0000 7f4136000640 1 -- 172.21.15.44:0/2129341841 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4130021020 con 0x7f4138036620 2024-09-13T08:37:09.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.892+0000 7f34a7c00640 1 -- 172.21.15.44:0/1306680485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a016e110 msgr2=0x7f34a006ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.892+0000 7f34a7c00640 1 --2- 172.21.15.44:0/1306680485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a016e110 0x7f34a006ec20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f34a00ca290 tx=0x7f348c01c960 comp rx=0 tx=0).stop 2024-09-13T08:37:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.892+0000 7f34a7c00640 1 -- 172.21.15.44:0/1306680485 shutdown_connections 2024-09-13T08:37:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.892+0000 7f34a7c00640 1 --2- 172.21.15.44:0/1306680485 >> v2:172.21.15.129:3300/0 conn(0x7f34a0073090 0x7f34a00734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.892+0000 7f34a7c00640 1 --2- 172.21.15.44:0/1306680485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a016e110 0x7f34a006ec20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.896+0000 7f34a7c00640 1 --2- 172.21.15.44:0/1306680485 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a016dd30 0x7f34a006e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.896+0000 7f34a7c00640 1 -- 172.21.15.44:0/1306680485 >> 172.21.15.44:0/1306680485 conn(0x7f34a00b1600 msgr2=0x7f34a00b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.896+0000 7f6fe1600640 1 -- 172.21.15.44:0/3252927735 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v9)=0 v9) ==== 117+0+0 (secure 0 0 0) 0x7f6fd800fc90 con 0x7f6fdc035f40 2024-09-13T08:37:09.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.896+0000 7f34a7c00640 1 -- 172.21.15.44:0/1306680485 shutdown_connections 2024-09-13T08:37:09.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.896+0000 7f34a7c00640 1 -- 172.21.15.44:0/1306680485 wait complete. 2024-09-13T08:37:09.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.900+0000 7f34a7c00640 1 Processor -- start 2024-09-13T08:37:09.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.904+0000 7f6fec000640 1 -- 172.21.15.44:0/3252927735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fdc035f40 msgr2=0x7f6fdc0da170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.904+0000 7f6fec000640 1 --2- 172.21.15.44:0/3252927735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fdc035f40 0x7f6fdc0da170 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800b4a0 tx=0x7f6fd800b960 comp rx=0 tx=0).stop 2024-09-13T08:37:09.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.908+0000 7f6fec000640 1 -- 172.21.15.44:0/3252927735 shutdown_connections 2024-09-13T08:37:09.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.908+0000 7f6fec000640 1 --2- 172.21.15.44:0/3252927735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6fdc0e2f10 0x7f6fdc0da6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.908+0000 7f6fec000640 1 --2- 172.21.15.44:0/3252927735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fdc035f40 0x7f6fdc0da170 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.908+0000 7f6fec000640 1 --2- 172.21.15.44:0/3252927735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc005a40 0x7f6fdc0dcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.908+0000 7f6fec000640 1 -- 172.21.15.44:0/3252927735 >> 172.21.15.44:0/3252927735 conn(0x7f6fdc01d7b0 msgr2=0x7f6fdc051910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.908+0000 7f34a7c00640 1 -- start start 2024-09-13T08:37:09.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.908+0000 7f6fec000640 1 -- 172.21.15.44:0/3252927735 shutdown_connections 2024-09-13T08:37:09.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f34a7c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a0073090 0x7f34a00beb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f34a7c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a016dd30 0x7f34a020cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f34a7c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a016e110 0x7f34a00c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f34a7c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34a00cc510 con 0x7f34a016dd30 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f34a7c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34a00cc390 con 0x7f34a016e110 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f34a7c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f34a00cc690 con 0x7f34a0073090 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f349fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a016dd30 0x7f34a020cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f349fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a016dd30 0x7f34a020cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45168/0 (socket says 172.21.15.44:45168) 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f349fe00640 1 -- 172.21.15.44:0/731766550 learned_addr learned my addr 172.21.15.44:0/731766550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f349fe00640 1 -- 172.21.15.44:0/731766550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a0073090 msgr2=0x7f34a00beb60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f349fe00640 1 --2- 172.21.15.44:0/731766550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a0073090 0x7f34a00beb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f349fe00640 1 -- 172.21.15.44:0/731766550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a016e110 msgr2=0x7f34a00c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f349fe00640 1 --2- 172.21.15.44:0/731766550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a016e110 0x7f34a00c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f349fe00640 1 -- 172.21.15.44:0/731766550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a020d3c0 con 0x7f34a016dd30 2024-09-13T08:37:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f349fe00640 1 --2- 172.21.15.44:0/731766550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a016dd30 0x7f34a020cc90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f348c01b010 tx=0x7f348c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f349cc00640 1 -- 172.21.15.44:0/731766550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c0bd070 con 0x7f34a016dd30 2024-09-13T08:37:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f34a7c00640 1 -- 172.21.15.44:0/731766550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a020d6a0 con 0x7f34a016dd30 2024-09-13T08:37:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.920+0000 7f34a7c00640 1 -- 172.21.15.44:0/731766550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a00c3550 con 0x7f34a016dd30 2024-09-13T08:37:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.916+0000 7f6fec000640 1 -- 172.21.15.44:0/3252927735 wait complete. 2024-09-13T08:37:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.920+0000 7f349cc00640 1 -- 172.21.15.44:0/731766550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f348c000e50 con 0x7f34a016dd30 2024-09-13T08:37:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.920+0000 7f349cc00640 1 -- 172.21.15.44:0/731766550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c0bd070 con 0x7f34a016dd30 2024-09-13T08:37:09.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.920+0000 7f349cc00640 1 -- 172.21.15.44:0/731766550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f348c0a5930 con 0x7f34a016dd30 2024-09-13T08:37:09.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.920+0000 7f349cc00640 1 -- 172.21.15.44:0/731766550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f348c004100 con 0x7f34a016dd30 2024-09-13T08:37:09.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.924+0000 7f34a7c00640 1 -- 172.21.15.44:0/731766550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a016d2e0 con 0x7f34a016dd30 2024-09-13T08:37:09.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.944+0000 7f349cc00640 1 -- 172.21.15.44:0/731766550 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f348c0165e0 con 0x7f34a016dd30 2024-09-13T08:37:09.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.944+0000 7fc3eae00640 1 -- 172.21.15.44:0/2554960551 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3dc036280 msgr2=0x7fc3dc0e7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.944+0000 7fc3eae00640 1 --2- 172.21.15.44:0/2554960551 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3dc036280 0x7fc3dc0e7120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00098f0 tx=0x7fc3d001c800 comp rx=0 tx=0).stop 2024-09-13T08:37:09.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.956+0000 7fc3eae00640 1 -- 172.21.15.44:0/2554960551 shutdown_connections 2024-09-13T08:37:09.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.956+0000 7fc3eae00640 1 --2- 172.21.15.44:0/2554960551 >> v2:172.21.15.129:3300/0 conn(0x7fc3dc036660 0x7fc3dc0330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.956+0000 7fc3eae00640 1 --2- 172.21.15.44:0/2554960551 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3dc036280 0x7fc3dc0e7120 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.956+0000 7fc3eae00640 1 -- 172.21.15.44:0/2554960551 >> 172.21.15.44:0/2554960551 conn(0x7fc3dc01d7b0 msgr2=0x7fc3dc0545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.961 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi044') 2024-09-13T08:37:09.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.960+0000 7fc3eae00640 1 -- 172.21.15.44:0/2554960551 shutdown_connections 2024-09-13T08:37:09.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.960+0000 7fc3eae00640 1 -- 172.21.15.44:0/2554960551 wait complete. 2024-09-13T08:37:09.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.960+0000 7fc3eae00640 1 Processor -- start 2024-09-13T08:37:09.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.964+0000 7fc3eae00640 1 -- start start 2024-09-13T08:37:09.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.964+0000 7fc3eae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3dc036280 0x7fc3dc048040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3dc036280 0x7fc3dc048040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3dc036280 0x7fc3dc048040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45194/0 (socket says 172.21.15.44:45194) 2024-09-13T08:37:09.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e3e00640 1 -- 172.21.15.44:0/2243513478 learned_addr learned my addr 172.21.15.44:0/2243513478 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:09.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3eae00640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3dc036660 0x7fc3dc0f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3eae00640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3dc0550a0 0x7fc3dc0f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e3400640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3dc036660 0x7fc3dc0f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc0e1210 con 0x7fc3dc036280 2024-09-13T08:37:09.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc0e1390 con 0x7fc3dc0550a0 2024-09-13T08:37:09.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc3dc0e1510 con 0x7fc3dc036660 2024-09-13T08:37:09.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e8e00640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3dc0550a0 0x7fc3dc0f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e8e00640 1 -- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3dc036660 msgr2=0x7fc3dc0f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e8e00640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3dc036660 0x7fc3dc0f2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e8e00640 1 -- 172.21.15.44:0/2243513478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3dc036280 msgr2=0x7fc3dc048040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e8e00640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3dc036280 0x7fc3dc048040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e8e00640 1 -- 172.21.15.44:0/2243513478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc04acb0 con 0x7fc3dc0550a0 2024-09-13T08:37:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.968+0000 7fc3e8e00640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3dc0550a0 0x7fc3dc0f2eb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800c3f0 tx=0x7fc3d800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:09.975 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi044') 2024-09-13T08:37:09.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.976+0000 7fc3d7e00640 1 -- 172.21.15.44:0/2243513478 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8017070 con 0x7fc3dc0550a0 2024-09-13T08:37:09.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.976+0000 7fc3d7e00640 1 -- 172.21.15.44:0/2243513478 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3d80052d0 con 0x7fc3dc0550a0 2024-09-13T08:37:09.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.976+0000 7fc3d7e00640 1 -- 172.21.15.44:0/2243513478 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80054c0 con 0x7fc3dc0550a0 2024-09-13T08:37:09.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.976+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc04af90 con 0x7fc3dc0550a0 2024-09-13T08:37:09.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.976+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc048580 con 0x7fc3dc0550a0 2024-09-13T08:37:09.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.976+0000 7fc3d7e00640 1 -- 172.21.15.44:0/2243513478 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3d801a070 con 0x7fc3dc0550a0 2024-09-13T08:37:09.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.976+0000 7fc3d7e00640 1 -- 172.21.15.44:0/2243513478 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc3d801c440 con 0x7fc3dc0550a0 2024-09-13T08:37:09.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.976+0000 7f1ccea00640 1 -- 172.21.15.44:0/3989014618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc80aecb0 msgr2=0x7f1cc8073fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.976+0000 7f1ccea00640 1 --2- 172.21.15.44:0/3989014618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc80aecb0 0x7f1cc8073fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80cd520 tx=0x7f1cc001cb10 comp rx=0 tx=0).stop 2024-09-13T08:37:09.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.976+0000 7f1ccea00640 1 -- 172.21.15.44:0/3989014618 shutdown_connections 2024-09-13T08:37:09.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.980+0000 7f1ccea00640 1 --2- 172.21.15.44:0/3989014618 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cc8072610 0x7f1cc806e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.980+0000 7f1ccea00640 1 --2- 172.21.15.44:0/3989014618 >> v2:172.21.15.44:3300/0 conn(0x7f1cc80cac60 0x7f1cc80744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.980+0000 7f1ccea00640 1 --2- 172.21.15.44:0/3989014618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc80aecb0 0x7f1cc8073fa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.980+0000 7f1ccea00640 1 -- 172.21.15.44:0/3989014618 >> 172.21.15.44:0/3989014618 conn(0x7f1cc80cfa00 msgr2=0x7f1cc80cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:09.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.980+0000 7f1ccea00640 1 -- 172.21.15.44:0/3989014618 shutdown_connections 2024-09-13T08:37:09.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.980+0000 7f1ccea00640 1 -- 172.21.15.44:0/3989014618 wait complete. 2024-09-13T08:37:09.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.984+0000 7f1ccea00640 1 Processor -- start 2024-09-13T08:37:09.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.980+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3dc0e7120 con 0x7fc3dc0550a0 2024-09-13T08:37:09.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.984+0000 7fc3d7e00640 1 -- 172.21.15.44:0/2243513478 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc3d8022020 con 0x7fc3dc0550a0 2024-09-13T08:37:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.988+0000 7f1ccea00640 1 -- start start 2024-09-13T08:37:09.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1ccea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc8072610 0x7f1cc82022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1ccea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc80aecb0 0x7f1cc815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1ccea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cc80cac60 0x7f1cc815f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:09.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1ccea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc815bbf0 con 0x7f1cc80aecb0 2024-09-13T08:37:09.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1ccea00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc815ba70 con 0x7f1cc8072610 2024-09-13T08:37:09.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1ccea00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc815bd70 con 0x7f1cc80cac60 2024-09-13T08:37:09.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1cce000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cc80cac60 0x7f1cc815f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:09.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1cce000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cc80cac60 0x7f1cc815f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:41068/0 (socket says 172.21.15.44:41068) 2024-09-13T08:37:09.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7fcdabc00640 1 -- 172.21.15.44:0/4008436388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fcda40d47b0 con 0x7fcda40b30b0 2024-09-13T08:37:09.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1cce000640 1 -- 172.21.15.44:0/2926867416 learned_addr learned my addr 172.21.15.44:0/2926867416 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:09.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7f2fcaa00640 1 -- 172.21.15.44:0/2277581672 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fc40cbcf0 msgr2=0x7f2fc4159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:09.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7f2fcaa00640 1 --2- 172.21.15.44:0/2277581672 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fc40cbcf0 0x7f2fc4159ef0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009a70 tx=0x7f2fbc01dbc0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7f2fcaa00640 1 -- 172.21.15.44:0/2277581672 shutdown_connections 2024-09-13T08:37:09.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7f2fcaa00640 1 --2- 172.21.15.44:0/2277581672 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fc40cbcf0 0x7f2fc4159ef0 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009a70 tx=0x7f2fbc01dbc0 comp rx=0 tx=0).stop 2024-09-13T08:37:09.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7f2fcaa00640 1 -- 172.21.15.44:0/2277581672 >> 172.21.15.44:0/2277581672 conn(0x7f2fc40b1600 msgr2=0x7f2fc40b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7fee12800640 1 -- 172.21.15.44:0/236844436 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fee040e2f10 msgr2=0x7fee04034370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7fee12800640 1 --2- 172.21.15.44:0/236844436 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fee040e2f10 0x7fee04034370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee00009920 tx=0x7fee0001c8f0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1cce000640 1 -- 172.21.15.44:0/2926867416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc8072610 msgr2=0x7f1cc82022e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T08:37:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1cce000640 1 --2- 172.21.15.44:0/2926867416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc8072610 0x7f1cc82022e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1cce000640 1 -- 172.21.15.44:0/2926867416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc80aecb0 msgr2=0x7f1cc815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1cce000640 1 --2- 172.21.15.44:0/2926867416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc80aecb0 0x7f1cc815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1cce000640 1 -- 172.21.15.44:0/2926867416 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc80c6ad0 con 0x7f1cc80cac60 2024-09-13T08:37:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1cce000640 1 --2- 172.21.15.44:0/2926867416 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cc80cac60 0x7f1cc815f020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00e970 tx=0x7f1cbc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.992+0000 7f1cc5600640 1 -- 172.21.15.44:0/2926867416 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00ccc0 con 0x7f1cc80cac60 2024-09-13T08:37:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7f1cc5600640 1 -- 172.21.15.44:0/2926867416 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cbc010430 con 0x7f1cc80cac60 2024-09-13T08:37:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7f1cc5600640 1 -- 172.21.15.44:0/2926867416 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc010880 con 0x7f1cc80cac60 2024-09-13T08:37:10.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7f2fcaa00640 1 -- 172.21.15.44:0/2277581672 shutdown_connections 2024-09-13T08:37:10.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7f1ccea00640 1 -- 172.21.15.44:0/2926867416 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc80c6d40 con 0x7f1cc80cac60 2024-09-13T08:37:10.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:09.996+0000 7f2fcaa00640 1 -- 172.21.15.44:0/2277581672 wait complete. 2024-09-13T08:37:10.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.000+0000 7f2fcaa00640 1 Processor -- start 2024-09-13T08:37:10.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.000+0000 7f2fcaa00640 1 -- start start 2024-09-13T08:37:10.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.000+0000 7f1ccea00640 1 -- 172.21.15.44:0/2926867416 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc80c7300 con 0x7f1cc80cac60 2024-09-13T08:37:10.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fcaa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc40cc440 0x7f2fc4206580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fcaa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fc4159b10 0x7f2fc4156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fcaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fc4156cb0 0x7f2fc4158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fcaa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc41687e0 con 0x7f2fc4156cb0 2024-09-13T08:37:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7fee12800640 1 -- 172.21.15.44:0/236844436 shutdown_connections 2024-09-13T08:37:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7fee12800640 1 --2- 172.21.15.44:0/236844436 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fee040e2f10 0x7fee04034370 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7fee12800640 1 -- 172.21.15.44:0/236844436 >> 172.21.15.44:0/236844436 conn(0x7fee0401d7b0 msgr2=0x7fee040545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fcaa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc4168660 con 0x7f2fc40cc440 2024-09-13T08:37:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f1cc5600640 1 -- 172.21.15.44:0/2926867416 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1cbc01f020 con 0x7f1cc80cac60 2024-09-13T08:37:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f1cc5600640 1 -- 172.21.15.44:0/2926867416 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1cbc005320 con 0x7f1cc80cac60 2024-09-13T08:37:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f1cbaa00640 1 -- 172.21.15.44:0/2926867416 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc806e650 con 0x7f1cc80cac60 2024-09-13T08:37:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fcaa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc4168960 con 0x7f2fc4159b10 2024-09-13T08:37:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fc8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fc4156cb0 0x7f2fc4158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fc8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fc4156cb0 0x7f2fc4158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45236/0 (socket says 172.21.15.44:45236) 2024-09-13T08:37:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2be5e00640 1 -- 172.21.15.44:0/2682249191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be006e740 msgr2=0x7f2be00ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2be5e00640 1 --2- 172.21.15.44:0/2682249191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be006e740 0x7f2be00ceb70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2be0074d00 tx=0x7f2bc801c770 comp rx=0 tx=0).stop 2024-09-13T08:37:10.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fc8a00640 1 -- 172.21.15.44:0/1638492776 learned_addr learned my addr 172.21.15.44:0/1638492776 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:10.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fc3e00640 1 --2- 172.21.15.44:0/1638492776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc40cc440 0x7f2fc4206580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7fee12800640 1 -- 172.21.15.44:0/236844436 shutdown_connections 2024-09-13T08:37:10.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fc8a00640 1 -- 172.21.15.44:0/1638492776 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fc4159b10 msgr2=0x7f2fc4156770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:37:10.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fc8a00640 1 --2- 172.21.15.44:0/1638492776 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fc4159b10 0x7f2fc4156770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fc8a00640 1 -- 172.21.15.44:0/1638492776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc40cc440 msgr2=0x7f2fc4206580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fc8a00640 1 --2- 172.21.15.44:0/1638492776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc40cc440 0x7f2fc4206580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fc8a00640 1 -- 172.21.15.44:0/1638492776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc42062a0 con 0x7f2fc4156cb0 2024-09-13T08:37:10.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2be5e00640 1 -- 172.21.15.44:0/2682249191 shutdown_connections 2024-09-13T08:37:10.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2be5e00640 1 --2- 172.21.15.44:0/2682249191 >> v2:172.21.15.44:3300/0 conn(0x7f2be00cce40 0x7f2be00cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2be5e00640 1 --2- 172.21.15.44:0/2682249191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be006e740 0x7f2be00ceb70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2be5e00640 1 -- 172.21.15.44:0/2682249191 >> 172.21.15.44:0/2682249191 conn(0x7f2be00d1be0 msgr2=0x7f2be00d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7fee12800640 1 -- 172.21.15.44:0/236844436 wait complete. 2024-09-13T08:37:10.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2be5e00640 1 -- 172.21.15.44:0/2682249191 shutdown_connections 2024-09-13T08:37:10.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2be5e00640 1 -- 172.21.15.44:0/2682249191 wait complete. 2024-09-13T08:37:10.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.004+0000 7f2fc8a00640 1 --2- 172.21.15.44:0/1638492776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fc4156cb0 0x7f2fc4158d60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c430 tx=0x7f2fb400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:10.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7f2be5e00640 1 Processor -- start 2024-09-13T08:37:10.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7f2fbbe00640 1 -- 172.21.15.44:0/1638492776 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4017070 con 0x7f2fc4156cb0 2024-09-13T08:37:10.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7f2fbbe00640 1 -- 172.21.15.44:0/1638492776 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fb4003600 con 0x7f2fc4156cb0 2024-09-13T08:37:10.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7f2fbbe00640 1 -- 172.21.15.44:0/1638492776 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4003a50 con 0x7f2fc4156cb0 2024-09-13T08:37:10.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7fcda0c00640 1 -- 172.21.15.44:0/4008436388 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7fcda40cbdd0 con 0x7fcda40b30b0 2024-09-13T08:37:10.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7fee12800640 1 Processor -- start 2024-09-13T08:37:10.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7fee12800640 1 -- start start 2024-09-13T08:37:10.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7f2be5e00640 1 -- start start 2024-09-13T08:37:10.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7fcdabc00640 1 -- 172.21.15.44:0/4008436388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcda40b30b0 msgr2=0x7fcda415b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7fcdabc00640 1 --2- 172.21.15.44:0/4008436388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcda40b30b0 0x7fcda415b630 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800b3f0 tx=0x7fcd9800b8b0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7f1cc5600640 1 -- 172.21.15.44:0/2926867416 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1cbc026070 con 0x7f1cc80cac60 2024-09-13T08:37:10.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7fee12800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee04035f40 0x7fee040d0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fcdabc00640 1 -- 172.21.15.44:0/4008436388 shutdown_connections 2024-09-13T08:37:10.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fcdabc00640 1 --2- 172.21.15.44:0/4008436388 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcda40d39f0 0x7fcda415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fcdabc00640 1 --2- 172.21.15.44:0/4008436388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcda40b7fc0 0x7fcda40cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fcdabc00640 1 --2- 172.21.15.44:0/4008436388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcda40b30b0 0x7fcda415b630 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fcdabc00640 1 -- 172.21.15.44:0/4008436388 >> 172.21.15.44:0/4008436388 conn(0x7fcda40d4050 msgr2=0x7fcda40b7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7f2fcaa00640 1 -- 172.21.15.44:0/1638492776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc40c7980 con 0x7f2fc4156cb0 2024-09-13T08:37:10.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7f2fcaa00640 1 -- 172.21.15.44:0/1638492776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc4206ac0 con 0x7f2fc4156cb0 2024-09-13T08:37:10.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.008+0000 7f2fcaa00640 1 -- 172.21.15.44:0/1638492776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc4159ef0 con 0x7f2fc4156cb0 2024-09-13T08:37:10.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2fbbe00640 1 -- 172.21.15.44:0/1638492776 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2fb401a070 con 0x7f2fc4156cb0 2024-09-13T08:37:10.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2be5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be006e740 0x7f2be00cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2be5e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be00cce40 0x7f2be0155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2be5e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be015e030 0x7f2be01fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2be5e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2be016aec0 con 0x7f2be006e740 2024-09-13T08:37:10.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2be5e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2be016ad40 con 0x7f2be00cce40 2024-09-13T08:37:10.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2be5e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2be016b040 con 0x7f2be015e030 2024-09-13T08:37:10.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fcdabc00640 1 -- 172.21.15.44:0/4008436388 shutdown_connections 2024-09-13T08:37:10.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fcdabc00640 1 -- 172.21.15.44:0/4008436388 wait complete. 2024-09-13T08:37:10.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fee12800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fee04052a80 0x7fee040392f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fee12800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee040e2f10 0x7fee040d8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fee0aa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fee04052a80 0x7fee040392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fee0aa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fee04052a80 0x7fee040392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:41100/0 (socket says 172.21.15.44:41100) 2024-09-13T08:37:10.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fee0aa00640 1 -- 172.21.15.44:0/1804420975 learned_addr learned my addr 172.21.15.44:0/1804420975 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:10.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fee040edfb0 con 0x7fee04035f40 2024-09-13T08:37:10.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2fbbe00640 1 -- 172.21.15.44:0/1638492776 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2fb4012740 con 0x7f2fc4156cb0 2024-09-13T08:37:10.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fee040ede30 con 0x7fee040e2f10 2024-09-13T08:37:10.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fee040ee130 con 0x7fee04052a80 2024-09-13T08:37:10.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd6000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be00cce40 0x7f2be0155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd6000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be00cce40 0x7f2be0155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:49342/0 (socket says 172.21.15.44:49342) 2024-09-13T08:37:10.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd6000640 1 -- 172.21.15.44:0/1849549657 learned_addr learned my addr 172.21.15.44:0/1849549657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:10.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bdf400640 1 --2- 172.21.15.44:0/1849549657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be015e030 0x7f2be01fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd6000640 1 -- 172.21.15.44:0/1849549657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be015e030 msgr2=0x7f2be01fafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd6000640 1 --2- 172.21.15.44:0/1849549657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be015e030 0x7f2be01fafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd6000640 1 -- 172.21.15.44:0/1849549657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be006e740 msgr2=0x7f2be00cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd6000640 1 --2- 172.21.15.44:0/1849549657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be006e740 0x7f2be00cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd6000640 1 -- 172.21.15.44:0/1849549657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be01567d0 con 0x7f2be00cce40 2024-09-13T08:37:10.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bdf400640 1 --2- 172.21.15.44:0/1849549657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be015e030 0x7f2be01fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:37:10.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd6000640 1 --2- 172.21.15.44:0/1849549657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be00cce40 0x7f2be0155390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009790 tx=0x7f2bc80041a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:10.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7fee0be00640 1 --2- 172.21.15.44:0/1804420975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee040e2f10 0x7fee040d8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd7e00640 1 -- 172.21.15.44:0/1849549657 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80bc030 con 0x7f2be00cce40 2024-09-13T08:37:10.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd7e00640 1 -- 172.21.15.44:0/1849549657 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bc80036f0 con 0x7f2be00cce40 2024-09-13T08:37:10.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.012+0000 7f2bd7e00640 1 -- 172.21.15.44:0/1849549657 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8003b00 con 0x7f2be00cce40 2024-09-13T08:37:10.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7f2be5e00640 1 -- 172.21.15.44:0/1849549657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be00cb950 con 0x7f2be00cce40 2024-09-13T08:37:10.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7f2be5e00640 1 -- 172.21.15.44:0/1849549657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be0156d50 con 0x7f2be00cce40 2024-09-13T08:37:10.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7f2bd7e00640 1 -- 172.21.15.44:0/1849549657 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2bc80bf070 con 0x7f2be00cce40 2024-09-13T08:37:10.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7f2bd7e00640 1 -- 172.21.15.44:0/1849549657 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2bc80c1ab0 con 0x7f2be00cce40 2024-09-13T08:37:10.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7f2be5e00640 1 -- 172.21.15.44:0/1849549657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be015d8d0 con 0x7f2be00cce40 2024-09-13T08:37:10.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fc3c6e00640 1 -- 172.21.15.44:0/1758309813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3b8054e50 msgr2=0x7fc3b80deb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fc3c6e00640 1 --2- 172.21.15.44:0/1758309813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3b8054e50 0x7fc3b80deb30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3c006bbe0 tx=0x7fc3c0098450 comp rx=0 tx=0).stop 2024-09-13T08:37:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fee0aa00640 1 -- 172.21.15.44:0/1804420975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee040e2f10 msgr2=0x7fee040d8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fee0aa00640 1 --2- 172.21.15.44:0/1804420975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee040e2f10 0x7fee040d8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fee0aa00640 1 -- 172.21.15.44:0/1804420975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee04035f40 msgr2=0x7fee040d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fee0aa00640 1 --2- 172.21.15.44:0/1804420975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee04035f40 0x7fee040d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fee0aa00640 1 -- 172.21.15.44:0/1804420975 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee04039cb0 con 0x7fee04052a80 2024-09-13T08:37:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fee0aa00640 1 --2- 172.21.15.44:0/1804420975 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fee04052a80 0x7fee040392f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c500 tx=0x7fedf800c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fedff400640 1 -- 172.21.15.44:0/1804420975 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8017070 con 0x7fee04052a80 2024-09-13T08:37:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fedff400640 1 -- 172.21.15.44:0/1804420975 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedf80052d0 con 0x7fee04052a80 2024-09-13T08:37:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fedff400640 1 -- 172.21.15.44:0/1804420975 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8012890 con 0x7fee04052a80 2024-09-13T08:37:10.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fc3c6e00640 1 -- 172.21.15.44:0/1758309813 shutdown_connections 2024-09-13T08:37:10.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fc3c6e00640 1 --2- 172.21.15.44:0/1758309813 >> v2:172.21.15.44:3300/0 conn(0x7fc3b804ed90 0x7fc3b804f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0417fef0 con 0x7fee04052a80 2024-09-13T08:37:10.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fee041804b0 con 0x7fee04052a80 2024-09-13T08:37:10.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fc3c6e00640 1 --2- 172.21.15.44:0/1758309813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3b8054e50 0x7fc3b80deb30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.016+0000 7fc3c6e00640 1 -- 172.21.15.44:0/1758309813 >> 172.21.15.44:0/1758309813 conn(0x7fc3b801d950 msgr2=0x7fc3b8054740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.020+0000 7fc3c6e00640 1 -- 172.21.15.44:0/1758309813 shutdown_connections 2024-09-13T08:37:10.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.020+0000 7f2bd7e00640 1 -- 172.21.15.44:0/1849549657 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2bc80c6020 con 0x7f2be00cce40 2024-09-13T08:37:10.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.020+0000 7f2fbbe00640 1 -- 172.21.15.44:0/1638492776 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2fb4021070 con 0x7f2fc4156cb0 2024-09-13T08:37:10.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.020+0000 7fedff400640 1 -- 172.21.15.44:0/1804420975 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fedf801a070 con 0x7fee04052a80 2024-09-13T08:37:10.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.020+0000 7fedff400640 1 -- 172.21.15.44:0/1804420975 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fedf801c780 con 0x7fee04052a80 2024-09-13T08:37:10.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.020+0000 7fc3c6e00640 1 -- 172.21.15.44:0/1758309813 wait complete. 2024-09-13T08:37:10.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.020+0000 7fc3c6e00640 1 Processor -- start 2024-09-13T08:37:10.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.020+0000 7fc3c6e00640 1 -- start start 2024-09-13T08:37:10.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c6e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3b804ed90 0x7fc3b80e0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede80052a0 con 0x7fee04052a80 2024-09-13T08:37:10.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.028+0000 7fedff400640 1 -- 172.21.15.44:0/1804420975 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fedf8021020 con 0x7fee04052a80 2024-09-13T08:37:10.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c6e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b8054e50 0x7fc3b8189cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c8400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3b804ed90 0x7fc3b80e0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c8400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3b804ed90 0x7fc3b80e0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:41126/0 (socket says 172.21.15.44:41126) 2024-09-13T08:37:10.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c8400640 1 -- 172.21.15.44:0/2369805743 learned_addr learned my addr 172.21.15.44:0/2369805743 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:10.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c6e00640 1 --2- 172.21.15.44:0/2369805743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8055330 0x7fc3b80494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c5000640 1 --2- 172.21.15.44:0/2369805743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b8054e50 0x7fc3b8189cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8005d60 con 0x7fc3b8054e50 2024-09-13T08:37:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8005be0 con 0x7fc3b8055330 2024-09-13T08:37:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b8005ee0 con 0x7fc3b804ed90 2024-09-13T08:37:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c5a00640 1 --2- 172.21.15.44:0/2369805743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8055330 0x7fc3b80494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c8400640 1 -- 172.21.15.44:0/2369805743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8055330 msgr2=0x7fc3b80494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c8400640 1 --2- 172.21.15.44:0/2369805743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8055330 0x7fc3b80494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c8400640 1 -- 172.21.15.44:0/2369805743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b8054e50 msgr2=0x7fc3b8189cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c8400640 1 --2- 172.21.15.44:0/2369805743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b8054e50 0x7fc3b8189cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.024+0000 7fc3c8400640 1 -- 172.21.15.44:0/2369805743 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b81866e0 con 0x7fc3b804ed90 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.028+0000 7fc3c8400640 1 --2- 172.21.15.44:0/2369805743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3b804ed90 0x7fc3b80e0370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc3c006b560 tx=0x7fc3c01265d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.028+0000 7fc3bd600640 1 -- 172.21.15.44:0/2369805743 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c012c040 con 0x7fc3b804ed90 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.028+0000 7fc3bd600640 1 -- 172.21.15.44:0/2369805743 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3c01268d0 con 0x7fc3b804ed90 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.028+0000 7fc3bd600640 1 -- 172.21.15.44:0/2369805743 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0126ca0 con 0x7fc3b804ed90 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.028+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b81869c0 con 0x7fc3b804ed90 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.028+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b8189790 con 0x7fc3b804ed90 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.028+0000 7fc3bd600640 1 -- 172.21.15.44:0/2369805743 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3c012f070 con 0x7fc3b804ed90 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.028+0000 7fc3bd600640 1 -- 172.21.15.44:0/2369805743 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc3c0131930 con 0x7fc3b804ed90 2024-09-13T08:37:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.032+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b8186ee0 con 0x7fc3b804ed90 2024-09-13T08:37:10.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.032+0000 7fc3bd600640 1 -- 172.21.15.44:0/2369805743 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc3b8186ee0 con 0x7fc3b804ed90 2024-09-13T08:37:10.061 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi044') 2024-09-13T08:37:10.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.092+0000 7f412b400640 1 -- 172.21.15.44:0/2129341841 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f4118005590 con 0x7f4138036620 2024-09-13T08:37:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.096+0000 7f4151e00640 1 -- 172.21.15.44:0/845874357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f414c15e300 msgr2=0x7f414c0cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.096+0000 7f4151e00640 1 --2- 172.21.15.44:0/845874357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f414c15e300 0x7f414c0cf430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f414c06eeb0 tx=0x7f413801c890 comp rx=0 tx=0).stop 2024-09-13T08:37:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.096+0000 7f4151e00640 1 -- 172.21.15.44:0/845874357 shutdown_connections 2024-09-13T08:37:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.096+0000 7f4151e00640 1 --2- 172.21.15.44:0/845874357 >> v2:172.21.15.44:3300/0 conn(0x7f414c0cd5d0 0x7f414c073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.096+0000 7f4151e00640 1 --2- 172.21.15.44:0/845874357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f414c15e300 0x7f414c0cf430 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.096+0000 7f4151e00640 1 -- 172.21.15.44:0/845874357 >> 172.21.15.44:0/845874357 conn(0x7f414c0cfa00 msgr2=0x7f414c0cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.100+0000 7f4151e00640 1 -- 172.21.15.44:0/845874357 shutdown_connections 2024-09-13T08:37:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.100+0000 7f4151e00640 1 -- 172.21.15.44:0/845874357 wait complete. 2024-09-13T08:37:10.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151e00640 1 Processor -- start 2024-09-13T08:37:10.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151e00640 1 -- start start 2024-09-13T08:37:10.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f414c0cd5d0 0x7f414c1f8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414c0cd9f0 0x7f414c0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f414c15e300 0x7f414c154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f414c168160 con 0x7f414c0cd5d0 2024-09-13T08:37:10.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f414c167fe0 con 0x7f414c15e300 2024-09-13T08:37:10.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f414c1682e0 con 0x7f414c0cd9f0 2024-09-13T08:37:10.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f414c15e300 0x7f414c154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f414c15e300 0x7f414c154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:49364/0 (socket says 172.21.15.44:49364) 2024-09-13T08:37:10.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.104+0000 7f4151400640 1 -- 172.21.15.44:0/1812299566 learned_addr learned my addr 172.21.15.44:0/1812299566 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:10.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.108+0000 7f4150a00640 1 --2- 172.21.15.44:0/1812299566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f414c0cd5d0 0x7f414c1f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.108+0000 7f414be00640 1 --2- 172.21.15.44:0/1812299566 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414c0cd9f0 0x7f414c0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.108+0000 7f4150a00640 1 -- 172.21.15.44:0/1812299566 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414c0cd9f0 msgr2=0x7f414c0c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.108+0000 7f4150a00640 1 --2- 172.21.15.44:0/1812299566 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414c0cd9f0 0x7f414c0c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.108+0000 7f4150a00640 1 -- 172.21.15.44:0/1812299566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f414c15e300 msgr2=0x7f414c154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.108+0000 7f4150a00640 1 --2- 172.21.15.44:0/1812299566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f414c15e300 0x7f414c154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.108+0000 7f4150a00640 1 -- 172.21.15.44:0/1812299566 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c0ca0e0 con 0x7f414c0cd5d0 2024-09-13T08:37:10.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.108+0000 7f414be00640 1 --2- 172.21.15.44:0/1812299566 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414c0cd9f0 0x7f414c0c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:37:10.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.108+0000 7f4136000640 1 -- 172.21.15.44:0/2129341841 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7f4130021020 con 0x7f4138036620 2024-09-13T08:37:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f412b400640 1 -- 172.21.15.44:0/2129341841 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4138036620 msgr2=0x7f4138188c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f412b400640 1 --2- 172.21.15.44:0/2129341841 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4138036620 0x7f4138188c00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f413000c3e0 tx=0x7f413000c8a0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f4150a00640 1 --2- 172.21.15.44:0/1812299566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f414c0cd5d0 0x7f414c1f8850 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f41380ac000 tx=0x7f4138004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f412b400640 1 -- 172.21.15.44:0/2129341841 shutdown_connections 2024-09-13T08:37:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f412b400640 1 --2- 172.21.15.44:0/2129341841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41380e2ee0 0x7f4138049280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f412b400640 1 --2- 172.21.15.44:0/2129341841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4138052a50 0x7f4138048d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f412b400640 1 --2- 172.21.15.44:0/2129341841 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4138036620 0x7f4138188c00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f412b400640 1 -- 172.21.15.44:0/2129341841 >> 172.21.15.44:0/2129341841 conn(0x7f413801d780 msgr2=0x7f4138039b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f4148c00640 1 -- 172.21.15.44:0/1812299566 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41380bc030 con 0x7f414c0cd5d0 2024-09-13T08:37:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f4148c00640 1 -- 172.21.15.44:0/1812299566 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4138004340 con 0x7f414c0cd5d0 2024-09-13T08:37:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f4151e00640 1 -- 172.21.15.44:0/1812299566 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c0ca3c0 con 0x7f414c0cd5d0 2024-09-13T08:37:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f412b400640 1 -- 172.21.15.44:0/2129341841 shutdown_connections 2024-09-13T08:37:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f412b400640 1 -- 172.21.15.44:0/2129341841 wait complete. 2024-09-13T08:37:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f4148c00640 1 -- 172.21.15.44:0/1812299566 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41380b2070 con 0x7f414c0cd5d0 2024-09-13T08:37:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f4151e00640 1 -- 172.21.15.44:0/1812299566 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c0ca980 con 0x7f414c0cd5d0 2024-09-13T08:37:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.116+0000 7f4148c00640 1 -- 172.21.15.44:0/1812299566 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f41380bf070 con 0x7f414c0cd5d0 2024-09-13T08:37:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.120+0000 7f4148c00640 1 -- 172.21.15.44:0/1812299566 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4138003940 con 0x7f414c0cd5d0 2024-09-13T08:37:10.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.124+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fc3dc00dc30 con 0x7fc3dc0550a0 2024-09-13T08:37:10.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.124+0000 7f4142000640 1 -- 172.21.15.44:0/1812299566 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41200052a0 con 0x7f414c0cd5d0 2024-09-13T08:37:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.128+0000 7f34a7c00640 1 -- 172.21.15.44:0/731766550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f34a00008e0 con 0x7f34a016dd30 2024-09-13T08:37:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.128+0000 7f4148c00640 1 -- 172.21.15.44:0/1812299566 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f41380c7020 con 0x7f414c0cd5d0 2024-09-13T08:37:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.136+0000 7fc3d7e00640 1 -- 172.21.15.44:0/2243513478 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7fc3d8003600 con 0x7fc3dc0550a0 2024-09-13T08:37:10.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.148+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3dc0550a0 msgr2=0x7fc3dc0f2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.148+0000 7fc3eae00640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3dc0550a0 0x7fc3dc0f2eb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800c3f0 tx=0x7fc3d800c8b0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.148+0000 7f349cc00640 1 -- 172.21.15.44:0/731766550 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v13)=0 v13) ==== 120+0+0 (secure 0 0 0) 0x7f348c003600 con 0x7f34a016dd30 2024-09-13T08:37:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.148+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 shutdown_connections 2024-09-13T08:37:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.148+0000 7fc3eae00640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3dc0550a0 0x7fc3dc0f2eb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.148+0000 7fc3eae00640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3dc036660 0x7fc3dc0f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.148+0000 7fc3eae00640 1 --2- 172.21.15.44:0/2243513478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3dc036280 0x7fc3dc048040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.148+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 >> 172.21.15.44:0/2243513478 conn(0x7fc3dc01d7b0 msgr2=0x7fc3dc04fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.148+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 shutdown_connections 2024-09-13T08:37:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.148+0000 7fc3eae00640 1 -- 172.21.15.44:0/2243513478 wait complete. 2024-09-13T08:37:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.152+0000 7f34a7c00640 1 -- 172.21.15.44:0/731766550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a016dd30 msgr2=0x7f34a020cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.152+0000 7f34a7c00640 1 --2- 172.21.15.44:0/731766550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a016dd30 0x7f34a020cc90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f348c01b010 tx=0x7f348c005770 comp rx=0 tx=0).stop 2024-09-13T08:37:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.152+0000 7f34a7c00640 1 -- 172.21.15.44:0/731766550 shutdown_connections 2024-09-13T08:37:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.152+0000 7f34a7c00640 1 --2- 172.21.15.44:0/731766550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a016e110 0x7f34a00c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.152+0000 7f34a7c00640 1 --2- 172.21.15.44:0/731766550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a016dd30 0x7f34a020cc90 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f348c01b010 tx=0x7f348c005770 comp rx=0 tx=0).stop 2024-09-13T08:37:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.152+0000 7f34a7c00640 1 --2- 172.21.15.44:0/731766550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a0073090 0x7f34a00beb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.152+0000 7f34a7c00640 1 -- 172.21.15.44:0/731766550 >> 172.21.15.44:0/731766550 conn(0x7f34a00b1600 msgr2=0x7f34a00b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.152+0000 7f34a7c00640 1 -- 172.21.15.44:0/731766550 shutdown_connections 2024-09-13T08:37:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.152+0000 7f34a7c00640 1 -- 172.21.15.44:0/731766550 wait complete. 2024-09-13T08:37:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.160+0000 7f1cbaa00640 1 -- 172.21.15.44:0/2926867416 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f1cc815b450 con 0x7f1cc80cac60 2024-09-13T08:37:10.161 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi044') 2024-09-13T08:37:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.164+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fc3b8189990 con 0x7fc3b804ed90 2024-09-13T08:37:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.172+0000 7f1cc5600640 1 -- 172.21.15.44:0/2926867416 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v14)=0 v14) ==== 119+0+0 (secure 0 0 0) 0x7f1cbc026070 con 0x7f1cc80cac60 2024-09-13T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.176+0000 7f1cbaa00640 1 -- 172.21.15.44:0/2926867416 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cc80cac60 msgr2=0x7f1cc815f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.176+0000 7f1cbaa00640 1 --2- 172.21.15.44:0/2926867416 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cc80cac60 0x7f1cc815f020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00e970 tx=0x7f1cbc00ee30 comp rx=0 tx=0).stop 2024-09-13T08:37:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.176+0000 7f1cbaa00640 1 -- 172.21.15.44:0/2926867416 shutdown_connections 2024-09-13T08:37:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.176+0000 7f1cbaa00640 1 --2- 172.21.15.44:0/2926867416 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cc80cac60 0x7f1cc815f020 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.176+0000 7f1cbaa00640 1 --2- 172.21.15.44:0/2926867416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc80aecb0 0x7f1cc815eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.176+0000 7f1cbaa00640 1 --2- 172.21.15.44:0/2926867416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc8072610 0x7f1cc82022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.176+0000 7f1cbaa00640 1 -- 172.21.15.44:0/2926867416 >> 172.21.15.44:0/2926867416 conn(0x7f1cc80cfa00 msgr2=0x7f1cc80b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.176+0000 7f1cbaa00640 1 -- 172.21.15.44:0/2926867416 shutdown_connections 2024-09-13T08:37:10.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.176+0000 7f1cbaa00640 1 -- 172.21.15.44:0/2926867416 wait complete. 2024-09-13T08:37:10.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.176+0000 7f2be5e00640 1 -- 172.21.15.44:0/1849549657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f2be0070c10 con 0x7f2be00cce40 2024-09-13T08:37:10.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.180+0000 7fc3bd600640 1 -- 172.21.15.44:0/2369805743 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7fc3b8189990 con 0x7fc3b804ed90 2024-09-13T08:37:10.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.184+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3b804ed90 msgr2=0x7fc3b80e0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.184+0000 7fc3c6e00640 1 --2- 172.21.15.44:0/2369805743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3b804ed90 0x7fc3b80e0370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc3c006b560 tx=0x7fc3c01265d0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.184+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 shutdown_connections 2024-09-13T08:37:10.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.184+0000 7fc3c6e00640 1 --2- 172.21.15.44:0/2369805743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8055330 0x7fc3b80494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.184+0000 7fc3c6e00640 1 --2- 172.21.15.44:0/2369805743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b8054e50 0x7fc3b8189cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.184+0000 7fc3c6e00640 1 --2- 172.21.15.44:0/2369805743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3b804ed90 0x7fc3b80e0370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.184+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 >> 172.21.15.44:0/2369805743 conn(0x7fc3b801d950 msgr2=0x7fc3b800ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.184+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 shutdown_connections 2024-09-13T08:37:10.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.184+0000 7fc3c6e00640 1 -- 172.21.15.44:0/2369805743 wait complete. 2024-09-13T08:37:10.191 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi044') 2024-09-13T08:37:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.188+0000 7f2bd7e00640 1 -- 172.21.15.44:0/1849549657 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f2bc80163f0 con 0x7f2be00cce40 2024-09-13T08:37:10.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.188+0000 7f2be5e00640 1 -- 172.21.15.44:0/1849549657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be00cce40 msgr2=0x7f2be0155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.188+0000 7f2be5e00640 1 --2- 172.21.15.44:0/1849549657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be00cce40 0x7f2be0155390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009790 tx=0x7f2bc80041a0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.192+0000 7f2be5e00640 1 -- 172.21.15.44:0/1849549657 shutdown_connections 2024-09-13T08:37:10.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.192+0000 7f2be5e00640 1 --2- 172.21.15.44:0/1849549657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be015e030 0x7f2be01fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.192+0000 7f2be5e00640 1 --2- 172.21.15.44:0/1849549657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be00cce40 0x7f2be0155390 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.192+0000 7f2be5e00640 1 --2- 172.21.15.44:0/1849549657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be006e740 0x7f2be00cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.192+0000 7f2be5e00640 1 -- 172.21.15.44:0/1849549657 >> 172.21.15.44:0/1849549657 conn(0x7f2be00d1be0 msgr2=0x7f2be0150c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.192+0000 7f2be5e00640 1 -- 172.21.15.44:0/1849549657 shutdown_connections 2024-09-13T08:37:10.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.192+0000 7f2be5e00640 1 -- 172.21.15.44:0/1849549657 wait complete. 2024-09-13T08:37:10.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.192+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fede8005590 con 0x7fee04052a80 2024-09-13T08:37:10.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.200+0000 7fedff400640 1 -- 172.21.15.44:0/1804420975 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v17)=0 v17) ==== 125+0+0 (secure 0 0 0) 0x7fedf8003e00 con 0x7fee04052a80 2024-09-13T08:37:10.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.200+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fee04052a80 msgr2=0x7fee040392f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.200+0000 7fee12800640 1 --2- 172.21.15.44:0/1804420975 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fee04052a80 0x7fee040392f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c500 tx=0x7fedf800c9c0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.204+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 shutdown_connections 2024-09-13T08:37:10.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.204+0000 7fee12800640 1 --2- 172.21.15.44:0/1804420975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee040e2f10 0x7fee040d8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.204+0000 7fee12800640 1 --2- 172.21.15.44:0/1804420975 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fee04052a80 0x7fee040392f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.204+0000 7fee12800640 1 --2- 172.21.15.44:0/1804420975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee04035f40 0x7fee040d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.204+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 >> 172.21.15.44:0/1804420975 conn(0x7fee0401d7b0 msgr2=0x7fee040d9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.204+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 shutdown_connections 2024-09-13T08:37:10.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.204+0000 7fee12800640 1 -- 172.21.15.44:0/1804420975 wait complete. 2024-09-13T08:37:10.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.204+0000 7f2fcaa00640 1 -- 172.21.15.44:0/1638492776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f2fc4165070 con 0x7f2fc4156cb0 2024-09-13T08:37:10.212 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi044') 2024-09-13T08:37:10.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.208+0000 7f2fbbe00640 1 -- 172.21.15.44:0/1638492776 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v18)=0 v18) ==== 120+0+0 (secure 0 0 0) 0x7f2fb4003bf0 con 0x7f2fc4156cb0 2024-09-13T08:37:10.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.212+0000 7f2fcaa00640 1 -- 172.21.15.44:0/1638492776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fc4156cb0 msgr2=0x7f2fc4158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.212+0000 7f2fcaa00640 1 --2- 172.21.15.44:0/1638492776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fc4156cb0 0x7f2fc4158d60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c430 tx=0x7f2fb400c8f0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.212+0000 7f2fcaa00640 1 -- 172.21.15.44:0/1638492776 shutdown_connections 2024-09-13T08:37:10.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.212+0000 7f2fcaa00640 1 --2- 172.21.15.44:0/1638492776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fc4156cb0 0x7f2fc4158d60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.212+0000 7f2fcaa00640 1 --2- 172.21.15.44:0/1638492776 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fc4159b10 0x7f2fc4156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.212+0000 7f2fcaa00640 1 --2- 172.21.15.44:0/1638492776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc40cc440 0x7f2fc4206580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.212+0000 7f2fcaa00640 1 -- 172.21.15.44:0/1638492776 >> 172.21.15.44:0/1638492776 conn(0x7f2fc40b1600 msgr2=0x7f2fc40b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.212+0000 7f2fcaa00640 1 -- 172.21.15.44:0/1638492776 shutdown_connections 2024-09-13T08:37:10.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.212+0000 7f2fcaa00640 1 -- 172.21.15.44:0/1638492776 wait complete. 2024-09-13T08:37:10.220 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-09-13T08:37:10.220 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-09-13T08:37:10.238 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi044') 2024-09-13T08:37:10.238 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi044') 2024-09-13T08:37:10.238 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi044') 2024-09-13T08:37:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.236+0000 7f4142000640 1 -- 172.21.15.44:0/1812299566 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f4120005860 con 0x7f414c0cd5d0 2024-09-13T08:37:10.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.236+0000 7f4148c00640 1 -- 172.21.15.44:0/1812299566 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f41380164b0 con 0x7f414c0cd5d0 2024-09-13T08:37:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.240+0000 7f4142000640 1 -- 172.21.15.44:0/1812299566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f414c0cd5d0 msgr2=0x7f414c1f8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.240+0000 7f4142000640 1 --2- 172.21.15.44:0/1812299566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f414c0cd5d0 0x7f414c1f8850 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f41380ac000 tx=0x7f4138004040 comp rx=0 tx=0).stop 2024-09-13T08:37:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.240+0000 7f4142000640 1 -- 172.21.15.44:0/1812299566 shutdown_connections 2024-09-13T08:37:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.240+0000 7f4142000640 1 --2- 172.21.15.44:0/1812299566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f414c15e300 0x7f414c154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.240+0000 7f4142000640 1 --2- 172.21.15.44:0/1812299566 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414c0cd9f0 0x7f414c0c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.240+0000 7f4142000640 1 --2- 172.21.15.44:0/1812299566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f414c0cd5d0 0x7f414c1f8850 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.240+0000 7f4142000640 1 -- 172.21.15.44:0/1812299566 >> 172.21.15.44:0/1812299566 conn(0x7f414c0cfa00 msgr2=0x7f414c0afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.240+0000 7f4142000640 1 -- 172.21.15.44:0/1812299566 shutdown_connections 2024-09-13T08:37:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.240+0000 7f4142000640 1 -- 172.21.15.44:0/1812299566 wait complete. 2024-09-13T08:37:10.260 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T08:37:10.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T08:37:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.444+0000 7f3523400640 1 -- 172.21.15.44:0/66755293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c0d00a0 msgr2=0x7f351c0d0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.444+0000 7f3523400640 1 --2- 172.21.15.44:0/66755293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c0d00a0 0x7f351c0d0530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f351c073540 tx=0x7f351401cd80 comp rx=0 tx=0).stop 2024-09-13T08:37:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.444+0000 7f3523400640 1 -- 172.21.15.44:0/66755293 shutdown_connections 2024-09-13T08:37:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.444+0000 7f3523400640 1 --2- 172.21.15.44:0/66755293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c0d00a0 0x7f351c0d0530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.444+0000 7f3523400640 1 --2- 172.21.15.44:0/66755293 >> v2:172.21.15.129:3300/0 conn(0x7f351c151960 0x7f351c151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.444+0000 7f3523400640 1 -- 172.21.15.44:0/66755293 >> 172.21.15.44:0/66755293 conn(0x7f351c0d0da0 msgr2=0x7f351c0d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.444+0000 7f3523400640 1 -- 172.21.15.44:0/66755293 shutdown_connections 2024-09-13T08:37:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.444+0000 7f3523400640 1 -- 172.21.15.44:0/66755293 wait complete. 2024-09-13T08:37:10.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.444+0000 7f3523400640 1 Processor -- start 2024-09-13T08:37:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.444+0000 7f3523400640 1 -- start start 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3523400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f351c070720 0x7f351c0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3523400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c0d00a0 0x7f351c0c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3523400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f351c151960 0x7f351c1f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3523400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f351c0cc510 con 0x7f351c0d00a0 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3523400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f351c15c1e0 con 0x7f351c070720 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3523400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f351c0cc690 con 0x7f351c151960 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f351be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c0d00a0 0x7f351c0c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3520a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f351c070720 0x7f351c0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f351be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c0d00a0 0x7f351c0c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45330/0 (socket says 172.21.15.44:45330) 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f351be00640 1 -- 172.21.15.44:0/352784761 learned_addr learned my addr 172.21.15.44:0/352784761 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3521400640 1 --2- 172.21.15.44:0/352784761 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f351c151960 0x7f351c1f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f351be00640 1 -- 172.21.15.44:0/352784761 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f351c151960 msgr2=0x7f351c1f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f351be00640 1 --2- 172.21.15.44:0/352784761 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f351c151960 0x7f351c1f8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f351be00640 1 -- 172.21.15.44:0/352784761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f351c070720 msgr2=0x7f351c0b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f351be00640 1 --2- 172.21.15.44:0/352784761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f351c070720 0x7f351c0b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f351be00640 1 -- 172.21.15.44:0/352784761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c1f55b0 con 0x7f351c0d00a0 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f351be00640 1 --2- 172.21.15.44:0/352784761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c0d00a0 0x7f351c0c7e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f350800c3c0 tx=0x7f350800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3518c00640 1 -- 172.21.15.44:0/352784761 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508017070 con 0x7f351c0d00a0 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3518c00640 1 -- 172.21.15.44:0/352784761 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35080053c0 con 0x7f351c0d00a0 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3523400640 1 -- 172.21.15.44:0/352784761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c1f5890 con 0x7f351c0d00a0 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3518c00640 1 -- 172.21.15.44:0/352784761 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508003860 con 0x7f351c0d00a0 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3523400640 1 -- 172.21.15.44:0/352784761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c1f52c0 con 0x7f351c0d00a0 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3518c00640 1 -- 172.21.15.44:0/352784761 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f350801a070 con 0x7f351c0d00a0 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3518c00640 1 -- 172.21.15.44:0/352784761 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f350801c780 con 0x7f351c0d00a0 2024-09-13T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.448+0000 7f3523400640 1 -- 172.21.15.44:0/352784761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351c0d0530 con 0x7f351c0d00a0 2024-09-13T08:37:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.452+0000 7f3518c00640 1 -- 172.21.15.44:0/352784761 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3508021020 con 0x7f351c0d00a0 2024-09-13T08:37:10.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.516+0000 7f3523400640 1 -- 172.21.15.44:0/352784761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f351c0b6d20 con 0x7f351c0d00a0 2024-09-13T08:37:10.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.520+0000 7f3518c00640 1 -- 172.21.15.44:0/352784761 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f351c074340 con 0x7f351c0d00a0 2024-09-13T08:37:10.523 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2024-09-13T08:37:10.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.520+0000 7f3523400640 1 -- 172.21.15.44:0/352784761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c0d00a0 msgr2=0x7f351c0c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:10.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.520+0000 7f3523400640 1 --2- 172.21.15.44:0/352784761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c0d00a0 0x7f351c0c7e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f350800c3c0 tx=0x7f350800c880 comp rx=0 tx=0).stop 2024-09-13T08:37:10.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.524+0000 7f3523400640 1 -- 172.21.15.44:0/352784761 shutdown_connections 2024-09-13T08:37:10.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.524+0000 7f3523400640 1 --2- 172.21.15.44:0/352784761 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f351c151960 0x7f351c1f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.524+0000 7f3523400640 1 --2- 172.21.15.44:0/352784761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c0d00a0 0x7f351c0c7e70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.524+0000 7f3523400640 1 --2- 172.21.15.44:0/352784761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f351c070720 0x7f351c0b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:10.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.524+0000 7f3523400640 1 -- 172.21.15.44:0/352784761 >> 172.21.15.44:0/352784761 conn(0x7f351c0d0da0 msgr2=0x7f351c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:10.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.524+0000 7f3523400640 1 -- 172.21.15.44:0/352784761 shutdown_connections 2024-09-13T08:37:10.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:10.524+0000 7f3523400640 1 -- 172.21.15.44:0/352784761 wait complete. 2024-09-13T08:37:10.544 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T08:37:10.545 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T08:37:10.545 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:10.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T08:37:10.566 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:10.566 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T08:37:10.628 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:10.628 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T08:37:10.688 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:10.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T08:37:10.748 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:10.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T08:37:10.769 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:10.769 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T08:37:10.832 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:10.832 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T08:37:10.892 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:10.892 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T08:37:10.952 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 552c332f-dfc0-4d0c-a685-8d43312f4cb0 0 2024-09-13T08:37:11.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 -- 172.21.15.129:0/2684101212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff900d29b0 msgr2=0x7eff900b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 --2- 172.21.15.129:0/2684101212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff900d29b0 0x7eff900b1b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7eff78009840 tx=0x7eff7801f010 comp rx=0 tx=0).stop 2024-09-13T08:37:11.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 -- 172.21.15.129:0/2684101212 shutdown_connections 2024-09-13T08:37:11.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 --2- 172.21.15.129:0/2684101212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff900d29b0 0x7eff900b1b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 --2- 172.21.15.129:0/2684101212 >> v2:172.21.15.129:3300/0 conn(0x7eff900d00f0 0x7eff900b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 -- 172.21.15.129:0/2684101212 >> 172.21.15.129:0/2684101212 conn(0x7eff900d1e90 msgr2=0x7eff900d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:11.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 -- 172.21.15.129:0/2684101212 shutdown_connections 2024-09-13T08:37:11.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 -- 172.21.15.129:0/2684101212 wait complete. 2024-09-13T08:37:11.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 Processor -- start 2024-09-13T08:37:11.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 -- start start 2024-09-13T08:37:11.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff900d00f0 0x7eff900b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff900d04d0 0x7eff900c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff900d29b0 0x7eff90158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff900b0ed0 con 0x7eff900d00f0 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7eff900b0d50 con 0x7eff900d29b0 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff95600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7eff900b04f0 con 0x7eff900d04d0 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff87e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff900d04d0 0x7eff900c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff8f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff900d29b0 0x7eff90158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff8ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff900d00f0 0x7eff900b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff87e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff900d04d0 0x7eff900c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.129:42762/0 (socket says 172.21.15.129:42762) 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff87e00640 1 -- 172.21.15.129:0/2702056740 learned_addr learned my addr 172.21.15.129:0/2702056740 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff87e00640 1 -- 172.21.15.129:0/2702056740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff900d29b0 msgr2=0x7eff90158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff87e00640 1 --2- 172.21.15.129:0/2702056740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff900d29b0 0x7eff90158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff87e00640 1 -- 172.21.15.129:0/2702056740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff900d00f0 msgr2=0x7eff900b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff87e00640 1 --2- 172.21.15.129:0/2702056740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff900d00f0 0x7eff900b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff87e00640 1 -- 172.21.15.129:0/2702056740 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9015e460 con 0x7eff900d04d0 2024-09-13T08:37:11.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.135+0000 7eff8ea00640 1 --2- 172.21.15.129:0/2702056740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff900d00f0 0x7eff900b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:11.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.139+0000 7eff87e00640 1 --2- 172.21.15.129:0/2702056740 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff900d04d0 0x7eff900c72e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7eff7801d6c0 tx=0x7eff78005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:11.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.139+0000 7eff86a00640 1 -- 172.21.15.129:0/2702056740 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff780be070 con 0x7eff900d04d0 2024-09-13T08:37:11.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.139+0000 7eff86a00640 1 -- 172.21.15.129:0/2702056740 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff780052f0 con 0x7eff900d04d0 2024-09-13T08:37:11.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.139+0000 7eff95600640 1 -- 172.21.15.129:0/2702056740 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff9015e740 con 0x7eff900d04d0 2024-09-13T08:37:11.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.139+0000 7eff86a00640 1 -- 172.21.15.129:0/2702056740 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff780b5040 con 0x7eff900d04d0 2024-09-13T08:37:11.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.139+0000 7eff95600640 1 -- 172.21.15.129:0/2702056740 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7eff9015ed00 con 0x7eff900d04d0 2024-09-13T08:37:11.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.139+0000 7eff86a00640 1 -- 172.21.15.129:0/2702056740 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7eff780c1070 con 0x7eff900d04d0 2024-09-13T08:37:11.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.139+0000 7eff86a00640 1 -- 172.21.15.129:0/2702056740 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7eff78003940 con 0x7eff900d04d0 2024-09-13T08:37:11.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.139+0000 7eff95600640 1 -- 172.21.15.129:0/2702056740 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff900b15e0 con 0x7eff900d04d0 2024-09-13T08:37:11.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.143+0000 7eff86a00640 1 -- 172.21.15.129:0/2702056740 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7eff78018b70 con 0x7eff900d04d0 2024-09-13T08:37:11.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.211+0000 7eff95600640 1 -- 172.21.15.129:0/2702056740 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "osd new", "uuid": "552c332f-dfc0-4d0c-a685-8d43312f4cb0", "id": 0} v 0) -- 0x7eff900b4870 con 0x7eff900d04d0 2024-09-13T08:37:11.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.215+0000 7eff86a00640 1 -- 172.21.15.129:0/2702056740 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "552c332f-dfc0-4d0c-a685-8d43312f4cb0", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7eff78003b40 con 0x7eff900d04d0 2024-09-13T08:37:11.214 INFO:teuthology.orchestra.run.smithi129.stdout:0 2024-09-13T08:37:11.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.215+0000 7eff95600640 1 -- 172.21.15.129:0/2702056740 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff900d04d0 msgr2=0x7eff900c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.215+0000 7eff95600640 1 --2- 172.21.15.129:0/2702056740 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff900d04d0 0x7eff900c72e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7eff7801d6c0 tx=0x7eff78005db0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.215+0000 7eff95600640 1 -- 172.21.15.129:0/2702056740 shutdown_connections 2024-09-13T08:37:11.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.215+0000 7eff95600640 1 --2- 172.21.15.129:0/2702056740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff900d29b0 0x7eff90158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.215+0000 7eff95600640 1 --2- 172.21.15.129:0/2702056740 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff900d04d0 0x7eff900c72e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.215+0000 7eff95600640 1 --2- 172.21.15.129:0/2702056740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff900d00f0 0x7eff900b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.215+0000 7eff95600640 1 -- 172.21.15.129:0/2702056740 >> 172.21.15.129:0/2702056740 conn(0x7eff900d1e90 msgr2=0x7eff901e84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:11.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.215+0000 7eff95600640 1 -- 172.21.15.129:0/2702056740 shutdown_connections 2024-09-13T08:37:11.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.215+0000 7eff95600640 1 -- 172.21.15.129:0/2702056740 wait complete. 2024-09-13T08:37:11.234 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new df0f39fa-83f5-4276-a3ab-ab685aaf8c55 1 2024-09-13T08:37:11.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.375+0000 7f247b400640 1 -- 172.21.15.129:0/1537144839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2474070720 msgr2=0x7f2474073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.375+0000 7f247b400640 1 --2- 172.21.15.129:0/1537144839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2474070720 0x7f2474073c30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2474070470 tx=0x7f246001c800 comp rx=0 tx=0).stop 2024-09-13T08:37:11.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 -- 172.21.15.129:0/1537144839 shutdown_connections 2024-09-13T08:37:11.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 --2- 172.21.15.129:0/1537144839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2474070720 0x7f2474073c30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 --2- 172.21.15.129:0/1537144839 >> v2:172.21.15.44:3300/0 conn(0x7f2474151960 0x7f2474151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 -- 172.21.15.129:0/1537144839 >> 172.21.15.129:0/1537144839 conn(0x7f24740d0da0 msgr2=0x7f24740d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:11.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 -- 172.21.15.129:0/1537144839 shutdown_connections 2024-09-13T08:37:11.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 -- 172.21.15.129:0/1537144839 wait complete. 2024-09-13T08:37:11.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 Processor -- start 2024-09-13T08:37:11.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 -- start start 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2474070720 0x7f24740c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24740d00a0 0x7f24740cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2474151960 0x7f24740b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24740ccd20 con 0x7f24740d00a0 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2474074380 con 0x7f2474070720 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f247b400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f24740ccea0 con 0x7f2474151960 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2478a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2474070720 0x7f24740c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2478a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2474070720 0x7f24740c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59220/0 (socket says 172.21.15.129:59220) 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2478a00640 1 -- 172.21.15.129:0/1716688872 learned_addr learned my addr 172.21.15.129:0/1716688872 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2479400640 1 --2- 172.21.15.129:0/1716688872 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2474151960 0x7f24740b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2473e00640 1 --2- 172.21.15.129:0/1716688872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24740d00a0 0x7f24740cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2478a00640 1 -- 172.21.15.129:0/1716688872 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2474151960 msgr2=0x7f24740b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2478a00640 1 --2- 172.21.15.129:0/1716688872 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2474151960 0x7f24740b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2478a00640 1 -- 172.21.15.129:0/1716688872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24740d00a0 msgr2=0x7f24740cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2478a00640 1 --2- 172.21.15.129:0/1716688872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24740d00a0 0x7f24740cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2478a00640 1 -- 172.21.15.129:0/1716688872 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247406a190 con 0x7f2474070720 2024-09-13T08:37:11.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.379+0000 7f2478a00640 1 --2- 172.21.15.129:0/1716688872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2474070720 0x7f24740c8300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f246800ea20 tx=0x7f246800eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.383+0000 7f2470c00640 1 -- 172.21.15.129:0/1716688872 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246800cd80 con 0x7f2474070720 2024-09-13T08:37:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.383+0000 7f2470c00640 1 -- 172.21.15.129:0/1716688872 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2468010430 con 0x7f2474070720 2024-09-13T08:37:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.383+0000 7f2470c00640 1 -- 172.21.15.129:0/1716688872 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468010860 con 0x7f2474070720 2024-09-13T08:37:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.383+0000 7f247b400640 1 -- 172.21.15.129:0/1716688872 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247415e250 con 0x7f2474070720 2024-09-13T08:37:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.383+0000 7f247b400640 1 -- 172.21.15.129:0/1716688872 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24741554c0 con 0x7f2474070720 2024-09-13T08:37:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.383+0000 7f2470c00640 1 -- 172.21.15.129:0/1716688872 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2468010a00 con 0x7f2474070720 2024-09-13T08:37:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.383+0000 7f2470c00640 1 -- 172.21.15.129:0/1716688872 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f24680217a0 con 0x7f2474070720 2024-09-13T08:37:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.383+0000 7f247b400640 1 -- 172.21.15.129:0/1716688872 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2474073c30 con 0x7f2474070720 2024-09-13T08:37:11.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.383+0000 7f2470c00640 1 -- 172.21.15.129:0/1716688872 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f24680219f0 con 0x7f2474070720 2024-09-13T08:37:11.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.455+0000 7f247b400640 1 -- 172.21.15.129:0/1716688872 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd new", "uuid": "df0f39fa-83f5-4276-a3ab-ab685aaf8c55", "id": 1} v 0) -- 0x7f24740b6d20 con 0x7f2474070720 2024-09-13T08:37:11.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.459+0000 7f2470c00640 1 -- 172.21.15.129:0/1716688872 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df0f39fa-83f5-4276-a3ab-ab685aaf8c55", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f2468021bd0 con 0x7f2474070720 2024-09-13T08:37:11.459 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-09-13T08:37:11.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.459+0000 7f247b400640 1 -- 172.21.15.129:0/1716688872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2474070720 msgr2=0x7f24740c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.459+0000 7f247b400640 1 --2- 172.21.15.129:0/1716688872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2474070720 0x7f24740c8300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f246800ea20 tx=0x7f246800eee0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.459+0000 7f247b400640 1 -- 172.21.15.129:0/1716688872 shutdown_connections 2024-09-13T08:37:11.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.459+0000 7f247b400640 1 --2- 172.21.15.129:0/1716688872 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2474151960 0x7f24740b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.459+0000 7f247b400640 1 --2- 172.21.15.129:0/1716688872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24740d00a0 0x7f24740cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.459+0000 7f247b400640 1 --2- 172.21.15.129:0/1716688872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2474070720 0x7f24740c8300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.459+0000 7f247b400640 1 -- 172.21.15.129:0/1716688872 >> 172.21.15.129:0/1716688872 conn(0x7f24740d0da0 msgr2=0x7f24740b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:11.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.459+0000 7f247b400640 1 -- 172.21.15.129:0/1716688872 shutdown_connections 2024-09-13T08:37:11.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.459+0000 7f247b400640 1 -- 172.21.15.129:0/1716688872 wait complete. 2024-09-13T08:37:11.478 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new bcb344c5-4c69-4f64-8990-956e8d477ab3 2 2024-09-13T08:37:11.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 -- 172.21.15.129:0/2612038182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d40cee80 msgr2=0x7f88d406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 --2- 172.21.15.129:0/2612038182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d40cee80 0x7f88d406d160 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f88d415eeb0 tx=0x7f88c001c870 comp rx=0 tx=0).stop 2024-09-13T08:37:11.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 -- 172.21.15.129:0/2612038182 shutdown_connections 2024-09-13T08:37:11.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 --2- 172.21.15.129:0/2612038182 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d40cb5b0 0x7f88d40cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 --2- 172.21.15.129:0/2612038182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d40cee80 0x7f88d406d160 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 --2- 172.21.15.129:0/2612038182 >> v2:172.21.15.44:3300/0 conn(0x7f88d415df20 0x7f88d406cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 -- 172.21.15.129:0/2612038182 >> 172.21.15.129:0/2612038182 conn(0x7f88d40d0da0 msgr2=0x7f88d40d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:11.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 -- 172.21.15.129:0/2612038182 shutdown_connections 2024-09-13T08:37:11.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 -- 172.21.15.129:0/2612038182 wait complete. 2024-09-13T08:37:11.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 Processor -- start 2024-09-13T08:37:11.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.619+0000 7f88d9600640 1 -- start start 2024-09-13T08:37:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d9600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d40cb5b0 0x7f88d4152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40cee80 0x7f88d4158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d9600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d415df20 0x7f88d40b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d9600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f88d4161bf0 con 0x7f88d40cee80 2024-09-13T08:37:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d9600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88d4074690 con 0x7f88d40cb5b0 2024-09-13T08:37:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d9600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f88d4161d70 con 0x7f88d415df20 2024-09-13T08:37:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d40cb5b0 0x7f88d4152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d40cb5b0 0x7f88d4152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59252/0 (socket says 172.21.15.129:59252) 2024-09-13T08:37:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d2a00640 1 -- 172.21.15.129:0/1207330145 learned_addr learned my addr 172.21.15.129:0/1207330145 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-13T08:37:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d3400640 1 --2- 172.21.15.129:0/1207330145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d415df20 0x7f88d40b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d2000640 1 --2- 172.21.15.129:0/1207330145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40cee80 0x7f88d4158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d2a00640 1 -- 172.21.15.129:0/1207330145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d415df20 msgr2=0x7f88d40b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d2a00640 1 --2- 172.21.15.129:0/1207330145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d415df20 0x7f88d40b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d2a00640 1 -- 172.21.15.129:0/1207330145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40cee80 msgr2=0x7f88d4158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d2a00640 1 --2- 172.21.15.129:0/1207330145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40cee80 0x7f88d4158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d2a00640 1 -- 172.21.15.129:0/1207330145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d4157f60 con 0x7f88d40cb5b0 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d3400640 1 --2- 172.21.15.129:0/1207330145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d415df20 0x7f88d40b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d2a00640 1 --2- 172.21.15.129:0/1207330145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d40cb5b0 0x7f88d4152720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f88c400d950 tx=0x7f88c400dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88caa00640 1 -- 172.21.15.129:0/1207330145 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4017070 con 0x7f88d40cb5b0 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88caa00640 1 -- 172.21.15.129:0/1207330145 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88c4009b10 con 0x7f88d40cb5b0 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d9600640 1 -- 172.21.15.129:0/1207330145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d4158240 con 0x7f88d40cb5b0 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88caa00640 1 -- 172.21.15.129:0/1207330145 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4010460 con 0x7f88d40cb5b0 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d9600640 1 -- 172.21.15.129:0/1207330145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d4152c60 con 0x7f88d40cb5b0 2024-09-13T08:37:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88caa00640 1 -- 172.21.15.129:0/1207330145 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f88c401a070 con 0x7f88d40cb5b0 2024-09-13T08:37:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88caa00640 1 -- 172.21.15.129:0/1207330145 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f88c4010a20 con 0x7f88d40cb5b0 2024-09-13T08:37:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.623+0000 7f88d9600640 1 -- 172.21.15.129:0/1207330145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d40cb9c0 con 0x7f88d40cb5b0 2024-09-13T08:37:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.627+0000 7f88caa00640 1 -- 172.21.15.129:0/1207330145 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f88c4010d00 con 0x7f88d40cb5b0 2024-09-13T08:37:11.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.695+0000 7f88d9600640 1 -- 172.21.15.129:0/1207330145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bcb344c5-4c69-4f64-8990-956e8d477ab3", "id": 2} v 0) -- 0x7f88d40c7c10 con 0x7f88d40cb5b0 2024-09-13T08:37:11.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.699+0000 7f88caa00640 1 -- 172.21.15.129:0/1207330145 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bcb344c5-4c69-4f64-8990-956e8d477ab3", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f88c4010600 con 0x7f88d40cb5b0 2024-09-13T08:37:11.700 INFO:teuthology.orchestra.run.smithi129.stdout:2 2024-09-13T08:37:11.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.703+0000 7f88d9600640 1 -- 172.21.15.129:0/1207330145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d40cb5b0 msgr2=0x7f88d4152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.703+0000 7f88d9600640 1 --2- 172.21.15.129:0/1207330145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d40cb5b0 0x7f88d4152720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f88c400d950 tx=0x7f88c400dd50 comp rx=0 tx=0).stop 2024-09-13T08:37:11.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.703+0000 7f88d9600640 1 -- 172.21.15.129:0/1207330145 shutdown_connections 2024-09-13T08:37:11.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.703+0000 7f88d9600640 1 --2- 172.21.15.129:0/1207330145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d415df20 0x7f88d40b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.703+0000 7f88d9600640 1 --2- 172.21.15.129:0/1207330145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40cee80 0x7f88d4158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.703+0000 7f88d9600640 1 --2- 172.21.15.129:0/1207330145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d40cb5b0 0x7f88d4152720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.703+0000 7f88d9600640 1 -- 172.21.15.129:0/1207330145 >> 172.21.15.129:0/1207330145 conn(0x7f88d40d0da0 msgr2=0x7f88d40b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:11.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.703+0000 7f88d9600640 1 -- 172.21.15.129:0/1207330145 shutdown_connections 2024-09-13T08:37:11.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.703+0000 7f88d9600640 1 -- 172.21.15.129:0/1207330145 wait complete. 2024-09-13T08:37:11.719 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new e8dff9fa-c0a5-40f5-995d-190975f1c6b7 3 2024-09-13T08:37:11.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.859+0000 7f9e22000640 1 -- 172.21.15.129:0/385727020 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e1c06d160 msgr2=0x7f9e1c0cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.859+0000 7f9e22000640 1 --2- 172.21.15.129:0/385727020 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e1c06d160 0x7f9e1c0cbcc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009c80 tx=0x7f9e1401cd70 comp rx=0 tx=0).stop 2024-09-13T08:37:11.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.859+0000 7f9e22000640 1 -- 172.21.15.129:0/385727020 shutdown_connections 2024-09-13T08:37:11.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.859+0000 7f9e22000640 1 --2- 172.21.15.129:0/385727020 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e1c06d160 0x7f9e1c0cbcc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.859+0000 7f9e22000640 1 --2- 172.21.15.129:0/385727020 >> v2:172.21.15.44:3300/0 conn(0x7f9e1c15df20 0x7f9e1c06cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.859+0000 7f9e22000640 1 -- 172.21.15.129:0/385727020 >> 172.21.15.129:0/385727020 conn(0x7f9e1c0d0da0 msgr2=0x7f9e1c0d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:11.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.859+0000 7f9e22000640 1 -- 172.21.15.129:0/385727020 shutdown_connections 2024-09-13T08:37:11.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.859+0000 7f9e22000640 1 -- 172.21.15.129:0/385727020 wait complete. 2024-09-13T08:37:11.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e22000640 1 Processor -- start 2024-09-13T08:37:11.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e22000640 1 -- start start 2024-09-13T08:37:11.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e22000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e1c06d160 0x7f9e1c14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e22000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e1c0cee80 0x7f9e1c15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e22000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1c15df20 0x7f9e1c15fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e22000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c16b0a0 con 0x7f9e1c15df20 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e22000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c16af20 con 0x7f9e1c0cee80 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e22000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c16b220 con 0x7f9e1c06d160 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e1b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e1c06d160 0x7f9e1c14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e12a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e1c0cee80 0x7f9e1c15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e1b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e1c06d160 0x7f9e1c14f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.129:42820/0 (socket says 172.21.15.129:42820) 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e1b400640 1 -- 172.21.15.129:0/3683405576 learned_addr learned my addr 172.21.15.129:0/3683405576 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e1be00640 1 --2- 172.21.15.129:0/3683405576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1c15df20 0x7f9e1c15fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e1b400640 1 -- 172.21.15.129:0/3683405576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e1c0cee80 msgr2=0x7f9e1c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e1b400640 1 --2- 172.21.15.129:0/3683405576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e1c0cee80 0x7f9e1c15d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e1b400640 1 -- 172.21.15.129:0/3683405576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1c15df20 msgr2=0x7f9e1c15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e1b400640 1 --2- 172.21.15.129:0/3683405576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1c15df20 0x7f9e1c15fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e1b400640 1 -- 172.21.15.129:0/3683405576 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c15a4d0 con 0x7f9e1c06d160 2024-09-13T08:37:11.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e1b400640 1 --2- 172.21.15.129:0/3683405576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e1c06d160 0x7f9e1c14f4a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00d950 tx=0x7f9e0c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:11.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e13e00640 1 -- 172.21.15.129:0/3683405576 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c017070 con 0x7f9e1c06d160 2024-09-13T08:37:11.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e13e00640 1 -- 172.21.15.129:0/3683405576 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e0c009d20 con 0x7f9e1c06d160 2024-09-13T08:37:11.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e13e00640 1 -- 172.21.15.129:0/3683405576 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0106c0 con 0x7f9e1c06d160 2024-09-13T08:37:11.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e22000640 1 -- 172.21.15.129:0/3683405576 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c14f9e0 con 0x7f9e1c06d160 2024-09-13T08:37:11.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e22000640 1 -- 172.21.15.129:0/3683405576 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c15c1c0 con 0x7f9e1c06d160 2024-09-13T08:37:11.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e13e00640 1 -- 172.21.15.129:0/3683405576 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e0c01a070 con 0x7f9e1c06d160 2024-09-13T08:37:11.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.863+0000 7f9e13e00640 1 -- 172.21.15.129:0/3683405576 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f9e0c010cf0 con 0x7f9e1c06d160 2024-09-13T08:37:11.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.867+0000 7f9e22000640 1 -- 172.21.15.129:0/3683405576 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c06cc20 con 0x7f9e1c06d160 2024-09-13T08:37:11.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.867+0000 7f9e13e00640 1 -- 172.21.15.129:0/3683405576 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9e0c010860 con 0x7f9e1c06d160 2024-09-13T08:37:11.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.939+0000 7f9e22000640 1 -- 172.21.15.129:0/3683405576 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e8dff9fa-c0a5-40f5-995d-190975f1c6b7", "id": 3} v 0) -- 0x7f9e1c0b6d20 con 0x7f9e1c06d160 2024-09-13T08:37:11.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.943+0000 7f9e13e00640 1 -- 172.21.15.129:0/3683405576 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8dff9fa-c0a5-40f5-995d-190975f1c6b7", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f9e0c010ac0 con 0x7f9e1c06d160 2024-09-13T08:37:11.942 INFO:teuthology.orchestra.run.smithi129.stdout:3 2024-09-13T08:37:11.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.943+0000 7f9e22000640 1 -- 172.21.15.129:0/3683405576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e1c06d160 msgr2=0x7f9e1c14f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:11.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.943+0000 7f9e22000640 1 --2- 172.21.15.129:0/3683405576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e1c06d160 0x7f9e1c14f4a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00d950 tx=0x7f9e0c00de10 comp rx=0 tx=0).stop 2024-09-13T08:37:11.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.943+0000 7f9e22000640 1 -- 172.21.15.129:0/3683405576 shutdown_connections 2024-09-13T08:37:11.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.943+0000 7f9e22000640 1 --2- 172.21.15.129:0/3683405576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1c15df20 0x7f9e1c15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.943+0000 7f9e22000640 1 --2- 172.21.15.129:0/3683405576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e1c0cee80 0x7f9e1c15d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.943+0000 7f9e22000640 1 --2- 172.21.15.129:0/3683405576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e1c06d160 0x7f9e1c14f4a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:11.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.943+0000 7f9e22000640 1 -- 172.21.15.129:0/3683405576 >> 172.21.15.129:0/3683405576 conn(0x7f9e1c0d0da0 msgr2=0x7f9e1c151130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:11.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.943+0000 7f9e22000640 1 -- 172.21.15.129:0/3683405576 shutdown_connections 2024-09-13T08:37:11.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:11.943+0000 7f9e22000640 1 -- 172.21.15.129:0/3683405576 wait complete. 2024-09-13T08:37:11.961 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new daf4f548-002d-444f-bdb0-b0c58efe6e41 4 2024-09-13T08:37:12.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.135+0000 7f4c35a00640 1 -- 172.21.15.129:0/3749885311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c30151d40 msgr2=0x7f4c30073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.135+0000 7f4c35a00640 1 --2- 172.21.15.129:0/3749885311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c30151d40 0x7f4c30073c30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0097c0 tx=0x7f4c1c01c6e0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.135+0000 7f4c35a00640 1 -- 172.21.15.129:0/3749885311 shutdown_connections 2024-09-13T08:37:12.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.135+0000 7f4c35a00640 1 --2- 172.21.15.129:0/3749885311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c30151d40 0x7f4c30073c30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.135+0000 7f4c35a00640 1 -- 172.21.15.129:0/3749885311 >> 172.21.15.129:0/3749885311 conn(0x7f4c300d0da0 msgr2=0x7f4c300d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:12.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.135+0000 7f4c35a00640 1 -- 172.21.15.129:0/3749885311 shutdown_connections 2024-09-13T08:37:12.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.135+0000 7f4c35a00640 1 -- 172.21.15.129:0/3749885311 wait complete. 2024-09-13T08:37:12.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c35a00640 1 Processor -- start 2024-09-13T08:37:12.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c35a00640 1 -- start start 2024-09-13T08:37:12.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c35a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c300d00a0 0x7f4c30148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c35a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c30151960 0x7f4c3014eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c35a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c30151d40 0x7f4c3014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c35a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4c300ccea0 con 0x7f4c30151d40 2024-09-13T08:37:12.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c35a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c300ccd20 con 0x7f4c30151960 2024-09-13T08:37:12.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c35a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4c300cd020 con 0x7f4c300d00a0 2024-09-13T08:37:12.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2ea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c300d00a0 0x7f4c30148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2e000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c30151960 0x7f4c3014eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2ea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c300d00a0 0x7f4c30148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.129:42826/0 (socket says 172.21.15.129:42826) 2024-09-13T08:37:12.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2ea00640 1 -- 172.21.15.129:0/217683407 learned_addr learned my addr 172.21.15.129:0/217683407 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-13T08:37:12.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2f400640 1 --2- 172.21.15.129:0/217683407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c30151d40 0x7f4c3014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2e000640 1 -- 172.21.15.129:0/217683407 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c300d00a0 msgr2=0x7f4c30148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2e000640 1 --2- 172.21.15.129:0/217683407 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c300d00a0 0x7f4c30148110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2e000640 1 -- 172.21.15.129:0/217683407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c30151d40 msgr2=0x7f4c3014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2e000640 1 --2- 172.21.15.129:0/217683407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c30151d40 0x7f4c3014f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2e000640 1 -- 172.21.15.129:0/217683407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c30154f10 con 0x7f4c30151960 2024-09-13T08:37:12.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2ea00640 1 --2- 172.21.15.129:0/217683407 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c300d00a0 0x7f4c30148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:12.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c2e000640 1 --2- 172.21.15.129:0/217683407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c30151960 0x7f4c3014eb90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0ac000 tx=0x7f4c1c0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:12.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c26a00640 1 -- 172.21.15.129:0/217683407 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0040f0 con 0x7f4c30151960 2024-09-13T08:37:12.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c26a00640 1 -- 172.21.15.129:0/217683407 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c1c004290 con 0x7f4c30151960 2024-09-13T08:37:12.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c35a00640 1 -- 172.21.15.129:0/217683407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c301551f0 con 0x7f4c30151960 2024-09-13T08:37:12.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c26a00640 1 -- 172.21.15.129:0/217683407 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0046d0 con 0x7f4c30151960 2024-09-13T08:37:12.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c35a00640 1 -- 172.21.15.129:0/217683407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c300b4100 con 0x7f4c30151960 2024-09-13T08:37:12.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c26a00640 1 -- 172.21.15.129:0/217683407 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4c1c004890 con 0x7f4c30151960 2024-09-13T08:37:12.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.139+0000 7f4c26a00640 1 -- 172.21.15.129:0/217683407 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f4c1c003c40 con 0x7f4c30151960 2024-09-13T08:37:12.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.143+0000 7f4c35a00640 1 -- 172.21.15.129:0/217683407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c30073c30 con 0x7f4c30151960 2024-09-13T08:37:12.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.143+0000 7f4c26a00640 1 -- 172.21.15.129:0/217683407 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4c1c0ac030 con 0x7f4c30151960 2024-09-13T08:37:12.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.211+0000 7f4c35a00640 1 -- 172.21.15.129:0/217683407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd new", "uuid": "daf4f548-002d-444f-bdb0-b0c58efe6e41", "id": 4} v 0) -- 0x7f4c300b6d20 con 0x7f4c30151960 2024-09-13T08:37:12.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.215+0000 7f4c26a00640 1 -- 172.21.15.129:0/217683407 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "daf4f548-002d-444f-bdb0-b0c58efe6e41", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f4c1c0162d0 con 0x7f4c30151960 2024-09-13T08:37:12.216 INFO:teuthology.orchestra.run.smithi129.stdout:4 2024-09-13T08:37:12.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.219+0000 7f4c35a00640 1 -- 172.21.15.129:0/217683407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c30151960 msgr2=0x7f4c3014eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.219+0000 7f4c35a00640 1 --2- 172.21.15.129:0/217683407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c30151960 0x7f4c3014eb90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0ac000 tx=0x7f4c1c0056d0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.219+0000 7f4c35a00640 1 -- 172.21.15.129:0/217683407 shutdown_connections 2024-09-13T08:37:12.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.219+0000 7f4c35a00640 1 --2- 172.21.15.129:0/217683407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c30151d40 0x7f4c3014f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.219+0000 7f4c35a00640 1 --2- 172.21.15.129:0/217683407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c30151960 0x7f4c3014eb90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.219+0000 7f4c35a00640 1 --2- 172.21.15.129:0/217683407 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c300d00a0 0x7f4c30148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.219+0000 7f4c35a00640 1 -- 172.21.15.129:0/217683407 >> 172.21.15.129:0/217683407 conn(0x7f4c300d0da0 msgr2=0x7f4c3015c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:12.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.219+0000 7f4c35a00640 1 -- 172.21.15.129:0/217683407 shutdown_connections 2024-09-13T08:37:12.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.219+0000 7f4c35a00640 1 -- 172.21.15.129:0/217683407 wait complete. 2024-09-13T08:37:12.235 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 7363ce33-d055-4b66-96ce-a9d3c2e7ad00 5 2024-09-13T08:37:12.356 INFO:tasks.ceph.mgr.x.smithi129.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T08:37:12.356 INFO:tasks.ceph.mgr.x.smithi129.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T08:37:12.356 INFO:tasks.ceph.mgr.x.smithi129.stderr: from numpy import show_config as show_numpy_config 2024-09-13T08:37:12.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.375+0000 7fed37800640 1 -- 172.21.15.129:0/818751216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed300d29b0 msgr2=0x7fed300b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.375+0000 7fed37800640 1 --2- 172.21.15.129:0/818751216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed300d29b0 0x7fed300b1cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed30074570 tx=0x7fed1801eff0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.375+0000 7fed37800640 1 -- 172.21.15.129:0/818751216 shutdown_connections 2024-09-13T08:37:12.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.375+0000 7fed37800640 1 --2- 172.21.15.129:0/818751216 >> v2:172.21.15.44:3300/0 conn(0x7fed300d04d0 0x7fed300af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.375+0000 7fed37800640 1 --2- 172.21.15.129:0/818751216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed300d29b0 0x7fed300b1cf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.375+0000 7fed37800640 1 -- 172.21.15.129:0/818751216 >> 172.21.15.129:0/818751216 conn(0x7fed300d1e90 msgr2=0x7fed300d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:12.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.375+0000 7fed37800640 1 -- 172.21.15.129:0/818751216 shutdown_connections 2024-09-13T08:37:12.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.375+0000 7fed37800640 1 -- 172.21.15.129:0/818751216 wait complete. 2024-09-13T08:37:12.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed37800640 1 Processor -- start 2024-09-13T08:37:12.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed37800640 1 -- start start 2024-09-13T08:37:12.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed37800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed300d00f0 0x7fed300cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed37800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed300d04d0 0x7fed300cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed37800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed300d29b0 0x7fed301f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed37800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed300b0ed0 con 0x7fed300d04d0 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed37800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed300b0d50 con 0x7fed300d29b0 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed37800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fed300af740 con 0x7fed300d00f0 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed35800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed300d29b0 0x7fed301f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed34e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed300d00f0 0x7fed300cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed35800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed300d29b0 0x7fed301f41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59302/0 (socket says 172.21.15.129:59302) 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed35800640 1 -- 172.21.15.129:0/3929086156 learned_addr learned my addr 172.21.15.129:0/3929086156 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed27e00640 1 --2- 172.21.15.129:0/3929086156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed300d04d0 0x7fed300cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed35800640 1 -- 172.21.15.129:0/3929086156 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed300d00f0 msgr2=0x7fed300cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed35800640 1 --2- 172.21.15.129:0/3929086156 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed300d00f0 0x7fed300cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed35800640 1 -- 172.21.15.129:0/3929086156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed300d04d0 msgr2=0x7fed300cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed35800640 1 --2- 172.21.15.129:0/3929086156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed300d04d0 0x7fed300cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed35800640 1 -- 172.21.15.129:0/3929086156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed301f8f90 con 0x7fed300d29b0 2024-09-13T08:37:12.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed35800640 1 --2- 172.21.15.129:0/3929086156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed300d29b0 0x7fed301f41b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fed2800ea20 tx=0x7fed2800eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:12.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed2d600640 1 -- 172.21.15.129:0/3929086156 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2800cde0 con 0x7fed300d29b0 2024-09-13T08:37:12.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed2d600640 1 -- 172.21.15.129:0/3929086156 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed28004c40 con 0x7fed300d29b0 2024-09-13T08:37:12.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed2d600640 1 -- 172.21.15.129:0/3929086156 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28005280 con 0x7fed300d29b0 2024-09-13T08:37:12.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed37800640 1 -- 172.21.15.129:0/3929086156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed300b7690 con 0x7fed300d29b0 2024-09-13T08:37:12.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed37800640 1 -- 172.21.15.129:0/3929086156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed300cec40 con 0x7fed300d29b0 2024-09-13T08:37:12.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed2d600640 1 -- 172.21.15.129:0/3929086156 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fed28005420 con 0x7fed300d29b0 2024-09-13T08:37:12.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.379+0000 7fed2d600640 1 -- 172.21.15.129:0/3929086156 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fed28005710 con 0x7fed300d29b0 2024-09-13T08:37:12.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.383+0000 7fed37800640 1 -- 172.21.15.129:0/3929086156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed300af200 con 0x7fed300d29b0 2024-09-13T08:37:12.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.383+0000 7fed2d600640 1 -- 172.21.15.129:0/3929086156 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fed28016030 con 0x7fed300d29b0 2024-09-13T08:37:12.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.467+0000 7fed37800640 1 -- 172.21.15.129:0/3929086156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7363ce33-d055-4b66-96ce-a9d3c2e7ad00", "id": 5} v 0) -- 0x7fed3006eb00 con 0x7fed300d29b0 2024-09-13T08:37:12.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.471+0000 7fed2d600640 1 -- 172.21.15.129:0/3929086156 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7363ce33-d055-4b66-96ce-a9d3c2e7ad00", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fed28011ba0 con 0x7fed300d29b0 2024-09-13T08:37:12.472 INFO:teuthology.orchestra.run.smithi129.stdout:5 2024-09-13T08:37:12.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.471+0000 7fed37800640 1 -- 172.21.15.129:0/3929086156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed300d29b0 msgr2=0x7fed301f41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.471+0000 7fed37800640 1 --2- 172.21.15.129:0/3929086156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed300d29b0 0x7fed301f41b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fed2800ea20 tx=0x7fed2800eee0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.471+0000 7fed37800640 1 -- 172.21.15.129:0/3929086156 shutdown_connections 2024-09-13T08:37:12.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.471+0000 7fed37800640 1 --2- 172.21.15.129:0/3929086156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed300d29b0 0x7fed301f41b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.471+0000 7fed37800640 1 --2- 172.21.15.129:0/3929086156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed300d04d0 0x7fed300cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.471+0000 7fed37800640 1 --2- 172.21.15.129:0/3929086156 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed300d00f0 0x7fed300cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.471+0000 7fed37800640 1 -- 172.21.15.129:0/3929086156 >> 172.21.15.129:0/3929086156 conn(0x7fed300d1e90 msgr2=0x7fed3006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:12.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.471+0000 7fed37800640 1 -- 172.21.15.129:0/3929086156 shutdown_connections 2024-09-13T08:37:12.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.475+0000 7fed37800640 1 -- 172.21.15.129:0/3929086156 wait complete. 2024-09-13T08:37:12.491 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 4af3eb16-8c24-4b52-a409-d34dafb11a13 6 2024-09-13T08:37:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.631+0000 7f6038e00640 1 -- 172.21.15.129:0/1546094145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f60340cda70 msgr2=0x7f603406eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.631+0000 7f6038e00640 1 --2- 172.21.15.129:0/1546094145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f60340cda70 0x7f603406eb10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f601c009840 tx=0x7f601c01c7f0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.631+0000 7f6038e00640 1 -- 172.21.15.129:0/1546094145 shutdown_connections 2024-09-13T08:37:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.631+0000 7f6038e00640 1 --2- 172.21.15.129:0/1546094145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f60340cda70 0x7f603406eb10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.631+0000 7f6038e00640 1 --2- 172.21.15.129:0/1546094145 >> v2:172.21.15.44:3300/0 conn(0x7f603415ab60 0x7f60340cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.631+0000 7f6038e00640 1 -- 172.21.15.129:0/1546094145 >> 172.21.15.129:0/1546094145 conn(0x7f60340b1ce0 msgr2=0x7f60340b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.631+0000 7f6038e00640 1 -- 172.21.15.129:0/1546094145 shutdown_connections 2024-09-13T08:37:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.631+0000 7f6038e00640 1 -- 172.21.15.129:0/1546094145 wait complete. 2024-09-13T08:37:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 Processor -- start 2024-09-13T08:37:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 -- start start 2024-09-13T08:37:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6034074a80 0x7f60340afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60340cda70 0x7f60341441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f603415ab60 0x7f60341446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6034168340 con 0x7f60340cda70 2024-09-13T08:37:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f603406f4c0 con 0x7f603415ab60 2024-09-13T08:37:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f60341684c0 con 0x7f6034074a80 2024-09-13T08:37:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6031e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6034074a80 0x7f60340afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6031e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6034074a80 0x7f60340afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.129:42850/0 (socket says 172.21.15.129:42850) 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6031e00640 1 -- 172.21.15.129:0/520763908 learned_addr learned my addr 172.21.15.129:0/520763908 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6032800640 1 --2- 172.21.15.129:0/520763908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f603415ab60 0x7f60341446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6029400640 1 --2- 172.21.15.129:0/520763908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60340cda70 0x7f60341441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6031e00640 1 -- 172.21.15.129:0/520763908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f603415ab60 msgr2=0x7f60341446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6031e00640 1 --2- 172.21.15.129:0/520763908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f603415ab60 0x7f60341446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6031e00640 1 -- 172.21.15.129:0/520763908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60340cda70 msgr2=0x7f60341441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6031e00640 1 --2- 172.21.15.129:0/520763908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60340cda70 0x7f60341441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6031e00640 1 -- 172.21.15.129:0/520763908 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6034144c30 con 0x7f6034074a80 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6031e00640 1 --2- 172.21.15.129:0/520763908 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6034074a80 0x7f60340afdb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f602400d750 tx=0x7f602400dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f602aa00640 1 -- 172.21.15.129:0/520763908 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024017070 con 0x7f6034074a80 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f602aa00640 1 -- 172.21.15.129:0/520763908 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6024009d20 con 0x7f6034074a80 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 -- 172.21.15.129:0/520763908 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60340c7170 con 0x7f6034074a80 2024-09-13T08:37:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f602aa00640 1 -- 172.21.15.129:0/520763908 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602400de60 con 0x7f6034074a80 2024-09-13T08:37:12.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 -- 172.21.15.129:0/520763908 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f60340c7730 con 0x7f6034074a80 2024-09-13T08:37:12.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f602aa00640 1 -- 172.21.15.129:0/520763908 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6024005020 con 0x7f6034074a80 2024-09-13T08:37:12.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f602aa00640 1 -- 172.21.15.129:0/520763908 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f60240055e0 con 0x7f6034074a80 2024-09-13T08:37:12.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.635+0000 7f6038e00640 1 -- 172.21.15.129:0/520763908 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603406eb10 con 0x7f6034074a80 2024-09-13T08:37:12.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.639+0000 7f602aa00640 1 -- 172.21.15.129:0/520763908 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6024020020 con 0x7f6034074a80 2024-09-13T08:37:12.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.707+0000 7f6038e00640 1 -- 172.21.15.129:0/520763908 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4af3eb16-8c24-4b52-a409-d34dafb11a13", "id": 6} v 0) -- 0x7f60340ce2d0 con 0x7f6034074a80 2024-09-13T08:37:12.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.715+0000 7f602aa00640 1 -- 172.21.15.129:0/520763908 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4af3eb16-8c24-4b52-a409-d34dafb11a13", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f6034144c30 con 0x7f6034074a80 2024-09-13T08:37:12.715 INFO:teuthology.orchestra.run.smithi129.stdout:6 2024-09-13T08:37:12.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.715+0000 7f6038e00640 1 -- 172.21.15.129:0/520763908 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6034074a80 msgr2=0x7f60340afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.715+0000 7f6038e00640 1 --2- 172.21.15.129:0/520763908 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6034074a80 0x7f60340afdb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f602400d750 tx=0x7f602400dc10 comp rx=0 tx=0).stop 2024-09-13T08:37:12.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.715+0000 7f6038e00640 1 -- 172.21.15.129:0/520763908 shutdown_connections 2024-09-13T08:37:12.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.715+0000 7f6038e00640 1 --2- 172.21.15.129:0/520763908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f603415ab60 0x7f60341446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.715+0000 7f6038e00640 1 --2- 172.21.15.129:0/520763908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60340cda70 0x7f60341441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.715+0000 7f6038e00640 1 --2- 172.21.15.129:0/520763908 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6034074a80 0x7f60340afdb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.715+0000 7f6038e00640 1 -- 172.21.15.129:0/520763908 >> 172.21.15.129:0/520763908 conn(0x7f60340b1ce0 msgr2=0x7f60340b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:12.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.715+0000 7f6038e00640 1 -- 172.21.15.129:0/520763908 shutdown_connections 2024-09-13T08:37:12.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.715+0000 7f6038e00640 1 -- 172.21.15.129:0/520763908 wait complete. 2024-09-13T08:37:12.733 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 9315c934-f5fb-4bd4-bd3c-cc01accf02d9 7 2024-09-13T08:37:12.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 -- 172.21.15.129:0/476898697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea5815b000 msgr2=0x7fea580cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 --2- 172.21.15.129:0/476898697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea5815b000 0x7fea580cba30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea44009840 tx=0x7fea4401c820 comp rx=0 tx=0).stop 2024-09-13T08:37:12.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 -- 172.21.15.129:0/476898697 shutdown_connections 2024-09-13T08:37:12.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 --2- 172.21.15.129:0/476898697 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea58073780 0x7fea580cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 --2- 172.21.15.129:0/476898697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea5815b000 0x7fea580cba30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 -- 172.21.15.129:0/476898697 >> 172.21.15.129:0/476898697 conn(0x7fea580d04f0 msgr2=0x7fea580d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:12.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 -- 172.21.15.129:0/476898697 shutdown_connections 2024-09-13T08:37:12.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 -- 172.21.15.129:0/476898697 wait complete. 2024-09-13T08:37:12.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 Processor -- start 2024-09-13T08:37:12.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 -- start start 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea58073780 0x7fea580c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea5815ac20 0x7fea580c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.875+0000 7fea5fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea5815b000 0x7fea5814a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fea58168530 con 0x7fea58073780 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fea581683b0 con 0x7fea5815b000 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fea581686b0 con 0x7fea5815ac20 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea5815ac20 0x7fea580c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea5815b000 0x7fea5814a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea5815ac20 0x7fea580c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.129:42868/0 (socket says 172.21.15.129:42868) 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5ca00640 1 -- 172.21.15.129:0/2769004041 learned_addr learned my addr 172.21.15.129:0/2769004041 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5d400640 1 --2- 172.21.15.129:0/2769004041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea58073780 0x7fea580c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:12.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5de00640 1 -- 172.21.15.129:0/2769004041 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea5815ac20 msgr2=0x7fea580c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5de00640 1 --2- 172.21.15.129:0/2769004041 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea5815ac20 0x7fea580c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5de00640 1 -- 172.21.15.129:0/2769004041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea58073780 msgr2=0x7fea580c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5de00640 1 --2- 172.21.15.129:0/2769004041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea58073780 0x7fea580c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5de00640 1 -- 172.21.15.129:0/2769004041 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea581f6c30 con 0x7fea5815b000 2024-09-13T08:37:12.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5de00640 1 --2- 172.21.15.129:0/2769004041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea5815b000 0x7fea5814a5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea50009f90 tx=0x7fea5000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:12.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea55600640 1 -- 172.21.15.129:0/2769004041 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50003980 con 0x7fea5815b000 2024-09-13T08:37:12.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea55600640 1 -- 172.21.15.129:0/2769004041 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea50003c10 con 0x7fea5815b000 2024-09-13T08:37:12.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea55600640 1 -- 172.21.15.129:0/2769004041 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50004020 con 0x7fea5815b000 2024-09-13T08:37:12.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5fe00640 1 -- 172.21.15.129:0/2769004041 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea580c2ea0 con 0x7fea5815b000 2024-09-13T08:37:12.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5fe00640 1 -- 172.21.15.129:0/2769004041 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5814ab00 con 0x7fea5815b000 2024-09-13T08:37:12.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea55600640 1 -- 172.21.15.129:0/2769004041 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fea50004b30 con 0x7fea5815b000 2024-09-13T08:37:12.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea55600640 1 -- 172.21.15.129:0/2769004041 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fea50009860 con 0x7fea5815b000 2024-09-13T08:37:12.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.879+0000 7fea5fe00640 1 -- 172.21.15.129:0/2769004041 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea580c2c90 con 0x7fea5815b000 2024-09-13T08:37:12.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.883+0000 7fea55600640 1 -- 172.21.15.129:0/2769004041 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fea50024070 con 0x7fea5815b000 2024-09-13T08:37:12.933 INFO:tasks.ceph.mgr.y.smithi044.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T08:37:12.933 INFO:tasks.ceph.mgr.y.smithi044.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T08:37:12.933 INFO:tasks.ceph.mgr.y.smithi044.stderr: from numpy import show_config as show_numpy_config 2024-09-13T08:37:12.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.951+0000 7fea5fe00640 1 -- 172.21.15.129:0/2769004041 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9315c934-f5fb-4bd4-bd3c-cc01accf02d9", "id": 7} v 0) -- 0x7fea58073070 con 0x7fea5815b000 2024-09-13T08:37:12.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.959+0000 7fea55600640 1 -- 172.21.15.129:0/2769004041 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9315c934-f5fb-4bd4-bd3c-cc01accf02d9", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fea5000b9d0 con 0x7fea5815b000 2024-09-13T08:37:12.958 INFO:teuthology.orchestra.run.smithi129.stdout:7 2024-09-13T08:37:12.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.959+0000 7fea5fe00640 1 -- 172.21.15.129:0/2769004041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea5815b000 msgr2=0x7fea5814a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:12.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.959+0000 7fea5fe00640 1 --2- 172.21.15.129:0/2769004041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea5815b000 0x7fea5814a5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea50009f90 tx=0x7fea5000c800 comp rx=0 tx=0).stop 2024-09-13T08:37:12.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.959+0000 7fea5fe00640 1 -- 172.21.15.129:0/2769004041 shutdown_connections 2024-09-13T08:37:12.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.959+0000 7fea5fe00640 1 --2- 172.21.15.129:0/2769004041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea5815b000 0x7fea5814a5c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.959+0000 7fea5fe00640 1 --2- 172.21.15.129:0/2769004041 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea5815ac20 0x7fea580c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.959+0000 7fea5fe00640 1 --2- 172.21.15.129:0/2769004041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea58073780 0x7fea580c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:12.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.959+0000 7fea5fe00640 1 -- 172.21.15.129:0/2769004041 >> 172.21.15.129:0/2769004041 conn(0x7fea580d04f0 msgr2=0x7fea580d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:12.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.959+0000 7fea5fe00640 1 -- 172.21.15.129:0/2769004041 shutdown_connections 2024-09-13T08:37:12.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:12.959+0000 7fea5fe00640 1 -- 172.21.15.129:0/2769004041 wait complete. 2024-09-13T08:37:12.977 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T08:37:12.977 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T08:37:12.980 INFO:tasks.ceph.osd.0:Started 2024-09-13T08:37:12.981 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T08:37:12.981 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T08:37:12.984 INFO:tasks.ceph.osd.1:Started 2024-09-13T08:37:12.984 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T08:37:12.984 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T08:37:12.987 INFO:tasks.ceph.osd.2:Started 2024-09-13T08:37:12.987 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T08:37:12.987 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T08:37:12.991 INFO:tasks.ceph.osd.3:Started 2024-09-13T08:37:12.991 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T08:37:12.991 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T08:37:12.994 INFO:tasks.ceph.osd.4:Started 2024-09-13T08:37:12.994 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T08:37:12.994 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T08:37:12.997 INFO:tasks.ceph.osd.5:Started 2024-09-13T08:37:12.997 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T08:37:12.997 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T08:37:13.000 INFO:tasks.ceph.osd.6:Started 2024-09-13T08:37:13.000 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T08:37:13.001 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T08:37:13.004 INFO:tasks.ceph.osd.7:Started 2024-09-13T08:37:13.008 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:37:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.176+0000 7f606b000640 1 -- 172.21.15.44:0/1587020061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606415c1c0 msgr2=0x7f606406d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.176+0000 7f606b000640 1 --2- 172.21.15.44:0/1587020061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606415c1c0 0x7f606406d000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6050009900 tx=0x7f605001c8b0 comp rx=0 tx=0).stop 2024-09-13T08:37:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.176+0000 7f606b000640 1 -- 172.21.15.44:0/1587020061 shutdown_connections 2024-09-13T08:37:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.176+0000 7f606b000640 1 --2- 172.21.15.44:0/1587020061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606415c1c0 0x7f606406d000 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.176+0000 7f606b000640 1 --2- 172.21.15.44:0/1587020061 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f606406cc20 0x7f60640737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.176+0000 7f606b000640 1 -- 172.21.15.44:0/1587020061 >> 172.21.15.44:0/1587020061 conn(0x7f60640d0c90 msgr2=0x7f60640d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.176+0000 7f606b000640 1 -- 172.21.15.44:0/1587020061 shutdown_connections 2024-09-13T08:37:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.176+0000 7f606b000640 1 -- 172.21.15.44:0/1587020061 wait complete. 2024-09-13T08:37:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.176+0000 7f606b000640 1 Processor -- start 2024-09-13T08:37:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.176+0000 7f606b000640 1 -- start start 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f606b000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606406cc20 0x7f60640b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f606b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f606415c1c0 0x7f6064154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f606b000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f606415f7b0 0x7f60640c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f606b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f60641694e0 con 0x7f606415c1c0 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f606b000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6064169360 con 0x7f606406cc20 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f606b000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6064169660 con 0x7f606415f7b0 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6063400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f606415c1c0 0x7f6064154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6063400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f606415c1c0 0x7f6064154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48634/0 (socket says 172.21.15.44:48634) 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6063400640 1 -- 172.21.15.44:0/2695243002 learned_addr learned my addr 172.21.15.44:0/2695243002 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6063e00640 1 --2- 172.21.15.44:0/2695243002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606406cc20 0x7f60640b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6069000640 1 --2- 172.21.15.44:0/2695243002 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f606415f7b0 0x7f60640c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:13.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6063400640 1 -- 172.21.15.44:0/2695243002 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f606415f7b0 msgr2=0x7f60640c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:13.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6063400640 1 --2- 172.21.15.44:0/2695243002 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f606415f7b0 0x7f60640c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:13.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6063400640 1 -- 172.21.15.44:0/2695243002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606406cc20 msgr2=0x7f60640b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:13.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6063400640 1 --2- 172.21.15.44:0/2695243002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606406cc20 0x7f60640b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:13.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6063400640 1 -- 172.21.15.44:0/2695243002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60640b1dd0 con 0x7f606415c1c0 2024-09-13T08:37:13.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f6063400640 1 --2- 172.21.15.44:0/2695243002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f606415c1c0 0x7f6064154d70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6050009850 tx=0x7f6050005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f605be00640 1 -- 172.21.15.44:0/2695243002 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60500b2040 con 0x7f606415c1c0 2024-09-13T08:37:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f605be00640 1 -- 172.21.15.44:0/2695243002 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60500053e0 con 0x7f606415c1c0 2024-09-13T08:37:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f605be00640 1 -- 172.21.15.44:0/2695243002 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60500046d0 con 0x7f606415c1c0 2024-09-13T08:37:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f606b000640 1 -- 172.21.15.44:0/2695243002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60640b20b0 con 0x7f606415c1c0 2024-09-13T08:37:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f606b000640 1 -- 172.21.15.44:0/2695243002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60640b2670 con 0x7f606415c1c0 2024-09-13T08:37:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f605be00640 1 -- 172.21.15.44:0/2695243002 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f60500be060 con 0x7f606415c1c0 2024-09-13T08:37:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f605be00640 1 -- 172.21.15.44:0/2695243002 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6050003c90 con 0x7f606415c1c0 2024-09-13T08:37:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.180+0000 7f606b000640 1 -- 172.21.15.44:0/2695243002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60640737c0 con 0x7f606415c1c0 2024-09-13T08:37:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.184+0000 7f605be00640 1 -- 172.21.15.44:0/2695243002 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f60500164d0 con 0x7f606415c1c0 2024-09-13T08:37:13.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.244+0000 7f606b000640 1 -- 172.21.15.44:0/2695243002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60640cb0c0 con 0x7f606415c1c0 2024-09-13T08:37:13.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.244+0000 7f605be00640 1 -- 172.21.15.44:0/2695243002 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f60500c5020 con 0x7f606415c1c0 2024-09-13T08:37:13.248 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":10,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:12.953327+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:37:13.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.248+0000 7f606b000640 1 -- 172.21.15.44:0/2695243002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f606415c1c0 msgr2=0x7f6064154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:13.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.248+0000 7f606b000640 1 --2- 172.21.15.44:0/2695243002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f606415c1c0 0x7f6064154d70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6050009850 tx=0x7f6050005e70 comp rx=0 tx=0).stop 2024-09-13T08:37:13.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.248+0000 7f606b000640 1 -- 172.21.15.44:0/2695243002 shutdown_connections 2024-09-13T08:37:13.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.248+0000 7f606b000640 1 --2- 172.21.15.44:0/2695243002 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f606415f7b0 0x7f60640c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:13.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.248+0000 7f606b000640 1 --2- 172.21.15.44:0/2695243002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f606415c1c0 0x7f6064154d70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:13.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.248+0000 7f606b000640 1 --2- 172.21.15.44:0/2695243002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606406cc20 0x7f60640b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:13.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.248+0000 7f606b000640 1 -- 172.21.15.44:0/2695243002 >> 172.21.15.44:0/2695243002 conn(0x7f60640d0c90 msgr2=0x7f60641662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:13.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.248+0000 7f606b000640 1 -- 172.21.15.44:0/2695243002 shutdown_connections 2024-09-13T08:37:13.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:13.248+0000 7f606b000640 1 -- 172.21.15.44:0/2695243002 wait complete. 2024-09-13T08:37:13.268 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T08:37:13.269 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T08:37:14.075 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T08:37:14.216 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.212+0000 7f6eee400640 1 -- 172.21.15.44:0/3115100352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ee815fad0 msgr2=0x7f6ee815ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:14.216 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.212+0000 7f6eee400640 1 --2- 172.21.15.44:0/3115100352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ee815fad0 0x7f6ee815ff60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009bc0 tx=0x7f6ee001cba0 comp rx=0 tx=0).stop 2024-09-13T08:37:14.216 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.212+0000 7f6eee400640 1 -- 172.21.15.44:0/3115100352 shutdown_connections 2024-09-13T08:37:14.216 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.212+0000 7f6eee400640 1 --2- 172.21.15.44:0/3115100352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ee815fad0 0x7f6ee815ff60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:14.216 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.212+0000 7f6eee400640 1 -- 172.21.15.44:0/3115100352 >> 172.21.15.44:0/3115100352 conn(0x7f6ee80d0c20 msgr2=0x7f6ee80d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:14.216 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.212+0000 7f6eee400640 1 -- 172.21.15.44:0/3115100352 shutdown_connections 2024-09-13T08:37:14.216 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.212+0000 7f6eee400640 1 -- 172.21.15.44:0/3115100352 wait complete. 2024-09-13T08:37:14.217 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.212+0000 7f6eee400640 1 Processor -- start 2024-09-13T08:37:14.217 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6eee400640 1 -- start start 2024-09-13T08:37:14.219 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6eee400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6ee806cc20 0x7f6ee8158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:14.219 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6eee400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee815c5b0 0x7f6ee80b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:14.219 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6eee400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ee815fad0 0x7f6ee81fac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:14.219 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6eee400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee816a210 con 0x7f6ee815c5b0 2024-09-13T08:37:14.219 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6eee400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee816a090 con 0x7f6ee815fad0 2024-09-13T08:37:14.219 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6eee400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee816a390 con 0x7f6ee806cc20 2024-09-13T08:37:14.219 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee815c5b0 0x7f6ee80b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:14.219 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee815c5b0 0x7f6ee80b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48644/0 (socket says 172.21.15.44:48644) 2024-09-13T08:37:14.219 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee6a00640 1 -- 172.21.15.44:0/3884242993 learned_addr learned my addr 172.21.15.44:0/3884242993 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:14.219 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee7400640 1 --2- 172.21.15.44:0/3884242993 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6ee806cc20 0x7f6ee8158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:14.220 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee7e00640 1 --2- 172.21.15.44:0/3884242993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ee815fad0 0x7f6ee81fac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:14.220 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee6a00640 1 -- 172.21.15.44:0/3884242993 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6ee806cc20 msgr2=0x7f6ee8158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:14.220 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee6a00640 1 --2- 172.21.15.44:0/3884242993 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6ee806cc20 0x7f6ee8158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:14.220 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee6a00640 1 -- 172.21.15.44:0/3884242993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ee815fad0 msgr2=0x7f6ee81fac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:14.220 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee6a00640 1 --2- 172.21.15.44:0/3884242993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ee815fad0 0x7f6ee81fac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:14.220 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee6a00640 1 -- 172.21.15.44:0/3884242993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee81fb940 con 0x7f6ee815c5b0 2024-09-13T08:37:14.220 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6ee6a00640 1 --2- 172.21.15.44:0/3884242993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee815c5b0 0x7f6ee80b2b10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c500 tx=0x7f6ed400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:14.221 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6edf400640 1 -- 172.21.15.44:0/3884242993 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4016020 con 0x7f6ee815c5b0 2024-09-13T08:37:14.221 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6edf400640 1 -- 172.21.15.44:0/3884242993 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ed4012920 con 0x7f6ee815c5b0 2024-09-13T08:37:14.221 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6eee400640 1 -- 172.21.15.44:0/3884242993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee80cf3a0 con 0x7f6ee815c5b0 2024-09-13T08:37:14.221 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6edf400640 1 -- 172.21.15.44:0/3884242993 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4012d50 con 0x7f6ee815c5b0 2024-09-13T08:37:14.221 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6eee400640 1 -- 172.21.15.44:0/3884242993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee80cf960 con 0x7f6ee815c5b0 2024-09-13T08:37:14.221 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6edf400640 1 -- 172.21.15.44:0/3884242993 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ed4019020 con 0x7f6ee815c5b0 2024-09-13T08:37:14.221 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6edf400640 1 -- 172.21.15.44:0/3884242993 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6ed401a3b0 con 0x7f6ee815c5b0 2024-09-13T08:37:14.221 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.216+0000 7f6eee400640 1 -- 172.21.15.44:0/3884242993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee815ff60 con 0x7f6ee815c5b0 2024-09-13T08:37:14.223 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.220+0000 7f6edf400640 1 -- 172.21.15.44:0/3884242993 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6ed4020030 con 0x7f6ee815c5b0 2024-09-13T08:37:14.285 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6eee400640 1 -- 172.21.15.44:0/3884242993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ee80b6ba0 con 0x7f6ee815c5b0 2024-09-13T08:37:14.286 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6edf400640 1 -- 172.21.15.44:0/3884242993 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6ed401a950 con 0x7f6ee815c5b0 2024-09-13T08:37:14.286 INFO:teuthology.misc.health.smithi044.stdout: 2024-09-13T08:37:14.286 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":10,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:12.953327+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:37:14.287 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6eee400640 1 -- 172.21.15.44:0/3884242993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee815c5b0 msgr2=0x7f6ee80b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:14.287 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6eee400640 1 --2- 172.21.15.44:0/3884242993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee815c5b0 0x7f6ee80b2b10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c500 tx=0x7f6ed400c9c0 comp rx=0 tx=0).stop 2024-09-13T08:37:14.287 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6eee400640 1 -- 172.21.15.44:0/3884242993 shutdown_connections 2024-09-13T08:37:14.287 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6eee400640 1 --2- 172.21.15.44:0/3884242993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ee815fad0 0x7f6ee81fac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:14.287 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6eee400640 1 --2- 172.21.15.44:0/3884242993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee815c5b0 0x7f6ee80b2b10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:14.287 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6eee400640 1 --2- 172.21.15.44:0/3884242993 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6ee806cc20 0x7f6ee8158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:14.287 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6eee400640 1 -- 172.21.15.44:0/3884242993 >> 172.21.15.44:0/3884242993 conn(0x7f6ee80d0c20 msgr2=0x7f6ee80c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:14.288 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6eee400640 1 -- 172.21.15.44:0/3884242993 shutdown_connections 2024-09-13T08:37:14.288 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:14.284+0000 7f6eee400640 1 -- 172.21.15.44:0/3884242993 wait complete. 2024-09-13T08:37:14.304 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T08:37:14.690 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-13T08:37:14.691+0000 7f7bdbbab7c0 -1 Falling back to public interface 2024-09-13T08:37:14.698 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T08:37:14.696+0000 7ff12b7c67c0 -1 Falling back to public interface 2024-09-13T08:37:14.706 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T08:37:14.704+0000 7f64427577c0 -1 Falling back to public interface 2024-09-13T08:37:14.714 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-13T08:37:14.715+0000 7f84b0f4a7c0 -1 Falling back to public interface 2024-09-13T08:37:14.715 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T08:37:14.715+0000 7fd0fd81e7c0 -1 Falling back to public interface 2024-09-13T08:37:14.750 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T08:37:14.748+0000 7fac9f4127c0 -1 Falling back to public interface 2024-09-13T08:37:14.906 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T08:37:14.907+0000 7fe599e6b7c0 -1 Falling back to public interface 2024-09-13T08:37:14.966 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-09-13T08:37:14.964+0000 7fd44112d7c0 -1 Falling back to public interface 2024-09-13T08:37:15.512 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:37:15.511+0000 7f8aee600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T08:37:15.512 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:37:15.511+0000 7f8aee600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T08:37:18.896 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-13T08:37:18.895+0000 7f7bdbbab7c0 -1 osd.6 0 log_to_monitors true 2024-09-13T08:37:19.019 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-13T08:37:19.019+0000 7f84b0f4a7c0 -1 osd.7 0 log_to_monitors true 2024-09-13T08:37:19.042 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T08:37:19.043+0000 7fd0fd81e7c0 -1 osd.5 0 log_to_monitors true 2024-09-13T08:37:19.063 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T08:37:19.060+0000 7ff12b7c67c0 -1 osd.0 0 log_to_monitors true 2024-09-13T08:37:19.102 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T08:37:19.100+0000 7f64427577c0 -1 osd.2 0 log_to_monitors true 2024-09-13T08:37:19.174 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-09-13T08:37:19.172+0000 7fd44112d7c0 -1 osd.3 0 log_to_monitors true 2024-09-13T08:37:19.380 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T08:37:19.376+0000 7fac9f4127c0 -1 osd.1 0 log_to_monitors true 2024-09-13T08:37:19.575 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T08:37:19.575+0000 7fe599e6b7c0 -1 osd.4 0 log_to_monitors true 2024-09-13T08:37:20.959 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-13T08:37:20.959+0000 7f7bd6a00640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T08:37:21.011 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-13T08:37:21.011+0000 7f7bcca00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T08:37:21.110 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T08:37:21.276 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.272+0000 7f434d800640 1 -- 172.21.15.44:0/639012441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f434815bfa0 msgr2=0x7f43480cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:21.276 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.272+0000 7f434d800640 1 --2- 172.21.15.44:0/639012441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f434815bfa0 0x7f43480cd830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f43480cc9a0 tx=0x7f433801c8c0 comp rx=0 tx=0).stop 2024-09-13T08:37:21.276 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.272+0000 7f434d800640 1 -- 172.21.15.44:0/639012441 shutdown_connections 2024-09-13T08:37:21.276 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.272+0000 7f434d800640 1 --2- 172.21.15.44:0/639012441 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43480cfe40 0x7f43480c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:21.276 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.272+0000 7f434d800640 1 --2- 172.21.15.44:0/639012441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f434815bfa0 0x7f43480cd830 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:21.276 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.272+0000 7f434d800640 1 -- 172.21.15.44:0/639012441 >> 172.21.15.44:0/639012441 conn(0x7f43480d1d30 msgr2=0x7f43480d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:21.276 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.272+0000 7f434d800640 1 -- 172.21.15.44:0/639012441 shutdown_connections 2024-09-13T08:37:21.277 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.272+0000 7f434d800640 1 -- 172.21.15.44:0/639012441 wait complete. 2024-09-13T08:37:21.278 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f434d800640 1 Processor -- start 2024-09-13T08:37:21.278 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f434d800640 1 -- start start 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f434d800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43480c5dc0 0x7f434814f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f434d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43480cfe40 0x7f43480b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f434d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43480b6110 0x7f4348154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f434d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43481671a0 con 0x7f43480cfe40 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f434d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4348167020 con 0x7f43480b6110 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f434d800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4348167320 con 0x7f43480c5dc0 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43480cfe40 0x7f43480b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43480cfe40 0x7f43480b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48712/0 (socket says 172.21.15.44:48712) 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346000640 1 -- 172.21.15.44:0/3892070243 learned_addr learned my addr 172.21.15.44:0/3892070243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346a00640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43480c5dc0 0x7f434814f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:21.280 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4347400640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43480b6110 0x7f4348154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:21.281 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346000640 1 -- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43480c5dc0 msgr2=0x7f434814f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346000640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43480c5dc0 0x7f434814f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346000640 1 -- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43480b6110 msgr2=0x7f4348154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346000640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43480b6110 0x7f4348154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346000640 1 -- 172.21.15.44:0/3892070243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4348157290 con 0x7f43480cfe40 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346a00640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43480c5dc0 0x7f434814f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f4346000640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43480cfe40 0x7f43480b5bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f433400ea20 tx=0x7f433400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f433ea00640 1 -- 172.21.15.44:0/3892070243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433400cd80 con 0x7f43480cfe40 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f433ea00640 1 -- 172.21.15.44:0/3892070243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4334010430 con 0x7f43480cfe40 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f434d800640 1 -- 172.21.15.44:0/3892070243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4348157570 con 0x7f43480cfe40 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f433ea00640 1 -- 172.21.15.44:0/3892070243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334010860 con 0x7f43480cfe40 2024-09-13T08:37:21.282 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.276+0000 7f434d800640 1 -- 172.21.15.44:0/3892070243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4348157b30 con 0x7f43480cfe40 2024-09-13T08:37:21.283 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.280+0000 7f434d800640 1 -- 172.21.15.44:0/3892070243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43480b6520 con 0x7f43480cfe40 2024-09-13T08:37:21.283 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.280+0000 7f433ea00640 1 -- 172.21.15.44:0/3892070243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) ==== 97024+0+0 (secure 0 0 0) 0x7f4334010a00 con 0x7f43480cfe40 2024-09-13T08:37:21.284 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.280+0000 7f433ea00640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4328072e30 0x7f43280752f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:21.284 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.280+0000 7f433ea00640 1 -- 172.21.15.44:0/3892070243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f4334093b10 con 0x7f43480cfe40 2024-09-13T08:37:21.285 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.280+0000 7f4346a00640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4328072e30 0x7f43280752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:21.286 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.284+0000 7f4346a00640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4328072e30 0x7f43280752f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f43380af000 tx=0x7f433801b100 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:21.290 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.288+0000 7f433ea00640 1 -- 172.21.15.44:0/3892070243 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f43480b6520 con 0x7f43480cfe40 2024-09-13T08:37:21.519 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.516+0000 7f434d800640 1 -- 172.21.15.44:0/3892070243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4348167830 con 0x7f43480cfe40 2024-09-13T08:37:21.520 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.516+0000 7f433ea00640 1 -- 172.21.15.44:0/3892070243 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f433405f4e0 con 0x7f43480cfe40 2024-09-13T08:37:21.520 INFO:teuthology.misc.health.smithi044.stdout: 2024-09-13T08:37:21.520 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":13,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:20.961713+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:37:21.523 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 -- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4328072e30 msgr2=0x7f43280752f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:21.523 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4328072e30 0x7f43280752f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f43380af000 tx=0x7f433801b100 comp rx=0 tx=0).stop 2024-09-13T08:37:21.523 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 -- 172.21.15.44:0/3892070243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43480cfe40 msgr2=0x7f43480b5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:21.523 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43480cfe40 0x7f43480b5bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f433400ea20 tx=0x7f433400eee0 comp rx=0 tx=0).stop 2024-09-13T08:37:21.523 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 -- 172.21.15.44:0/3892070243 shutdown_connections 2024-09-13T08:37:21.523 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4328072e30 0x7f43280752f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:21.523 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43480b6110 0x7f4348154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:21.523 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43480cfe40 0x7f43480b5bd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:21.524 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 --2- 172.21.15.44:0/3892070243 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43480c5dc0 0x7f434814f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:21.524 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 -- 172.21.15.44:0/3892070243 >> 172.21.15.44:0/3892070243 conn(0x7f43480d1d30 msgr2=0x7f43480b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:21.524 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 -- 172.21.15.44:0/3892070243 shutdown_connections 2024-09-13T08:37:21.524 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:21.520+0000 7f434d800640 1 -- 172.21.15.44:0/3892070243 wait complete. 2024-09-13T08:37:21.549 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T08:37:22.798 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T08:37:22.799+0000 7fe594e00640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T08:37:22.805 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-13T08:37:22.803+0000 7f84abe00640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T08:37:22.822 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T08:37:22.823+0000 7fe58ae00640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T08:37:22.835 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T08:37:22.835+0000 7fd0f9000640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T08:37:22.842 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-13T08:37:22.843+0000 7f84a1e00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T08:37:22.870 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T08:37:22.871+0000 7fd0ee600640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T08:37:23.214 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-09-13T08:37:23.212+0000 7fd43c000640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T08:37:23.222 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T08:37:23.220+0000 7fac9a200640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T08:37:23.241 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T08:37:23.240+0000 7ff126600640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T08:37:23.246 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-09-13T08:37:23.244+0000 7fd432000640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T08:37:23.262 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T08:37:23.260+0000 7f643d600640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T08:37:23.266 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T08:37:23.264+0000 7fac90200640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T08:37:23.282 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T08:37:23.280+0000 7ff11c600640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T08:37:23.302 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T08:37:23.300+0000 7f6433600640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T08:37:28.356 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T08:37:28.504 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.500+0000 7f7852a00640 1 -- 172.21.15.44:0/3170544482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c0c5f90 msgr2=0x7f784c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:28.505 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.500+0000 7f7852a00640 1 --2- 172.21.15.44:0/3170544482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c0c5f90 0x7f784c0ce740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f784c15f860 tx=0x7f784401cb80 comp rx=0 tx=0).stop 2024-09-13T08:37:28.505 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.500+0000 7f7852a00640 1 -- 172.21.15.44:0/3170544482 shutdown_connections 2024-09-13T08:37:28.505 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.500+0000 7f7852a00640 1 --2- 172.21.15.44:0/3170544482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c0c5f90 0x7f784c0ce740 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.505 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.500+0000 7f7852a00640 1 --2- 172.21.15.44:0/3170544482 >> v2:172.21.15.129:3300/0 conn(0x7f784c0cfe40 0x7f784c0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.505 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.500+0000 7f7852a00640 1 -- 172.21.15.44:0/3170544482 >> 172.21.15.44:0/3170544482 conn(0x7f784c0d1d30 msgr2=0x7f784c0d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:28.505 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.500+0000 7f7852a00640 1 -- 172.21.15.44:0/3170544482 shutdown_connections 2024-09-13T08:37:28.505 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.500+0000 7f7852a00640 1 -- 172.21.15.44:0/3170544482 wait complete. 2024-09-13T08:37:28.506 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7852a00640 1 Processor -- start 2024-09-13T08:37:28.506 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7852a00640 1 -- start start 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7852a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f784c0c5f90 0x7f784c156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7852a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f784c0cfe40 0x7f784c1fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7852a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c0b72f0 0x7f784c154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7852a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f784c1695f0 con 0x7f784c0b72f0 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7852a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f784c169470 con 0x7f784c0cfe40 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7852a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f784c169770 con 0x7f784c0c5f90 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7850a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c0b72f0 0x7f784c154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7850a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c0b72f0 0x7f784c154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42624/0 (socket says 172.21.15.44:42624) 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7850a00640 1 -- 172.21.15.44:0/3514288560 learned_addr learned my addr 172.21.15.44:0/3514288560 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f784b400640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f784c0cfe40 0x7f784c1fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:28.508 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f784be00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f784c0c5f90 0x7f784c156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:28.509 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7850a00640 1 -- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f784c0c5f90 msgr2=0x7f784c156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:28.509 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7850a00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f784c0c5f90 0x7f784c156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.509 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7850a00640 1 -- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f784c0cfe40 msgr2=0x7f784c1fbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:28.509 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7850a00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f784c0cfe40 0x7f784c1fbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.509 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7850a00640 1 -- 172.21.15.44:0/3514288560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c0b7840 con 0x7f784c0b72f0 2024-09-13T08:37:28.510 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7850a00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c0b72f0 0x7f784c154f30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7844009b10 tx=0x7f7844009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:28.510 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7843e00640 1 -- 172.21.15.44:0/3514288560 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440bb070 con 0x7f784c0b72f0 2024-09-13T08:37:28.510 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7843e00640 1 -- 172.21.15.44:0/3514288560 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7844004b40 con 0x7f784c0b72f0 2024-09-13T08:37:28.510 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7852a00640 1 -- 172.21.15.44:0/3514288560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c0b5780 con 0x7f784c0b72f0 2024-09-13T08:37:28.510 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7843e00640 1 -- 172.21.15.44:0/3514288560 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440b2040 con 0x7f784c0b72f0 2024-09-13T08:37:28.510 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.504+0000 7f7852a00640 1 -- 172.21.15.44:0/3514288560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c0b5d40 con 0x7f784c0b72f0 2024-09-13T08:37:28.510 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.508+0000 7f7852a00640 1 -- 172.21.15.44:0/3514288560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784c0b5f40 con 0x7f784c0b72f0 2024-09-13T08:37:28.511 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.508+0000 7f7843e00640 1 -- 172.21.15.44:0/3514288560 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f78440be070 con 0x7f784c0b72f0 2024-09-13T08:37:28.511 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.508+0000 7f7843e00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f782c072ef0 0x7f782c0753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:28.512 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.508+0000 7f7843e00640 1 -- 172.21.15.44:0/3514288560 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f784412d4b0 con 0x7f784c0b72f0 2024-09-13T08:37:28.512 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.508+0000 7f784be00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f782c072ef0 0x7f782c0753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:28.513 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.508+0000 7f784be00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f782c072ef0 0x7f782c0753b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f784c0cc9a0 tx=0x7f783c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:28.516 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.512+0000 7f7843e00640 1 -- 172.21.15.44:0/3514288560 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f784c0b5f40 con 0x7f784c0b72f0 2024-09-13T08:37:28.699 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.696+0000 7f7852a00640 1 -- 172.21.15.44:0/3514288560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f784c076200 con 0x7f784c0b72f0 2024-09-13T08:37:28.701 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.696+0000 7f7843e00640 1 -- 172.21.15.44:0/3514288560 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12908 (secure 0 0 0) 0x7f7844016770 con 0x7f784c0b72f0 2024-09-13T08:37:28.701 INFO:teuthology.misc.health.smithi044.stdout: 2024-09-13T08:37:28.702 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":18,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:26.820529+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:37:28.703 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 -- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f782c072ef0 msgr2=0x7f782c0753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:28.703 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f782c072ef0 0x7f782c0753b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f784c0cc9a0 tx=0x7f783c009040 comp rx=0 tx=0).stop 2024-09-13T08:37:28.703 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 -- 172.21.15.44:0/3514288560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c0b72f0 msgr2=0x7f784c154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:28.703 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c0b72f0 0x7f784c154f30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7844009b10 tx=0x7f7844009300 comp rx=0 tx=0).stop 2024-09-13T08:37:28.703 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 -- 172.21.15.44:0/3514288560 shutdown_connections 2024-09-13T08:37:28.703 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f782c072ef0 0x7f782c0753b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.703 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c0b72f0 0x7f784c154f30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.703 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f784c0cfe40 0x7f784c1fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.703 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 --2- 172.21.15.44:0/3514288560 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f784c0c5f90 0x7f784c156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.704 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 -- 172.21.15.44:0/3514288560 >> 172.21.15.44:0/3514288560 conn(0x7f784c0d1d30 msgr2=0x7f784c0708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:28.704 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 -- 172.21.15.44:0/3514288560 shutdown_connections 2024-09-13T08:37:28.704 INFO:teuthology.misc.health.smithi044.stderr:2024-09-13T08:37:28.700+0000 7f7852a00640 1 -- 172.21.15.44:0/3514288560 wait complete. 2024-09-13T08:37:28.720 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T08:37:28.720 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T08:37:28.721 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T08:37:28.721 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T08:37:28.724 INFO:tasks.ceph.mds.a:Started 2024-09-13T08:37:28.724 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T08:37:28.724 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T08:37:28.727 INFO:tasks.ceph.mds.c:Started 2024-09-13T08:37:28.727 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T08:37:28.727 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T08:37:28.731 INFO:tasks.ceph.mds.b:Started 2024-09-13T08:37:28.731 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T08:37:28.731 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T08:37:28.734 INFO:tasks.ceph.mds.d:Started 2024-09-13T08:37:28.734 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-13T08:37:28.734 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:37:28.820 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:37:28.820 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:37:28.861 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:37:28.864 INFO:tasks.ceph.mds.b.smithi129.stdout:starting mds.b at 2024-09-13T08:37:28.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.884+0000 7f7f53e00640 1 -- 172.21.15.44:0/1546742171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f540b4d80 msgr2=0x7f7f540b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:28.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.884+0000 7f7f53e00640 1 --2- 172.21.15.44:0/1546742171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f540b4d80 0x7f7f540b11a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009840 tx=0x7f7f4401c740 comp rx=0 tx=0).stop 2024-09-13T08:37:28.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.884+0000 7f7f53e00640 1 -- 172.21.15.44:0/1546742171 shutdown_connections 2024-09-13T08:37:28.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.884+0000 7f7f53e00640 1 --2- 172.21.15.44:0/1546742171 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f540b5870 0x7f7f54165670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.884+0000 7f7f53e00640 1 --2- 172.21.15.44:0/1546742171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f540b4d80 0x7f7f540b11a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.884+0000 7f7f53e00640 1 -- 172.21.15.44:0/1546742171 >> 172.21.15.44:0/1546742171 conn(0x7f7f5409aa10 msgr2=0x7f7f5409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:28.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.884+0000 7f7f53e00640 1 -- 172.21.15.44:0/1546742171 shutdown_connections 2024-09-13T08:37:28.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.884+0000 7f7f53e00640 1 -- 172.21.15.44:0/1546742171 wait complete. 2024-09-13T08:37:28.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 Processor -- start 2024-09-13T08:37:28.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 -- start start 2024-09-13T08:37:28.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f540b0ce0 0x7f7f541f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:28.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f540b4d80 0x7f7f5415b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:28.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f540b5870 0x7f7f54155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:28.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54168a70 con 0x7f7f540b4d80 2024-09-13T08:37:28.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541688f0 con 0x7f7f540b0ce0 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7f54168bf0 con 0x7f7f540b5870 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f540b4d80 0x7f7f5415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f540b4d80 0x7f7f5415b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42662/0 (socket says 172.21.15.44:42662) 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52000640 1 -- 172.21.15.44:0/788205039 learned_addr learned my addr 172.21.15.44:0/788205039 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f540b0ce0 0x7f7f541f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53400640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f540b5870 0x7f7f54155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52000640 1 -- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f540b5870 msgr2=0x7f7f54155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52000640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f540b5870 0x7f7f54155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52000640 1 -- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f540b0ce0 msgr2=0x7f7f541f8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52000640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f540b0ce0 0x7f7f541f8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52000640 1 -- 172.21.15.44:0/788205039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5414f110 con 0x7f7f540b4d80 2024-09-13T08:37:28.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52a00640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f540b0ce0 0x7f7f541f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:28.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f52000640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f540b4d80 0x7f7f5415b410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000c760 tx=0x7f7f4000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:28.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f4b400640 1 -- 172.21.15.44:0/788205039 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40016020 con 0x7f7f540b4d80 2024-09-13T08:37:28.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f4b400640 1 -- 172.21.15.44:0/788205039 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f4000a8f0 con 0x7f7f540b4d80 2024-09-13T08:37:28.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f4b400640 1 -- 172.21.15.44:0/788205039 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4000ad40 con 0x7f7f540b4d80 2024-09-13T08:37:28.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 -- 172.21.15.44:0/788205039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5414f3f0 con 0x7f7f540b4d80 2024-09-13T08:37:28.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 -- 172.21.15.44:0/788205039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5414f9b0 con 0x7f7f540b4d80 2024-09-13T08:37:28.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.888+0000 7f7f53e00640 1 -- 172.21.15.44:0/788205039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f540b11a0 con 0x7f7f540b4d80 2024-09-13T08:37:28.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.892+0000 7f7f4b400640 1 -- 172.21.15.44:0/788205039 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f7f40019020 con 0x7f7f540b4d80 2024-09-13T08:37:28.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.892+0000 7f7f4b400640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f34072ef0 0x7f7f340753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:28.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.892+0000 7f7f4b400640 1 -- 172.21.15.44:0/788205039 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f7f40093b60 con 0x7f7f540b4d80 2024-09-13T08:37:28.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.892+0000 7f7f52a00640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f34072ef0 0x7f7f340753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:28.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.892+0000 7f7f52a00640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f34072ef0 0x7f7f340753b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f440af000 tx=0x7f7f440b4040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:28.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:28.896+0000 7f7f4b400640 1 -- 172.21.15.44:0/788205039 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7f40020020 con 0x7f7f540b4d80 2024-09-13T08:37:29.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 -- 172.21.15.44:0/788205039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f540b1e20 con 0x7f7f540b4d80 2024-09-13T08:37:29.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f4b400640 1 -- 172.21.15.44:0/788205039 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7f7f4005ea10 con 0x7f7f540b4d80 2024-09-13T08:37:29.118 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:29.119 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"btime":"2024-09-13T08:37:28:828351+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4335,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/853789824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":853789824},{"type":"v1","addr":"172.21.15.44:6834","nonce":853789824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-13T08:37:29.119 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 2 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 -- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f34072ef0 msgr2=0x7f7f340753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f34072ef0 0x7f7f340753b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f440af000 tx=0x7f7f440b4040 comp rx=0 tx=0).stop 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 -- 172.21.15.44:0/788205039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f540b4d80 msgr2=0x7f7f5415b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f540b4d80 0x7f7f5415b410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000c760 tx=0x7f7f4000cc20 comp rx=0 tx=0).stop 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 -- 172.21.15.44:0/788205039 shutdown_connections 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f34072ef0 0x7f7f340753b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f540b5870 0x7f7f54155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f540b4d80 0x7f7f5415b410 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 --2- 172.21.15.44:0/788205039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f540b0ce0 0x7f7f541f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 -- 172.21.15.44:0/788205039 >> 172.21.15.44:0/788205039 conn(0x7f7f5409aa10 msgr2=0x7f7f540b3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 -- 172.21.15.44:0/788205039 shutdown_connections 2024-09-13T08:37:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:29.116+0000 7f7f53e00640 1 -- 172.21.15.44:0/788205039 wait complete. 2024-09-13T08:37:30.875 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:37:30.878 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:37:31.140 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:37:31.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.299+0000 7fe736a00640 1 -- 172.21.15.44:0/105961678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe714003fe0 msgr2=0x7fe714004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:31.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.299+0000 7fe736a00640 1 --2- 172.21.15.44:0/105961678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe714003fe0 0x7fe714004810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe71800c310 tx=0x7fe71801cca0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.299+0000 7fe734a00640 1 -- 172.21.15.44:0/105961678 reap_dead start 2024-09-13T08:37:31.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.299+0000 7fe736a00640 1 -- 172.21.15.44:0/105961678 shutdown_connections 2024-09-13T08:37:31.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.299+0000 7fe736a00640 1 -- 172.21.15.44:0/105961678 >> 172.21.15.44:0/105961678 conn(0x7fe7300d1ea0 msgr2=0x7fe7300d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:31.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.299+0000 7fe736a00640 1 -- 172.21.15.44:0/105961678 shutdown_connections 2024-09-13T08:37:31.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.299+0000 7fe736a00640 1 -- 172.21.15.44:0/105961678 wait complete. 2024-09-13T08:37:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.299+0000 7fe736a00640 1 Processor -- start 2024-09-13T08:37:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.299+0000 7fe736a00640 1 -- start start 2024-09-13T08:37:31.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.299+0000 7fe736a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe714003fe0 0x7fe7301583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe736a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe730158900 0x7fe730163d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe736a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe730157f40 0x7fe730164260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe736a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe730167360 con 0x7fe714003fe0 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe736a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7301671e0 con 0x7fe730157f40 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe736a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe7301674e0 con 0x7fe730158900 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe714003fe0 0x7fe7301583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe714003fe0 0x7fe7301583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42676/0 (socket says 172.21.15.44:42676) 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72fe00640 1 -- 172.21.15.44:0/2720951671 learned_addr learned my addr 172.21.15.44:0/2720951671 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe727e00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe730158900 0x7fe730163d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe734a00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe730157f40 0x7fe730164260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72fe00640 1 -- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe730158900 msgr2=0x7fe730163d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72fe00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe730158900 0x7fe730163d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72fe00640 1 -- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe730157f40 msgr2=0x7fe730164260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:31.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72fe00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe730157f40 0x7fe730164260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72fe00640 1 -- 172.21.15.44:0/2720951671 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe730162a90 con 0x7fe714003fe0 2024-09-13T08:37:31.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72fe00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe714003fe0 0x7fe7301583c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe720011a80 tx=0x7fe720011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:31.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72cc00640 1 -- 172.21.15.44:0/2720951671 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72000fe10 con 0x7fe714003fe0 2024-09-13T08:37:31.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72cc00640 1 -- 172.21.15.44:0/2720951671 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe720016070 con 0x7fe714003fe0 2024-09-13T08:37:31.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72cc00640 1 -- 172.21.15.44:0/2720951671 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720012780 con 0x7fe714003fe0 2024-09-13T08:37:31.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe736a00640 1 -- 172.21.15.44:0/2720951671 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe730162d70 con 0x7fe714003fe0 2024-09-13T08:37:31.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe736a00640 1 -- 172.21.15.44:0/2720951671 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe730163330 con 0x7fe714003fe0 2024-09-13T08:37:31.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe736a00640 1 -- 172.21.15.44:0/2720951671 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7300c5dd0 con 0x7fe714003fe0 2024-09-13T08:37:31.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.303+0000 7fe72cc00640 1 -- 172.21.15.44:0/2720951671 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fe720018030 con 0x7fe714003fe0 2024-09-13T08:37:31.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.307+0000 7fe72cc00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c072ec0 0x7fe70c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:31.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.307+0000 7fe72cc00640 1 -- 172.21.15.44:0/2720951671 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fe7200943b0 con 0x7fe714003fe0 2024-09-13T08:37:31.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.307+0000 7fe727e00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c072ec0 0x7fe70c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:31.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.307+0000 7fe727e00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c072ec0 0x7fe70c075380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe7180ac000 tx=0x7fe718019e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.311+0000 7fe72cc00640 1 -- 172.21.15.44:0/2720951671 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe72005f8b0 con 0x7fe714003fe0 2024-09-13T08:37:31.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 -- 172.21.15.44:0/2720951671 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe730076360 con 0x7fe714003fe0 2024-09-13T08:37:31.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe72cc00640 1 -- 172.21.15.44:0/2720951671 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3906 (secure 0 0 0) 0x7fe720064e20 con 0x7fe714003fe0 2024-09-13T08:37:31.523 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:31.523 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":3,"btime":"2024-09-13T08:37:29:829848+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2931832597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2931832597},{"type":"v1","addr":"172.21.15.129:6837","nonce":2931832597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4335,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/853789824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":853789824},{"type":"v1","addr":"172.21.15.44:6834","nonce":853789824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/686971411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":686971411},{"type":"v1","addr":"172.21.15.44:6835","nonce":686971411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1614710769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1614710769},{"type":"v1","addr":"172.21.15.129:6835","nonce":1614710769}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-13T08:37:31.524 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 3 2024-09-13T08:37:31.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 -- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c072ec0 msgr2=0x7fe70c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c072ec0 0x7fe70c075380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe7180ac000 tx=0x7fe718019e00 comp rx=0 tx=0).stop 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 -- 172.21.15.44:0/2720951671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe714003fe0 msgr2=0x7fe7301583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe714003fe0 0x7fe7301583c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe720011a80 tx=0x7fe720011f40 comp rx=0 tx=0).stop 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 -- 172.21.15.44:0/2720951671 shutdown_connections 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c072ec0 0x7fe70c075380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe730157f40 0x7fe730164260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe730158900 0x7fe730163d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 --2- 172.21.15.44:0/2720951671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe714003fe0 0x7fe7301583c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.519+0000 7fe736a00640 1 -- 172.21.15.44:0/2720951671 >> 172.21.15.44:0/2720951671 conn(0x7fe7300d1ea0 msgr2=0x7fe730166130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.523+0000 7fe736a00640 1 -- 172.21.15.44:0/2720951671 shutdown_connections 2024-09-13T08:37:31.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.523+0000 7fe736a00640 1 -- 172.21.15.44:0/2720951671 wait complete. 2024-09-13T08:37:31.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T08:37:31.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 -- 172.21.15.44:0/2755337263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cdc0b11d0 msgr2=0x7f0cdc0afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:31.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 --2- 172.21.15.44:0/2755337263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cdc0b11d0 0x7f0cdc0afcd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc074010 tx=0x7f0cd401cc60 comp rx=0 tx=0).stop 2024-09-13T08:37:31.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 -- 172.21.15.44:0/2755337263 shutdown_connections 2024-09-13T08:37:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 --2- 172.21.15.44:0/2755337263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cdc0b11d0 0x7f0cdc0afcd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 --2- 172.21.15.44:0/2755337263 >> v2:172.21.15.129:3300/0 conn(0x7f0cdc15e330 0x7f0cdc06ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 --2- 172.21.15.44:0/2755337263 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0cdc15df50 0x7f0cdc06e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 -- 172.21.15.44:0/2755337263 >> 172.21.15.44:0/2755337263 conn(0x7f0cdc0d1dc0 msgr2=0x7f0cdc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 -- 172.21.15.44:0/2755337263 shutdown_connections 2024-09-13T08:37:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 -- 172.21.15.44:0/2755337263 wait complete. 2024-09-13T08:37:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 Processor -- start 2024-09-13T08:37:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.687+0000 7f0ce0e00640 1 -- start start 2024-09-13T08:37:31.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0ce0e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cdc0b11d0 0x7f0cdc161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:31.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0ce0e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0cdc15df50 0x7f0cdc1fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0ce0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cdc15e330 0x7f0cdc0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0ce0e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc0b2ef0 con 0x7f0cdc15e330 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0ce0e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc0b2d70 con 0x7f0cdc0b11d0 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0ce0e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0cdc0b3070 con 0x7f0cdc15df50 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cda800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cdc15e330 0x7f0cdc0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cd9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cdc0b11d0 0x7f0cdc161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cda800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cdc15e330 0x7f0cdc0cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42684/0 (socket says 172.21.15.44:42684) 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cda800640 1 -- 172.21.15.44:0/211054789 learned_addr learned my addr 172.21.15.44:0/211054789 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cd9400640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0cdc15df50 0x7f0cdc1fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cda800640 1 -- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0cdc15df50 msgr2=0x7f0cdc1fbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cda800640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0cdc15df50 0x7f0cdc1fbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cda800640 1 -- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cdc0b11d0 msgr2=0x7f0cdc161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:31.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cda800640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cdc0b11d0 0x7f0cdc161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cda800640 1 -- 172.21.15.44:0/211054789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc09f8c0 con 0x7f0cdc15e330 2024-09-13T08:37:31.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cda800640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cdc15e330 0x7f0cdc0cd690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0cd401b310 tx=0x7f0cd4009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:31.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cd2000640 1 -- 172.21.15.44:0/211054789 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40bb070 con 0x7f0cdc15e330 2024-09-13T08:37:31.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cd2000640 1 -- 172.21.15.44:0/211054789 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cd4004d40 con 0x7f0cdc15e330 2024-09-13T08:37:31.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cd2000640 1 -- 172.21.15.44:0/211054789 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40b2040 con 0x7f0cdc15e330 2024-09-13T08:37:31.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0ce0e00640 1 -- 172.21.15.44:0/211054789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cdc160990 con 0x7f0cdc15e330 2024-09-13T08:37:31.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0ce0e00640 1 -- 172.21.15.44:0/211054789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cdc160f50 con 0x7f0cdc15e330 2024-09-13T08:37:31.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0ce0e00640 1 -- 172.21.15.44:0/211054789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cdc161150 con 0x7f0cdc15e330 2024-09-13T08:37:31.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.691+0000 7f0cd2000640 1 -- 172.21.15.44:0/211054789 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f0cd40be070 con 0x7f0cdc15e330 2024-09-13T08:37:31.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.695+0000 7f0cd2000640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cbc072ec0 0x7f0cbc075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:31.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.695+0000 7f0cd2000640 1 -- 172.21.15.44:0/211054789 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f0cd412d730 con 0x7f0cdc15e330 2024-09-13T08:37:31.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.695+0000 7f0cd9e00640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cbc072ec0 0x7f0cbc075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:31.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.695+0000 7f0cd9e00640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cbc072ec0 0x7f0cbc075380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc004630 tx=0x7f0ccc00b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:31.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.699+0000 7f0cd2000640 1 -- 172.21.15.44:0/211054789 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0cdc161150 con 0x7f0cdc15e330 2024-09-13T08:37:31.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.875+0000 7f0ce0e00640 1 -- 172.21.15.44:0/211054789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0cdc160780 con 0x7f0cdc15e330 2024-09-13T08:37:31.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.875+0000 7f0cd2000640 1 -- 172.21.15.44:0/211054789 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f0cd4016970 con 0x7f0cdc15e330 2024-09-13T08:37:31.881 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:31.881 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:37:31.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:37:31.881 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T08:37:31.881 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T08:37:31.882 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:37:31.882 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:37:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 -- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cbc072ec0 msgr2=0x7f0cbc075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cbc072ec0 0x7f0cbc075380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc004630 tx=0x7f0ccc00b390 comp rx=0 tx=0).stop 2024-09-13T08:37:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 -- 172.21.15.44:0/211054789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cdc15e330 msgr2=0x7f0cdc0cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cdc15e330 0x7f0cdc0cd690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0cd401b310 tx=0x7f0cd4009370 comp rx=0 tx=0).stop 2024-09-13T08:37:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 -- 172.21.15.44:0/211054789 shutdown_connections 2024-09-13T08:37:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cbc072ec0 0x7f0cbc075380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cdc15e330 0x7f0cdc0cd690 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0cdc15df50 0x7f0cdc1fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 --2- 172.21.15.44:0/211054789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cdc0b11d0 0x7f0cdc161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 -- 172.21.15.44:0/211054789 >> 172.21.15.44:0/211054789 conn(0x7f0cdc0d1dc0 msgr2=0x7f0cdc0715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 -- 172.21.15.44:0/211054789 shutdown_connections 2024-09-13T08:37:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:31.879+0000 7f0ce0e00640 1 -- 172.21.15.44:0/211054789 wait complete. 2024-09-13T08:37:31.902 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T08:37:31.902 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T08:37:32.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.047+0000 7ff4dc200640 1 -- 172.21.15.44:0/1637863178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4d415be10 msgr2=0x7ff4d40ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:32.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.047+0000 7ff4dc200640 1 --2- 172.21.15.44:0/1637863178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4d415be10 0x7ff4d40ce700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009840 tx=0x7ff4c801c820 comp rx=0 tx=0).stop 2024-09-13T08:37:32.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.047+0000 7ff4dc200640 1 -- 172.21.15.44:0/1637863178 shutdown_connections 2024-09-13T08:37:32.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.047+0000 7ff4dc200640 1 --2- 172.21.15.44:0/1637863178 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff4d40b0b60 0x7ff4d40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:32.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.047+0000 7ff4dc200640 1 --2- 172.21.15.44:0/1637863178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4d415be10 0x7ff4d40ce700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:32.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.047+0000 7ff4dc200640 1 -- 172.21.15.44:0/1637863178 >> 172.21.15.44:0/1637863178 conn(0x7ff4d40d1da0 msgr2=0x7ff4d40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:32.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.047+0000 7ff4dc200640 1 -- 172.21.15.44:0/1637863178 shutdown_connections 2024-09-13T08:37:32.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.047+0000 7ff4dc200640 1 -- 172.21.15.44:0/1637863178 wait complete. 2024-09-13T08:37:32.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 Processor -- start 2024-09-13T08:37:32.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 -- start start 2024-09-13T08:37:32.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4d40b0b60 0x7ff4d40c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff4d415be10 0x7ff4d4150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4d415c1f0 0x7ff4d4150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4169040 con 0x7ff4d415c1f0 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4168ec0 con 0x7ff4d40b0b60 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d41691c0 con 0x7ff4d415be10 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4da200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4d415c1f0 0x7ff4d4150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4da200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4d415c1f0 0x7ff4d4150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42700/0 (socket says 172.21.15.44:42700) 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4da200640 1 -- 172.21.15.44:0/74805033 learned_addr learned my addr 172.21.15.44:0/74805033 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4d9800640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4d40b0b60 0x7ff4d40c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4d8e00640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff4d415be10 0x7ff4d4150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:32.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4da200640 1 -- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff4d415be10 msgr2=0x7ff4d4150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4da200640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff4d415be10 0x7ff4d4150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4da200640 1 -- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4d40b0b60 msgr2=0x7ff4d40c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4da200640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4d40b0b60 0x7ff4d40c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4da200640 1 -- 172.21.15.44:0/74805033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d40cadf0 con 0x7ff4d415c1f0 2024-09-13T08:37:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4da200640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4d415c1f0 0x7ff4d4150e90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00b9a0 tx=0x7ff4cc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:32.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4d1600640 1 -- 172.21.15.44:0/74805033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc019070 con 0x7ff4d415c1f0 2024-09-13T08:37:32.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4d1600640 1 -- 172.21.15.44:0/74805033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4cc00ccb0 con 0x7ff4d415c1f0 2024-09-13T08:37:32.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4d1600640 1 -- 172.21.15.44:0/74805033 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0146a0 con 0x7ff4d415c1f0 2024-09-13T08:37:32.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 -- 172.21.15.44:0/74805033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d40c6380 con 0x7ff4d415c1f0 2024-09-13T08:37:32.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 -- 172.21.15.44:0/74805033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d40c6940 con 0x7ff4d415c1f0 2024-09-13T08:37:32.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.051+0000 7ff4dc200640 1 -- 172.21.15.44:0/74805033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d40ccbd0 con 0x7ff4d415c1f0 2024-09-13T08:37:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.055+0000 7ff4d1600640 1 -- 172.21.15.44:0/74805033 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7ff4cc01c070 con 0x7ff4d415c1f0 2024-09-13T08:37:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.055+0000 7ff4d1600640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff4b4072ec0 0x7ff4b4075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.055+0000 7ff4d1600640 1 -- 172.21.15.44:0/74805033 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7ff4cc0938f0 con 0x7ff4d415c1f0 2024-09-13T08:37:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.055+0000 7ff4d9800640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff4b4072ec0 0x7ff4b4075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:32.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.055+0000 7ff4d9800640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff4b4072ec0 0x7ff4b4075380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80af000 tx=0x7ff4c801b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.059+0000 7ff4d1600640 1 -- 172.21.15.44:0/74805033 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff4cc05e710 con 0x7ff4d415c1f0 2024-09-13T08:37:32.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.235+0000 7ff4dc200640 1 -- 172.21.15.44:0/74805033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff4d4070350 con 0x7ff4d415c1f0 2024-09-13T08:37:32.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.847+0000 7ff4d1600640 1 -- 172.21.15.44:0/74805033 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7ff4cc063140 con 0x7ff4d415c1f0 2024-09-13T08:37:32.853 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T08:37:32.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 -- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff4b4072ec0 msgr2=0x7ff4b4075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:32.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff4b4072ec0 0x7ff4b4075380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80af000 tx=0x7ff4c801b060 comp rx=0 tx=0).stop 2024-09-13T08:37:32.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 -- 172.21.15.44:0/74805033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4d415c1f0 msgr2=0x7ff4d4150e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:32.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4d415c1f0 0x7ff4d4150e90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00b9a0 tx=0x7ff4cc00be60 comp rx=0 tx=0).stop 2024-09-13T08:37:32.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 -- 172.21.15.44:0/74805033 shutdown_connections 2024-09-13T08:37:32.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff4b4072ec0 0x7ff4b4075380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:32.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4d415c1f0 0x7ff4d4150e90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:32.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff4d415be10 0x7ff4d4150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:32.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 --2- 172.21.15.44:0/74805033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4d40b0b60 0x7ff4d40c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:32.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 -- 172.21.15.44:0/74805033 >> 172.21.15.44:0/74805033 conn(0x7ff4d40d1da0 msgr2=0x7ff4d40ce9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:32.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 -- 172.21.15.44:0/74805033 shutdown_connections 2024-09-13T08:37:32.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:32.851+0000 7ff4dc200640 1 -- 172.21.15.44:0/74805033 wait complete. 2024-09-13T08:37:32.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T08:37:33.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.039+0000 7f4a62a00640 1 -- 172.21.15.44:0/1201309199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a400052f0 msgr2=0x7f4a4000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:33.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.039+0000 7f4a62a00640 1 --2- 172.21.15.44:0/1201309199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a400052f0 0x7f4a4000b9d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4a54006fd0 tx=0x7f4a5401caf0 comp rx=0 tx=0).stop 2024-09-13T08:37:33.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.039+0000 7f4a62a00640 1 -- 172.21.15.44:0/1201309199 shutdown_connections 2024-09-13T08:37:33.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.039+0000 7f4a62a00640 1 --2- 172.21.15.44:0/1201309199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a400052f0 0x7f4a4000b9d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:33.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.039+0000 7f4a62a00640 1 -- 172.21.15.44:0/1201309199 >> 172.21.15.44:0/1201309199 conn(0x7f4a5c0d1dd0 msgr2=0x7f4a5c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:33.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.039+0000 7f4a62a00640 1 -- 172.21.15.44:0/1201309199 shutdown_connections 2024-09-13T08:37:33.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.039+0000 7f4a62a00640 1 -- 172.21.15.44:0/1201309199 wait complete. 2024-09-13T08:37:33.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 Processor -- start 2024-09-13T08:37:33.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 -- start start 2024-09-13T08:37:33.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a400052f0 0x7f4a5c0ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:33.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a5c0c80b0 0x7f4a5c0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:33.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c0b6ab0 0x7f4a5c0b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:33.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c0b1220 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c0b0ee0 con 0x7f4a5c0c80b0 2024-09-13T08:37:33.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5c0b13a0 con 0x7f4a400052f0 2024-09-13T08:37:33.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a60a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c0b6ab0 0x7f4a5c0b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:33.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a60a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c0b6ab0 0x7f4a5c0b6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60564/0 (socket says 172.21.15.44:60564) 2024-09-13T08:37:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a60a00640 1 -- 172.21.15.44:0/117239186 learned_addr learned my addr 172.21.15.44:0/117239186 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a53400640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a5c0c80b0 0x7f4a5c0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a5be00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a400052f0 0x7f4a5c0ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a60a00640 1 -- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a400052f0 msgr2=0x7f4a5c0ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a60a00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a400052f0 0x7f4a5c0ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a60a00640 1 -- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a5c0c80b0 msgr2=0x7f4a5c0c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a60a00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a5c0c80b0 0x7f4a5c0c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a60a00640 1 -- 172.21.15.44:0/117239186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c0c8a50 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a5be00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a400052f0 0x7f4a5c0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a60a00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c0b6ab0 0x7f4a5c0b6f60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a5401c5b0 tx=0x7f4a54004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:33.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a58c00640 1 -- 172.21.15.44:0/117239186 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54006cb0 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a58c00640 1 -- 172.21.15.44:0/117239186 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a540062e0 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a58c00640 1 -- 172.21.15.44:0/117239186 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540066d0 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 -- 172.21.15.44:0/117239186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c0b5cf0 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 -- 172.21.15.44:0/117239186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c0b62b0 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.043+0000 7f4a62a00640 1 -- 172.21.15.44:0/117239186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5c0d0690 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.047+0000 7f4a58c00640 1 -- 172.21.15.44:0/117239186 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f4a540c4430 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.047+0000 7f4a58c00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a38072ec0 0x7f4a38075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:33.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.047+0000 7f4a58c00640 1 -- 172.21.15.44:0/117239186 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f4a5412ea70 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.047+0000 7f4a5be00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a38072ec0 0x7f4a38075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:33.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.047+0000 7f4a5be00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a38072ec0 0x7f4a38075380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000fc0 tx=0x7f4a4c0067b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:33.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.051+0000 7f4a58c00640 1 -- 172.21.15.44:0/117239186 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4a54016850 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.227+0000 7f4a62a00640 1 -- 172.21.15.44:0/117239186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4a5c0b1f20 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.859+0000 7f4a58c00640 1 -- 172.21.15.44:0/117239186 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f4a540f99d0 con 0x7f4a5c0b6ab0 2024-09-13T08:37:33.864 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T08:37:33.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.863+0000 7f4a62a00640 1 -- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a38072ec0 msgr2=0x7f4a38075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:33.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.863+0000 7f4a62a00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a38072ec0 0x7f4a38075380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000fc0 tx=0x7f4a4c0067b0 comp rx=0 tx=0).stop 2024-09-13T08:37:33.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.863+0000 7f4a62a00640 1 -- 172.21.15.44:0/117239186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c0b6ab0 msgr2=0x7f4a5c0b6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:33.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.863+0000 7f4a62a00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c0b6ab0 0x7f4a5c0b6f60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a5401c5b0 tx=0x7f4a54004ff0 comp rx=0 tx=0).stop 2024-09-13T08:37:33.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.867+0000 7f4a62a00640 1 -- 172.21.15.44:0/117239186 shutdown_connections 2024-09-13T08:37:33.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.867+0000 7f4a62a00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a38072ec0 0x7f4a38075380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:33.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.867+0000 7f4a62a00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c0b6ab0 0x7f4a5c0b6f60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:33.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.867+0000 7f4a62a00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a5c0c80b0 0x7f4a5c0c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:33.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.867+0000 7f4a62a00640 1 --2- 172.21.15.44:0/117239186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a400052f0 0x7f4a5c0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:33.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.867+0000 7f4a62a00640 1 -- 172.21.15.44:0/117239186 >> 172.21.15.44:0/117239186 conn(0x7f4a5c0d1dd0 msgr2=0x7f4a5c071510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:33.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.867+0000 7f4a62a00640 1 -- 172.21.15.44:0/117239186 shutdown_connections 2024-09-13T08:37:33.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:33.867+0000 7f4a62a00640 1 -- 172.21.15.44:0/117239186 wait complete. 2024-09-13T08:37:33.914 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T08:37:34.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.063+0000 7f136e800640 1 -- 172.21.15.44:0/1729724278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13680cf1d0 msgr2=0x7f136815fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:34.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.063+0000 7f136e800640 1 --2- 172.21.15.44:0/1729724278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13680cf1d0 0x7f136815fd80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1350009840 tx=0x7f135001c7f0 comp rx=0 tx=0).stop 2024-09-13T08:37:34.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.063+0000 7f136e800640 1 -- 172.21.15.44:0/1729724278 shutdown_connections 2024-09-13T08:37:34.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.063+0000 7f136e800640 1 --2- 172.21.15.44:0/1729724278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13680cf1d0 0x7f136815fd80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:34.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.063+0000 7f136e800640 1 --2- 172.21.15.44:0/1729724278 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f136815f9a0 0x7f136806e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:34.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.063+0000 7f136e800640 1 -- 172.21.15.44:0/1729724278 >> 172.21.15.44:0/1729724278 conn(0x7f13680b18f0 msgr2=0x7f13680b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:34.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.063+0000 7f136e800640 1 -- 172.21.15.44:0/1729724278 shutdown_connections 2024-09-13T08:37:34.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.063+0000 7f136e800640 1 -- 172.21.15.44:0/1729724278 wait complete. 2024-09-13T08:37:34.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.063+0000 7f136e800640 1 Processor -- start 2024-09-13T08:37:34.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.063+0000 7f136e800640 1 -- start start 2024-09-13T08:37:34.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f136e800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13680cf1d0 0x7f13681fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:34.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f136e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f136815f9a0 0x7f136809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:34.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f136e800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f13681602c0 0x7f1368148540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:34.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f136e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1368169720 con 0x7f136815f9a0 2024-09-13T08:37:34.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f136e800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f13681695a0 con 0x7f13680cf1d0 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f136e800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f13681698a0 con 0x7f13681602c0 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f136815f9a0 0x7f136809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f136815f9a0 0x7f136809aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60576/0 (socket says 172.21.15.44:60576) 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135ea00640 1 -- 172.21.15.44:0/3751968796 learned_addr learned my addr 172.21.15.44:0/3751968796 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f1367400640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13680cf1d0 0x7f13681fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f1367e00640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f13681602c0 0x7f1368148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135ea00640 1 -- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f13681602c0 msgr2=0x7f1368148540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135ea00640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f13681602c0 0x7f1368148540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135ea00640 1 -- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13680cf1d0 msgr2=0x7f13681fbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135ea00640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13680cf1d0 0x7f13681fbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135ea00640 1 -- 172.21.15.44:0/3751968796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1368165160 con 0x7f136815f9a0 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135ea00640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f136815f9a0 0x7f136809aa10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1350009790 tx=0x7f1350004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:34.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135fe00640 1 -- 172.21.15.44:0/3751968796 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500b2040 con 0x7f136815f9a0 2024-09-13T08:37:34.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135fe00640 1 -- 172.21.15.44:0/3751968796 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1350004430 con 0x7f136815f9a0 2024-09-13T08:37:34.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f135fe00640 1 -- 172.21.15.44:0/3751968796 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350005280 con 0x7f136815f9a0 2024-09-13T08:37:34.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f136e800640 1 -- 172.21.15.44:0/3751968796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1368165420 con 0x7f136815f9a0 2024-09-13T08:37:34.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f136e800640 1 -- 172.21.15.44:0/3751968796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1368148a80 con 0x7f136815f9a0 2024-09-13T08:37:34.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.067+0000 7f136e800640 1 -- 172.21.15.44:0/3751968796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1368069410 con 0x7f136815f9a0 2024-09-13T08:37:34.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.071+0000 7f135fe00640 1 -- 172.21.15.44:0/3751968796 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f1350005420 con 0x7f136815f9a0 2024-09-13T08:37:34.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.071+0000 7f135fe00640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1344072ec0 0x7f1344075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:34.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.071+0000 7f135fe00640 1 -- 172.21.15.44:0/3751968796 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f135012dd90 con 0x7f136815f9a0 2024-09-13T08:37:34.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.071+0000 7f1367400640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1344072ec0 0x7f1344075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:34.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.071+0000 7f1367400640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1344072ec0 0x7f1344075380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1358005c50 tx=0x7f1358005be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:34.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.075+0000 7f135fe00640 1 -- 172.21.15.44:0/3751968796 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1350016410 con 0x7f136815f9a0 2024-09-13T08:37:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.283+0000 7f136e800640 1 -- 172.21.15.44:0/3751968796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f136815fd80 con 0x7f136815f9a0 2024-09-13T08:37:34.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.863+0000 7f135fe00640 1 -- 172.21.15.44:0/3751968796 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f13500f8a60 con 0x7f136815f9a0 2024-09-13T08:37:34.866 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T08:37:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.863+0000 7f136e800640 1 -- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1344072ec0 msgr2=0x7f1344075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.863+0000 7f136e800640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1344072ec0 0x7f1344075380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1358005c50 tx=0x7f1358005be0 comp rx=0 tx=0).stop 2024-09-13T08:37:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.863+0000 7f136e800640 1 -- 172.21.15.44:0/3751968796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f136815f9a0 msgr2=0x7f136809aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.863+0000 7f136e800640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f136815f9a0 0x7f136809aa10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1350009790 tx=0x7f1350004040 comp rx=0 tx=0).stop 2024-09-13T08:37:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.867+0000 7f136e800640 1 -- 172.21.15.44:0/3751968796 shutdown_connections 2024-09-13T08:37:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.867+0000 7f136e800640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1344072ec0 0x7f1344075380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.867+0000 7f136e800640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f13681602c0 0x7f1368148540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.867+0000 7f136e800640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f136815f9a0 0x7f136809aa10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.867+0000 7f136e800640 1 --2- 172.21.15.44:0/3751968796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13680cf1d0 0x7f13681fbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.867+0000 7f136e800640 1 -- 172.21.15.44:0/3751968796 >> 172.21.15.44:0/3751968796 conn(0x7f13680b18f0 msgr2=0x7f1368159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.867+0000 7f136e800640 1 -- 172.21.15.44:0/3751968796 shutdown_connections 2024-09-13T08:37:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:34.867+0000 7f136e800640 1 -- 172.21.15.44:0/3751968796 wait complete. 2024-09-13T08:37:34.918 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:37:35.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 -- 172.21.15.44:0/342594154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0480cebf0 msgr2=0x7fa0480cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 --2- 172.21.15.44:0/342594154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0480cebf0 0x7fa0480cc8b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa030009840 tx=0x7fa03001c820 comp rx=0 tx=0).stop 2024-09-13T08:37:35.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 -- 172.21.15.44:0/342594154 shutdown_connections 2024-09-13T08:37:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 --2- 172.21.15.44:0/342594154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0480cebf0 0x7fa0480cc8b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 -- 172.21.15.44:0/342594154 >> 172.21.15.44:0/342594154 conn(0x7fa0480d1da0 msgr2=0x7fa0480d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 -- 172.21.15.44:0/342594154 shutdown_connections 2024-09-13T08:37:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 -- 172.21.15.44:0/342594154 wait complete. 2024-09-13T08:37:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 Processor -- start 2024-09-13T08:37:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 -- start start 2024-09-13T08:37:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0480b6400 0x7fa048154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0480cebf0 0x7fa048156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa04815be80 0x7fa048156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa04816a360 con 0x7fa04815be80 2024-09-13T08:37:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa04816a1e0 con 0x7fa0480cebf0 2024-09-13T08:37:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa04e800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa04816a4e0 con 0x7fa0480b6400 2024-09-13T08:37:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa047e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa04815be80 0x7fa048156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa047e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa04815be80 0x7fa048156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60594/0 (socket says 172.21.15.44:60594) 2024-09-13T08:37:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa047e00640 1 -- 172.21.15.44:0/1152834345 learned_addr learned my addr 172.21.15.44:0/1152834345 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa047400640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0480b6400 0x7fa048154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa03fe00640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0480cebf0 0x7fa048156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa047e00640 1 -- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0480b6400 msgr2=0x7fa048154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa047e00640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0480b6400 0x7fa048154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa047e00640 1 -- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0480cebf0 msgr2=0x7fa048156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa047e00640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0480cebf0 0x7fa048156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa047e00640 1 -- 172.21.15.44:0/1152834345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0480cf670 con 0x7fa04815be80 2024-09-13T08:37:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.063+0000 7fa047400640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0480b6400 0x7fa048154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa047e00640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa04815be80 0x7fa048156570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa04000cbc0 tx=0x7fa04000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa03f400640 1 -- 172.21.15.44:0/1152834345 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040016020 con 0x7fa04815be80 2024-09-13T08:37:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa03f400640 1 -- 172.21.15.44:0/1152834345 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa04000ab40 con 0x7fa04815be80 2024-09-13T08:37:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa03f400640 1 -- 172.21.15.44:0/1152834345 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040012420 con 0x7fa04815be80 2024-09-13T08:37:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa04e800640 1 -- 172.21.15.44:0/1152834345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0481fb460 con 0x7fa04815be80 2024-09-13T08:37:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa04e800640 1 -- 172.21.15.44:0/1152834345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0481fba20 con 0x7fa04815be80 2024-09-13T08:37:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa04e800640 1 -- 172.21.15.44:0/1152834345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0480ce4e0 con 0x7fa04815be80 2024-09-13T08:37:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa03f400640 1 -- 172.21.15.44:0/1152834345 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fa040019020 con 0x7fa04815be80 2024-09-13T08:37:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa03f400640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa024072ec0 0x7fa024075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa03f400640 1 -- 172.21.15.44:0/1152834345 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa040093c60 con 0x7fa04815be80 2024-09-13T08:37:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.067+0000 7fa047400640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa024072ec0 0x7fa024075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.071+0000 7fa047400640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa024072ec0 0x7fa024075380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0480c72a0 tx=0x7fa038009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.071+0000 7fa03f400640 1 -- 172.21.15.44:0/1152834345 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa040020030 con 0x7fa04815be80 2024-09-13T08:37:35.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 -- 172.21.15.44:0/1152834345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0480cdfb0 con 0x7fa04815be80 2024-09-13T08:37:35.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa03f400640 1 -- 172.21.15.44:0/1152834345 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16694 (secure 0 0 0) 0x7fa0480d0350 con 0x7fa04815be80 2024-09-13T08:37:35.259 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:35.259 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":21,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:34.290410+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:37:32.247896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T08:37:33.241202+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:37:35.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 -- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa024072ec0 msgr2=0x7fa024075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa024072ec0 0x7fa024075380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0480c72a0 tx=0x7fa038009040 comp rx=0 tx=0).stop 2024-09-13T08:37:35.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 -- 172.21.15.44:0/1152834345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa04815be80 msgr2=0x7fa048156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa04815be80 0x7fa048156570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa04000cbc0 tx=0x7fa04000ccb0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 -- 172.21.15.44:0/1152834345 shutdown_connections 2024-09-13T08:37:35.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa024072ec0 0x7fa024075380 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa04815be80 0x7fa048156570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0480cebf0 0x7fa048156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 --2- 172.21.15.44:0/1152834345 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0480b6400 0x7fa048154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 -- 172.21.15.44:0/1152834345 >> 172.21.15.44:0/1152834345 conn(0x7fa0480d1da0 msgr2=0x7fa048071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 -- 172.21.15.44:0/1152834345 shutdown_connections 2024-09-13T08:37:35.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.255+0000 7fa04e800640 1 -- 172.21.15.44:0/1152834345 wait complete. 2024-09-13T08:37:35.281 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:37:35.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.423+0000 7feef8400640 1 -- 172.21.15.44:0/1454337575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed8004d50 msgr2=0x7feed80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.423+0000 7feef8400640 1 --2- 172.21.15.44:0/1454337575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed8004d50 0x7feed80051d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feee8006fd0 tx=0x7feee801cb80 comp rx=0 tx=0).stop 2024-09-13T08:37:35.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.423+0000 7feef6400640 1 -- 172.21.15.44:0/1454337575 reap_dead start 2024-09-13T08:37:35.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.423+0000 7feef8400640 1 -- 172.21.15.44:0/1454337575 shutdown_connections 2024-09-13T08:37:35.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.423+0000 7feef8400640 1 -- 172.21.15.44:0/1454337575 >> 172.21.15.44:0/1454337575 conn(0x7feef00d1dc0 msgr2=0x7feef00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:35.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.423+0000 7feef8400640 1 -- 172.21.15.44:0/1454337575 shutdown_connections 2024-09-13T08:37:35.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.423+0000 7feef8400640 1 -- 172.21.15.44:0/1454337575 wait complete. 2024-09-13T08:37:35.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 Processor -- start 2024-09-13T08:37:35.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 -- start start 2024-09-13T08:37:35.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed8004d50 0x7feef015e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feef015af50 0x7feef00b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feef00c4d40 0x7feef00b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feef00ae9b0 con 0x7feed8004d50 2024-09-13T08:37:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feef00d2af0 con 0x7feef00c4d40 2024-09-13T08:37:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7feef00aeb30 con 0x7feef015af50 2024-09-13T08:37:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed8004d50 0x7feef015e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed8004d50 0x7feef015e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60616/0 (socket says 172.21.15.44:60616) 2024-09-13T08:37:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef5a00640 1 -- 172.21.15.44:0/3822335643 learned_addr learned my addr 172.21.15.44:0/3822335643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef6400640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feef00c4d40 0x7feef00b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef5000640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feef015af50 0x7feef00b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef5a00640 1 -- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feef015af50 msgr2=0x7feef00b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef5a00640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feef015af50 0x7feef00b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef5a00640 1 -- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feef00c4d40 msgr2=0x7feef00b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef5a00640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feef00c4d40 0x7feef00b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef5a00640 1 -- 172.21.15.44:0/3822335643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef00b6df0 con 0x7feed8004d50 2024-09-13T08:37:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef5a00640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed8004d50 0x7feef015e120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feee4012610 tx=0x7feee4012ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feeed600640 1 -- 172.21.15.44:0/3822335643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4013a40 con 0x7feed8004d50 2024-09-13T08:37:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feeed600640 1 -- 172.21.15.44:0/3822335643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feee4017030 con 0x7feed8004d50 2024-09-13T08:37:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 -- 172.21.15.44:0/3822335643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef00b70c0 con 0x7feed8004d50 2024-09-13T08:37:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feeed600640 1 -- 172.21.15.44:0/3822335643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee401d430 con 0x7feed8004d50 2024-09-13T08:37:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 -- 172.21.15.44:0/3822335643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef015d920 con 0x7feed8004d50 2024-09-13T08:37:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.427+0000 7feef8400640 1 -- 172.21.15.44:0/3822335643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef00cfcf0 con 0x7feed8004d50 2024-09-13T08:37:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.431+0000 7feeed600640 1 -- 172.21.15.44:0/3822335643 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7feee40103e0 con 0x7feed8004d50 2024-09-13T08:37:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.431+0000 7feeed600640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feed0072ec0 0x7feed0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.431+0000 7feeed600640 1 -- 172.21.15.44:0/3822335643 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feee4097a20 con 0x7feed8004d50 2024-09-13T08:37:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.431+0000 7feef5000640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feed0072ec0 0x7feed0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.431+0000 7feef5000640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feed0072ec0 0x7feed0075380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feef00b7e40 tx=0x7feedc0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:35.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.435+0000 7feeed600640 1 -- 172.21.15.44:0/3822335643 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7feee4062d00 con 0x7feed8004d50 2024-09-13T08:37:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.611+0000 7feef8400640 1 -- 172.21.15.44:0/3822335643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feef01eac40 con 0x7feed8004d50 2024-09-13T08:37:35.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.611+0000 7feeed600640 1 -- 172.21.15.44:0/3822335643 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16694 (secure 0 0 0) 0x7feee4062600 con 0x7feed8004d50 2024-09-13T08:37:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:35.617 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":21,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:34.290410+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:37:32.247896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T08:37:33.241202+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:37:35.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 -- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feed0072ec0 msgr2=0x7feed0075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feed0072ec0 0x7feed0075380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feef00b7e40 tx=0x7feedc0091e0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 -- 172.21.15.44:0/3822335643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed8004d50 msgr2=0x7feef015e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed8004d50 0x7feef015e120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feee4012610 tx=0x7feee4012ad0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 -- 172.21.15.44:0/3822335643 shutdown_connections 2024-09-13T08:37:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feed0072ec0 0x7feed0075380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feef00c4d40 0x7feef00b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feef015af50 0x7feef00b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 --2- 172.21.15.44:0/3822335643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed8004d50 0x7feef015e120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 -- 172.21.15.44:0/3822335643 >> 172.21.15.44:0/3822335643 conn(0x7feef00d1dc0 msgr2=0x7feef0071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 -- 172.21.15.44:0/3822335643 shutdown_connections 2024-09-13T08:37:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.615+0000 7feef8400640 1 -- 172.21.15.44:0/3822335643 wait complete. 2024-09-13T08:37:35.639 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T08:37:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.783+0000 7f9bfea00640 1 -- 172.21.15.44:0/3422400149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bf81613c0 msgr2=0x7f9bf8161bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.783+0000 7f9bfea00640 1 --2- 172.21.15.44:0/3422400149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bf81613c0 0x7f9bf8161bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009900 tx=0x7f9be801c790 comp rx=0 tx=0).stop 2024-09-13T08:37:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.783+0000 7f9bfea00640 1 -- 172.21.15.44:0/3422400149 shutdown_connections 2024-09-13T08:37:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.783+0000 7f9bfea00640 1 --2- 172.21.15.44:0/3422400149 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9bf81617a0 0x7f9bf8162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.783+0000 7f9bfea00640 1 --2- 172.21.15.44:0/3422400149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bf81613c0 0x7f9bf8161bf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.783+0000 7f9bfea00640 1 -- 172.21.15.44:0/3422400149 >> 172.21.15.44:0/3422400149 conn(0x7f9bf80d1dd0 msgr2=0x7f9bf80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.783+0000 7f9bfea00640 1 -- 172.21.15.44:0/3422400149 shutdown_connections 2024-09-13T08:37:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.783+0000 7f9bfea00640 1 -- 172.21.15.44:0/3422400149 wait complete. 2024-09-13T08:37:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 Processor -- start 2024-09-13T08:37:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 -- start start 2024-09-13T08:37:35.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80b3190 0x7f9bf8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bf81613c0 0x7f9bf80b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9bf81617a0 0x7f9bf80b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf816b530 con 0x7f9bf80b3190 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf816b3b0 con 0x7f9bf81613c0 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf816b6b0 con 0x7f9bf81617a0 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80b3190 0x7f9bf8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80b3190 0x7f9bf8159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60618/0 (socket says 172.21.15.44:60618) 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf7e00640 1 -- 172.21.15.44:0/2669436939 learned_addr learned my addr 172.21.15.44:0/2669436939 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfca00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9bf81617a0 0x7f9bf80b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9befe00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bf81613c0 0x7f9bf80b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf7e00640 1 -- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9bf81617a0 msgr2=0x7f9bf80b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf7e00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9bf81617a0 0x7f9bf80b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf7e00640 1 -- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bf81613c0 msgr2=0x7f9bf80b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:35.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf7e00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bf81613c0 0x7f9bf80b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:35.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf7e00640 1 -- 172.21.15.44:0/2669436939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf815a4f0 con 0x7f9bf80b3190 2024-09-13T08:37:35.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf7e00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80b3190 0x7f9bf8159c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9be80b0040 tx=0x7f9be8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:35.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf4c00640 1 -- 172.21.15.44:0/2669436939 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be80b2040 con 0x7f9bf80b3190 2024-09-13T08:37:35.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf4c00640 1 -- 172.21.15.44:0/2669436939 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9be80a5bd0 con 0x7f9bf80b3190 2024-09-13T08:37:35.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bf4c00640 1 -- 172.21.15.44:0/2669436939 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8005030 con 0x7f9bf80b3190 2024-09-13T08:37:35.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 -- 172.21.15.44:0/2669436939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf815a7d0 con 0x7f9bf80b3190 2024-09-13T08:37:35.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 -- 172.21.15.44:0/2669436939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf815e720 con 0x7f9bf80b3190 2024-09-13T08:37:35.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.787+0000 7f9bfea00640 1 -- 172.21.15.44:0/2669436939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf806dd30 con 0x7f9bf80b3190 2024-09-13T08:37:35.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.791+0000 7f9bf4c00640 1 -- 172.21.15.44:0/2669436939 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f9be80051f0 con 0x7f9bf80b3190 2024-09-13T08:37:35.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.791+0000 7f9bf4c00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bd4072ec0 0x7f9bd4075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:35.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.791+0000 7f9bf4c00640 1 -- 172.21.15.44:0/2669436939 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9be812dcb0 con 0x7f9bf80b3190 2024-09-13T08:37:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.791+0000 7f9befe00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bd4072ec0 0x7f9bd4075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:35.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.791+0000 7f9befe00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bd4072ec0 0x7f9bd4075380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80cb6f0 tx=0x7f9be00091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:35.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:35.795+0000 7f9bf4c00640 1 -- 172.21.15.44:0/2669436939 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9be80164d0 con 0x7f9bf80b3190 2024-09-13T08:37:36.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.007+0000 7f9bfea00640 1 -- 172.21.15.44:0/2669436939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9bf8070a80 con 0x7f9bf80b3190 2024-09-13T08:37:36.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.887+0000 7f9bf4c00640 1 -- 172.21.15.44:0/2669436939 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f9be80f8e60 con 0x7f9bf80b3190 2024-09-13T08:37:36.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 -- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bd4072ec0 msgr2=0x7f9bd4075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bd4072ec0 0x7f9bd4075380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80cb6f0 tx=0x7f9be00091e0 comp rx=0 tx=0).stop 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 -- 172.21.15.44:0/2669436939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80b3190 msgr2=0x7f9bf8159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80b3190 0x7f9bf8159c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9be80b0040 tx=0x7f9be8005e70 comp rx=0 tx=0).stop 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 -- 172.21.15.44:0/2669436939 shutdown_connections 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bd4072ec0 0x7f9bd4075380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9bf81617a0 0x7f9bf80b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bf81613c0 0x7f9bf80b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 --2- 172.21.15.44:0/2669436939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80b3190 0x7f9bf8159c60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 -- 172.21.15.44:0/2669436939 >> 172.21.15.44:0/2669436939 conn(0x7f9bf80d1dd0 msgr2=0x7f9bf80aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 -- 172.21.15.44:0/2669436939 shutdown_connections 2024-09-13T08:37:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:36.891+0000 7f9bfea00640 1 -- 172.21.15.44:0/2669436939 wait complete. 2024-09-13T08:37:36.919 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T08:37:36.919 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T08:37:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 -- 172.21.15.44:0/2701984789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834003a60 msgr2=0x7fa834003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 --2- 172.21.15.44:0/2701984789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834003a60 0x7fa834003e80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa84000bb10 tx=0x7fa84001cda0 comp rx=0 tx=0).stop 2024-09-13T08:37:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa84fe00640 1 -- 172.21.15.44:0/2701984789 reap_dead start 2024-09-13T08:37:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 -- 172.21.15.44:0/2701984789 shutdown_connections 2024-09-13T08:37:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 -- 172.21.15.44:0/2701984789 >> 172.21.15.44:0/2701984789 conn(0x7fa8500b18f0 msgr2=0x7fa8500b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 -- 172.21.15.44:0/2701984789 shutdown_connections 2024-09-13T08:37:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 -- 172.21.15.44:0/2701984789 wait complete. 2024-09-13T08:37:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 Processor -- start 2024-09-13T08:37:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 -- start start 2024-09-13T08:37:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834003a60 0x7fa8501f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8500b2cf0 0x7fa8500ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8500b3880 0x7fa8500add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa85015c940 con 0x7fa834003a60 2024-09-13T08:37:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa85015c7c0 con 0x7fa8500b3880 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa856800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa85015cac0 con 0x7fa8500b2cf0 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa84f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834003a60 0x7fa8501f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa84f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834003a60 0x7fa8501f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60648/0 (socket says 172.21.15.44:60648) 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa84f400640 1 -- 172.21.15.44:0/3751149985 learned_addr learned my addr 172.21.15.44:0/3751149985 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa84fe00640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8500b3880 0x7fa8500add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa847e00640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8500b2cf0 0x7fa8500ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa84f400640 1 -- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8500b2cf0 msgr2=0x7fa8500ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa84f400640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8500b2cf0 0x7fa8500ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa84f400640 1 -- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8500b3880 msgr2=0x7fa8500add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa84f400640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8500b3880 0x7fa8500add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.071+0000 7fa84f400640 1 -- 172.21.15.44:0/3751149985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8501f8cb0 con 0x7fa834003a60 2024-09-13T08:37:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.075+0000 7fa84f400640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834003a60 0x7fa8501f86d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa84001cbd0 tx=0x7fa840005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.075+0000 7fa847400640 1 -- 172.21.15.44:0/3751149985 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8400ab040 con 0x7fa834003a60 2024-09-13T08:37:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.075+0000 7fa847400640 1 -- 172.21.15.44:0/3751149985 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8400be070 con 0x7fa834003a60 2024-09-13T08:37:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.075+0000 7fa847400640 1 -- 172.21.15.44:0/3751149985 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840004b20 con 0x7fa834003a60 2024-09-13T08:37:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.075+0000 7fa856800640 1 -- 172.21.15.44:0/3751149985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8501f8f90 con 0x7fa834003a60 2024-09-13T08:37:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.075+0000 7fa856800640 1 -- 172.21.15.44:0/3751149985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8500b4180 con 0x7fa834003a60 2024-09-13T08:37:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.075+0000 7fa856800640 1 -- 172.21.15.44:0/3751149985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa850069410 con 0x7fa834003a60 2024-09-13T08:37:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.075+0000 7fa847400640 1 -- 172.21.15.44:0/3751149985 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fa8400b6070 con 0x7fa834003a60 2024-09-13T08:37:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.075+0000 7fa847400640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa82c072ef0 0x7fa82c0753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.075+0000 7fa847400640 1 -- 172.21.15.44:0/3751149985 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa84012eae0 con 0x7fa834003a60 2024-09-13T08:37:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.079+0000 7fa847e00640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa82c072ef0 0x7fa82c0753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:37.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.079+0000 7fa847e00640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa82c072ef0 0x7fa82c0753b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8500cbed0 tx=0x7fa838005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:37.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.083+0000 7fa847400640 1 -- 172.21.15.44:0/3751149985 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa840016ab0 con 0x7fa834003a60 2024-09-13T08:37:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.291+0000 7fa856800640 1 -- 172.21.15.44:0/3751149985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa8500b4710 con 0x7fa834003a60 2024-09-13T08:37:37.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa847400640 1 -- 172.21.15.44:0/3751149985 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fa8400f9c00 con 0x7fa834003a60 2024-09-13T08:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 -- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa82c072ef0 msgr2=0x7fa82c0753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa82c072ef0 0x7fa82c0753b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8500cbed0 tx=0x7fa838005f90 comp rx=0 tx=0).stop 2024-09-13T08:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 -- 172.21.15.44:0/3751149985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834003a60 msgr2=0x7fa8501f86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834003a60 0x7fa8501f86d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa84001cbd0 tx=0x7fa840005620 comp rx=0 tx=0).stop 2024-09-13T08:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 -- 172.21.15.44:0/3751149985 shutdown_connections 2024-09-13T08:37:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa82c072ef0 0x7fa82c0753b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:37.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8500b3880 0x7fa8500add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:37.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8500b2cf0 0x7fa8500ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:37.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 --2- 172.21.15.44:0/3751149985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834003a60 0x7fa8501f86d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:37.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 -- 172.21.15.44:0/3751149985 >> 172.21.15.44:0/3751149985 conn(0x7fa8500b18f0 msgr2=0x7fa8500b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:37.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 -- 172.21.15.44:0/3751149985 shutdown_connections 2024-09-13T08:37:37.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:37.895+0000 7fa856800640 1 -- 172.21.15.44:0/3751149985 wait complete. 2024-09-13T08:37:37.924 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T08:37:37.924 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:37:38.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.071+0000 7f55e4200640 1 -- 172.21.15.44:0/2631209598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55dc15dae0 msgr2=0x7f55dc0acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.071+0000 7f55e4200640 1 --2- 172.21.15.44:0/2631209598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55dc15dae0 0x7f55dc0acd40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55dc070d30 tx=0x7f55d001c6e0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.071+0000 7f55e4200640 1 -- 172.21.15.44:0/2631209598 shutdown_connections 2024-09-13T08:37:38.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.071+0000 7f55e4200640 1 --2- 172.21.15.44:0/2631209598 >> v2:172.21.15.129:3300/0 conn(0x7f55dc0b49e0 0x7f55dc06f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.071+0000 7f55e4200640 1 --2- 172.21.15.44:0/2631209598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55dc15dae0 0x7f55dc0acd40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.071+0000 7f55e4200640 1 -- 172.21.15.44:0/2631209598 >> 172.21.15.44:0/2631209598 conn(0x7f55dc0b18f0 msgr2=0x7f55dc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:38.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.071+0000 7f55e4200640 1 -- 172.21.15.44:0/2631209598 shutdown_connections 2024-09-13T08:37:38.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.071+0000 7f55e4200640 1 -- 172.21.15.44:0/2631209598 wait complete. 2024-09-13T08:37:38.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.071+0000 7f55e4200640 1 Processor -- start 2024-09-13T08:37:38.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.071+0000 7f55e4200640 1 -- start start 2024-09-13T08:37:38.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e4200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55dc0b49e0 0x7f55dc1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e4200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55dc0cd100 0x7f55dc1fad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e4200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55dc15dae0 0x7f55dc14d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e4200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc0cf190 con 0x7f55dc15dae0 2024-09-13T08:37:38.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e4200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc0cf010 con 0x7f55dc0b49e0 2024-09-13T08:37:38.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e4200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f55dc0cf310 con 0x7f55dc0cd100 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e2200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55dc15dae0 0x7f55dc14d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e2200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55dc15dae0 0x7f55dc14d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60656/0 (socket says 172.21.15.44:60656) 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e2200640 1 -- 172.21.15.44:0/3228528278 learned_addr learned my addr 172.21.15.44:0/3228528278 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e1800640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55dc0b49e0 0x7f55dc1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e0e00640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55dc0cd100 0x7f55dc1fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e2200640 1 -- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55dc0cd100 msgr2=0x7f55dc1fad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e2200640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55dc0cd100 0x7f55dc1fad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e2200640 1 -- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55dc0b49e0 msgr2=0x7f55dc1564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e2200640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55dc0b49e0 0x7f55dc1564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e2200640 1 -- 172.21.15.44:0/3228528278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc14d580 con 0x7f55dc15dae0 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e2200640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55dc15dae0 0x7f55dc14d040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55d400ec60 tx=0x7f55d400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:38.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55d9600640 1 -- 172.21.15.44:0/3228528278 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4004d30 con 0x7f55dc15dae0 2024-09-13T08:37:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55d9600640 1 -- 172.21.15.44:0/3228528278 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55d400b570 con 0x7f55dc15dae0 2024-09-13T08:37:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55d9600640 1 -- 172.21.15.44:0/3228528278 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d400b980 con 0x7f55dc15dae0 2024-09-13T08:37:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e4200640 1 -- 172.21.15.44:0/3228528278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc1f7760 con 0x7f55dc15dae0 2024-09-13T08:37:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e4200640 1 -- 172.21.15.44:0/3228528278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1fb7a0 con 0x7f55dc15dae0 2024-09-13T08:37:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55e4200640 1 -- 172.21.15.44:0/3228528278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55dc069410 con 0x7f55dc15dae0 2024-09-13T08:37:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.075+0000 7f55d9600640 1 -- 172.21.15.44:0/3228528278 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f55d400bb20 con 0x7f55dc15dae0 2024-09-13T08:37:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.079+0000 7f55d9600640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55bc072ec0 0x7f55bc075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.079+0000 7f55d9600640 1 -- 172.21.15.44:0/3228528278 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f55d40949d0 con 0x7f55dc15dae0 2024-09-13T08:37:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.079+0000 7f55e1800640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55bc072ec0 0x7f55bc075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.079+0000 7f55e1800640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55bc072ec0 0x7f55bc075380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f55d00af000 tx=0x7f55d001af20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:38.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.083+0000 7f55d9600640 1 -- 172.21.15.44:0/3228528278 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f55dc14d580 con 0x7f55dc15dae0 2024-09-13T08:37:38.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 -- 172.21.15.44:0/3228528278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55dc0acd40 con 0x7f55dc15dae0 2024-09-13T08:37:38.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55d9600640 1 -- 172.21.15.44:0/3228528278 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f55d405fc10 con 0x7f55dc15dae0 2024-09-13T08:37:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:38.298 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"btime":"2024-09-13T08:37:37:896662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2931832597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2931832597},{"type":"v1","addr":"172.21.15.129:6837","nonce":2931832597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4335,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/853789824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":853789824},{"type":"v1","addr":"172.21.15.44:6834","nonce":853789824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/686971411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":686971411},{"type":"v1","addr":"172.21.15.44:6835","nonce":686971411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:37:34.290379+0000","modified":"2024-09-13T08:37:37.298049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4360":{"gid":4360,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6835/1614710769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1614710769},{"type":"v1","addr":"172.21.15.129:6835","nonce":1614710769}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4360,"qdb_cluster":[4360]},"id":1}]} 2024-09-13T08:37:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-09-13T08:37:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 -- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55bc072ec0 msgr2=0x7f55bc075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55bc072ec0 0x7f55bc075380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f55d00af000 tx=0x7f55d001af20 comp rx=0 tx=0).stop 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 -- 172.21.15.44:0/3228528278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55dc15dae0 msgr2=0x7f55dc14d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55dc15dae0 0x7f55dc14d040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55d400ec60 tx=0x7f55d400c7a0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 -- 172.21.15.44:0/3228528278 shutdown_connections 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55bc072ec0 0x7f55bc075380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55dc15dae0 0x7f55dc14d040 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55dc0cd100 0x7f55dc1fad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 --2- 172.21.15.44:0/3228528278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55dc0b49e0 0x7f55dc1564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 -- 172.21.15.44:0/3228528278 >> 172.21.15.44:0/3228528278 conn(0x7f55dc0b18f0 msgr2=0x7f55dc0b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 -- 172.21.15.44:0/3228528278 shutdown_connections 2024-09-13T08:37:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.295+0000 7f55e4200640 1 -- 172.21.15.44:0/3228528278 wait complete. 2024-09-13T08:37:38.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:37:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.459+0000 7fde4f600640 1 -- 172.21.15.44:0/2321640206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde480ced80 msgr2=0x7fde480ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.459+0000 7fde4f600640 1 --2- 172.21.15.44:0/2321640206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde480ced80 0x7fde480ae9b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde30009900 tx=0x7fde3001c8e0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.459+0000 7fde4f600640 1 -- 172.21.15.44:0/2321640206 shutdown_connections 2024-09-13T08:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.459+0000 7fde4f600640 1 --2- 172.21.15.44:0/2321640206 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde480cdb30 0x7fde480aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.459+0000 7fde4f600640 1 --2- 172.21.15.44:0/2321640206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde480ced80 0x7fde480ae9b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.459+0000 7fde4f600640 1 -- 172.21.15.44:0/2321640206 >> 172.21.15.44:0/2321640206 conn(0x7fde480d1dc0 msgr2=0x7fde480d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.459+0000 7fde4f600640 1 -- 172.21.15.44:0/2321640206 shutdown_connections 2024-09-13T08:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.459+0000 7fde4f600640 1 -- 172.21.15.44:0/2321640206 wait complete. 2024-09-13T08:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4f600640 1 Processor -- start 2024-09-13T08:37:38.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4f600640 1 -- start start 2024-09-13T08:37:38.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4f600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde480cdb30 0x7fde4815a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4f600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde480ced80 0x7fde48156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4f600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde4815bea0 0x7fde48156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4f600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fde48169340 con 0x7fde480cdb30 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4f600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fde481691c0 con 0x7fde4815bea0 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4f600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fde481694c0 con 0x7fde480ced80 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde480cdb30 0x7fde4815a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde480cdb30 0x7fde4815a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60672/0 (socket says 172.21.15.44:60672) 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4cc00640 1 -- 172.21.15.44:0/2881797184 learned_addr learned my addr 172.21.15.44:0/2881797184 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde3fe00640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde480ced80 0x7fde48156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4d600640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde4815bea0 0x7fde48156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4cc00640 1 -- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde480ced80 msgr2=0x7fde48156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4cc00640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde480ced80 0x7fde48156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4cc00640 1 -- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde4815bea0 msgr2=0x7fde48156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4cc00640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde4815bea0 0x7fde48156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4cc00640 1 -- 172.21.15.44:0/2881797184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde480b7c70 con 0x7fde480cdb30 2024-09-13T08:37:38.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4cc00640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde480cdb30 0x7fde4815a3a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c3f0 tx=0x7fde3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:38.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde45600640 1 -- 172.21.15.44:0/2881797184 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38017070 con 0x7fde480cdb30 2024-09-13T08:37:38.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde45600640 1 -- 172.21.15.44:0/2881797184 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde380053c0 con 0x7fde480cdb30 2024-09-13T08:37:38.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4f600640 1 -- 172.21.15.44:0/2881797184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde480cb4f0 con 0x7fde480cdb30 2024-09-13T08:37:38.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde45600640 1 -- 172.21.15.44:0/2881797184 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38003860 con 0x7fde480cdb30 2024-09-13T08:37:38.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.463+0000 7fde4f600640 1 -- 172.21.15.44:0/2881797184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde480cba70 con 0x7fde480cdb30 2024-09-13T08:37:38.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.467+0000 7fde4f600640 1 -- 172.21.15.44:0/2881797184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde480cbdb0 con 0x7fde480cdb30 2024-09-13T08:37:38.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.467+0000 7fde45600640 1 -- 172.21.15.44:0/2881797184 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fde3801a070 con 0x7fde480cdb30 2024-09-13T08:37:38.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.467+0000 7fde45600640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde24072ec0 0x7fde24075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.467+0000 7fde45600640 1 -- 172.21.15.44:0/2881797184 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fde38093150 con 0x7fde480cdb30 2024-09-13T08:37:38.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.467+0000 7fde3fe00640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde24072ec0 0x7fde24075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.467+0000 7fde3fe00640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde24072ec0 0x7fde24075380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde300af000 tx=0x7fde3001b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:38.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.471+0000 7fde45600640 1 -- 172.21.15.44:0/2881797184 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fde480cbdb0 con 0x7fde480cdb30 2024-09-13T08:37:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.647+0000 7fde4f600640 1 -- 172.21.15.44:0/2881797184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde480ce670 con 0x7fde480cdb30 2024-09-13T08:37:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde45600640 1 -- 172.21.15.44:0/2881797184 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16681 (secure 0 0 0) 0x7fde3805dcb0 con 0x7fde480cdb30 2024-09-13T08:37:38.654 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:38.655 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":22,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:35.883263+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:37:32.247896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T08:37:33.241202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 -- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde24072ec0 msgr2=0x7fde24075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde24072ec0 0x7fde24075380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde300af000 tx=0x7fde3001b120 comp rx=0 tx=0).stop 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 -- 172.21.15.44:0/2881797184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde480cdb30 msgr2=0x7fde4815a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde480cdb30 0x7fde4815a3a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c3f0 tx=0x7fde3800c8b0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 -- 172.21.15.44:0/2881797184 shutdown_connections 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde24072ec0 0x7fde24075380 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde4815bea0 0x7fde48156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde480ced80 0x7fde48156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 --2- 172.21.15.44:0/2881797184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde480cdb30 0x7fde4815a3a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 -- 172.21.15.44:0/2881797184 >> 172.21.15.44:0/2881797184 conn(0x7fde480d1dc0 msgr2=0x7fde4806f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 -- 172.21.15.44:0/2881797184 shutdown_connections 2024-09-13T08:37:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.651+0000 7fde4f600640 1 -- 172.21.15.44:0/2881797184 wait complete. 2024-09-13T08:37:38.677 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:37:38.677 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:37:38.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.819+0000 7f428d000640 1 -- 172.21.15.44:0/1363331742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f426c003c90 msgr2=0x7f426c004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.819+0000 7f428d000640 1 --2- 172.21.15.44:0/1363331742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f426c003c90 0x7f426c004090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4278006f50 tx=0x7f427801cec0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.819+0000 7f428d000640 1 -- 172.21.15.44:0/1363331742 shutdown_connections 2024-09-13T08:37:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.819+0000 7f428d000640 1 --2- 172.21.15.44:0/1363331742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f426c003c90 0x7f426c004090 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.819+0000 7f428d000640 1 -- 172.21.15.44:0/1363331742 >> 172.21.15.44:0/1363331742 conn(0x7f42880d1dc0 msgr2=0x7f42880d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.819+0000 7f428d000640 1 -- 172.21.15.44:0/1363331742 shutdown_connections 2024-09-13T08:37:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.819+0000 7f428d000640 1 -- 172.21.15.44:0/1363331742 wait complete. 2024-09-13T08:37:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.819+0000 7f428d000640 1 Processor -- start 2024-09-13T08:37:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.819+0000 7f428d000640 1 -- start start 2024-09-13T08:37:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f428d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f426c003c90 0x7f42880c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f428d000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f428815b1b0 0x7f42880c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f428d000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428815a820 0x7f428815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f428d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42880cd570 con 0x7f426c003c90 2024-09-13T08:37:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f428d000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f42880cd3f0 con 0x7f428815a820 2024-09-13T08:37:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f428d000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f42880cd6f0 con 0x7f428815b1b0 2024-09-13T08:37:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4285e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f426c003c90 0x7f42880c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4285e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f426c003c90 0x7f42880c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60682/0 (socket says 172.21.15.44:60682) 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4285e00640 1 -- 172.21.15.44:0/1897062703 learned_addr learned my addr 172.21.15.44:0/1897062703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4285e00640 1 -- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f428815b1b0 msgr2=0x7f42880c5590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4285e00640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f428815b1b0 0x7f42880c5590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4286800640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428815a820 0x7f428815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4285e00640 1 -- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428815a820 msgr2=0x7f428815e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4285e00640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428815a820 0x7f428815e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4285e00640 1 -- 172.21.15.44:0/1897062703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42880c8130 con 0x7f426c003c90 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4286800640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428815a820 0x7f428815e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f4285e00640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f426c003c90 0x7f42880c88c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4278098980 tx=0x7f4278006600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f427e000640 1 -- 172.21.15.44:0/1897062703 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278006b70 con 0x7f426c003c90 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f427e000640 1 -- 172.21.15.44:0/1897062703 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4278001500 con 0x7f426c003c90 2024-09-13T08:37:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f427e000640 1 -- 172.21.15.44:0/1897062703 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278003860 con 0x7f426c003c90 2024-09-13T08:37:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f428d000640 1 -- 172.21.15.44:0/1897062703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42880ca450 con 0x7f426c003c90 2024-09-13T08:37:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f428d000640 1 -- 172.21.15.44:0/1897062703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42880c83a0 con 0x7f426c003c90 2024-09-13T08:37:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.823+0000 7f428d000640 1 -- 172.21.15.44:0/1897062703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42880aeca0 con 0x7f426c003c90 2024-09-13T08:37:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.827+0000 7f427e000640 1 -- 172.21.15.44:0/1897062703 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f4278003a00 con 0x7f426c003c90 2024-09-13T08:37:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.827+0000 7f427e000640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4264072ec0 0x7f4264075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.827+0000 7f427e000640 1 -- 172.21.15.44:0/1897062703 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f427812d870 con 0x7f426c003c90 2024-09-13T08:37:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.827+0000 7f427fe00640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4264072ec0 0x7f4264075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.827+0000 7f427fe00640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4264072ec0 0x7f4264075380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f428815f6c0 tx=0x7f4270006ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:38.831+0000 7f427e000640 1 -- 172.21.15.44:0/1897062703 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f42780bf030 con 0x7f426c003c90 2024-09-13T08:37:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.003+0000 7f428d000640 1 -- 172.21.15.44:0/1897062703 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42880b5200 con 0x7f4264072ec0 2024-09-13T08:37:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.011+0000 7f427e000640 1 -- 172.21.15.44:0/1897062703 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247213 (secure 0 0 0) 0x7f42880b5200 con 0x7f4264072ec0 2024-09-13T08:37:39.017 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:39.019 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:37:39.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 -- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4264072ec0 msgr2=0x7f4264075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4264072ec0 0x7f4264075380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f428815f6c0 tx=0x7f4270006ca0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 -- 172.21.15.44:0/1897062703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f426c003c90 msgr2=0x7f42880c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f426c003c90 0x7f42880c88c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4278098980 tx=0x7f4278006600 comp rx=0 tx=0).stop 2024-09-13T08:37:39.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 -- 172.21.15.44:0/1897062703 shutdown_connections 2024-09-13T08:37:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4264072ec0 0x7f4264075380 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428815a820 0x7f428815e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f428815b1b0 0x7f42880c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 --2- 172.21.15.44:0/1897062703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f426c003c90 0x7f42880c88c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 -- 172.21.15.44:0/1897062703 >> 172.21.15.44:0/1897062703 conn(0x7f42880d1dc0 msgr2=0x7f42880b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 -- 172.21.15.44:0/1897062703 shutdown_connections 2024-09-13T08:37:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.015+0000 7f428d000640 1 -- 172.21.15.44:0/1897062703 wait complete. 2024-09-13T08:37:39.037 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-13T08:37:38.511365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218896,"kb_used_data":2660,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755824,"statfs":{"total":773094113280,"available":772869963776,"internally_reserved":0,"allocated":2723840,"data_stored":1668108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.654417"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.888188+0000","last_change":"2024-09-13T08:37:33.888188+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.888188+0000","last_undegraded":"2024-09-13T08:37:33.888188+0000","last_fullsized":"2024-09-13T08:37:33.888188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876585+0000","last_change":"2024-09-13T08:37:33.876585+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876585+0000","last_undegraded":"2024-09-13T08:37:33.876585+0000","last_fullsized":"2024-09-13T08:37:33.876585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873515+0000","last_change":"2024-09-13T08:37:33.873515+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873515+0000","last_undegraded":"2024-09-13T08:37:33.873515+0000","last_fullsized":"2024-09-13T08:37:33.873515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870231+0000","last_change":"2024-09-13T08:37:33.870231+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870231+0000","last_undegraded":"2024-09-13T08:37:33.870231+0000","last_fullsized":"2024-09-13T08:37:33.870231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.887043+0000","last_change":"2024-09-13T08:37:33.887043+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.887043+0000","last_undegraded":"2024-09-13T08:37:33.887043+0000","last_fullsized":"2024-09-13T08:37:33.887043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.869808+0000","last_change":"2024-09-13T08:37:33.869808+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.869808+0000","last_undegraded":"2024-09-13T08:37:33.869808+0000","last_fullsized":"2024-09-13T08:37:33.869808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.871829+0000","last_change":"2024-09-13T08:37:33.871829+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.871829+0000","last_undegraded":"2024-09-13T08:37:33.871829+0000","last_fullsized":"2024-09-13T08:37:33.871829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875627+0000","last_change":"2024-09-13T08:37:33.875627+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875627+0000","last_undegraded":"2024-09-13T08:37:33.875627+0000","last_fullsized":"2024-09-13T08:37:33.875627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874425+0000","last_change":"2024-09-13T08:37:33.874425+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874425+0000","last_undegraded":"2024-09-13T08:37:33.874425+0000","last_fullsized":"2024-09-13T08:37:33.874425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874047+0000","last_change":"2024-09-13T08:37:33.874047+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874047+0000","last_undegraded":"2024-09-13T08:37:33.874047+0000","last_fullsized":"2024-09-13T08:37:33.874047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.881267+0000","last_change":"2024-09-13T08:37:33.881267+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.881267+0000","last_undegraded":"2024-09-13T08:37:33.881267+0000","last_fullsized":"2024-09-13T08:37:33.881267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.877798+0000","last_change":"2024-09-13T08:37:33.877798+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.877798+0000","last_undegraded":"2024-09-13T08:37:33.877798+0000","last_fullsized":"2024-09-13T08:37:33.877798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894980+0000","last_change":"2024-09-13T08:37:34.903037+0000","last_active":"2024-09-13T08:37:35.894980+0000","last_peered":"2024-09-13T08:37:35.894980+0000","last_clean":"2024-09-13T08:37:35.894980+0000","last_became_active":"2024-09-13T08:37:34.894805+0000","last_became_peered":"2024-09-13T08:37:34.894805+0000","last_unstale":"2024-09-13T08:37:35.894980+0000","last_undegraded":"2024-09-13T08:37:35.894980+0000","last_fullsized":"2024-09-13T08:37:35.894980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:49.712241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894573+0000","last_change":"2024-09-13T08:37:34.901603+0000","last_active":"2024-09-13T08:37:35.894573+0000","last_peered":"2024-09-13T08:37:35.894573+0000","last_clean":"2024-09-13T08:37:35.894573+0000","last_became_active":"2024-09-13T08:37:34.899884+0000","last_became_peered":"2024-09-13T08:37:34.899884+0000","last_unstale":"2024-09-13T08:37:35.894573+0000","last_undegraded":"2024-09-13T08:37:35.894573+0000","last_fullsized":"2024-09-13T08:37:35.894573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:05.260600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874667+0000","last_change":"2024-09-13T08:37:33.874667+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874667+0000","last_undegraded":"2024-09-13T08:37:33.874667+0000","last_fullsized":"2024-09-13T08:37:33.874667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876422+0000","last_change":"2024-09-13T08:37:33.876422+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876422+0000","last_undegraded":"2024-09-13T08:37:33.876422+0000","last_fullsized":"2024-09-13T08:37:33.876422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874801+0000","last_change":"2024-09-13T08:37:33.874801+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874801+0000","last_undegraded":"2024-09-13T08:37:33.874801+0000","last_fullsized":"2024-09-13T08:37:33.874801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.878051+0000","last_change":"2024-09-13T08:37:33.878051+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.878051+0000","last_undegraded":"2024-09-13T08:37:33.878051+0000","last_fullsized":"2024-09-13T08:37:33.878051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875550+0000","last_change":"2024-09-13T08:37:33.875550+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875550+0000","last_undegraded":"2024-09-13T08:37:33.875550+0000","last_fullsized":"2024-09-13T08:37:33.875550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875619+0000","last_change":"2024-09-13T08:37:33.875619+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875619+0000","last_undegraded":"2024-09-13T08:37:33.875619+0000","last_fullsized":"2024-09-13T08:37:33.875619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.881053+0000","last_change":"2024-09-13T08:37:33.881053+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.881053+0000","last_undegraded":"2024-09-13T08:37:33.881053+0000","last_fullsized":"2024-09-13T08:37:33.881053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875830+0000","last_change":"2024-09-13T08:37:33.875830+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875830+0000","last_undegraded":"2024-09-13T08:37:33.875830+0000","last_fullsized":"2024-09-13T08:37:33.875830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873389+0000","last_change":"2024-09-13T08:37:33.873389+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873389+0000","last_undegraded":"2024-09-13T08:37:33.873389+0000","last_fullsized":"2024-09-13T08:37:33.873389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875855+0000","last_change":"2024-09-13T08:37:33.875855+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875855+0000","last_undegraded":"2024-09-13T08:37:33.875855+0000","last_fullsized":"2024-09-13T08:37:33.875855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.877601+0000","last_change":"2024-09-13T08:37:33.877601+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.877601+0000","last_undegraded":"2024-09-13T08:37:33.877601+0000","last_fullsized":"2024-09-13T08:37:33.877601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.878497+0000","last_change":"2024-09-13T08:37:33.878497+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.878497+0000","last_undegraded":"2024-09-13T08:37:33.878497+0000","last_fullsized":"2024-09-13T08:37:33.878497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895541+0000","last_change":"2024-09-13T08:37:34.902546+0000","last_active":"2024-09-13T08:37:35.895541+0000","last_peered":"2024-09-13T08:37:35.895541+0000","last_clean":"2024-09-13T08:37:35.895541+0000","last_became_active":"2024-09-13T08:37:34.901841+0000","last_became_peered":"2024-09-13T08:37:34.901841+0000","last_unstale":"2024-09-13T08:37:35.895541+0000","last_undegraded":"2024-09-13T08:37:35.895541+0000","last_fullsized":"2024-09-13T08:37:35.895541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:40.635073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895078+0000","last_change":"2024-09-13T08:37:34.907277+0000","last_active":"2024-09-13T08:37:35.895078+0000","last_peered":"2024-09-13T08:37:35.895078+0000","last_clean":"2024-09-13T08:37:35.895078+0000","last_became_active":"2024-09-13T08:37:34.906984+0000","last_became_peered":"2024-09-13T08:37:34.906984+0000","last_unstale":"2024-09-13T08:37:35.895078+0000","last_undegraded":"2024-09-13T08:37:35.895078+0000","last_fullsized":"2024-09-13T08:37:35.895078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:23.664559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872673+0000","last_change":"2024-09-13T08:37:33.872673+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872673+0000","last_undegraded":"2024-09-13T08:37:33.872673+0000","last_fullsized":"2024-09-13T08:37:33.872673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874060+0000","last_change":"2024-09-13T08:37:33.874060+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874060+0000","last_undegraded":"2024-09-13T08:37:33.874060+0000","last_fullsized":"2024-09-13T08:37:33.874060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874453+0000","last_change":"2024-09-13T08:37:33.874453+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874453+0000","last_undegraded":"2024-09-13T08:37:33.874453+0000","last_fullsized":"2024-09-13T08:37:33.874453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872506+0000","last_change":"2024-09-13T08:37:33.872506+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872506+0000","last_undegraded":"2024-09-13T08:37:33.872506+0000","last_fullsized":"2024-09-13T08:37:33.872506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875197+0000","last_change":"2024-09-13T08:37:33.875197+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875197+0000","last_undegraded":"2024-09-13T08:37:33.875197+0000","last_fullsized":"2024-09-13T08:37:33.875197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.892711+0000","last_change":"2024-09-13T08:37:33.888852+0000","last_active":"2024-09-13T08:37:33.892711+0000","last_peered":"2024-09-13T08:37:33.892711+0000","last_clean":"2024-09-13T08:37:33.892711+0000","last_became_active":"2024-09-13T08:37:33.888409+0000","last_became_peered":"2024-09-13T08:37:33.888409+0000","last_unstale":"2024-09-13T08:37:33.892711+0000","last_undegraded":"2024-09-13T08:37:33.892711+0000","last_fullsized":"2024-09-13T08:37:33.892711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872451+0000","last_change":"2024-09-13T08:37:33.872451+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872451+0000","last_undegraded":"2024-09-13T08:37:33.872451+0000","last_fullsized":"2024-09-13T08:37:33.872451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.895745+0000","last_change":"2024-09-13T08:37:33.892308+0000","last_active":"2024-09-13T08:37:33.895745+0000","last_peered":"2024-09-13T08:37:33.895745+0000","last_clean":"2024-09-13T08:37:33.895745+0000","last_became_active":"2024-09-13T08:37:33.891875+0000","last_became_peered":"2024-09-13T08:37:33.891875+0000","last_unstale":"2024-09-13T08:37:33.895745+0000","last_undegraded":"2024-09-13T08:37:33.895745+0000","last_fullsized":"2024-09-13T08:37:33.895745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:22.150995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894251+0000","last_change":"2024-09-13T08:37:33.896561+0000","last_active":"2024-09-13T08:37:35.894251+0000","last_peered":"2024-09-13T08:37:35.894251+0000","last_clean":"2024-09-13T08:37:35.894251+0000","last_became_active":"2024-09-13T08:37:33.896314+0000","last_became_peered":"2024-09-13T08:37:33.896314+0000","last_unstale":"2024-09-13T08:37:35.894251+0000","last_undegraded":"2024-09-13T08:37:35.894251+0000","last_fullsized":"2024-09-13T08:37:35.894251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:28.728254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872534+0000","last_change":"2024-09-13T08:37:33.872534+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872534+0000","last_undegraded":"2024-09-13T08:37:33.872534+0000","last_fullsized":"2024-09-13T08:37:33.872534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.889489+0000","last_change":"2024-09-13T08:37:33.883156+0000","last_active":"2024-09-13T08:37:33.889489+0000","last_peered":"2024-09-13T08:37:33.889489+0000","last_clean":"2024-09-13T08:37:33.889489+0000","last_became_active":"2024-09-13T08:37:33.882800+0000","last_became_peered":"2024-09-13T08:37:33.882800+0000","last_unstale":"2024-09-13T08:37:33.889489+0000","last_undegraded":"2024-09-13T08:37:33.889489+0000","last_fullsized":"2024-09-13T08:37:33.889489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.879388+0000","last_change":"2024-09-13T08:37:33.879388+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.879388+0000","last_undegraded":"2024-09-13T08:37:33.879388+0000","last_fullsized":"2024-09-13T08:37:33.879388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894588+0000","last_change":"2024-09-13T08:37:33.891385+0000","last_active":"2024-09-13T08:37:33.894588+0000","last_peered":"2024-09-13T08:37:33.894588+0000","last_clean":"2024-09-13T08:37:33.894588+0000","last_became_active":"2024-09-13T08:37:33.891112+0000","last_became_peered":"2024-09-13T08:37:33.891112+0000","last_unstale":"2024-09-13T08:37:33.894588+0000","last_undegraded":"2024-09-13T08:37:33.894588+0000","last_fullsized":"2024-09-13T08:37:33.894588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870875+0000","last_change":"2024-09-13T08:37:33.870875+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870875+0000","last_undegraded":"2024-09-13T08:37:33.870875+0000","last_fullsized":"2024-09-13T08:37:33.870875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.904015+0000","last_change":"2024-09-13T08:37:33.899147+0000","last_active":"2024-09-13T08:37:33.904015+0000","last_peered":"2024-09-13T08:37:33.904015+0000","last_clean":"2024-09-13T08:37:33.904015+0000","last_became_active":"2024-09-13T08:37:33.896929+0000","last_became_peered":"2024-09-13T08:37:33.896929+0000","last_unstale":"2024-09-13T08:37:33.904015+0000","last_undegraded":"2024-09-13T08:37:33.904015+0000","last_fullsized":"2024-09-13T08:37:33.904015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876422+0000","last_change":"2024-09-13T08:37:33.876422+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876422+0000","last_undegraded":"2024-09-13T08:37:33.876422+0000","last_fullsized":"2024-09-13T08:37:33.876422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900065+0000","last_change":"2024-09-13T08:37:33.892872+0000","last_active":"2024-09-13T08:37:33.900065+0000","last_peered":"2024-09-13T08:37:33.900065+0000","last_clean":"2024-09-13T08:37:33.900065+0000","last_became_active":"2024-09-13T08:37:33.892579+0000","last_became_peered":"2024-09-13T08:37:33.892579+0000","last_unstale":"2024-09-13T08:37:33.900065+0000","last_undegraded":"2024-09-13T08:37:33.900065+0000","last_fullsized":"2024-09-13T08:37:33.900065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874425+0000","last_change":"2024-09-13T08:37:33.874425+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874425+0000","last_undegraded":"2024-09-13T08:37:33.874425+0000","last_fullsized":"2024-09-13T08:37:33.874425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894771+0000","last_change":"2024-09-13T08:37:33.891105+0000","last_active":"2024-09-13T08:37:33.894771+0000","last_peered":"2024-09-13T08:37:33.894771+0000","last_clean":"2024-09-13T08:37:33.894771+0000","last_became_active":"2024-09-13T08:37:33.890783+0000","last_became_peered":"2024-09-13T08:37:33.890783+0000","last_unstale":"2024-09-13T08:37:33.894771+0000","last_undegraded":"2024-09-13T08:37:33.894771+0000","last_fullsized":"2024-09-13T08:37:33.894771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:17.466021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.878005+0000","last_change":"2024-09-13T08:37:33.878005+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.878005+0000","last_undegraded":"2024-09-13T08:37:33.878005+0000","last_fullsized":"2024-09-13T08:37:33.878005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.905653+0000","last_change":"2024-09-13T08:37:33.902610+0000","last_active":"2024-09-13T08:37:33.905653+0000","last_peered":"2024-09-13T08:37:33.905653+0000","last_clean":"2024-09-13T08:37:33.905653+0000","last_became_active":"2024-09-13T08:37:33.902275+0000","last_became_peered":"2024-09-13T08:37:33.902275+0000","last_unstale":"2024-09-13T08:37:33.905653+0000","last_undegraded":"2024-09-13T08:37:33.905653+0000","last_fullsized":"2024-09-13T08:37:33.905653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894679+0000","last_change":"2024-09-13T08:37:34.904983+0000","last_active":"2024-09-13T08:37:35.894679+0000","last_peered":"2024-09-13T08:37:35.894679+0000","last_clean":"2024-09-13T08:37:35.894679+0000","last_became_active":"2024-09-13T08:37:34.904618+0000","last_became_peered":"2024-09-13T08:37:34.904618+0000","last_unstale":"2024-09-13T08:37:35.894679+0000","last_undegraded":"2024-09-13T08:37:35.894679+0000","last_fullsized":"2024-09-13T08:37:35.894679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:30.070140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.895692+0000","last_change":"2024-09-13T08:37:33.888874+0000","last_active":"2024-09-13T08:37:33.895692+0000","last_peered":"2024-09-13T08:37:33.895692+0000","last_clean":"2024-09-13T08:37:33.895692+0000","last_became_active":"2024-09-13T08:37:33.888111+0000","last_became_peered":"2024-09-13T08:37:33.888111+0000","last_unstale":"2024-09-13T08:37:33.895692+0000","last_undegraded":"2024-09-13T08:37:33.895692+0000","last_fullsized":"2024-09-13T08:37:33.895692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874869+0000","last_change":"2024-09-13T08:37:33.874869+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874869+0000","last_undegraded":"2024-09-13T08:37:33.874869+0000","last_fullsized":"2024-09-13T08:37:33.874869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.899257+0000","last_change":"2024-09-13T08:37:33.892008+0000","last_active":"2024-09-13T08:37:33.899257+0000","last_peered":"2024-09-13T08:37:33.899257+0000","last_clean":"2024-09-13T08:37:33.899257+0000","last_became_active":"2024-09-13T08:37:33.891758+0000","last_became_peered":"2024-09-13T08:37:33.891758+0000","last_unstale":"2024-09-13T08:37:33.899257+0000","last_undegraded":"2024-09-13T08:37:33.899257+0000","last_fullsized":"2024-09-13T08:37:33.899257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874510+0000","last_change":"2024-09-13T08:37:33.874510+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874510+0000","last_undegraded":"2024-09-13T08:37:33.874510+0000","last_fullsized":"2024-09-13T08:37:33.874510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894614+0000","last_change":"2024-09-13T08:37:33.888556+0000","last_active":"2024-09-13T08:37:33.894614+0000","last_peered":"2024-09-13T08:37:33.894614+0000","last_clean":"2024-09-13T08:37:33.894614+0000","last_became_active":"2024-09-13T08:37:33.887997+0000","last_became_peered":"2024-09-13T08:37:33.887997+0000","last_unstale":"2024-09-13T08:37:33.894614+0000","last_undegraded":"2024-09-13T08:37:33.894614+0000","last_fullsized":"2024-09-13T08:37:33.894614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872360+0000","last_change":"2024-09-13T08:37:33.872360+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872360+0000","last_undegraded":"2024-09-13T08:37:33.872360+0000","last_fullsized":"2024-09-13T08:37:33.872360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900661+0000","last_change":"2024-09-13T08:37:33.895134+0000","last_active":"2024-09-13T08:37:33.900661+0000","last_peered":"2024-09-13T08:37:33.900661+0000","last_clean":"2024-09-13T08:37:33.900661+0000","last_became_active":"2024-09-13T08:37:33.894893+0000","last_became_peered":"2024-09-13T08:37:33.894893+0000","last_unstale":"2024-09-13T08:37:33.900661+0000","last_undegraded":"2024-09-13T08:37:33.900661+0000","last_fullsized":"2024-09-13T08:37:33.900661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870305+0000","last_change":"2024-09-13T08:37:33.870305+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870305+0000","last_undegraded":"2024-09-13T08:37:33.870305+0000","last_fullsized":"2024-09-13T08:37:33.870305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894667+0000","last_change":"2024-09-13T08:37:33.888156+0000","last_active":"2024-09-13T08:37:33.894667+0000","last_peered":"2024-09-13T08:37:33.894667+0000","last_clean":"2024-09-13T08:37:33.894667+0000","last_became_active":"2024-09-13T08:37:33.886607+0000","last_became_peered":"2024-09-13T08:37:33.886607+0000","last_unstale":"2024-09-13T08:37:33.894667+0000","last_undegraded":"2024-09-13T08:37:33.894667+0000","last_fullsized":"2024-09-13T08:37:33.894667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874608+0000","last_change":"2024-09-13T08:37:33.874608+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874608+0000","last_undegraded":"2024-09-13T08:37:33.874608+0000","last_fullsized":"2024-09-13T08:37:33.874608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.891315+0000","last_change":"2024-09-13T08:37:33.885405+0000","last_active":"2024-09-13T08:37:33.891315+0000","last_peered":"2024-09-13T08:37:33.891315+0000","last_clean":"2024-09-13T08:37:33.891315+0000","last_became_active":"2024-09-13T08:37:33.885048+0000","last_became_peered":"2024-09-13T08:37:33.885048+0000","last_unstale":"2024-09-13T08:37:33.891315+0000","last_undegraded":"2024-09-13T08:37:33.891315+0000","last_fullsized":"2024-09-13T08:37:33.891315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873750+0000","last_change":"2024-09-13T08:37:33.873750+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873750+0000","last_undegraded":"2024-09-13T08:37:33.873750+0000","last_fullsized":"2024-09-13T08:37:33.873750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895446+0000","last_change":"2024-09-13T08:37:33.902815+0000","last_active":"2024-09-13T08:37:35.895446+0000","last_peered":"2024-09-13T08:37:35.895446+0000","last_clean":"2024-09-13T08:37:35.895446+0000","last_became_active":"2024-09-13T08:37:33.902490+0000","last_became_peered":"2024-09-13T08:37:33.902490+0000","last_unstale":"2024-09-13T08:37:35.895446+0000","last_undegraded":"2024-09-13T08:37:35.895446+0000","last_fullsized":"2024-09-13T08:37:35.895446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:32.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872340+0000","last_change":"2024-09-13T08:37:33.872340+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872340+0000","last_undegraded":"2024-09-13T08:37:33.872340+0000","last_fullsized":"2024-09-13T08:37:33.872340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":107,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.877260+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:37:33.877260+0000","last_peered":"2024-09-13T08:37:33.877260+0000","last_clean":"2024-09-13T08:37:33.877260+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:37:33.877260+0000","last_undegraded":"2024-09-13T08:37:33.877260+0000","last_fullsized":"2024-09-13T08:37:33.877260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.893747+0000","last_change":"2024-09-13T08:37:33.883826+0000","last_active":"2024-09-13T08:37:33.893747+0000","last_peered":"2024-09-13T08:37:33.893747+0000","last_clean":"2024-09-13T08:37:33.893747+0000","last_became_active":"2024-09-13T08:37:33.883511+0000","last_became_peered":"2024-09-13T08:37:33.883511+0000","last_unstale":"2024-09-13T08:37:33.893747+0000","last_undegraded":"2024-09-13T08:37:33.893747+0000","last_fullsized":"2024-09-13T08:37:33.893747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.877074+0000","last_change":"2024-09-13T08:37:33.877074+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.877074+0000","last_undegraded":"2024-09-13T08:37:33.877074+0000","last_fullsized":"2024-09-13T08:37:33.877074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.893960+0000","last_change":"2024-09-13T08:37:33.885219+0000","last_active":"2024-09-13T08:37:33.893960+0000","last_peered":"2024-09-13T08:37:33.893960+0000","last_clean":"2024-09-13T08:37:33.893960+0000","last_became_active":"2024-09-13T08:37:33.884889+0000","last_became_peered":"2024-09-13T08:37:33.884889+0000","last_unstale":"2024-09-13T08:37:33.893960+0000","last_undegraded":"2024-09-13T08:37:33.893960+0000","last_fullsized":"2024-09-13T08:37:33.893960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894451+0000","last_change":"2024-09-13T08:37:34.908186+0000","last_active":"2024-09-13T08:37:35.894451+0000","last_peered":"2024-09-13T08:37:35.894451+0000","last_clean":"2024-09-13T08:37:35.894451+0000","last_became_active":"2024-09-13T08:37:34.907886+0000","last_became_peered":"2024-09-13T08:37:34.907886+0000","last_unstale":"2024-09-13T08:37:35.894451+0000","last_undegraded":"2024-09-13T08:37:35.894451+0000","last_fullsized":"2024-09-13T08:37:35.894451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:11:14.712673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.897126+0000","last_change":"2024-09-13T08:37:33.890556+0000","last_active":"2024-09-13T08:37:33.897126+0000","last_peered":"2024-09-13T08:37:33.897126+0000","last_clean":"2024-09-13T08:37:33.897126+0000","last_became_active":"2024-09-13T08:37:33.890252+0000","last_became_peered":"2024-09-13T08:37:33.890252+0000","last_unstale":"2024-09-13T08:37:33.897126+0000","last_undegraded":"2024-09-13T08:37:33.897126+0000","last_fullsized":"2024-09-13T08:37:33.897126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875140+0000","last_change":"2024-09-13T08:37:33.875140+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875140+0000","last_undegraded":"2024-09-13T08:37:33.875140+0000","last_fullsized":"2024-09-13T08:37:33.875140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.895720+0000","last_change":"2024-09-13T08:37:33.889192+0000","last_active":"2024-09-13T08:37:33.895720+0000","last_peered":"2024-09-13T08:37:33.895720+0000","last_clean":"2024-09-13T08:37:33.895720+0000","last_became_active":"2024-09-13T08:37:33.888556+0000","last_became_peered":"2024-09-13T08:37:33.888556+0000","last_unstale":"2024-09-13T08:37:33.895720+0000","last_undegraded":"2024-09-13T08:37:33.895720+0000","last_fullsized":"2024-09-13T08:37:33.895720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872530+0000","last_change":"2024-09-13T08:37:33.872530+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872530+0000","last_undegraded":"2024-09-13T08:37:33.872530+0000","last_fullsized":"2024-09-13T08:37:33.872530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894811+0000","last_change":"2024-09-13T08:37:33.889699+0000","last_active":"2024-09-13T08:37:33.894811+0000","last_peered":"2024-09-13T08:37:33.894811+0000","last_clean":"2024-09-13T08:37:33.894811+0000","last_became_active":"2024-09-13T08:37:33.889224+0000","last_became_peered":"2024-09-13T08:37:33.889224+0000","last_unstale":"2024-09-13T08:37:33.894811+0000","last_undegraded":"2024-09-13T08:37:33.894811+0000","last_fullsized":"2024-09-13T08:37:33.894811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.871796+0000","last_change":"2024-09-13T08:37:33.871796+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.871796+0000","last_undegraded":"2024-09-13T08:37:33.871796+0000","last_fullsized":"2024-09-13T08:37:33.871796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900269+0000","last_change":"2024-09-13T08:37:33.893542+0000","last_active":"2024-09-13T08:37:33.900269+0000","last_peered":"2024-09-13T08:37:33.900269+0000","last_clean":"2024-09-13T08:37:33.900269+0000","last_became_active":"2024-09-13T08:37:33.893231+0000","last_became_peered":"2024-09-13T08:37:33.893231+0000","last_unstale":"2024-09-13T08:37:33.900269+0000","last_undegraded":"2024-09-13T08:37:33.900269+0000","last_fullsized":"2024-09-13T08:37:33.900269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:36.540294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895292+0000","last_change":"2024-09-13T08:37:34.901999+0000","last_active":"2024-09-13T08:37:35.895292+0000","last_peered":"2024-09-13T08:37:35.895292+0000","last_clean":"2024-09-13T08:37:35.895292+0000","last_became_active":"2024-09-13T08:37:34.901765+0000","last_became_peered":"2024-09-13T08:37:34.901765+0000","last_unstale":"2024-09-13T08:37:35.895292+0000","last_undegraded":"2024-09-13T08:37:35.895292+0000","last_fullsized":"2024-09-13T08:37:35.895292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:12.422693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.903257+0000","last_change":"2024-09-13T08:37:33.898217+0000","last_active":"2024-09-13T08:37:33.903257+0000","last_peered":"2024-09-13T08:37:33.903257+0000","last_clean":"2024-09-13T08:37:33.903257+0000","last_became_active":"2024-09-13T08:37:33.897954+0000","last_became_peered":"2024-09-13T08:37:33.897954+0000","last_unstale":"2024-09-13T08:37:33.903257+0000","last_undegraded":"2024-09-13T08:37:33.903257+0000","last_fullsized":"2024-09-13T08:37:33.903257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.879326+0000","last_change":"2024-09-13T08:37:33.879326+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.879326+0000","last_undegraded":"2024-09-13T08:37:33.879326+0000","last_fullsized":"2024-09-13T08:37:33.879326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.901668+0000","last_change":"2024-09-13T08:37:33.897139+0000","last_active":"2024-09-13T08:37:33.901668+0000","last_peered":"2024-09-13T08:37:33.901668+0000","last_clean":"2024-09-13T08:37:33.901668+0000","last_became_active":"2024-09-13T08:37:33.896576+0000","last_became_peered":"2024-09-13T08:37:33.896576+0000","last_unstale":"2024-09-13T08:37:33.901668+0000","last_undegraded":"2024-09-13T08:37:33.901668+0000","last_fullsized":"2024-09-13T08:37:33.901668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:05.995257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.879322+0000","last_change":"2024-09-13T08:37:33.879322+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.879322+0000","last_undegraded":"2024-09-13T08:37:33.879322+0000","last_fullsized":"2024-09-13T08:37:33.879322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900384+0000","last_change":"2024-09-13T08:37:33.893088+0000","last_active":"2024-09-13T08:37:33.900384+0000","last_peered":"2024-09-13T08:37:33.900384+0000","last_clean":"2024-09-13T08:37:33.900384+0000","last_became_active":"2024-09-13T08:37:33.892731+0000","last_became_peered":"2024-09-13T08:37:33.892731+0000","last_unstale":"2024-09-13T08:37:33.900384+0000","last_undegraded":"2024-09-13T08:37:33.900384+0000","last_fullsized":"2024-09-13T08:37:33.900384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870235+0000","last_change":"2024-09-13T08:37:33.870235+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870235+0000","last_undegraded":"2024-09-13T08:37:33.870235+0000","last_fullsized":"2024-09-13T08:37:33.870235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.903271+0000","last_change":"2024-09-13T08:37:33.898275+0000","last_active":"2024-09-13T08:37:33.903271+0000","last_peered":"2024-09-13T08:37:33.903271+0000","last_clean":"2024-09-13T08:37:33.903271+0000","last_became_active":"2024-09-13T08:37:33.897960+0000","last_became_peered":"2024-09-13T08:37:33.897960+0000","last_unstale":"2024-09-13T08:37:33.903271+0000","last_undegraded":"2024-09-13T08:37:33.903271+0000","last_fullsized":"2024-09-13T08:37:33.903271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874612+0000","last_change":"2024-09-13T08:37:33.874612+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874612+0000","last_undegraded":"2024-09-13T08:37:33.874612+0000","last_fullsized":"2024-09-13T08:37:33.874612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894655+0000","last_change":"2024-09-13T08:37:33.891510+0000","last_active":"2024-09-13T08:37:35.894655+0000","last_peered":"2024-09-13T08:37:35.894655+0000","last_clean":"2024-09-13T08:37:35.894655+0000","last_became_active":"2024-09-13T08:37:33.890774+0000","last_became_peered":"2024-09-13T08:37:33.890774+0000","last_unstale":"2024-09-13T08:37:35.894655+0000","last_undegraded":"2024-09-13T08:37:35.894655+0000","last_fullsized":"2024-09-13T08:37:35.894655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:25.370931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876319+0000","last_change":"2024-09-13T08:37:33.876319+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876319+0000","last_undegraded":"2024-09-13T08:37:33.876319+0000","last_fullsized":"2024-09-13T08:37:33.876319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.893774+0000","last_change":"2024-09-13T08:37:33.887243+0000","last_active":"2024-09-13T08:37:33.893774+0000","last_peered":"2024-09-13T08:37:33.893774+0000","last_clean":"2024-09-13T08:37:33.893774+0000","last_became_active":"2024-09-13T08:37:33.886882+0000","last_became_peered":"2024-09-13T08:37:33.886882+0000","last_unstale":"2024-09-13T08:37:33.893774+0000","last_undegraded":"2024-09-13T08:37:33.893774+0000","last_fullsized":"2024-09-13T08:37:33.893774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.871566+0000","last_change":"2024-09-13T08:37:33.871566+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.871566+0000","last_undegraded":"2024-09-13T08:37:33.871566+0000","last_fullsized":"2024-09-13T08:37:33.871566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875076+0000","last_change":"2024-09-13T08:37:33.875076+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875076+0000","last_undegraded":"2024-09-13T08:37:33.875076+0000","last_fullsized":"2024-09-13T08:37:33.875076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.904025+0000","last_change":"2024-09-13T08:37:33.900929+0000","last_active":"2024-09-13T08:37:33.904025+0000","last_peered":"2024-09-13T08:37:33.904025+0000","last_clean":"2024-09-13T08:37:33.904025+0000","last_became_active":"2024-09-13T08:37:33.899776+0000","last_became_peered":"2024-09-13T08:37:33.899776+0000","last_unstale":"2024-09-13T08:37:33.904025+0000","last_undegraded":"2024-09-13T08:37:33.904025+0000","last_fullsized":"2024-09-13T08:37:33.904025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873649+0000","last_change":"2024-09-13T08:37:33.873649+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873649+0000","last_undegraded":"2024-09-13T08:37:33.873649+0000","last_fullsized":"2024-09-13T08:37:33.873649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900424+0000","last_change":"2024-09-13T08:37:33.894985+0000","last_active":"2024-09-13T08:37:33.900424+0000","last_peered":"2024-09-13T08:37:33.900424+0000","last_clean":"2024-09-13T08:37:33.900424+0000","last_became_active":"2024-09-13T08:37:33.894087+0000","last_became_peered":"2024-09-13T08:37:33.894087+0000","last_unstale":"2024-09-13T08:37:33.900424+0000","last_undegraded":"2024-09-13T08:37:33.900424+0000","last_fullsized":"2024-09-13T08:37:33.900424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876978+0000","last_change":"2024-09-13T08:37:33.876978+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876978+0000","last_undegraded":"2024-09-13T08:37:33.876978+0000","last_fullsized":"2024-09-13T08:37:33.876978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.892622+0000","last_change":"2024-09-13T08:37:33.887848+0000","last_active":"2024-09-13T08:37:33.892622+0000","last_peered":"2024-09-13T08:37:33.892622+0000","last_clean":"2024-09-13T08:37:33.892622+0000","last_became_active":"2024-09-13T08:37:33.887508+0000","last_became_peered":"2024-09-13T08:37:33.887508+0000","last_unstale":"2024-09-13T08:37:33.892622+0000","last_undegraded":"2024-09-13T08:37:33.892622+0000","last_fullsized":"2024-09-13T08:37:33.892622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875975+0000","last_change":"2024-09-13T08:37:33.875975+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875975+0000","last_undegraded":"2024-09-13T08:37:33.875975+0000","last_fullsized":"2024-09-13T08:37:33.875975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.897760+0000","last_change":"2024-09-13T08:37:33.894082+0000","last_active":"2024-09-13T08:37:33.897760+0000","last_peered":"2024-09-13T08:37:33.897760+0000","last_clean":"2024-09-13T08:37:33.897760+0000","last_became_active":"2024-09-13T08:37:33.893820+0000","last_became_peered":"2024-09-13T08:37:33.893820+0000","last_unstale":"2024-09-13T08:37:33.897760+0000","last_undegraded":"2024-09-13T08:37:33.897760+0000","last_fullsized":"2024-09-13T08:37:33.897760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":653577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:37:39.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:37:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 -- 172.21.15.44:0/1949228549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac15bea0 msgr2=0x7f62ac0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 --2- 172.21.15.44:0/1949228549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac15bea0 0x7f62ac0ce670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f629c009900 tx=0x7f629c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 -- 172.21.15.44:0/1949228549 shutdown_connections 2024-09-13T08:37:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 --2- 172.21.15.44:0/1949228549 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f62ac0cdb30 0x7f62ac0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 --2- 172.21.15.44:0/1949228549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac15bea0 0x7f62ac0ce670 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 -- 172.21.15.44:0/1949228549 >> 172.21.15.44:0/1949228549 conn(0x7f62ac0d1dc0 msgr2=0x7f62ac0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 -- 172.21.15.44:0/1949228549 shutdown_connections 2024-09-13T08:37:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 -- 172.21.15.44:0/1949228549 wait complete. 2024-09-13T08:37:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 Processor -- start 2024-09-13T08:37:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 -- start start 2024-09-13T08:37:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac0cdb30 0x7f62ac0c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac0d28e0 0x7f62ac15ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f62ac15bea0 0x7f62ac151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac169020 con 0x7f62ac0cdb30 2024-09-13T08:37:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac168ea0 con 0x7f62ac0d28e0 2024-09-13T08:37:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.223+0000 7f62b0e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f62ac1691a0 con 0x7f62ac15bea0 2024-09-13T08:37:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac0cdb30 0x7f62ac0c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac0cdb30 0x7f62ac0c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60686/0 (socket says 172.21.15.44:60686) 2024-09-13T08:37:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a9e00640 1 -- 172.21.15.44:0/2765296381 learned_addr learned my addr 172.21.15.44:0/2765296381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a3e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac0d28e0 0x7f62ac15ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:39.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62aa800640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f62ac15bea0 0x7f62ac151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a9e00640 1 -- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f62ac15bea0 msgr2=0x7f62ac151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a9e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f62ac15bea0 0x7f62ac151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a9e00640 1 -- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac0d28e0 msgr2=0x7f62ac15ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a9e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac0d28e0 0x7f62ac15ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a9e00640 1 -- 172.21.15.44:0/2765296381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac0c9230 con 0x7f62ac0cdb30 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a9e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac0cdb30 0x7f62ac0c7ef0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f629c0b0040 tx=0x7f629c0a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a2000640 1 -- 172.21.15.44:0/2765296381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c0b2040 con 0x7f62ac0cdb30 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62b0e00640 1 -- 172.21.15.44:0/2765296381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac0c9510 con 0x7f62ac0cdb30 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a2000640 1 -- 172.21.15.44:0/2765296381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f629c0043b0 con 0x7f62ac0cdb30 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a2000640 1 -- 172.21.15.44:0/2765296381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c005290 con 0x7f62ac0cdb30 2024-09-13T08:37:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62b0e00640 1 -- 172.21.15.44:0/2765296381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac0c9ad0 con 0x7f62ac0cdb30 2024-09-13T08:37:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62b0e00640 1 -- 172.21.15.44:0/2765296381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ac0cc9d0 con 0x7f62ac0cdb30 2024-09-13T08:37:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.227+0000 7f62a2000640 1 -- 172.21.15.44:0/2765296381 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f629c0be060 con 0x7f62ac0cdb30 2024-09-13T08:37:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.231+0000 7f62a2000640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6288072ec0 0x7f6288075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.231+0000 7f62a2000640 1 -- 172.21.15.44:0/2765296381 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f629c12d1e0 con 0x7f62ac0cdb30 2024-09-13T08:37:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.231+0000 7f62a3e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6288072ec0 0x7f6288075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:39.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.231+0000 7f62a3e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6288072ec0 0x7f6288075380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6294009600 tx=0x7f62940091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.235+0000 7f62a2000640 1 -- 172.21.15.44:0/2765296381 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f629c0164d0 con 0x7f62ac0cdb30 2024-09-13T08:37:39.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.407+0000 7f62b0e00640 1 -- 172.21.15.44:0/2765296381 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62ac15b710 con 0x7f6288072ec0 2024-09-13T08:37:39.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.411+0000 7f62a2000640 1 -- 172.21.15.44:0/2765296381 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247213 (secure 0 0 0) 0x7f62ac15b710 con 0x7f6288072ec0 2024-09-13T08:37:39.418 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:39.419 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:37:39.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 -- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6288072ec0 msgr2=0x7f6288075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6288072ec0 0x7f6288075380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6294009600 tx=0x7f62940091e0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 -- 172.21.15.44:0/2765296381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac0cdb30 msgr2=0x7f62ac0c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac0cdb30 0x7f62ac0c7ef0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f629c0b0040 tx=0x7f629c0a5730 comp rx=0 tx=0).stop 2024-09-13T08:37:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 -- 172.21.15.44:0/2765296381 shutdown_connections 2024-09-13T08:37:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6288072ec0 0x7f6288075380 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f62ac15bea0 0x7f62ac151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac0d28e0 0x7f62ac15ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 --2- 172.21.15.44:0/2765296381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac0cdb30 0x7f62ac0c7ef0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 -- 172.21.15.44:0/2765296381 >> 172.21.15.44:0/2765296381 conn(0x7f62ac0d1dc0 msgr2=0x7f62ac0b6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 -- 172.21.15.44:0/2765296381 shutdown_connections 2024-09-13T08:37:39.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.415+0000 7f62b0e00640 1 -- 172.21.15.44:0/2765296381 wait complete. 2024-09-13T08:37:39.438 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-13T08:37:38.511365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218896,"kb_used_data":2660,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755824,"statfs":{"total":773094113280,"available":772869963776,"internally_reserved":0,"allocated":2723840,"data_stored":1668108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.654417"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.888188+0000","last_change":"2024-09-13T08:37:33.888188+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.888188+0000","last_undegraded":"2024-09-13T08:37:33.888188+0000","last_fullsized":"2024-09-13T08:37:33.888188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876585+0000","last_change":"2024-09-13T08:37:33.876585+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876585+0000","last_undegraded":"2024-09-13T08:37:33.876585+0000","last_fullsized":"2024-09-13T08:37:33.876585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873515+0000","last_change":"2024-09-13T08:37:33.873515+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873515+0000","last_undegraded":"2024-09-13T08:37:33.873515+0000","last_fullsized":"2024-09-13T08:37:33.873515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870231+0000","last_change":"2024-09-13T08:37:33.870231+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870231+0000","last_undegraded":"2024-09-13T08:37:33.870231+0000","last_fullsized":"2024-09-13T08:37:33.870231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.887043+0000","last_change":"2024-09-13T08:37:33.887043+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.887043+0000","last_undegraded":"2024-09-13T08:37:33.887043+0000","last_fullsized":"2024-09-13T08:37:33.887043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.869808+0000","last_change":"2024-09-13T08:37:33.869808+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.869808+0000","last_undegraded":"2024-09-13T08:37:33.869808+0000","last_fullsized":"2024-09-13T08:37:33.869808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.871829+0000","last_change":"2024-09-13T08:37:33.871829+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.871829+0000","last_undegraded":"2024-09-13T08:37:33.871829+0000","last_fullsized":"2024-09-13T08:37:33.871829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875627+0000","last_change":"2024-09-13T08:37:33.875627+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875627+0000","last_undegraded":"2024-09-13T08:37:33.875627+0000","last_fullsized":"2024-09-13T08:37:33.875627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874425+0000","last_change":"2024-09-13T08:37:33.874425+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874425+0000","last_undegraded":"2024-09-13T08:37:33.874425+0000","last_fullsized":"2024-09-13T08:37:33.874425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874047+0000","last_change":"2024-09-13T08:37:33.874047+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874047+0000","last_undegraded":"2024-09-13T08:37:33.874047+0000","last_fullsized":"2024-09-13T08:37:33.874047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.881267+0000","last_change":"2024-09-13T08:37:33.881267+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.881267+0000","last_undegraded":"2024-09-13T08:37:33.881267+0000","last_fullsized":"2024-09-13T08:37:33.881267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.877798+0000","last_change":"2024-09-13T08:37:33.877798+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.877798+0000","last_undegraded":"2024-09-13T08:37:33.877798+0000","last_fullsized":"2024-09-13T08:37:33.877798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894980+0000","last_change":"2024-09-13T08:37:34.903037+0000","last_active":"2024-09-13T08:37:35.894980+0000","last_peered":"2024-09-13T08:37:35.894980+0000","last_clean":"2024-09-13T08:37:35.894980+0000","last_became_active":"2024-09-13T08:37:34.894805+0000","last_became_peered":"2024-09-13T08:37:34.894805+0000","last_unstale":"2024-09-13T08:37:35.894980+0000","last_undegraded":"2024-09-13T08:37:35.894980+0000","last_fullsized":"2024-09-13T08:37:35.894980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:49.712241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894573+0000","last_change":"2024-09-13T08:37:34.901603+0000","last_active":"2024-09-13T08:37:35.894573+0000","last_peered":"2024-09-13T08:37:35.894573+0000","last_clean":"2024-09-13T08:37:35.894573+0000","last_became_active":"2024-09-13T08:37:34.899884+0000","last_became_peered":"2024-09-13T08:37:34.899884+0000","last_unstale":"2024-09-13T08:37:35.894573+0000","last_undegraded":"2024-09-13T08:37:35.894573+0000","last_fullsized":"2024-09-13T08:37:35.894573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:05.260600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874667+0000","last_change":"2024-09-13T08:37:33.874667+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874667+0000","last_undegraded":"2024-09-13T08:37:33.874667+0000","last_fullsized":"2024-09-13T08:37:33.874667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876422+0000","last_change":"2024-09-13T08:37:33.876422+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876422+0000","last_undegraded":"2024-09-13T08:37:33.876422+0000","last_fullsized":"2024-09-13T08:37:33.876422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874801+0000","last_change":"2024-09-13T08:37:33.874801+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874801+0000","last_undegraded":"2024-09-13T08:37:33.874801+0000","last_fullsized":"2024-09-13T08:37:33.874801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.878051+0000","last_change":"2024-09-13T08:37:33.878051+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.878051+0000","last_undegraded":"2024-09-13T08:37:33.878051+0000","last_fullsized":"2024-09-13T08:37:33.878051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875550+0000","last_change":"2024-09-13T08:37:33.875550+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875550+0000","last_undegraded":"2024-09-13T08:37:33.875550+0000","last_fullsized":"2024-09-13T08:37:33.875550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875619+0000","last_change":"2024-09-13T08:37:33.875619+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875619+0000","last_undegraded":"2024-09-13T08:37:33.875619+0000","last_fullsized":"2024-09-13T08:37:33.875619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.881053+0000","last_change":"2024-09-13T08:37:33.881053+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.881053+0000","last_undegraded":"2024-09-13T08:37:33.881053+0000","last_fullsized":"2024-09-13T08:37:33.881053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875830+0000","last_change":"2024-09-13T08:37:33.875830+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875830+0000","last_undegraded":"2024-09-13T08:37:33.875830+0000","last_fullsized":"2024-09-13T08:37:33.875830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873389+0000","last_change":"2024-09-13T08:37:33.873389+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873389+0000","last_undegraded":"2024-09-13T08:37:33.873389+0000","last_fullsized":"2024-09-13T08:37:33.873389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875855+0000","last_change":"2024-09-13T08:37:33.875855+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875855+0000","last_undegraded":"2024-09-13T08:37:33.875855+0000","last_fullsized":"2024-09-13T08:37:33.875855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.877601+0000","last_change":"2024-09-13T08:37:33.877601+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.877601+0000","last_undegraded":"2024-09-13T08:37:33.877601+0000","last_fullsized":"2024-09-13T08:37:33.877601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.878497+0000","last_change":"2024-09-13T08:37:33.878497+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.878497+0000","last_undegraded":"2024-09-13T08:37:33.878497+0000","last_fullsized":"2024-09-13T08:37:33.878497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895541+0000","last_change":"2024-09-13T08:37:34.902546+0000","last_active":"2024-09-13T08:37:35.895541+0000","last_peered":"2024-09-13T08:37:35.895541+0000","last_clean":"2024-09-13T08:37:35.895541+0000","last_became_active":"2024-09-13T08:37:34.901841+0000","last_became_peered":"2024-09-13T08:37:34.901841+0000","last_unstale":"2024-09-13T08:37:35.895541+0000","last_undegraded":"2024-09-13T08:37:35.895541+0000","last_fullsized":"2024-09-13T08:37:35.895541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:40.635073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895078+0000","last_change":"2024-09-13T08:37:34.907277+0000","last_active":"2024-09-13T08:37:35.895078+0000","last_peered":"2024-09-13T08:37:35.895078+0000","last_clean":"2024-09-13T08:37:35.895078+0000","last_became_active":"2024-09-13T08:37:34.906984+0000","last_became_peered":"2024-09-13T08:37:34.906984+0000","last_unstale":"2024-09-13T08:37:35.895078+0000","last_undegraded":"2024-09-13T08:37:35.895078+0000","last_fullsized":"2024-09-13T08:37:35.895078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:23.664559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872673+0000","last_change":"2024-09-13T08:37:33.872673+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872673+0000","last_undegraded":"2024-09-13T08:37:33.872673+0000","last_fullsized":"2024-09-13T08:37:33.872673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874060+0000","last_change":"2024-09-13T08:37:33.874060+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874060+0000","last_undegraded":"2024-09-13T08:37:33.874060+0000","last_fullsized":"2024-09-13T08:37:33.874060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874453+0000","last_change":"2024-09-13T08:37:33.874453+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874453+0000","last_undegraded":"2024-09-13T08:37:33.874453+0000","last_fullsized":"2024-09-13T08:37:33.874453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872506+0000","last_change":"2024-09-13T08:37:33.872506+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872506+0000","last_undegraded":"2024-09-13T08:37:33.872506+0000","last_fullsized":"2024-09-13T08:37:33.872506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875197+0000","last_change":"2024-09-13T08:37:33.875197+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875197+0000","last_undegraded":"2024-09-13T08:37:33.875197+0000","last_fullsized":"2024-09-13T08:37:33.875197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.892711+0000","last_change":"2024-09-13T08:37:33.888852+0000","last_active":"2024-09-13T08:37:33.892711+0000","last_peered":"2024-09-13T08:37:33.892711+0000","last_clean":"2024-09-13T08:37:33.892711+0000","last_became_active":"2024-09-13T08:37:33.888409+0000","last_became_peered":"2024-09-13T08:37:33.888409+0000","last_unstale":"2024-09-13T08:37:33.892711+0000","last_undegraded":"2024-09-13T08:37:33.892711+0000","last_fullsized":"2024-09-13T08:37:33.892711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872451+0000","last_change":"2024-09-13T08:37:33.872451+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872451+0000","last_undegraded":"2024-09-13T08:37:33.872451+0000","last_fullsized":"2024-09-13T08:37:33.872451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.895745+0000","last_change":"2024-09-13T08:37:33.892308+0000","last_active":"2024-09-13T08:37:33.895745+0000","last_peered":"2024-09-13T08:37:33.895745+0000","last_clean":"2024-09-13T08:37:33.895745+0000","last_became_active":"2024-09-13T08:37:33.891875+0000","last_became_peered":"2024-09-13T08:37:33.891875+0000","last_unstale":"2024-09-13T08:37:33.895745+0000","last_undegraded":"2024-09-13T08:37:33.895745+0000","last_fullsized":"2024-09-13T08:37:33.895745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:22.150995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894251+0000","last_change":"2024-09-13T08:37:33.896561+0000","last_active":"2024-09-13T08:37:35.894251+0000","last_peered":"2024-09-13T08:37:35.894251+0000","last_clean":"2024-09-13T08:37:35.894251+0000","last_became_active":"2024-09-13T08:37:33.896314+0000","last_became_peered":"2024-09-13T08:37:33.896314+0000","last_unstale":"2024-09-13T08:37:35.894251+0000","last_undegraded":"2024-09-13T08:37:35.894251+0000","last_fullsized":"2024-09-13T08:37:35.894251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:28.728254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872534+0000","last_change":"2024-09-13T08:37:33.872534+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872534+0000","last_undegraded":"2024-09-13T08:37:33.872534+0000","last_fullsized":"2024-09-13T08:37:33.872534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.889489+0000","last_change":"2024-09-13T08:37:33.883156+0000","last_active":"2024-09-13T08:37:33.889489+0000","last_peered":"2024-09-13T08:37:33.889489+0000","last_clean":"2024-09-13T08:37:33.889489+0000","last_became_active":"2024-09-13T08:37:33.882800+0000","last_became_peered":"2024-09-13T08:37:33.882800+0000","last_unstale":"2024-09-13T08:37:33.889489+0000","last_undegraded":"2024-09-13T08:37:33.889489+0000","last_fullsized":"2024-09-13T08:37:33.889489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.879388+0000","last_change":"2024-09-13T08:37:33.879388+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.879388+0000","last_undegraded":"2024-09-13T08:37:33.879388+0000","last_fullsized":"2024-09-13T08:37:33.879388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894588+0000","last_change":"2024-09-13T08:37:33.891385+0000","last_active":"2024-09-13T08:37:33.894588+0000","last_peered":"2024-09-13T08:37:33.894588+0000","last_clean":"2024-09-13T08:37:33.894588+0000","last_became_active":"2024-09-13T08:37:33.891112+0000","last_became_peered":"2024-09-13T08:37:33.891112+0000","last_unstale":"2024-09-13T08:37:33.894588+0000","last_undegraded":"2024-09-13T08:37:33.894588+0000","last_fullsized":"2024-09-13T08:37:33.894588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870875+0000","last_change":"2024-09-13T08:37:33.870875+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870875+0000","last_undegraded":"2024-09-13T08:37:33.870875+0000","last_fullsized":"2024-09-13T08:37:33.870875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.904015+0000","last_change":"2024-09-13T08:37:33.899147+0000","last_active":"2024-09-13T08:37:33.904015+0000","last_peered":"2024-09-13T08:37:33.904015+0000","last_clean":"2024-09-13T08:37:33.904015+0000","last_became_active":"2024-09-13T08:37:33.896929+0000","last_became_peered":"2024-09-13T08:37:33.896929+0000","last_unstale":"2024-09-13T08:37:33.904015+0000","last_undegraded":"2024-09-13T08:37:33.904015+0000","last_fullsized":"2024-09-13T08:37:33.904015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876422+0000","last_change":"2024-09-13T08:37:33.876422+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876422+0000","last_undegraded":"2024-09-13T08:37:33.876422+0000","last_fullsized":"2024-09-13T08:37:33.876422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900065+0000","last_change":"2024-09-13T08:37:33.892872+0000","last_active":"2024-09-13T08:37:33.900065+0000","last_peered":"2024-09-13T08:37:33.900065+0000","last_clean":"2024-09-13T08:37:33.900065+0000","last_became_active":"2024-09-13T08:37:33.892579+0000","last_became_peered":"2024-09-13T08:37:33.892579+0000","last_unstale":"2024-09-13T08:37:33.900065+0000","last_undegraded":"2024-09-13T08:37:33.900065+0000","last_fullsized":"2024-09-13T08:37:33.900065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874425+0000","last_change":"2024-09-13T08:37:33.874425+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874425+0000","last_undegraded":"2024-09-13T08:37:33.874425+0000","last_fullsized":"2024-09-13T08:37:33.874425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894771+0000","last_change":"2024-09-13T08:37:33.891105+0000","last_active":"2024-09-13T08:37:33.894771+0000","last_peered":"2024-09-13T08:37:33.894771+0000","last_clean":"2024-09-13T08:37:33.894771+0000","last_became_active":"2024-09-13T08:37:33.890783+0000","last_became_peered":"2024-09-13T08:37:33.890783+0000","last_unstale":"2024-09-13T08:37:33.894771+0000","last_undegraded":"2024-09-13T08:37:33.894771+0000","last_fullsized":"2024-09-13T08:37:33.894771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:17.466021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.878005+0000","last_change":"2024-09-13T08:37:33.878005+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.878005+0000","last_undegraded":"2024-09-13T08:37:33.878005+0000","last_fullsized":"2024-09-13T08:37:33.878005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.905653+0000","last_change":"2024-09-13T08:37:33.902610+0000","last_active":"2024-09-13T08:37:33.905653+0000","last_peered":"2024-09-13T08:37:33.905653+0000","last_clean":"2024-09-13T08:37:33.905653+0000","last_became_active":"2024-09-13T08:37:33.902275+0000","last_became_peered":"2024-09-13T08:37:33.902275+0000","last_unstale":"2024-09-13T08:37:33.905653+0000","last_undegraded":"2024-09-13T08:37:33.905653+0000","last_fullsized":"2024-09-13T08:37:33.905653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894679+0000","last_change":"2024-09-13T08:37:34.904983+0000","last_active":"2024-09-13T08:37:35.894679+0000","last_peered":"2024-09-13T08:37:35.894679+0000","last_clean":"2024-09-13T08:37:35.894679+0000","last_became_active":"2024-09-13T08:37:34.904618+0000","last_became_peered":"2024-09-13T08:37:34.904618+0000","last_unstale":"2024-09-13T08:37:35.894679+0000","last_undegraded":"2024-09-13T08:37:35.894679+0000","last_fullsized":"2024-09-13T08:37:35.894679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:30.070140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.895692+0000","last_change":"2024-09-13T08:37:33.888874+0000","last_active":"2024-09-13T08:37:33.895692+0000","last_peered":"2024-09-13T08:37:33.895692+0000","last_clean":"2024-09-13T08:37:33.895692+0000","last_became_active":"2024-09-13T08:37:33.888111+0000","last_became_peered":"2024-09-13T08:37:33.888111+0000","last_unstale":"2024-09-13T08:37:33.895692+0000","last_undegraded":"2024-09-13T08:37:33.895692+0000","last_fullsized":"2024-09-13T08:37:33.895692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874869+0000","last_change":"2024-09-13T08:37:33.874869+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874869+0000","last_undegraded":"2024-09-13T08:37:33.874869+0000","last_fullsized":"2024-09-13T08:37:33.874869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.899257+0000","last_change":"2024-09-13T08:37:33.892008+0000","last_active":"2024-09-13T08:37:33.899257+0000","last_peered":"2024-09-13T08:37:33.899257+0000","last_clean":"2024-09-13T08:37:33.899257+0000","last_became_active":"2024-09-13T08:37:33.891758+0000","last_became_peered":"2024-09-13T08:37:33.891758+0000","last_unstale":"2024-09-13T08:37:33.899257+0000","last_undegraded":"2024-09-13T08:37:33.899257+0000","last_fullsized":"2024-09-13T08:37:33.899257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874510+0000","last_change":"2024-09-13T08:37:33.874510+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874510+0000","last_undegraded":"2024-09-13T08:37:33.874510+0000","last_fullsized":"2024-09-13T08:37:33.874510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894614+0000","last_change":"2024-09-13T08:37:33.888556+0000","last_active":"2024-09-13T08:37:33.894614+0000","last_peered":"2024-09-13T08:37:33.894614+0000","last_clean":"2024-09-13T08:37:33.894614+0000","last_became_active":"2024-09-13T08:37:33.887997+0000","last_became_peered":"2024-09-13T08:37:33.887997+0000","last_unstale":"2024-09-13T08:37:33.894614+0000","last_undegraded":"2024-09-13T08:37:33.894614+0000","last_fullsized":"2024-09-13T08:37:33.894614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872360+0000","last_change":"2024-09-13T08:37:33.872360+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872360+0000","last_undegraded":"2024-09-13T08:37:33.872360+0000","last_fullsized":"2024-09-13T08:37:33.872360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900661+0000","last_change":"2024-09-13T08:37:33.895134+0000","last_active":"2024-09-13T08:37:33.900661+0000","last_peered":"2024-09-13T08:37:33.900661+0000","last_clean":"2024-09-13T08:37:33.900661+0000","last_became_active":"2024-09-13T08:37:33.894893+0000","last_became_peered":"2024-09-13T08:37:33.894893+0000","last_unstale":"2024-09-13T08:37:33.900661+0000","last_undegraded":"2024-09-13T08:37:33.900661+0000","last_fullsized":"2024-09-13T08:37:33.900661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870305+0000","last_change":"2024-09-13T08:37:33.870305+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870305+0000","last_undegraded":"2024-09-13T08:37:33.870305+0000","last_fullsized":"2024-09-13T08:37:33.870305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894667+0000","last_change":"2024-09-13T08:37:33.888156+0000","last_active":"2024-09-13T08:37:33.894667+0000","last_peered":"2024-09-13T08:37:33.894667+0000","last_clean":"2024-09-13T08:37:33.894667+0000","last_became_active":"2024-09-13T08:37:33.886607+0000","last_became_peered":"2024-09-13T08:37:33.886607+0000","last_unstale":"2024-09-13T08:37:33.894667+0000","last_undegraded":"2024-09-13T08:37:33.894667+0000","last_fullsized":"2024-09-13T08:37:33.894667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874608+0000","last_change":"2024-09-13T08:37:33.874608+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874608+0000","last_undegraded":"2024-09-13T08:37:33.874608+0000","last_fullsized":"2024-09-13T08:37:33.874608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.891315+0000","last_change":"2024-09-13T08:37:33.885405+0000","last_active":"2024-09-13T08:37:33.891315+0000","last_peered":"2024-09-13T08:37:33.891315+0000","last_clean":"2024-09-13T08:37:33.891315+0000","last_became_active":"2024-09-13T08:37:33.885048+0000","last_became_peered":"2024-09-13T08:37:33.885048+0000","last_unstale":"2024-09-13T08:37:33.891315+0000","last_undegraded":"2024-09-13T08:37:33.891315+0000","last_fullsized":"2024-09-13T08:37:33.891315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873750+0000","last_change":"2024-09-13T08:37:33.873750+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873750+0000","last_undegraded":"2024-09-13T08:37:33.873750+0000","last_fullsized":"2024-09-13T08:37:33.873750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895446+0000","last_change":"2024-09-13T08:37:33.902815+0000","last_active":"2024-09-13T08:37:35.895446+0000","last_peered":"2024-09-13T08:37:35.895446+0000","last_clean":"2024-09-13T08:37:35.895446+0000","last_became_active":"2024-09-13T08:37:33.902490+0000","last_became_peered":"2024-09-13T08:37:33.902490+0000","last_unstale":"2024-09-13T08:37:35.895446+0000","last_undegraded":"2024-09-13T08:37:35.895446+0000","last_fullsized":"2024-09-13T08:37:35.895446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:32.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872340+0000","last_change":"2024-09-13T08:37:33.872340+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872340+0000","last_undegraded":"2024-09-13T08:37:33.872340+0000","last_fullsized":"2024-09-13T08:37:33.872340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":107,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.877260+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:37:33.877260+0000","last_peered":"2024-09-13T08:37:33.877260+0000","last_clean":"2024-09-13T08:37:33.877260+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:37:33.877260+0000","last_undegraded":"2024-09-13T08:37:33.877260+0000","last_fullsized":"2024-09-13T08:37:33.877260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.893747+0000","last_change":"2024-09-13T08:37:33.883826+0000","last_active":"2024-09-13T08:37:33.893747+0000","last_peered":"2024-09-13T08:37:33.893747+0000","last_clean":"2024-09-13T08:37:33.893747+0000","last_became_active":"2024-09-13T08:37:33.883511+0000","last_became_peered":"2024-09-13T08:37:33.883511+0000","last_unstale":"2024-09-13T08:37:33.893747+0000","last_undegraded":"2024-09-13T08:37:33.893747+0000","last_fullsized":"2024-09-13T08:37:33.893747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.877074+0000","last_change":"2024-09-13T08:37:33.877074+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.877074+0000","last_undegraded":"2024-09-13T08:37:33.877074+0000","last_fullsized":"2024-09-13T08:37:33.877074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.893960+0000","last_change":"2024-09-13T08:37:33.885219+0000","last_active":"2024-09-13T08:37:33.893960+0000","last_peered":"2024-09-13T08:37:33.893960+0000","last_clean":"2024-09-13T08:37:33.893960+0000","last_became_active":"2024-09-13T08:37:33.884889+0000","last_became_peered":"2024-09-13T08:37:33.884889+0000","last_unstale":"2024-09-13T08:37:33.893960+0000","last_undegraded":"2024-09-13T08:37:33.893960+0000","last_fullsized":"2024-09-13T08:37:33.893960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894451+0000","last_change":"2024-09-13T08:37:34.908186+0000","last_active":"2024-09-13T08:37:35.894451+0000","last_peered":"2024-09-13T08:37:35.894451+0000","last_clean":"2024-09-13T08:37:35.894451+0000","last_became_active":"2024-09-13T08:37:34.907886+0000","last_became_peered":"2024-09-13T08:37:34.907886+0000","last_unstale":"2024-09-13T08:37:35.894451+0000","last_undegraded":"2024-09-13T08:37:35.894451+0000","last_fullsized":"2024-09-13T08:37:35.894451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:11:14.712673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.897126+0000","last_change":"2024-09-13T08:37:33.890556+0000","last_active":"2024-09-13T08:37:33.897126+0000","last_peered":"2024-09-13T08:37:33.897126+0000","last_clean":"2024-09-13T08:37:33.897126+0000","last_became_active":"2024-09-13T08:37:33.890252+0000","last_became_peered":"2024-09-13T08:37:33.890252+0000","last_unstale":"2024-09-13T08:37:33.897126+0000","last_undegraded":"2024-09-13T08:37:33.897126+0000","last_fullsized":"2024-09-13T08:37:33.897126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875140+0000","last_change":"2024-09-13T08:37:33.875140+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875140+0000","last_undegraded":"2024-09-13T08:37:33.875140+0000","last_fullsized":"2024-09-13T08:37:33.875140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.895720+0000","last_change":"2024-09-13T08:37:33.889192+0000","last_active":"2024-09-13T08:37:33.895720+0000","last_peered":"2024-09-13T08:37:33.895720+0000","last_clean":"2024-09-13T08:37:33.895720+0000","last_became_active":"2024-09-13T08:37:33.888556+0000","last_became_peered":"2024-09-13T08:37:33.888556+0000","last_unstale":"2024-09-13T08:37:33.895720+0000","last_undegraded":"2024-09-13T08:37:33.895720+0000","last_fullsized":"2024-09-13T08:37:33.895720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872530+0000","last_change":"2024-09-13T08:37:33.872530+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872530+0000","last_undegraded":"2024-09-13T08:37:33.872530+0000","last_fullsized":"2024-09-13T08:37:33.872530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894811+0000","last_change":"2024-09-13T08:37:33.889699+0000","last_active":"2024-09-13T08:37:33.894811+0000","last_peered":"2024-09-13T08:37:33.894811+0000","last_clean":"2024-09-13T08:37:33.894811+0000","last_became_active":"2024-09-13T08:37:33.889224+0000","last_became_peered":"2024-09-13T08:37:33.889224+0000","last_unstale":"2024-09-13T08:37:33.894811+0000","last_undegraded":"2024-09-13T08:37:33.894811+0000","last_fullsized":"2024-09-13T08:37:33.894811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.871796+0000","last_change":"2024-09-13T08:37:33.871796+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.871796+0000","last_undegraded":"2024-09-13T08:37:33.871796+0000","last_fullsized":"2024-09-13T08:37:33.871796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900269+0000","last_change":"2024-09-13T08:37:33.893542+0000","last_active":"2024-09-13T08:37:33.900269+0000","last_peered":"2024-09-13T08:37:33.900269+0000","last_clean":"2024-09-13T08:37:33.900269+0000","last_became_active":"2024-09-13T08:37:33.893231+0000","last_became_peered":"2024-09-13T08:37:33.893231+0000","last_unstale":"2024-09-13T08:37:33.900269+0000","last_undegraded":"2024-09-13T08:37:33.900269+0000","last_fullsized":"2024-09-13T08:37:33.900269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:36.540294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895292+0000","last_change":"2024-09-13T08:37:34.901999+0000","last_active":"2024-09-13T08:37:35.895292+0000","last_peered":"2024-09-13T08:37:35.895292+0000","last_clean":"2024-09-13T08:37:35.895292+0000","last_became_active":"2024-09-13T08:37:34.901765+0000","last_became_peered":"2024-09-13T08:37:34.901765+0000","last_unstale":"2024-09-13T08:37:35.895292+0000","last_undegraded":"2024-09-13T08:37:35.895292+0000","last_fullsized":"2024-09-13T08:37:35.895292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:12.422693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.903257+0000","last_change":"2024-09-13T08:37:33.898217+0000","last_active":"2024-09-13T08:37:33.903257+0000","last_peered":"2024-09-13T08:37:33.903257+0000","last_clean":"2024-09-13T08:37:33.903257+0000","last_became_active":"2024-09-13T08:37:33.897954+0000","last_became_peered":"2024-09-13T08:37:33.897954+0000","last_unstale":"2024-09-13T08:37:33.903257+0000","last_undegraded":"2024-09-13T08:37:33.903257+0000","last_fullsized":"2024-09-13T08:37:33.903257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.879326+0000","last_change":"2024-09-13T08:37:33.879326+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.879326+0000","last_undegraded":"2024-09-13T08:37:33.879326+0000","last_fullsized":"2024-09-13T08:37:33.879326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.901668+0000","last_change":"2024-09-13T08:37:33.897139+0000","last_active":"2024-09-13T08:37:33.901668+0000","last_peered":"2024-09-13T08:37:33.901668+0000","last_clean":"2024-09-13T08:37:33.901668+0000","last_became_active":"2024-09-13T08:37:33.896576+0000","last_became_peered":"2024-09-13T08:37:33.896576+0000","last_unstale":"2024-09-13T08:37:33.901668+0000","last_undegraded":"2024-09-13T08:37:33.901668+0000","last_fullsized":"2024-09-13T08:37:33.901668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:05.995257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.879322+0000","last_change":"2024-09-13T08:37:33.879322+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.879322+0000","last_undegraded":"2024-09-13T08:37:33.879322+0000","last_fullsized":"2024-09-13T08:37:33.879322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900384+0000","last_change":"2024-09-13T08:37:33.893088+0000","last_active":"2024-09-13T08:37:33.900384+0000","last_peered":"2024-09-13T08:37:33.900384+0000","last_clean":"2024-09-13T08:37:33.900384+0000","last_became_active":"2024-09-13T08:37:33.892731+0000","last_became_peered":"2024-09-13T08:37:33.892731+0000","last_unstale":"2024-09-13T08:37:33.900384+0000","last_undegraded":"2024-09-13T08:37:33.900384+0000","last_fullsized":"2024-09-13T08:37:33.900384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870235+0000","last_change":"2024-09-13T08:37:33.870235+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870235+0000","last_undegraded":"2024-09-13T08:37:33.870235+0000","last_fullsized":"2024-09-13T08:37:33.870235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.903271+0000","last_change":"2024-09-13T08:37:33.898275+0000","last_active":"2024-09-13T08:37:33.903271+0000","last_peered":"2024-09-13T08:37:33.903271+0000","last_clean":"2024-09-13T08:37:33.903271+0000","last_became_active":"2024-09-13T08:37:33.897960+0000","last_became_peered":"2024-09-13T08:37:33.897960+0000","last_unstale":"2024-09-13T08:37:33.903271+0000","last_undegraded":"2024-09-13T08:37:33.903271+0000","last_fullsized":"2024-09-13T08:37:33.903271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874612+0000","last_change":"2024-09-13T08:37:33.874612+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874612+0000","last_undegraded":"2024-09-13T08:37:33.874612+0000","last_fullsized":"2024-09-13T08:37:33.874612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894655+0000","last_change":"2024-09-13T08:37:33.891510+0000","last_active":"2024-09-13T08:37:35.894655+0000","last_peered":"2024-09-13T08:37:35.894655+0000","last_clean":"2024-09-13T08:37:35.894655+0000","last_became_active":"2024-09-13T08:37:33.890774+0000","last_became_peered":"2024-09-13T08:37:33.890774+0000","last_unstale":"2024-09-13T08:37:35.894655+0000","last_undegraded":"2024-09-13T08:37:35.894655+0000","last_fullsized":"2024-09-13T08:37:35.894655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:25.370931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876319+0000","last_change":"2024-09-13T08:37:33.876319+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876319+0000","last_undegraded":"2024-09-13T08:37:33.876319+0000","last_fullsized":"2024-09-13T08:37:33.876319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.893774+0000","last_change":"2024-09-13T08:37:33.887243+0000","last_active":"2024-09-13T08:37:33.893774+0000","last_peered":"2024-09-13T08:37:33.893774+0000","last_clean":"2024-09-13T08:37:33.893774+0000","last_became_active":"2024-09-13T08:37:33.886882+0000","last_became_peered":"2024-09-13T08:37:33.886882+0000","last_unstale":"2024-09-13T08:37:33.893774+0000","last_undegraded":"2024-09-13T08:37:33.893774+0000","last_fullsized":"2024-09-13T08:37:33.893774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.871566+0000","last_change":"2024-09-13T08:37:33.871566+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.871566+0000","last_undegraded":"2024-09-13T08:37:33.871566+0000","last_fullsized":"2024-09-13T08:37:33.871566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875076+0000","last_change":"2024-09-13T08:37:33.875076+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875076+0000","last_undegraded":"2024-09-13T08:37:33.875076+0000","last_fullsized":"2024-09-13T08:37:33.875076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.904025+0000","last_change":"2024-09-13T08:37:33.900929+0000","last_active":"2024-09-13T08:37:33.904025+0000","last_peered":"2024-09-13T08:37:33.904025+0000","last_clean":"2024-09-13T08:37:33.904025+0000","last_became_active":"2024-09-13T08:37:33.899776+0000","last_became_peered":"2024-09-13T08:37:33.899776+0000","last_unstale":"2024-09-13T08:37:33.904025+0000","last_undegraded":"2024-09-13T08:37:33.904025+0000","last_fullsized":"2024-09-13T08:37:33.904025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873649+0000","last_change":"2024-09-13T08:37:33.873649+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873649+0000","last_undegraded":"2024-09-13T08:37:33.873649+0000","last_fullsized":"2024-09-13T08:37:33.873649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900424+0000","last_change":"2024-09-13T08:37:33.894985+0000","last_active":"2024-09-13T08:37:33.900424+0000","last_peered":"2024-09-13T08:37:33.900424+0000","last_clean":"2024-09-13T08:37:33.900424+0000","last_became_active":"2024-09-13T08:37:33.894087+0000","last_became_peered":"2024-09-13T08:37:33.894087+0000","last_unstale":"2024-09-13T08:37:33.900424+0000","last_undegraded":"2024-09-13T08:37:33.900424+0000","last_fullsized":"2024-09-13T08:37:33.900424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876978+0000","last_change":"2024-09-13T08:37:33.876978+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876978+0000","last_undegraded":"2024-09-13T08:37:33.876978+0000","last_fullsized":"2024-09-13T08:37:33.876978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.892622+0000","last_change":"2024-09-13T08:37:33.887848+0000","last_active":"2024-09-13T08:37:33.892622+0000","last_peered":"2024-09-13T08:37:33.892622+0000","last_clean":"2024-09-13T08:37:33.892622+0000","last_became_active":"2024-09-13T08:37:33.887508+0000","last_became_peered":"2024-09-13T08:37:33.887508+0000","last_unstale":"2024-09-13T08:37:33.892622+0000","last_undegraded":"2024-09-13T08:37:33.892622+0000","last_fullsized":"2024-09-13T08:37:33.892622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875975+0000","last_change":"2024-09-13T08:37:33.875975+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875975+0000","last_undegraded":"2024-09-13T08:37:33.875975+0000","last_fullsized":"2024-09-13T08:37:33.875975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.897760+0000","last_change":"2024-09-13T08:37:33.894082+0000","last_active":"2024-09-13T08:37:33.897760+0000","last_peered":"2024-09-13T08:37:33.897760+0000","last_clean":"2024-09-13T08:37:33.897760+0000","last_became_active":"2024-09-13T08:37:33.893820+0000","last_became_peered":"2024-09-13T08:37:33.893820+0000","last_unstale":"2024-09-13T08:37:33.897760+0000","last_undegraded":"2024-09-13T08:37:33.897760+0000","last_fullsized":"2024-09-13T08:37:33.897760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":653577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:37:39.441 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:37:39.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.627+0000 7fef59200640 1 -- 172.21.15.44:0/3032609835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef38003c70 msgr2=0x7fef38004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.627+0000 7fef59200640 1 --2- 172.21.15.44:0/3032609835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef38003c70 0x7fef38004070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fef44001860 tx=0x7fef4401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.627+0000 7fef53e00640 1 -- 172.21.15.44:0/3032609835 reap_dead start 2024-09-13T08:37:39.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.627+0000 7fef59200640 1 -- 172.21.15.44:0/3032609835 shutdown_connections 2024-09-13T08:37:39.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.627+0000 7fef59200640 1 -- 172.21.15.44:0/3032609835 >> 172.21.15.44:0/3032609835 conn(0x7fef5409aa10 msgr2=0x7fef5409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:39.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.627+0000 7fef59200640 1 -- 172.21.15.44:0/3032609835 shutdown_connections 2024-09-13T08:37:39.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.627+0000 7fef59200640 1 -- 172.21.15.44:0/3032609835 wait complete. 2024-09-13T08:37:39.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.627+0000 7fef59200640 1 Processor -- start 2024-09-13T08:37:39.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.627+0000 7fef59200640 1 -- start start 2024-09-13T08:37:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef59200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fef38003c70 0x7fef54146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef59200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef540c7ba0 0x7fef5415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef59200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef540c8020 0x7fef5415e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef59200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fef54167f10 con 0x7fef540c8020 2024-09-13T08:37:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef59200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef54167d90 con 0x7fef540c7ba0 2024-09-13T08:37:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef59200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fef54168090 con 0x7fef38003c70 2024-09-13T08:37:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef540c8020 0x7fef5415e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef540c8020 0x7fef5415e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60696/0 (socket says 172.21.15.44:60696) 2024-09-13T08:37:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef53e00640 1 -- 172.21.15.44:0/810280808 learned_addr learned my addr 172.21.15.44:0/810280808 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef53400640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fef38003c70 0x7fef54146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef4be00640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef540c7ba0 0x7fef5415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef53e00640 1 -- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fef38003c70 msgr2=0x7fef54146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef53e00640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fef38003c70 0x7fef54146f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef53e00640 1 -- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef540c7ba0 msgr2=0x7fef5415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef53e00640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef540c7ba0 0x7fef5415dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef53e00640 1 -- 172.21.15.44:0/810280808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef540c63a0 con 0x7fef540c8020 2024-09-13T08:37:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef53e00640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef540c8020 0x7fef5415e160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef54074520 tx=0x7fef4c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:39.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef4b400640 1 -- 172.21.15.44:0/810280808 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c004b30 con 0x7fef540c8020 2024-09-13T08:37:39.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef4b400640 1 -- 172.21.15.44:0/810280808 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef4c004cd0 con 0x7fef540c8020 2024-09-13T08:37:39.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef4b400640 1 -- 172.21.15.44:0/810280808 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c003be0 con 0x7fef540c8020 2024-09-13T08:37:39.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef59200640 1 -- 172.21.15.44:0/810280808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef540c6680 con 0x7fef540c8020 2024-09-13T08:37:39.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef59200640 1 -- 172.21.15.44:0/810280808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef540c73a0 con 0x7fef540c8020 2024-09-13T08:37:39.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.631+0000 7fef59200640 1 -- 172.21.15.44:0/810280808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef540b2590 con 0x7fef540c8020 2024-09-13T08:37:39.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.635+0000 7fef4b400640 1 -- 172.21.15.44:0/810280808 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fef4c004030 con 0x7fef540c8020 2024-09-13T08:37:39.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.635+0000 7fef4b400640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef30072ec0 0x7fef30075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:39.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.635+0000 7fef4b400640 1 -- 172.21.15.44:0/810280808 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fef4c0946a0 con 0x7fef540c8020 2024-09-13T08:37:39.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.635+0000 7fef53400640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef30072ec0 0x7fef30075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:39.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.635+0000 7fef53400640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef30072ec0 0x7fef30075380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef44001830 tx=0x7fef44019720 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.639+0000 7fef4b400640 1 -- 172.21.15.44:0/810280808 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fef4c05f290 con 0x7fef540c8020 2024-09-13T08:37:39.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.811+0000 7fef59200640 1 -- 172.21.15.44:0/810280808 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef540cc0f0 con 0x7fef30072ec0 2024-09-13T08:37:39.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.815+0000 7fef4b400640 1 -- 172.21.15.44:0/810280808 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247213 (secure 0 0 0) 0x7fef440fa5c0 con 0x7fef30072ec0 2024-09-13T08:37:39.824 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:39.825 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.819+0000 7fef59200640 1 -- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef30072ec0 msgr2=0x7fef30075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.819+0000 7fef59200640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef30072ec0 0x7fef30075380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef44001830 tx=0x7fef44019720 comp rx=0 tx=0).stop 2024-09-13T08:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.819+0000 7fef59200640 1 -- 172.21.15.44:0/810280808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef540c8020 msgr2=0x7fef5415e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.819+0000 7fef59200640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef540c8020 0x7fef5415e160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef54074520 tx=0x7fef4c00f3c0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.819+0000 7fef59200640 1 -- 172.21.15.44:0/810280808 shutdown_connections 2024-09-13T08:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.819+0000 7fef59200640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef30072ec0 0x7fef30075380 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.819+0000 7fef59200640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef540c8020 0x7fef5415e160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.819+0000 7fef59200640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef540c7ba0 0x7fef5415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.819+0000 7fef59200640 1 --2- 172.21.15.44:0/810280808 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fef38003c70 0x7fef54146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:39.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.819+0000 7fef59200640 1 -- 172.21.15.44:0/810280808 >> 172.21.15.44:0/810280808 conn(0x7fef5409aa10 msgr2=0x7fef5409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:39.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.823+0000 7fef59200640 1 -- 172.21.15.44:0/810280808 shutdown_connections 2024-09-13T08:37:39.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:39.823+0000 7fef59200640 1 -- 172.21.15.44:0/810280808 wait complete. 2024-09-13T08:37:39.842 INFO:tasks.ceph.mds.d.smithi129.stderr:2024-09-13T08:37:39.842+0000 7ff048c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:37:39.845 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-13T08:37:38.511365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218896,"kb_used_data":2660,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755824,"statfs":{"total":773094113280,"available":772869963776,"internally_reserved":0,"allocated":2723840,"data_stored":1668108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.654417"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.888188+0000","last_change":"2024-09-13T08:37:33.888188+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.888188+0000","last_undegraded":"2024-09-13T08:37:33.888188+0000","last_fullsized":"2024-09-13T08:37:33.888188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876585+0000","last_change":"2024-09-13T08:37:33.876585+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876585+0000","last_undegraded":"2024-09-13T08:37:33.876585+0000","last_fullsized":"2024-09-13T08:37:33.876585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873515+0000","last_change":"2024-09-13T08:37:33.873515+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873515+0000","last_undegraded":"2024-09-13T08:37:33.873515+0000","last_fullsized":"2024-09-13T08:37:33.873515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870231+0000","last_change":"2024-09-13T08:37:33.870231+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870231+0000","last_undegraded":"2024-09-13T08:37:33.870231+0000","last_fullsized":"2024-09-13T08:37:33.870231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.887043+0000","last_change":"2024-09-13T08:37:33.887043+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.887043+0000","last_undegraded":"2024-09-13T08:37:33.887043+0000","last_fullsized":"2024-09-13T08:37:33.887043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.869808+0000","last_change":"2024-09-13T08:37:33.869808+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.869808+0000","last_undegraded":"2024-09-13T08:37:33.869808+0000","last_fullsized":"2024-09-13T08:37:33.869808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.871829+0000","last_change":"2024-09-13T08:37:33.871829+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.871829+0000","last_undegraded":"2024-09-13T08:37:33.871829+0000","last_fullsized":"2024-09-13T08:37:33.871829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875627+0000","last_change":"2024-09-13T08:37:33.875627+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875627+0000","last_undegraded":"2024-09-13T08:37:33.875627+0000","last_fullsized":"2024-09-13T08:37:33.875627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874425+0000","last_change":"2024-09-13T08:37:33.874425+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874425+0000","last_undegraded":"2024-09-13T08:37:33.874425+0000","last_fullsized":"2024-09-13T08:37:33.874425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874047+0000","last_change":"2024-09-13T08:37:33.874047+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874047+0000","last_undegraded":"2024-09-13T08:37:33.874047+0000","last_fullsized":"2024-09-13T08:37:33.874047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.881267+0000","last_change":"2024-09-13T08:37:33.881267+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.881267+0000","last_undegraded":"2024-09-13T08:37:33.881267+0000","last_fullsized":"2024-09-13T08:37:33.881267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.877798+0000","last_change":"2024-09-13T08:37:33.877798+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.877798+0000","last_undegraded":"2024-09-13T08:37:33.877798+0000","last_fullsized":"2024-09-13T08:37:33.877798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894980+0000","last_change":"2024-09-13T08:37:34.903037+0000","last_active":"2024-09-13T08:37:35.894980+0000","last_peered":"2024-09-13T08:37:35.894980+0000","last_clean":"2024-09-13T08:37:35.894980+0000","last_became_active":"2024-09-13T08:37:34.894805+0000","last_became_peered":"2024-09-13T08:37:34.894805+0000","last_unstale":"2024-09-13T08:37:35.894980+0000","last_undegraded":"2024-09-13T08:37:35.894980+0000","last_fullsized":"2024-09-13T08:37:35.894980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:49.712241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894573+0000","last_change":"2024-09-13T08:37:34.901603+0000","last_active":"2024-09-13T08:37:35.894573+0000","last_peered":"2024-09-13T08:37:35.894573+0000","last_clean":"2024-09-13T08:37:35.894573+0000","last_became_active":"2024-09-13T08:37:34.899884+0000","last_became_peered":"2024-09-13T08:37:34.899884+0000","last_unstale":"2024-09-13T08:37:35.894573+0000","last_undegraded":"2024-09-13T08:37:35.894573+0000","last_fullsized":"2024-09-13T08:37:35.894573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:05.260600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874667+0000","last_change":"2024-09-13T08:37:33.874667+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874667+0000","last_undegraded":"2024-09-13T08:37:33.874667+0000","last_fullsized":"2024-09-13T08:37:33.874667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876422+0000","last_change":"2024-09-13T08:37:33.876422+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876422+0000","last_undegraded":"2024-09-13T08:37:33.876422+0000","last_fullsized":"2024-09-13T08:37:33.876422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874801+0000","last_change":"2024-09-13T08:37:33.874801+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874801+0000","last_undegraded":"2024-09-13T08:37:33.874801+0000","last_fullsized":"2024-09-13T08:37:33.874801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.878051+0000","last_change":"2024-09-13T08:37:33.878051+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.878051+0000","last_undegraded":"2024-09-13T08:37:33.878051+0000","last_fullsized":"2024-09-13T08:37:33.878051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875550+0000","last_change":"2024-09-13T08:37:33.875550+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875550+0000","last_undegraded":"2024-09-13T08:37:33.875550+0000","last_fullsized":"2024-09-13T08:37:33.875550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875619+0000","last_change":"2024-09-13T08:37:33.875619+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875619+0000","last_undegraded":"2024-09-13T08:37:33.875619+0000","last_fullsized":"2024-09-13T08:37:33.875619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.881053+0000","last_change":"2024-09-13T08:37:33.881053+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.881053+0000","last_undegraded":"2024-09-13T08:37:33.881053+0000","last_fullsized":"2024-09-13T08:37:33.881053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875830+0000","last_change":"2024-09-13T08:37:33.875830+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875830+0000","last_undegraded":"2024-09-13T08:37:33.875830+0000","last_fullsized":"2024-09-13T08:37:33.875830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873389+0000","last_change":"2024-09-13T08:37:33.873389+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873389+0000","last_undegraded":"2024-09-13T08:37:33.873389+0000","last_fullsized":"2024-09-13T08:37:33.873389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875855+0000","last_change":"2024-09-13T08:37:33.875855+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875855+0000","last_undegraded":"2024-09-13T08:37:33.875855+0000","last_fullsized":"2024-09-13T08:37:33.875855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.877601+0000","last_change":"2024-09-13T08:37:33.877601+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.877601+0000","last_undegraded":"2024-09-13T08:37:33.877601+0000","last_fullsized":"2024-09-13T08:37:33.877601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.878497+0000","last_change":"2024-09-13T08:37:33.878497+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.878497+0000","last_undegraded":"2024-09-13T08:37:33.878497+0000","last_fullsized":"2024-09-13T08:37:33.878497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895541+0000","last_change":"2024-09-13T08:37:34.902546+0000","last_active":"2024-09-13T08:37:35.895541+0000","last_peered":"2024-09-13T08:37:35.895541+0000","last_clean":"2024-09-13T08:37:35.895541+0000","last_became_active":"2024-09-13T08:37:34.901841+0000","last_became_peered":"2024-09-13T08:37:34.901841+0000","last_unstale":"2024-09-13T08:37:35.895541+0000","last_undegraded":"2024-09-13T08:37:35.895541+0000","last_fullsized":"2024-09-13T08:37:35.895541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:40.635073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895078+0000","last_change":"2024-09-13T08:37:34.907277+0000","last_active":"2024-09-13T08:37:35.895078+0000","last_peered":"2024-09-13T08:37:35.895078+0000","last_clean":"2024-09-13T08:37:35.895078+0000","last_became_active":"2024-09-13T08:37:34.906984+0000","last_became_peered":"2024-09-13T08:37:34.906984+0000","last_unstale":"2024-09-13T08:37:35.895078+0000","last_undegraded":"2024-09-13T08:37:35.895078+0000","last_fullsized":"2024-09-13T08:37:35.895078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:23.664559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872673+0000","last_change":"2024-09-13T08:37:33.872673+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872673+0000","last_undegraded":"2024-09-13T08:37:33.872673+0000","last_fullsized":"2024-09-13T08:37:33.872673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874060+0000","last_change":"2024-09-13T08:37:33.874060+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874060+0000","last_undegraded":"2024-09-13T08:37:33.874060+0000","last_fullsized":"2024-09-13T08:37:33.874060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874453+0000","last_change":"2024-09-13T08:37:33.874453+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874453+0000","last_undegraded":"2024-09-13T08:37:33.874453+0000","last_fullsized":"2024-09-13T08:37:33.874453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872506+0000","last_change":"2024-09-13T08:37:33.872506+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872506+0000","last_undegraded":"2024-09-13T08:37:33.872506+0000","last_fullsized":"2024-09-13T08:37:33.872506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875197+0000","last_change":"2024-09-13T08:37:33.875197+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875197+0000","last_undegraded":"2024-09-13T08:37:33.875197+0000","last_fullsized":"2024-09-13T08:37:33.875197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.892711+0000","last_change":"2024-09-13T08:37:33.888852+0000","last_active":"2024-09-13T08:37:33.892711+0000","last_peered":"2024-09-13T08:37:33.892711+0000","last_clean":"2024-09-13T08:37:33.892711+0000","last_became_active":"2024-09-13T08:37:33.888409+0000","last_became_peered":"2024-09-13T08:37:33.888409+0000","last_unstale":"2024-09-13T08:37:33.892711+0000","last_undegraded":"2024-09-13T08:37:33.892711+0000","last_fullsized":"2024-09-13T08:37:33.892711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872451+0000","last_change":"2024-09-13T08:37:33.872451+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872451+0000","last_undegraded":"2024-09-13T08:37:33.872451+0000","last_fullsized":"2024-09-13T08:37:33.872451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.895745+0000","last_change":"2024-09-13T08:37:33.892308+0000","last_active":"2024-09-13T08:37:33.895745+0000","last_peered":"2024-09-13T08:37:33.895745+0000","last_clean":"2024-09-13T08:37:33.895745+0000","last_became_active":"2024-09-13T08:37:33.891875+0000","last_became_peered":"2024-09-13T08:37:33.891875+0000","last_unstale":"2024-09-13T08:37:33.895745+0000","last_undegraded":"2024-09-13T08:37:33.895745+0000","last_fullsized":"2024-09-13T08:37:33.895745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:22.150995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894251+0000","last_change":"2024-09-13T08:37:33.896561+0000","last_active":"2024-09-13T08:37:35.894251+0000","last_peered":"2024-09-13T08:37:35.894251+0000","last_clean":"2024-09-13T08:37:35.894251+0000","last_became_active":"2024-09-13T08:37:33.896314+0000","last_became_peered":"2024-09-13T08:37:33.896314+0000","last_unstale":"2024-09-13T08:37:35.894251+0000","last_undegraded":"2024-09-13T08:37:35.894251+0000","last_fullsized":"2024-09-13T08:37:35.894251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:28.728254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872534+0000","last_change":"2024-09-13T08:37:33.872534+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872534+0000","last_undegraded":"2024-09-13T08:37:33.872534+0000","last_fullsized":"2024-09-13T08:37:33.872534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.889489+0000","last_change":"2024-09-13T08:37:33.883156+0000","last_active":"2024-09-13T08:37:33.889489+0000","last_peered":"2024-09-13T08:37:33.889489+0000","last_clean":"2024-09-13T08:37:33.889489+0000","last_became_active":"2024-09-13T08:37:33.882800+0000","last_became_peered":"2024-09-13T08:37:33.882800+0000","last_unstale":"2024-09-13T08:37:33.889489+0000","last_undegraded":"2024-09-13T08:37:33.889489+0000","last_fullsized":"2024-09-13T08:37:33.889489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.879388+0000","last_change":"2024-09-13T08:37:33.879388+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.879388+0000","last_undegraded":"2024-09-13T08:37:33.879388+0000","last_fullsized":"2024-09-13T08:37:33.879388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894588+0000","last_change":"2024-09-13T08:37:33.891385+0000","last_active":"2024-09-13T08:37:33.894588+0000","last_peered":"2024-09-13T08:37:33.894588+0000","last_clean":"2024-09-13T08:37:33.894588+0000","last_became_active":"2024-09-13T08:37:33.891112+0000","last_became_peered":"2024-09-13T08:37:33.891112+0000","last_unstale":"2024-09-13T08:37:33.894588+0000","last_undegraded":"2024-09-13T08:37:33.894588+0000","last_fullsized":"2024-09-13T08:37:33.894588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870875+0000","last_change":"2024-09-13T08:37:33.870875+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870875+0000","last_undegraded":"2024-09-13T08:37:33.870875+0000","last_fullsized":"2024-09-13T08:37:33.870875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.904015+0000","last_change":"2024-09-13T08:37:33.899147+0000","last_active":"2024-09-13T08:37:33.904015+0000","last_peered":"2024-09-13T08:37:33.904015+0000","last_clean":"2024-09-13T08:37:33.904015+0000","last_became_active":"2024-09-13T08:37:33.896929+0000","last_became_peered":"2024-09-13T08:37:33.896929+0000","last_unstale":"2024-09-13T08:37:33.904015+0000","last_undegraded":"2024-09-13T08:37:33.904015+0000","last_fullsized":"2024-09-13T08:37:33.904015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876422+0000","last_change":"2024-09-13T08:37:33.876422+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876422+0000","last_undegraded":"2024-09-13T08:37:33.876422+0000","last_fullsized":"2024-09-13T08:37:33.876422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900065+0000","last_change":"2024-09-13T08:37:33.892872+0000","last_active":"2024-09-13T08:37:33.900065+0000","last_peered":"2024-09-13T08:37:33.900065+0000","last_clean":"2024-09-13T08:37:33.900065+0000","last_became_active":"2024-09-13T08:37:33.892579+0000","last_became_peered":"2024-09-13T08:37:33.892579+0000","last_unstale":"2024-09-13T08:37:33.900065+0000","last_undegraded":"2024-09-13T08:37:33.900065+0000","last_fullsized":"2024-09-13T08:37:33.900065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874425+0000","last_change":"2024-09-13T08:37:33.874425+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874425+0000","last_undegraded":"2024-09-13T08:37:33.874425+0000","last_fullsized":"2024-09-13T08:37:33.874425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894771+0000","last_change":"2024-09-13T08:37:33.891105+0000","last_active":"2024-09-13T08:37:33.894771+0000","last_peered":"2024-09-13T08:37:33.894771+0000","last_clean":"2024-09-13T08:37:33.894771+0000","last_became_active":"2024-09-13T08:37:33.890783+0000","last_became_peered":"2024-09-13T08:37:33.890783+0000","last_unstale":"2024-09-13T08:37:33.894771+0000","last_undegraded":"2024-09-13T08:37:33.894771+0000","last_fullsized":"2024-09-13T08:37:33.894771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:17.466021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.878005+0000","last_change":"2024-09-13T08:37:33.878005+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.878005+0000","last_undegraded":"2024-09-13T08:37:33.878005+0000","last_fullsized":"2024-09-13T08:37:33.878005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.905653+0000","last_change":"2024-09-13T08:37:33.902610+0000","last_active":"2024-09-13T08:37:33.905653+0000","last_peered":"2024-09-13T08:37:33.905653+0000","last_clean":"2024-09-13T08:37:33.905653+0000","last_became_active":"2024-09-13T08:37:33.902275+0000","last_became_peered":"2024-09-13T08:37:33.902275+0000","last_unstale":"2024-09-13T08:37:33.905653+0000","last_undegraded":"2024-09-13T08:37:33.905653+0000","last_fullsized":"2024-09-13T08:37:33.905653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894679+0000","last_change":"2024-09-13T08:37:34.904983+0000","last_active":"2024-09-13T08:37:35.894679+0000","last_peered":"2024-09-13T08:37:35.894679+0000","last_clean":"2024-09-13T08:37:35.894679+0000","last_became_active":"2024-09-13T08:37:34.904618+0000","last_became_peered":"2024-09-13T08:37:34.904618+0000","last_unstale":"2024-09-13T08:37:35.894679+0000","last_undegraded":"2024-09-13T08:37:35.894679+0000","last_fullsized":"2024-09-13T08:37:35.894679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:30.070140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.895692+0000","last_change":"2024-09-13T08:37:33.888874+0000","last_active":"2024-09-13T08:37:33.895692+0000","last_peered":"2024-09-13T08:37:33.895692+0000","last_clean":"2024-09-13T08:37:33.895692+0000","last_became_active":"2024-09-13T08:37:33.888111+0000","last_became_peered":"2024-09-13T08:37:33.888111+0000","last_unstale":"2024-09-13T08:37:33.895692+0000","last_undegraded":"2024-09-13T08:37:33.895692+0000","last_fullsized":"2024-09-13T08:37:33.895692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874869+0000","last_change":"2024-09-13T08:37:33.874869+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874869+0000","last_undegraded":"2024-09-13T08:37:33.874869+0000","last_fullsized":"2024-09-13T08:37:33.874869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.899257+0000","last_change":"2024-09-13T08:37:33.892008+0000","last_active":"2024-09-13T08:37:33.899257+0000","last_peered":"2024-09-13T08:37:33.899257+0000","last_clean":"2024-09-13T08:37:33.899257+0000","last_became_active":"2024-09-13T08:37:33.891758+0000","last_became_peered":"2024-09-13T08:37:33.891758+0000","last_unstale":"2024-09-13T08:37:33.899257+0000","last_undegraded":"2024-09-13T08:37:33.899257+0000","last_fullsized":"2024-09-13T08:37:33.899257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874510+0000","last_change":"2024-09-13T08:37:33.874510+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874510+0000","last_undegraded":"2024-09-13T08:37:33.874510+0000","last_fullsized":"2024-09-13T08:37:33.874510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894614+0000","last_change":"2024-09-13T08:37:33.888556+0000","last_active":"2024-09-13T08:37:33.894614+0000","last_peered":"2024-09-13T08:37:33.894614+0000","last_clean":"2024-09-13T08:37:33.894614+0000","last_became_active":"2024-09-13T08:37:33.887997+0000","last_became_peered":"2024-09-13T08:37:33.887997+0000","last_unstale":"2024-09-13T08:37:33.894614+0000","last_undegraded":"2024-09-13T08:37:33.894614+0000","last_fullsized":"2024-09-13T08:37:33.894614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872360+0000","last_change":"2024-09-13T08:37:33.872360+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872360+0000","last_undegraded":"2024-09-13T08:37:33.872360+0000","last_fullsized":"2024-09-13T08:37:33.872360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900661+0000","last_change":"2024-09-13T08:37:33.895134+0000","last_active":"2024-09-13T08:37:33.900661+0000","last_peered":"2024-09-13T08:37:33.900661+0000","last_clean":"2024-09-13T08:37:33.900661+0000","last_became_active":"2024-09-13T08:37:33.894893+0000","last_became_peered":"2024-09-13T08:37:33.894893+0000","last_unstale":"2024-09-13T08:37:33.900661+0000","last_undegraded":"2024-09-13T08:37:33.900661+0000","last_fullsized":"2024-09-13T08:37:33.900661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870305+0000","last_change":"2024-09-13T08:37:33.870305+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870305+0000","last_undegraded":"2024-09-13T08:37:33.870305+0000","last_fullsized":"2024-09-13T08:37:33.870305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894667+0000","last_change":"2024-09-13T08:37:33.888156+0000","last_active":"2024-09-13T08:37:33.894667+0000","last_peered":"2024-09-13T08:37:33.894667+0000","last_clean":"2024-09-13T08:37:33.894667+0000","last_became_active":"2024-09-13T08:37:33.886607+0000","last_became_peered":"2024-09-13T08:37:33.886607+0000","last_unstale":"2024-09-13T08:37:33.894667+0000","last_undegraded":"2024-09-13T08:37:33.894667+0000","last_fullsized":"2024-09-13T08:37:33.894667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874608+0000","last_change":"2024-09-13T08:37:33.874608+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874608+0000","last_undegraded":"2024-09-13T08:37:33.874608+0000","last_fullsized":"2024-09-13T08:37:33.874608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.891315+0000","last_change":"2024-09-13T08:37:33.885405+0000","last_active":"2024-09-13T08:37:33.891315+0000","last_peered":"2024-09-13T08:37:33.891315+0000","last_clean":"2024-09-13T08:37:33.891315+0000","last_became_active":"2024-09-13T08:37:33.885048+0000","last_became_peered":"2024-09-13T08:37:33.885048+0000","last_unstale":"2024-09-13T08:37:33.891315+0000","last_undegraded":"2024-09-13T08:37:33.891315+0000","last_fullsized":"2024-09-13T08:37:33.891315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873750+0000","last_change":"2024-09-13T08:37:33.873750+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873750+0000","last_undegraded":"2024-09-13T08:37:33.873750+0000","last_fullsized":"2024-09-13T08:37:33.873750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895446+0000","last_change":"2024-09-13T08:37:33.902815+0000","last_active":"2024-09-13T08:37:35.895446+0000","last_peered":"2024-09-13T08:37:35.895446+0000","last_clean":"2024-09-13T08:37:35.895446+0000","last_became_active":"2024-09-13T08:37:33.902490+0000","last_became_peered":"2024-09-13T08:37:33.902490+0000","last_unstale":"2024-09-13T08:37:35.895446+0000","last_undegraded":"2024-09-13T08:37:35.895446+0000","last_fullsized":"2024-09-13T08:37:35.895446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:32.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872340+0000","last_change":"2024-09-13T08:37:33.872340+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872340+0000","last_undegraded":"2024-09-13T08:37:33.872340+0000","last_fullsized":"2024-09-13T08:37:33.872340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":107,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.877260+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:37:33.877260+0000","last_peered":"2024-09-13T08:37:33.877260+0000","last_clean":"2024-09-13T08:37:33.877260+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:37:33.877260+0000","last_undegraded":"2024-09-13T08:37:33.877260+0000","last_fullsized":"2024-09-13T08:37:33.877260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.893747+0000","last_change":"2024-09-13T08:37:33.883826+0000","last_active":"2024-09-13T08:37:33.893747+0000","last_peered":"2024-09-13T08:37:33.893747+0000","last_clean":"2024-09-13T08:37:33.893747+0000","last_became_active":"2024-09-13T08:37:33.883511+0000","last_became_peered":"2024-09-13T08:37:33.883511+0000","last_unstale":"2024-09-13T08:37:33.893747+0000","last_undegraded":"2024-09-13T08:37:33.893747+0000","last_fullsized":"2024-09-13T08:37:33.893747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.877074+0000","last_change":"2024-09-13T08:37:33.877074+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.877074+0000","last_undegraded":"2024-09-13T08:37:33.877074+0000","last_fullsized":"2024-09-13T08:37:33.877074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.893960+0000","last_change":"2024-09-13T08:37:33.885219+0000","last_active":"2024-09-13T08:37:33.893960+0000","last_peered":"2024-09-13T08:37:33.893960+0000","last_clean":"2024-09-13T08:37:33.893960+0000","last_became_active":"2024-09-13T08:37:33.884889+0000","last_became_peered":"2024-09-13T08:37:33.884889+0000","last_unstale":"2024-09-13T08:37:33.893960+0000","last_undegraded":"2024-09-13T08:37:33.893960+0000","last_fullsized":"2024-09-13T08:37:33.893960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894451+0000","last_change":"2024-09-13T08:37:34.908186+0000","last_active":"2024-09-13T08:37:35.894451+0000","last_peered":"2024-09-13T08:37:35.894451+0000","last_clean":"2024-09-13T08:37:35.894451+0000","last_became_active":"2024-09-13T08:37:34.907886+0000","last_became_peered":"2024-09-13T08:37:34.907886+0000","last_unstale":"2024-09-13T08:37:35.894451+0000","last_undegraded":"2024-09-13T08:37:35.894451+0000","last_fullsized":"2024-09-13T08:37:35.894451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:11:14.712673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.897126+0000","last_change":"2024-09-13T08:37:33.890556+0000","last_active":"2024-09-13T08:37:33.897126+0000","last_peered":"2024-09-13T08:37:33.897126+0000","last_clean":"2024-09-13T08:37:33.897126+0000","last_became_active":"2024-09-13T08:37:33.890252+0000","last_became_peered":"2024-09-13T08:37:33.890252+0000","last_unstale":"2024-09-13T08:37:33.897126+0000","last_undegraded":"2024-09-13T08:37:33.897126+0000","last_fullsized":"2024-09-13T08:37:33.897126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:14.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875140+0000","last_change":"2024-09-13T08:37:33.875140+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875140+0000","last_undegraded":"2024-09-13T08:37:33.875140+0000","last_fullsized":"2024-09-13T08:37:33.875140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.895720+0000","last_change":"2024-09-13T08:37:33.889192+0000","last_active":"2024-09-13T08:37:33.895720+0000","last_peered":"2024-09-13T08:37:33.895720+0000","last_clean":"2024-09-13T08:37:33.895720+0000","last_became_active":"2024-09-13T08:37:33.888556+0000","last_became_peered":"2024-09-13T08:37:33.888556+0000","last_unstale":"2024-09-13T08:37:33.895720+0000","last_undegraded":"2024-09-13T08:37:33.895720+0000","last_fullsized":"2024-09-13T08:37:33.895720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.872530+0000","last_change":"2024-09-13T08:37:33.872530+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.872530+0000","last_undegraded":"2024-09-13T08:37:33.872530+0000","last_fullsized":"2024-09-13T08:37:33.872530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.894811+0000","last_change":"2024-09-13T08:37:33.889699+0000","last_active":"2024-09-13T08:37:33.894811+0000","last_peered":"2024-09-13T08:37:33.894811+0000","last_clean":"2024-09-13T08:37:33.894811+0000","last_became_active":"2024-09-13T08:37:33.889224+0000","last_became_peered":"2024-09-13T08:37:33.889224+0000","last_unstale":"2024-09-13T08:37:33.894811+0000","last_undegraded":"2024-09-13T08:37:33.894811+0000","last_fullsized":"2024-09-13T08:37:33.894811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.871796+0000","last_change":"2024-09-13T08:37:33.871796+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.871796+0000","last_undegraded":"2024-09-13T08:37:33.871796+0000","last_fullsized":"2024-09-13T08:37:33.871796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900269+0000","last_change":"2024-09-13T08:37:33.893542+0000","last_active":"2024-09-13T08:37:33.900269+0000","last_peered":"2024-09-13T08:37:33.900269+0000","last_clean":"2024-09-13T08:37:33.900269+0000","last_became_active":"2024-09-13T08:37:33.893231+0000","last_became_peered":"2024-09-13T08:37:33.893231+0000","last_unstale":"2024-09-13T08:37:33.900269+0000","last_undegraded":"2024-09-13T08:37:33.900269+0000","last_fullsized":"2024-09-13T08:37:33.900269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:36.540294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895292+0000","last_change":"2024-09-13T08:37:34.901999+0000","last_active":"2024-09-13T08:37:35.895292+0000","last_peered":"2024-09-13T08:37:35.895292+0000","last_clean":"2024-09-13T08:37:35.895292+0000","last_became_active":"2024-09-13T08:37:34.901765+0000","last_became_peered":"2024-09-13T08:37:34.901765+0000","last_unstale":"2024-09-13T08:37:35.895292+0000","last_undegraded":"2024-09-13T08:37:35.895292+0000","last_fullsized":"2024-09-13T08:37:35.895292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:12.422693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.903257+0000","last_change":"2024-09-13T08:37:33.898217+0000","last_active":"2024-09-13T08:37:33.903257+0000","last_peered":"2024-09-13T08:37:33.903257+0000","last_clean":"2024-09-13T08:37:33.903257+0000","last_became_active":"2024-09-13T08:37:33.897954+0000","last_became_peered":"2024-09-13T08:37:33.897954+0000","last_unstale":"2024-09-13T08:37:33.903257+0000","last_undegraded":"2024-09-13T08:37:33.903257+0000","last_fullsized":"2024-09-13T08:37:33.903257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.879326+0000","last_change":"2024-09-13T08:37:33.879326+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.879326+0000","last_undegraded":"2024-09-13T08:37:33.879326+0000","last_fullsized":"2024-09-13T08:37:33.879326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.901668+0000","last_change":"2024-09-13T08:37:33.897139+0000","last_active":"2024-09-13T08:37:33.901668+0000","last_peered":"2024-09-13T08:37:33.901668+0000","last_clean":"2024-09-13T08:37:33.901668+0000","last_became_active":"2024-09-13T08:37:33.896576+0000","last_became_peered":"2024-09-13T08:37:33.896576+0000","last_unstale":"2024-09-13T08:37:33.901668+0000","last_undegraded":"2024-09-13T08:37:33.901668+0000","last_fullsized":"2024-09-13T08:37:33.901668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:05.995257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.879322+0000","last_change":"2024-09-13T08:37:33.879322+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.879322+0000","last_undegraded":"2024-09-13T08:37:33.879322+0000","last_fullsized":"2024-09-13T08:37:33.879322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900384+0000","last_change":"2024-09-13T08:37:33.893088+0000","last_active":"2024-09-13T08:37:33.900384+0000","last_peered":"2024-09-13T08:37:33.900384+0000","last_clean":"2024-09-13T08:37:33.900384+0000","last_became_active":"2024-09-13T08:37:33.892731+0000","last_became_peered":"2024-09-13T08:37:33.892731+0000","last_unstale":"2024-09-13T08:37:33.900384+0000","last_undegraded":"2024-09-13T08:37:33.900384+0000","last_fullsized":"2024-09-13T08:37:33.900384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.870235+0000","last_change":"2024-09-13T08:37:33.870235+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.870235+0000","last_undegraded":"2024-09-13T08:37:33.870235+0000","last_fullsized":"2024-09-13T08:37:33.870235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.903271+0000","last_change":"2024-09-13T08:37:33.898275+0000","last_active":"2024-09-13T08:37:33.903271+0000","last_peered":"2024-09-13T08:37:33.903271+0000","last_clean":"2024-09-13T08:37:33.903271+0000","last_became_active":"2024-09-13T08:37:33.897960+0000","last_became_peered":"2024-09-13T08:37:33.897960+0000","last_unstale":"2024-09-13T08:37:33.903271+0000","last_undegraded":"2024-09-13T08:37:33.903271+0000","last_fullsized":"2024-09-13T08:37:33.903271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.874612+0000","last_change":"2024-09-13T08:37:33.874612+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.874612+0000","last_undegraded":"2024-09-13T08:37:33.874612+0000","last_fullsized":"2024-09-13T08:37:33.874612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894655+0000","last_change":"2024-09-13T08:37:33.891510+0000","last_active":"2024-09-13T08:37:35.894655+0000","last_peered":"2024-09-13T08:37:35.894655+0000","last_clean":"2024-09-13T08:37:35.894655+0000","last_became_active":"2024-09-13T08:37:33.890774+0000","last_became_peered":"2024-09-13T08:37:33.890774+0000","last_unstale":"2024-09-13T08:37:35.894655+0000","last_undegraded":"2024-09-13T08:37:35.894655+0000","last_fullsized":"2024-09-13T08:37:35.894655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:25.370931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876319+0000","last_change":"2024-09-13T08:37:33.876319+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876319+0000","last_undegraded":"2024-09-13T08:37:33.876319+0000","last_fullsized":"2024-09-13T08:37:33.876319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.893774+0000","last_change":"2024-09-13T08:37:33.887243+0000","last_active":"2024-09-13T08:37:33.893774+0000","last_peered":"2024-09-13T08:37:33.893774+0000","last_clean":"2024-09-13T08:37:33.893774+0000","last_became_active":"2024-09-13T08:37:33.886882+0000","last_became_peered":"2024-09-13T08:37:33.886882+0000","last_unstale":"2024-09-13T08:37:33.893774+0000","last_undegraded":"2024-09-13T08:37:33.893774+0000","last_fullsized":"2024-09-13T08:37:33.893774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.871566+0000","last_change":"2024-09-13T08:37:33.871566+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.871566+0000","last_undegraded":"2024-09-13T08:37:33.871566+0000","last_fullsized":"2024-09-13T08:37:33.871566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875076+0000","last_change":"2024-09-13T08:37:33.875076+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875076+0000","last_undegraded":"2024-09-13T08:37:33.875076+0000","last_fullsized":"2024-09-13T08:37:33.875076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.904025+0000","last_change":"2024-09-13T08:37:33.900929+0000","last_active":"2024-09-13T08:37:33.904025+0000","last_peered":"2024-09-13T08:37:33.904025+0000","last_clean":"2024-09-13T08:37:33.904025+0000","last_became_active":"2024-09-13T08:37:33.899776+0000","last_became_peered":"2024-09-13T08:37:33.899776+0000","last_unstale":"2024-09-13T08:37:33.904025+0000","last_undegraded":"2024-09-13T08:37:33.904025+0000","last_fullsized":"2024-09-13T08:37:33.904025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.873649+0000","last_change":"2024-09-13T08:37:33.873649+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.873649+0000","last_undegraded":"2024-09-13T08:37:33.873649+0000","last_fullsized":"2024-09-13T08:37:33.873649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.900424+0000","last_change":"2024-09-13T08:37:33.894985+0000","last_active":"2024-09-13T08:37:33.900424+0000","last_peered":"2024-09-13T08:37:33.900424+0000","last_clean":"2024-09-13T08:37:33.900424+0000","last_became_active":"2024-09-13T08:37:33.894087+0000","last_became_peered":"2024-09-13T08:37:33.894087+0000","last_unstale":"2024-09-13T08:37:33.900424+0000","last_undegraded":"2024-09-13T08:37:33.900424+0000","last_fullsized":"2024-09-13T08:37:33.900424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.876978+0000","last_change":"2024-09-13T08:37:33.876978+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.876978+0000","last_undegraded":"2024-09-13T08:37:33.876978+0000","last_fullsized":"2024-09-13T08:37:33.876978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.892622+0000","last_change":"2024-09-13T08:37:33.887848+0000","last_active":"2024-09-13T08:37:33.892622+0000","last_peered":"2024-09-13T08:37:33.892622+0000","last_clean":"2024-09-13T08:37:33.892622+0000","last_became_active":"2024-09-13T08:37:33.887508+0000","last_became_peered":"2024-09-13T08:37:33.887508+0000","last_unstale":"2024-09-13T08:37:33.892622+0000","last_undegraded":"2024-09-13T08:37:33.892622+0000","last_fullsized":"2024-09-13T08:37:33.892622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:12:45.242739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-13T08:37:33.875975+0000","last_change":"2024-09-13T08:37:33.875975+0000","last_active":"2024-09-13T08:37:33.854563+0000","last_peered":"2024-09-13T08:37:33.854563+0000","last_clean":"2024-09-13T08:37:33.854563+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:37:33.875975+0000","last_undegraded":"2024-09-13T08:37:33.875975+0000","last_fullsized":"2024-09-13T08:37:33.875975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T08:37:33.897760+0000","last_change":"2024-09-13T08:37:33.894082+0000","last_active":"2024-09-13T08:37:33.897760+0000","last_peered":"2024-09-13T08:37:33.897760+0000","last_clean":"2024-09-13T08:37:33.897760+0000","last_became_active":"2024-09-13T08:37:33.893820+0000","last_became_peered":"2024-09-13T08:37:33.893820+0000","last_unstale":"2024-09-13T08:37:33.897760+0000","last_undegraded":"2024-09-13T08:37:33.897760+0000","last_fullsized":"2024-09-13T08:37:33.897760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:24.586658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":653577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:37:39.848 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-13T08:37:39.848 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.888188+0000', 'last_change': '2024-09-13T08:37:33.888188+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.888188+0000', 'last_undegraded': '2024-09-13T08:37:33.888188+0000', 'last_fullsized': '2024-09-13T08:37:33.888188+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.876585+0000', 'last_change': '2024-09-13T08:37:33.876585+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.876585+0000', 'last_undegraded': '2024-09-13T08:37:33.876585+0000', 'last_fullsized': '2024-09-13T08:37:33.876585+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.873515+0000', 'last_change': '2024-09-13T08:37:33.873515+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.873515+0000', 'last_undegraded': '2024-09-13T08:37:33.873515+0000', 'last_fullsized': '2024-09-13T08:37:33.873515+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.870231+0000', 'last_change': '2024-09-13T08:37:33.870231+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.870231+0000', 'last_undegraded': '2024-09-13T08:37:33.870231+0000', 'last_fullsized': '2024-09-13T08:37:33.870231+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.887043+0000', 'last_change': '2024-09-13T08:37:33.887043+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.887043+0000', 'last_undegraded': '2024-09-13T08:37:33.887043+0000', 'last_fullsized': '2024-09-13T08:37:33.887043+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.869808+0000', 'last_change': '2024-09-13T08:37:33.869808+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.869808+0000', 'last_undegraded': '2024-09-13T08:37:33.869808+0000', 'last_fullsized': '2024-09-13T08:37:33.869808+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:37:39.849 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.871829+0000', 'last_change': '2024-09-13T08:37:33.871829+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.871829+0000', 'last_undegraded': '2024-09-13T08:37:33.871829+0000', 'last_fullsized': '2024-09-13T08:37:33.871829+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.875627+0000', 'last_change': '2024-09-13T08:37:33.875627+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.875627+0000', 'last_undegraded': '2024-09-13T08:37:33.875627+0000', 'last_fullsized': '2024-09-13T08:37:33.875627+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874425+0000', 'last_change': '2024-09-13T08:37:33.874425+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874425+0000', 'last_undegraded': '2024-09-13T08:37:33.874425+0000', 'last_fullsized': '2024-09-13T08:37:33.874425+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874047+0000', 'last_change': '2024-09-13T08:37:33.874047+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874047+0000', 'last_undegraded': '2024-09-13T08:37:33.874047+0000', 'last_fullsized': '2024-09-13T08:37:33.874047+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.881267+0000', 'last_change': '2024-09-13T08:37:33.881267+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.881267+0000', 'last_undegraded': '2024-09-13T08:37:33.881267+0000', 'last_fullsized': '2024-09-13T08:37:33.881267+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.877798+0000', 'last_change': '2024-09-13T08:37:33.877798+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.877798+0000', 'last_undegraded': '2024-09-13T08:37:33.877798+0000', 'last_fullsized': '2024-09-13T08:37:33.877798+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.850 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874667+0000', 'last_change': '2024-09-13T08:37:33.874667+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874667+0000', 'last_undegraded': '2024-09-13T08:37:33.874667+0000', 'last_fullsized': '2024-09-13T08:37:33.874667+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.876422+0000', 'last_change': '2024-09-13T08:37:33.876422+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.876422+0000', 'last_undegraded': '2024-09-13T08:37:33.876422+0000', 'last_fullsized': '2024-09-13T08:37:33.876422+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874801+0000', 'last_change': '2024-09-13T08:37:33.874801+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874801+0000', 'last_undegraded': '2024-09-13T08:37:33.874801+0000', 'last_fullsized': '2024-09-13T08:37:33.874801+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.878051+0000', 'last_change': '2024-09-13T08:37:33.878051+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.878051+0000', 'last_undegraded': '2024-09-13T08:37:33.878051+0000', 'last_fullsized': '2024-09-13T08:37:33.878051+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.875550+0000', 'last_change': '2024-09-13T08:37:33.875550+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.875550+0000', 'last_undegraded': '2024-09-13T08:37:33.875550+0000', 'last_fullsized': '2024-09-13T08:37:33.875550+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.875619+0000', 'last_change': '2024-09-13T08:37:33.875619+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.875619+0000', 'last_undegraded': '2024-09-13T08:37:33.875619+0000', 'last_fullsized': '2024-09-13T08:37:33.875619+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.851 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-13T08:37:39.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.881053+0000', 'last_change': '2024-09-13T08:37:33.881053+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.881053+0000', 'last_undegraded': '2024-09-13T08:37:33.881053+0000', 'last_fullsized': '2024-09-13T08:37:33.881053+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-13T08:37:39.852 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-13T08:37:39.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.875830+0000', 'last_change': '2024-09-13T08:37:33.875830+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.875830+0000', 'last_undegraded': '2024-09-13T08:37:33.875830+0000', 'last_fullsized': '2024-09-13T08:37:33.875830+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T08:37:39.852 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-13T08:37:39.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.873389+0000', 'last_change': '2024-09-13T08:37:33.873389+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.873389+0000', 'last_undegraded': '2024-09-13T08:37:33.873389+0000', 'last_fullsized': '2024-09-13T08:37:33.873389+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.852 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-13T08:37:39.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.875855+0000', 'last_change': '2024-09-13T08:37:33.875855+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.875855+0000', 'last_undegraded': '2024-09-13T08:37:33.875855+0000', 'last_fullsized': '2024-09-13T08:37:33.875855+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.852 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-13T08:37:39.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.877601+0000', 'last_change': '2024-09-13T08:37:33.877601+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.877601+0000', 'last_undegraded': '2024-09-13T08:37:33.877601+0000', 'last_fullsized': '2024-09-13T08:37:33.877601+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.878497+0000', 'last_change': '2024-09-13T08:37:33.878497+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.878497+0000', 'last_undegraded': '2024-09-13T08:37:33.878497+0000', 'last_fullsized': '2024-09-13T08:37:33.878497+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.872673+0000', 'last_change': '2024-09-13T08:37:33.872673+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.872673+0000', 'last_undegraded': '2024-09-13T08:37:33.872673+0000', 'last_fullsized': '2024-09-13T08:37:33.872673+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874060+0000', 'last_change': '2024-09-13T08:37:33.874060+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874060+0000', 'last_undegraded': '2024-09-13T08:37:33.874060+0000', 'last_fullsized': '2024-09-13T08:37:33.874060+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874453+0000', 'last_change': '2024-09-13T08:37:33.874453+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874453+0000', 'last_undegraded': '2024-09-13T08:37:33.874453+0000', 'last_fullsized': '2024-09-13T08:37:33.874453+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.872506+0000', 'last_change': '2024-09-13T08:37:33.872506+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.872506+0000', 'last_undegraded': '2024-09-13T08:37:33.872506+0000', 'last_fullsized': '2024-09-13T08:37:33.872506+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-13T08:37:39.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.875197+0000', 'last_change': '2024-09-13T08:37:33.875197+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.875197+0000', 'last_undegraded': '2024-09-13T08:37:33.875197+0000', 'last_fullsized': '2024-09-13T08:37:33.875197+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.872451+0000', 'last_change': '2024-09-13T08:37:33.872451+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.872451+0000', 'last_undegraded': '2024-09-13T08:37:33.872451+0000', 'last_fullsized': '2024-09-13T08:37:33.872451+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.872534+0000', 'last_change': '2024-09-13T08:37:33.872534+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.872534+0000', 'last_undegraded': '2024-09-13T08:37:33.872534+0000', 'last_fullsized': '2024-09-13T08:37:33.872534+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.879388+0000', 'last_change': '2024-09-13T08:37:33.879388+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.879388+0000', 'last_undegraded': '2024-09-13T08:37:33.879388+0000', 'last_fullsized': '2024-09-13T08:37:33.879388+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.870875+0000', 'last_change': '2024-09-13T08:37:33.870875+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.870875+0000', 'last_undegraded': '2024-09-13T08:37:33.870875+0000', 'last_fullsized': '2024-09-13T08:37:33.870875+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.876422+0000', 'last_change': '2024-09-13T08:37:33.876422+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.876422+0000', 'last_undegraded': '2024-09-13T08:37:33.876422+0000', 'last_fullsized': '2024-09-13T08:37:33.876422+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874425+0000', 'last_change': '2024-09-13T08:37:33.874425+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874425+0000', 'last_undegraded': '2024-09-13T08:37:33.874425+0000', 'last_fullsized': '2024-09-13T08:37:33.874425+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.854 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.878005+0000', 'last_change': '2024-09-13T08:37:33.878005+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.878005+0000', 'last_undegraded': '2024-09-13T08:37:33.878005+0000', 'last_fullsized': '2024-09-13T08:37:33.878005+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874869+0000', 'last_change': '2024-09-13T08:37:33.874869+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874869+0000', 'last_undegraded': '2024-09-13T08:37:33.874869+0000', 'last_fullsized': '2024-09-13T08:37:33.874869+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874510+0000', 'last_change': '2024-09-13T08:37:33.874510+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874510+0000', 'last_undegraded': '2024-09-13T08:37:33.874510+0000', 'last_fullsized': '2024-09-13T08:37:33.874510+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.872360+0000', 'last_change': '2024-09-13T08:37:33.872360+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.872360+0000', 'last_undegraded': '2024-09-13T08:37:33.872360+0000', 'last_fullsized': '2024-09-13T08:37:33.872360+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.870305+0000', 'last_change': '2024-09-13T08:37:33.870305+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.870305+0000', 'last_undegraded': '2024-09-13T08:37:33.870305+0000', 'last_fullsized': '2024-09-13T08:37:33.870305+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874608+0000', 'last_change': '2024-09-13T08:37:33.874608+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874608+0000', 'last_undegraded': '2024-09-13T08:37:33.874608+0000', 'last_fullsized': '2024-09-13T08:37:33.874608+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.855 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.873750+0000', 'last_change': '2024-09-13T08:37:33.873750+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.873750+0000', 'last_undegraded': '2024-09-13T08:37:33.873750+0000', 'last_fullsized': '2024-09-13T08:37:33.873750+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.872340+0000', 'last_change': '2024-09-13T08:37:33.872340+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.872340+0000', 'last_undegraded': '2024-09-13T08:37:33.872340+0000', 'last_fullsized': '2024-09-13T08:37:33.872340+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.877074+0000', 'last_change': '2024-09-13T08:37:33.877074+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.877074+0000', 'last_undegraded': '2024-09-13T08:37:33.877074+0000', 'last_fullsized': '2024-09-13T08:37:33.877074+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.875140+0000', 'last_change': '2024-09-13T08:37:33.875140+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.875140+0000', 'last_undegraded': '2024-09-13T08:37:33.875140+0000', 'last_fullsized': '2024-09-13T08:37:33.875140+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.872530+0000', 'last_change': '2024-09-13T08:37:33.872530+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.872530+0000', 'last_undegraded': '2024-09-13T08:37:33.872530+0000', 'last_fullsized': '2024-09-13T08:37:33.872530+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-13T08:37:39.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.871796+0000', 'last_change': '2024-09-13T08:37:33.871796+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.871796+0000', 'last_undegraded': '2024-09-13T08:37:33.871796+0000', 'last_fullsized': '2024-09-13T08:37:33.871796+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:37:39.857 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-13T08:37:39.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.879326+0000', 'last_change': '2024-09-13T08:37:33.879326+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.879326+0000', 'last_undegraded': '2024-09-13T08:37:33.879326+0000', 'last_fullsized': '2024-09-13T08:37:33.879326+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-13T08:37:39.857 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-13T08:37:39.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.879322+0000', 'last_change': '2024-09-13T08:37:33.879322+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.879322+0000', 'last_undegraded': '2024-09-13T08:37:33.879322+0000', 'last_fullsized': '2024-09-13T08:37:33.879322+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-13T08:37:39.857 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-13T08:37:39.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.870235+0000', 'last_change': '2024-09-13T08:37:33.870235+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.870235+0000', 'last_undegraded': '2024-09-13T08:37:33.870235+0000', 'last_fullsized': '2024-09-13T08:37:33.870235+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.857 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-13T08:37:39.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.874612+0000', 'last_change': '2024-09-13T08:37:33.874612+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.874612+0000', 'last_undegraded': '2024-09-13T08:37:33.874612+0000', 'last_fullsized': '2024-09-13T08:37:33.874612+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-13T08:37:39.857 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-13T08:37:39.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.876319+0000', 'last_change': '2024-09-13T08:37:33.876319+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.876319+0000', 'last_undegraded': '2024-09-13T08:37:33.876319+0000', 'last_fullsized': '2024-09-13T08:37:33.876319+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-13T08:37:39.858 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-13T08:37:39.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.871566+0000', 'last_change': '2024-09-13T08:37:33.871566+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.871566+0000', 'last_undegraded': '2024-09-13T08:37:33.871566+0000', 'last_fullsized': '2024-09-13T08:37:33.871566+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.858 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-13T08:37:39.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.875076+0000', 'last_change': '2024-09-13T08:37:33.875076+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.875076+0000', 'last_undegraded': '2024-09-13T08:37:33.875076+0000', 'last_fullsized': '2024-09-13T08:37:33.875076+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:37:39.858 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-13T08:37:39.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.873649+0000', 'last_change': '2024-09-13T08:37:33.873649+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.873649+0000', 'last_undegraded': '2024-09-13T08:37:33.873649+0000', 'last_fullsized': '2024-09-13T08:37:33.873649+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T08:37:39.858 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-13T08:37:39.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.876978+0000', 'last_change': '2024-09-13T08:37:33.876978+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.876978+0000', 'last_undegraded': '2024-09-13T08:37:33.876978+0000', 'last_fullsized': '2024-09-13T08:37:33.876978+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.858 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-13T08:37:39.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:37:33.875975+0000', 'last_change': '2024-09-13T08:37:33.875975+0000', 'last_active': '2024-09-13T08:37:33.854563+0000', 'last_peered': '2024-09-13T08:37:33.854563+0000', 'last_clean': '2024-09-13T08:37:33.854563+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:37:33.875975+0000', 'last_undegraded': '2024-09-13T08:37:33.875975+0000', 'last_fullsized': '2024-09-13T08:37:33.875975+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'last_clean_scrub_stamp': '2024-09-13T08:37:33.854563+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:37:39.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T08:37:40.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 -- 172.21.15.44:0/2921532026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa63c0b2760 msgr2=0x7fa63c1605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:40.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 --2- 172.21.15.44:0/2921532026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa63c0b2760 0x7fa63c1605d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa634009c80 tx=0x7fa63401cd10 comp rx=0 tx=0).stop 2024-09-13T08:37:40.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 -- 172.21.15.44:0/2921532026 shutdown_connections 2024-09-13T08:37:40.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 --2- 172.21.15.44:0/2921532026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa63c0b2760 0x7fa63c1605d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 --2- 172.21.15.44:0/2921532026 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa63c0cfe80 0x7fa63c0d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 -- 172.21.15.44:0/2921532026 >> 172.21.15.44:0/2921532026 conn(0x7fa63c0d1e40 msgr2=0x7fa63c0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:40.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 -- 172.21.15.44:0/2921532026 shutdown_connections 2024-09-13T08:37:40.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 -- 172.21.15.44:0/2921532026 wait complete. 2024-09-13T08:37:40.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 Processor -- start 2024-09-13T08:37:40.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 -- start start 2024-09-13T08:37:40.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa63c0b2760 0x7fa63c154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:40.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa63c0b2d00 0x7fa63c1f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:40.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c0cfe80 0x7fa63c0ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:40.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c0b0940 con 0x7fa63c0cfe80 2024-09-13T08:37:40.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c0b07c0 con 0x7fa63c0b2d00 2024-09-13T08:37:40.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa643400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa63c0b0ac0 con 0x7fa63c0b2760 2024-09-13T08:37:40.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa640a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa63c0b2760 0x7fa63c154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:40.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.035+0000 7fa641400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c0cfe80 0x7fa63c0ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:40.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa640a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa63c0b2760 0x7fa63c154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:57802/0 (socket says 172.21.15.44:57802) 2024-09-13T08:37:40.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa640a00640 1 -- 172.21.15.44:0/3527796225 learned_addr learned my addr 172.21.15.44:0/3527796225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:40.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa63be00640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa63c0b2d00 0x7fa63c1f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:40.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa641400640 1 -- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa63c0b2760 msgr2=0x7fa63c154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:40.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa641400640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa63c0b2760 0x7fa63c154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa641400640 1 -- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa63c0b2d00 msgr2=0x7fa63c1f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:40.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa641400640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa63c0b2d00 0x7fa63c1f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa641400640 1 -- 172.21.15.44:0/3527796225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63c1fb110 con 0x7fa63c0cfe80 2024-09-13T08:37:40.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa641400640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c0cfe80 0x7fa63c0ca420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6340ac000 tx=0x7fa634009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:40.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa638c00640 1 -- 172.21.15.44:0/3527796225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6340bb070 con 0x7fa63c0cfe80 2024-09-13T08:37:40.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa638c00640 1 -- 172.21.15.44:0/3527796225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa634004d40 con 0x7fa63c0cfe80 2024-09-13T08:37:40.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa643400640 1 -- 172.21.15.44:0/3527796225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63c1fb3f0 con 0x7fa63c0cfe80 2024-09-13T08:37:40.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa638c00640 1 -- 172.21.15.44:0/3527796225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6340b2040 con 0x7fa63c0cfe80 2024-09-13T08:37:40.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa643400640 1 -- 172.21.15.44:0/3527796225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63c1fb9b0 con 0x7fa63c0cfe80 2024-09-13T08:37:40.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa643400640 1 -- 172.21.15.44:0/3527796225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63c0d0260 con 0x7fa63c0cfe80 2024-09-13T08:37:40.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.039+0000 7fa638c00640 1 -- 172.21.15.44:0/3527796225 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fa6340a5ae0 con 0x7fa63c0cfe80 2024-09-13T08:37:40.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.043+0000 7fa638c00640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa61c072ef0 0x7fa61c0753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:40.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.043+0000 7fa638c00640 1 -- 172.21.15.44:0/3527796225 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa63412db70 con 0x7fa63c0cfe80 2024-09-13T08:37:40.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.043+0000 7fa640a00640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa61c072ef0 0x7fa61c0753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:40.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.043+0000 7fa640a00640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa61c072ef0 0x7fa61c0753b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa630009680 tx=0x7fa6300091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:40.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.043+0000 7fa638c00640 1 -- 172.21.15.44:0/3527796225 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa634016990 con 0x7fa63c0cfe80 2024-09-13T08:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 -- 172.21.15.44:0/3527796225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa63c1fbd20 con 0x7fa63c0cfe80 2024-09-13T08:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa638c00640 1 -- 172.21.15.44:0/3527796225 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1074 (secure 0 0 0) 0x7fa6340f8650 con 0x7fa63c0cfe80 2024-09-13T08:37:40.287 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:40.287 INFO:teuthology.orchestra.run.smithi044.stdout:{"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1726216643,"num_in_osds":8,"osd_in_since":1726216632,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":57},{"state_name":"active+clean","count":40}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":224149504,"bytes_avail":772869963776,"bytes_total":773094113280,"inactive_pgs_ratio":0.58762884140014648},"fsmap":{"epoch":8,"btime":"2024-09-13T08:37:37:896662+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4360}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-13T08:37:09.259109+0000","services":{}},"progress_events":{}} 2024-09-13T08:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 -- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa61c072ef0 msgr2=0x7fa61c0753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa61c072ef0 0x7fa61c0753b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa630009680 tx=0x7fa6300091e0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 -- 172.21.15.44:0/3527796225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c0cfe80 msgr2=0x7fa63c0ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c0cfe80 0x7fa63c0ca420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6340ac000 tx=0x7fa634009330 comp rx=0 tx=0).stop 2024-09-13T08:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 -- 172.21.15.44:0/3527796225 shutdown_connections 2024-09-13T08:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa61c072ef0 0x7fa61c0753b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c0cfe80 0x7fa63c0ca420 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa63c0b2d00 0x7fa63c1f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 --2- 172.21.15.44:0/3527796225 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa63c0b2760 0x7fa63c154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 -- 172.21.15.44:0/3527796225 >> 172.21.15.44:0/3527796225 conn(0x7fa63c0d1e40 msgr2=0x7fa63c06e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:40.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 -- 172.21.15.44:0/3527796225 shutdown_connections 2024-09-13T08:37:40.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.283+0000 7fa643400640 1 -- 172.21.15.44:0/3527796225 wait complete. 2024-09-13T08:37:40.308 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-13T08:37:40.308 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:37:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.447+0000 7fabeac00640 1 -- 172.21.15.44:0/303302045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabe40d28e0 msgr2=0x7fabe40cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.447+0000 7fabeac00640 1 --2- 172.21.15.44:0/303302045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabe40d28e0 0x7fabe40cc9d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fabe4074010 tx=0x7fabdc01cd80 comp rx=0 tx=0).stop 2024-09-13T08:37:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.447+0000 7fabeac00640 1 -- 172.21.15.44:0/303302045 shutdown_connections 2024-09-13T08:37:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.447+0000 7fabeac00640 1 --2- 172.21.15.44:0/303302045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabe40d28e0 0x7fabe40cc9d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.447+0000 7fabeac00640 1 --2- 172.21.15.44:0/303302045 >> v2:172.21.15.129:3300/0 conn(0x7fabe40cdb30 0x7fabe40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.447+0000 7fabeac00640 1 --2- 172.21.15.44:0/303302045 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabe415bea0 0x7fabe40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.447+0000 7fabeac00640 1 -- 172.21.15.44:0/303302045 >> 172.21.15.44:0/303302045 conn(0x7fabe40d1dc0 msgr2=0x7fabe40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 -- 172.21.15.44:0/303302045 shutdown_connections 2024-09-13T08:37:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 -- 172.21.15.44:0/303302045 wait complete. 2024-09-13T08:37:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 Processor -- start 2024-09-13T08:37:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 -- start start 2024-09-13T08:37:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabe40cdb30 0x7fabe40c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabe40d28e0 0x7fabe414d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabe415bea0 0x7fabe40b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fabe40af1b0 con 0x7fabe40cdb30 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fabe40af030 con 0x7fabe415bea0 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fabe40af330 con 0x7fabe40d28e0 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabe40cdb30 0x7fabe40c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabe40cdb30 0x7fabe40c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60726/0 (socket says 172.21.15.44:60726) 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe3e00640 1 -- 172.21.15.44:0/1934894664 learned_addr learned my addr 172.21.15.44:0/1934894664 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabdbe00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabe40d28e0 0x7fabe414d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe8c00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabe415bea0 0x7fabe40b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe3e00640 1 -- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabe40d28e0 msgr2=0x7fabe414d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe3e00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabe40d28e0 0x7fabe414d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe3e00640 1 -- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabe415bea0 msgr2=0x7fabe40b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe3e00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabe415bea0 0x7fabe40b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe3e00640 1 -- 172.21.15.44:0/1934894664 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe40ca240 con 0x7fabe40cdb30 2024-09-13T08:37:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe3e00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabe40cdb30 0x7fabe40c8e90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d720 tx=0x7fabd400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe0c00640 1 -- 172.21.15.44:0/1934894664 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4017070 con 0x7fabe40cdb30 2024-09-13T08:37:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe0c00640 1 -- 172.21.15.44:0/1934894664 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabd4009d20 con 0x7fabe40cdb30 2024-09-13T08:37:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabe0c00640 1 -- 172.21.15.44:0/1934894664 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4005280 con 0x7fabe40cdb30 2024-09-13T08:37:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.451+0000 7fabeac00640 1 -- 172.21.15.44:0/1934894664 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe40c9600 con 0x7fabe40cdb30 2024-09-13T08:37:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.455+0000 7fabeac00640 1 -- 172.21.15.44:0/1934894664 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe40c9bc0 con 0x7fabe40cdb30 2024-09-13T08:37:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.455+0000 7fabeac00640 1 -- 172.21.15.44:0/1934894664 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe40af8c0 con 0x7fabe40cdb30 2024-09-13T08:37:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.455+0000 7fabe0c00640 1 -- 172.21.15.44:0/1934894664 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fabd4005420 con 0x7fabe40cdb30 2024-09-13T08:37:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.455+0000 7fabe0c00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fabc0072ec0 0x7fabc0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.455+0000 7fabe0c00640 1 -- 172.21.15.44:0/1934894664 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fabd4093ec0 con 0x7fabe40cdb30 2024-09-13T08:37:40.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.455+0000 7fabdbe00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fabc0072ec0 0x7fabc0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:40.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.459+0000 7fabdbe00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fabc0072ec0 0x7fabc0075380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabcc005fd0 tx=0x7fabcc005e60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.459+0000 7fabe0c00640 1 -- 172.21.15.44:0/1934894664 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fabd405f110 con 0x7fabe40cdb30 2024-09-13T08:37:40.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.631+0000 7fabeac00640 1 -- 172.21.15.44:0/1934894664 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabe409cde0 con 0x7fabc0072ec0 2024-09-13T08:37:40.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.639+0000 7fabe0c00640 1 -- 172.21.15.44:0/1934894664 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7fabcc00b2c0 con 0x7fabc0072ec0 2024-09-13T08:37:40.644 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:40.646 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.639+0000 7fabeac00640 1 -- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fabc0072ec0 msgr2=0x7fabc0075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.639+0000 7fabeac00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fabc0072ec0 0x7fabc0075380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabcc005fd0 tx=0x7fabcc005e60 comp rx=0 tx=0).stop 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.639+0000 7fabeac00640 1 -- 172.21.15.44:0/1934894664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabe40cdb30 msgr2=0x7fabe40c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.639+0000 7fabeac00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabe40cdb30 0x7fabe40c8e90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d720 tx=0x7fabd400dbe0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.643+0000 7fabeac00640 1 -- 172.21.15.44:0/1934894664 shutdown_connections 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.643+0000 7fabeac00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fabc0072ec0 0x7fabc0075380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.643+0000 7fabeac00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabe415bea0 0x7fabe40b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.643+0000 7fabeac00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabe40d28e0 0x7fabe414d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.643+0000 7fabeac00640 1 --2- 172.21.15.44:0/1934894664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabe40cdb30 0x7fabe40c8e90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.643+0000 7fabeac00640 1 -- 172.21.15.44:0/1934894664 >> 172.21.15.44:0/1934894664 conn(0x7fabe40d1dc0 msgr2=0x7fabe415afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:40.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.643+0000 7fabeac00640 1 -- 172.21.15.44:0/1934894664 shutdown_connections 2024-09-13T08:37:40.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:40.643+0000 7fabeac00640 1 -- 172.21.15.44:0/1934894664 wait complete. 2024-09-13T08:37:40.664 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-13T08:37:40.511871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219224,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755496,"statfs":{"total":773094113280,"available":772869627904,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.643054"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877562+0000","last_change":"2024-09-13T08:37:34.901083+0000","last_active":"2024-09-13T08:37:36.877562+0000","last_peered":"2024-09-13T08:37:36.877562+0000","last_clean":"2024-09-13T08:37:36.877562+0000","last_became_active":"2024-09-13T08:37:34.900414+0000","last_became_peered":"2024-09-13T08:37:34.900414+0000","last_unstale":"2024-09-13T08:37:36.877562+0000","last_undegraded":"2024-09-13T08:37:36.877562+0000","last_fullsized":"2024-09-13T08:37:36.877562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:28:50.949101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896264+0000","last_change":"2024-09-13T08:37:34.900607+0000","last_active":"2024-09-13T08:37:35.896264+0000","last_peered":"2024-09-13T08:37:35.896264+0000","last_clean":"2024-09-13T08:37:35.896264+0000","last_became_active":"2024-09-13T08:37:34.900222+0000","last_became_peered":"2024-09-13T08:37:34.900222+0000","last_unstale":"2024-09-13T08:37:35.896264+0000","last_undegraded":"2024-09-13T08:37:35.896264+0000","last_fullsized":"2024-09-13T08:37:35.896264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:47.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895933+0000","last_change":"2024-09-13T08:37:34.906335+0000","last_active":"2024-09-13T08:37:35.895933+0000","last_peered":"2024-09-13T08:37:35.895933+0000","last_clean":"2024-09-13T08:37:35.895933+0000","last_became_active":"2024-09-13T08:37:34.905961+0000","last_became_peered":"2024-09-13T08:37:34.905961+0000","last_unstale":"2024-09-13T08:37:35.895933+0000","last_undegraded":"2024-09-13T08:37:35.895933+0000","last_fullsized":"2024-09-13T08:37:35.895933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:54.979663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510756+0000","last_change":"2024-09-13T08:37:34.904837+0000","last_active":"2024-09-13T08:37:36.510756+0000","last_peered":"2024-09-13T08:37:36.510756+0000","last_clean":"2024-09-13T08:37:36.510756+0000","last_became_active":"2024-09-13T08:37:34.898571+0000","last_became_peered":"2024-09-13T08:37:34.898571+0000","last_unstale":"2024-09-13T08:37:36.510756+0000","last_undegraded":"2024-09-13T08:37:36.510756+0000","last_fullsized":"2024-09-13T08:37:36.510756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877279+0000","last_change":"2024-09-13T08:37:34.888564+0000","last_active":"2024-09-13T08:37:36.877279+0000","last_peered":"2024-09-13T08:37:36.877279+0000","last_clean":"2024-09-13T08:37:36.877279+0000","last_became_active":"2024-09-13T08:37:34.888280+0000","last_became_peered":"2024-09-13T08:37:34.888280+0000","last_unstale":"2024-09-13T08:37:36.877279+0000","last_undegraded":"2024-09-13T08:37:36.877279+0000","last_fullsized":"2024-09-13T08:37:36.877279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:27:32.330596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510582+0000","last_change":"2024-09-13T08:37:34.895044+0000","last_active":"2024-09-13T08:37:36.510582+0000","last_peered":"2024-09-13T08:37:36.510582+0000","last_clean":"2024-09-13T08:37:36.510582+0000","last_became_active":"2024-09-13T08:37:34.894777+0000","last_became_peered":"2024-09-13T08:37:34.894777+0000","last_unstale":"2024-09-13T08:37:36.510582+0000","last_undegraded":"2024-09-13T08:37:36.510582+0000","last_fullsized":"2024-09-13T08:37:36.510582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877014+0000","last_change":"2024-09-13T08:37:34.888796+0000","last_active":"2024-09-13T08:37:36.877014+0000","last_peered":"2024-09-13T08:37:36.877014+0000","last_clean":"2024-09-13T08:37:36.877014+0000","last_became_active":"2024-09-13T08:37:34.888488+0000","last_became_peered":"2024-09-13T08:37:34.888488+0000","last_unstale":"2024-09-13T08:37:36.877014+0000","last_undegraded":"2024-09-13T08:37:36.877014+0000","last_fullsized":"2024-09-13T08:37:36.877014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894878+0000","last_change":"2024-09-13T08:37:34.910054+0000","last_active":"2024-09-13T08:37:35.894878+0000","last_peered":"2024-09-13T08:37:35.894878+0000","last_clean":"2024-09-13T08:37:35.894878+0000","last_became_active":"2024-09-13T08:37:34.909596+0000","last_became_peered":"2024-09-13T08:37:34.909596+0000","last_unstale":"2024-09-13T08:37:35.894878+0000","last_undegraded":"2024-09-13T08:37:35.894878+0000","last_fullsized":"2024-09-13T08:37:35.894878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:55.615355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892431+0000","last_change":"2024-09-13T08:37:34.885156+0000","last_active":"2024-09-13T08:37:35.892431+0000","last_peered":"2024-09-13T08:37:35.892431+0000","last_clean":"2024-09-13T08:37:35.892431+0000","last_became_active":"2024-09-13T08:37:34.884851+0000","last_became_peered":"2024-09-13T08:37:34.884851+0000","last_unstale":"2024-09-13T08:37:35.892431+0000","last_undegraded":"2024-09-13T08:37:35.892431+0000","last_fullsized":"2024-09-13T08:37:35.892431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:06.047971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877522+0000","last_change":"2024-09-13T08:37:34.909843+0000","last_active":"2024-09-13T08:37:36.877522+0000","last_peered":"2024-09-13T08:37:36.877522+0000","last_clean":"2024-09-13T08:37:36.877522+0000","last_became_active":"2024-09-13T08:37:34.909552+0000","last_became_peered":"2024-09-13T08:37:34.909552+0000","last_unstale":"2024-09-13T08:37:36.877522+0000","last_undegraded":"2024-09-13T08:37:36.877522+0000","last_fullsized":"2024-09-13T08:37:36.877522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:08.493271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877666+0000","last_change":"2024-09-13T08:37:34.902054+0000","last_active":"2024-09-13T08:37:36.877666+0000","last_peered":"2024-09-13T08:37:36.877666+0000","last_clean":"2024-09-13T08:37:36.877666+0000","last_became_active":"2024-09-13T08:37:34.901788+0000","last_became_peered":"2024-09-13T08:37:34.901788+0000","last_unstale":"2024-09-13T08:37:36.877666+0000","last_undegraded":"2024-09-13T08:37:36.877666+0000","last_fullsized":"2024-09-13T08:37:36.877666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:14.002956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877261+0000","last_change":"2024-09-13T08:37:34.892602+0000","last_active":"2024-09-13T08:37:36.877261+0000","last_peered":"2024-09-13T08:37:36.877261+0000","last_clean":"2024-09-13T08:37:36.877261+0000","last_became_active":"2024-09-13T08:37:34.890000+0000","last_became_peered":"2024-09-13T08:37:34.890000+0000","last_unstale":"2024-09-13T08:37:36.877261+0000","last_undegraded":"2024-09-13T08:37:36.877261+0000","last_fullsized":"2024-09-13T08:37:36.877261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:06.530173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894980+0000","last_change":"2024-09-13T08:37:34.903037+0000","last_active":"2024-09-13T08:37:35.894980+0000","last_peered":"2024-09-13T08:37:35.894980+0000","last_clean":"2024-09-13T08:37:35.894980+0000","last_became_active":"2024-09-13T08:37:34.894805+0000","last_became_peered":"2024-09-13T08:37:34.894805+0000","last_unstale":"2024-09-13T08:37:35.894980+0000","last_undegraded":"2024-09-13T08:37:35.894980+0000","last_fullsized":"2024-09-13T08:37:35.894980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:49.712241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894573+0000","last_change":"2024-09-13T08:37:34.901603+0000","last_active":"2024-09-13T08:37:35.894573+0000","last_peered":"2024-09-13T08:37:35.894573+0000","last_clean":"2024-09-13T08:37:35.894573+0000","last_became_active":"2024-09-13T08:37:34.899884+0000","last_became_peered":"2024-09-13T08:37:34.899884+0000","last_unstale":"2024-09-13T08:37:35.894573+0000","last_undegraded":"2024-09-13T08:37:35.894573+0000","last_fullsized":"2024-09-13T08:37:35.894573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:05.260600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895761+0000","last_change":"2024-09-13T08:37:34.896921+0000","last_active":"2024-09-13T08:37:35.895761+0000","last_peered":"2024-09-13T08:37:35.895761+0000","last_clean":"2024-09-13T08:37:35.895761+0000","last_became_active":"2024-09-13T08:37:34.896593+0000","last_became_peered":"2024-09-13T08:37:34.896593+0000","last_unstale":"2024-09-13T08:37:35.895761+0000","last_undegraded":"2024-09-13T08:37:35.895761+0000","last_fullsized":"2024-09-13T08:37:35.895761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:06.530173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084616+0000","last_change":"2024-09-13T08:37:34.908031+0000","last_active":"2024-09-13T08:37:37.084616+0000","last_peered":"2024-09-13T08:37:37.084616+0000","last_clean":"2024-09-13T08:37:37.084616+0000","last_became_active":"2024-09-13T08:37:34.907734+0000","last_became_peered":"2024-09-13T08:37:34.907734+0000","last_unstale":"2024-09-13T08:37:37.084616+0000","last_undegraded":"2024-09-13T08:37:37.084616+0000","last_fullsized":"2024-09-13T08:37:37.084616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:14.002956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085576+0000","last_change":"2024-09-13T08:37:34.911572+0000","last_active":"2024-09-13T08:37:37.085576+0000","last_peered":"2024-09-13T08:37:37.085576+0000","last_clean":"2024-09-13T08:37:37.085576+0000","last_became_active":"2024-09-13T08:37:34.911105+0000","last_became_peered":"2024-09-13T08:37:34.911105+0000","last_unstale":"2024-09-13T08:37:37.085576+0000","last_undegraded":"2024-09-13T08:37:37.085576+0000","last_fullsized":"2024-09-13T08:37:37.085576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:49:35.223067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.530220+0000","last_change":"2024-09-13T08:37:34.907767+0000","last_active":"2024-09-13T08:37:36.530220+0000","last_peered":"2024-09-13T08:37:36.530220+0000","last_clean":"2024-09-13T08:37:36.530220+0000","last_became_active":"2024-09-13T08:37:34.907100+0000","last_became_peered":"2024-09-13T08:37:34.907100+0000","last_unstale":"2024-09-13T08:37:36.530220+0000","last_undegraded":"2024-09-13T08:37:36.530220+0000","last_fullsized":"2024-09-13T08:37:36.530220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085543+0000","last_change":"2024-09-13T08:37:34.895822+0000","last_active":"2024-09-13T08:37:37.085543+0000","last_peered":"2024-09-13T08:37:37.085543+0000","last_clean":"2024-09-13T08:37:37.085543+0000","last_became_active":"2024-09-13T08:37:34.895430+0000","last_became_peered":"2024-09-13T08:37:34.895430+0000","last_unstale":"2024-09-13T08:37:37.085543+0000","last_undegraded":"2024-09-13T08:37:37.085543+0000","last_fullsized":"2024-09-13T08:37:37.085543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:33:54.897556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877407+0000","last_change":"2024-09-13T08:37:34.891580+0000","last_active":"2024-09-13T08:37:36.877407+0000","last_peered":"2024-09-13T08:37:36.877407+0000","last_clean":"2024-09-13T08:37:36.877407+0000","last_became_active":"2024-09-13T08:37:34.889710+0000","last_became_peered":"2024-09-13T08:37:34.889710+0000","last_unstale":"2024-09-13T08:37:36.877407+0000","last_undegraded":"2024-09-13T08:37:36.877407+0000","last_fullsized":"2024-09-13T08:37:36.877407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:47.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585799+0000","last_change":"2024-09-13T08:37:34.894615+0000","last_active":"2024-09-13T08:37:36.585799+0000","last_peered":"2024-09-13T08:37:36.585799+0000","last_clean":"2024-09-13T08:37:36.585799+0000","last_became_active":"2024-09-13T08:37:34.894327+0000","last_became_peered":"2024-09-13T08:37:34.894327+0000","last_unstale":"2024-09-13T08:37:36.585799+0000","last_undegraded":"2024-09-13T08:37:36.585799+0000","last_fullsized":"2024-09-13T08:37:36.585799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:55.615355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891945+0000","last_change":"2024-09-13T08:37:34.883272+0000","last_active":"2024-09-13T08:37:35.891945+0000","last_peered":"2024-09-13T08:37:35.891945+0000","last_clean":"2024-09-13T08:37:35.891945+0000","last_became_active":"2024-09-13T08:37:34.882941+0000","last_became_peered":"2024-09-13T08:37:34.882941+0000","last_unstale":"2024-09-13T08:37:35.891945+0000","last_undegraded":"2024-09-13T08:37:35.891945+0000","last_fullsized":"2024-09-13T08:37:35.891945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895545+0000","last_change":"2024-09-13T08:37:34.903123+0000","last_active":"2024-09-13T08:37:35.895545+0000","last_peered":"2024-09-13T08:37:35.895545+0000","last_clean":"2024-09-13T08:37:35.895545+0000","last_became_active":"2024-09-13T08:37:34.901167+0000","last_became_peered":"2024-09-13T08:37:34.901167+0000","last_unstale":"2024-09-13T08:37:35.895545+0000","last_undegraded":"2024-09-13T08:37:35.895545+0000","last_fullsized":"2024-09-13T08:37:35.895545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084614+0000","last_change":"2024-09-13T08:37:34.891584+0000","last_active":"2024-09-13T08:37:37.084614+0000","last_peered":"2024-09-13T08:37:37.084614+0000","last_clean":"2024-09-13T08:37:37.084614+0000","last_became_active":"2024-09-13T08:37:34.891268+0000","last_became_peered":"2024-09-13T08:37:34.891268+0000","last_unstale":"2024-09-13T08:37:37.084614+0000","last_undegraded":"2024-09-13T08:37:37.084614+0000","last_fullsized":"2024-09-13T08:37:37.084614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:59.406808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896325+0000","last_change":"2024-09-13T08:37:34.902079+0000","last_active":"2024-09-13T08:37:35.896325+0000","last_peered":"2024-09-13T08:37:35.896325+0000","last_clean":"2024-09-13T08:37:35.896325+0000","last_became_active":"2024-09-13T08:37:34.900499+0000","last_became_peered":"2024-09-13T08:37:34.900499+0000","last_unstale":"2024-09-13T08:37:35.896325+0000","last_undegraded":"2024-09-13T08:37:35.896325+0000","last_fullsized":"2024-09-13T08:37:35.896325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:45.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511137+0000","last_change":"2024-09-13T08:37:34.888787+0000","last_active":"2024-09-13T08:37:36.511137+0000","last_peered":"2024-09-13T08:37:36.511137+0000","last_clean":"2024-09-13T08:37:36.511137+0000","last_became_active":"2024-09-13T08:37:34.888486+0000","last_became_peered":"2024-09-13T08:37:34.888486+0000","last_unstale":"2024-09-13T08:37:36.511137+0000","last_undegraded":"2024-09-13T08:37:36.511137+0000","last_fullsized":"2024-09-13T08:37:36.511137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:56.908418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895541+0000","last_change":"2024-09-13T08:37:34.902546+0000","last_active":"2024-09-13T08:37:35.895541+0000","last_peered":"2024-09-13T08:37:35.895541+0000","last_clean":"2024-09-13T08:37:35.895541+0000","last_became_active":"2024-09-13T08:37:34.901841+0000","last_became_peered":"2024-09-13T08:37:34.901841+0000","last_unstale":"2024-09-13T08:37:35.895541+0000","last_undegraded":"2024-09-13T08:37:35.895541+0000","last_fullsized":"2024-09-13T08:37:35.895541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:40.635073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895078+0000","last_change":"2024-09-13T08:37:34.907277+0000","last_active":"2024-09-13T08:37:35.895078+0000","last_peered":"2024-09-13T08:37:35.895078+0000","last_clean":"2024-09-13T08:37:35.895078+0000","last_became_active":"2024-09-13T08:37:34.906984+0000","last_became_peered":"2024-09-13T08:37:34.906984+0000","last_unstale":"2024-09-13T08:37:35.895078+0000","last_undegraded":"2024-09-13T08:37:35.895078+0000","last_fullsized":"2024-09-13T08:37:35.895078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:23.664559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895840+0000","last_change":"2024-09-13T08:37:34.902627+0000","last_active":"2024-09-13T08:37:35.895840+0000","last_peered":"2024-09-13T08:37:35.895840+0000","last_clean":"2024-09-13T08:37:35.895840+0000","last_became_active":"2024-09-13T08:37:34.901894+0000","last_became_peered":"2024-09-13T08:37:34.901894+0000","last_unstale":"2024-09-13T08:37:35.895840+0000","last_undegraded":"2024-09-13T08:37:35.895840+0000","last_fullsized":"2024-09-13T08:37:35.895840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:06.723541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084874+0000","last_change":"2024-09-13T08:37:34.891317+0000","last_active":"2024-09-13T08:37:37.084874+0000","last_peered":"2024-09-13T08:37:37.084874+0000","last_clean":"2024-09-13T08:37:37.084874+0000","last_became_active":"2024-09-13T08:37:34.891030+0000","last_became_peered":"2024-09-13T08:37:34.891030+0000","last_unstale":"2024-09-13T08:37:37.084874+0000","last_undegraded":"2024-09-13T08:37:37.084874+0000","last_fullsized":"2024-09-13T08:37:37.084874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:46.738660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529933+0000","last_change":"2024-09-13T08:37:34.894275+0000","last_active":"2024-09-13T08:37:36.529933+0000","last_peered":"2024-09-13T08:37:36.529933+0000","last_clean":"2024-09-13T08:37:36.529933+0000","last_became_active":"2024-09-13T08:37:34.893030+0000","last_became_peered":"2024-09-13T08:37:34.893030+0000","last_unstale":"2024-09-13T08:37:36.529933+0000","last_undegraded":"2024-09-13T08:37:36.529933+0000","last_fullsized":"2024-09-13T08:37:36.529933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510646+0000","last_change":"2024-09-13T08:37:34.900318+0000","last_active":"2024-09-13T08:37:36.510646+0000","last_peered":"2024-09-13T08:37:36.510646+0000","last_clean":"2024-09-13T08:37:36.510646+0000","last_became_active":"2024-09-13T08:37:34.899949+0000","last_became_peered":"2024-09-13T08:37:34.899949+0000","last_unstale":"2024-09-13T08:37:36.510646+0000","last_undegraded":"2024-09-13T08:37:36.510646+0000","last_fullsized":"2024-09-13T08:37:36.510646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892117+0000","last_change":"2024-09-13T08:37:34.889876+0000","last_active":"2024-09-13T08:37:35.892117+0000","last_peered":"2024-09-13T08:37:35.892117+0000","last_clean":"2024-09-13T08:37:35.892117+0000","last_became_active":"2024-09-13T08:37:34.889622+0000","last_became_peered":"2024-09-13T08:37:34.889622+0000","last_unstale":"2024-09-13T08:37:35.892117+0000","last_undegraded":"2024-09-13T08:37:35.892117+0000","last_fullsized":"2024-09-13T08:37:35.892117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895867+0000","last_change":"2024-09-13T08:37:33.888852+0000","last_active":"2024-09-13T08:37:35.895867+0000","last_peered":"2024-09-13T08:37:35.895867+0000","last_clean":"2024-09-13T08:37:35.895867+0000","last_became_active":"2024-09-13T08:37:33.888409+0000","last_became_peered":"2024-09-13T08:37:33.888409+0000","last_unstale":"2024-09-13T08:37:35.895867+0000","last_undegraded":"2024-09-13T08:37:35.895867+0000","last_fullsized":"2024-09-13T08:37:35.895867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585747+0000","last_change":"2024-09-13T08:37:34.890809+0000","last_active":"2024-09-13T08:37:36.585747+0000","last_peered":"2024-09-13T08:37:36.585747+0000","last_clean":"2024-09-13T08:37:36.585747+0000","last_became_active":"2024-09-13T08:37:34.887213+0000","last_became_peered":"2024-09-13T08:37:34.887213+0000","last_unstale":"2024-09-13T08:37:36.585747+0000","last_undegraded":"2024-09-13T08:37:36.585747+0000","last_fullsized":"2024-09-13T08:37:36.585747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:19:06.310722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085055+0000","last_change":"2024-09-13T08:37:33.892308+0000","last_active":"2024-09-13T08:37:37.085055+0000","last_peered":"2024-09-13T08:37:37.085055+0000","last_clean":"2024-09-13T08:37:37.085055+0000","last_became_active":"2024-09-13T08:37:33.891875+0000","last_became_peered":"2024-09-13T08:37:33.891875+0000","last_unstale":"2024-09-13T08:37:37.085055+0000","last_undegraded":"2024-09-13T08:37:37.085055+0000","last_fullsized":"2024-09-13T08:37:37.085055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:05.039750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894251+0000","last_change":"2024-09-13T08:37:33.896561+0000","last_active":"2024-09-13T08:37:35.894251+0000","last_peered":"2024-09-13T08:37:35.894251+0000","last_clean":"2024-09-13T08:37:35.894251+0000","last_became_active":"2024-09-13T08:37:33.896314+0000","last_became_peered":"2024-09-13T08:37:33.896314+0000","last_unstale":"2024-09-13T08:37:35.894251+0000","last_undegraded":"2024-09-13T08:37:35.894251+0000","last_fullsized":"2024-09-13T08:37:35.894251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:28.728254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895122+0000","last_change":"2024-09-13T08:37:34.900565+0000","last_active":"2024-09-13T08:37:35.895122+0000","last_peered":"2024-09-13T08:37:35.895122+0000","last_clean":"2024-09-13T08:37:35.895122+0000","last_became_active":"2024-09-13T08:37:34.900257+0000","last_became_peered":"2024-09-13T08:37:34.900257+0000","last_unstale":"2024-09-13T08:37:35.895122+0000","last_undegraded":"2024-09-13T08:37:35.895122+0000","last_fullsized":"2024-09-13T08:37:35.895122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:34.291551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084825+0000","last_change":"2024-09-13T08:37:33.883156+0000","last_active":"2024-09-13T08:37:37.084825+0000","last_peered":"2024-09-13T08:37:37.084825+0000","last_clean":"2024-09-13T08:37:37.084825+0000","last_became_active":"2024-09-13T08:37:33.882800+0000","last_became_peered":"2024-09-13T08:37:33.882800+0000","last_unstale":"2024-09-13T08:37:37.084825+0000","last_undegraded":"2024-09-13T08:37:37.084825+0000","last_fullsized":"2024-09-13T08:37:37.084825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:10.413313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585694+0000","last_change":"2024-09-13T08:37:34.888188+0000","last_active":"2024-09-13T08:37:36.585694+0000","last_peered":"2024-09-13T08:37:36.585694+0000","last_clean":"2024-09-13T08:37:36.585694+0000","last_became_active":"2024-09-13T08:37:34.887475+0000","last_became_peered":"2024-09-13T08:37:34.887475+0000","last_unstale":"2024-09-13T08:37:36.585694+0000","last_undegraded":"2024-09-13T08:37:36.585694+0000","last_fullsized":"2024-09-13T08:37:36.585694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:14.743221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892396+0000","last_change":"2024-09-13T08:37:33.891385+0000","last_active":"2024-09-13T08:37:35.892396+0000","last_peered":"2024-09-13T08:37:35.892396+0000","last_clean":"2024-09-13T08:37:35.892396+0000","last_became_active":"2024-09-13T08:37:33.891112+0000","last_became_peered":"2024-09-13T08:37:33.891112+0000","last_unstale":"2024-09-13T08:37:35.892396+0000","last_undegraded":"2024-09-13T08:37:35.892396+0000","last_fullsized":"2024-09-13T08:37:35.892396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511114+0000","last_change":"2024-09-13T08:37:34.899512+0000","last_active":"2024-09-13T08:37:36.511114+0000","last_peered":"2024-09-13T08:37:36.511114+0000","last_clean":"2024-09-13T08:37:36.511114+0000","last_became_active":"2024-09-13T08:37:34.887883+0000","last_became_peered":"2024-09-13T08:37:34.887883+0000","last_unstale":"2024-09-13T08:37:36.511114+0000","last_undegraded":"2024-09-13T08:37:36.511114+0000","last_fullsized":"2024-09-13T08:37:36.511114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:56.149863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877483+0000","last_change":"2024-09-13T08:37:33.899147+0000","last_active":"2024-09-13T08:37:36.877483+0000","last_peered":"2024-09-13T08:37:36.877483+0000","last_clean":"2024-09-13T08:37:36.877483+0000","last_became_active":"2024-09-13T08:37:33.896929+0000","last_became_peered":"2024-09-13T08:37:33.896929+0000","last_unstale":"2024-09-13T08:37:36.877483+0000","last_undegraded":"2024-09-13T08:37:36.877483+0000","last_fullsized":"2024-09-13T08:37:36.877483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:08:15.453880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896504+0000","last_change":"2024-09-13T08:37:34.900790+0000","last_active":"2024-09-13T08:37:35.896504+0000","last_peered":"2024-09-13T08:37:35.896504+0000","last_clean":"2024-09-13T08:37:35.896504+0000","last_became_active":"2024-09-13T08:37:34.898939+0000","last_became_peered":"2024-09-13T08:37:34.898939+0000","last_unstale":"2024-09-13T08:37:35.896504+0000","last_undegraded":"2024-09-13T08:37:35.896504+0000","last_fullsized":"2024-09-13T08:37:35.896504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:07.812549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.530096+0000","last_change":"2024-09-13T08:37:33.892872+0000","last_active":"2024-09-13T08:37:36.530096+0000","last_peered":"2024-09-13T08:37:36.530096+0000","last_clean":"2024-09-13T08:37:36.530096+0000","last_became_active":"2024-09-13T08:37:33.892579+0000","last_became_peered":"2024-09-13T08:37:33.892579+0000","last_unstale":"2024-09-13T08:37:36.530096+0000","last_undegraded":"2024-09-13T08:37:36.530096+0000","last_fullsized":"2024-09-13T08:37:36.530096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085280+0000","last_change":"2024-09-13T08:37:34.887215+0000","last_active":"2024-09-13T08:37:37.085280+0000","last_peered":"2024-09-13T08:37:37.085280+0000","last_clean":"2024-09-13T08:37:37.085280+0000","last_became_active":"2024-09-13T08:37:34.886889+0000","last_became_peered":"2024-09-13T08:37:34.886889+0000","last_unstale":"2024-09-13T08:37:37.085280+0000","last_undegraded":"2024-09-13T08:37:37.085280+0000","last_fullsized":"2024-09-13T08:37:37.085280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:57.113272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510921+0000","last_change":"2024-09-13T08:37:33.891105+0000","last_active":"2024-09-13T08:37:36.510921+0000","last_peered":"2024-09-13T08:37:36.510921+0000","last_clean":"2024-09-13T08:37:36.510921+0000","last_became_active":"2024-09-13T08:37:33.890783+0000","last_became_peered":"2024-09-13T08:37:33.890783+0000","last_unstale":"2024-09-13T08:37:36.510921+0000","last_undegraded":"2024-09-13T08:37:36.510921+0000","last_fullsized":"2024-09-13T08:37:36.510921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:30:45.112157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877636+0000","last_change":"2024-09-13T08:37:34.893268+0000","last_active":"2024-09-13T08:37:36.877636+0000","last_peered":"2024-09-13T08:37:36.877636+0000","last_clean":"2024-09-13T08:37:36.877636+0000","last_became_active":"2024-09-13T08:37:34.892944+0000","last_became_peered":"2024-09-13T08:37:34.892944+0000","last_unstale":"2024-09-13T08:37:36.877636+0000","last_undegraded":"2024-09-13T08:37:36.877636+0000","last_fullsized":"2024-09-13T08:37:36.877636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:45.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529953+0000","last_change":"2024-09-13T08:37:33.902610+0000","last_active":"2024-09-13T08:37:36.529953+0000","last_peered":"2024-09-13T08:37:36.529953+0000","last_clean":"2024-09-13T08:37:36.529953+0000","last_became_active":"2024-09-13T08:37:33.902275+0000","last_became_peered":"2024-09-13T08:37:33.902275+0000","last_unstale":"2024-09-13T08:37:36.529953+0000","last_undegraded":"2024-09-13T08:37:36.529953+0000","last_fullsized":"2024-09-13T08:37:36.529953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894679+0000","last_change":"2024-09-13T08:37:34.904983+0000","last_active":"2024-09-13T08:37:35.894679+0000","last_peered":"2024-09-13T08:37:35.894679+0000","last_clean":"2024-09-13T08:37:35.894679+0000","last_became_active":"2024-09-13T08:37:34.904618+0000","last_became_peered":"2024-09-13T08:37:34.904618+0000","last_unstale":"2024-09-13T08:37:35.894679+0000","last_undegraded":"2024-09-13T08:37:35.894679+0000","last_fullsized":"2024-09-13T08:37:35.894679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:30.070140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085096+0000","last_change":"2024-09-13T08:37:33.888874+0000","last_active":"2024-09-13T08:37:37.085096+0000","last_peered":"2024-09-13T08:37:37.085096+0000","last_clean":"2024-09-13T08:37:37.085096+0000","last_became_active":"2024-09-13T08:37:33.888111+0000","last_became_peered":"2024-09-13T08:37:33.888111+0000","last_unstale":"2024-09-13T08:37:37.085096+0000","last_undegraded":"2024-09-13T08:37:37.085096+0000","last_fullsized":"2024-09-13T08:37:37.085096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:55.141642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084977+0000","last_change":"2024-09-13T08:37:34.893432+0000","last_active":"2024-09-13T08:37:37.084977+0000","last_peered":"2024-09-13T08:37:37.084977+0000","last_clean":"2024-09-13T08:37:37.084977+0000","last_became_active":"2024-09-13T08:37:34.893160+0000","last_became_peered":"2024-09-13T08:37:34.893160+0000","last_unstale":"2024-09-13T08:37:37.084977+0000","last_undegraded":"2024-09-13T08:37:37.084977+0000","last_fullsized":"2024-09-13T08:37:37.084977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:46.900298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.876999+0000","last_change":"2024-09-13T08:37:33.892008+0000","last_active":"2024-09-13T08:37:36.876999+0000","last_peered":"2024-09-13T08:37:36.876999+0000","last_clean":"2024-09-13T08:37:36.876999+0000","last_became_active":"2024-09-13T08:37:33.891758+0000","last_became_peered":"2024-09-13T08:37:33.891758+0000","last_unstale":"2024-09-13T08:37:36.876999+0000","last_undegraded":"2024-09-13T08:37:36.876999+0000","last_fullsized":"2024-09-13T08:37:36.876999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:27.615101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891708+0000","last_change":"2024-09-13T08:37:34.889786+0000","last_active":"2024-09-13T08:37:35.891708+0000","last_peered":"2024-09-13T08:37:35.891708+0000","last_clean":"2024-09-13T08:37:35.891708+0000","last_became_active":"2024-09-13T08:37:34.889505+0000","last_became_peered":"2024-09-13T08:37:34.889505+0000","last_unstale":"2024-09-13T08:37:35.891708+0000","last_undegraded":"2024-09-13T08:37:35.891708+0000","last_fullsized":"2024-09-13T08:37:35.891708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:50.000602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585415+0000","last_change":"2024-09-13T08:37:33.888556+0000","last_active":"2024-09-13T08:37:36.585415+0000","last_peered":"2024-09-13T08:37:36.585415+0000","last_clean":"2024-09-13T08:37:36.585415+0000","last_became_active":"2024-09-13T08:37:33.887997+0000","last_became_peered":"2024-09-13T08:37:33.887997+0000","last_unstale":"2024-09-13T08:37:36.585415+0000","last_undegraded":"2024-09-13T08:37:36.585415+0000","last_fullsized":"2024-09-13T08:37:36.585415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084670+0000","last_change":"2024-09-13T08:37:34.908412+0000","last_active":"2024-09-13T08:37:37.084670+0000","last_peered":"2024-09-13T08:37:37.084670+0000","last_clean":"2024-09-13T08:37:37.084670+0000","last_became_active":"2024-09-13T08:37:34.908135+0000","last_became_peered":"2024-09-13T08:37:34.908135+0000","last_unstale":"2024-09-13T08:37:37.084670+0000","last_undegraded":"2024-09-13T08:37:37.084670+0000","last_fullsized":"2024-09-13T08:37:37.084670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:15.303677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877915+0000","last_change":"2024-09-13T08:37:33.895134+0000","last_active":"2024-09-13T08:37:36.877915+0000","last_peered":"2024-09-13T08:37:36.877915+0000","last_clean":"2024-09-13T08:37:36.877915+0000","last_became_active":"2024-09-13T08:37:33.894893+0000","last_became_peered":"2024-09-13T08:37:33.894893+0000","last_unstale":"2024-09-13T08:37:36.877915+0000","last_undegraded":"2024-09-13T08:37:36.877915+0000","last_fullsized":"2024-09-13T08:37:36.877915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:03:36.459606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511239+0000","last_change":"2024-09-13T08:37:34.915800+0000","last_active":"2024-09-13T08:37:36.511239+0000","last_peered":"2024-09-13T08:37:36.511239+0000","last_clean":"2024-09-13T08:37:36.511239+0000","last_became_active":"2024-09-13T08:37:34.915478+0000","last_became_peered":"2024-09-13T08:37:34.915478+0000","last_unstale":"2024-09-13T08:37:36.511239+0000","last_undegraded":"2024-09-13T08:37:36.511239+0000","last_fullsized":"2024-09-13T08:37:36.511239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:55.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586002+0000","last_change":"2024-09-13T08:37:33.888156+0000","last_active":"2024-09-13T08:37:36.586002+0000","last_peered":"2024-09-13T08:37:36.586002+0000","last_clean":"2024-09-13T08:37:36.586002+0000","last_became_active":"2024-09-13T08:37:33.886607+0000","last_became_peered":"2024-09-13T08:37:33.886607+0000","last_unstale":"2024-09-13T08:37:36.586002+0000","last_undegraded":"2024-09-13T08:37:36.586002+0000","last_fullsized":"2024-09-13T08:37:36.586002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:36.540294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877863+0000","last_change":"2024-09-13T08:37:34.891196+0000","last_active":"2024-09-13T08:37:36.877863+0000","last_peered":"2024-09-13T08:37:36.877863+0000","last_clean":"2024-09-13T08:37:36.877863+0000","last_became_active":"2024-09-13T08:37:34.890848+0000","last_became_peered":"2024-09-13T08:37:34.890848+0000","last_unstale":"2024-09-13T08:37:36.877863+0000","last_undegraded":"2024-09-13T08:37:36.877863+0000","last_fullsized":"2024-09-13T08:37:36.877863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:07.683461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896137+0000","last_change":"2024-09-13T08:37:33.885405+0000","last_active":"2024-09-13T08:37:35.896137+0000","last_peered":"2024-09-13T08:37:35.896137+0000","last_clean":"2024-09-13T08:37:35.896137+0000","last_became_active":"2024-09-13T08:37:33.885048+0000","last_became_peered":"2024-09-13T08:37:33.885048+0000","last_unstale":"2024-09-13T08:37:35.896137+0000","last_undegraded":"2024-09-13T08:37:35.896137+0000","last_fullsized":"2024-09-13T08:37:35.896137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:33:47.115618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877736+0000","last_change":"2024-09-13T08:37:34.898475+0000","last_active":"2024-09-13T08:37:36.877736+0000","last_peered":"2024-09-13T08:37:36.877736+0000","last_clean":"2024-09-13T08:37:36.877736+0000","last_became_active":"2024-09-13T08:37:34.897854+0000","last_became_peered":"2024-09-13T08:37:34.897854+0000","last_unstale":"2024-09-13T08:37:36.877736+0000","last_undegraded":"2024-09-13T08:37:36.877736+0000","last_fullsized":"2024-09-13T08:37:36.877736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:15.303677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895446+0000","last_change":"2024-09-13T08:37:33.902815+0000","last_active":"2024-09-13T08:37:35.895446+0000","last_peered":"2024-09-13T08:37:35.895446+0000","last_clean":"2024-09-13T08:37:35.895446+0000","last_became_active":"2024-09-13T08:37:33.902490+0000","last_became_peered":"2024-09-13T08:37:33.902490+0000","last_unstale":"2024-09-13T08:37:35.895446+0000","last_undegraded":"2024-09-13T08:37:35.895446+0000","last_fullsized":"2024-09-13T08:37:35.895446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:32.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085416+0000","last_change":"2024-09-13T08:37:34.891378+0000","last_active":"2024-09-13T08:37:37.085416+0000","last_peered":"2024-09-13T08:37:37.085416+0000","last_clean":"2024-09-13T08:37:37.085416+0000","last_became_active":"2024-09-13T08:37:34.885687+0000","last_became_peered":"2024-09-13T08:37:34.885687+0000","last_unstale":"2024-09-13T08:37:37.085416+0000","last_undegraded":"2024-09-13T08:37:37.085416+0000","last_fullsized":"2024-09-13T08:37:37.085416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:42.639292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891693+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:37:35.891693+0000","last_peered":"2024-09-13T08:37:35.891693+0000","last_clean":"2024-09-13T08:37:35.891693+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:37:35.891693+0000","last_undegraded":"2024-09-13T08:37:35.891693+0000","last_fullsized":"2024-09-13T08:37:35.891693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085218+0000","last_change":"2024-09-13T08:37:33.883826+0000","last_active":"2024-09-13T08:37:37.085218+0000","last_peered":"2024-09-13T08:37:37.085218+0000","last_clean":"2024-09-13T08:37:37.085218+0000","last_became_active":"2024-09-13T08:37:33.883511+0000","last_became_peered":"2024-09-13T08:37:33.883511+0000","last_unstale":"2024-09-13T08:37:37.085218+0000","last_undegraded":"2024-09-13T08:37:37.085218+0000","last_fullsized":"2024-09-13T08:37:37.085218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:39:48.743211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585719+0000","last_change":"2024-09-13T08:37:34.901372+0000","last_active":"2024-09-13T08:37:36.585719+0000","last_peered":"2024-09-13T08:37:36.585719+0000","last_clean":"2024-09-13T08:37:36.585719+0000","last_became_active":"2024-09-13T08:37:34.900931+0000","last_became_peered":"2024-09-13T08:37:34.900931+0000","last_unstale":"2024-09-13T08:37:36.585719+0000","last_undegraded":"2024-09-13T08:37:36.585719+0000","last_fullsized":"2024-09-13T08:37:36.585719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:16.328343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084832+0000","last_change":"2024-09-13T08:37:33.885219+0000","last_active":"2024-09-13T08:37:37.084832+0000","last_peered":"2024-09-13T08:37:37.084832+0000","last_clean":"2024-09-13T08:37:37.084832+0000","last_became_active":"2024-09-13T08:37:33.884889+0000","last_became_peered":"2024-09-13T08:37:33.884889+0000","last_unstale":"2024-09-13T08:37:37.084832+0000","last_undegraded":"2024-09-13T08:37:37.084832+0000","last_fullsized":"2024-09-13T08:37:37.084832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:55.495409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894451+0000","last_change":"2024-09-13T08:37:34.908186+0000","last_active":"2024-09-13T08:37:35.894451+0000","last_peered":"2024-09-13T08:37:35.894451+0000","last_clean":"2024-09-13T08:37:35.894451+0000","last_became_active":"2024-09-13T08:37:34.907886+0000","last_became_peered":"2024-09-13T08:37:34.907886+0000","last_unstale":"2024-09-13T08:37:35.894451+0000","last_undegraded":"2024-09-13T08:37:35.894451+0000","last_fullsized":"2024-09-13T08:37:35.894451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:11:14.712673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529782+0000","last_change":"2024-09-13T08:37:33.890556+0000","last_active":"2024-09-13T08:37:36.529782+0000","last_peered":"2024-09-13T08:37:36.529782+0000","last_clean":"2024-09-13T08:37:36.529782+0000","last_became_active":"2024-09-13T08:37:33.890252+0000","last_became_peered":"2024-09-13T08:37:33.890252+0000","last_unstale":"2024-09-13T08:37:36.529782+0000","last_undegraded":"2024-09-13T08:37:36.529782+0000","last_fullsized":"2024-09-13T08:37:36.529782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894861+0000","last_change":"2024-09-13T08:37:34.901633+0000","last_active":"2024-09-13T08:37:35.894861+0000","last_peered":"2024-09-13T08:37:35.894861+0000","last_clean":"2024-09-13T08:37:35.894861+0000","last_became_active":"2024-09-13T08:37:34.901323+0000","last_became_peered":"2024-09-13T08:37:34.901323+0000","last_unstale":"2024-09-13T08:37:35.894861+0000","last_undegraded":"2024-09-13T08:37:35.894861+0000","last_fullsized":"2024-09-13T08:37:35.894861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:37.744328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085159+0000","last_change":"2024-09-13T08:37:33.889192+0000","last_active":"2024-09-13T08:37:37.085159+0000","last_peered":"2024-09-13T08:37:37.085159+0000","last_clean":"2024-09-13T08:37:37.085159+0000","last_became_active":"2024-09-13T08:37:33.888556+0000","last_became_peered":"2024-09-13T08:37:33.888556+0000","last_unstale":"2024-09-13T08:37:37.085159+0000","last_undegraded":"2024-09-13T08:37:37.085159+0000","last_fullsized":"2024-09-13T08:37:37.085159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:55.900198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877366+0000","last_change":"2024-09-13T08:37:34.905109+0000","last_active":"2024-09-13T08:37:36.877366+0000","last_peered":"2024-09-13T08:37:36.877366+0000","last_clean":"2024-09-13T08:37:36.877366+0000","last_became_active":"2024-09-13T08:37:34.904766+0000","last_became_peered":"2024-09-13T08:37:34.904766+0000","last_unstale":"2024-09-13T08:37:36.877366+0000","last_undegraded":"2024-09-13T08:37:36.877366+0000","last_fullsized":"2024-09-13T08:37:36.877366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:54.979663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895896+0000","last_change":"2024-09-13T08:37:33.889699+0000","last_active":"2024-09-13T08:37:35.895896+0000","last_peered":"2024-09-13T08:37:35.895896+0000","last_clean":"2024-09-13T08:37:35.895896+0000","last_became_active":"2024-09-13T08:37:33.889224+0000","last_became_peered":"2024-09-13T08:37:33.889224+0000","last_unstale":"2024-09-13T08:37:35.895896+0000","last_undegraded":"2024-09-13T08:37:35.895896+0000","last_fullsized":"2024-09-13T08:37:35.895896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:24.413843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510786+0000","last_change":"2024-09-13T08:37:34.901224+0000","last_active":"2024-09-13T08:37:36.510786+0000","last_peered":"2024-09-13T08:37:36.510786+0000","last_clean":"2024-09-13T08:37:36.510786+0000","last_became_active":"2024-09-13T08:37:34.900890+0000","last_became_peered":"2024-09-13T08:37:34.900890+0000","last_unstale":"2024-09-13T08:37:36.510786+0000","last_undegraded":"2024-09-13T08:37:36.510786+0000","last_fullsized":"2024-09-13T08:37:36.510786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:06.047971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085567+0000","last_change":"2024-09-13T08:37:33.893542+0000","last_active":"2024-09-13T08:37:37.085567+0000","last_peered":"2024-09-13T08:37:37.085567+0000","last_clean":"2024-09-13T08:37:37.085567+0000","last_became_active":"2024-09-13T08:37:33.893231+0000","last_became_peered":"2024-09-13T08:37:33.893231+0000","last_unstale":"2024-09-13T08:37:37.085567+0000","last_undegraded":"2024-09-13T08:37:37.085567+0000","last_fullsized":"2024-09-13T08:37:37.085567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:48.992381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895292+0000","last_change":"2024-09-13T08:37:34.901999+0000","last_active":"2024-09-13T08:37:35.895292+0000","last_peered":"2024-09-13T08:37:35.895292+0000","last_clean":"2024-09-13T08:37:35.895292+0000","last_became_active":"2024-09-13T08:37:34.901765+0000","last_became_peered":"2024-09-13T08:37:34.901765+0000","last_unstale":"2024-09-13T08:37:35.895292+0000","last_undegraded":"2024-09-13T08:37:35.895292+0000","last_fullsized":"2024-09-13T08:37:35.895292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:12.422693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586075+0000","last_change":"2024-09-13T08:37:33.898217+0000","last_active":"2024-09-13T08:37:36.586075+0000","last_peered":"2024-09-13T08:37:36.586075+0000","last_clean":"2024-09-13T08:37:36.586075+0000","last_became_active":"2024-09-13T08:37:33.897954+0000","last_became_peered":"2024-09-13T08:37:33.897954+0000","last_unstale":"2024-09-13T08:37:36.586075+0000","last_undegraded":"2024-09-13T08:37:36.586075+0000","last_fullsized":"2024-09-13T08:37:36.586075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:05.995257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586135+0000","last_change":"2024-09-13T08:37:34.897630+0000","last_active":"2024-09-13T08:37:36.586135+0000","last_peered":"2024-09-13T08:37:36.586135+0000","last_clean":"2024-09-13T08:37:36.586135+0000","last_became_active":"2024-09-13T08:37:34.896603+0000","last_became_peered":"2024-09-13T08:37:34.896603+0000","last_unstale":"2024-09-13T08:37:36.586135+0000","last_undegraded":"2024-09-13T08:37:36.586135+0000","last_fullsized":"2024-09-13T08:37:36.586135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085624+0000","last_change":"2024-09-13T08:37:33.897139+0000","last_active":"2024-09-13T08:37:37.085624+0000","last_peered":"2024-09-13T08:37:37.085624+0000","last_clean":"2024-09-13T08:37:37.085624+0000","last_became_active":"2024-09-13T08:37:33.896576+0000","last_became_peered":"2024-09-13T08:37:33.896576+0000","last_unstale":"2024-09-13T08:37:37.085624+0000","last_undegraded":"2024-09-13T08:37:37.085624+0000","last_fullsized":"2024-09-13T08:37:37.085624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:54.815810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585844+0000","last_change":"2024-09-13T08:37:34.896701+0000","last_active":"2024-09-13T08:37:36.585844+0000","last_peered":"2024-09-13T08:37:36.585844+0000","last_clean":"2024-09-13T08:37:36.585844+0000","last_became_active":"2024-09-13T08:37:34.893772+0000","last_became_peered":"2024-09-13T08:37:34.893772+0000","last_unstale":"2024-09-13T08:37:36.585844+0000","last_undegraded":"2024-09-13T08:37:36.585844+0000","last_fullsized":"2024-09-13T08:37:36.585844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:34.291551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.876764+0000","last_change":"2024-09-13T08:37:33.893088+0000","last_active":"2024-09-13T08:37:36.876764+0000","last_peered":"2024-09-13T08:37:36.876764+0000","last_clean":"2024-09-13T08:37:36.876764+0000","last_became_active":"2024-09-13T08:37:33.892731+0000","last_became_peered":"2024-09-13T08:37:33.892731+0000","last_unstale":"2024-09-13T08:37:36.876764+0000","last_undegraded":"2024-09-13T08:37:36.876764+0000","last_fullsized":"2024-09-13T08:37:36.876764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084725+0000","last_change":"2024-09-13T08:37:34.895132+0000","last_active":"2024-09-13T08:37:37.084725+0000","last_peered":"2024-09-13T08:37:37.084725+0000","last_clean":"2024-09-13T08:37:37.084725+0000","last_became_active":"2024-09-13T08:37:34.894785+0000","last_became_peered":"2024-09-13T08:37:34.894785+0000","last_unstale":"2024-09-13T08:37:37.084725+0000","last_undegraded":"2024-09-13T08:37:37.084725+0000","last_fullsized":"2024-09-13T08:37:37.084725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:07.683461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585602+0000","last_change":"2024-09-13T08:37:33.898275+0000","last_active":"2024-09-13T08:37:36.585602+0000","last_peered":"2024-09-13T08:37:36.585602+0000","last_clean":"2024-09-13T08:37:36.585602+0000","last_became_active":"2024-09-13T08:37:33.897960+0000","last_became_peered":"2024-09-13T08:37:33.897960+0000","last_unstale":"2024-09-13T08:37:36.585602+0000","last_undegraded":"2024-09-13T08:37:36.585602+0000","last_fullsized":"2024-09-13T08:37:36.585602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:22.150995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585465+0000","last_change":"2024-09-13T08:37:34.891285+0000","last_active":"2024-09-13T08:37:36.585465+0000","last_peered":"2024-09-13T08:37:36.585465+0000","last_clean":"2024-09-13T08:37:36.585465+0000","last_became_active":"2024-09-13T08:37:34.890996+0000","last_became_peered":"2024-09-13T08:37:34.890996+0000","last_unstale":"2024-09-13T08:37:36.585465+0000","last_undegraded":"2024-09-13T08:37:36.585465+0000","last_fullsized":"2024-09-13T08:37:36.585465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:47.564367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894655+0000","last_change":"2024-09-13T08:37:33.891510+0000","last_active":"2024-09-13T08:37:35.894655+0000","last_peered":"2024-09-13T08:37:35.894655+0000","last_clean":"2024-09-13T08:37:35.894655+0000","last_became_active":"2024-09-13T08:37:33.890774+0000","last_became_peered":"2024-09-13T08:37:33.890774+0000","last_unstale":"2024-09-13T08:37:35.894655+0000","last_undegraded":"2024-09-13T08:37:35.894655+0000","last_fullsized":"2024-09-13T08:37:35.894655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:25.370931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529890+0000","last_change":"2024-09-13T08:37:34.895928+0000","last_active":"2024-09-13T08:37:36.529890+0000","last_peered":"2024-09-13T08:37:36.529890+0000","last_clean":"2024-09-13T08:37:36.529890+0000","last_became_active":"2024-09-13T08:37:34.895612+0000","last_became_peered":"2024-09-13T08:37:34.895612+0000","last_unstale":"2024-09-13T08:37:36.529890+0000","last_undegraded":"2024-09-13T08:37:36.529890+0000","last_fullsized":"2024-09-13T08:37:36.529890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085214+0000","last_change":"2024-09-13T08:37:33.887243+0000","last_active":"2024-09-13T08:37:37.085214+0000","last_peered":"2024-09-13T08:37:37.085214+0000","last_clean":"2024-09-13T08:37:37.085214+0000","last_became_active":"2024-09-13T08:37:33.886882+0000","last_became_peered":"2024-09-13T08:37:33.886882+0000","last_unstale":"2024-09-13T08:37:37.085214+0000","last_undegraded":"2024-09-13T08:37:37.085214+0000","last_fullsized":"2024-09-13T08:37:37.085214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:31.044508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084923+0000","last_change":"2024-09-13T08:37:34.894740+0000","last_active":"2024-09-13T08:37:37.084923+0000","last_peered":"2024-09-13T08:37:37.084923+0000","last_clean":"2024-09-13T08:37:37.084923+0000","last_became_active":"2024-09-13T08:37:34.894284+0000","last_became_peered":"2024-09-13T08:37:34.894284+0000","last_unstale":"2024-09-13T08:37:37.084923+0000","last_undegraded":"2024-09-13T08:37:37.084923+0000","last_fullsized":"2024-09-13T08:37:37.084923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:28.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085351+0000","last_change":"2024-09-13T08:37:34.892338+0000","last_active":"2024-09-13T08:37:37.085351+0000","last_peered":"2024-09-13T08:37:37.085351+0000","last_clean":"2024-09-13T08:37:37.085351+0000","last_became_active":"2024-09-13T08:37:34.891790+0000","last_became_peered":"2024-09-13T08:37:34.891790+0000","last_unstale":"2024-09-13T08:37:37.085351+0000","last_undegraded":"2024-09-13T08:37:37.085351+0000","last_fullsized":"2024-09-13T08:37:37.085351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:24:06.938838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877706+0000","last_change":"2024-09-13T08:37:33.900929+0000","last_active":"2024-09-13T08:37:36.877706+0000","last_peered":"2024-09-13T08:37:36.877706+0000","last_clean":"2024-09-13T08:37:36.877706+0000","last_became_active":"2024-09-13T08:37:33.899776+0000","last_became_peered":"2024-09-13T08:37:33.899776+0000","last_unstale":"2024-09-13T08:37:36.877706+0000","last_undegraded":"2024-09-13T08:37:36.877706+0000","last_fullsized":"2024-09-13T08:37:36.877706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:27.711361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892184+0000","last_change":"2024-09-13T08:37:34.893833+0000","last_active":"2024-09-13T08:37:35.892184+0000","last_peered":"2024-09-13T08:37:35.892184+0000","last_clean":"2024-09-13T08:37:35.892184+0000","last_became_active":"2024-09-13T08:37:34.893512+0000","last_became_peered":"2024-09-13T08:37:34.893512+0000","last_unstale":"2024-09-13T08:37:35.892184+0000","last_undegraded":"2024-09-13T08:37:35.892184+0000","last_fullsized":"2024-09-13T08:37:35.892184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877847+0000","last_change":"2024-09-13T08:37:33.894985+0000","last_active":"2024-09-13T08:37:36.877847+0000","last_peered":"2024-09-13T08:37:36.877847+0000","last_clean":"2024-09-13T08:37:36.877847+0000","last_became_active":"2024-09-13T08:37:33.894087+0000","last_became_peered":"2024-09-13T08:37:33.894087+0000","last_unstale":"2024-09-13T08:37:36.877847+0000","last_undegraded":"2024-09-13T08:37:36.877847+0000","last_fullsized":"2024-09-13T08:37:36.877847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:51.688613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896328+0000","last_change":"2024-09-13T08:37:34.901345+0000","last_active":"2024-09-13T08:37:35.896328+0000","last_peered":"2024-09-13T08:37:35.896328+0000","last_clean":"2024-09-13T08:37:35.896328+0000","last_became_active":"2024-09-13T08:37:34.900290+0000","last_became_peered":"2024-09-13T08:37:34.900290+0000","last_unstale":"2024-09-13T08:37:35.896328+0000","last_undegraded":"2024-09-13T08:37:35.896328+0000","last_fullsized":"2024-09-13T08:37:35.896328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:08.493271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896451+0000","last_change":"2024-09-13T08:37:33.887848+0000","last_active":"2024-09-13T08:37:35.896451+0000","last_peered":"2024-09-13T08:37:35.896451+0000","last_clean":"2024-09-13T08:37:35.896451+0000","last_became_active":"2024-09-13T08:37:33.887508+0000","last_became_peered":"2024-09-13T08:37:33.887508+0000","last_unstale":"2024-09-13T08:37:35.896451+0000","last_undegraded":"2024-09-13T08:37:35.896451+0000","last_fullsized":"2024-09-13T08:37:35.896451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:27.615101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896328+0000","last_change":"2024-09-13T08:37:34.907107+0000","last_active":"2024-09-13T08:37:35.896328+0000","last_peered":"2024-09-13T08:37:35.896328+0000","last_clean":"2024-09-13T08:37:35.896328+0000","last_became_active":"2024-09-13T08:37:34.904160+0000","last_became_peered":"2024-09-13T08:37:34.904160+0000","last_unstale":"2024-09-13T08:37:35.896328+0000","last_undegraded":"2024-09-13T08:37:35.896328+0000","last_fullsized":"2024-09-13T08:37:35.896328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:55.775775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586114+0000","last_change":"2024-09-13T08:37:33.894082+0000","last_active":"2024-09-13T08:37:36.586114+0000","last_peered":"2024-09-13T08:37:36.586114+0000","last_clean":"2024-09-13T08:37:36.586114+0000","last_became_active":"2024-09-13T08:37:33.893820+0000","last_became_peered":"2024-09-13T08:37:33.893820+0000","last_unstale":"2024-09-13T08:37:36.586114+0000","last_undegraded":"2024-09-13T08:37:36.586114+0000","last_fullsized":"2024-09-13T08:37:36.586114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:48.992381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:37:43.672 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:37:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.827+0000 7f09ebc00640 1 -- 172.21.15.44:0/543252449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cdb30 msgr2=0x7f09e40af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.827+0000 7f09ebc00640 1 --2- 172.21.15.44:0/543252449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cdb30 0x7f09e40af0c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009bc0 tx=0x7f09dc01cba0 comp rx=0 tx=0).stop 2024-09-13T08:37:43.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.827+0000 7f09ebc00640 1 -- 172.21.15.44:0/543252449 shutdown_connections 2024-09-13T08:37:43.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.827+0000 7f09ebc00640 1 --2- 172.21.15.44:0/543252449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cdb30 0x7f09e40af0c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:43.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.827+0000 7f09ebc00640 1 -- 172.21.15.44:0/543252449 >> 172.21.15.44:0/543252449 conn(0x7f09e40d1dc0 msgr2=0x7f09e40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:43.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.827+0000 7f09ebc00640 1 -- 172.21.15.44:0/543252449 shutdown_connections 2024-09-13T08:37:43.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.827+0000 7f09ebc00640 1 -- 172.21.15.44:0/543252449 wait complete. 2024-09-13T08:37:43.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.827+0000 7f09ebc00640 1 Processor -- start 2024-09-13T08:37:43.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.827+0000 7f09ebc00640 1 -- start start 2024-09-13T08:37:43.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09ebc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f09e40cdb30 0x7f09e40c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:43.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09ebc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09e40ced80 0x7f09e40b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:43.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09ebc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e415bea0 0x7f09e41fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:43.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09ebc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f09e416a8a0 con 0x7f09e415bea0 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09ebc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f09e416a720 con 0x7f09e40ced80 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09ebc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f09e416aa20 con 0x7f09e40cdb30 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e415bea0 0x7f09e41fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e415bea0 0x7f09e41fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36206/0 (socket says 172.21.15.44:36206) 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9c00640 1 -- 172.21.15.44:0/2108526683 learned_addr learned my addr 172.21.15.44:0/2108526683 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e3e00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09e40ced80 0x7f09e40b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9200640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f09e40cdb30 0x7f09e40c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9c00640 1 -- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f09e40cdb30 msgr2=0x7f09e40c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9c00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f09e40cdb30 0x7f09e40c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9c00640 1 -- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09e40ced80 msgr2=0x7f09e40b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9c00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09e40ced80 0x7f09e40b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9c00640 1 -- 172.21.15.44:0/2108526683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e409bf10 con 0x7f09e415bea0 2024-09-13T08:37:43.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9200640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f09e40cdb30 0x7f09e40c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:43.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e9c00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e415bea0 0x7f09e41fb8a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009b10 tx=0x7f09dc0a5b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:43.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e0c00640 1 -- 172.21.15.44:0/2108526683 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0bb070 con 0x7f09e415bea0 2024-09-13T08:37:43.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e0c00640 1 -- 172.21.15.44:0/2108526683 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09dc004e00 con 0x7f09e415bea0 2024-09-13T08:37:43.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09e0c00640 1 -- 172.21.15.44:0/2108526683 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0b2040 con 0x7f09e415bea0 2024-09-13T08:37:43.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09ebc00640 1 -- 172.21.15.44:0/2108526683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e40b1520 con 0x7f09e415bea0 2024-09-13T08:37:43.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09ebc00640 1 -- 172.21.15.44:0/2108526683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e409bb30 con 0x7f09e415bea0 2024-09-13T08:37:43.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.831+0000 7f09ebc00640 1 -- 172.21.15.44:0/2108526683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e40b01a0 con 0x7f09e415bea0 2024-09-13T08:37:43.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.835+0000 7f09e0c00640 1 -- 172.21.15.44:0/2108526683 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f09dc004360 con 0x7f09e415bea0 2024-09-13T08:37:43.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.835+0000 7f09e0c00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f09c4072ec0 0x7f09c4075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:43.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.835+0000 7f09e0c00640 1 -- 172.21.15.44:0/2108526683 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f09dc12dd40 con 0x7f09e415bea0 2024-09-13T08:37:43.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.835+0000 7f09e9200640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f09c4072ec0 0x7f09c4075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:43.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.835+0000 7f09e9200640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f09c4072ec0 0x7f09c4075380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f09d80016b0 tx=0x7f09d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:43.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:43.839+0000 7f09e0c00640 1 -- 172.21.15.44:0/2108526683 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f09dc0a5e20 con 0x7f09e415bea0 2024-09-13T08:37:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.011+0000 7f09ebc00640 1 -- 172.21.15.44:0/2108526683 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09e40af2a0 con 0x7f09c4072ec0 2024-09-13T08:37:44.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.015+0000 7f09e0c00640 1 -- 172.21.15.44:0/2108526683 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7f09e40af2a0 con 0x7f09c4072ec0 2024-09-13T08:37:44.022 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:37:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 -- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f09c4072ec0 msgr2=0x7f09c4075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f09c4072ec0 0x7f09c4075380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f09d80016b0 tx=0x7f09d8009040 comp rx=0 tx=0).stop 2024-09-13T08:37:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 -- 172.21.15.44:0/2108526683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e415bea0 msgr2=0x7f09e41fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e415bea0 0x7f09e41fb8a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009b10 tx=0x7f09dc0a5b00 comp rx=0 tx=0).stop 2024-09-13T08:37:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 -- 172.21.15.44:0/2108526683 shutdown_connections 2024-09-13T08:37:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f09c4072ec0 0x7f09c4075380 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e415bea0 0x7f09e41fb8a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09e40ced80 0x7f09e40b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 --2- 172.21.15.44:0/2108526683 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f09e40cdb30 0x7f09e40c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 -- 172.21.15.44:0/2108526683 >> 172.21.15.44:0/2108526683 conn(0x7f09e40d1dc0 msgr2=0x7f09e40d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 -- 172.21.15.44:0/2108526683 shutdown_connections 2024-09-13T08:37:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.019+0000 7f09ebc00640 1 -- 172.21.15.44:0/2108526683 wait complete. 2024-09-13T08:37:44.044 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-13T08:37:42.512535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219224,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755496,"statfs":{"total":773094113280,"available":772869627904,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002241"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877562+0000","last_change":"2024-09-13T08:37:34.901083+0000","last_active":"2024-09-13T08:37:36.877562+0000","last_peered":"2024-09-13T08:37:36.877562+0000","last_clean":"2024-09-13T08:37:36.877562+0000","last_became_active":"2024-09-13T08:37:34.900414+0000","last_became_peered":"2024-09-13T08:37:34.900414+0000","last_unstale":"2024-09-13T08:37:36.877562+0000","last_undegraded":"2024-09-13T08:37:36.877562+0000","last_fullsized":"2024-09-13T08:37:36.877562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:28:50.949101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896264+0000","last_change":"2024-09-13T08:37:34.900607+0000","last_active":"2024-09-13T08:37:35.896264+0000","last_peered":"2024-09-13T08:37:35.896264+0000","last_clean":"2024-09-13T08:37:35.896264+0000","last_became_active":"2024-09-13T08:37:34.900222+0000","last_became_peered":"2024-09-13T08:37:34.900222+0000","last_unstale":"2024-09-13T08:37:35.896264+0000","last_undegraded":"2024-09-13T08:37:35.896264+0000","last_fullsized":"2024-09-13T08:37:35.896264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:47.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895933+0000","last_change":"2024-09-13T08:37:34.906335+0000","last_active":"2024-09-13T08:37:35.895933+0000","last_peered":"2024-09-13T08:37:35.895933+0000","last_clean":"2024-09-13T08:37:35.895933+0000","last_became_active":"2024-09-13T08:37:34.905961+0000","last_became_peered":"2024-09-13T08:37:34.905961+0000","last_unstale":"2024-09-13T08:37:35.895933+0000","last_undegraded":"2024-09-13T08:37:35.895933+0000","last_fullsized":"2024-09-13T08:37:35.895933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:54.979663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510756+0000","last_change":"2024-09-13T08:37:34.904837+0000","last_active":"2024-09-13T08:37:36.510756+0000","last_peered":"2024-09-13T08:37:36.510756+0000","last_clean":"2024-09-13T08:37:36.510756+0000","last_became_active":"2024-09-13T08:37:34.898571+0000","last_became_peered":"2024-09-13T08:37:34.898571+0000","last_unstale":"2024-09-13T08:37:36.510756+0000","last_undegraded":"2024-09-13T08:37:36.510756+0000","last_fullsized":"2024-09-13T08:37:36.510756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877279+0000","last_change":"2024-09-13T08:37:34.888564+0000","last_active":"2024-09-13T08:37:36.877279+0000","last_peered":"2024-09-13T08:37:36.877279+0000","last_clean":"2024-09-13T08:37:36.877279+0000","last_became_active":"2024-09-13T08:37:34.888280+0000","last_became_peered":"2024-09-13T08:37:34.888280+0000","last_unstale":"2024-09-13T08:37:36.877279+0000","last_undegraded":"2024-09-13T08:37:36.877279+0000","last_fullsized":"2024-09-13T08:37:36.877279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:27:32.330596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510582+0000","last_change":"2024-09-13T08:37:34.895044+0000","last_active":"2024-09-13T08:37:36.510582+0000","last_peered":"2024-09-13T08:37:36.510582+0000","last_clean":"2024-09-13T08:37:36.510582+0000","last_became_active":"2024-09-13T08:37:34.894777+0000","last_became_peered":"2024-09-13T08:37:34.894777+0000","last_unstale":"2024-09-13T08:37:36.510582+0000","last_undegraded":"2024-09-13T08:37:36.510582+0000","last_fullsized":"2024-09-13T08:37:36.510582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877014+0000","last_change":"2024-09-13T08:37:34.888796+0000","last_active":"2024-09-13T08:37:36.877014+0000","last_peered":"2024-09-13T08:37:36.877014+0000","last_clean":"2024-09-13T08:37:36.877014+0000","last_became_active":"2024-09-13T08:37:34.888488+0000","last_became_peered":"2024-09-13T08:37:34.888488+0000","last_unstale":"2024-09-13T08:37:36.877014+0000","last_undegraded":"2024-09-13T08:37:36.877014+0000","last_fullsized":"2024-09-13T08:37:36.877014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894878+0000","last_change":"2024-09-13T08:37:34.910054+0000","last_active":"2024-09-13T08:37:35.894878+0000","last_peered":"2024-09-13T08:37:35.894878+0000","last_clean":"2024-09-13T08:37:35.894878+0000","last_became_active":"2024-09-13T08:37:34.909596+0000","last_became_peered":"2024-09-13T08:37:34.909596+0000","last_unstale":"2024-09-13T08:37:35.894878+0000","last_undegraded":"2024-09-13T08:37:35.894878+0000","last_fullsized":"2024-09-13T08:37:35.894878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:55.615355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892431+0000","last_change":"2024-09-13T08:37:34.885156+0000","last_active":"2024-09-13T08:37:35.892431+0000","last_peered":"2024-09-13T08:37:35.892431+0000","last_clean":"2024-09-13T08:37:35.892431+0000","last_became_active":"2024-09-13T08:37:34.884851+0000","last_became_peered":"2024-09-13T08:37:34.884851+0000","last_unstale":"2024-09-13T08:37:35.892431+0000","last_undegraded":"2024-09-13T08:37:35.892431+0000","last_fullsized":"2024-09-13T08:37:35.892431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:06.047971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877522+0000","last_change":"2024-09-13T08:37:34.909843+0000","last_active":"2024-09-13T08:37:36.877522+0000","last_peered":"2024-09-13T08:37:36.877522+0000","last_clean":"2024-09-13T08:37:36.877522+0000","last_became_active":"2024-09-13T08:37:34.909552+0000","last_became_peered":"2024-09-13T08:37:34.909552+0000","last_unstale":"2024-09-13T08:37:36.877522+0000","last_undegraded":"2024-09-13T08:37:36.877522+0000","last_fullsized":"2024-09-13T08:37:36.877522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:08.493271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877666+0000","last_change":"2024-09-13T08:37:34.902054+0000","last_active":"2024-09-13T08:37:36.877666+0000","last_peered":"2024-09-13T08:37:36.877666+0000","last_clean":"2024-09-13T08:37:36.877666+0000","last_became_active":"2024-09-13T08:37:34.901788+0000","last_became_peered":"2024-09-13T08:37:34.901788+0000","last_unstale":"2024-09-13T08:37:36.877666+0000","last_undegraded":"2024-09-13T08:37:36.877666+0000","last_fullsized":"2024-09-13T08:37:36.877666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:14.002956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877261+0000","last_change":"2024-09-13T08:37:34.892602+0000","last_active":"2024-09-13T08:37:36.877261+0000","last_peered":"2024-09-13T08:37:36.877261+0000","last_clean":"2024-09-13T08:37:36.877261+0000","last_became_active":"2024-09-13T08:37:34.890000+0000","last_became_peered":"2024-09-13T08:37:34.890000+0000","last_unstale":"2024-09-13T08:37:36.877261+0000","last_undegraded":"2024-09-13T08:37:36.877261+0000","last_fullsized":"2024-09-13T08:37:36.877261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:06.530173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894980+0000","last_change":"2024-09-13T08:37:34.903037+0000","last_active":"2024-09-13T08:37:35.894980+0000","last_peered":"2024-09-13T08:37:35.894980+0000","last_clean":"2024-09-13T08:37:35.894980+0000","last_became_active":"2024-09-13T08:37:34.894805+0000","last_became_peered":"2024-09-13T08:37:34.894805+0000","last_unstale":"2024-09-13T08:37:35.894980+0000","last_undegraded":"2024-09-13T08:37:35.894980+0000","last_fullsized":"2024-09-13T08:37:35.894980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:49.712241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894573+0000","last_change":"2024-09-13T08:37:34.901603+0000","last_active":"2024-09-13T08:37:35.894573+0000","last_peered":"2024-09-13T08:37:35.894573+0000","last_clean":"2024-09-13T08:37:35.894573+0000","last_became_active":"2024-09-13T08:37:34.899884+0000","last_became_peered":"2024-09-13T08:37:34.899884+0000","last_unstale":"2024-09-13T08:37:35.894573+0000","last_undegraded":"2024-09-13T08:37:35.894573+0000","last_fullsized":"2024-09-13T08:37:35.894573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:05.260600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895761+0000","last_change":"2024-09-13T08:37:34.896921+0000","last_active":"2024-09-13T08:37:35.895761+0000","last_peered":"2024-09-13T08:37:35.895761+0000","last_clean":"2024-09-13T08:37:35.895761+0000","last_became_active":"2024-09-13T08:37:34.896593+0000","last_became_peered":"2024-09-13T08:37:34.896593+0000","last_unstale":"2024-09-13T08:37:35.895761+0000","last_undegraded":"2024-09-13T08:37:35.895761+0000","last_fullsized":"2024-09-13T08:37:35.895761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:06.530173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084616+0000","last_change":"2024-09-13T08:37:34.908031+0000","last_active":"2024-09-13T08:37:37.084616+0000","last_peered":"2024-09-13T08:37:37.084616+0000","last_clean":"2024-09-13T08:37:37.084616+0000","last_became_active":"2024-09-13T08:37:34.907734+0000","last_became_peered":"2024-09-13T08:37:34.907734+0000","last_unstale":"2024-09-13T08:37:37.084616+0000","last_undegraded":"2024-09-13T08:37:37.084616+0000","last_fullsized":"2024-09-13T08:37:37.084616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:14.002956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085576+0000","last_change":"2024-09-13T08:37:34.911572+0000","last_active":"2024-09-13T08:37:37.085576+0000","last_peered":"2024-09-13T08:37:37.085576+0000","last_clean":"2024-09-13T08:37:37.085576+0000","last_became_active":"2024-09-13T08:37:34.911105+0000","last_became_peered":"2024-09-13T08:37:34.911105+0000","last_unstale":"2024-09-13T08:37:37.085576+0000","last_undegraded":"2024-09-13T08:37:37.085576+0000","last_fullsized":"2024-09-13T08:37:37.085576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:49:35.223067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.530220+0000","last_change":"2024-09-13T08:37:34.907767+0000","last_active":"2024-09-13T08:37:36.530220+0000","last_peered":"2024-09-13T08:37:36.530220+0000","last_clean":"2024-09-13T08:37:36.530220+0000","last_became_active":"2024-09-13T08:37:34.907100+0000","last_became_peered":"2024-09-13T08:37:34.907100+0000","last_unstale":"2024-09-13T08:37:36.530220+0000","last_undegraded":"2024-09-13T08:37:36.530220+0000","last_fullsized":"2024-09-13T08:37:36.530220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085543+0000","last_change":"2024-09-13T08:37:34.895822+0000","last_active":"2024-09-13T08:37:37.085543+0000","last_peered":"2024-09-13T08:37:37.085543+0000","last_clean":"2024-09-13T08:37:37.085543+0000","last_became_active":"2024-09-13T08:37:34.895430+0000","last_became_peered":"2024-09-13T08:37:34.895430+0000","last_unstale":"2024-09-13T08:37:37.085543+0000","last_undegraded":"2024-09-13T08:37:37.085543+0000","last_fullsized":"2024-09-13T08:37:37.085543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:33:54.897556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877407+0000","last_change":"2024-09-13T08:37:34.891580+0000","last_active":"2024-09-13T08:37:36.877407+0000","last_peered":"2024-09-13T08:37:36.877407+0000","last_clean":"2024-09-13T08:37:36.877407+0000","last_became_active":"2024-09-13T08:37:34.889710+0000","last_became_peered":"2024-09-13T08:37:34.889710+0000","last_unstale":"2024-09-13T08:37:36.877407+0000","last_undegraded":"2024-09-13T08:37:36.877407+0000","last_fullsized":"2024-09-13T08:37:36.877407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:47.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585799+0000","last_change":"2024-09-13T08:37:34.894615+0000","last_active":"2024-09-13T08:37:36.585799+0000","last_peered":"2024-09-13T08:37:36.585799+0000","last_clean":"2024-09-13T08:37:36.585799+0000","last_became_active":"2024-09-13T08:37:34.894327+0000","last_became_peered":"2024-09-13T08:37:34.894327+0000","last_unstale":"2024-09-13T08:37:36.585799+0000","last_undegraded":"2024-09-13T08:37:36.585799+0000","last_fullsized":"2024-09-13T08:37:36.585799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:55.615355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891945+0000","last_change":"2024-09-13T08:37:34.883272+0000","last_active":"2024-09-13T08:37:35.891945+0000","last_peered":"2024-09-13T08:37:35.891945+0000","last_clean":"2024-09-13T08:37:35.891945+0000","last_became_active":"2024-09-13T08:37:34.882941+0000","last_became_peered":"2024-09-13T08:37:34.882941+0000","last_unstale":"2024-09-13T08:37:35.891945+0000","last_undegraded":"2024-09-13T08:37:35.891945+0000","last_fullsized":"2024-09-13T08:37:35.891945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895545+0000","last_change":"2024-09-13T08:37:34.903123+0000","last_active":"2024-09-13T08:37:35.895545+0000","last_peered":"2024-09-13T08:37:35.895545+0000","last_clean":"2024-09-13T08:37:35.895545+0000","last_became_active":"2024-09-13T08:37:34.901167+0000","last_became_peered":"2024-09-13T08:37:34.901167+0000","last_unstale":"2024-09-13T08:37:35.895545+0000","last_undegraded":"2024-09-13T08:37:35.895545+0000","last_fullsized":"2024-09-13T08:37:35.895545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084614+0000","last_change":"2024-09-13T08:37:34.891584+0000","last_active":"2024-09-13T08:37:37.084614+0000","last_peered":"2024-09-13T08:37:37.084614+0000","last_clean":"2024-09-13T08:37:37.084614+0000","last_became_active":"2024-09-13T08:37:34.891268+0000","last_became_peered":"2024-09-13T08:37:34.891268+0000","last_unstale":"2024-09-13T08:37:37.084614+0000","last_undegraded":"2024-09-13T08:37:37.084614+0000","last_fullsized":"2024-09-13T08:37:37.084614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:59.406808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896325+0000","last_change":"2024-09-13T08:37:34.902079+0000","last_active":"2024-09-13T08:37:35.896325+0000","last_peered":"2024-09-13T08:37:35.896325+0000","last_clean":"2024-09-13T08:37:35.896325+0000","last_became_active":"2024-09-13T08:37:34.900499+0000","last_became_peered":"2024-09-13T08:37:34.900499+0000","last_unstale":"2024-09-13T08:37:35.896325+0000","last_undegraded":"2024-09-13T08:37:35.896325+0000","last_fullsized":"2024-09-13T08:37:35.896325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:45.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511137+0000","last_change":"2024-09-13T08:37:34.888787+0000","last_active":"2024-09-13T08:37:36.511137+0000","last_peered":"2024-09-13T08:37:36.511137+0000","last_clean":"2024-09-13T08:37:36.511137+0000","last_became_active":"2024-09-13T08:37:34.888486+0000","last_became_peered":"2024-09-13T08:37:34.888486+0000","last_unstale":"2024-09-13T08:37:36.511137+0000","last_undegraded":"2024-09-13T08:37:36.511137+0000","last_fullsized":"2024-09-13T08:37:36.511137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:56.908418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895541+0000","last_change":"2024-09-13T08:37:34.902546+0000","last_active":"2024-09-13T08:37:35.895541+0000","last_peered":"2024-09-13T08:37:35.895541+0000","last_clean":"2024-09-13T08:37:35.895541+0000","last_became_active":"2024-09-13T08:37:34.901841+0000","last_became_peered":"2024-09-13T08:37:34.901841+0000","last_unstale":"2024-09-13T08:37:35.895541+0000","last_undegraded":"2024-09-13T08:37:35.895541+0000","last_fullsized":"2024-09-13T08:37:35.895541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:40.635073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895078+0000","last_change":"2024-09-13T08:37:34.907277+0000","last_active":"2024-09-13T08:37:35.895078+0000","last_peered":"2024-09-13T08:37:35.895078+0000","last_clean":"2024-09-13T08:37:35.895078+0000","last_became_active":"2024-09-13T08:37:34.906984+0000","last_became_peered":"2024-09-13T08:37:34.906984+0000","last_unstale":"2024-09-13T08:37:35.895078+0000","last_undegraded":"2024-09-13T08:37:35.895078+0000","last_fullsized":"2024-09-13T08:37:35.895078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:23.664559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895840+0000","last_change":"2024-09-13T08:37:34.902627+0000","last_active":"2024-09-13T08:37:35.895840+0000","last_peered":"2024-09-13T08:37:35.895840+0000","last_clean":"2024-09-13T08:37:35.895840+0000","last_became_active":"2024-09-13T08:37:34.901894+0000","last_became_peered":"2024-09-13T08:37:34.901894+0000","last_unstale":"2024-09-13T08:37:35.895840+0000","last_undegraded":"2024-09-13T08:37:35.895840+0000","last_fullsized":"2024-09-13T08:37:35.895840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:06.723541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084874+0000","last_change":"2024-09-13T08:37:34.891317+0000","last_active":"2024-09-13T08:37:37.084874+0000","last_peered":"2024-09-13T08:37:37.084874+0000","last_clean":"2024-09-13T08:37:37.084874+0000","last_became_active":"2024-09-13T08:37:34.891030+0000","last_became_peered":"2024-09-13T08:37:34.891030+0000","last_unstale":"2024-09-13T08:37:37.084874+0000","last_undegraded":"2024-09-13T08:37:37.084874+0000","last_fullsized":"2024-09-13T08:37:37.084874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:46.738660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529933+0000","last_change":"2024-09-13T08:37:34.894275+0000","last_active":"2024-09-13T08:37:36.529933+0000","last_peered":"2024-09-13T08:37:36.529933+0000","last_clean":"2024-09-13T08:37:36.529933+0000","last_became_active":"2024-09-13T08:37:34.893030+0000","last_became_peered":"2024-09-13T08:37:34.893030+0000","last_unstale":"2024-09-13T08:37:36.529933+0000","last_undegraded":"2024-09-13T08:37:36.529933+0000","last_fullsized":"2024-09-13T08:37:36.529933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510646+0000","last_change":"2024-09-13T08:37:34.900318+0000","last_active":"2024-09-13T08:37:36.510646+0000","last_peered":"2024-09-13T08:37:36.510646+0000","last_clean":"2024-09-13T08:37:36.510646+0000","last_became_active":"2024-09-13T08:37:34.899949+0000","last_became_peered":"2024-09-13T08:37:34.899949+0000","last_unstale":"2024-09-13T08:37:36.510646+0000","last_undegraded":"2024-09-13T08:37:36.510646+0000","last_fullsized":"2024-09-13T08:37:36.510646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892117+0000","last_change":"2024-09-13T08:37:34.889876+0000","last_active":"2024-09-13T08:37:35.892117+0000","last_peered":"2024-09-13T08:37:35.892117+0000","last_clean":"2024-09-13T08:37:35.892117+0000","last_became_active":"2024-09-13T08:37:34.889622+0000","last_became_peered":"2024-09-13T08:37:34.889622+0000","last_unstale":"2024-09-13T08:37:35.892117+0000","last_undegraded":"2024-09-13T08:37:35.892117+0000","last_fullsized":"2024-09-13T08:37:35.892117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895867+0000","last_change":"2024-09-13T08:37:33.888852+0000","last_active":"2024-09-13T08:37:35.895867+0000","last_peered":"2024-09-13T08:37:35.895867+0000","last_clean":"2024-09-13T08:37:35.895867+0000","last_became_active":"2024-09-13T08:37:33.888409+0000","last_became_peered":"2024-09-13T08:37:33.888409+0000","last_unstale":"2024-09-13T08:37:35.895867+0000","last_undegraded":"2024-09-13T08:37:35.895867+0000","last_fullsized":"2024-09-13T08:37:35.895867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585747+0000","last_change":"2024-09-13T08:37:34.890809+0000","last_active":"2024-09-13T08:37:36.585747+0000","last_peered":"2024-09-13T08:37:36.585747+0000","last_clean":"2024-09-13T08:37:36.585747+0000","last_became_active":"2024-09-13T08:37:34.887213+0000","last_became_peered":"2024-09-13T08:37:34.887213+0000","last_unstale":"2024-09-13T08:37:36.585747+0000","last_undegraded":"2024-09-13T08:37:36.585747+0000","last_fullsized":"2024-09-13T08:37:36.585747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:19:06.310722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085055+0000","last_change":"2024-09-13T08:37:33.892308+0000","last_active":"2024-09-13T08:37:37.085055+0000","last_peered":"2024-09-13T08:37:37.085055+0000","last_clean":"2024-09-13T08:37:37.085055+0000","last_became_active":"2024-09-13T08:37:33.891875+0000","last_became_peered":"2024-09-13T08:37:33.891875+0000","last_unstale":"2024-09-13T08:37:37.085055+0000","last_undegraded":"2024-09-13T08:37:37.085055+0000","last_fullsized":"2024-09-13T08:37:37.085055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:05.039750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894251+0000","last_change":"2024-09-13T08:37:33.896561+0000","last_active":"2024-09-13T08:37:35.894251+0000","last_peered":"2024-09-13T08:37:35.894251+0000","last_clean":"2024-09-13T08:37:35.894251+0000","last_became_active":"2024-09-13T08:37:33.896314+0000","last_became_peered":"2024-09-13T08:37:33.896314+0000","last_unstale":"2024-09-13T08:37:35.894251+0000","last_undegraded":"2024-09-13T08:37:35.894251+0000","last_fullsized":"2024-09-13T08:37:35.894251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:28.728254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895122+0000","last_change":"2024-09-13T08:37:34.900565+0000","last_active":"2024-09-13T08:37:35.895122+0000","last_peered":"2024-09-13T08:37:35.895122+0000","last_clean":"2024-09-13T08:37:35.895122+0000","last_became_active":"2024-09-13T08:37:34.900257+0000","last_became_peered":"2024-09-13T08:37:34.900257+0000","last_unstale":"2024-09-13T08:37:35.895122+0000","last_undegraded":"2024-09-13T08:37:35.895122+0000","last_fullsized":"2024-09-13T08:37:35.895122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:34.291551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084825+0000","last_change":"2024-09-13T08:37:33.883156+0000","last_active":"2024-09-13T08:37:37.084825+0000","last_peered":"2024-09-13T08:37:37.084825+0000","last_clean":"2024-09-13T08:37:37.084825+0000","last_became_active":"2024-09-13T08:37:33.882800+0000","last_became_peered":"2024-09-13T08:37:33.882800+0000","last_unstale":"2024-09-13T08:37:37.084825+0000","last_undegraded":"2024-09-13T08:37:37.084825+0000","last_fullsized":"2024-09-13T08:37:37.084825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:10.413313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585694+0000","last_change":"2024-09-13T08:37:34.888188+0000","last_active":"2024-09-13T08:37:36.585694+0000","last_peered":"2024-09-13T08:37:36.585694+0000","last_clean":"2024-09-13T08:37:36.585694+0000","last_became_active":"2024-09-13T08:37:34.887475+0000","last_became_peered":"2024-09-13T08:37:34.887475+0000","last_unstale":"2024-09-13T08:37:36.585694+0000","last_undegraded":"2024-09-13T08:37:36.585694+0000","last_fullsized":"2024-09-13T08:37:36.585694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:14.743221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892396+0000","last_change":"2024-09-13T08:37:33.891385+0000","last_active":"2024-09-13T08:37:35.892396+0000","last_peered":"2024-09-13T08:37:35.892396+0000","last_clean":"2024-09-13T08:37:35.892396+0000","last_became_active":"2024-09-13T08:37:33.891112+0000","last_became_peered":"2024-09-13T08:37:33.891112+0000","last_unstale":"2024-09-13T08:37:35.892396+0000","last_undegraded":"2024-09-13T08:37:35.892396+0000","last_fullsized":"2024-09-13T08:37:35.892396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511114+0000","last_change":"2024-09-13T08:37:34.899512+0000","last_active":"2024-09-13T08:37:36.511114+0000","last_peered":"2024-09-13T08:37:36.511114+0000","last_clean":"2024-09-13T08:37:36.511114+0000","last_became_active":"2024-09-13T08:37:34.887883+0000","last_became_peered":"2024-09-13T08:37:34.887883+0000","last_unstale":"2024-09-13T08:37:36.511114+0000","last_undegraded":"2024-09-13T08:37:36.511114+0000","last_fullsized":"2024-09-13T08:37:36.511114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:56.149863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877483+0000","last_change":"2024-09-13T08:37:33.899147+0000","last_active":"2024-09-13T08:37:36.877483+0000","last_peered":"2024-09-13T08:37:36.877483+0000","last_clean":"2024-09-13T08:37:36.877483+0000","last_became_active":"2024-09-13T08:37:33.896929+0000","last_became_peered":"2024-09-13T08:37:33.896929+0000","last_unstale":"2024-09-13T08:37:36.877483+0000","last_undegraded":"2024-09-13T08:37:36.877483+0000","last_fullsized":"2024-09-13T08:37:36.877483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:08:15.453880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896504+0000","last_change":"2024-09-13T08:37:34.900790+0000","last_active":"2024-09-13T08:37:35.896504+0000","last_peered":"2024-09-13T08:37:35.896504+0000","last_clean":"2024-09-13T08:37:35.896504+0000","last_became_active":"2024-09-13T08:37:34.898939+0000","last_became_peered":"2024-09-13T08:37:34.898939+0000","last_unstale":"2024-09-13T08:37:35.896504+0000","last_undegraded":"2024-09-13T08:37:35.896504+0000","last_fullsized":"2024-09-13T08:37:35.896504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:07.812549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.530096+0000","last_change":"2024-09-13T08:37:33.892872+0000","last_active":"2024-09-13T08:37:36.530096+0000","last_peered":"2024-09-13T08:37:36.530096+0000","last_clean":"2024-09-13T08:37:36.530096+0000","last_became_active":"2024-09-13T08:37:33.892579+0000","last_became_peered":"2024-09-13T08:37:33.892579+0000","last_unstale":"2024-09-13T08:37:36.530096+0000","last_undegraded":"2024-09-13T08:37:36.530096+0000","last_fullsized":"2024-09-13T08:37:36.530096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085280+0000","last_change":"2024-09-13T08:37:34.887215+0000","last_active":"2024-09-13T08:37:37.085280+0000","last_peered":"2024-09-13T08:37:37.085280+0000","last_clean":"2024-09-13T08:37:37.085280+0000","last_became_active":"2024-09-13T08:37:34.886889+0000","last_became_peered":"2024-09-13T08:37:34.886889+0000","last_unstale":"2024-09-13T08:37:37.085280+0000","last_undegraded":"2024-09-13T08:37:37.085280+0000","last_fullsized":"2024-09-13T08:37:37.085280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:57.113272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510921+0000","last_change":"2024-09-13T08:37:33.891105+0000","last_active":"2024-09-13T08:37:36.510921+0000","last_peered":"2024-09-13T08:37:36.510921+0000","last_clean":"2024-09-13T08:37:36.510921+0000","last_became_active":"2024-09-13T08:37:33.890783+0000","last_became_peered":"2024-09-13T08:37:33.890783+0000","last_unstale":"2024-09-13T08:37:36.510921+0000","last_undegraded":"2024-09-13T08:37:36.510921+0000","last_fullsized":"2024-09-13T08:37:36.510921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:30:45.112157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877636+0000","last_change":"2024-09-13T08:37:34.893268+0000","last_active":"2024-09-13T08:37:36.877636+0000","last_peered":"2024-09-13T08:37:36.877636+0000","last_clean":"2024-09-13T08:37:36.877636+0000","last_became_active":"2024-09-13T08:37:34.892944+0000","last_became_peered":"2024-09-13T08:37:34.892944+0000","last_unstale":"2024-09-13T08:37:36.877636+0000","last_undegraded":"2024-09-13T08:37:36.877636+0000","last_fullsized":"2024-09-13T08:37:36.877636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:45.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529953+0000","last_change":"2024-09-13T08:37:33.902610+0000","last_active":"2024-09-13T08:37:36.529953+0000","last_peered":"2024-09-13T08:37:36.529953+0000","last_clean":"2024-09-13T08:37:36.529953+0000","last_became_active":"2024-09-13T08:37:33.902275+0000","last_became_peered":"2024-09-13T08:37:33.902275+0000","last_unstale":"2024-09-13T08:37:36.529953+0000","last_undegraded":"2024-09-13T08:37:36.529953+0000","last_fullsized":"2024-09-13T08:37:36.529953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894679+0000","last_change":"2024-09-13T08:37:34.904983+0000","last_active":"2024-09-13T08:37:35.894679+0000","last_peered":"2024-09-13T08:37:35.894679+0000","last_clean":"2024-09-13T08:37:35.894679+0000","last_became_active":"2024-09-13T08:37:34.904618+0000","last_became_peered":"2024-09-13T08:37:34.904618+0000","last_unstale":"2024-09-13T08:37:35.894679+0000","last_undegraded":"2024-09-13T08:37:35.894679+0000","last_fullsized":"2024-09-13T08:37:35.894679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:30.070140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085096+0000","last_change":"2024-09-13T08:37:33.888874+0000","last_active":"2024-09-13T08:37:37.085096+0000","last_peered":"2024-09-13T08:37:37.085096+0000","last_clean":"2024-09-13T08:37:37.085096+0000","last_became_active":"2024-09-13T08:37:33.888111+0000","last_became_peered":"2024-09-13T08:37:33.888111+0000","last_unstale":"2024-09-13T08:37:37.085096+0000","last_undegraded":"2024-09-13T08:37:37.085096+0000","last_fullsized":"2024-09-13T08:37:37.085096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:55.141642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084977+0000","last_change":"2024-09-13T08:37:34.893432+0000","last_active":"2024-09-13T08:37:37.084977+0000","last_peered":"2024-09-13T08:37:37.084977+0000","last_clean":"2024-09-13T08:37:37.084977+0000","last_became_active":"2024-09-13T08:37:34.893160+0000","last_became_peered":"2024-09-13T08:37:34.893160+0000","last_unstale":"2024-09-13T08:37:37.084977+0000","last_undegraded":"2024-09-13T08:37:37.084977+0000","last_fullsized":"2024-09-13T08:37:37.084977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:46.900298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.876999+0000","last_change":"2024-09-13T08:37:33.892008+0000","last_active":"2024-09-13T08:37:36.876999+0000","last_peered":"2024-09-13T08:37:36.876999+0000","last_clean":"2024-09-13T08:37:36.876999+0000","last_became_active":"2024-09-13T08:37:33.891758+0000","last_became_peered":"2024-09-13T08:37:33.891758+0000","last_unstale":"2024-09-13T08:37:36.876999+0000","last_undegraded":"2024-09-13T08:37:36.876999+0000","last_fullsized":"2024-09-13T08:37:36.876999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:27.615101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891708+0000","last_change":"2024-09-13T08:37:34.889786+0000","last_active":"2024-09-13T08:37:35.891708+0000","last_peered":"2024-09-13T08:37:35.891708+0000","last_clean":"2024-09-13T08:37:35.891708+0000","last_became_active":"2024-09-13T08:37:34.889505+0000","last_became_peered":"2024-09-13T08:37:34.889505+0000","last_unstale":"2024-09-13T08:37:35.891708+0000","last_undegraded":"2024-09-13T08:37:35.891708+0000","last_fullsized":"2024-09-13T08:37:35.891708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:50.000602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585415+0000","last_change":"2024-09-13T08:37:33.888556+0000","last_active":"2024-09-13T08:37:36.585415+0000","last_peered":"2024-09-13T08:37:36.585415+0000","last_clean":"2024-09-13T08:37:36.585415+0000","last_became_active":"2024-09-13T08:37:33.887997+0000","last_became_peered":"2024-09-13T08:37:33.887997+0000","last_unstale":"2024-09-13T08:37:36.585415+0000","last_undegraded":"2024-09-13T08:37:36.585415+0000","last_fullsized":"2024-09-13T08:37:36.585415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084670+0000","last_change":"2024-09-13T08:37:34.908412+0000","last_active":"2024-09-13T08:37:37.084670+0000","last_peered":"2024-09-13T08:37:37.084670+0000","last_clean":"2024-09-13T08:37:37.084670+0000","last_became_active":"2024-09-13T08:37:34.908135+0000","last_became_peered":"2024-09-13T08:37:34.908135+0000","last_unstale":"2024-09-13T08:37:37.084670+0000","last_undegraded":"2024-09-13T08:37:37.084670+0000","last_fullsized":"2024-09-13T08:37:37.084670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:15.303677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877915+0000","last_change":"2024-09-13T08:37:33.895134+0000","last_active":"2024-09-13T08:37:36.877915+0000","last_peered":"2024-09-13T08:37:36.877915+0000","last_clean":"2024-09-13T08:37:36.877915+0000","last_became_active":"2024-09-13T08:37:33.894893+0000","last_became_peered":"2024-09-13T08:37:33.894893+0000","last_unstale":"2024-09-13T08:37:36.877915+0000","last_undegraded":"2024-09-13T08:37:36.877915+0000","last_fullsized":"2024-09-13T08:37:36.877915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:03:36.459606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511239+0000","last_change":"2024-09-13T08:37:34.915800+0000","last_active":"2024-09-13T08:37:36.511239+0000","last_peered":"2024-09-13T08:37:36.511239+0000","last_clean":"2024-09-13T08:37:36.511239+0000","last_became_active":"2024-09-13T08:37:34.915478+0000","last_became_peered":"2024-09-13T08:37:34.915478+0000","last_unstale":"2024-09-13T08:37:36.511239+0000","last_undegraded":"2024-09-13T08:37:36.511239+0000","last_fullsized":"2024-09-13T08:37:36.511239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:55.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586002+0000","last_change":"2024-09-13T08:37:33.888156+0000","last_active":"2024-09-13T08:37:36.586002+0000","last_peered":"2024-09-13T08:37:36.586002+0000","last_clean":"2024-09-13T08:37:36.586002+0000","last_became_active":"2024-09-13T08:37:33.886607+0000","last_became_peered":"2024-09-13T08:37:33.886607+0000","last_unstale":"2024-09-13T08:37:36.586002+0000","last_undegraded":"2024-09-13T08:37:36.586002+0000","last_fullsized":"2024-09-13T08:37:36.586002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:36.540294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877863+0000","last_change":"2024-09-13T08:37:34.891196+0000","last_active":"2024-09-13T08:37:36.877863+0000","last_peered":"2024-09-13T08:37:36.877863+0000","last_clean":"2024-09-13T08:37:36.877863+0000","last_became_active":"2024-09-13T08:37:34.890848+0000","last_became_peered":"2024-09-13T08:37:34.890848+0000","last_unstale":"2024-09-13T08:37:36.877863+0000","last_undegraded":"2024-09-13T08:37:36.877863+0000","last_fullsized":"2024-09-13T08:37:36.877863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:07.683461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896137+0000","last_change":"2024-09-13T08:37:33.885405+0000","last_active":"2024-09-13T08:37:35.896137+0000","last_peered":"2024-09-13T08:37:35.896137+0000","last_clean":"2024-09-13T08:37:35.896137+0000","last_became_active":"2024-09-13T08:37:33.885048+0000","last_became_peered":"2024-09-13T08:37:33.885048+0000","last_unstale":"2024-09-13T08:37:35.896137+0000","last_undegraded":"2024-09-13T08:37:35.896137+0000","last_fullsized":"2024-09-13T08:37:35.896137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:33:47.115618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877736+0000","last_change":"2024-09-13T08:37:34.898475+0000","last_active":"2024-09-13T08:37:36.877736+0000","last_peered":"2024-09-13T08:37:36.877736+0000","last_clean":"2024-09-13T08:37:36.877736+0000","last_became_active":"2024-09-13T08:37:34.897854+0000","last_became_peered":"2024-09-13T08:37:34.897854+0000","last_unstale":"2024-09-13T08:37:36.877736+0000","last_undegraded":"2024-09-13T08:37:36.877736+0000","last_fullsized":"2024-09-13T08:37:36.877736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:15.303677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895446+0000","last_change":"2024-09-13T08:37:33.902815+0000","last_active":"2024-09-13T08:37:35.895446+0000","last_peered":"2024-09-13T08:37:35.895446+0000","last_clean":"2024-09-13T08:37:35.895446+0000","last_became_active":"2024-09-13T08:37:33.902490+0000","last_became_peered":"2024-09-13T08:37:33.902490+0000","last_unstale":"2024-09-13T08:37:35.895446+0000","last_undegraded":"2024-09-13T08:37:35.895446+0000","last_fullsized":"2024-09-13T08:37:35.895446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:32.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085416+0000","last_change":"2024-09-13T08:37:34.891378+0000","last_active":"2024-09-13T08:37:37.085416+0000","last_peered":"2024-09-13T08:37:37.085416+0000","last_clean":"2024-09-13T08:37:37.085416+0000","last_became_active":"2024-09-13T08:37:34.885687+0000","last_became_peered":"2024-09-13T08:37:34.885687+0000","last_unstale":"2024-09-13T08:37:37.085416+0000","last_undegraded":"2024-09-13T08:37:37.085416+0000","last_fullsized":"2024-09-13T08:37:37.085416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:42.639292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891693+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:37:35.891693+0000","last_peered":"2024-09-13T08:37:35.891693+0000","last_clean":"2024-09-13T08:37:35.891693+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:37:35.891693+0000","last_undegraded":"2024-09-13T08:37:35.891693+0000","last_fullsized":"2024-09-13T08:37:35.891693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085218+0000","last_change":"2024-09-13T08:37:33.883826+0000","last_active":"2024-09-13T08:37:37.085218+0000","last_peered":"2024-09-13T08:37:37.085218+0000","last_clean":"2024-09-13T08:37:37.085218+0000","last_became_active":"2024-09-13T08:37:33.883511+0000","last_became_peered":"2024-09-13T08:37:33.883511+0000","last_unstale":"2024-09-13T08:37:37.085218+0000","last_undegraded":"2024-09-13T08:37:37.085218+0000","last_fullsized":"2024-09-13T08:37:37.085218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:39:48.743211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585719+0000","last_change":"2024-09-13T08:37:34.901372+0000","last_active":"2024-09-13T08:37:36.585719+0000","last_peered":"2024-09-13T08:37:36.585719+0000","last_clean":"2024-09-13T08:37:36.585719+0000","last_became_active":"2024-09-13T08:37:34.900931+0000","last_became_peered":"2024-09-13T08:37:34.900931+0000","last_unstale":"2024-09-13T08:37:36.585719+0000","last_undegraded":"2024-09-13T08:37:36.585719+0000","last_fullsized":"2024-09-13T08:37:36.585719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:16.328343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084832+0000","last_change":"2024-09-13T08:37:33.885219+0000","last_active":"2024-09-13T08:37:37.084832+0000","last_peered":"2024-09-13T08:37:37.084832+0000","last_clean":"2024-09-13T08:37:37.084832+0000","last_became_active":"2024-09-13T08:37:33.884889+0000","last_became_peered":"2024-09-13T08:37:33.884889+0000","last_unstale":"2024-09-13T08:37:37.084832+0000","last_undegraded":"2024-09-13T08:37:37.084832+0000","last_fullsized":"2024-09-13T08:37:37.084832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:55.495409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894451+0000","last_change":"2024-09-13T08:37:34.908186+0000","last_active":"2024-09-13T08:37:35.894451+0000","last_peered":"2024-09-13T08:37:35.894451+0000","last_clean":"2024-09-13T08:37:35.894451+0000","last_became_active":"2024-09-13T08:37:34.907886+0000","last_became_peered":"2024-09-13T08:37:34.907886+0000","last_unstale":"2024-09-13T08:37:35.894451+0000","last_undegraded":"2024-09-13T08:37:35.894451+0000","last_fullsized":"2024-09-13T08:37:35.894451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:11:14.712673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529782+0000","last_change":"2024-09-13T08:37:33.890556+0000","last_active":"2024-09-13T08:37:36.529782+0000","last_peered":"2024-09-13T08:37:36.529782+0000","last_clean":"2024-09-13T08:37:36.529782+0000","last_became_active":"2024-09-13T08:37:33.890252+0000","last_became_peered":"2024-09-13T08:37:33.890252+0000","last_unstale":"2024-09-13T08:37:36.529782+0000","last_undegraded":"2024-09-13T08:37:36.529782+0000","last_fullsized":"2024-09-13T08:37:36.529782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894861+0000","last_change":"2024-09-13T08:37:34.901633+0000","last_active":"2024-09-13T08:37:35.894861+0000","last_peered":"2024-09-13T08:37:35.894861+0000","last_clean":"2024-09-13T08:37:35.894861+0000","last_became_active":"2024-09-13T08:37:34.901323+0000","last_became_peered":"2024-09-13T08:37:34.901323+0000","last_unstale":"2024-09-13T08:37:35.894861+0000","last_undegraded":"2024-09-13T08:37:35.894861+0000","last_fullsized":"2024-09-13T08:37:35.894861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:37.744328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085159+0000","last_change":"2024-09-13T08:37:33.889192+0000","last_active":"2024-09-13T08:37:37.085159+0000","last_peered":"2024-09-13T08:37:37.085159+0000","last_clean":"2024-09-13T08:37:37.085159+0000","last_became_active":"2024-09-13T08:37:33.888556+0000","last_became_peered":"2024-09-13T08:37:33.888556+0000","last_unstale":"2024-09-13T08:37:37.085159+0000","last_undegraded":"2024-09-13T08:37:37.085159+0000","last_fullsized":"2024-09-13T08:37:37.085159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:55.900198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877366+0000","last_change":"2024-09-13T08:37:34.905109+0000","last_active":"2024-09-13T08:37:36.877366+0000","last_peered":"2024-09-13T08:37:36.877366+0000","last_clean":"2024-09-13T08:37:36.877366+0000","last_became_active":"2024-09-13T08:37:34.904766+0000","last_became_peered":"2024-09-13T08:37:34.904766+0000","last_unstale":"2024-09-13T08:37:36.877366+0000","last_undegraded":"2024-09-13T08:37:36.877366+0000","last_fullsized":"2024-09-13T08:37:36.877366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:54.979663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895896+0000","last_change":"2024-09-13T08:37:33.889699+0000","last_active":"2024-09-13T08:37:35.895896+0000","last_peered":"2024-09-13T08:37:35.895896+0000","last_clean":"2024-09-13T08:37:35.895896+0000","last_became_active":"2024-09-13T08:37:33.889224+0000","last_became_peered":"2024-09-13T08:37:33.889224+0000","last_unstale":"2024-09-13T08:37:35.895896+0000","last_undegraded":"2024-09-13T08:37:35.895896+0000","last_fullsized":"2024-09-13T08:37:35.895896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:24.413843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510786+0000","last_change":"2024-09-13T08:37:34.901224+0000","last_active":"2024-09-13T08:37:36.510786+0000","last_peered":"2024-09-13T08:37:36.510786+0000","last_clean":"2024-09-13T08:37:36.510786+0000","last_became_active":"2024-09-13T08:37:34.900890+0000","last_became_peered":"2024-09-13T08:37:34.900890+0000","last_unstale":"2024-09-13T08:37:36.510786+0000","last_undegraded":"2024-09-13T08:37:36.510786+0000","last_fullsized":"2024-09-13T08:37:36.510786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:06.047971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085567+0000","last_change":"2024-09-13T08:37:33.893542+0000","last_active":"2024-09-13T08:37:37.085567+0000","last_peered":"2024-09-13T08:37:37.085567+0000","last_clean":"2024-09-13T08:37:37.085567+0000","last_became_active":"2024-09-13T08:37:33.893231+0000","last_became_peered":"2024-09-13T08:37:33.893231+0000","last_unstale":"2024-09-13T08:37:37.085567+0000","last_undegraded":"2024-09-13T08:37:37.085567+0000","last_fullsized":"2024-09-13T08:37:37.085567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:48.992381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895292+0000","last_change":"2024-09-13T08:37:34.901999+0000","last_active":"2024-09-13T08:37:35.895292+0000","last_peered":"2024-09-13T08:37:35.895292+0000","last_clean":"2024-09-13T08:37:35.895292+0000","last_became_active":"2024-09-13T08:37:34.901765+0000","last_became_peered":"2024-09-13T08:37:34.901765+0000","last_unstale":"2024-09-13T08:37:35.895292+0000","last_undegraded":"2024-09-13T08:37:35.895292+0000","last_fullsized":"2024-09-13T08:37:35.895292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:12.422693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586075+0000","last_change":"2024-09-13T08:37:33.898217+0000","last_active":"2024-09-13T08:37:36.586075+0000","last_peered":"2024-09-13T08:37:36.586075+0000","last_clean":"2024-09-13T08:37:36.586075+0000","last_became_active":"2024-09-13T08:37:33.897954+0000","last_became_peered":"2024-09-13T08:37:33.897954+0000","last_unstale":"2024-09-13T08:37:36.586075+0000","last_undegraded":"2024-09-13T08:37:36.586075+0000","last_fullsized":"2024-09-13T08:37:36.586075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:05.995257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586135+0000","last_change":"2024-09-13T08:37:34.897630+0000","last_active":"2024-09-13T08:37:36.586135+0000","last_peered":"2024-09-13T08:37:36.586135+0000","last_clean":"2024-09-13T08:37:36.586135+0000","last_became_active":"2024-09-13T08:37:34.896603+0000","last_became_peered":"2024-09-13T08:37:34.896603+0000","last_unstale":"2024-09-13T08:37:36.586135+0000","last_undegraded":"2024-09-13T08:37:36.586135+0000","last_fullsized":"2024-09-13T08:37:36.586135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085624+0000","last_change":"2024-09-13T08:37:33.897139+0000","last_active":"2024-09-13T08:37:37.085624+0000","last_peered":"2024-09-13T08:37:37.085624+0000","last_clean":"2024-09-13T08:37:37.085624+0000","last_became_active":"2024-09-13T08:37:33.896576+0000","last_became_peered":"2024-09-13T08:37:33.896576+0000","last_unstale":"2024-09-13T08:37:37.085624+0000","last_undegraded":"2024-09-13T08:37:37.085624+0000","last_fullsized":"2024-09-13T08:37:37.085624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:54.815810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585844+0000","last_change":"2024-09-13T08:37:34.896701+0000","last_active":"2024-09-13T08:37:36.585844+0000","last_peered":"2024-09-13T08:37:36.585844+0000","last_clean":"2024-09-13T08:37:36.585844+0000","last_became_active":"2024-09-13T08:37:34.893772+0000","last_became_peered":"2024-09-13T08:37:34.893772+0000","last_unstale":"2024-09-13T08:37:36.585844+0000","last_undegraded":"2024-09-13T08:37:36.585844+0000","last_fullsized":"2024-09-13T08:37:36.585844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:34.291551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.876764+0000","last_change":"2024-09-13T08:37:33.893088+0000","last_active":"2024-09-13T08:37:36.876764+0000","last_peered":"2024-09-13T08:37:36.876764+0000","last_clean":"2024-09-13T08:37:36.876764+0000","last_became_active":"2024-09-13T08:37:33.892731+0000","last_became_peered":"2024-09-13T08:37:33.892731+0000","last_unstale":"2024-09-13T08:37:36.876764+0000","last_undegraded":"2024-09-13T08:37:36.876764+0000","last_fullsized":"2024-09-13T08:37:36.876764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084725+0000","last_change":"2024-09-13T08:37:34.895132+0000","last_active":"2024-09-13T08:37:37.084725+0000","last_peered":"2024-09-13T08:37:37.084725+0000","last_clean":"2024-09-13T08:37:37.084725+0000","last_became_active":"2024-09-13T08:37:34.894785+0000","last_became_peered":"2024-09-13T08:37:34.894785+0000","last_unstale":"2024-09-13T08:37:37.084725+0000","last_undegraded":"2024-09-13T08:37:37.084725+0000","last_fullsized":"2024-09-13T08:37:37.084725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:07.683461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585602+0000","last_change":"2024-09-13T08:37:33.898275+0000","last_active":"2024-09-13T08:37:36.585602+0000","last_peered":"2024-09-13T08:37:36.585602+0000","last_clean":"2024-09-13T08:37:36.585602+0000","last_became_active":"2024-09-13T08:37:33.897960+0000","last_became_peered":"2024-09-13T08:37:33.897960+0000","last_unstale":"2024-09-13T08:37:36.585602+0000","last_undegraded":"2024-09-13T08:37:36.585602+0000","last_fullsized":"2024-09-13T08:37:36.585602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:22.150995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585465+0000","last_change":"2024-09-13T08:37:34.891285+0000","last_active":"2024-09-13T08:37:36.585465+0000","last_peered":"2024-09-13T08:37:36.585465+0000","last_clean":"2024-09-13T08:37:36.585465+0000","last_became_active":"2024-09-13T08:37:34.890996+0000","last_became_peered":"2024-09-13T08:37:34.890996+0000","last_unstale":"2024-09-13T08:37:36.585465+0000","last_undegraded":"2024-09-13T08:37:36.585465+0000","last_fullsized":"2024-09-13T08:37:36.585465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:47.564367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894655+0000","last_change":"2024-09-13T08:37:33.891510+0000","last_active":"2024-09-13T08:37:35.894655+0000","last_peered":"2024-09-13T08:37:35.894655+0000","last_clean":"2024-09-13T08:37:35.894655+0000","last_became_active":"2024-09-13T08:37:33.890774+0000","last_became_peered":"2024-09-13T08:37:33.890774+0000","last_unstale":"2024-09-13T08:37:35.894655+0000","last_undegraded":"2024-09-13T08:37:35.894655+0000","last_fullsized":"2024-09-13T08:37:35.894655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:25.370931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529890+0000","last_change":"2024-09-13T08:37:34.895928+0000","last_active":"2024-09-13T08:37:36.529890+0000","last_peered":"2024-09-13T08:37:36.529890+0000","last_clean":"2024-09-13T08:37:36.529890+0000","last_became_active":"2024-09-13T08:37:34.895612+0000","last_became_peered":"2024-09-13T08:37:34.895612+0000","last_unstale":"2024-09-13T08:37:36.529890+0000","last_undegraded":"2024-09-13T08:37:36.529890+0000","last_fullsized":"2024-09-13T08:37:36.529890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085214+0000","last_change":"2024-09-13T08:37:33.887243+0000","last_active":"2024-09-13T08:37:37.085214+0000","last_peered":"2024-09-13T08:37:37.085214+0000","last_clean":"2024-09-13T08:37:37.085214+0000","last_became_active":"2024-09-13T08:37:33.886882+0000","last_became_peered":"2024-09-13T08:37:33.886882+0000","last_unstale":"2024-09-13T08:37:37.085214+0000","last_undegraded":"2024-09-13T08:37:37.085214+0000","last_fullsized":"2024-09-13T08:37:37.085214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:31.044508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084923+0000","last_change":"2024-09-13T08:37:34.894740+0000","last_active":"2024-09-13T08:37:37.084923+0000","last_peered":"2024-09-13T08:37:37.084923+0000","last_clean":"2024-09-13T08:37:37.084923+0000","last_became_active":"2024-09-13T08:37:34.894284+0000","last_became_peered":"2024-09-13T08:37:34.894284+0000","last_unstale":"2024-09-13T08:37:37.084923+0000","last_undegraded":"2024-09-13T08:37:37.084923+0000","last_fullsized":"2024-09-13T08:37:37.084923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:28.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085351+0000","last_change":"2024-09-13T08:37:34.892338+0000","last_active":"2024-09-13T08:37:37.085351+0000","last_peered":"2024-09-13T08:37:37.085351+0000","last_clean":"2024-09-13T08:37:37.085351+0000","last_became_active":"2024-09-13T08:37:34.891790+0000","last_became_peered":"2024-09-13T08:37:34.891790+0000","last_unstale":"2024-09-13T08:37:37.085351+0000","last_undegraded":"2024-09-13T08:37:37.085351+0000","last_fullsized":"2024-09-13T08:37:37.085351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:24:06.938838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877706+0000","last_change":"2024-09-13T08:37:33.900929+0000","last_active":"2024-09-13T08:37:36.877706+0000","last_peered":"2024-09-13T08:37:36.877706+0000","last_clean":"2024-09-13T08:37:36.877706+0000","last_became_active":"2024-09-13T08:37:33.899776+0000","last_became_peered":"2024-09-13T08:37:33.899776+0000","last_unstale":"2024-09-13T08:37:36.877706+0000","last_undegraded":"2024-09-13T08:37:36.877706+0000","last_fullsized":"2024-09-13T08:37:36.877706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:27.711361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892184+0000","last_change":"2024-09-13T08:37:34.893833+0000","last_active":"2024-09-13T08:37:35.892184+0000","last_peered":"2024-09-13T08:37:35.892184+0000","last_clean":"2024-09-13T08:37:35.892184+0000","last_became_active":"2024-09-13T08:37:34.893512+0000","last_became_peered":"2024-09-13T08:37:34.893512+0000","last_unstale":"2024-09-13T08:37:35.892184+0000","last_undegraded":"2024-09-13T08:37:35.892184+0000","last_fullsized":"2024-09-13T08:37:35.892184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877847+0000","last_change":"2024-09-13T08:37:33.894985+0000","last_active":"2024-09-13T08:37:36.877847+0000","last_peered":"2024-09-13T08:37:36.877847+0000","last_clean":"2024-09-13T08:37:36.877847+0000","last_became_active":"2024-09-13T08:37:33.894087+0000","last_became_peered":"2024-09-13T08:37:33.894087+0000","last_unstale":"2024-09-13T08:37:36.877847+0000","last_undegraded":"2024-09-13T08:37:36.877847+0000","last_fullsized":"2024-09-13T08:37:36.877847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:51.688613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896328+0000","last_change":"2024-09-13T08:37:34.901345+0000","last_active":"2024-09-13T08:37:35.896328+0000","last_peered":"2024-09-13T08:37:35.896328+0000","last_clean":"2024-09-13T08:37:35.896328+0000","last_became_active":"2024-09-13T08:37:34.900290+0000","last_became_peered":"2024-09-13T08:37:34.900290+0000","last_unstale":"2024-09-13T08:37:35.896328+0000","last_undegraded":"2024-09-13T08:37:35.896328+0000","last_fullsized":"2024-09-13T08:37:35.896328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:08.493271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896451+0000","last_change":"2024-09-13T08:37:33.887848+0000","last_active":"2024-09-13T08:37:35.896451+0000","last_peered":"2024-09-13T08:37:35.896451+0000","last_clean":"2024-09-13T08:37:35.896451+0000","last_became_active":"2024-09-13T08:37:33.887508+0000","last_became_peered":"2024-09-13T08:37:33.887508+0000","last_unstale":"2024-09-13T08:37:35.896451+0000","last_undegraded":"2024-09-13T08:37:35.896451+0000","last_fullsized":"2024-09-13T08:37:35.896451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:27.615101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896328+0000","last_change":"2024-09-13T08:37:34.907107+0000","last_active":"2024-09-13T08:37:35.896328+0000","last_peered":"2024-09-13T08:37:35.896328+0000","last_clean":"2024-09-13T08:37:35.896328+0000","last_became_active":"2024-09-13T08:37:34.904160+0000","last_became_peered":"2024-09-13T08:37:34.904160+0000","last_unstale":"2024-09-13T08:37:35.896328+0000","last_undegraded":"2024-09-13T08:37:35.896328+0000","last_fullsized":"2024-09-13T08:37:35.896328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:55.775775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586114+0000","last_change":"2024-09-13T08:37:33.894082+0000","last_active":"2024-09-13T08:37:36.586114+0000","last_peered":"2024-09-13T08:37:36.586114+0000","last_clean":"2024-09-13T08:37:36.586114+0000","last_became_active":"2024-09-13T08:37:33.893820+0000","last_became_peered":"2024-09-13T08:37:33.893820+0000","last_unstale":"2024-09-13T08:37:36.586114+0000","last_undegraded":"2024-09-13T08:37:36.586114+0000","last_fullsized":"2024-09-13T08:37:36.586114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:48.992381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:37:44.047 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:37:44.048 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-13T08:37:44.048 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T08:37:44.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.231+0000 7f077e200640 1 -- 172.21.15.44:0/45140655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c0039f0 msgr2=0x7f075c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.231+0000 7f077e200640 1 --2- 172.21.15.44:0/45140655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c0039f0 0x7f075c003dd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0768009b30 tx=0x7f076801d210 comp rx=0 tx=0).stop 2024-09-13T08:37:44.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.231+0000 7f0777e00640 1 -- 172.21.15.44:0/45140655 reap_dead start 2024-09-13T08:37:44.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.231+0000 7f077e200640 1 -- 172.21.15.44:0/45140655 shutdown_connections 2024-09-13T08:37:44.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.231+0000 7f077e200640 1 -- 172.21.15.44:0/45140655 >> 172.21.15.44:0/45140655 conn(0x7f07780d1dc0 msgr2=0x7f07780d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:44.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.231+0000 7f077e200640 1 -- 172.21.15.44:0/45140655 shutdown_connections 2024-09-13T08:37:44.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.231+0000 7f077e200640 1 -- 172.21.15.44:0/45140655 wait complete. 2024-09-13T08:37:44.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f077e200640 1 Processor -- start 2024-09-13T08:37:44.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f077e200640 1 -- start start 2024-09-13T08:37:44.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f077e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c0039f0 0x7f077815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:44.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f077e200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07780c8e90 0x7f077815af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:44.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f077e200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07781f5e70 0x7f07781f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:44.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f077e200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f077815c400 con 0x7f075c0039f0 2024-09-13T08:37:44.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f077e200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f077815c280 con 0x7f07781f5e70 2024-09-13T08:37:44.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f077e200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f07780ce670 con 0x7f07780c8e90 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f0777400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c0039f0 0x7f077815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f0777400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c0039f0 0x7f077815a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36220/0 (socket says 172.21.15.44:36220) 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f0777400640 1 -- 172.21.15.44:0/1356646825 learned_addr learned my addr 172.21.15.44:0/1356646825 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f076fe00640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07780c8e90 0x7f077815af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f0777e00640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07781f5e70 0x7f07781f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f0777400640 1 -- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07780c8e90 msgr2=0x7f077815af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f0777400640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07780c8e90 0x7f077815af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f0777400640 1 -- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07781f5e70 msgr2=0x7f07781f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f0777400640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07781f5e70 0x7f07781f6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f0777400640 1 -- 172.21.15.44:0/1356646825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07780c93c0 con 0x7f075c0039f0 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f0777400640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c0039f0 0x7f077815a9e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f076801cc80 tx=0x7f0768005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f076f400640 1 -- 172.21.15.44:0/1356646825 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07680b1040 con 0x7f075c0039f0 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f076f400640 1 -- 172.21.15.44:0/1356646825 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07680bb070 con 0x7f075c0039f0 2024-09-13T08:37:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f077e200640 1 -- 172.21.15.44:0/1356646825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07781f2730 con 0x7f075c0039f0 2024-09-13T08:37:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f076f400640 1 -- 172.21.15.44:0/1356646825 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768004290 con 0x7f075c0039f0 2024-09-13T08:37:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.235+0000 7f077e200640 1 -- 172.21.15.44:0/1356646825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07781f2cf0 con 0x7f075c0039f0 2024-09-13T08:37:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.239+0000 7f077e200640 1 -- 172.21.15.44:0/1356646825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07780ceb90 con 0x7f075c0039f0 2024-09-13T08:37:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.239+0000 7f076f400640 1 -- 172.21.15.44:0/1356646825 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f07680046d0 con 0x7f075c0039f0 2024-09-13T08:37:44.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.239+0000 7f076f400640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0754072ec0 0x7f0754075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:44.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.239+0000 7f076f400640 1 -- 172.21.15.44:0/1356646825 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f076812ec70 con 0x7f075c0039f0 2024-09-13T08:37:44.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.239+0000 7f076fe00640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0754072ec0 0x7f0754075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:44.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.239+0000 7f076fe00640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0754072ec0 0x7f0754075380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0760001060 tx=0x7f07600067b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:44.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.243+0000 7f076f400640 1 -- 172.21.15.44:0/1356646825 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0768016f20 con 0x7f075c0039f0 2024-09-13T08:37:44.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.419+0000 7f077e200640 1 -- 172.21.15.44:0/1356646825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f07781e1c60 con 0x7f075c0039f0 2024-09-13T08:37:44.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.419+0000 7f076f400640 1 -- 172.21.15.44:0/1356646825 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7f07680f97d0 con 0x7f075c0039f0 2024-09-13T08:37:44.425 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:44.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 -- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0754072ec0 msgr2=0x7f0754075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0754072ec0 0x7f0754075380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0760001060 tx=0x7f07600067b0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 -- 172.21.15.44:0/1356646825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c0039f0 msgr2=0x7f077815a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c0039f0 0x7f077815a9e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f076801cc80 tx=0x7f0768005560 comp rx=0 tx=0).stop 2024-09-13T08:37:44.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 -- 172.21.15.44:0/1356646825 shutdown_connections 2024-09-13T08:37:44.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0754072ec0 0x7f0754075380 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07781f5e70 0x7f07781f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07780c8e90 0x7f077815af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 --2- 172.21.15.44:0/1356646825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c0039f0 0x7f077815a9e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 -- 172.21.15.44:0/1356646825 >> 172.21.15.44:0/1356646825 conn(0x7f07780d1dc0 msgr2=0x7f07780cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:44.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 -- 172.21.15.44:0/1356646825 shutdown_connections 2024-09-13T08:37:44.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.423+0000 7f077e200640 1 -- 172.21.15.44:0/1356646825 wait complete. 2024-09-13T08:37:44.446 INFO:teuthology.orchestra.run.smithi044.stdout:[{"version":27,"timestamp":"2024-09-13T08:37:23.263998+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"3978.263337"}]}] 2024-09-13T08:37:44.447 INFO:tasks.ceph_manager:config epoch is 27 2024-09-13T08:37:44.447 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T08:37:44.447 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T08:37:44.447 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T08:37:44.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.587+0000 7f0f8a800640 1 -- 172.21.15.44:0/414460056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f840737c0 msgr2=0x7f0f8415c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.587+0000 7f0f8a800640 1 --2- 172.21.15.44:0/414460056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f840737c0 0x7f0f8415c1c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009840 tx=0x7f0f7001c7b0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.587+0000 7f0f8a800640 1 -- 172.21.15.44:0/414460056 shutdown_connections 2024-09-13T08:37:44.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.587+0000 7f0f8a800640 1 --2- 172.21.15.44:0/414460056 >> v2:172.21.15.129:3300/0 conn(0x7f0f84071030 0x7f0f84071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.587+0000 7f0f8a800640 1 --2- 172.21.15.44:0/414460056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f840737c0 0x7f0f8415c1c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.587+0000 7f0f8a800640 1 -- 172.21.15.44:0/414460056 >> 172.21.15.44:0/414460056 conn(0x7f0f840d0c90 msgr2=0x7f0f840d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:44.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.587+0000 7f0f8a800640 1 -- 172.21.15.44:0/414460056 shutdown_connections 2024-09-13T08:37:44.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.587+0000 7f0f8a800640 1 -- 172.21.15.44:0/414460056 wait complete. 2024-09-13T08:37:44.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f8a800640 1 Processor -- start 2024-09-13T08:37:44.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f8a800640 1 -- start start 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f8a800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f8406cc20 0x7f0f840b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f8a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f84071030 0x7f0f8415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f8a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f840737c0 0x7f0f8415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f8a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84169020 con 0x7f0f84071030 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f8a800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84168ea0 con 0x7f0f840737c0 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f8a800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0f841691a0 con 0x7f0f8406cc20 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f82a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f84071030 0x7f0f8415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f82a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f84071030 0x7f0f8415abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36242/0 (socket says 172.21.15.44:36242) 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f82a00640 1 -- 172.21.15.44:0/3376653905 learned_addr learned my addr 172.21.15.44:0/3376653905 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f83400640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f8406cc20 0x7f0f840b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f83e00640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f840737c0 0x7f0f8415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f82a00640 1 -- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f8406cc20 msgr2=0x7f0f840b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f82a00640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f8406cc20 0x7f0f840b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f82a00640 1 -- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f840737c0 msgr2=0x7f0f8415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f82a00640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f840737c0 0x7f0f8415b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f82a00640 1 -- 172.21.15.44:0/3376653905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f84156a40 con 0x7f0f84071030 2024-09-13T08:37:44.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f83e00640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f840737c0 0x7f0f8415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:44.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f83400640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f8406cc20 0x7f0f840b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:44.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f82a00640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f84071030 0x7f0f8415abe0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009790 tx=0x7f0f700a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:44.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f7b400640 1 -- 172.21.15.44:0/3376653905 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f700bb070 con 0x7f0f84071030 2024-09-13T08:37:44.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f7b400640 1 -- 172.21.15.44:0/3376653905 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f70004470 con 0x7f0f84071030 2024-09-13T08:37:44.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f8a800640 1 -- 172.21.15.44:0/3376653905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f84156d20 con 0x7f0f84071030 2024-09-13T08:37:44.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.591+0000 7f0f7b400640 1 -- 172.21.15.44:0/3376653905 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f700b2040 con 0x7f0f84071030 2024-09-13T08:37:44.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.595+0000 7f0f8a800640 1 -- 172.21.15.44:0/3376653905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841572e0 con 0x7f0f84071030 2024-09-13T08:37:44.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.595+0000 7f0f8a800640 1 -- 172.21.15.44:0/3376653905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f84071460 con 0x7f0f84071030 2024-09-13T08:37:44.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.595+0000 7f0f7b400640 1 -- 172.21.15.44:0/3376653905 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f0f700be070 con 0x7f0f84071030 2024-09-13T08:37:44.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.595+0000 7f0f7b400640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f64072ec0 0x7f0f64075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:44.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.595+0000 7f0f7b400640 1 -- 172.21.15.44:0/3376653905 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0f7012d640 con 0x7f0f84071030 2024-09-13T08:37:44.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.595+0000 7f0f83400640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f64072ec0 0x7f0f64075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:44.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.599+0000 7f0f83400640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f64072ec0 0x7f0f64075380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0f74007fd0 tx=0x7f0f74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:44.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.599+0000 7f0f7b400640 1 -- 172.21.15.44:0/3376653905 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0f700163d0 con 0x7f0f84071030 2024-09-13T08:37:44.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.819+0000 7f0f8a800640 1 -- 172.21.15.44:0/3376653905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0f84160650 con 0x7f0f84071030 2024-09-13T08:37:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.823+0000 7f0f7b400640 1 -- 172.21.15.44:0/3376653905 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186090 (secure 0 0 0) 0x7f0f700f8800 con 0x7f0f84071030 2024-09-13T08:37:44.829 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:44.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 -- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f64072ec0 msgr2=0x7f0f64075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f64072ec0 0x7f0f64075380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0f74007fd0 tx=0x7f0f74009040 comp rx=0 tx=0).stop 2024-09-13T08:37:44.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 -- 172.21.15.44:0/3376653905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f84071030 msgr2=0x7f0f8415abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:44.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f84071030 0x7f0f8415abe0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009790 tx=0x7f0f700a5620 comp rx=0 tx=0).stop 2024-09-13T08:37:44.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 -- 172.21.15.44:0/3376653905 shutdown_connections 2024-09-13T08:37:44.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f64072ec0 0x7f0f64075380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f840737c0 0x7f0f8415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f84071030 0x7f0f8415abe0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 --2- 172.21.15.44:0/3376653905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f8406cc20 0x7f0f840b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:44.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 -- 172.21.15.44:0/3376653905 >> 172.21.15.44:0/3376653905 conn(0x7f0f840d0c90 msgr2=0x7f0f840cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:44.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 -- 172.21.15.44:0/3376653905 shutdown_connections 2024-09-13T08:37:44.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.827+0000 7f0f8a800640 1 -- 172.21.15.44:0/3376653905 wait complete. 2024-09-13T08:37:44.849 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3066969387},{"type":"v1","addr":"172.21.15.129:6801","nonce":3066969387}]},"active_addr":"172.21.15.129:6801/3066969387","active_change":"2024-09-13T08:37:14.494278+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":3062519430}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":1712626197}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":1157544519}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":3928153165}]}]} 2024-09-13T08:37:44.852 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T08:37:44.852 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T08:37:44.852 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:37:45.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.995+0000 7fbf2de00640 1 -- 172.21.15.44:0/3730979910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf0c005290 msgr2=0x7fbf0c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.995+0000 7fbf2de00640 1 --2- 172.21.15.44:0/3730979910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf0c005290 0x7fbf0c00b9d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbf20001f50 tx=0x7fbf2001f3b0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.995+0000 7fbf2de00640 1 -- 172.21.15.44:0/3730979910 shutdown_connections 2024-09-13T08:37:45.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.995+0000 7fbf2de00640 1 --2- 172.21.15.44:0/3730979910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf0c005290 0x7fbf0c00b9d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.995+0000 7fbf2de00640 1 -- 172.21.15.44:0/3730979910 >> 172.21.15.44:0/3730979910 conn(0x7fbf280d1dc0 msgr2=0x7fbf280d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.995+0000 7fbf2de00640 1 -- 172.21.15.44:0/3730979910 shutdown_connections 2024-09-13T08:37:45.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.995+0000 7fbf2de00640 1 -- 172.21.15.44:0/3730979910 wait complete. 2024-09-13T08:37:45.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.995+0000 7fbf2de00640 1 Processor -- start 2024-09-13T08:37:45.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.995+0000 7fbf2de00640 1 -- start start 2024-09-13T08:37:45.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf2de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf0c005290 0x7fbf2814e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf2de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf281f65c0 0x7fbf280b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf2de00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf281e3a40 0x7fbf280b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf2de00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf280ce670 con 0x7fbf0c005290 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf2de00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf280cfbf0 con 0x7fbf281f65c0 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf2de00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fbf280ce7f0 con 0x7fbf281e3a40 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf26a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf0c005290 0x7fbf2814e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf26a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf0c005290 0x7fbf2814e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36260/0 (socket says 172.21.15.44:36260) 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf26a00640 1 -- 172.21.15.44:0/3574843529 learned_addr learned my addr 172.21.15.44:0/3574843529 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf26a00640 1 -- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf281e3a40 msgr2=0x7fbf280b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf26a00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf281e3a40 0x7fbf280b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf26a00640 1 -- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf281f65c0 msgr2=0x7fbf280b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf1fe00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf281f65c0 0x7fbf280b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf26a00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf281f65c0 0x7fbf280b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf26a00640 1 -- 172.21.15.44:0/3574843529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf280b15d0 con 0x7fbf0c005290 2024-09-13T08:37:45.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf1fe00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf281f65c0 0x7fbf280b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:37:45.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf26a00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf0c005290 0x7fbf2814e1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf18011ab0 tx=0x7fbf18011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf1f400640 1 -- 172.21.15.44:0/3574843529 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1801e070 con 0x7fbf0c005290 2024-09-13T08:37:45.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf1f400640 1 -- 172.21.15.44:0/3574843529 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf1800fe70 con 0x7fbf0c005290 2024-09-13T08:37:45.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf1f400640 1 -- 172.21.15.44:0/3574843529 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf180042b0 con 0x7fbf0c005290 2024-09-13T08:37:45.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf2de00640 1 -- 172.21.15.44:0/3574843529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf280b18b0 con 0x7fbf0c005290 2024-09-13T08:37:45.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf2de00640 1 -- 172.21.15.44:0/3574843529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf280b1e70 con 0x7fbf0c005290 2024-09-13T08:37:45.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:44.999+0000 7fbf2de00640 1 -- 172.21.15.44:0/3574843529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf280ced50 con 0x7fbf0c005290 2024-09-13T08:37:45.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.003+0000 7fbf1f400640 1 -- 172.21.15.44:0/3574843529 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fbf18021070 con 0x7fbf0c005290 2024-09-13T08:37:45.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.003+0000 7fbf1f400640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf04072ec0 0x7fbf04075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.003+0000 7fbf1f400640 1 -- 172.21.15.44:0/3574843529 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbf18094850 con 0x7fbf0c005290 2024-09-13T08:37:45.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.003+0000 7fbf1fe00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf04072ec0 0x7fbf04075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.003+0000 7fbf1fe00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf04072ec0 0x7fbf04075380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbf281f69c0 tx=0x7fbf10000fb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.007+0000 7fbf1f400640 1 -- 172.21.15.44:0/3574843529 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fbf1805f450 con 0x7fbf0c005290 2024-09-13T08:37:45.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.187+0000 7fbf2de00640 1 -- 172.21.15.44:0/3574843529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf280af980 con 0x7fbf0c005290 2024-09-13T08:37:45.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf1f400640 1 -- 172.21.15.44:0/3574843529 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16681 (secure 0 0 0) 0x7fbf18063e80 con 0x7fbf0c005290 2024-09-13T08:37:45.195 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:45.195 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":22,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:35.883263+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:37:32.247896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T08:37:33.241202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:37:45.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 -- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf04072ec0 msgr2=0x7fbf04075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf04072ec0 0x7fbf04075380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbf281f69c0 tx=0x7fbf10000fb0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 -- 172.21.15.44:0/3574843529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf0c005290 msgr2=0x7fbf2814e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf0c005290 0x7fbf2814e1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf18011ab0 tx=0x7fbf18011f70 comp rx=0 tx=0).stop 2024-09-13T08:37:45.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 -- 172.21.15.44:0/3574843529 shutdown_connections 2024-09-13T08:37:45.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf04072ec0 0x7fbf04075380 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf281e3a40 0x7fbf280b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf281f65c0 0x7fbf280b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 --2- 172.21.15.44:0/3574843529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf0c005290 0x7fbf2814e1a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 -- 172.21.15.44:0/3574843529 >> 172.21.15.44:0/3574843529 conn(0x7fbf280d1dc0 msgr2=0x7fbf280b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 -- 172.21.15.44:0/3574843529 shutdown_connections 2024-09-13T08:37:45.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.191+0000 7fbf2de00640 1 -- 172.21.15.44:0/3574843529 wait complete. 2024-09-13T08:37:45.215 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T08:37:45.215 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:37:45.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 -- 172.21.15.44:0/1145884073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10e0003c20 msgr2=0x7f10e0004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 --2- 172.21.15.44:0/1145884073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10e0003c20 0x7f10e0004020 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f10ec001890 tx=0x7f10ec01c8c0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10fda00640 1 -- 172.21.15.44:0/1145884073 reap_dead start 2024-09-13T08:37:45.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 -- 172.21.15.44:0/1145884073 shutdown_connections 2024-09-13T08:37:45.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 -- 172.21.15.44:0/1145884073 >> 172.21.15.44:0/1145884073 conn(0x7f10f80b2890 msgr2=0x7f10f80b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 -- 172.21.15.44:0/1145884073 shutdown_connections 2024-09-13T08:37:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 -- 172.21.15.44:0/1145884073 wait complete. 2024-09-13T08:37:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 Processor -- start 2024-09-13T08:37:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 -- start start 2024-09-13T08:37:45.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10e0003c20 0x7f10f815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f10f81ebff0 0x7f10f8145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f81ec470 0x7f10f8145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10f80711c0 con 0x7f10f81ec470 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8071040 con 0x7f10e0003c20 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10ffa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f10f8071340 con 0x7f10f81ebff0 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10fda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f81ec470 0x7f10f8145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10fd000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10e0003c20 0x7f10f815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10fda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f81ec470 0x7f10f8145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36270/0 (socket says 172.21.15.44:36270) 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.359+0000 7f10fda00640 1 -- 172.21.15.44:0/1430463203 learned_addr learned my addr 172.21.15.44:0/1430463203 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10f7e00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f10f81ebff0 0x7f10f8145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10fda00640 1 -- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f10f81ebff0 msgr2=0x7f10f8145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10fda00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f10f81ebff0 0x7f10f8145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10fda00640 1 -- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10e0003c20 msgr2=0x7f10f815b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10fda00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10e0003c20 0x7f10f815b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10fda00640 1 -- 172.21.15.44:0/1430463203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f8159af0 con 0x7f10f81ec470 2024-09-13T08:37:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10fda00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f81ec470 0x7f10f8145ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f10f80b4ce0 tx=0x7f10f00129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10f4c00640 1 -- 172.21.15.44:0/1430463203 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f000c260 con 0x7f10f81ec470 2024-09-13T08:37:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10f4c00640 1 -- 172.21.15.44:0/1430463203 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10f0003d90 con 0x7f10f81ec470 2024-09-13T08:37:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10f4c00640 1 -- 172.21.15.44:0/1430463203 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0004290 con 0x7f10f81ec470 2024-09-13T08:37:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10ffa00640 1 -- 172.21.15.44:0/1430463203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f80cda90 con 0x7f10f81ec470 2024-09-13T08:37:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10ffa00640 1 -- 172.21.15.44:0/1430463203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f80ce050 con 0x7f10f81ec470 2024-09-13T08:37:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10ffa00640 1 -- 172.21.15.44:0/1430463203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f81477b0 con 0x7f10f81ec470 2024-09-13T08:37:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.363+0000 7f10f4c00640 1 -- 172.21.15.44:0/1430463203 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f10f0004430 con 0x7f10f81ec470 2024-09-13T08:37:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.367+0000 7f10f4c00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f10d8072ec0 0x7f10d8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.367+0000 7f10f4c00640 1 -- 172.21.15.44:0/1430463203 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f10f0098540 con 0x7f10f81ec470 2024-09-13T08:37:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.367+0000 7f10fd000640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f10d8072ec0 0x7f10d8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.367+0000 7f10fd000640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f10d8072ec0 0x7f10d8075380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10ec001860 tx=0x7f10ec019700 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.371+0000 7f10f4c00640 1 -- 172.21.15.44:0/1430463203 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f10f0022020 con 0x7f10f81ec470 2024-09-13T08:37:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.547+0000 7f10ffa00640 1 -- 172.21.15.44:0/1430463203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10f8074a00 con 0x7f10f81ec470 2024-09-13T08:37:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.547+0000 7f10f4c00640 1 -- 172.21.15.44:0/1430463203 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16681 (secure 0 0 0) 0x7f10f00636f0 con 0x7f10f81ec470 2024-09-13T08:37:45.552 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:45.552 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":22,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:35.883263+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:37:32.247896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T08:37:33.241202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:37:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 -- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f10d8072ec0 msgr2=0x7f10d8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f10d8072ec0 0x7f10d8075380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10ec001860 tx=0x7f10ec019700 comp rx=0 tx=0).stop 2024-09-13T08:37:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 -- 172.21.15.44:0/1430463203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f81ec470 msgr2=0x7f10f8145ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f81ec470 0x7f10f8145ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f10f80b4ce0 tx=0x7f10f00129e0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 -- 172.21.15.44:0/1430463203 shutdown_connections 2024-09-13T08:37:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f10d8072ec0 0x7f10d8075380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f81ec470 0x7f10f8145ed0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f10f81ebff0 0x7f10f8145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 --2- 172.21.15.44:0/1430463203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10e0003c20 0x7f10f815b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 -- 172.21.15.44:0/1430463203 >> 172.21.15.44:0/1430463203 conn(0x7f10f80b2890 msgr2=0x7f10f80b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 -- 172.21.15.44:0/1430463203 shutdown_connections 2024-09-13T08:37:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.551+0000 7f10ffa00640 1 -- 172.21.15.44:0/1430463203 wait complete. 2024-09-13T08:37:45.575 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T08:37:45.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T08:37:45.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T08:37:45.577 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T08:37:45.577 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T08:37:45.578 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T08:37:45.578 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T08:37:45.578 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T08:37:45.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.827+0000 7f90da200640 1 -- 172.21.15.44:0/1852310131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc004190 msgr2=0x7f90bc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.827+0000 7f90da200640 1 --2- 172.21.15.44:0/1852310131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc004190 0x7f90bc004810 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009890 tx=0x7f90c001c6e0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.827+0000 7f90d3e00640 1 -- 172.21.15.44:0/1852310131 reap_dead start 2024-09-13T08:37:45.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.827+0000 7f90da200640 1 -- 172.21.15.44:0/1852310131 shutdown_connections 2024-09-13T08:37:45.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.827+0000 7f90da200640 1 -- 172.21.15.44:0/1852310131 >> 172.21.15.44:0/1852310131 conn(0x7f90d40b1b90 msgr2=0x7f90d40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.827+0000 7f90da200640 1 -- 172.21.15.44:0/1852310131 shutdown_connections 2024-09-13T08:37:45.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.827+0000 7f90da200640 1 -- 172.21.15.44:0/1852310131 wait complete. 2024-09-13T08:37:45.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.831+0000 7f90da200640 1 Processor -- start 2024-09-13T08:37:45.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.831+0000 7f90da200640 1 -- start start 2024-09-13T08:37:45.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.831+0000 7f90da200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc004190 0x7f90d415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.831+0000 7f90da200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d41f4180 0x7f90d41602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.831+0000 7f90da200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d4160820 0x7f90d41f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.831+0000 7f90d2a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d41f4180 0x7f90d41602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.831+0000 7f90d3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d4160820 0x7f90d41f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90d3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d4160820 0x7f90d41f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:52500/0 (socket says 172.21.15.44:52500) 2024-09-13T08:37:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90d2a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d41f4180 0x7f90d41602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37158/0 (socket says 172.21.15.44:37158) 2024-09-13T08:37:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90d3e00640 1 -- 172.21.15.44:0/2158863971 learned_addr learned my addr 172.21.15.44:0/2158863971 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90d3400640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc004190 0x7f90d415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4070360 con 0x7f90bc004190 2024-09-13T08:37:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90d40701e0 con 0x7f90d4160820 2024-09-13T08:37:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f90d40704e0 con 0x7f90d41f4180 2024-09-13T08:37:45.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90d3e00640 1 -- 172.21.15.44:0/2158863971 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d41f4180 msgr2=0x7f90d41602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90d3e00640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d41f4180 0x7f90d41602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90d3e00640 1 -- 172.21.15.44:0/2158863971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc004190 msgr2=0x7f90d415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90d3e00640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc004190 0x7f90d415e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90d3e00640 1 -- 172.21.15.44:0/2158863971 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d4160ce0 con 0x7f90d4160820 2024-09-13T08:37:45.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90d3e00640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d4160820 0x7f90d41f3500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f90d415e270 tx=0x7f90cc00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90c7400640 1 -- 172.21.15.44:0/2158863971 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc009e30 con 0x7f90d4160820 2024-09-13T08:37:45.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90c7400640 1 -- 172.21.15.44:0/2158863971 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90cc01a070 con 0x7f90d4160820 2024-09-13T08:37:45.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90c7400640 1 -- 172.21.15.44:0/2158863971 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0146c0 con 0x7f90d4160820 2024-09-13T08:37:45.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d4160fc0 con 0x7f90d4160820 2024-09-13T08:37:45.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d415e7e0 con 0x7f90d4160820 2024-09-13T08:37:45.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f984ec00640 1 -- 172.21.15.44:0/3107128210 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f984815abe0 msgr2=0x7f984806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f3706400640 1 -- 172.21.15.44:0/1226083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37001614d0 msgr2=0x7f37000d1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f3706400640 1 --2- 172.21.15.44:0/1226083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37001614d0 0x7f37000d1710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f37000cfbe0 tx=0x7f36ec01c670 comp rx=0 tx=0).stop 2024-09-13T08:37:45.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90d4161470 con 0x7f90d4160820 2024-09-13T08:37:45.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f984ec00640 1 --2- 172.21.15.44:0/3107128210 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f984815abe0 0x7f984806ecb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9834009900 tx=0x7f983401c900 comp rx=0 tx=0).stop 2024-09-13T08:37:45.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f984ec00640 1 -- 172.21.15.44:0/3107128210 shutdown_connections 2024-09-13T08:37:45.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f984ec00640 1 --2- 172.21.15.44:0/3107128210 >> v2:172.21.15.129:3300/0 conn(0x7f98480b2bf0 0x7f98480b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f984ec00640 1 --2- 172.21.15.44:0/3107128210 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f984815abe0 0x7f984806ecb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.835+0000 7f984ec00640 1 -- 172.21.15.44:0/3107128210 >> 172.21.15.44:0/3107128210 conn(0x7f98480b18f0 msgr2=0x7f98480b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.839+0000 7f984ec00640 1 -- 172.21.15.44:0/3107128210 shutdown_connections 2024-09-13T08:37:45.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.839+0000 7f984ec00640 1 -- 172.21.15.44:0/3107128210 wait complete. 2024-09-13T08:37:45.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.839+0000 7f3706400640 1 -- 172.21.15.44:0/1226083 shutdown_connections 2024-09-13T08:37:45.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.839+0000 7f3706400640 1 --2- 172.21.15.44:0/1226083 >> v2:172.21.15.129:3300/0 conn(0x7f37001618b0 0x7f37000aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.839+0000 7f3706400640 1 --2- 172.21.15.44:0/1226083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37001614d0 0x7f37000d1710 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.839+0000 7f3706400640 1 -- 172.21.15.44:0/1226083 >> 172.21.15.44:0/1226083 conn(0x7f37000d1dc0 msgr2=0x7f37000d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.839+0000 7f3706400640 1 -- 172.21.15.44:0/1226083 shutdown_connections 2024-09-13T08:37:45.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.839+0000 7f3706400640 1 -- 172.21.15.44:0/1226083 wait complete. 2024-09-13T08:37:45.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.839+0000 7f90c7400640 1 -- 172.21.15.44:0/2158863971 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f90cc012050 con 0x7f90d4160820 2024-09-13T08:37:45.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.839+0000 7f984ec00640 1 Processor -- start 2024-09-13T08:37:45.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f984ec00640 1 -- start start 2024-09-13T08:37:45.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f90c7400640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4072ec0 0x7f90b4075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f3706400640 1 Processor -- start 2024-09-13T08:37:45.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f3706400640 1 -- start start 2024-09-13T08:37:45.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f90d3400640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4072ec0 0x7f90b4075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f90c7400640 1 -- 172.21.15.44:0/2158863971 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f90cc094830 con 0x7f90d4160820 2024-09-13T08:37:45.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f1482c00640 1 -- 172.21.15.44:0/2798228804 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f147c0b3d30 msgr2=0x7f147c0b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f1482c00640 1 --2- 172.21.15.44:0/2798228804 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f147c0b3d30 0x7f147c0b41c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1474009800 tx=0x7f147401dd80 comp rx=0 tx=0).stop 2024-09-13T08:37:45.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f1482c00640 1 -- 172.21.15.44:0/2798228804 shutdown_connections 2024-09-13T08:37:45.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f1482c00640 1 --2- 172.21.15.44:0/2798228804 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f147c0b3d30 0x7f147c0b41c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f1482c00640 1 --2- 172.21.15.44:0/2798228804 >> v2:172.21.15.44:3300/0 conn(0x7f147c06f8c0 0x7f147c06fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f1482c00640 1 -- 172.21.15.44:0/2798228804 >> 172.21.15.44:0/2798228804 conn(0x7f147c0b18f0 msgr2=0x7f147c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f1482c00640 1 -- 172.21.15.44:0/2798228804 shutdown_connections 2024-09-13T08:37:45.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f3706400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f370006dd30 0x7f37000c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f1482c00640 1 -- 172.21.15.44:0/2798228804 wait complete. 2024-09-13T08:37:45.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f90c7400640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] conn(0x7f90b407b0e0 0x7f90b407d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f3706400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001614d0 0x7f370015ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f3706400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37001618b0 0x7f370015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f3706400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f370016b140 con 0x7f37001618b0 2024-09-13T08:37:45.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f3706400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f370016afc0 con 0x7f37001614d0 2024-09-13T08:37:45.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f3706400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f370016b2c0 con 0x7f370006dd30 2024-09-13T08:37:45.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f90d3400640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4072ec0 0x7f90b4075380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f90c8000f70 tx=0x7f90c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f90c7400640 1 -- 172.21.15.44:0/2158863971 --> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90cc00df90 con 0x7f90b407b0e0 2024-09-13T08:37:45.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f90c7400640 1 -- 172.21.15.44:0/2158863971 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f90cc094c20 con 0x7f90d4160820 2024-09-13T08:37:45.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f90d2a00640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] conn(0x7f90b407b0e0 0x7f90b407d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f984ec00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98480b2bf0 0x7f9848154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f984ec00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98480cebf0 0x7f98480adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f984ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f984815abe0 0x7f98480ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f984ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9848167d70 con 0x7f984815abe0 2024-09-13T08:37:45.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f984ec00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9848167bf0 con 0x7f98480cebf0 2024-09-13T08:37:45.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f984ec00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9848167ef0 con 0x7f98480b2bf0 2024-09-13T08:37:45.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f23b8e00640 1 -- 172.21.15.44:0/4096361789 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23b40cf1d0 msgr2=0x7f23b4160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f23b8e00640 1 --2- 172.21.15.44:0/4096361789 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23b40cf1d0 0x7f23b4160490 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009900 tx=0x7f23a001c8d0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f9847e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98480b2bf0 0x7f9848154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f9847400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98480cebf0 0x7f98480adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f23b8e00640 1 -- 172.21.15.44:0/4096361789 shutdown_connections 2024-09-13T08:37:45.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f23b8e00640 1 --2- 172.21.15.44:0/4096361789 >> v2:172.21.15.129:3300/0 conn(0x7f23b406e7a0 0x7f23b40ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f23b8e00640 1 --2- 172.21.15.44:0/4096361789 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23b40cf1d0 0x7f23b4160490 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f9847400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98480cebf0 0x7f98480adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:52526/0 (socket says 172.21.15.44:52526) 2024-09-13T08:37:45.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f9847400640 1 -- 172.21.15.44:0/1995204752 learned_addr learned my addr 172.21.15.44:0/1995204752 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:45.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f984cc00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f984815abe0 0x7f98480ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f0341800640 1 -- 172.21.15.44:0/1241172499 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f033c0d2060 msgr2=0x7f033c0cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f0341800640 1 --2- 172.21.15.44:0/1241172499 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f033c0d2060 0x7f033c0cf660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f033c08c5a0 tx=0x7f032801c8e0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f36fea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001614d0 0x7f370015ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f23b8e00640 1 -- 172.21.15.44:0/4096361789 >> 172.21.15.44:0/4096361789 conn(0x7f23b40b18f0 msgr2=0x7f23b40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f36ffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37001618b0 0x7f370015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f36ffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37001618b0 0x7f370015b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36330/0 (socket says 172.21.15.44:36330) 2024-09-13T08:37:45.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f36ffe00640 1 -- 172.21.15.44:0/3275075031 learned_addr learned my addr 172.21.15.44:0/3275075031 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:45.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f36ff400640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f370006dd30 0x7f37000c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.843+0000 7f36ff400640 1 -- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001614d0 msgr2=0x7f370015ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f23b8e00640 1 -- 172.21.15.44:0/4096361789 shutdown_connections 2024-09-13T08:37:45.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f36ff400640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001614d0 0x7f370015ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f23b8e00640 1 -- 172.21.15.44:0/4096361789 wait complete. 2024-09-13T08:37:45.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f90d2a00640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] conn(0x7f90b407b0e0 0x7f90b407d500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f36ff400640 1 -- 172.21.15.44:0/3275075031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37001618b0 msgr2=0x7f370015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f36ff400640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37001618b0 0x7f370015b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f36ff400640 1 -- 172.21.15.44:0/3275075031 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37001ff8b0 con 0x7f370006dd30 2024-09-13T08:37:45.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f36ffe00640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37001618b0 0x7f370015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:45.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f36fea00640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001614d0 0x7f370015ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:37:45.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f0341800640 1 -- 172.21.15.44:0/1241172499 shutdown_connections 2024-09-13T08:37:45.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f0341800640 1 --2- 172.21.15.44:0/1241172499 >> v2:172.21.15.44:3300/0 conn(0x7f033c0d09f0 0x7f033c0d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f0341800640 1 --2- 172.21.15.44:0/1241172499 >> v2:172.21.15.129:3300/0 conn(0x7f033c0d2440 0x7f033c0d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f0341800640 1 --2- 172.21.15.44:0/1241172499 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f033c0d2060 0x7f033c0cf660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f0341800640 1 -- 172.21.15.44:0/1241172499 >> 172.21.15.44:0/1241172499 conn(0x7f033c0d3fe0 msgr2=0x7f033c0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f9847400640 1 -- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98480b2bf0 msgr2=0x7f9848154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f9847400640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98480b2bf0 0x7f9848154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f9847400640 1 -- 172.21.15.44:0/1995204752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f984815abe0 msgr2=0x7f98480ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f9847400640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f984815abe0 0x7f98480ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f9847400640 1 -- 172.21.15.44:0/1995204752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984814ca00 con 0x7f98480cebf0 2024-09-13T08:37:45.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f9847e00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98480b2bf0 0x7f9848154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:45.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f36ff400640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f370006dd30 0x7f37000c6090 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3700073d10 tx=0x7f36ec004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f0341800640 1 -- 172.21.15.44:0/1241172499 shutdown_connections 2024-09-13T08:37:45.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f9847400640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98480cebf0 0x7f98480adec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f984000c7c0 tx=0x7f984000cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f0341800640 1 -- 172.21.15.44:0/1241172499 wait complete. 2024-09-13T08:37:45.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f1482c00640 1 Processor -- start 2024-09-13T08:37:45.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f23b8e00640 1 Processor -- start 2024-09-13T08:37:45.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f1482c00640 1 -- start start 2024-09-13T08:37:45.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f36f7400640 1 -- 172.21.15.44:0/3275075031 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec003600 con 0x7f370006dd30 2024-09-13T08:37:45.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f90c7400640 1 -- 172.21.15.44:0/2158863971 <== osd.0 v2:172.21.15.44:6800/1438138297 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90cc00df90 con 0x7f90b407b0e0 2024-09-13T08:37:45.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f36f7400640 1 -- 172.21.15.44:0/3275075031 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36ec0037c0 con 0x7f370006dd30 2024-09-13T08:37:45.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f36f7400640 1 -- 172.21.15.44:0/3275075031 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec003c10 con 0x7f370006dd30 2024-09-13T08:37:45.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.847+0000 7f23b8e00640 1 -- start start 2024-09-13T08:37:45.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37001ffb90 con 0x7f370006dd30 2024-09-13T08:37:45.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3700200150 con 0x7f370006dd30 2024-09-13T08:37:45.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b8e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23b406e7a0 0x7f23b41fc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f1482c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f147c06f8c0 0x7f147c1375f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f1482c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f147c0b3d30 0x7f147c158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f1482c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147c0cc740 0x7f147c13f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f1482c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f147c167500 con 0x7f147c0cc740 2024-09-13T08:37:45.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f1482c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f147c167380 con 0x7f147c0b3d30 2024-09-13T08:37:45.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f1482c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f147c167680 con 0x7f147c06f8c0 2024-09-13T08:37:45.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b8e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23b40cf1d0 0x7f23b41550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23b415f9a0 0x7f23b415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f0341800640 1 Processor -- start 2024-09-13T08:37:45.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f983fe00640 1 -- 172.21.15.44:0/1995204752 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840016020 con 0x7f98480cebf0 2024-09-13T08:37:45.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f983fe00640 1 -- 172.21.15.44:0/1995204752 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f984000ab40 con 0x7f98480cebf0 2024-09-13T08:37:45.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f983fe00640 1 -- 172.21.15.44:0/1995204752 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840012420 con 0x7f98480cebf0 2024-09-13T08:37:45.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b1e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23b406e7a0 0x7f23b41fc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37000b4c10 con 0x7f370006dd30 2024-09-13T08:37:45.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f1480c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147c0cc740 0x7f147c13f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b1e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23b406e7a0 0x7f23b41fc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37226/0 (socket says 172.21.15.44:37226) 2024-09-13T08:37:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23b415f9a0 0x7f23b415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23b415f9a0 0x7f23b415e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36366/0 (socket says 172.21.15.44:36366) 2024-09-13T08:37:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b1e00640 1 -- 172.21.15.44:0/151801090 learned_addr learned my addr 172.21.15.44:0/151801090 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b8e00640 1 -- 172.21.15.44:0/151801090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23b4169200 con 0x7f23b415f9a0 2024-09-13T08:37:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b8e00640 1 -- 172.21.15.44:0/151801090 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23b4169080 con 0x7f23b40cf1d0 2024-09-13T08:37:45.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f1480c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147c0cc740 0x7f147c13f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36376/0 (socket says 172.21.15.44:36376) 2024-09-13T08:37:45.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b8e00640 1 -- 172.21.15.44:0/151801090 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f23b4169380 con 0x7f23b406e7a0 2024-09-13T08:37:45.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f1480c00640 1 -- 172.21.15.44:0/2098091329 learned_addr learned my addr 172.21.15.44:0/2098091329 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:45.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f147b400640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f147c0b3d30 0x7f147c158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f147be00640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f147c06f8c0 0x7f147c1375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b1400640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23b40cf1d0 0x7f23b41550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b1e00640 1 -- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23b40cf1d0 msgr2=0x7f23b41550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b1e00640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23b40cf1d0 0x7f23b41550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f147b400640 1 -- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f147c06f8c0 msgr2=0x7f147c1375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f147b400640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f147c06f8c0 0x7f147c1375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f147b400640 1 -- 172.21.15.44:0/2098091329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147c0cc740 msgr2=0x7f147c13f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f147b400640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147c0cc740 0x7f147c13f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f147b400640 1 -- 172.21.15.44:0/2098091329 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147c13cd00 con 0x7f147c0b3d30 2024-09-13T08:37:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b1e00640 1 -- 172.21.15.44:0/151801090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23b415f9a0 msgr2=0x7f23b415e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b1e00640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23b415f9a0 0x7f23b415e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b1e00640 1 -- 172.21.15.44:0/151801090 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b415f2c0 con 0x7f23b406e7a0 2024-09-13T08:37:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b2800640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23b415f9a0 0x7f23b415e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f0341800640 1 -- start start 2024-09-13T08:37:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b1e00640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23b406e7a0 0x7f23b41fc470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f23a400b410 tx=0x7f23a400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f36f7400640 1 -- 172.21.15.44:0/3275075031 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f36ec0a6040 con 0x7f370006dd30 2024-09-13T08:37:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f147b400640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f147c0b3d30 0x7f147c158270 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f146800c390 tx=0x7f146800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23aa000640 1 -- 172.21.15.44:0/151801090 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4015030 con 0x7f23b406e7a0 2024-09-13T08:37:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23aa000640 1 -- 172.21.15.44:0/151801090 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23a4004020 con 0x7f23b406e7a0 2024-09-13T08:37:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f82f0600640 1 -- 172.21.15.44:0/1475960549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82e80d8f10 msgr2=0x7f82e8163aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f82f0600640 1 --2- 172.21.15.44:0/1475960549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82e80d8f10 0x7f82e8163aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009900 tx=0x7f82dc01c840 comp rx=0 tx=0).stop 2024-09-13T08:37:45.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23aa000640 1 -- 172.21.15.44:0/151801090 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a40044b0 con 0x7f23b406e7a0 2024-09-13T08:37:45.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f23b8e00640 1 -- 172.21.15.44:0/151801090 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b40b2f40 con 0x7f23b406e7a0 2024-09-13T08:37:45.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f0341800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f033c0d09f0 0x7f033c14f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f47ddc00640 1 -- 172.21.15.44:0/3240263064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47d8070960 msgr2=0x7f47d80cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f47ddc00640 1 --2- 172.21.15.44:0/3240263064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47d8070960 0x7f47d80cc740 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47d8074ca0 tx=0x7f47c001db60 comp rx=0 tx=0).stop 2024-09-13T08:37:45.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98481619b0 con 0x7f98480cebf0 2024-09-13T08:37:45.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.851+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98480aead0 con 0x7f98480cebf0 2024-09-13T08:37:45.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f36f7400640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f36e0072fc0 0x7f36e0075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98481f92c0 con 0x7f98480cebf0 2024-09-13T08:37:45.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f23b8e00640 1 -- 172.21.15.44:0/151801090 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b40b3500 con 0x7f23b406e7a0 2024-09-13T08:37:45.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f23b8e00640 1 -- 172.21.15.44:0/151801090 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23b40b2cf0 con 0x7f23b406e7a0 2024-09-13T08:37:45.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f47ddc00640 1 -- 172.21.15.44:0/3240263064 shutdown_connections 2024-09-13T08:37:45.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f1473e00640 1 -- 172.21.15.44:0/2098091329 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468017070 con 0x7f147c0b3d30 2024-09-13T08:37:45.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f47ddc00640 1 --2- 172.21.15.44:0/3240263064 >> v2:172.21.15.44:3300/0 conn(0x7f47d806eeb0 0x7f47d8164fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f1473e00640 1 -- 172.21.15.44:0/2098091329 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14680053c0 con 0x7f147c0b3d30 2024-09-13T08:37:45.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f1473e00640 1 -- 172.21.15.44:0/2098091329 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468003840 con 0x7f147c0b3d30 2024-09-13T08:37:45.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147c13cfe0 con 0x7f147c0b3d30 2024-09-13T08:37:45.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f147c13b530 con 0x7f147c0b3d30 2024-09-13T08:37:45.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f47ddc00640 1 --2- 172.21.15.44:0/3240263064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47d8070960 0x7f47d80cc740 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f36f7400640 1 -- 172.21.15.44:0/3275075031 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f36ec12e910 con 0x7f370006dd30 2024-09-13T08:37:45.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f36f7400640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] conn(0x7f36e007b1e0 0x7f36e007d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f36f7400640 1 -- 172.21.15.44:0/3275075031 --> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36e007dcb0 con 0x7f36e007b1e0 2024-09-13T08:37:45.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f0341800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f033c0d2060 0x7f033c09e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f82f0600640 1 -- 172.21.15.44:0/1475960549 shutdown_connections 2024-09-13T08:37:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f82f0600640 1 --2- 172.21.15.44:0/1475960549 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82e81629e0 0x7f82e8163fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f82f0600640 1 --2- 172.21.15.44:0/1475960549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82e80d8f10 0x7f82e8163aa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f82f0600640 1 -- 172.21.15.44:0/1475960549 >> 172.21.15.44:0/1475960549 conn(0x7f82e80d83f0 msgr2=0x7f82e80d8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f0341800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f033c0d2440 0x7f033c1fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f0341800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f033c0b8650 con 0x7f033c0d2060 2024-09-13T08:37:45.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f0341800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f033c0b84d0 con 0x7f033c0d09f0 2024-09-13T08:37:45.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f0341800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f033c0b87d0 con 0x7f033c0d2440 2024-09-13T08:37:45.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f47ddc00640 1 -- 172.21.15.44:0/3240263064 >> 172.21.15.44:0/3240263064 conn(0x7f47d80b18f0 msgr2=0x7f47d80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f033c0d2440 0x7f033c1fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f36f7400640 1 -- 172.21.15.44:0/3275075031 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f36ec12ed00 con 0x7f370006dd30 2024-09-13T08:37:45.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f033c0d2440 0x7f033c1fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37234/0 (socket says 172.21.15.44:37234) 2024-09-13T08:37:45.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033b400640 1 -- 172.21.15.44:0/3066681922 learned_addr learned my addr 172.21.15.44:0/3066681922 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:45.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033aa00640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f033c0d09f0 0x7f033c14f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f36fea00640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f36e0072fc0 0x7f36e0075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f47ddc00640 1 -- 172.21.15.44:0/3240263064 shutdown_connections 2024-09-13T08:37:45.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f82f0600640 1 -- 172.21.15.44:0/1475960549 shutdown_connections 2024-09-13T08:37:45.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f36ffe00640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] conn(0x7f36e007b1e0 0x7f36e007d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033a000640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f033c0d2060 0x7f033c09e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f147c13cab0 con 0x7f147c0b3d30 2024-09-13T08:37:45.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f983fe00640 1 -- 172.21.15.44:0/1995204752 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f9840019020 con 0x7f98480cebf0 2024-09-13T08:37:45.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f983fe00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9828072f10 0x7f98280753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f983fe00640 1 -- 172.21.15.44:0/1995204752 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9840093ab0 con 0x7f98480cebf0 2024-09-13T08:37:45.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f983fe00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] conn(0x7f982807b130 0x7f982807d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f983fe00640 1 -- 172.21.15.44:0/1995204752 --> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f982807dc00 con 0x7f982807b130 2024-09-13T08:37:45.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f983fe00640 1 -- 172.21.15.44:0/1995204752 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9840093f90 con 0x7f98480cebf0 2024-09-13T08:37:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f36ffe00640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] conn(0x7f36e007b1e0 0x7f36e007d600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f36fea00640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f36e0072fc0 0x7f36e0075480 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f37000d2bc0 tx=0x7f36f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033aa00640 1 -- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f033c0d2440 msgr2=0x7f033c1fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033aa00640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f033c0d2440 0x7f033c1fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033aa00640 1 -- 172.21.15.44:0/3066681922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f033c0d2060 msgr2=0x7f033c09e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033aa00640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f033c0d2060 0x7f033c09e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033aa00640 1 -- 172.21.15.44:0/3066681922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c0a0090 con 0x7f033c0d09f0 2024-09-13T08:37:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f033a000640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f033c0d2060 0x7f033c09e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:37:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f033aa00640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f033c0d09f0 0x7f033c14f2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03280ac000 tx=0x7f03280a5650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f0332a00640 1 -- 172.21.15.44:0/3066681922 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280b2040 con 0x7f033c0d09f0 2024-09-13T08:37:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f0332a00640 1 -- 172.21.15.44:0/3066681922 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03280042e0 con 0x7f033c0d09f0 2024-09-13T08:37:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f0332a00640 1 -- 172.21.15.44:0/3066681922 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328005290 con 0x7f033c0d09f0 2024-09-13T08:37:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f47ddc00640 1 -- 172.21.15.44:0/3240263064 wait complete. 2024-09-13T08:37:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.855+0000 7f82f0600640 1 -- 172.21.15.44:0/1475960549 wait complete. 2024-09-13T08:37:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82f0600640 1 Processor -- start 2024-09-13T08:37:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f23aa000640 1 -- 172.21.15.44:0/151801090 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f23a4018070 con 0x7f23b406e7a0 2024-09-13T08:37:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82f0600640 1 -- start start 2024-09-13T08:37:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82f0600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82e8070e10 0x7f82e80bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f9847e00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9828072f10 0x7f98280753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f0341800640 1 -- 172.21.15.44:0/3066681922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033c0a0350 con 0x7f033c0d09f0 2024-09-13T08:37:45.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f0341800640 1 -- 172.21.15.44:0/3066681922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033c09e170 con 0x7f033c0d09f0 2024-09-13T08:37:45.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82f0600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e80d8f10 0x7f82e81645f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82f0600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82e81629e0 0x7f82e8164b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f984cc00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] conn(0x7f982807b130 0x7f982807d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82f0600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f82e816f690 con 0x7f82e80d8f10 2024-09-13T08:37:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82f0600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f82e816f510 con 0x7f82e8070e10 2024-09-13T08:37:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82f0600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f82e816f810 con 0x7f82e81629e0 2024-09-13T08:37:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82ed200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e80d8f10 0x7f82e81645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82ee600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82e81629e0 0x7f82e8164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f1473e00640 1 -- 172.21.15.44:0/2098091329 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f146801a070 con 0x7f147c0b3d30 2024-09-13T08:37:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f1473e00640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f145c072f10 0x7f145c0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82ee600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82e81629e0 0x7f82e8164b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37250/0 (socket says 172.21.15.44:37250) 2024-09-13T08:37:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82ee600640 1 -- 172.21.15.44:0/629555346 learned_addr learned my addr 172.21.15.44:0/629555346 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f1473e00640 1 -- 172.21.15.44:0/2098091329 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1468093480 con 0x7f147c0b3d30 2024-09-13T08:37:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f23aa000640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2394072ef0 0x7f23940753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f23aa000640 1 -- 172.21.15.44:0/151801090 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f23a4094010 con 0x7f23b406e7a0 2024-09-13T08:37:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82edc00640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82e8070e10 0x7f82e80bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f23aa000640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] conn(0x7f239407b090 0x7f239407d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f23aa000640 1 -- 172.21.15.44:0/151801090 --> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23a4016e50 con 0x7f239407b090 2024-09-13T08:37:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f23aa000640 1 -- 172.21.15.44:0/151801090 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f23a40943c0 con 0x7f23b406e7a0 2024-09-13T08:37:45.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f1473e00640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] conn(0x7f145c07b130 0x7f145c07d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f1473e00640 1 -- 172.21.15.44:0/2098091329 --> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f146801fe50 con 0x7f145c07b130 2024-09-13T08:37:45.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82ee600640 1 -- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82e8070e10 msgr2=0x7f82e80bcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82ee600640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82e8070e10 0x7f82e80bcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f82ee600640 1 -- 172.21.15.44:0/629555346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e80d8f10 msgr2=0x7f82e81645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f1473e00640 1 -- 172.21.15.44:0/2098091329 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1468093830 con 0x7f147c0b3d30 2024-09-13T08:37:45.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f82ee600640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e80d8f10 0x7f82e81645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f0327e00640 1 -- 172.21.15.44:0/3066681922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f033c069b20 con 0x7f033c0d09f0 2024-09-13T08:37:45.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f23b2800640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] conn(0x7f239407b090 0x7f239407d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f1480c00640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] conn(0x7f145c07b130 0x7f145c07d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f82ee600640 1 -- 172.21.15.44:0/629555346 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e80bd460 con 0x7f82e81629e0 2024-09-13T08:37:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f82ed200640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e80d8f10 0x7f82e81645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f9847e00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9828072f10 0x7f98280753d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98340af000 tx=0x7f983401b140 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f147be00640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f145c072f10 0x7f145c0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f47ddc00640 1 Processor -- start 2024-09-13T08:37:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f23b1400640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2394072ef0 0x7f23940753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.859+0000 7f36f7400640 1 -- 172.21.15.44:0/3275075031 <== osd.2 v2:172.21.15.44:6808/231873797 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36e007dcb0 con 0x7f36e007b1e0 2024-09-13T08:37:45.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f23b2800640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] conn(0x7f239407b090 0x7f239407d4b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f47ddc00640 1 -- start start 2024-09-13T08:37:45.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f0332a00640 1 -- 172.21.15.44:0/3066681922 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f0328005430 con 0x7f033c0d09f0 2024-09-13T08:37:45.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f82ee600640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82e81629e0 0x7f82e8164b30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82e000ea80 tx=0x7f82e000ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f23b1400640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2394072ef0 0x7f23940753b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23a00af000 tx=0x7f23a0019670 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f984cc00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] conn(0x7f982807b130 0x7f982807d550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f23aa000640 1 -- 172.21.15.44:0/151801090 <== osd.5 v2:172.21.15.129:6811/3446737198 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23a4016e50 con 0x7f239407b090 2024-09-13T08:37:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f983fe00640 1 -- 172.21.15.44:0/1995204752 <== osd.1 v2:172.21.15.44:6816/3258852146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f982807dc00 con 0x7f982807b130 2024-09-13T08:37:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f82e5600640 1 -- 172.21.15.44:0/629555346 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0004b30 con 0x7f82e81629e0 2024-09-13T08:37:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f82e5600640 1 -- 172.21.15.44:0/629555346 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82e0004cf0 con 0x7f82e81629e0 2024-09-13T08:37:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f47ddc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47d806eeb0 0x7f47d8141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f47ddc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47d8070960 0x7f47d809c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f47ddc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47d80ccc80 0x7f47d81593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f47ddc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f47d81676a0 con 0x7f47d806eeb0 2024-09-13T08:37:45.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f47ddc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47d8167520 con 0x7f47d80ccc80 2024-09-13T08:37:45.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f47ddc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f47d8167820 con 0x7f47d8070960 2024-09-13T08:37:45.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f147be00640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f145c072f10 0x7f145c0753d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f146c009c20 tx=0x7f146c009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d7400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47d80ccc80 0x7f47d81593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f1480c00640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] conn(0x7f145c07b130 0x7f145c07d550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d7400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47d80ccc80 0x7f47d81593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:52610/0 (socket says 172.21.15.44:52610) 2024-09-13T08:37:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d6000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47d8070960 0x7f47d809c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d6000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47d8070960 0x7f47d809c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37264/0 (socket says 172.21.15.44:37264) 2024-09-13T08:37:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d6000640 1 -- 172.21.15.44:0/1509174257 learned_addr learned my addr 172.21.15.44:0/1509174257 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e81fc3d0 con 0x7f82e81629e0 2024-09-13T08:37:45.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.863+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e81fc990 con 0x7f82e81629e0 2024-09-13T08:37:45.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f82e5600640 1 -- 172.21.15.44:0/629555346 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0005280 con 0x7f82e81629e0 2024-09-13T08:37:45.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f0332a00640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f031c072ec0 0x7f031c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f0332a00640 1 -- 172.21.15.44:0/3066681922 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f032812ec30 con 0x7f033c0d09f0 2024-09-13T08:37:45.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f0332a00640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] conn(0x7f031c07b060 0x7f031c07d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f0332a00640 1 -- 172.21.15.44:0/3066681922 --> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03280c2e80 con 0x7f031c07b060 2024-09-13T08:37:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f0332a00640 1 -- 172.21.15.44:0/3066681922 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f032812efe0 con 0x7f033c0d09f0 2024-09-13T08:37:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f033a000640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f031c072ec0 0x7f031c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f033a000640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f031c072ec0 0x7f031c075380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f033c1f9e20 tx=0x7f032c005d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f1473e00640 1 -- 172.21.15.44:0/2098091329 <== osd.3 v2:172.21.15.44:6824/3593887774 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f146801fe50 con 0x7f145c07b130 2024-09-13T08:37:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f033b400640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] conn(0x7f031c07b060 0x7f031c07d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82c4001040 con 0x7f82e81629e0 2024-09-13T08:37:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f033b400640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] conn(0x7f031c07b060 0x7f031c07d480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d6000640 1 -- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47d80ccc80 msgr2=0x7f47d81593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d6000640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47d80ccc80 0x7f47d81593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d6000640 1 -- 172.21.15.44:0/1509174257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47d806eeb0 msgr2=0x7f47d8141570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d6000640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47d806eeb0 0x7f47d8141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d6000640 1 -- 172.21.15.44:0/1509174257 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d81fa4b0 con 0x7f47d8070960 2024-09-13T08:37:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.867+0000 7f47d6000640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47d8070960 0x7f47d809c9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c790 tx=0x7f47c800cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f0332a00640 1 -- 172.21.15.44:0/3066681922 <== osd.4 v2:172.21.15.129:6826/3123318411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03280c2e80 con 0x7f031c07b060 2024-09-13T08:37:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f47cea00640 1 -- 172.21.15.44:0/1509174257 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8003b80 con 0x7f47d8070960 2024-09-13T08:37:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f47cea00640 1 -- 172.21.15.44:0/1509174257 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47c8003d20 con 0x7f47d8070960 2024-09-13T08:37:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f47cea00640 1 -- 172.21.15.44:0/1509174257 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8005310 con 0x7f47d8070960 2024-09-13T08:37:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d809b920 con 0x7f47d8070960 2024-09-13T08:37:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d809bec0 con 0x7f47d8070960 2024-09-13T08:37:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47b0001040 con 0x7f47d8070960 2024-09-13T08:37:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f82e5600640 1 -- 172.21.15.44:0/629555346 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f82e0005420 con 0x7f82e81629e0 2024-09-13T08:37:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f82e5600640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f82cc072fc0 0x7f82cc075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f82e5600640 1 -- 172.21.15.44:0/629555346 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f82e0094b00 con 0x7f82e81629e0 2024-09-13T08:37:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f82e5600640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] conn(0x7f82cc07b160 0x7f82cc07d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f82e5600640 1 -- 172.21.15.44:0/629555346 --> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82e0019e70 con 0x7f82cc07b160 2024-09-13T08:37:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f82e5600640 1 -- 172.21.15.44:0/629555346 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f82e005bc80 con 0x7f82e81629e0 2024-09-13T08:37:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.871+0000 7f82ed200640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] conn(0x7f82cc07b160 0x7f82cc07d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.875+0000 7f82edc00640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f82cc072fc0 0x7f82cc075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.875+0000 7f82edc00640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f82cc072fc0 0x7f82cc075480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82d8000f70 tx=0x7f82d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.875+0000 7f82ed200640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] conn(0x7f82cc07b160 0x7f82cc07d580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.875+0000 7f47cea00640 1 -- 172.21.15.44:0/1509174257 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f47c80056c0 con 0x7f47d8070960 2024-09-13T08:37:45.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.875+0000 7f82e5600640 1 -- 172.21.15.44:0/629555346 <== osd.7 v2:172.21.15.129:6810/1299717984 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f82e0019e70 con 0x7f82cc07b160 2024-09-13T08:37:45.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.875+0000 7f47cea00640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47b4072f90 0x7f47b4075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.875+0000 7f47cea00640 1 -- 172.21.15.44:0/1509174257 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f47c8094160 con 0x7f47d8070960 2024-09-13T08:37:45.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.875+0000 7f47cea00640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] conn(0x7f47b407b1b0 0x7f47b407d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:45.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.875+0000 7f47cea00640 1 -- 172.21.15.44:0/1509174257 --> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47b407dc80 con 0x7f47b407b1b0 2024-09-13T08:37:45.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.875+0000 7f47cea00640 1 -- 172.21.15.44:0/1509174257 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f47c8094550 con 0x7f47d8070960 2024-09-13T08:37:45.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.879+0000 7f47d6a00640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47b4072f90 0x7f47b4075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.879+0000 7f47d7400640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] conn(0x7f47b407b1b0 0x7f47b407d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.879+0000 7f47d6a00640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47b4072f90 0x7f47b4075450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47c00b0000 tx=0x7f47c001a900 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.879+0000 7f47d7400640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] conn(0x7f47b407b1b0 0x7f47b407d5d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.879+0000 7f47cea00640 1 -- 172.21.15.44:0/1509174257 <== osd.6 v2:172.21.15.129:6802/2280849631 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47b407dc80 con 0x7f47b407b1b0 2024-09-13T08:37:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.883+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 --> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f90d40af440 con 0x7f90b407b0e0 2024-09-13T08:37:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.883+0000 7f90c7400640 1 -- 172.21.15.44:0/2158863971 <== osd.0 v2:172.21.15.44:6800/1438138297 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f90d40af440 con 0x7f90b407b0e0 2024-09-13T08:37:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 >> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] conn(0x7f90b407b0e0 msgr2=0x7f90b407d500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90da200640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] conn(0x7f90b407b0e0 0x7f90b407d500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4072ec0 msgr2=0x7f90b4075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90da200640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4072ec0 0x7f90b4075380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f90c8000f70 tx=0x7f90c8009040 comp rx=0 tx=0).stop 2024-09-13T08:37:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d4160820 msgr2=0x7f90d41f3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90da200640 1 --2- 172.21.15.44:0/2158863971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d4160820 0x7f90d41f3500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f90d415e270 tx=0x7f90cc00c490 comp rx=0 tx=0).stop 2024-09-13T08:37:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90d3e00640 1 -- 172.21.15.44:0/2158863971 reap_dead start 2024-09-13T08:37:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 shutdown_connections 2024-09-13T08:37:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 >> 172.21.15.44:0/2158863971 conn(0x7f90d40b1b90 msgr2=0x7f90d40afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 shutdown_connections 2024-09-13T08:37:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.887+0000 7f90da200640 1 -- 172.21.15.44:0/2158863971 wait complete. 2024-09-13T08:37:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.891+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 --> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37000d1710 con 0x7f36e007b1e0 2024-09-13T08:37:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 --> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f98480b3060 con 0x7f982807b130 2024-09-13T08:37:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f36f7400640 1 -- 172.21.15.44:0/3275075031 <== osd.2 v2:172.21.15.44:6808/231873797 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f37000d1710 con 0x7f36e007b1e0 2024-09-13T08:37:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 >> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] conn(0x7f36e007b1e0 msgr2=0x7f36e007d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f3706400640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] conn(0x7f36e007b1e0 0x7f36e007d600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f36e0072fc0 msgr2=0x7f36e0075480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f3706400640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f36e0072fc0 0x7f36e0075480 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f37000d2bc0 tx=0x7f36f0008040 comp rx=0 tx=0).stop 2024-09-13T08:37:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f370006dd30 msgr2=0x7f37000c6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f3706400640 1 --2- 172.21.15.44:0/3275075031 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f370006dd30 0x7f37000c6090 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3700073d10 tx=0x7f36ec004270 comp rx=0 tx=0).stop 2024-09-13T08:37:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f36ffe00640 1 -- 172.21.15.44:0/3275075031 reap_dead start 2024-09-13T08:37:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 shutdown_connections 2024-09-13T08:37:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 >> 172.21.15.44:0/3275075031 conn(0x7f37000d1dc0 msgr2=0x7f370015e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f983fe00640 1 -- 172.21.15.44:0/1995204752 <== osd.1 v2:172.21.15.44:6816/3258852146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f98480b3060 con 0x7f982807b130 2024-09-13T08:37:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 shutdown_connections 2024-09-13T08:37:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.895+0000 7f3706400640 1 -- 172.21.15.44:0/3275075031 wait complete. 2024-09-13T08:37:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 >> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] conn(0x7f982807b130 msgr2=0x7f982807d550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984ec00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] conn(0x7f982807b130 0x7f982807d550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f0327e00640 1 -- 172.21.15.44:0/3066681922 --> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f033c0b8d60 con 0x7f031c07b060 2024-09-13T08:37:45.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f23b8e00640 1 -- 172.21.15.44:0/151801090 --> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f23b40ce2e0 con 0x7f239407b090 2024-09-13T08:37:45.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 --> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f147c13b080 con 0x7f145c07b130 2024-09-13T08:37:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9828072f10 msgr2=0x7f98280753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984ec00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9828072f10 0x7f98280753d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98340af000 tx=0x7f983401b140 comp rx=0 tx=0).stop 2024-09-13T08:37:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98480cebf0 msgr2=0x7f98480adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984ec00640 1 --2- 172.21.15.44:0/1995204752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98480cebf0 0x7f98480adec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f984000c7c0 tx=0x7f984000cc80 comp rx=0 tx=0).stop 2024-09-13T08:37:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984cc00640 1 -- 172.21.15.44:0/1995204752 reap_dead start 2024-09-13T08:37:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 shutdown_connections 2024-09-13T08:37:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 >> 172.21.15.44:0/1995204752 conn(0x7f98480b18f0 msgr2=0x7f984815afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 shutdown_connections 2024-09-13T08:37:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f984ec00640 1 -- 172.21.15.44:0/1995204752 wait complete. 2024-09-13T08:37:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1473e00640 1 -- 172.21.15.44:0/2098091329 <== osd.3 v2:172.21.15.44:6824/3593887774 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f147c13b080 con 0x7f145c07b130 2024-09-13T08:37:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 >> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] conn(0x7f145c07b130 msgr2=0x7f145c07d550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1482c00640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] conn(0x7f145c07b130 0x7f145c07d550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f145c072f10 msgr2=0x7f145c0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1482c00640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f145c072f10 0x7f145c0753d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f146c009c20 tx=0x7f146c009260 comp rx=0 tx=0).stop 2024-09-13T08:37:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f147c0b3d30 msgr2=0x7f147c158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1482c00640 1 --2- 172.21.15.44:0/2098091329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f147c0b3d30 0x7f147c158270 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f146800c390 tx=0x7f146800c850 comp rx=0 tx=0).stop 2024-09-13T08:37:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1480c00640 1 -- 172.21.15.44:0/2098091329 reap_dead start 2024-09-13T08:37:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 shutdown_connections 2024-09-13T08:37:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.899+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 >> 172.21.15.44:0/2098091329 conn(0x7f147c0b18f0 msgr2=0x7f147c0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f0332a00640 1 -- 172.21.15.44:0/3066681922 <== osd.4 v2:172.21.15.129:6826/3123318411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f033c0b8d60 con 0x7f031c07b060 2024-09-13T08:37:45.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f23aa000640 1 -- 172.21.15.44:0/151801090 <== osd.5 v2:172.21.15.129:6811/3446737198 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f23b40ce2e0 con 0x7f239407b090 2024-09-13T08:37:45.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f239f400640 1 -- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] conn(0x7f239407b090 msgr2=0x7f239407d4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f239f400640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] conn(0x7f239407b090 0x7f239407d4b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f0327e00640 1 -- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] conn(0x7f031c07b060 msgr2=0x7f031c07d480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f0327e00640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] conn(0x7f031c07b060 0x7f031c07d480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f239f400640 1 -- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2394072ef0 msgr2=0x7f23940753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f239f400640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2394072ef0 0x7f23940753b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23a00af000 tx=0x7f23a0019670 comp rx=0 tx=0).stop 2024-09-13T08:37:45.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f239f400640 1 -- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23b406e7a0 msgr2=0x7f23b41fc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f239f400640 1 --2- 172.21.15.44:0/151801090 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23b406e7a0 0x7f23b41fc470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f23a400b410 tx=0x7f23a400b8d0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f23b2800640 1 -- 172.21.15.44:0/151801090 reap_dead start 2024-09-13T08:37:45.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 shutdown_connections 2024-09-13T08:37:45.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f1482c00640 1 -- 172.21.15.44:0/2098091329 wait complete. 2024-09-13T08:37:45.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f0327e00640 1 -- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f031c072ec0 msgr2=0x7f031c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f0327e00640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f031c072ec0 0x7f031c075380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f033c1f9e20 tx=0x7f032c005d30 comp rx=0 tx=0).stop 2024-09-13T08:37:45.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f0327e00640 1 -- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f033c0d09f0 msgr2=0x7f033c14f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f0327e00640 1 --2- 172.21.15.44:0/3066681922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f033c0d09f0 0x7f033c14f2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03280ac000 tx=0x7f03280a5650 comp rx=0 tx=0).stop 2024-09-13T08:37:45.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.903+0000 7f033b400640 1 -- 172.21.15.44:0/3066681922 reap_dead start 2024-09-13T08:37:45.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f239f400640 1 -- 172.21.15.44:0/151801090 shutdown_connections 2024-09-13T08:37:45.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f239f400640 1 -- 172.21.15.44:0/151801090 >> 172.21.15.44:0/151801090 conn(0x7f23b40b18f0 msgr2=0x7f23b4073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f0327e00640 1 -- 172.21.15.44:0/3066681922 shutdown_connections 2024-09-13T08:37:45.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f0327e00640 1 -- 172.21.15.44:0/3066681922 >> 172.21.15.44:0/3066681922 conn(0x7f033c0d3fe0 msgr2=0x7f033c15f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f239f400640 1 -- 172.21.15.44:0/151801090 shutdown_connections 2024-09-13T08:37:45.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f0327e00640 1 -- 172.21.15.44:0/3066681922 shutdown_connections 2024-09-13T08:37:45.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f0327e00640 1 -- 172.21.15.44:0/3066681922 wait complete. 2024-09-13T08:37:45.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f239f400640 1 -- 172.21.15.44:0/151801090 wait complete. 2024-09-13T08:37:45.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 --> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82c4002de0 con 0x7f82cc07b160 2024-09-13T08:37:45.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f82e5600640 1 -- 172.21.15.44:0/629555346 <== osd.7 v2:172.21.15.129:6810/1299717984 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f82c4002de0 con 0x7f82cc07b160 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] conn(0x7f82cc07b160 msgr2=0x7f82cc07d580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.907+0000 7f82f0600640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] conn(0x7f82cc07b160 0x7f82cc07d580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.911+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f82cc072fc0 msgr2=0x7f82cc075480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.911+0000 7f82f0600640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f82cc072fc0 0x7f82cc075480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82d8000f70 tx=0x7f82d8009040 comp rx=0 tx=0).stop 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.911+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82e81629e0 msgr2=0x7f82e8164b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.911+0000 7f82f0600640 1 --2- 172.21.15.44:0/629555346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82e81629e0 0x7f82e8164b30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82e000ea80 tx=0x7f82e000ef40 comp rx=0 tx=0).stop 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.911+0000 7f82ee600640 1 -- 172.21.15.44:0/629555346 reap_dead start 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.911+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 shutdown_connections 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.911+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 >> 172.21.15.44:0/629555346 conn(0x7f82e80d83f0 msgr2=0x7f82e81636e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.911+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 shutdown_connections 2024-09-13T08:37:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.911+0000 7f82f0600640 1 -- 172.21.15.44:0/629555346 wait complete. 2024-09-13T08:37:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.911+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 --> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47b0002de0 con 0x7f47b407b1b0 2024-09-13T08:37:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47cea00640 1 -- 172.21.15.44:0/1509174257 <== osd.6 v2:172.21.15.129:6802/2280849631 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f47b0002de0 con 0x7f47b407b1b0 2024-09-13T08:37:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] conn(0x7f47b407b1b0 msgr2=0x7f47b407d5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47ddc00640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] conn(0x7f47b407b1b0 0x7f47b407d5d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:45.928 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-09-13T08:37:45.928 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T08:37:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47b4072f90 msgr2=0x7f47b4075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47ddc00640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47b4072f90 0x7f47b4075450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47c00b0000 tx=0x7f47c001a900 comp rx=0 tx=0).stop 2024-09-13T08:37:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47d8070960 msgr2=0x7f47d809c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47ddc00640 1 --2- 172.21.15.44:0/1509174257 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47d8070960 0x7f47d809c9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c790 tx=0x7f47c800cb90 comp rx=0 tx=0).stop 2024-09-13T08:37:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47d7400640 1 -- 172.21.15.44:0/1509174257 reap_dead start 2024-09-13T08:37:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 shutdown_connections 2024-09-13T08:37:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 >> 172.21.15.44:0/1509174257 conn(0x7f47d80b18f0 msgr2=0x7f47d80b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 shutdown_connections 2024-09-13T08:37:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:45.919+0000 7f47ddc00640 1 -- 172.21.15.44:0/1509174257 wait complete. 2024-09-13T08:37:45.941 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-09-13T08:37:45.942 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-09-13T08:37:45.942 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T08:37:45.943 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T08:37:45.952 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-09-13T08:37:45.953 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T08:37:45.954 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-09-13T08:37:45.954 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T08:37:45.956 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-09-13T08:37:45.956 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T08:37:45.962 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-09-13T08:37:45.962 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T08:37:45.963 INFO:teuthology.orchestra.run.smithi044.stdout:60129542152 2024-09-13T08:37:45.963 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T08:37:46.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.139+0000 7f6248800640 1 -- 172.21.15.44:0/1056920661 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f62400d03f0 msgr2=0x7f624006e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.139+0000 7f6248800640 1 --2- 172.21.15.44:0/1056920661 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f62400d03f0 0x7f624006e6d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f624006f820 tx=0x7f623c01cae0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.139+0000 7f6248800640 1 -- 172.21.15.44:0/1056920661 shutdown_connections 2024-09-13T08:37:46.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.139+0000 7f6248800640 1 --2- 172.21.15.44:0/1056920661 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f62400d03f0 0x7f624006e6d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.139+0000 7f6248800640 1 --2- 172.21.15.44:0/1056920661 >> v2:172.21.15.129:3300/0 conn(0x7f62400b2a50 0x7f62400cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.139+0000 7f6248800640 1 --2- 172.21.15.44:0/1056920661 >> v2:172.21.15.44:3300/0 conn(0x7f624013cc20 0x7f62400b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.139+0000 7f6248800640 1 -- 172.21.15.44:0/1056920661 >> 172.21.15.44:0/1056920661 conn(0x7f624006cc20 msgr2=0x7f624006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.139+0000 7f6248800640 1 -- 172.21.15.44:0/1056920661 shutdown_connections 2024-09-13T08:37:46.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.139+0000 7f6248800640 1 -- 172.21.15.44:0/1056920661 wait complete. 2024-09-13T08:37:46.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.143+0000 7f6248800640 1 Processor -- start 2024-09-13T08:37:46.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.143+0000 7f6248800640 1 -- start start 2024-09-13T08:37:46.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6248800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62400b2a50 0x7f624013d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6248800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62400d03f0 0x7f62401fbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6248800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f624013cc20 0x7f62400ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6248800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f624015a2b0 con 0x7f62400d03f0 2024-09-13T08:37:46.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6248800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62400cfc70 con 0x7f62400b2a50 2024-09-13T08:37:46.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6248800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f624015a430 con 0x7f624013cc20 2024-09-13T08:37:46.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6246800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f624013cc20 0x7f62400ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6246800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f624013cc20 0x7f62400ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37292/0 (socket says 172.21.15.44:37292) 2024-09-13T08:37:46.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6246800640 1 -- 172.21.15.44:0/23660930 learned_addr learned my addr 172.21.15.44:0/23660930 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:46.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6245400640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62400d03f0 0x7f62401fbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6246800640 1 -- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62400b2a50 msgr2=0x7f624013d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6246800640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62400b2a50 0x7f624013d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6246800640 1 -- 172.21.15.44:0/23660930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62400d03f0 msgr2=0x7f62401fbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6246800640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62400d03f0 0x7f62401fbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6246800640 1 -- 172.21.15.44:0/23660930 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624013d790 con 0x7f624013cc20 2024-09-13T08:37:46.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6245400640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62400d03f0 0x7f62401fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:46.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6246800640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f624013cc20 0x7f62400ce950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f623c01b190 tx=0x7f623c009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f623a000640 1 -- 172.21.15.44:0/23660930 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c0bb070 con 0x7f624013cc20 2024-09-13T08:37:46.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f623a000640 1 -- 172.21.15.44:0/23660930 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f623c004d40 con 0x7f624013cc20 2024-09-13T08:37:46.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f623a000640 1 -- 172.21.15.44:0/23660930 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c0bb070 con 0x7f624013cc20 2024-09-13T08:37:46.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.147+0000 7f6248800640 1 -- 172.21.15.44:0/23660930 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62401fc1c0 con 0x7f624013cc20 2024-09-13T08:37:46.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.151+0000 7f6248800640 1 -- 172.21.15.44:0/23660930 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62401fc740 con 0x7f624013cc20 2024-09-13T08:37:46.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.155+0000 7f623a000640 1 -- 172.21.15.44:0/23660930 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f623c004000 con 0x7f624013cc20 2024-09-13T08:37:46.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.155+0000 7f622f400640 1 -- 172.21.15.44:0/23660930 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624006e6d0 con 0x7f624013cc20 2024-09-13T08:37:46.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.155+0000 7f623a000640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62240730c0 0x7f6224075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.155+0000 7f623a000640 1 -- 172.21.15.44:0/23660930 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f623c12c130 con 0x7f624013cc20 2024-09-13T08:37:46.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.155+0000 7f24e4000640 1 -- 172.21.15.44:0/799376029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc06fa40 msgr2=0x7f24dc151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.155+0000 7f24e4000640 1 --2- 172.21.15.44:0/799376029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc06fa40 0x7f24dc151430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f24d000d930 tx=0x7f24d001e780 comp rx=0 tx=0).stop 2024-09-13T08:37:46.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f24e4000640 1 -- 172.21.15.44:0/799376029 shutdown_connections 2024-09-13T08:37:46.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f24e4000640 1 --2- 172.21.15.44:0/799376029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc06fa40 0x7f24dc151430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f24e4000640 1 -- 172.21.15.44:0/799376029 >> 172.21.15.44:0/799376029 conn(0x7f24dc06cc20 msgr2=0x7f24dc06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f6245e00640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62240730c0 0x7f6224075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f24e4000640 1 -- 172.21.15.44:0/799376029 shutdown_connections 2024-09-13T08:37:46.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f24e4000640 1 -- 172.21.15.44:0/799376029 wait complete. 2024-09-13T08:37:46.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f1910e00640 1 -- 172.21.15.44:0/360293357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18f0004e70 msgr2=0x7f18f000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f1910e00640 1 --2- 172.21.15.44:0/360293357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18f0004e70 0x7f18f000b9d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c2e0 tx=0x7f18f801d010 comp rx=0 tx=0).stop 2024-09-13T08:37:46.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f190a800640 1 -- 172.21.15.44:0/360293357 reap_dead start 2024-09-13T08:37:46.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f6245e00640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62240730c0 0x7f6224075580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f62340058d0 tx=0x7f6234005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f1910e00640 1 -- 172.21.15.44:0/360293357 shutdown_connections 2024-09-13T08:37:46.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f1910e00640 1 -- 172.21.15.44:0/360293357 >> 172.21.15.44:0/360293357 conn(0x7f190c06cc20 msgr2=0x7f190c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f1910e00640 1 -- 172.21.15.44:0/360293357 shutdown_connections 2024-09-13T08:37:46.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f1910e00640 1 -- 172.21.15.44:0/360293357 wait complete. 2024-09-13T08:37:46.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f24e4000640 1 Processor -- start 2024-09-13T08:37:46.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f24e4000640 1 -- start start 2024-09-13T08:37:46.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f24e4000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24dc06fa40 0x7f24dc1404d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.159+0000 7f24e4000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24dc06fe20 0x7f24dc14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e4000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc15bcf0 0x7f24dc14c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e4000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc167aa0 con 0x7f24dc15bcf0 2024-09-13T08:37:46.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e4000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc167920 con 0x7f24dc06fe20 2024-09-13T08:37:46.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e4000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f24dc167c20 con 0x7f24dc06fa40 2024-09-13T08:37:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e1600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24dc06fa40 0x7f24dc1404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc15bcf0 0x7f24dc14c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc15bcf0 0x7f24dc14c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36468/0 (socket says 172.21.15.44:36468) 2024-09-13T08:37:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e2000640 1 -- 172.21.15.44:0/1413675540 learned_addr learned my addr 172.21.15.44:0/1413675540 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e1600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24dc06fa40 0x7f24dc1404d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37316/0 (socket says 172.21.15.44:37316) 2024-09-13T08:37:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e0c00640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24dc06fe20 0x7f24dc14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1910e00640 1 Processor -- start 2024-09-13T08:37:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f623a000640 1 -- 172.21.15.44:0/23660930 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f623c016990 con 0x7f624013cc20 2024-09-13T08:37:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e2000640 1 -- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24dc06fa40 msgr2=0x7f24dc1404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e2000640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24dc06fa40 0x7f24dc1404d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e2000640 1 -- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24dc06fe20 msgr2=0x7f24dc14c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e2000640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24dc06fe20 0x7f24dc14c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e2000640 1 -- 172.21.15.44:0/1413675540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc0cded0 con 0x7f24dc15bcf0 2024-09-13T08:37:46.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1910e00640 1 -- start start 2024-09-13T08:37:46.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1910e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f18f0004e70 0x7f190c0af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1910e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f190c1f4000 0x7f190c1f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1910e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f190c1f7f10 0x7f190c13a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1910e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f190c15ed40 con 0x7f190c1f4000 2024-09-13T08:37:46.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1910e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f190c15f480 con 0x7f190c1f7f10 2024-09-13T08:37:46.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1910e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f190c15eec0 con 0x7f18f0004e70 2024-09-13T08:37:46.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e2000640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc15bcf0 0x7f24dc14c930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24c800c3c0 tx=0x7f24c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24d9600640 1 -- 172.21.15.44:0/1413675540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8011790 con 0x7f24dc15bcf0 2024-09-13T08:37:46.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24d9600640 1 -- 172.21.15.44:0/1413675540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24c8017070 con 0x7f24dc15bcf0 2024-09-13T08:37:46.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e4000640 1 -- 172.21.15.44:0/1413675540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc0ce1b0 con 0x7f24dc15bcf0 2024-09-13T08:37:46.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24e4000640 1 -- 172.21.15.44:0/1413675540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc0ce770 con 0x7f24dc15bcf0 2024-09-13T08:37:46.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f24d9600640 1 -- 172.21.15.44:0/1413675540 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8011790 con 0x7f24dc15bcf0 2024-09-13T08:37:46.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1909e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f18f0004e70 0x7f190c0af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1909e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f18f0004e70 0x7f190c0af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37322/0 (socket says 172.21.15.44:37322) 2024-09-13T08:37:46.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.163+0000 7f1909e00640 1 -- 172.21.15.44:0/298874047 learned_addr learned my addr 172.21.15.44:0/298874047 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:46.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.167+0000 7f190a800640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f190c1f7f10 0x7f190c13a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.167+0000 7f1909e00640 1 -- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f190c1f7f10 msgr2=0x7f190c13a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.167+0000 7f1909e00640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f190c1f7f10 0x7f190c13a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.167+0000 7f1909e00640 1 -- 172.21.15.44:0/298874047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f190c1f4000 msgr2=0x7f190c1f79d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T08:37:46.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.167+0000 7f24e4000640 1 -- 172.21.15.44:0/1413675540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc15e5f0 con 0x7f24dc15bcf0 2024-09-13T08:37:46.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.167+0000 7f1909400640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f190c1f4000 0x7f190c1f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.167+0000 7f1909e00640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f190c1f4000 0x7f190c1f79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.167+0000 7f1909e00640 1 -- 172.21.15.44:0/298874047 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c15fe40 con 0x7f18f0004e70 2024-09-13T08:37:46.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.167+0000 7f190a800640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f190c1f7f10 0x7f190c13a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:37:46.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.167+0000 7f1909e00640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f18f0004e70 0x7f190c0af4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f190400cef0 tx=0x7f190400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.171+0000 7f1902000640 1 -- 172.21.15.44:0/298874047 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190400f860 con 0x7f18f0004e70 2024-09-13T08:37:46.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.171+0000 7f1902000640 1 -- 172.21.15.44:0/298874047 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f190400ae50 con 0x7f18f0004e70 2024-09-13T08:37:46.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.171+0000 7f1902000640 1 -- 172.21.15.44:0/298874047 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190400fd90 con 0x7f18f0004e70 2024-09-13T08:37:46.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.171+0000 7f1910e00640 1 -- 172.21.15.44:0/298874047 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c0b1ba0 con 0x7f18f0004e70 2024-09-13T08:37:46.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.171+0000 7f1910e00640 1 -- 172.21.15.44:0/298874047 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c0b1e40 con 0x7f18f0004e70 2024-09-13T08:37:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.171+0000 7f24d9600640 1 -- 172.21.15.44:0/1413675540 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f24c8011a60 con 0x7f24dc15bcf0 2024-09-13T08:37:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.175+0000 7f24d9600640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24bc0730c0 0x7f24bc075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.175+0000 7f24d9600640 1 -- 172.21.15.44:0/1413675540 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f24c80940e0 con 0x7f24dc15bcf0 2024-09-13T08:37:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.179+0000 7f24e1600640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24bc0730c0 0x7f24bc075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.175+0000 7f1910e00640 1 -- 172.21.15.44:0/298874047 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e40052a0 con 0x7f18f0004e70 2024-09-13T08:37:46.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.175+0000 7f1902000640 1 -- 172.21.15.44:0/298874047 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f1904018400 con 0x7f18f0004e70 2024-09-13T08:37:46.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.179+0000 7f1902000640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f18ec072f90 0x7f18ec075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.179+0000 7f1902000640 1 -- 172.21.15.44:0/298874047 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1904094c90 con 0x7f18f0004e70 2024-09-13T08:37:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.179+0000 7f24e1600640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24bc0730c0 0x7f24bc075580 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f24d00b1000 tx=0x7f24d001cfc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.179+0000 7f1909400640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f18ec072f90 0x7f18ec075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.179+0000 7f1909400640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f18ec072f90 0x7f18ec075450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f190c0d0160 tx=0x7f18fc00a250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.179+0000 7f24d9600640 1 -- 172.21.15.44:0/1413675540 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f24c805ecd0 con 0x7f24dc15bcf0 2024-09-13T08:37:46.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.187+0000 7f1902000640 1 -- 172.21.15.44:0/298874047 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1904014030 con 0x7f18f0004e70 2024-09-13T08:37:46.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.247+0000 7f2dfee00640 1 -- 172.21.15.44:0/1728316212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2df80d03f0 msgr2=0x7f2df806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.247+0000 7f2df6a00640 1 -- 172.21.15.44:0/1728316212 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df00b2070 con 0x7f2df80d03f0 2024-09-13T08:37:46.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.247+0000 7f2dfee00640 1 --2- 172.21.15.44:0/1728316212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2df80d03f0 0x7f2df806e6d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2df80d0a70 tx=0x7f2df001cd80 comp rx=0 tx=0).stop 2024-09-13T08:37:46.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.251+0000 7f2dfee00640 1 -- 172.21.15.44:0/1728316212 shutdown_connections 2024-09-13T08:37:46.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.251+0000 7f2dfee00640 1 --2- 172.21.15.44:0/1728316212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2df80d03f0 0x7f2df806e6d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.251+0000 7f2dfee00640 1 --2- 172.21.15.44:0/1728316212 >> v2:172.21.15.44:3300/0 conn(0x7f2df80b2a50 0x7f2df80cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.251+0000 7f2dfee00640 1 --2- 172.21.15.44:0/1728316212 >> v2:172.21.15.129:3300/0 conn(0x7f2df813cc20 0x7f2df80b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.251+0000 7f2dfee00640 1 -- 172.21.15.44:0/1728316212 >> 172.21.15.44:0/1728316212 conn(0x7f2df806cc20 msgr2=0x7f2df806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.251+0000 7f2dfee00640 1 -- 172.21.15.44:0/1728316212 shutdown_connections 2024-09-13T08:37:46.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.251+0000 7f2dfee00640 1 -- 172.21.15.44:0/1728316212 wait complete. 2024-09-13T08:37:46.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.255+0000 7f2dfee00640 1 Processor -- start 2024-09-13T08:37:46.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.255+0000 7f2dfee00640 1 -- start start 2024-09-13T08:37:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2dfee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df80b2a50 0x7f2df815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2dfee00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2df80d03f0 0x7f2df8136140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2dfee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2df813cc20 0x7f2df815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2dfee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2df815a640 con 0x7f2df80b2a50 2024-09-13T08:37:46.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2dfee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2df815a4c0 con 0x7f2df813cc20 2024-09-13T08:37:46.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2dfee00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2df815a7c0 con 0x7f2df80d03f0 2024-09-13T08:37:46.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2df7400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2df80d03f0 0x7f2df8136140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2df7400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2df80d03f0 0x7f2df8136140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37358/0 (socket says 172.21.15.44:37358) 2024-09-13T08:37:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2df7400640 1 -- 172.21.15.44:0/1528898218 learned_addr learned my addr 172.21.15.44:0/1528898218 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2dfce00640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2df813cc20 0x7f2df815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2df7e00640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df80b2a50 0x7f2df815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2df7e00640 1 -- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2df80d03f0 msgr2=0x7f2df8136140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2df7e00640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2df80d03f0 0x7f2df8136140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2df7e00640 1 -- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2df813cc20 msgr2=0x7f2df815bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2df7e00640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2df813cc20 0x7f2df815bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.259+0000 7f2df7e00640 1 -- 172.21.15.44:0/1528898218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df815d2f0 con 0x7f2df80b2a50 2024-09-13T08:37:46.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.263+0000 7f2dfce00640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2df813cc20 0x7f2df815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:46.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.263+0000 7f2df7e00640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df80b2a50 0x7f2df815cdb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2de40094d0 tx=0x7f2de4009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.263+0000 7f2defe00640 1 -- 172.21.15.44:0/1528898218 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4018030 con 0x7f2df80b2a50 2024-09-13T08:37:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.263+0000 7f2defe00640 1 -- 172.21.15.44:0/1528898218 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2de4004130 con 0x7f2df80b2a50 2024-09-13T08:37:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.263+0000 7f2defe00640 1 -- 172.21.15.44:0/1528898218 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4005020 con 0x7f2df80b2a50 2024-09-13T08:37:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f2dfee00640 1 -- 172.21.15.44:0/1528898218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df80af830 con 0x7f2df80b2a50 2024-09-13T08:37:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f2dfee00640 1 -- 172.21.15.44:0/1528898218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df80afdf0 con 0x7f2df80b2a50 2024-09-13T08:37:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f1281c00640 1 -- 172.21.15.44:0/799281991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1274057d90 msgr2=0x7f12740e8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f1281c00640 1 --2- 172.21.15.44:0/799281991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1274057d90 0x7f12740e8240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f127c07af00 tx=0x7f127c0985e0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f1281c00640 1 -- 172.21.15.44:0/799281991 shutdown_connections 2024-09-13T08:37:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f1281c00640 1 --2- 172.21.15.44:0/799281991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1274057d90 0x7f12740e8240 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f1281c00640 1 --2- 172.21.15.44:0/799281991 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f127403ef50 0x7f12740e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f1281c00640 1 -- 172.21.15.44:0/799281991 >> 172.21.15.44:0/799281991 conn(0x7f127405cdc0 msgr2=0x7f127405d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f1281c00640 1 -- 172.21.15.44:0/799281991 shutdown_connections 2024-09-13T08:37:46.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f1281c00640 1 -- 172.21.15.44:0/799281991 wait complete. 2024-09-13T08:37:46.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.267+0000 7f2ded600640 1 -- 172.21.15.44:0/1528898218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df806e6d0 con 0x7f2df80b2a50 2024-09-13T08:37:46.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7ff55fe00640 1 -- 172.21.15.44:0/2954061896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff560146ed0 msgr2=0x7ff5600cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7ff55fe00640 1 --2- 172.21.15.44:0/2954061896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff560146ed0 0x7ff5600cdb40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5600aee10 tx=0x7ff54c01c770 comp rx=0 tx=0).stop 2024-09-13T08:37:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f1281c00640 1 Processor -- start 2024-09-13T08:37:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7ff55fe00640 1 -- 172.21.15.44:0/2954061896 shutdown_connections 2024-09-13T08:37:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7ff55fe00640 1 --2- 172.21.15.44:0/2954061896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff560146ed0 0x7ff5600cdb40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7ff55fe00640 1 --2- 172.21.15.44:0/2954061896 >> v2:172.21.15.44:3300/0 conn(0x7ff560072680 0x7ff5600cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7ff55fe00640 1 --2- 172.21.15.44:0/2954061896 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5600744b0 0x7ff560162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7ff55fe00640 1 -- 172.21.15.44:0/2954061896 >> 172.21.15.44:0/2954061896 conn(0x7ff5600b1600 msgr2=0x7ff5600b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7ff55fe00640 1 -- 172.21.15.44:0/2954061896 shutdown_connections 2024-09-13T08:37:46.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f1281c00640 1 -- start start 2024-09-13T08:37:46.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f2defe00640 1 -- 172.21.15.44:0/1528898218 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f2de40051c0 con 0x7f2df80b2a50 2024-09-13T08:37:46.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f1281c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f127403ef50 0x7f12740d9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f1281c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1274057d90 0x7f12740d9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f1281c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12740ddbe0 0x7f12740e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f1281c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12740f3a30 con 0x7f127403ef50 2024-09-13T08:37:46.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7ff55fe00640 1 -- 172.21.15.44:0/2954061896 wait complete. 2024-09-13T08:37:46.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f2defe00640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dd8072f90 0x7f2dd8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7ff55fe00640 1 Processor -- start 2024-09-13T08:37:46.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f127be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f127403ef50 0x7f12740d9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f127be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f127403ef50 0x7f12740d9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36532/0 (socket says 172.21.15.44:36532) 2024-09-13T08:37:46.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f2defe00640 1 -- 172.21.15.44:0/1528898218 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2de4093af0 con 0x7f2df80b2a50 2024-09-13T08:37:46.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f127be00640 1 -- 172.21.15.44:0/1767447771 learned_addr learned my addr 172.21.15.44:0/1767447771 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:46.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.271+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f12740f38b0 con 0x7f12740ddbe0 2024-09-13T08:37:46.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f12740f3bb0 con 0x7f1274057d90 2024-09-13T08:37:46.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f2df7400640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dd8072f90 0x7f2dd8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55fe00640 1 -- start start 2024-09-13T08:37:46.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f1281200640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12740ddbe0 0x7f12740e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f127be00640 1 -- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1274057d90 msgr2=0x7f12740d9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f127be00640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1274057d90 0x7f12740d9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f127be00640 1 -- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12740ddbe0 msgr2=0x7f12740e4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f127be00640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12740ddbe0 0x7f12740e4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f127be00640 1 -- 172.21.15.44:0/1767447771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127403e720 con 0x7f127403ef50 2024-09-13T08:37:46.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f1281200640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12740ddbe0 0x7f12740e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:37:46.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f2df7400640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dd8072f90 0x7f2dd8075450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2df81543e0 tx=0x7f2de800b440 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7f127be00640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f127403ef50 0x7f12740d9770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f127000b3e0 tx=0x7f127000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff560072680 0x7ff5601608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5600744b0 0x7ff560204b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560146ed0 0x7ff5600acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff5600703d0 con 0x7ff560146ed0 2024-09-13T08:37:46.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff560070250 con 0x7ff5600744b0 2024-09-13T08:37:46.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff5600abcf0 con 0x7ff560072680 2024-09-13T08:37:46.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55ea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff560072680 0x7ff5601608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55ea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff560072680 0x7ff5601608d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37378/0 (socket says 172.21.15.44:37378) 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55ea00640 1 -- 172.21.15.44:0/857902958 learned_addr learned my addr 172.21.15.44:0/857902958 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55e000640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5600744b0 0x7ff560204b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55f400640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560146ed0 0x7ff5600acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55ea00640 1 -- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5600744b0 msgr2=0x7ff560204b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55ea00640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5600744b0 0x7ff560204b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55ea00640 1 -- 172.21.15.44:0/857902958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560146ed0 msgr2=0x7ff5600acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55ea00640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560146ed0 0x7ff5600acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55ea00640 1 -- 172.21.15.44:0/857902958 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5600abfd0 con 0x7ff560072680 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55f400640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560146ed0 0x7ff5600acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55e000640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5600744b0 0x7ff560204b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.275+0000 7ff55ea00640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff560072680 0x7ff5601608d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c970 tx=0x7ff55000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7ff556a00640 1 -- 172.21.15.44:0/857902958 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550007bb0 con 0x7ff560072680 2024-09-13T08:37:46.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7ff556a00640 1 -- 172.21.15.44:0/857902958 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff550007e40 con 0x7ff560072680 2024-09-13T08:37:46.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7ff55fe00640 1 -- 172.21.15.44:0/857902958 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5600ac310 con 0x7ff560072680 2024-09-13T08:37:46.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7ff55fe00640 1 -- 172.21.15.44:0/857902958 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff560160e10 con 0x7ff560072680 2024-09-13T08:37:46.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7ff556a00640 1 -- 172.21.15.44:0/857902958 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550005260 con 0x7ff560072680 2024-09-13T08:37:46.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7ff55fe00640 1 -- 172.21.15.44:0/857902958 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff560162250 con 0x7ff560072680 2024-09-13T08:37:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7f126fe00640 1 -- 172.21.15.44:0/1767447771 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270015020 con 0x7f127403ef50 2024-09-13T08:37:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7f126fe00640 1 -- 172.21.15.44:0/1767447771 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1270011940 con 0x7f127403ef50 2024-09-13T08:37:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7f126fe00640 1 -- 172.21.15.44:0/1767447771 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270011d90 con 0x7f127403ef50 2024-09-13T08:37:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.283+0000 7f2defe00640 1 -- 172.21.15.44:0/1528898218 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2de4011070 con 0x7f2df80b2a50 2024-09-13T08:37:46.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12740e5bf0 con 0x7f127403ef50 2024-09-13T08:37:46.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.279+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127403e360 con 0x7f127403ef50 2024-09-13T08:37:46.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.283+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12740e7bb0 con 0x7f127403ef50 2024-09-13T08:37:46.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.283+0000 7ff556a00640 1 -- 172.21.15.44:0/857902958 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7ff550005500 con 0x7ff560072680 2024-09-13T08:37:46.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.283+0000 7ff556a00640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff540072f90 0x7ff540075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.287+0000 7ff556a00640 1 -- 172.21.15.44:0/857902958 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff550094680 con 0x7ff560072680 2024-09-13T08:37:46.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.287+0000 7f126fe00640 1 -- 172.21.15.44:0/1767447771 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f1270018020 con 0x7f127403ef50 2024-09-13T08:37:46.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.291+0000 7ff55e000640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff540072f90 0x7ff540075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.291+0000 7ff556a00640 1 -- 172.21.15.44:0/857902958 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff5600ac310 con 0x7ff560072680 2024-09-13T08:37:46.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.291+0000 7ff55e000640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff540072f90 0x7ff540075450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5600cd0f0 tx=0x7ff558009500 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.291+0000 7f126fe00640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f125c072f90 0x7f125c075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.291+0000 7f126fe00640 1 -- 172.21.15.44:0/1767447771 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1270094830 con 0x7f127403ef50 2024-09-13T08:37:46.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.291+0000 7f127b400640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f125c072f90 0x7f125c075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.295+0000 7f126fe00640 1 -- 172.21.15.44:0/1767447771 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f127001f020 con 0x7f127403ef50 2024-09-13T08:37:46.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.295+0000 7f127b400640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f125c072f90 0x7f125c075450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12680058d0 tx=0x7f1268005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.327+0000 7f6e93800640 1 -- 172.21.15.44:0/2051830577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c06e740 msgr2=0x7f6e8c06eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.327+0000 7f6e93800640 1 --2- 172.21.15.44:0/2051830577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c06e740 0x7f6e8c06eb20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009900 tx=0x7f6e7c01c8d0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.331+0000 7f6e93800640 1 -- 172.21.15.44:0/2051830577 shutdown_connections 2024-09-13T08:37:46.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.331+0000 7f6e93800640 1 --2- 172.21.15.44:0/2051830577 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e8c0b11d0 0x7f6e8c161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.331+0000 7f6e93800640 1 --2- 172.21.15.44:0/2051830577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c06e740 0x7f6e8c06eb20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.331+0000 7f6e93800640 1 -- 172.21.15.44:0/2051830577 >> 172.21.15.44:0/2051830577 conn(0x7f6e8c0d1dc0 msgr2=0x7f6e8c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.331+0000 7f6e93800640 1 -- 172.21.15.44:0/2051830577 shutdown_connections 2024-09-13T08:37:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.331+0000 7f6e93800640 1 -- 172.21.15.44:0/2051830577 wait complete. 2024-09-13T08:37:46.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e93800640 1 Processor -- start 2024-09-13T08:37:46.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e93800640 1 -- start start 2024-09-13T08:37:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e93800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e8c06e740 0x7f6e8c0c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e93800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e8c0b11d0 0x7f6e8c15cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e93800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c162200 0x7f6e8c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e93800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c0b0e00 con 0x7f6e8c06e740 2024-09-13T08:37:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e93800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c0b0c80 con 0x7f6e8c162200 2024-09-13T08:37:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e93800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8c0d1a30 con 0x7f6e8c0b11d0 2024-09-13T08:37:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e90e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e8c06e740 0x7f6e8c0c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e91800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c162200 0x7f6e8c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e91800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c162200 0x7f6e8c0b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:52672/0 (socket says 172.21.15.44:52672) 2024-09-13T08:37:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e91800640 1 -- 172.21.15.44:0/3677555636 learned_addr learned my addr 172.21.15.44:0/3677555636 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.335+0000 7f6e8be00640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e8c0b11d0 0x7f6e8c15cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e90e00640 1 -- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e8c0b11d0 msgr2=0x7f6e8c15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e90e00640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e8c0b11d0 0x7f6e8c15cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e90e00640 1 -- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c162200 msgr2=0x7f6e8c0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e90e00640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c162200 0x7f6e8c0b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e90e00640 1 -- 172.21.15.44:0/3677555636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c0b4d90 con 0x7f6e8c06e740 2024-09-13T08:37:46.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e90e00640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e8c06e740 0x7f6e8c0c62e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0ac000 tx=0x7f6e7c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7fc214c00640 1 -- 172.21.15.44:0/185323907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc2080341b0 msgr2=0x7fc208035ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7fc20e000640 1 -- 172.21.15.44:0/185323907 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2040ad070 con 0x7fc2080341b0 2024-09-13T08:37:46.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7fc214c00640 1 --2- 172.21.15.44:0/185323907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc2080341b0 0x7fc208035ef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2040098f0 tx=0x7fc20401c820 comp rx=0 tx=0).stop 2024-09-13T08:37:46.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e88c00640 1 -- 172.21.15.44:0/3677555636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0b2040 con 0x7f6e8c06e740 2024-09-13T08:37:46.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e88c00640 1 -- 172.21.15.44:0/3677555636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e7c005270 con 0x7f6e8c06e740 2024-09-13T08:37:46.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e88c00640 1 -- 172.21.15.44:0/3677555636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0046d0 con 0x7f6e8c06e740 2024-09-13T08:37:46.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7fc214c00640 1 -- 172.21.15.44:0/185323907 shutdown_connections 2024-09-13T08:37:46.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7fc214c00640 1 --2- 172.21.15.44:0/185323907 >> v2:172.21.15.44:3300/0 conn(0x7fc208051860 0x7fc208051d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e93800640 1 -- 172.21.15.44:0/3677555636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8c0b5070 con 0x7f6e8c06e740 2024-09-13T08:37:46.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7fc214c00640 1 --2- 172.21.15.44:0/185323907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc2080341b0 0x7fc208035ef0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7f6e93800640 1 -- 172.21.15.44:0/3677555636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8c0b5630 con 0x7f6e8c06e740 2024-09-13T08:37:46.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7fc214c00640 1 -- 172.21.15.44:0/185323907 >> 172.21.15.44:0/185323907 conn(0x7fc20801e060 msgr2=0x7fc208054e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7fc214c00640 1 -- 172.21.15.44:0/185323907 shutdown_connections 2024-09-13T08:37:46.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.339+0000 7fc214c00640 1 -- 172.21.15.44:0/185323907 wait complete. 2024-09-13T08:37:46.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.343+0000 7fc214c00640 1 Processor -- start 2024-09-13T08:37:46.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.343+0000 7fc214c00640 1 -- start start 2024-09-13T08:37:46.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.343+0000 7fc214c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2080341b0 0x7fc20804b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.343+0000 7f6e93800640 1 -- 172.21.15.44:0/3677555636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8c161cc0 con 0x7f6e8c06e740 2024-09-13T08:37:46.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.343+0000 7f6e88c00640 1 -- 172.21.15.44:0/3677555636 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f6e7c0a58d0 con 0x7f6e8c06e740 2024-09-13T08:37:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.343+0000 7fc214c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc208034f90 0x7fc20817f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.343+0000 7fc214c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc208051860 0x7fc20817fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.343+0000 7fc214c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2080ed7d0 con 0x7fc2080341b0 2024-09-13T08:37:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.343+0000 7fc214c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2080ed650 con 0x7fc208051860 2024-09-13T08:37:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.343+0000 7fc214c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc2080ed950 con 0x7fc208034f90 2024-09-13T08:37:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc20ea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc208034f90 0x7fc20817f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7f6e88c00640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e6c072f40 0x7f6e6c075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7f6e8be00640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e6c072f40 0x7f6e6c075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc20ea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc208034f90 0x7fc20817f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37408/0 (socket says 172.21.15.44:37408) 2024-09-13T08:37:46.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc20ea00640 1 -- 172.21.15.44:0/4065799221 learned_addr learned my addr 172.21.15.44:0/4065799221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:46.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc20fe00640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc208051860 0x7fc20817fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7f6e88c00640 1 -- 172.21.15.44:0/3677555636 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6e7c0f4b10 con 0x7f6e8c06e740 2024-09-13T08:37:46.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc20fe00640 1 -- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc208034f90 msgr2=0x7fc20817f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc20fe00640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc208034f90 0x7fc20817f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc20fe00640 1 -- 172.21.15.44:0/4065799221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2080341b0 msgr2=0x7fc20804b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc20fe00640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2080341b0 0x7fc20804b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc20fe00640 1 -- 172.21.15.44:0/4065799221 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2080dd830 con 0x7fc208051860 2024-09-13T08:37:46.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc20fe00640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc208051860 0x7fc20817fa80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc2100773e0 tx=0x7fc210069d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7f6e8be00640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e6c072f40 0x7f6e6c075400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0d2ec0 tx=0x7f6e780091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc203400640 1 -- 172.21.15.44:0/4065799221 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100748e0 con 0x7fc208051860 2024-09-13T08:37:46.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc203400640 1 -- 172.21.15.44:0/4065799221 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc21008f070 con 0x7fc208051860 2024-09-13T08:37:46.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc203400640 1 -- 172.21.15.44:0/4065799221 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210074df0 con 0x7fc208051860 2024-09-13T08:37:46.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc214c00640 1 -- 172.21.15.44:0/4065799221 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20804de30 con 0x7fc208051860 2024-09-13T08:37:46.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc214c00640 1 -- 172.21.15.44:0/4065799221 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20804e3f0 con 0x7fc208051860 2024-09-13T08:37:46.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.347+0000 7fc214c00640 1 -- 172.21.15.44:0/4065799221 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2080dda00 con 0x7fc208051860 2024-09-13T08:37:46.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.351+0000 7fc203400640 1 -- 172.21.15.44:0/4065799221 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fc210094470 con 0x7fc208051860 2024-09-13T08:37:46.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.351+0000 7fc203400640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc1f0072ec0 0x7fc1f0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:46.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.351+0000 7fc203400640 1 -- 172.21.15.44:0/4065799221 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc21010e3d0 con 0x7fc208051860 2024-09-13T08:37:46.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.355+0000 7fc20f400640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc1f0072ec0 0x7fc1f0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:46.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.355+0000 7fc20f400640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc1f0072ec0 0x7fc1f0075380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2040af000 tx=0x7fc20400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:46.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.355+0000 7f6e88c00640 1 -- 172.21.15.44:0/3677555636 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6e7c0164f0 con 0x7f6e8c06e740 2024-09-13T08:37:46.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.359+0000 7fc203400640 1 -- 172.21.15.44:0/4065799221 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc2100dafe0 con 0x7fc208051860 2024-09-13T08:37:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.543+0000 7f622f400640 1 -- 172.21.15.44:0/23660930 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f62400cee90 con 0x7f624013cc20 2024-09-13T08:37:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f24e4000640 1 -- 172.21.15.44:0/1413675540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f24dc1506b0 con 0x7f24dc15bcf0 2024-09-13T08:37:46.554 INFO:teuthology.orchestra.run.smithi044.stdout:64424509445 2024-09-13T08:37:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f623a000640 1 -- 172.21.15.44:0/23660930 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f623c0f8dd0 con 0x7f624013cc20 2024-09-13T08:37:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f24d9600640 1 -- 172.21.15.44:0/1413675540 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f24c8063700 con 0x7f24dc15bcf0 2024-09-13T08:37:46.556 INFO:teuthology.orchestra.run.smithi044.stdout:64424509445 2024-09-13T08:37:46.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f622f400640 1 -- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62240730c0 msgr2=0x7f6224075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f622f400640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62240730c0 0x7f6224075580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f62340058d0 tx=0x7f6234005860 comp rx=0 tx=0).stop 2024-09-13T08:37:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f622f400640 1 -- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f624013cc20 msgr2=0x7f62400ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f622f400640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f624013cc20 0x7f62400ce950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f623c01b190 tx=0x7f623c009300 comp rx=0 tx=0).stop 2024-09-13T08:37:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f622f400640 1 -- 172.21.15.44:0/23660930 shutdown_connections 2024-09-13T08:37:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f622f400640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62240730c0 0x7f6224075580 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f622f400640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f624013cc20 0x7f62400ce950 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f622f400640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62400d03f0 0x7f62401fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f622f400640 1 --2- 172.21.15.44:0/23660930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62400b2a50 0x7f624013d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.551+0000 7f622f400640 1 -- 172.21.15.44:0/23660930 >> 172.21.15.44:0/23660930 conn(0x7f624006cc20 msgr2=0x7f6240150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f622f400640 1 -- 172.21.15.44:0/23660930 shutdown_connections 2024-09-13T08:37:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f622f400640 1 -- 172.21.15.44:0/23660930 wait complete. 2024-09-13T08:37:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 -- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24bc0730c0 msgr2=0x7f24bc075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24bc0730c0 0x7f24bc075580 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f24d00b1000 tx=0x7f24d001cfc0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 -- 172.21.15.44:0/1413675540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc15bcf0 msgr2=0x7f24dc14c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc15bcf0 0x7f24dc14c930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24c800c3c0 tx=0x7f24c800c880 comp rx=0 tx=0).stop 2024-09-13T08:37:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 -- 172.21.15.44:0/1413675540 shutdown_connections 2024-09-13T08:37:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24bc0730c0 0x7f24bc075580 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc15bcf0 0x7f24dc14c930 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24dc06fe20 0x7f24dc14c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 --2- 172.21.15.44:0/1413675540 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24dc06fa40 0x7f24dc1404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 -- 172.21.15.44:0/1413675540 >> 172.21.15.44:0/1413675540 conn(0x7f24dc06cc20 msgr2=0x7f24dc1508d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 -- 172.21.15.44:0/1413675540 shutdown_connections 2024-09-13T08:37:46.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.555+0000 7f24e4000640 1 -- 172.21.15.44:0/1413675540 wait complete. 2024-09-13T08:37:46.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.559+0000 7f1910e00640 1 -- 172.21.15.44:0/298874047 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f18e4005590 con 0x7f18f0004e70 2024-09-13T08:37:46.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.559+0000 7f1902000640 1 -- 172.21.15.44:0/298874047 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f190405ff50 con 0x7f18f0004e70 2024-09-13T08:37:46.569 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-09-13T08:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 -- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f18ec072f90 msgr2=0x7f18ec075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f18ec072f90 0x7f18ec075450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f190c0d0160 tx=0x7f18fc00a250 comp rx=0 tx=0).stop 2024-09-13T08:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 -- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f18f0004e70 msgr2=0x7f190c0af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f18f0004e70 0x7f190c0af4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f190400cef0 tx=0x7f190400a620 comp rx=0 tx=0).stop 2024-09-13T08:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 -- 172.21.15.44:0/298874047 shutdown_connections 2024-09-13T08:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f18ec072f90 0x7f18ec075450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f190c1f7f10 0x7f190c13a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f190c1f4000 0x7f190c1f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 --2- 172.21.15.44:0/298874047 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f18f0004e70 0x7f190c0af4e0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f190400cef0 tx=0x7f190400a620 comp rx=0 tx=0).stop 2024-09-13T08:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 -- 172.21.15.44:0/298874047 >> 172.21.15.44:0/298874047 conn(0x7f190c06cc20 msgr2=0x7f190c0b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 -- 172.21.15.44:0/298874047 shutdown_connections 2024-09-13T08:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.567+0000 7f1910e00640 1 -- 172.21.15.44:0/298874047 wait complete. 2024-09-13T08:37:46.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.0 2024-09-13T08:37:46.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-09-13T08:37:46.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.1 2024-09-13T08:37:46.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.623+0000 7ff55fe00640 1 -- 172.21.15.44:0/857902958 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff5600acab0 con 0x7ff560072680 2024-09-13T08:37:46.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.623+0000 7ff556a00640 1 -- 172.21.15.44:0/857902958 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff550064e20 con 0x7ff560072680 2024-09-13T08:37:46.629 INFO:teuthology.orchestra.run.smithi044.stdout:64424509445 2024-09-13T08:37:46.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 -- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff540072f90 msgr2=0x7ff540075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff540072f90 0x7ff540075450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5600cd0f0 tx=0x7ff558009500 comp rx=0 tx=0).stop 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 -- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff560072680 msgr2=0x7ff5601608d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff560072680 0x7ff5601608d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c970 tx=0x7ff55000ce30 comp rx=0 tx=0).stop 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 -- 172.21.15.44:0/857902958 shutdown_connections 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff540072f90 0x7ff540075450 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560146ed0 0x7ff5600acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5600744b0 0x7ff560204b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 --2- 172.21.15.44:0/857902958 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff560072680 0x7ff5601608d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 -- 172.21.15.44:0/857902958 >> 172.21.15.44:0/857902958 conn(0x7ff5600b1600 msgr2=0x7ff560168230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 -- 172.21.15.44:0/857902958 shutdown_connections 2024-09-13T08:37:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.627+0000 7ff55fe00640 1 -- 172.21.15.44:0/857902958 wait complete. 2024-09-13T08:37:46.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2024-09-13T08:37:46.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.651+0000 7f2ded600640 1 -- 172.21.15.44:0/1528898218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2df80d0150 con 0x7f2df80b2a50 2024-09-13T08:37:46.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2defe00640 1 -- 172.21.15.44:0/1528898218 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2de4011070 con 0x7f2df80b2a50 2024-09-13T08:37:46.658 INFO:teuthology.orchestra.run.smithi044.stdout:64424509445 2024-09-13T08:37:46.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 -- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dd8072f90 msgr2=0x7f2dd8075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dd8072f90 0x7f2dd8075450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2df81543e0 tx=0x7f2de800b440 comp rx=0 tx=0).stop 2024-09-13T08:37:46.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 -- 172.21.15.44:0/1528898218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df80b2a50 msgr2=0x7f2df815cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df80b2a50 0x7f2df815cdb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2de40094d0 tx=0x7f2de4009990 comp rx=0 tx=0).stop 2024-09-13T08:37:46.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 -- 172.21.15.44:0/1528898218 shutdown_connections 2024-09-13T08:37:46.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dd8072f90 0x7f2dd8075450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2df813cc20 0x7f2df815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2df80d03f0 0x7f2df8136140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 --2- 172.21.15.44:0/1528898218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df80b2a50 0x7f2df815cdb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 -- 172.21.15.44:0/1528898218 >> 172.21.15.44:0/1528898218 conn(0x7f2df806cc20 msgr2=0x7f2df813d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 -- 172.21.15.44:0/1528898218 shutdown_connections 2024-09-13T08:37:46.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.655+0000 7f2ded600640 1 -- 172.21.15.44:0/1528898218 wait complete. 2024-09-13T08:37:46.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.663+0000 7f6e93800640 1 -- 172.21.15.44:0/3677555636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6e8c0b33f0 con 0x7f6e8c06e740 2024-09-13T08:37:46.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.663+0000 7f6e88c00640 1 -- 172.21.15.44:0/3677555636 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6e7c0f8fa0 con 0x7f6e8c06e740 2024-09-13T08:37:46.669 INFO:teuthology.orchestra.run.smithi044.stdout:60129542151 2024-09-13T08:37:46.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 -- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e6c072f40 msgr2=0x7f6e6c075400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e6c072f40 0x7f6e6c075400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0d2ec0 tx=0x7f6e780091e0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 -- 172.21.15.44:0/3677555636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e8c06e740 msgr2=0x7f6e8c0c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e8c06e740 0x7f6e8c0c62e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0ac000 tx=0x7f6e7c005d60 comp rx=0 tx=0).stop 2024-09-13T08:37:46.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 -- 172.21.15.44:0/3677555636 shutdown_connections 2024-09-13T08:37:46.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e6c072f40 0x7f6e6c075400 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c162200 0x7f6e8c0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e8c0b11d0 0x7f6e8c15cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 --2- 172.21.15.44:0/3677555636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e8c06e740 0x7f6e8c0c62e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 -- 172.21.15.44:0/3677555636 >> 172.21.15.44:0/3677555636 conn(0x7f6e8c0d1dc0 msgr2=0x7f6e8c06eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 -- 172.21.15.44:0/3677555636 shutdown_connections 2024-09-13T08:37:46.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f6e93800640 1 -- 172.21.15.44:0/3677555636 wait complete. 2024-09-13T08:37:46.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f127405af60 con 0x7f127403ef50 2024-09-13T08:37:46.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.667+0000 7f126fe00640 1 -- 172.21.15.44:0/1767447771 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f127005f010 con 0x7f127403ef50 2024-09-13T08:37:46.673 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-09-13T08:37:46.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f125c072f90 msgr2=0x7f125c075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f125c072f90 0x7f125c075450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12680058d0 tx=0x7f1268005860 comp rx=0 tx=0).stop 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f127403ef50 msgr2=0x7f12740d9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f127403ef50 0x7f12740d9770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f127000b3e0 tx=0x7f127000b8a0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 shutdown_connections 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f125c072f90 0x7f125c075450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12740ddbe0 0x7f12740e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1274057d90 0x7f12740d9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 --2- 172.21.15.44:0/1767447771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f127403ef50 0x7f12740d9770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 >> 172.21.15.44:0/1767447771 conn(0x7f127405cdc0 msgr2=0x7f1274042250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 shutdown_connections 2024-09-13T08:37:46.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.671+0000 7f1281c00640 1 -- 172.21.15.44:0/1767447771 wait complete. 2024-09-13T08:37:46.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2024-09-13T08:37:46.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.6 2024-09-13T08:37:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.691+0000 7fc214c00640 1 -- 172.21.15.44:0/4065799221 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc2080dfe10 con 0x7fc208051860 2024-09-13T08:37:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.691+0000 7fc203400640 1 -- 172.21.15.44:0/4065799221 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc2100df3c0 con 0x7fc208051860 2024-09-13T08:37:46.696 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-09-13T08:37:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 -- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc1f0072ec0 msgr2=0x7fc1f0075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc1f0072ec0 0x7fc1f0075380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2040af000 tx=0x7fc20400b040 comp rx=0 tx=0).stop 2024-09-13T08:37:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 -- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc208051860 msgr2=0x7fc20817fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc208051860 0x7fc20817fa80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc2100773e0 tx=0x7fc210069d40 comp rx=0 tx=0).stop 2024-09-13T08:37:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 -- 172.21.15.44:0/4065799221 shutdown_connections 2024-09-13T08:37:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc1f0072ec0 0x7fc1f0075380 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc208051860 0x7fc20817fa80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc208034f90 0x7fc20817f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 --2- 172.21.15.44:0/4065799221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2080341b0 0x7fc20804b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 -- 172.21.15.44:0/4065799221 >> 172.21.15.44:0/4065799221 conn(0x7fc20801e060 msgr2=0x7fc208006a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:46.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2024-09-13T08:37:46.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 -- 172.21.15.44:0/4065799221 shutdown_connections 2024-09-13T08:37:46.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:46.695+0000 7fc214c00640 1 -- 172.21.15.44:0/4065799221 wait complete. 2024-09-13T08:37:46.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-09-13T08:37:47.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T08:37:47.600 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T08:37:47.610 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T08:37:47.654 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T08:37:47.695 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T08:37:47.695 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T08:37:47.700 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T08:37:47.718 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T08:37:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.783+0000 7f90bb200640 1 -- 172.21.15.44:0/4198884067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc15ad90 msgr2=0x7f90bc0ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.783+0000 7f90bb200640 1 --2- 172.21.15.44:0/4198884067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc15ad90 0x7f90bc0ca2b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009bc0 tx=0x7f90b401ca10 comp rx=0 tx=0).stop 2024-09-13T08:37:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.783+0000 7f90bb200640 1 -- 172.21.15.44:0/4198884067 shutdown_connections 2024-09-13T08:37:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.783+0000 7f90bb200640 1 --2- 172.21.15.44:0/4198884067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc15ad90 0x7f90bc0ca2b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.783+0000 7f90bb200640 1 --2- 172.21.15.44:0/4198884067 >> v2:172.21.15.44:3300/0 conn(0x7f90bc06f7f0 0x7f90bc06fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.783+0000 7f90bb200640 1 -- 172.21.15.44:0/4198884067 >> 172.21.15.44:0/4198884067 conn(0x7f90bc0b1600 msgr2=0x7f90bc0b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.783+0000 7f90bb200640 1 -- 172.21.15.44:0/4198884067 shutdown_connections 2024-09-13T08:37:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.783+0000 7f90bb200640 1 -- 172.21.15.44:0/4198884067 wait complete. 2024-09-13T08:37:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90bb200640 1 Processor -- start 2024-09-13T08:37:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90bb200640 1 -- start start 2024-09-13T08:37:47.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90bb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc06f7f0 0x7f90bc156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90bb200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0ca7f0 0x7f90bc1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90b9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc06f7f0 0x7f90bc156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90bb200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc15ad90 0x7f90bc1f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90b9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc06f7f0 0x7f90bc156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36588/0 (socket says 172.21.15.44:36588) 2024-09-13T08:37:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90b9e00640 1 -- 172.21.15.44:0/82271216 learned_addr learned my addr 172.21.15.44:0/82271216 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc167e40 con 0x7f90bc06f7f0 2024-09-13T08:37:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc167cc0 con 0x7f90bc0ca7f0 2024-09-13T08:37:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.787+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f90bc167fc0 con 0x7f90bc15ad90 2024-09-13T08:37:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90b9400640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0ca7f0 0x7f90bc1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90ba800640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc15ad90 0x7f90bc1f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90b9e00640 1 -- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc15ad90 msgr2=0x7f90bc1f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90b9e00640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc15ad90 0x7f90bc1f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90b9e00640 1 -- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0ca7f0 msgr2=0x7f90bc1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90b9e00640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0ca7f0 0x7f90bc1558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90b9e00640 1 -- 172.21.15.44:0/82271216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc155fd0 con 0x7f90bc06f7f0 2024-09-13T08:37:47.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90b9e00640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc06f7f0 0x7f90bc156ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00ea20 tx=0x7f90ac00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90b2000640 1 -- 172.21.15.44:0/82271216 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00cd80 con 0x7f90bc06f7f0 2024-09-13T08:37:47.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc1562b0 con 0x7f90bc06f7f0 2024-09-13T08:37:47.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc150d10 con 0x7f90bc06f7f0 2024-09-13T08:37:47.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90b2000640 1 -- 172.21.15.44:0/82271216 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90ac010430 con 0x7f90bc06f7f0 2024-09-13T08:37:47.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.791+0000 7f90b2000640 1 -- 172.21.15.44:0/82271216 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00cd80 con 0x7f90bc06f7f0 2024-09-13T08:37:47.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.795+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90940052a0 con 0x7f90bc06f7f0 2024-09-13T08:37:47.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.795+0000 7f90b2000640 1 -- 172.21.15.44:0/82271216 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f90ac01f020 con 0x7f90bc06f7f0 2024-09-13T08:37:47.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.799+0000 7f90b2000640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c0732c0 0x7f909c075780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.807+0000 7f90b2000640 1 -- 172.21.15.44:0/82271216 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f90ac00cb80 con 0x7f90bc06f7f0 2024-09-13T08:37:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.807+0000 7f90b9400640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c0732c0 0x7f909c075780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.807+0000 7f90b9400640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c0732c0 0x7f909c075780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90b40af000 tx=0x7f90b401b250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.811+0000 7f90b2000640 1 -- 172.21.15.44:0/82271216 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f90ac05f170 con 0x7f90bc06f7f0 2024-09-13T08:37:47.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7f1b6e200640 1 -- 172.21.15.44:0/1481723497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b50004810 msgr2=0x7f1b50004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7f1b6e200640 1 --2- 172.21.15.44:0/1481723497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b50004810 0x7f1b50004c90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b54006fb0 tx=0x7f1b5401f660 comp rx=0 tx=0).stop 2024-09-13T08:37:47.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7f1b67e00640 1 -- 172.21.15.44:0/1481723497 reap_dead start 2024-09-13T08:37:47.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7f1b6e200640 1 -- 172.21.15.44:0/1481723497 shutdown_connections 2024-09-13T08:37:47.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7f1b6e200640 1 -- 172.21.15.44:0/1481723497 >> 172.21.15.44:0/1481723497 conn(0x7f1b6001d780 msgr2=0x7f1b60054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:47.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7f1b6e200640 1 -- 172.21.15.44:0/1481723497 shutdown_connections 2024-09-13T08:37:47.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7ffaee600640 1 -- 172.21.15.44:0/2063664039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffae000a730 msgr2=0x7ffae000ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7f1b6e200640 1 -- 172.21.15.44:0/1481723497 wait complete. 2024-09-13T08:37:47.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7ffaee600640 1 --2- 172.21.15.44:0/2063664039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffae000a730 0x7ffae000ad50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffae806b080 tx=0x7ffae80985d0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7ffaee600640 1 -- 172.21.15.44:0/2063664039 shutdown_connections 2024-09-13T08:37:47.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7ffaee600640 1 --2- 172.21.15.44:0/2063664039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffae000a730 0x7ffae000ad50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7ffaee600640 1 --2- 172.21.15.44:0/2063664039 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffae0035fc0 0x7ffae000a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.863+0000 7ffaee600640 1 -- 172.21.15.44:0/2063664039 >> 172.21.15.44:0/2063664039 conn(0x7ffae0054540 msgr2=0x7ffae0054950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:47.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.867+0000 7f1b6e200640 1 Processor -- start 2024-09-13T08:37:47.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.867+0000 7f1b6e200640 1 -- start start 2024-09-13T08:37:47.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7ffaee600640 1 -- 172.21.15.44:0/2063664039 shutdown_connections 2024-09-13T08:37:47.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7ffaee600640 1 -- 172.21.15.44:0/2063664039 wait complete. 2024-09-13T08:37:47.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b6e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b50004810 0x7f1b600d7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b6e200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b60052310 0x7f1b60052790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b50004810 0x7f1b600d7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b50004810 0x7f1b600d7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36598/0 (socket says 172.21.15.44:36598) 2024-09-13T08:37:47.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67400640 1 -- 172.21.15.44:0/2239777787 learned_addr learned my addr 172.21.15.44:0/2239777787 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:47.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b6e200640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b60020160 0x7f1b600205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b66a00640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b60052310 0x7f1b60052790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b600e7240 con 0x7f1b50004810 2024-09-13T08:37:47.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1b600e70c0 con 0x7f1b60020160 2024-09-13T08:37:47.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1b600e73c0 con 0x7f1b60052310 2024-09-13T08:37:47.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67e00640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b60020160 0x7f1b600205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67e00640 1 -- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b60052310 msgr2=0x7f1b60052790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67e00640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b60052310 0x7f1b60052790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67e00640 1 -- 172.21.15.44:0/2239777787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b50004810 msgr2=0x7f1b600d7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67e00640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b50004810 0x7f1b600d7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67e00640 1 -- 172.21.15.44:0/2239777787 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b600d8230 con 0x7f1b60020160 2024-09-13T08:37:47.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b66a00640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b60052310 0x7f1b60052790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:37:47.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67400640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b50004810 0x7f1b600d7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:47.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7f1b67e00640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b60020160 0x7f1b600205e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b68069ff0 tx=0x7f1b68069e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.871+0000 7ffaee600640 1 Processor -- start 2024-09-13T08:37:47.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7f1b5b400640 1 -- 172.21.15.44:0/2239777787 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68094630 con 0x7f1b60020160 2024-09-13T08:37:47.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7f1b5b400640 1 -- 172.21.15.44:0/2239777787 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b68094920 con 0x7f1b60020160 2024-09-13T08:37:47.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7f1b5b400640 1 -- 172.21.15.44:0/2239777787 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68094d70 con 0x7f1b60020160 2024-09-13T08:37:47.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7ffaee600640 1 -- start start 2024-09-13T08:37:47.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b600e3c40 con 0x7f1b60020160 2024-09-13T08:37:47.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b600e4200 con 0x7f1b60020160 2024-09-13T08:37:47.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b440052a0 con 0x7f1b60020160 2024-09-13T08:37:47.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7ffaee600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffae000a730 0x7ffae00e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7ffaee600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffae0035fc0 0x7ffae004b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7ffaee600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffae0038f60 0x7ffae004bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7ffaee600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffae00ecdb0 con 0x7ffae0038f60 2024-09-13T08:37:47.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7ffaee600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffae00ecc30 con 0x7ffae000a730 2024-09-13T08:37:47.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7ffaee600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ffae00ecf30 con 0x7ffae0035fc0 2024-09-13T08:37:47.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7ffae7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffae0038f60 0x7ffae004bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7ffae7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffae0038f60 0x7ffae004bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36606/0 (socket says 172.21.15.44:36606) 2024-09-13T08:37:47.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.875+0000 7ffae7e00640 1 -- 172.21.15.44:0/3869038316 learned_addr learned my addr 172.21.15.44:0/3869038316 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:47.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffae7400640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffae000a730 0x7ffae00e1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7f1b5b400640 1 -- 172.21.15.44:0/2239777787 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f1b68074460 con 0x7f1b60020160 2024-09-13T08:37:47.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffae6a00640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffae0035fc0 0x7ffae004b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffae7400640 1 -- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffae0035fc0 msgr2=0x7ffae004b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffae7400640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffae0035fc0 0x7ffae004b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffae7400640 1 -- 172.21.15.44:0/3869038316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffae0038f60 msgr2=0x7ffae004bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffae7400640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffae0038f60 0x7ffae004bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffae7400640 1 -- 172.21.15.44:0/3869038316 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae004c1f0 con 0x7ffae000a730 2024-09-13T08:37:47.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffae7e00640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffae0038f60 0x7ffae004bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:47.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7f1b5b400640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b48072fc0 0x7f1b48075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7f1b5b400640 1 -- 172.21.15.44:0/2239777787 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1b6810ff40 con 0x7f1b60020160 2024-09-13T08:37:47.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffae7400640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffae000a730 0x7ffae00e1fe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffad400c3e0 tx=0x7ffad400c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7f1b67400640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b48072fc0 0x7f1b48075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7f1b67400640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b48072fc0 0x7f1b48075480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009b10 tx=0x7f1b5c005d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffadb400640 1 -- 172.21.15.44:0/3869038316 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4017070 con 0x7ffae000a730 2024-09-13T08:37:47.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffadb400640 1 -- 172.21.15.44:0/3869038316 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffad4003600 con 0x7ffae000a730 2024-09-13T08:37:47.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffadb400640 1 -- 172.21.15.44:0/3869038316 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4003a90 con 0x7ffae000a730 2024-09-13T08:37:47.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.879+0000 7ffaee600640 1 -- 172.21.15.44:0/3869038316 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae004c4d0 con 0x7ffae000a730 2024-09-13T08:37:47.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.883+0000 7ffaee600640 1 -- 172.21.15.44:0/3869038316 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae004ca90 con 0x7ffae000a730 2024-09-13T08:37:47.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.883+0000 7f1b5b400640 1 -- 172.21.15.44:0/2239777787 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1b6809a060 con 0x7f1b60020160 2024-09-13T08:37:47.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.887+0000 7ffaee600640 1 -- 172.21.15.44:0/3869038316 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae00ea4c0 con 0x7ffae000a730 2024-09-13T08:37:47.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.887+0000 7ffadb400640 1 -- 172.21.15.44:0/3869038316 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7ffad4003c30 con 0x7ffae000a730 2024-09-13T08:37:47.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.891+0000 7ffadb400640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffac8072ec0 0x7ffac8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.891+0000 7ffadb400640 1 -- 172.21.15.44:0/3869038316 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ffad40931f0 con 0x7ffae000a730 2024-09-13T08:37:47.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.891+0000 7ffae6a00640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffac8072ec0 0x7ffac8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.891+0000 7ffadb400640 1 -- 172.21.15.44:0/3869038316 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ffad405de60 con 0x7ffae000a730 2024-09-13T08:37:47.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.891+0000 7ffae6a00640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffac8072ec0 0x7ffac8075380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffae0035b10 tx=0x7ffadc0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.943+0000 7fd996200640 1 -- 172.21.15.44:0/405406787 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd978004810 msgr2=0x7fd978004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.943+0000 7fd996200640 1 --2- 172.21.15.44:0/405406787 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd978004810 0x7fd978004c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009910 tx=0x7fd97c01c6d0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.943+0000 7fd995800640 1 -- 172.21.15.44:0/405406787 reap_dead start 2024-09-13T08:37:47.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.943+0000 7fd996200640 1 -- 172.21.15.44:0/405406787 shutdown_connections 2024-09-13T08:37:47.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.943+0000 7fd996200640 1 -- 172.21.15.44:0/405406787 >> 172.21.15.44:0/405406787 conn(0x7fd98801e8d0 msgr2=0x7fd988055660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.943+0000 7fd996200640 1 -- 172.21.15.44:0/405406787 shutdown_connections 2024-09-13T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.943+0000 7fd996200640 1 -- 172.21.15.44:0/405406787 wait complete. 2024-09-13T08:37:47.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.943+0000 7fd996200640 1 Processor -- start 2024-09-13T08:37:47.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd996200640 1 -- start start 2024-09-13T08:37:47.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd996200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9880081a0 0x7fd9880deb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd996200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd988008580 0x7fd9880e2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd996200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9880e3320 0x7fd9880e5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd996200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9880e7870 con 0x7fd988008580 2024-09-13T08:37:47.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd996200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd988034cd0 con 0x7fd9880081a0 2024-09-13T08:37:47.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd996200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd988055270 con 0x7fd9880e3320 2024-09-13T08:37:47.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd98fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd988008580 0x7fd9880e2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd98fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd988008580 0x7fd9880e2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36656/0 (socket says 172.21.15.44:36656) 2024-09-13T08:37:47.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd98fe00640 1 -- 172.21.15.44:0/714497231 learned_addr learned my addr 172.21.15.44:0/714497231 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:47.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd994e00640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9880081a0 0x7fd9880deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd995800640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9880e3320 0x7fd9880e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd98fe00640 1 -- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9880e3320 msgr2=0x7fd9880e5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd98fe00640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9880e3320 0x7fd9880e5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd98fe00640 1 -- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9880081a0 msgr2=0x7fd9880deb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd98fe00640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9880081a0 0x7fd9880deb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd98fe00640 1 -- 172.21.15.44:0/714497231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9880e0210 con 0x7fd988008580 2024-09-13T08:37:47.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.947+0000 7fd98fe00640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd988008580 0x7fd9880e2de0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd97c01cf70 tx=0x7fd97c0a7ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7fd98cc00640 1 -- 172.21.15.44:0/714497231 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0bd070 con 0x7fd988008580 2024-09-13T08:37:47.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7fd98cc00640 1 -- 172.21.15.44:0/714497231 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd97c004230 con 0x7fd988008580 2024-09-13T08:37:47.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7fd98cc00640 1 -- 172.21.15.44:0/714497231 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b2040 con 0x7fd988008580 2024-09-13T08:37:47.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7fd996200640 1 -- 172.21.15.44:0/714497231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9880e04f0 con 0x7fd988008580 2024-09-13T08:37:47.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7fd996200640 1 -- 172.21.15.44:0/714497231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9880e0ab0 con 0x7fd988008580 2024-09-13T08:37:47.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7f34d5800640 1 -- 172.21.15.44:0/3782066660 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34d00cdd10 msgr2=0x7f34d006f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7f34d5800640 1 --2- 172.21.15.44:0/3782066660 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34d00cdd10 0x7f34d006f6f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009bc0 tx=0x7f34c801c990 comp rx=0 tx=0).stop 2024-09-13T08:37:47.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7f34d5800640 1 -- 172.21.15.44:0/3782066660 shutdown_connections 2024-09-13T08:37:47.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7f34d5800640 1 --2- 172.21.15.44:0/3782066660 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34d00cdd10 0x7f34d006f6f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7f34d5800640 1 --2- 172.21.15.44:0/3782066660 >> v2:172.21.15.44:3300/0 conn(0x7f34d00744b0 0x7f34d0162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7f34d5800640 1 -- 172.21.15.44:0/3782066660 >> 172.21.15.44:0/3782066660 conn(0x7f34d00b1600 msgr2=0x7f34d00b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:47.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7f34d5800640 1 -- 172.21.15.44:0/3782066660 shutdown_connections 2024-09-13T08:37:47.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7f34d5800640 1 -- 172.21.15.44:0/3782066660 wait complete. 2024-09-13T08:37:47.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.955+0000 7fd996200640 1 -- 172.21.15.44:0/714497231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9880346c0 con 0x7fd988008580 2024-09-13T08:37:47.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34d5800640 1 Processor -- start 2024-09-13T08:37:47.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34d5800640 1 -- start start 2024-09-13T08:37:47.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7fd98cc00640 1 -- 172.21.15.44:0/714497231 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fd97c0047e0 con 0x7fd988008580 2024-09-13T08:37:47.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f0445800640 1 -- 172.21.15.44:0/3110900648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0440159cc0 msgr2=0x7f0440072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f0445800640 1 --2- 172.21.15.44:0/3110900648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0440159cc0 0x7f0440072680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0430009900 tx=0x7f043001c8b0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f0445800640 1 -- 172.21.15.44:0/3110900648 shutdown_connections 2024-09-13T08:37:47.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f0445800640 1 --2- 172.21.15.44:0/3110900648 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04400cf570 0x7f04400cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f0445800640 1 --2- 172.21.15.44:0/3110900648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0440159cc0 0x7f0440072680 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f0445800640 1 -- 172.21.15.44:0/3110900648 >> 172.21.15.44:0/3110900648 conn(0x7f04400cfa30 msgr2=0x7f04400cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:47.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34d5800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34d0072680 0x7f34d015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34d5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00744b0 0x7f34d00ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34d5800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34d00cdd10 0x7f34d00c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34d5800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34d00a9980 con 0x7f34d00744b0 2024-09-13T08:37:47.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34d5800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34d00a9800 con 0x7f34d0072680 2024-09-13T08:37:47.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34d5800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f34d00a9b00 con 0x7f34d00cdd10 2024-09-13T08:37:47.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00744b0 0x7f34d00ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7fd98cc00640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd970072f40 0x7fd970075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00744b0 0x7f34d00ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36672/0 (socket says 172.21.15.44:36672) 2024-09-13T08:37:47.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34cfe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34d0072680 0x7f34d015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34cf400640 1 -- 172.21.15.44:0/1921422327 learned_addr learned my addr 172.21.15.44:0/1921422327 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:47.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34d4e00640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34d00cdd10 0x7f34d00c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7fd994e00640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd970072f40 0x7fd970075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34cfe00640 1 -- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34d00cdd10 msgr2=0x7f34d00c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34cfe00640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34d00cdd10 0x7f34d00c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34cfe00640 1 -- 172.21.15.44:0/1921422327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00744b0 msgr2=0x7f34d00ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34cfe00640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00744b0 0x7f34d00ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7f34cfe00640 1 -- 172.21.15.44:0/1921422327 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d00ad960 con 0x7f34d0072680 2024-09-13T08:37:47.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.959+0000 7fd98cc00640 1 -- 172.21.15.44:0/714497231 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd97c0f4a10 con 0x7fd988008580 2024-09-13T08:37:47.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f34cfe00640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34d0072680 0x7f34d015b940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f34c000d690 tx=0x7f34c000db50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7fd994e00640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd970072f40 0x7fd970075400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd9880520b0 tx=0x7fd984009420 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f0445800640 1 -- 172.21.15.44:0/3110900648 shutdown_connections 2024-09-13T08:37:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f34c7e00640 1 -- 172.21.15.44:0/1921422327 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0017070 con 0x7f34d0072680 2024-09-13T08:37:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f34c7e00640 1 -- 172.21.15.44:0/1921422327 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34c0005310 con 0x7f34d0072680 2024-09-13T08:37:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f34c7e00640 1 -- 172.21.15.44:0/1921422327 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0009820 con 0x7f34d0072680 2024-09-13T08:37:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f0445800640 1 -- 172.21.15.44:0/3110900648 wait complete. 2024-09-13T08:37:47.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f34d5800640 1 -- 172.21.15.44:0/1921422327 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d00c6570 con 0x7f34d0072680 2024-09-13T08:37:47.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f34d5800640 1 -- 172.21.15.44:0/1921422327 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d00c5550 con 0x7f34d0072680 2024-09-13T08:37:47.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f0445800640 1 Processor -- start 2024-09-13T08:37:47.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f34d5800640 1 -- 172.21.15.44:0/1921422327 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a80052a0 con 0x7f34d0072680 2024-09-13T08:37:47.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.963+0000 7f0445800640 1 -- start start 2024-09-13T08:37:47.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f34c7e00640 1 -- 172.21.15.44:0/1921422327 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f34c00099c0 con 0x7f34d0072680 2024-09-13T08:37:47.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0445800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04400cd600 0x7f04400c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0445800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04400cf570 0x7f04400c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0445800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0440159cc0 0x7f04400c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0445800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f044006f280 con 0x7f04400cf570 2024-09-13T08:37:47.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0445800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f044006f400 con 0x7f0440159cc0 2024-09-13T08:37:47.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0445800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f044006f580 con 0x7f04400cd600 2024-09-13T08:37:47.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f043fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04400cd600 0x7f04400c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f043fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04400cd600 0x7f04400c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37516/0 (socket says 172.21.15.44:37516) 2024-09-13T08:37:47.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f043fe00640 1 -- 172.21.15.44:0/1965260462 learned_addr learned my addr 172.21.15.44:0/1965260462 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:47.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f34c7e00640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f34b0072ec0 0x7f34b0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0444e00640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0440159cc0 0x7f04400c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f34c7e00640 1 -- 172.21.15.44:0/1921422327 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f34c0093c00 con 0x7f34d0072680 2024-09-13T08:37:47.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f043f400640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04400cf570 0x7f04400c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0444e00640 1 -- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04400cd600 msgr2=0x7f04400c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0444e00640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04400cd600 0x7f04400c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0444e00640 1 -- 172.21.15.44:0/1965260462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04400cf570 msgr2=0x7f04400c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0444e00640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04400cf570 0x7f04400c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.967+0000 7f0444e00640 1 -- 172.21.15.44:0/1965260462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04400c62f0 con 0x7f0440159cc0 2024-09-13T08:37:47.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.971+0000 7f0444e00640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0440159cc0 0x7f04400c5520 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f043800c760 tx=0x7f043800cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.971+0000 7f34cf400640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f34b0072ec0 0x7f34b0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.971+0000 7f34cf400640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f34b0072ec0 0x7f34b0075380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f34d00c4fb0 tx=0x7f34bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.971+0000 7fd98cc00640 1 -- 172.21.15.44:0/714497231 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd97c016430 con 0x7fd988008580 2024-09-13T08:37:47.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.971+0000 7f0437e00640 1 -- 172.21.15.44:0/1965260462 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438003b80 con 0x7f0440159cc0 2024-09-13T08:37:47.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.971+0000 7f0437e00640 1 -- 172.21.15.44:0/1965260462 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0438003d20 con 0x7f0440159cc0 2024-09-13T08:37:47.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.971+0000 7f0437e00640 1 -- 172.21.15.44:0/1965260462 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043800aab0 con 0x7f0440159cc0 2024-09-13T08:37:47.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.971+0000 7f34c7e00640 1 -- 172.21.15.44:0/1921422327 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f34c009a080 con 0x7f34d0072680 2024-09-13T08:37:47.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.971+0000 7f0445800640 1 -- 172.21.15.44:0/1965260462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04400c47e0 con 0x7f0440159cc0 2024-09-13T08:37:47.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.971+0000 7f0445800640 1 -- 172.21.15.44:0/1965260462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0440154080 con 0x7f0440159cc0 2024-09-13T08:37:47.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.975+0000 7f0445800640 1 -- 172.21.15.44:0/1965260462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04400cbd20 con 0x7f0440159cc0 2024-09-13T08:37:47.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.975+0000 7f4081c00640 1 -- 172.21.15.44:0/3537021937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c15df30 msgr2=0x7f407c15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.975+0000 7f4081c00640 1 --2- 172.21.15.44:0/3537021937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c15df30 0x7f407c15e320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f407c06dc00 tx=0x7f407401dcd0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.975+0000 7f4081c00640 1 -- 172.21.15.44:0/3537021937 shutdown_connections 2024-09-13T08:37:47.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.975+0000 7f4081c00640 1 --2- 172.21.15.44:0/3537021937 >> v2:172.21.15.44:3300/0 conn(0x7f407c072a50 0x7f407c15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.975+0000 7f4081c00640 1 --2- 172.21.15.44:0/3537021937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c15df30 0x7f407c15e320 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.975+0000 7f4081c00640 1 -- 172.21.15.44:0/3537021937 >> 172.21.15.44:0/3537021937 conn(0x7f407c06cc20 msgr2=0x7f407c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:47.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.975+0000 7f4081c00640 1 -- 172.21.15.44:0/3537021937 shutdown_connections 2024-09-13T08:37:47.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.975+0000 7f4081c00640 1 -- 172.21.15.44:0/3537021937 wait complete. 2024-09-13T08:37:47.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.979+0000 7f4081c00640 1 Processor -- start 2024-09-13T08:37:47.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.975+0000 7f0437e00640 1 -- 172.21.15.44:0/1965260462 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f043800ac50 con 0x7f0440159cc0 2024-09-13T08:37:47.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.979+0000 7f0437e00640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0420072f90 0x7f0420075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.979+0000 7f0437e00640 1 -- 172.21.15.44:0/1965260462 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0438093b00 con 0x7f0440159cc0 2024-09-13T08:37:47.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.979+0000 7f4081c00640 1 -- start start 2024-09-13T08:37:47.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.979+0000 7f043fe00640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0420072f90 0x7f0420075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.979+0000 7f0437e00640 1 -- 172.21.15.44:0/1965260462 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0438020060 con 0x7f0440159cc0 2024-09-13T08:37:47.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.979+0000 7f043fe00640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0420072f90 0x7f0420075450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f043001a4e0 tx=0x7f043001b0f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.979+0000 7f4081c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f407c072a50 0x7f407c16acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f4081c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c0cfe50 0x7f407c1fad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f4081c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f407c1fb280 0x7f407c153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f4081c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f407c1694c0 con 0x7f407c072a50 2024-09-13T08:37:47.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f4081c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f407c0d07c0 con 0x7f407c0cfe50 2024-09-13T08:37:47.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f4081c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f407c169640 con 0x7f407c1fb280 2024-09-13T08:37:47.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f407b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c0cfe50 0x7f407c1fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f407b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c0cfe50 0x7f407c1fad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:52808/0 (socket says 172.21.15.44:52808) 2024-09-13T08:37:47.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f407b400640 1 -- 172.21.15.44:0/489841798 learned_addr learned my addr 172.21.15.44:0/489841798 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:47.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f4081200640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f407c1fb280 0x7f407c153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f407b400640 1 -- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f407c1fb280 msgr2=0x7f407c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f407b400640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f407c1fb280 0x7f407c153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f407b400640 1 -- 172.21.15.44:0/489841798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f407c072a50 msgr2=0x7f407c16acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f407b400640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f407c072a50 0x7f407c16acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f407b400640 1 -- 172.21.15.44:0/489841798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c169f30 con 0x7f407c0cfe50 2024-09-13T08:37:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f4081200640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f407c1fb280 0x7f407c153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f407b400640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c0cfe50 0x7f407c1fad40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f406c00e9f0 tx=0x7f406c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:47.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f4073e00640 1 -- 172.21.15.44:0/489841798 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c00cd80 con 0x7f407c0cfe50 2024-09-13T08:37:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.983+0000 7f4073e00640 1 -- 172.21.15.44:0/489841798 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f406c010430 con 0x7f407c0cfe50 2024-09-13T08:37:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.987+0000 7f4073e00640 1 -- 172.21.15.44:0/489841798 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c010860 con 0x7f407c0cfe50 2024-09-13T08:37:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.987+0000 7f4081c00640 1 -- 172.21.15.44:0/489841798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407c16a1f0 con 0x7f407c0cfe50 2024-09-13T08:37:47.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.987+0000 7f4081c00640 1 -- 172.21.15.44:0/489841798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407c16a7b0 con 0x7f407c0cfe50 2024-09-13T08:37:47.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.987+0000 7f4081c00640 1 -- 172.21.15.44:0/489841798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f407c15ac30 con 0x7f407c0cfe50 2024-09-13T08:37:47.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.991+0000 7f4073e00640 1 -- 172.21.15.44:0/489841798 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f406c010a00 con 0x7f407c0cfe50 2024-09-13T08:37:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.991+0000 7f4073e00640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f405c072ec0 0x7f405c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.991+0000 7f4073e00640 1 -- 172.21.15.44:0/489841798 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f406c094990 con 0x7f407c0cfe50 2024-09-13T08:37:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.991+0000 7f407be00640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f405c072ec0 0x7f405c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.999+0000 7f407be00640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f405c072ec0 0x7f405c075380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40740b0000 tx=0x7f407401aa70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:47.999+0000 7f4073e00640 1 -- 172.21.15.44:0/489841798 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f406c05f5b0 con 0x7f407c0cfe50 2024-09-13T08:37:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.023+0000 7f66e6400640 1 -- 172.21.15.44:0/3190839118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66c40064e0 msgr2=0x7f66c400cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.023+0000 7f66dea00640 1 -- 172.21.15.44:0/3190839118 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d80042c0 con 0x7f66c40064e0 2024-09-13T08:37:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.023+0000 7f66e6400640 1 --2- 172.21.15.44:0/3190839118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66c40064e0 0x7f66c400cb20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66d8001f80 tx=0x7f66d801daf0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.023+0000 7f66dfe00640 1 -- 172.21.15.44:0/3190839118 reap_dead start 2024-09-13T08:37:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.023+0000 7f66e6400640 1 -- 172.21.15.44:0/3190839118 shutdown_connections 2024-09-13T08:37:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.023+0000 7f66e6400640 1 -- 172.21.15.44:0/3190839118 >> 172.21.15.44:0/3190839118 conn(0x7f66e00b18f0 msgr2=0x7f66e00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.023+0000 7f66e6400640 1 -- 172.21.15.44:0/3190839118 shutdown_connections 2024-09-13T08:37:48.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.023+0000 7f66e6400640 1 -- 172.21.15.44:0/3190839118 wait complete. 2024-09-13T08:37:48.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.027+0000 7f66e6400640 1 Processor -- start 2024-09-13T08:37:48.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.027+0000 7f66e6400640 1 -- start start 2024-09-13T08:37:48.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.027+0000 7f66e6400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c40064e0 0x7f66e0219a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.027+0000 7f66df400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c40064e0 0x7f66e0219a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.027+0000 7f66df400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c40064e0 0x7f66e0219a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36728/0 (socket says 172.21.15.44:36728) 2024-09-13T08:37:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66df400640 1 -- 172.21.15.44:0/1207078411 learned_addr learned my addr 172.21.15.44:0/1207078411 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66e6400640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e008aa70 0x7f66e0219f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66e6400640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66e0216720 0x7f66e0216bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66e00ccce0 con 0x7f66c40064e0 2024-09-13T08:37:48.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66e00ccb60 con 0x7f66e008aa70 2024-09-13T08:37:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f66e00cce60 con 0x7f66e0216720 2024-09-13T08:37:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66d6a00640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e008aa70 0x7f66e0219f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66dfe00640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66e0216720 0x7f66e0216bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66d6a00640 1 -- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66e0216720 msgr2=0x7f66e0216bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66d6a00640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66e0216720 0x7f66e0216bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66d6a00640 1 -- 172.21.15.44:0/1207078411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c40064e0 msgr2=0x7f66e0219a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66d6a00640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c40064e0 0x7f66e0219a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66d6a00640 1 -- 172.21.15.44:0/1207078411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e021a8b0 con 0x7f66e008aa70 2024-09-13T08:37:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.031+0000 7f66d6a00640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e008aa70 0x7f66e0219f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66e00af460 tx=0x7f66c800ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:48.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.035+0000 7f66d7e00640 1 -- 172.21.15.44:0/1207078411 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c800e880 con 0x7f66e008aa70 2024-09-13T08:37:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.035+0000 7f66d7e00640 1 -- 172.21.15.44:0/1207078411 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66c800eb10 con 0x7f66e008aa70 2024-09-13T08:37:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.035+0000 7f66d7e00640 1 -- 172.21.15.44:0/1207078411 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c800e880 con 0x7f66e008aa70 2024-09-13T08:37:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.035+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e008af30 con 0x7f66e008aa70 2024-09-13T08:37:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.035+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e009ae70 con 0x7f66e008aa70 2024-09-13T08:37:48.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.035+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b80052a0 con 0x7f66e008aa70 2024-09-13T08:37:48.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.039+0000 7f66d7e00640 1 -- 172.21.15.44:0/1207078411 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f66c800f450 con 0x7f66e008aa70 2024-09-13T08:37:48.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.039+0000 7f66d7e00640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66bc0732f0 0x7f66bc0757b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:48.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.039+0000 7f66df400640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66bc0732f0 0x7f66bc0757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:48.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.039+0000 7f66d7e00640 1 -- 172.21.15.44:0/1207078411 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f66c800e680 con 0x7f66e008aa70 2024-09-13T08:37:48.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.043+0000 7f66df400640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66bc0732f0 0x7f66bc0757b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66d0005fd0 tx=0x7f66d000a2d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:48.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.047+0000 7f66d7e00640 1 -- 172.21.15.44:0/1207078411 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f66c8014030 con 0x7f66e008aa70 2024-09-13T08:37:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.179+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9094005860 con 0x7f90bc06f7f0 2024-09-13T08:37:48.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.179+0000 7f90b2000640 1 -- 172.21.15.44:0/82271216 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90ac05ea70 con 0x7f90bc06f7f0 2024-09-13T08:37:48.183 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-09-13T08:37:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c0732c0 msgr2=0x7f909c075780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c0732c0 0x7f909c075780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90b40af000 tx=0x7f90b401b250 comp rx=0 tx=0).stop 2024-09-13T08:37:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc06f7f0 msgr2=0x7f90bc156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc06f7f0 0x7f90bc156ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00ea20 tx=0x7f90ac00eee0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 shutdown_connections 2024-09-13T08:37:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c0732c0 0x7f909c075780 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc15ad90 0x7f90bc1f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0ca7f0 0x7f90bc1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 --2- 172.21.15.44:0/82271216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc06f7f0 0x7f90bc156ff0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 >> 172.21.15.44:0/82271216 conn(0x7f90bc0b1600 msgr2=0x7f90bc0b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 shutdown_connections 2024-09-13T08:37:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.183+0000 7f90bb200640 1 -- 172.21.15.44:0/82271216 wait complete. 2024-09-13T08:37:48.224 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.0 2024-09-13T08:37:48.224 DEBUG:teuthology.parallel:result is None 2024-09-13T08:37:48.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.235+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1b44005860 con 0x7f1b60020160 2024-09-13T08:37:48.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b5b400640 1 -- 172.21.15.44:0/2239777787 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b680db200 con 0x7f1b60020160 2024-09-13T08:37:48.242 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-09-13T08:37:48.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b48072fc0 msgr2=0x7f1b48075480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b48072fc0 0x7f1b48075480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009b10 tx=0x7f1b5c005d60 comp rx=0 tx=0).stop 2024-09-13T08:37:48.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b60020160 msgr2=0x7f1b600205e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b60020160 0x7f1b600205e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b68069ff0 tx=0x7f1b68069e00 comp rx=0 tx=0).stop 2024-09-13T08:37:48.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 shutdown_connections 2024-09-13T08:37:48.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b48072fc0 0x7f1b48075480 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b60020160 0x7f1b600205e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b60052310 0x7f1b60052790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 --2- 172.21.15.44:0/2239777787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b50004810 0x7f1b600d7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 >> 172.21.15.44:0/2239777787 conn(0x7f1b6001d780 msgr2=0x7f1b6004a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 shutdown_connections 2024-09-13T08:37:48.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.239+0000 7f1b6e200640 1 -- 172.21.15.44:0/2239777787 wait complete. 2024-09-13T08:37:48.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-09-13T08:37:48.268 DEBUG:teuthology.parallel:result is None 2024-09-13T08:37:48.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.275+0000 7ffaee600640 1 -- 172.21.15.44:0/3869038316 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ffae0052ce0 con 0x7ffae000a730 2024-09-13T08:37:48.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.275+0000 7ffadb400640 1 -- 172.21.15.44:0/3869038316 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffad4062890 con 0x7ffae000a730 2024-09-13T08:37:48.279 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-09-13T08:37:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 -- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffac8072ec0 msgr2=0x7ffac8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffac8072ec0 0x7ffac8075380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffae0035b10 tx=0x7ffadc0091e0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 -- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffae000a730 msgr2=0x7ffae00e1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffae000a730 0x7ffae00e1fe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffad400c3e0 tx=0x7ffad400c8a0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 -- 172.21.15.44:0/3869038316 shutdown_connections 2024-09-13T08:37:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffac8072ec0 0x7ffac8075380 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffae0038f60 0x7ffae004bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffae0035fc0 0x7ffae004b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 --2- 172.21.15.44:0/3869038316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffae000a730 0x7ffae00e1fe0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 -- 172.21.15.44:0/3869038316 >> 172.21.15.44:0/3869038316 conn(0x7ffae0054540 msgr2=0x7ffae00537c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 -- 172.21.15.44:0/3869038316 shutdown_connections 2024-09-13T08:37:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.279+0000 7ffaee600640 1 -- 172.21.15.44:0/3869038316 wait complete. 2024-09-13T08:37:48.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.1 2024-09-13T08:37:48.311 DEBUG:teuthology.parallel:result is None 2024-09-13T08:37:48.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 -- 172.21.15.44:0/489841798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f407c0cc5c0 con 0x7f407c0cfe50 2024-09-13T08:37:48.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4073e00640 1 -- 172.21.15.44:0/489841798 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f406c063fe0 con 0x7f407c0cfe50 2024-09-13T08:37:48.339 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-09-13T08:37:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 -- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f405c072ec0 msgr2=0x7f405c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f405c072ec0 0x7f405c075380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40740b0000 tx=0x7f407401aa70 comp rx=0 tx=0).stop 2024-09-13T08:37:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 -- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c0cfe50 msgr2=0x7f407c1fad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c0cfe50 0x7f407c1fad40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f406c00e9f0 tx=0x7f406c00eeb0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 -- 172.21.15.44:0/489841798 shutdown_connections 2024-09-13T08:37:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f405c072ec0 0x7f405c075380 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f407c1fb280 0x7f407c153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c0cfe50 0x7f407c1fad40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 --2- 172.21.15.44:0/489841798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f407c072a50 0x7f407c16acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 -- 172.21.15.44:0/489841798 >> 172.21.15.44:0/489841798 conn(0x7f407c06cc20 msgr2=0x7f407c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 -- 172.21.15.44:0/489841798 shutdown_connections 2024-09-13T08:37:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.335+0000 7f4081c00640 1 -- 172.21.15.44:0/489841798 wait complete. 2024-09-13T08:37:48.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.343+0000 7fd996200640 1 -- 172.21.15.44:0/714497231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd9880df050 con 0x7fd988008580 2024-09-13T08:37:48.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.343+0000 7fd98cc00640 1 -- 172.21.15.44:0/714497231 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd97c016430 con 0x7fd988008580 2024-09-13T08:37:48.346 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-09-13T08:37:48.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.343+0000 7fd996200640 1 -- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd970072f40 msgr2=0x7fd970075400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.343+0000 7fd996200640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd970072f40 0x7fd970075400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd9880520b0 tx=0x7fd984009420 comp rx=0 tx=0).stop 2024-09-13T08:37:48.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.343+0000 7fd996200640 1 -- 172.21.15.44:0/714497231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd988008580 msgr2=0x7fd9880e2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.343+0000 7fd996200640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd988008580 0x7fd9880e2de0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd97c01cf70 tx=0x7fd97c0a7ee0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.347+0000 7fd996200640 1 -- 172.21.15.44:0/714497231 shutdown_connections 2024-09-13T08:37:48.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.347+0000 7fd996200640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd970072f40 0x7fd970075400 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.347+0000 7fd996200640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9880e3320 0x7fd9880e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.347+0000 7fd996200640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd988008580 0x7fd9880e2de0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.347+0000 7fd996200640 1 --2- 172.21.15.44:0/714497231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9880081a0 0x7fd9880deb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.347+0000 7fd996200640 1 -- 172.21.15.44:0/714497231 >> 172.21.15.44:0/714497231 conn(0x7fd98801e8d0 msgr2=0x7fd98804bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.347+0000 7fd996200640 1 -- 172.21.15.44:0/714497231 shutdown_connections 2024-09-13T08:37:48.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.347+0000 7fd996200640 1 -- 172.21.15.44:0/714497231 wait complete. 2024-09-13T08:37:48.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.351+0000 7f34d5800640 1 -- 172.21.15.44:0/1921422327 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f34a8005860 con 0x7f34d0072680 2024-09-13T08:37:48.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.351+0000 7f34c7e00640 1 -- 172.21.15.44:0/1921422327 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f34c005e800 con 0x7f34d0072680 2024-09-13T08:37:48.356 INFO:teuthology.orchestra.run.smithi044.stdout:60129542152 2024-09-13T08:37:48.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f0445800640 1 -- 172.21.15.44:0/1965260462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f04400c4c90 con 0x7f0440159cc0 2024-09-13T08:37:48.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f0437e00640 1 -- 172.21.15.44:0/1965260462 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f043805edc0 con 0x7f0440159cc0 2024-09-13T08:37:48.359 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 -- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f34b0072ec0 msgr2=0x7f34b0075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f34b0072ec0 0x7f34b0075380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f34d00c4fb0 tx=0x7f34bc009040 comp rx=0 tx=0).stop 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 -- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34d0072680 msgr2=0x7f34d015b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34d0072680 0x7f34d015b940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f34c000d690 tx=0x7f34c000db50 comp rx=0 tx=0).stop 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 -- 172.21.15.44:0/1921422327 shutdown_connections 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f34b0072ec0 0x7f34b0075380 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34d00cdd10 0x7f34d00c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00744b0 0x7f34d00ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 --2- 172.21.15.44:0/1921422327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34d0072680 0x7f34d015b940 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f34c000d690 tx=0x7f34c000db50 comp rx=0 tx=0).stop 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 -- 172.21.15.44:0/1921422327 >> 172.21.15.44:0/1921422327 conn(0x7f34d00b1600 msgr2=0x7f34d015e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 -- 172.21.15.44:0/1921422327 shutdown_connections 2024-09-13T08:37:48.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.355+0000 7f34d5800640 1 -- 172.21.15.44:0/1921422327 wait complete. 2024-09-13T08:37:48.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 -- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0420072f90 msgr2=0x7f0420075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0420072f90 0x7f0420075450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f043001a4e0 tx=0x7f043001b0f0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 -- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0440159cc0 msgr2=0x7f04400c5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0440159cc0 0x7f04400c5520 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f043800c760 tx=0x7f043800cc20 comp rx=0 tx=0).stop 2024-09-13T08:37:48.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2024-09-13T08:37:48.364 DEBUG:teuthology.parallel:result is None 2024-09-13T08:37:48.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 -- 172.21.15.44:0/1965260462 shutdown_connections 2024-09-13T08:37:48.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0420072f90 0x7f0420075450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0440159cc0 0x7f04400c5520 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04400cf570 0x7f04400c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 --2- 172.21.15.44:0/1965260462 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04400cd600 0x7f04400c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 -- 172.21.15.44:0/1965260462 >> 172.21.15.44:0/1965260462 conn(0x7f04400cfa30 msgr2=0x7f0440073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 -- 172.21.15.44:0/1965260462 shutdown_connections 2024-09-13T08:37:48.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.359+0000 7f0445800640 1 -- 172.21.15.44:0/1965260462 wait complete. 2024-09-13T08:37:48.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2024-09-13T08:37:48.385 DEBUG:teuthology.parallel:result is None 2024-09-13T08:37:48.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.6 2024-09-13T08:37:48.386 DEBUG:teuthology.parallel:result is None 2024-09-13T08:37:48.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2024-09-13T08:37:48.387 DEBUG:teuthology.parallel:result is None 2024-09-13T08:37:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f66b8005860 con 0x7f66e008aa70 2024-09-13T08:37:48.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66d7e00640 1 -- 172.21.15.44:0/1207078411 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f66c80630d0 con 0x7f66e008aa70 2024-09-13T08:37:48.399 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-09-13T08:37:48.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66bc0732f0 msgr2=0x7f66bc0757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66bc0732f0 0x7f66bc0757b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66d0005fd0 tx=0x7f66d000a2d0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e008aa70 msgr2=0x7f66e0219f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e008aa70 0x7f66e0219f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66e00af460 tx=0x7f66c800ca80 comp rx=0 tx=0).stop 2024-09-13T08:37:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 shutdown_connections 2024-09-13T08:37:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66bc0732f0 0x7f66bc0757b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66e0216720 0x7f66e0216bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e008aa70 0x7f66e0219f80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 --2- 172.21.15.44:0/1207078411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c40064e0 0x7f66e0219a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 >> 172.21.15.44:0/1207078411 conn(0x7f66e00b18f0 msgr2=0x7f66e00b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 shutdown_connections 2024-09-13T08:37:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.395+0000 7f66e6400640 1 -- 172.21.15.44:0/1207078411 wait complete. 2024-09-13T08:37:48.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-13T08:37:48.420 DEBUG:teuthology.parallel:result is None 2024-09-13T08:37:48.421 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:37:48.421 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:37:48.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.607+0000 7fd9d2800640 1 -- 172.21.15.44:0/2236804380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9b0004060 msgr2=0x7fd9b0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.607+0000 7fd9d2800640 1 --2- 172.21.15.44:0/2236804380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9b0004060 0x7fd9b0004810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc072c80 tx=0x7fd9b401c750 comp rx=0 tx=0).stop 2024-09-13T08:37:48.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.607+0000 7fd9cbe00640 1 -- 172.21.15.44:0/2236804380 reap_dead start 2024-09-13T08:37:48.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.607+0000 7fd9d2800640 1 -- 172.21.15.44:0/2236804380 shutdown_connections 2024-09-13T08:37:48.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.607+0000 7fd9d2800640 1 -- 172.21.15.44:0/2236804380 >> 172.21.15.44:0/2236804380 conn(0x7fd9cc0b1f10 msgr2=0x7fd9cc0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.607+0000 7fd9d2800640 1 -- 172.21.15.44:0/2236804380 shutdown_connections 2024-09-13T08:37:48.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.607+0000 7fd9d2800640 1 -- 172.21.15.44:0/2236804380 wait complete. 2024-09-13T08:37:48.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.607+0000 7fd9d2800640 1 Processor -- start 2024-09-13T08:37:48.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.607+0000 7fd9d2800640 1 -- start start 2024-09-13T08:37:48.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9d2800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9b0004060 0x7fd9cc1f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:48.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9d2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9cc06fc40 0x7fd9cc1f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:48.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9d2800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9cc0700a0 0x7fd9cc1f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9d2800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc068fd0 con 0x7fd9cc06fc40 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9d2800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc068e50 con 0x7fd9b0004060 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9d2800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd9cc0ce700 con 0x7fd9cc0700a0 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9cc06fc40 0x7fd9cc1f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9cc06fc40 0x7fd9cc1f3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36754/0 (socket says 172.21.15.44:36754) 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3e00640 1 -- 172.21.15.44:0/4231284907 learned_addr learned my addr 172.21.15.44:0/4231284907 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9cbe00640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9cc0700a0 0x7fd9cc1f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9cb400640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9b0004060 0x7fd9cc1f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3e00640 1 -- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9cc0700a0 msgr2=0x7fd9cc1f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3e00640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9cc0700a0 0x7fd9cc1f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3e00640 1 -- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9b0004060 msgr2=0x7fd9cc1f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3e00640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9b0004060 0x7fd9cc1f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3e00640 1 -- 172.21.15.44:0/4231284907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc0b5040 con 0x7fd9cc06fc40 2024-09-13T08:37:48.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9cbe00640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9cc0700a0 0x7fd9cc1f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:48.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9cb400640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9b0004060 0x7fd9cc1f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:48.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3e00640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9cc06fc40 0x7fd9cc1f3c50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd9b401ae50 tx=0x7fd9b4001420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:48.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3400640 1 -- 172.21.15.44:0/4231284907 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4003810 con 0x7fd9cc06fc40 2024-09-13T08:37:48.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3400640 1 -- 172.21.15.44:0/4231284907 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9b4003aa0 con 0x7fd9cc06fc40 2024-09-13T08:37:48.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9c3400640 1 -- 172.21.15.44:0/4231284907 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4003e50 con 0x7fd9cc06fc40 2024-09-13T08:37:48.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9d2800640 1 -- 172.21.15.44:0/4231284907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9cc0b5320 con 0x7fd9cc06fc40 2024-09-13T08:37:48.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9d2800640 1 -- 172.21.15.44:0/4231284907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9cc0b5bc0 con 0x7fd9cc06fc40 2024-09-13T08:37:48.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.611+0000 7fd9d2800640 1 -- 172.21.15.44:0/4231284907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9cc0b5710 con 0x7fd9cc06fc40 2024-09-13T08:37:48.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.615+0000 7fd9c3400640 1 -- 172.21.15.44:0/4231284907 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fd9b40a55b0 con 0x7fd9cc06fc40 2024-09-13T08:37:48.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.615+0000 7fd9c3400640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9a8073060 0x7fd9a8075520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:48.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.615+0000 7fd9c3400640 1 -- 172.21.15.44:0/4231284907 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd9b412d600 con 0x7fd9cc06fc40 2024-09-13T08:37:48.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.615+0000 7fd9cb400640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9a8073060 0x7fd9a8075520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:48.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.615+0000 7fd9cb400640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9a8073060 0x7fd9a8075520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009c60 tx=0x7fd9bc0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:48.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.619+0000 7fd9c3400640 1 -- 172.21.15.44:0/4231284907 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd9b40b3070 con 0x7fd9cc06fc40 2024-09-13T08:37:48.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.791+0000 7fd9d2800640 1 -- 172.21.15.44:0/4231284907 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd9cc0b0760 con 0x7fd9a8073060 2024-09-13T08:37:48.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.795+0000 7fd9c3400640 1 -- 172.21.15.44:0/4231284907 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7fd9bc05e7c0 con 0x7fd9a8073060 2024-09-13T08:37:48.802 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:48.803 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:37:48.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 -- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9a8073060 msgr2=0x7fd9a8075520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9a8073060 0x7fd9a8075520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009c60 tx=0x7fd9bc0091e0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 -- 172.21.15.44:0/4231284907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9cc06fc40 msgr2=0x7fd9cc1f3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:48.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9cc06fc40 0x7fd9cc1f3c50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd9b401ae50 tx=0x7fd9b4001420 comp rx=0 tx=0).stop 2024-09-13T08:37:48.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 -- 172.21.15.44:0/4231284907 shutdown_connections 2024-09-13T08:37:48.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9a8073060 0x7fd9a8075520 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9cc0700a0 0x7fd9cc1f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9cc06fc40 0x7fd9cc1f3c50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 --2- 172.21.15.44:0/4231284907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9b0004060 0x7fd9cc1f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 -- 172.21.15.44:0/4231284907 >> 172.21.15.44:0/4231284907 conn(0x7fd9cc0b1f10 msgr2=0x7fd9cc0cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 -- 172.21.15.44:0/4231284907 shutdown_connections 2024-09-13T08:37:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:48.799+0000 7fd9d2800640 1 -- 172.21.15.44:0/4231284907 wait complete. 2024-09-13T08:37:48.823 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-13T08:37:48.514218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219224,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755496,"statfs":{"total":773094113280,"available":772869627904,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003268"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877562+0000","last_change":"2024-09-13T08:37:34.901083+0000","last_active":"2024-09-13T08:37:36.877562+0000","last_peered":"2024-09-13T08:37:36.877562+0000","last_clean":"2024-09-13T08:37:36.877562+0000","last_became_active":"2024-09-13T08:37:34.900414+0000","last_became_peered":"2024-09-13T08:37:34.900414+0000","last_unstale":"2024-09-13T08:37:36.877562+0000","last_undegraded":"2024-09-13T08:37:36.877562+0000","last_fullsized":"2024-09-13T08:37:36.877562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:28:50.949101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896264+0000","last_change":"2024-09-13T08:37:34.900607+0000","last_active":"2024-09-13T08:37:35.896264+0000","last_peered":"2024-09-13T08:37:35.896264+0000","last_clean":"2024-09-13T08:37:35.896264+0000","last_became_active":"2024-09-13T08:37:34.900222+0000","last_became_peered":"2024-09-13T08:37:34.900222+0000","last_unstale":"2024-09-13T08:37:35.896264+0000","last_undegraded":"2024-09-13T08:37:35.896264+0000","last_fullsized":"2024-09-13T08:37:35.896264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:47.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895933+0000","last_change":"2024-09-13T08:37:34.906335+0000","last_active":"2024-09-13T08:37:35.895933+0000","last_peered":"2024-09-13T08:37:35.895933+0000","last_clean":"2024-09-13T08:37:35.895933+0000","last_became_active":"2024-09-13T08:37:34.905961+0000","last_became_peered":"2024-09-13T08:37:34.905961+0000","last_unstale":"2024-09-13T08:37:35.895933+0000","last_undegraded":"2024-09-13T08:37:35.895933+0000","last_fullsized":"2024-09-13T08:37:35.895933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:54.979663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510756+0000","last_change":"2024-09-13T08:37:34.904837+0000","last_active":"2024-09-13T08:37:36.510756+0000","last_peered":"2024-09-13T08:37:36.510756+0000","last_clean":"2024-09-13T08:37:36.510756+0000","last_became_active":"2024-09-13T08:37:34.898571+0000","last_became_peered":"2024-09-13T08:37:34.898571+0000","last_unstale":"2024-09-13T08:37:36.510756+0000","last_undegraded":"2024-09-13T08:37:36.510756+0000","last_fullsized":"2024-09-13T08:37:36.510756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877279+0000","last_change":"2024-09-13T08:37:34.888564+0000","last_active":"2024-09-13T08:37:36.877279+0000","last_peered":"2024-09-13T08:37:36.877279+0000","last_clean":"2024-09-13T08:37:36.877279+0000","last_became_active":"2024-09-13T08:37:34.888280+0000","last_became_peered":"2024-09-13T08:37:34.888280+0000","last_unstale":"2024-09-13T08:37:36.877279+0000","last_undegraded":"2024-09-13T08:37:36.877279+0000","last_fullsized":"2024-09-13T08:37:36.877279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:27:32.330596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510582+0000","last_change":"2024-09-13T08:37:34.895044+0000","last_active":"2024-09-13T08:37:36.510582+0000","last_peered":"2024-09-13T08:37:36.510582+0000","last_clean":"2024-09-13T08:37:36.510582+0000","last_became_active":"2024-09-13T08:37:34.894777+0000","last_became_peered":"2024-09-13T08:37:34.894777+0000","last_unstale":"2024-09-13T08:37:36.510582+0000","last_undegraded":"2024-09-13T08:37:36.510582+0000","last_fullsized":"2024-09-13T08:37:36.510582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877014+0000","last_change":"2024-09-13T08:37:34.888796+0000","last_active":"2024-09-13T08:37:36.877014+0000","last_peered":"2024-09-13T08:37:36.877014+0000","last_clean":"2024-09-13T08:37:36.877014+0000","last_became_active":"2024-09-13T08:37:34.888488+0000","last_became_peered":"2024-09-13T08:37:34.888488+0000","last_unstale":"2024-09-13T08:37:36.877014+0000","last_undegraded":"2024-09-13T08:37:36.877014+0000","last_fullsized":"2024-09-13T08:37:36.877014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894878+0000","last_change":"2024-09-13T08:37:34.910054+0000","last_active":"2024-09-13T08:37:35.894878+0000","last_peered":"2024-09-13T08:37:35.894878+0000","last_clean":"2024-09-13T08:37:35.894878+0000","last_became_active":"2024-09-13T08:37:34.909596+0000","last_became_peered":"2024-09-13T08:37:34.909596+0000","last_unstale":"2024-09-13T08:37:35.894878+0000","last_undegraded":"2024-09-13T08:37:35.894878+0000","last_fullsized":"2024-09-13T08:37:35.894878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:55.615355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892431+0000","last_change":"2024-09-13T08:37:34.885156+0000","last_active":"2024-09-13T08:37:35.892431+0000","last_peered":"2024-09-13T08:37:35.892431+0000","last_clean":"2024-09-13T08:37:35.892431+0000","last_became_active":"2024-09-13T08:37:34.884851+0000","last_became_peered":"2024-09-13T08:37:34.884851+0000","last_unstale":"2024-09-13T08:37:35.892431+0000","last_undegraded":"2024-09-13T08:37:35.892431+0000","last_fullsized":"2024-09-13T08:37:35.892431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:06.047971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877522+0000","last_change":"2024-09-13T08:37:34.909843+0000","last_active":"2024-09-13T08:37:36.877522+0000","last_peered":"2024-09-13T08:37:36.877522+0000","last_clean":"2024-09-13T08:37:36.877522+0000","last_became_active":"2024-09-13T08:37:34.909552+0000","last_became_peered":"2024-09-13T08:37:34.909552+0000","last_unstale":"2024-09-13T08:37:36.877522+0000","last_undegraded":"2024-09-13T08:37:36.877522+0000","last_fullsized":"2024-09-13T08:37:36.877522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:08.493271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877666+0000","last_change":"2024-09-13T08:37:34.902054+0000","last_active":"2024-09-13T08:37:36.877666+0000","last_peered":"2024-09-13T08:37:36.877666+0000","last_clean":"2024-09-13T08:37:36.877666+0000","last_became_active":"2024-09-13T08:37:34.901788+0000","last_became_peered":"2024-09-13T08:37:34.901788+0000","last_unstale":"2024-09-13T08:37:36.877666+0000","last_undegraded":"2024-09-13T08:37:36.877666+0000","last_fullsized":"2024-09-13T08:37:36.877666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:14.002956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877261+0000","last_change":"2024-09-13T08:37:34.892602+0000","last_active":"2024-09-13T08:37:36.877261+0000","last_peered":"2024-09-13T08:37:36.877261+0000","last_clean":"2024-09-13T08:37:36.877261+0000","last_became_active":"2024-09-13T08:37:34.890000+0000","last_became_peered":"2024-09-13T08:37:34.890000+0000","last_unstale":"2024-09-13T08:37:36.877261+0000","last_undegraded":"2024-09-13T08:37:36.877261+0000","last_fullsized":"2024-09-13T08:37:36.877261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:06.530173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894980+0000","last_change":"2024-09-13T08:37:34.903037+0000","last_active":"2024-09-13T08:37:35.894980+0000","last_peered":"2024-09-13T08:37:35.894980+0000","last_clean":"2024-09-13T08:37:35.894980+0000","last_became_active":"2024-09-13T08:37:34.894805+0000","last_became_peered":"2024-09-13T08:37:34.894805+0000","last_unstale":"2024-09-13T08:37:35.894980+0000","last_undegraded":"2024-09-13T08:37:35.894980+0000","last_fullsized":"2024-09-13T08:37:35.894980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:49.712241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894573+0000","last_change":"2024-09-13T08:37:34.901603+0000","last_active":"2024-09-13T08:37:35.894573+0000","last_peered":"2024-09-13T08:37:35.894573+0000","last_clean":"2024-09-13T08:37:35.894573+0000","last_became_active":"2024-09-13T08:37:34.899884+0000","last_became_peered":"2024-09-13T08:37:34.899884+0000","last_unstale":"2024-09-13T08:37:35.894573+0000","last_undegraded":"2024-09-13T08:37:35.894573+0000","last_fullsized":"2024-09-13T08:37:35.894573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:05.260600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895761+0000","last_change":"2024-09-13T08:37:34.896921+0000","last_active":"2024-09-13T08:37:35.895761+0000","last_peered":"2024-09-13T08:37:35.895761+0000","last_clean":"2024-09-13T08:37:35.895761+0000","last_became_active":"2024-09-13T08:37:34.896593+0000","last_became_peered":"2024-09-13T08:37:34.896593+0000","last_unstale":"2024-09-13T08:37:35.895761+0000","last_undegraded":"2024-09-13T08:37:35.895761+0000","last_fullsized":"2024-09-13T08:37:35.895761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:06.530173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084616+0000","last_change":"2024-09-13T08:37:34.908031+0000","last_active":"2024-09-13T08:37:37.084616+0000","last_peered":"2024-09-13T08:37:37.084616+0000","last_clean":"2024-09-13T08:37:37.084616+0000","last_became_active":"2024-09-13T08:37:34.907734+0000","last_became_peered":"2024-09-13T08:37:34.907734+0000","last_unstale":"2024-09-13T08:37:37.084616+0000","last_undegraded":"2024-09-13T08:37:37.084616+0000","last_fullsized":"2024-09-13T08:37:37.084616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:14.002956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085576+0000","last_change":"2024-09-13T08:37:34.911572+0000","last_active":"2024-09-13T08:37:37.085576+0000","last_peered":"2024-09-13T08:37:37.085576+0000","last_clean":"2024-09-13T08:37:37.085576+0000","last_became_active":"2024-09-13T08:37:34.911105+0000","last_became_peered":"2024-09-13T08:37:34.911105+0000","last_unstale":"2024-09-13T08:37:37.085576+0000","last_undegraded":"2024-09-13T08:37:37.085576+0000","last_fullsized":"2024-09-13T08:37:37.085576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:49:35.223067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.530220+0000","last_change":"2024-09-13T08:37:34.907767+0000","last_active":"2024-09-13T08:37:36.530220+0000","last_peered":"2024-09-13T08:37:36.530220+0000","last_clean":"2024-09-13T08:37:36.530220+0000","last_became_active":"2024-09-13T08:37:34.907100+0000","last_became_peered":"2024-09-13T08:37:34.907100+0000","last_unstale":"2024-09-13T08:37:36.530220+0000","last_undegraded":"2024-09-13T08:37:36.530220+0000","last_fullsized":"2024-09-13T08:37:36.530220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085543+0000","last_change":"2024-09-13T08:37:34.895822+0000","last_active":"2024-09-13T08:37:37.085543+0000","last_peered":"2024-09-13T08:37:37.085543+0000","last_clean":"2024-09-13T08:37:37.085543+0000","last_became_active":"2024-09-13T08:37:34.895430+0000","last_became_peered":"2024-09-13T08:37:34.895430+0000","last_unstale":"2024-09-13T08:37:37.085543+0000","last_undegraded":"2024-09-13T08:37:37.085543+0000","last_fullsized":"2024-09-13T08:37:37.085543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:33:54.897556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877407+0000","last_change":"2024-09-13T08:37:34.891580+0000","last_active":"2024-09-13T08:37:36.877407+0000","last_peered":"2024-09-13T08:37:36.877407+0000","last_clean":"2024-09-13T08:37:36.877407+0000","last_became_active":"2024-09-13T08:37:34.889710+0000","last_became_peered":"2024-09-13T08:37:34.889710+0000","last_unstale":"2024-09-13T08:37:36.877407+0000","last_undegraded":"2024-09-13T08:37:36.877407+0000","last_fullsized":"2024-09-13T08:37:36.877407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:47.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585799+0000","last_change":"2024-09-13T08:37:34.894615+0000","last_active":"2024-09-13T08:37:36.585799+0000","last_peered":"2024-09-13T08:37:36.585799+0000","last_clean":"2024-09-13T08:37:36.585799+0000","last_became_active":"2024-09-13T08:37:34.894327+0000","last_became_peered":"2024-09-13T08:37:34.894327+0000","last_unstale":"2024-09-13T08:37:36.585799+0000","last_undegraded":"2024-09-13T08:37:36.585799+0000","last_fullsized":"2024-09-13T08:37:36.585799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:55.615355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891945+0000","last_change":"2024-09-13T08:37:34.883272+0000","last_active":"2024-09-13T08:37:35.891945+0000","last_peered":"2024-09-13T08:37:35.891945+0000","last_clean":"2024-09-13T08:37:35.891945+0000","last_became_active":"2024-09-13T08:37:34.882941+0000","last_became_peered":"2024-09-13T08:37:34.882941+0000","last_unstale":"2024-09-13T08:37:35.891945+0000","last_undegraded":"2024-09-13T08:37:35.891945+0000","last_fullsized":"2024-09-13T08:37:35.891945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895545+0000","last_change":"2024-09-13T08:37:34.903123+0000","last_active":"2024-09-13T08:37:35.895545+0000","last_peered":"2024-09-13T08:37:35.895545+0000","last_clean":"2024-09-13T08:37:35.895545+0000","last_became_active":"2024-09-13T08:37:34.901167+0000","last_became_peered":"2024-09-13T08:37:34.901167+0000","last_unstale":"2024-09-13T08:37:35.895545+0000","last_undegraded":"2024-09-13T08:37:35.895545+0000","last_fullsized":"2024-09-13T08:37:35.895545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084614+0000","last_change":"2024-09-13T08:37:34.891584+0000","last_active":"2024-09-13T08:37:37.084614+0000","last_peered":"2024-09-13T08:37:37.084614+0000","last_clean":"2024-09-13T08:37:37.084614+0000","last_became_active":"2024-09-13T08:37:34.891268+0000","last_became_peered":"2024-09-13T08:37:34.891268+0000","last_unstale":"2024-09-13T08:37:37.084614+0000","last_undegraded":"2024-09-13T08:37:37.084614+0000","last_fullsized":"2024-09-13T08:37:37.084614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:59.406808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896325+0000","last_change":"2024-09-13T08:37:34.902079+0000","last_active":"2024-09-13T08:37:35.896325+0000","last_peered":"2024-09-13T08:37:35.896325+0000","last_clean":"2024-09-13T08:37:35.896325+0000","last_became_active":"2024-09-13T08:37:34.900499+0000","last_became_peered":"2024-09-13T08:37:34.900499+0000","last_unstale":"2024-09-13T08:37:35.896325+0000","last_undegraded":"2024-09-13T08:37:35.896325+0000","last_fullsized":"2024-09-13T08:37:35.896325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:45.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511137+0000","last_change":"2024-09-13T08:37:34.888787+0000","last_active":"2024-09-13T08:37:36.511137+0000","last_peered":"2024-09-13T08:37:36.511137+0000","last_clean":"2024-09-13T08:37:36.511137+0000","last_became_active":"2024-09-13T08:37:34.888486+0000","last_became_peered":"2024-09-13T08:37:34.888486+0000","last_unstale":"2024-09-13T08:37:36.511137+0000","last_undegraded":"2024-09-13T08:37:36.511137+0000","last_fullsized":"2024-09-13T08:37:36.511137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:56.908418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895541+0000","last_change":"2024-09-13T08:37:34.902546+0000","last_active":"2024-09-13T08:37:35.895541+0000","last_peered":"2024-09-13T08:37:35.895541+0000","last_clean":"2024-09-13T08:37:35.895541+0000","last_became_active":"2024-09-13T08:37:34.901841+0000","last_became_peered":"2024-09-13T08:37:34.901841+0000","last_unstale":"2024-09-13T08:37:35.895541+0000","last_undegraded":"2024-09-13T08:37:35.895541+0000","last_fullsized":"2024-09-13T08:37:35.895541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:40.635073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895078+0000","last_change":"2024-09-13T08:37:34.907277+0000","last_active":"2024-09-13T08:37:35.895078+0000","last_peered":"2024-09-13T08:37:35.895078+0000","last_clean":"2024-09-13T08:37:35.895078+0000","last_became_active":"2024-09-13T08:37:34.906984+0000","last_became_peered":"2024-09-13T08:37:34.906984+0000","last_unstale":"2024-09-13T08:37:35.895078+0000","last_undegraded":"2024-09-13T08:37:35.895078+0000","last_fullsized":"2024-09-13T08:37:35.895078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:23.664559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895840+0000","last_change":"2024-09-13T08:37:34.902627+0000","last_active":"2024-09-13T08:37:35.895840+0000","last_peered":"2024-09-13T08:37:35.895840+0000","last_clean":"2024-09-13T08:37:35.895840+0000","last_became_active":"2024-09-13T08:37:34.901894+0000","last_became_peered":"2024-09-13T08:37:34.901894+0000","last_unstale":"2024-09-13T08:37:35.895840+0000","last_undegraded":"2024-09-13T08:37:35.895840+0000","last_fullsized":"2024-09-13T08:37:35.895840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:06.723541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084874+0000","last_change":"2024-09-13T08:37:34.891317+0000","last_active":"2024-09-13T08:37:37.084874+0000","last_peered":"2024-09-13T08:37:37.084874+0000","last_clean":"2024-09-13T08:37:37.084874+0000","last_became_active":"2024-09-13T08:37:34.891030+0000","last_became_peered":"2024-09-13T08:37:34.891030+0000","last_unstale":"2024-09-13T08:37:37.084874+0000","last_undegraded":"2024-09-13T08:37:37.084874+0000","last_fullsized":"2024-09-13T08:37:37.084874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:46.738660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529933+0000","last_change":"2024-09-13T08:37:34.894275+0000","last_active":"2024-09-13T08:37:36.529933+0000","last_peered":"2024-09-13T08:37:36.529933+0000","last_clean":"2024-09-13T08:37:36.529933+0000","last_became_active":"2024-09-13T08:37:34.893030+0000","last_became_peered":"2024-09-13T08:37:34.893030+0000","last_unstale":"2024-09-13T08:37:36.529933+0000","last_undegraded":"2024-09-13T08:37:36.529933+0000","last_fullsized":"2024-09-13T08:37:36.529933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510646+0000","last_change":"2024-09-13T08:37:34.900318+0000","last_active":"2024-09-13T08:37:36.510646+0000","last_peered":"2024-09-13T08:37:36.510646+0000","last_clean":"2024-09-13T08:37:36.510646+0000","last_became_active":"2024-09-13T08:37:34.899949+0000","last_became_peered":"2024-09-13T08:37:34.899949+0000","last_unstale":"2024-09-13T08:37:36.510646+0000","last_undegraded":"2024-09-13T08:37:36.510646+0000","last_fullsized":"2024-09-13T08:37:36.510646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892117+0000","last_change":"2024-09-13T08:37:34.889876+0000","last_active":"2024-09-13T08:37:35.892117+0000","last_peered":"2024-09-13T08:37:35.892117+0000","last_clean":"2024-09-13T08:37:35.892117+0000","last_became_active":"2024-09-13T08:37:34.889622+0000","last_became_peered":"2024-09-13T08:37:34.889622+0000","last_unstale":"2024-09-13T08:37:35.892117+0000","last_undegraded":"2024-09-13T08:37:35.892117+0000","last_fullsized":"2024-09-13T08:37:35.892117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895867+0000","last_change":"2024-09-13T08:37:33.888852+0000","last_active":"2024-09-13T08:37:35.895867+0000","last_peered":"2024-09-13T08:37:35.895867+0000","last_clean":"2024-09-13T08:37:35.895867+0000","last_became_active":"2024-09-13T08:37:33.888409+0000","last_became_peered":"2024-09-13T08:37:33.888409+0000","last_unstale":"2024-09-13T08:37:35.895867+0000","last_undegraded":"2024-09-13T08:37:35.895867+0000","last_fullsized":"2024-09-13T08:37:35.895867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585747+0000","last_change":"2024-09-13T08:37:34.890809+0000","last_active":"2024-09-13T08:37:36.585747+0000","last_peered":"2024-09-13T08:37:36.585747+0000","last_clean":"2024-09-13T08:37:36.585747+0000","last_became_active":"2024-09-13T08:37:34.887213+0000","last_became_peered":"2024-09-13T08:37:34.887213+0000","last_unstale":"2024-09-13T08:37:36.585747+0000","last_undegraded":"2024-09-13T08:37:36.585747+0000","last_fullsized":"2024-09-13T08:37:36.585747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:19:06.310722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085055+0000","last_change":"2024-09-13T08:37:33.892308+0000","last_active":"2024-09-13T08:37:37.085055+0000","last_peered":"2024-09-13T08:37:37.085055+0000","last_clean":"2024-09-13T08:37:37.085055+0000","last_became_active":"2024-09-13T08:37:33.891875+0000","last_became_peered":"2024-09-13T08:37:33.891875+0000","last_unstale":"2024-09-13T08:37:37.085055+0000","last_undegraded":"2024-09-13T08:37:37.085055+0000","last_fullsized":"2024-09-13T08:37:37.085055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:05.039750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894251+0000","last_change":"2024-09-13T08:37:33.896561+0000","last_active":"2024-09-13T08:37:35.894251+0000","last_peered":"2024-09-13T08:37:35.894251+0000","last_clean":"2024-09-13T08:37:35.894251+0000","last_became_active":"2024-09-13T08:37:33.896314+0000","last_became_peered":"2024-09-13T08:37:33.896314+0000","last_unstale":"2024-09-13T08:37:35.894251+0000","last_undegraded":"2024-09-13T08:37:35.894251+0000","last_fullsized":"2024-09-13T08:37:35.894251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:28.728254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895122+0000","last_change":"2024-09-13T08:37:34.900565+0000","last_active":"2024-09-13T08:37:35.895122+0000","last_peered":"2024-09-13T08:37:35.895122+0000","last_clean":"2024-09-13T08:37:35.895122+0000","last_became_active":"2024-09-13T08:37:34.900257+0000","last_became_peered":"2024-09-13T08:37:34.900257+0000","last_unstale":"2024-09-13T08:37:35.895122+0000","last_undegraded":"2024-09-13T08:37:35.895122+0000","last_fullsized":"2024-09-13T08:37:35.895122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:34.291551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084825+0000","last_change":"2024-09-13T08:37:33.883156+0000","last_active":"2024-09-13T08:37:37.084825+0000","last_peered":"2024-09-13T08:37:37.084825+0000","last_clean":"2024-09-13T08:37:37.084825+0000","last_became_active":"2024-09-13T08:37:33.882800+0000","last_became_peered":"2024-09-13T08:37:33.882800+0000","last_unstale":"2024-09-13T08:37:37.084825+0000","last_undegraded":"2024-09-13T08:37:37.084825+0000","last_fullsized":"2024-09-13T08:37:37.084825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:10.413313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585694+0000","last_change":"2024-09-13T08:37:34.888188+0000","last_active":"2024-09-13T08:37:36.585694+0000","last_peered":"2024-09-13T08:37:36.585694+0000","last_clean":"2024-09-13T08:37:36.585694+0000","last_became_active":"2024-09-13T08:37:34.887475+0000","last_became_peered":"2024-09-13T08:37:34.887475+0000","last_unstale":"2024-09-13T08:37:36.585694+0000","last_undegraded":"2024-09-13T08:37:36.585694+0000","last_fullsized":"2024-09-13T08:37:36.585694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:14.743221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892396+0000","last_change":"2024-09-13T08:37:33.891385+0000","last_active":"2024-09-13T08:37:35.892396+0000","last_peered":"2024-09-13T08:37:35.892396+0000","last_clean":"2024-09-13T08:37:35.892396+0000","last_became_active":"2024-09-13T08:37:33.891112+0000","last_became_peered":"2024-09-13T08:37:33.891112+0000","last_unstale":"2024-09-13T08:37:35.892396+0000","last_undegraded":"2024-09-13T08:37:35.892396+0000","last_fullsized":"2024-09-13T08:37:35.892396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511114+0000","last_change":"2024-09-13T08:37:34.899512+0000","last_active":"2024-09-13T08:37:36.511114+0000","last_peered":"2024-09-13T08:37:36.511114+0000","last_clean":"2024-09-13T08:37:36.511114+0000","last_became_active":"2024-09-13T08:37:34.887883+0000","last_became_peered":"2024-09-13T08:37:34.887883+0000","last_unstale":"2024-09-13T08:37:36.511114+0000","last_undegraded":"2024-09-13T08:37:36.511114+0000","last_fullsized":"2024-09-13T08:37:36.511114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:56.149863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877483+0000","last_change":"2024-09-13T08:37:33.899147+0000","last_active":"2024-09-13T08:37:36.877483+0000","last_peered":"2024-09-13T08:37:36.877483+0000","last_clean":"2024-09-13T08:37:36.877483+0000","last_became_active":"2024-09-13T08:37:33.896929+0000","last_became_peered":"2024-09-13T08:37:33.896929+0000","last_unstale":"2024-09-13T08:37:36.877483+0000","last_undegraded":"2024-09-13T08:37:36.877483+0000","last_fullsized":"2024-09-13T08:37:36.877483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:08:15.453880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896504+0000","last_change":"2024-09-13T08:37:34.900790+0000","last_active":"2024-09-13T08:37:35.896504+0000","last_peered":"2024-09-13T08:37:35.896504+0000","last_clean":"2024-09-13T08:37:35.896504+0000","last_became_active":"2024-09-13T08:37:34.898939+0000","last_became_peered":"2024-09-13T08:37:34.898939+0000","last_unstale":"2024-09-13T08:37:35.896504+0000","last_undegraded":"2024-09-13T08:37:35.896504+0000","last_fullsized":"2024-09-13T08:37:35.896504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:07.812549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.530096+0000","last_change":"2024-09-13T08:37:33.892872+0000","last_active":"2024-09-13T08:37:36.530096+0000","last_peered":"2024-09-13T08:37:36.530096+0000","last_clean":"2024-09-13T08:37:36.530096+0000","last_became_active":"2024-09-13T08:37:33.892579+0000","last_became_peered":"2024-09-13T08:37:33.892579+0000","last_unstale":"2024-09-13T08:37:36.530096+0000","last_undegraded":"2024-09-13T08:37:36.530096+0000","last_fullsized":"2024-09-13T08:37:36.530096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085280+0000","last_change":"2024-09-13T08:37:34.887215+0000","last_active":"2024-09-13T08:37:37.085280+0000","last_peered":"2024-09-13T08:37:37.085280+0000","last_clean":"2024-09-13T08:37:37.085280+0000","last_became_active":"2024-09-13T08:37:34.886889+0000","last_became_peered":"2024-09-13T08:37:34.886889+0000","last_unstale":"2024-09-13T08:37:37.085280+0000","last_undegraded":"2024-09-13T08:37:37.085280+0000","last_fullsized":"2024-09-13T08:37:37.085280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:57.113272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510921+0000","last_change":"2024-09-13T08:37:33.891105+0000","last_active":"2024-09-13T08:37:36.510921+0000","last_peered":"2024-09-13T08:37:36.510921+0000","last_clean":"2024-09-13T08:37:36.510921+0000","last_became_active":"2024-09-13T08:37:33.890783+0000","last_became_peered":"2024-09-13T08:37:33.890783+0000","last_unstale":"2024-09-13T08:37:36.510921+0000","last_undegraded":"2024-09-13T08:37:36.510921+0000","last_fullsized":"2024-09-13T08:37:36.510921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:30:45.112157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877636+0000","last_change":"2024-09-13T08:37:34.893268+0000","last_active":"2024-09-13T08:37:36.877636+0000","last_peered":"2024-09-13T08:37:36.877636+0000","last_clean":"2024-09-13T08:37:36.877636+0000","last_became_active":"2024-09-13T08:37:34.892944+0000","last_became_peered":"2024-09-13T08:37:34.892944+0000","last_unstale":"2024-09-13T08:37:36.877636+0000","last_undegraded":"2024-09-13T08:37:36.877636+0000","last_fullsized":"2024-09-13T08:37:36.877636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:45.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529953+0000","last_change":"2024-09-13T08:37:33.902610+0000","last_active":"2024-09-13T08:37:36.529953+0000","last_peered":"2024-09-13T08:37:36.529953+0000","last_clean":"2024-09-13T08:37:36.529953+0000","last_became_active":"2024-09-13T08:37:33.902275+0000","last_became_peered":"2024-09-13T08:37:33.902275+0000","last_unstale":"2024-09-13T08:37:36.529953+0000","last_undegraded":"2024-09-13T08:37:36.529953+0000","last_fullsized":"2024-09-13T08:37:36.529953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894679+0000","last_change":"2024-09-13T08:37:34.904983+0000","last_active":"2024-09-13T08:37:35.894679+0000","last_peered":"2024-09-13T08:37:35.894679+0000","last_clean":"2024-09-13T08:37:35.894679+0000","last_became_active":"2024-09-13T08:37:34.904618+0000","last_became_peered":"2024-09-13T08:37:34.904618+0000","last_unstale":"2024-09-13T08:37:35.894679+0000","last_undegraded":"2024-09-13T08:37:35.894679+0000","last_fullsized":"2024-09-13T08:37:35.894679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:30.070140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085096+0000","last_change":"2024-09-13T08:37:33.888874+0000","last_active":"2024-09-13T08:37:37.085096+0000","last_peered":"2024-09-13T08:37:37.085096+0000","last_clean":"2024-09-13T08:37:37.085096+0000","last_became_active":"2024-09-13T08:37:33.888111+0000","last_became_peered":"2024-09-13T08:37:33.888111+0000","last_unstale":"2024-09-13T08:37:37.085096+0000","last_undegraded":"2024-09-13T08:37:37.085096+0000","last_fullsized":"2024-09-13T08:37:37.085096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:55.141642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084977+0000","last_change":"2024-09-13T08:37:34.893432+0000","last_active":"2024-09-13T08:37:37.084977+0000","last_peered":"2024-09-13T08:37:37.084977+0000","last_clean":"2024-09-13T08:37:37.084977+0000","last_became_active":"2024-09-13T08:37:34.893160+0000","last_became_peered":"2024-09-13T08:37:34.893160+0000","last_unstale":"2024-09-13T08:37:37.084977+0000","last_undegraded":"2024-09-13T08:37:37.084977+0000","last_fullsized":"2024-09-13T08:37:37.084977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:46.900298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.876999+0000","last_change":"2024-09-13T08:37:33.892008+0000","last_active":"2024-09-13T08:37:36.876999+0000","last_peered":"2024-09-13T08:37:36.876999+0000","last_clean":"2024-09-13T08:37:36.876999+0000","last_became_active":"2024-09-13T08:37:33.891758+0000","last_became_peered":"2024-09-13T08:37:33.891758+0000","last_unstale":"2024-09-13T08:37:36.876999+0000","last_undegraded":"2024-09-13T08:37:36.876999+0000","last_fullsized":"2024-09-13T08:37:36.876999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:27.615101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891708+0000","last_change":"2024-09-13T08:37:34.889786+0000","last_active":"2024-09-13T08:37:35.891708+0000","last_peered":"2024-09-13T08:37:35.891708+0000","last_clean":"2024-09-13T08:37:35.891708+0000","last_became_active":"2024-09-13T08:37:34.889505+0000","last_became_peered":"2024-09-13T08:37:34.889505+0000","last_unstale":"2024-09-13T08:37:35.891708+0000","last_undegraded":"2024-09-13T08:37:35.891708+0000","last_fullsized":"2024-09-13T08:37:35.891708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:50.000602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585415+0000","last_change":"2024-09-13T08:37:33.888556+0000","last_active":"2024-09-13T08:37:36.585415+0000","last_peered":"2024-09-13T08:37:36.585415+0000","last_clean":"2024-09-13T08:37:36.585415+0000","last_became_active":"2024-09-13T08:37:33.887997+0000","last_became_peered":"2024-09-13T08:37:33.887997+0000","last_unstale":"2024-09-13T08:37:36.585415+0000","last_undegraded":"2024-09-13T08:37:36.585415+0000","last_fullsized":"2024-09-13T08:37:36.585415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084670+0000","last_change":"2024-09-13T08:37:34.908412+0000","last_active":"2024-09-13T08:37:37.084670+0000","last_peered":"2024-09-13T08:37:37.084670+0000","last_clean":"2024-09-13T08:37:37.084670+0000","last_became_active":"2024-09-13T08:37:34.908135+0000","last_became_peered":"2024-09-13T08:37:34.908135+0000","last_unstale":"2024-09-13T08:37:37.084670+0000","last_undegraded":"2024-09-13T08:37:37.084670+0000","last_fullsized":"2024-09-13T08:37:37.084670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:15.303677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877915+0000","last_change":"2024-09-13T08:37:33.895134+0000","last_active":"2024-09-13T08:37:36.877915+0000","last_peered":"2024-09-13T08:37:36.877915+0000","last_clean":"2024-09-13T08:37:36.877915+0000","last_became_active":"2024-09-13T08:37:33.894893+0000","last_became_peered":"2024-09-13T08:37:33.894893+0000","last_unstale":"2024-09-13T08:37:36.877915+0000","last_undegraded":"2024-09-13T08:37:36.877915+0000","last_fullsized":"2024-09-13T08:37:36.877915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:03:36.459606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511239+0000","last_change":"2024-09-13T08:37:34.915800+0000","last_active":"2024-09-13T08:37:36.511239+0000","last_peered":"2024-09-13T08:37:36.511239+0000","last_clean":"2024-09-13T08:37:36.511239+0000","last_became_active":"2024-09-13T08:37:34.915478+0000","last_became_peered":"2024-09-13T08:37:34.915478+0000","last_unstale":"2024-09-13T08:37:36.511239+0000","last_undegraded":"2024-09-13T08:37:36.511239+0000","last_fullsized":"2024-09-13T08:37:36.511239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:55.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586002+0000","last_change":"2024-09-13T08:37:33.888156+0000","last_active":"2024-09-13T08:37:36.586002+0000","last_peered":"2024-09-13T08:37:36.586002+0000","last_clean":"2024-09-13T08:37:36.586002+0000","last_became_active":"2024-09-13T08:37:33.886607+0000","last_became_peered":"2024-09-13T08:37:33.886607+0000","last_unstale":"2024-09-13T08:37:36.586002+0000","last_undegraded":"2024-09-13T08:37:36.586002+0000","last_fullsized":"2024-09-13T08:37:36.586002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:36.540294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877863+0000","last_change":"2024-09-13T08:37:34.891196+0000","last_active":"2024-09-13T08:37:36.877863+0000","last_peered":"2024-09-13T08:37:36.877863+0000","last_clean":"2024-09-13T08:37:36.877863+0000","last_became_active":"2024-09-13T08:37:34.890848+0000","last_became_peered":"2024-09-13T08:37:34.890848+0000","last_unstale":"2024-09-13T08:37:36.877863+0000","last_undegraded":"2024-09-13T08:37:36.877863+0000","last_fullsized":"2024-09-13T08:37:36.877863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:07.683461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896137+0000","last_change":"2024-09-13T08:37:33.885405+0000","last_active":"2024-09-13T08:37:35.896137+0000","last_peered":"2024-09-13T08:37:35.896137+0000","last_clean":"2024-09-13T08:37:35.896137+0000","last_became_active":"2024-09-13T08:37:33.885048+0000","last_became_peered":"2024-09-13T08:37:33.885048+0000","last_unstale":"2024-09-13T08:37:35.896137+0000","last_undegraded":"2024-09-13T08:37:35.896137+0000","last_fullsized":"2024-09-13T08:37:35.896137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:33:47.115618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877736+0000","last_change":"2024-09-13T08:37:34.898475+0000","last_active":"2024-09-13T08:37:36.877736+0000","last_peered":"2024-09-13T08:37:36.877736+0000","last_clean":"2024-09-13T08:37:36.877736+0000","last_became_active":"2024-09-13T08:37:34.897854+0000","last_became_peered":"2024-09-13T08:37:34.897854+0000","last_unstale":"2024-09-13T08:37:36.877736+0000","last_undegraded":"2024-09-13T08:37:36.877736+0000","last_fullsized":"2024-09-13T08:37:36.877736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:15.303677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895446+0000","last_change":"2024-09-13T08:37:33.902815+0000","last_active":"2024-09-13T08:37:35.895446+0000","last_peered":"2024-09-13T08:37:35.895446+0000","last_clean":"2024-09-13T08:37:35.895446+0000","last_became_active":"2024-09-13T08:37:33.902490+0000","last_became_peered":"2024-09-13T08:37:33.902490+0000","last_unstale":"2024-09-13T08:37:35.895446+0000","last_undegraded":"2024-09-13T08:37:35.895446+0000","last_fullsized":"2024-09-13T08:37:35.895446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:32.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085416+0000","last_change":"2024-09-13T08:37:34.891378+0000","last_active":"2024-09-13T08:37:37.085416+0000","last_peered":"2024-09-13T08:37:37.085416+0000","last_clean":"2024-09-13T08:37:37.085416+0000","last_became_active":"2024-09-13T08:37:34.885687+0000","last_became_peered":"2024-09-13T08:37:34.885687+0000","last_unstale":"2024-09-13T08:37:37.085416+0000","last_undegraded":"2024-09-13T08:37:37.085416+0000","last_fullsized":"2024-09-13T08:37:37.085416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:42.639292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891693+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:37:35.891693+0000","last_peered":"2024-09-13T08:37:35.891693+0000","last_clean":"2024-09-13T08:37:35.891693+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:37:35.891693+0000","last_undegraded":"2024-09-13T08:37:35.891693+0000","last_fullsized":"2024-09-13T08:37:35.891693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085218+0000","last_change":"2024-09-13T08:37:33.883826+0000","last_active":"2024-09-13T08:37:37.085218+0000","last_peered":"2024-09-13T08:37:37.085218+0000","last_clean":"2024-09-13T08:37:37.085218+0000","last_became_active":"2024-09-13T08:37:33.883511+0000","last_became_peered":"2024-09-13T08:37:33.883511+0000","last_unstale":"2024-09-13T08:37:37.085218+0000","last_undegraded":"2024-09-13T08:37:37.085218+0000","last_fullsized":"2024-09-13T08:37:37.085218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:39:48.743211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585719+0000","last_change":"2024-09-13T08:37:34.901372+0000","last_active":"2024-09-13T08:37:36.585719+0000","last_peered":"2024-09-13T08:37:36.585719+0000","last_clean":"2024-09-13T08:37:36.585719+0000","last_became_active":"2024-09-13T08:37:34.900931+0000","last_became_peered":"2024-09-13T08:37:34.900931+0000","last_unstale":"2024-09-13T08:37:36.585719+0000","last_undegraded":"2024-09-13T08:37:36.585719+0000","last_fullsized":"2024-09-13T08:37:36.585719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:16.328343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084832+0000","last_change":"2024-09-13T08:37:33.885219+0000","last_active":"2024-09-13T08:37:37.084832+0000","last_peered":"2024-09-13T08:37:37.084832+0000","last_clean":"2024-09-13T08:37:37.084832+0000","last_became_active":"2024-09-13T08:37:33.884889+0000","last_became_peered":"2024-09-13T08:37:33.884889+0000","last_unstale":"2024-09-13T08:37:37.084832+0000","last_undegraded":"2024-09-13T08:37:37.084832+0000","last_fullsized":"2024-09-13T08:37:37.084832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:55.495409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894451+0000","last_change":"2024-09-13T08:37:34.908186+0000","last_active":"2024-09-13T08:37:35.894451+0000","last_peered":"2024-09-13T08:37:35.894451+0000","last_clean":"2024-09-13T08:37:35.894451+0000","last_became_active":"2024-09-13T08:37:34.907886+0000","last_became_peered":"2024-09-13T08:37:34.907886+0000","last_unstale":"2024-09-13T08:37:35.894451+0000","last_undegraded":"2024-09-13T08:37:35.894451+0000","last_fullsized":"2024-09-13T08:37:35.894451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:11:14.712673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529782+0000","last_change":"2024-09-13T08:37:33.890556+0000","last_active":"2024-09-13T08:37:36.529782+0000","last_peered":"2024-09-13T08:37:36.529782+0000","last_clean":"2024-09-13T08:37:36.529782+0000","last_became_active":"2024-09-13T08:37:33.890252+0000","last_became_peered":"2024-09-13T08:37:33.890252+0000","last_unstale":"2024-09-13T08:37:36.529782+0000","last_undegraded":"2024-09-13T08:37:36.529782+0000","last_fullsized":"2024-09-13T08:37:36.529782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894861+0000","last_change":"2024-09-13T08:37:34.901633+0000","last_active":"2024-09-13T08:37:35.894861+0000","last_peered":"2024-09-13T08:37:35.894861+0000","last_clean":"2024-09-13T08:37:35.894861+0000","last_became_active":"2024-09-13T08:37:34.901323+0000","last_became_peered":"2024-09-13T08:37:34.901323+0000","last_unstale":"2024-09-13T08:37:35.894861+0000","last_undegraded":"2024-09-13T08:37:35.894861+0000","last_fullsized":"2024-09-13T08:37:35.894861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:37.744328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085159+0000","last_change":"2024-09-13T08:37:33.889192+0000","last_active":"2024-09-13T08:37:37.085159+0000","last_peered":"2024-09-13T08:37:37.085159+0000","last_clean":"2024-09-13T08:37:37.085159+0000","last_became_active":"2024-09-13T08:37:33.888556+0000","last_became_peered":"2024-09-13T08:37:33.888556+0000","last_unstale":"2024-09-13T08:37:37.085159+0000","last_undegraded":"2024-09-13T08:37:37.085159+0000","last_fullsized":"2024-09-13T08:37:37.085159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:55.900198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877366+0000","last_change":"2024-09-13T08:37:34.905109+0000","last_active":"2024-09-13T08:37:36.877366+0000","last_peered":"2024-09-13T08:37:36.877366+0000","last_clean":"2024-09-13T08:37:36.877366+0000","last_became_active":"2024-09-13T08:37:34.904766+0000","last_became_peered":"2024-09-13T08:37:34.904766+0000","last_unstale":"2024-09-13T08:37:36.877366+0000","last_undegraded":"2024-09-13T08:37:36.877366+0000","last_fullsized":"2024-09-13T08:37:36.877366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:54.979663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895896+0000","last_change":"2024-09-13T08:37:33.889699+0000","last_active":"2024-09-13T08:37:35.895896+0000","last_peered":"2024-09-13T08:37:35.895896+0000","last_clean":"2024-09-13T08:37:35.895896+0000","last_became_active":"2024-09-13T08:37:33.889224+0000","last_became_peered":"2024-09-13T08:37:33.889224+0000","last_unstale":"2024-09-13T08:37:35.895896+0000","last_undegraded":"2024-09-13T08:37:35.895896+0000","last_fullsized":"2024-09-13T08:37:35.895896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:24.413843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510786+0000","last_change":"2024-09-13T08:37:34.901224+0000","last_active":"2024-09-13T08:37:36.510786+0000","last_peered":"2024-09-13T08:37:36.510786+0000","last_clean":"2024-09-13T08:37:36.510786+0000","last_became_active":"2024-09-13T08:37:34.900890+0000","last_became_peered":"2024-09-13T08:37:34.900890+0000","last_unstale":"2024-09-13T08:37:36.510786+0000","last_undegraded":"2024-09-13T08:37:36.510786+0000","last_fullsized":"2024-09-13T08:37:36.510786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:06.047971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085567+0000","last_change":"2024-09-13T08:37:33.893542+0000","last_active":"2024-09-13T08:37:37.085567+0000","last_peered":"2024-09-13T08:37:37.085567+0000","last_clean":"2024-09-13T08:37:37.085567+0000","last_became_active":"2024-09-13T08:37:33.893231+0000","last_became_peered":"2024-09-13T08:37:33.893231+0000","last_unstale":"2024-09-13T08:37:37.085567+0000","last_undegraded":"2024-09-13T08:37:37.085567+0000","last_fullsized":"2024-09-13T08:37:37.085567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:48.992381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895292+0000","last_change":"2024-09-13T08:37:34.901999+0000","last_active":"2024-09-13T08:37:35.895292+0000","last_peered":"2024-09-13T08:37:35.895292+0000","last_clean":"2024-09-13T08:37:35.895292+0000","last_became_active":"2024-09-13T08:37:34.901765+0000","last_became_peered":"2024-09-13T08:37:34.901765+0000","last_unstale":"2024-09-13T08:37:35.895292+0000","last_undegraded":"2024-09-13T08:37:35.895292+0000","last_fullsized":"2024-09-13T08:37:35.895292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:12.422693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586075+0000","last_change":"2024-09-13T08:37:33.898217+0000","last_active":"2024-09-13T08:37:36.586075+0000","last_peered":"2024-09-13T08:37:36.586075+0000","last_clean":"2024-09-13T08:37:36.586075+0000","last_became_active":"2024-09-13T08:37:33.897954+0000","last_became_peered":"2024-09-13T08:37:33.897954+0000","last_unstale":"2024-09-13T08:37:36.586075+0000","last_undegraded":"2024-09-13T08:37:36.586075+0000","last_fullsized":"2024-09-13T08:37:36.586075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:05.995257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586135+0000","last_change":"2024-09-13T08:37:34.897630+0000","last_active":"2024-09-13T08:37:36.586135+0000","last_peered":"2024-09-13T08:37:36.586135+0000","last_clean":"2024-09-13T08:37:36.586135+0000","last_became_active":"2024-09-13T08:37:34.896603+0000","last_became_peered":"2024-09-13T08:37:34.896603+0000","last_unstale":"2024-09-13T08:37:36.586135+0000","last_undegraded":"2024-09-13T08:37:36.586135+0000","last_fullsized":"2024-09-13T08:37:36.586135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085624+0000","last_change":"2024-09-13T08:37:33.897139+0000","last_active":"2024-09-13T08:37:37.085624+0000","last_peered":"2024-09-13T08:37:37.085624+0000","last_clean":"2024-09-13T08:37:37.085624+0000","last_became_active":"2024-09-13T08:37:33.896576+0000","last_became_peered":"2024-09-13T08:37:33.896576+0000","last_unstale":"2024-09-13T08:37:37.085624+0000","last_undegraded":"2024-09-13T08:37:37.085624+0000","last_fullsized":"2024-09-13T08:37:37.085624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:54.815810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585844+0000","last_change":"2024-09-13T08:37:34.896701+0000","last_active":"2024-09-13T08:37:36.585844+0000","last_peered":"2024-09-13T08:37:36.585844+0000","last_clean":"2024-09-13T08:37:36.585844+0000","last_became_active":"2024-09-13T08:37:34.893772+0000","last_became_peered":"2024-09-13T08:37:34.893772+0000","last_unstale":"2024-09-13T08:37:36.585844+0000","last_undegraded":"2024-09-13T08:37:36.585844+0000","last_fullsized":"2024-09-13T08:37:36.585844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:34.291551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.876764+0000","last_change":"2024-09-13T08:37:33.893088+0000","last_active":"2024-09-13T08:37:36.876764+0000","last_peered":"2024-09-13T08:37:36.876764+0000","last_clean":"2024-09-13T08:37:36.876764+0000","last_became_active":"2024-09-13T08:37:33.892731+0000","last_became_peered":"2024-09-13T08:37:33.892731+0000","last_unstale":"2024-09-13T08:37:36.876764+0000","last_undegraded":"2024-09-13T08:37:36.876764+0000","last_fullsized":"2024-09-13T08:37:36.876764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084725+0000","last_change":"2024-09-13T08:37:34.895132+0000","last_active":"2024-09-13T08:37:37.084725+0000","last_peered":"2024-09-13T08:37:37.084725+0000","last_clean":"2024-09-13T08:37:37.084725+0000","last_became_active":"2024-09-13T08:37:34.894785+0000","last_became_peered":"2024-09-13T08:37:34.894785+0000","last_unstale":"2024-09-13T08:37:37.084725+0000","last_undegraded":"2024-09-13T08:37:37.084725+0000","last_fullsized":"2024-09-13T08:37:37.084725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:07.683461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585602+0000","last_change":"2024-09-13T08:37:33.898275+0000","last_active":"2024-09-13T08:37:36.585602+0000","last_peered":"2024-09-13T08:37:36.585602+0000","last_clean":"2024-09-13T08:37:36.585602+0000","last_became_active":"2024-09-13T08:37:33.897960+0000","last_became_peered":"2024-09-13T08:37:33.897960+0000","last_unstale":"2024-09-13T08:37:36.585602+0000","last_undegraded":"2024-09-13T08:37:36.585602+0000","last_fullsized":"2024-09-13T08:37:36.585602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:22.150995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585465+0000","last_change":"2024-09-13T08:37:34.891285+0000","last_active":"2024-09-13T08:37:36.585465+0000","last_peered":"2024-09-13T08:37:36.585465+0000","last_clean":"2024-09-13T08:37:36.585465+0000","last_became_active":"2024-09-13T08:37:34.890996+0000","last_became_peered":"2024-09-13T08:37:34.890996+0000","last_unstale":"2024-09-13T08:37:36.585465+0000","last_undegraded":"2024-09-13T08:37:36.585465+0000","last_fullsized":"2024-09-13T08:37:36.585465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:47.564367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894655+0000","last_change":"2024-09-13T08:37:33.891510+0000","last_active":"2024-09-13T08:37:35.894655+0000","last_peered":"2024-09-13T08:37:35.894655+0000","last_clean":"2024-09-13T08:37:35.894655+0000","last_became_active":"2024-09-13T08:37:33.890774+0000","last_became_peered":"2024-09-13T08:37:33.890774+0000","last_unstale":"2024-09-13T08:37:35.894655+0000","last_undegraded":"2024-09-13T08:37:35.894655+0000","last_fullsized":"2024-09-13T08:37:35.894655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:25.370931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529890+0000","last_change":"2024-09-13T08:37:34.895928+0000","last_active":"2024-09-13T08:37:36.529890+0000","last_peered":"2024-09-13T08:37:36.529890+0000","last_clean":"2024-09-13T08:37:36.529890+0000","last_became_active":"2024-09-13T08:37:34.895612+0000","last_became_peered":"2024-09-13T08:37:34.895612+0000","last_unstale":"2024-09-13T08:37:36.529890+0000","last_undegraded":"2024-09-13T08:37:36.529890+0000","last_fullsized":"2024-09-13T08:37:36.529890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085214+0000","last_change":"2024-09-13T08:37:33.887243+0000","last_active":"2024-09-13T08:37:37.085214+0000","last_peered":"2024-09-13T08:37:37.085214+0000","last_clean":"2024-09-13T08:37:37.085214+0000","last_became_active":"2024-09-13T08:37:33.886882+0000","last_became_peered":"2024-09-13T08:37:33.886882+0000","last_unstale":"2024-09-13T08:37:37.085214+0000","last_undegraded":"2024-09-13T08:37:37.085214+0000","last_fullsized":"2024-09-13T08:37:37.085214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:31.044508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084923+0000","last_change":"2024-09-13T08:37:34.894740+0000","last_active":"2024-09-13T08:37:37.084923+0000","last_peered":"2024-09-13T08:37:37.084923+0000","last_clean":"2024-09-13T08:37:37.084923+0000","last_became_active":"2024-09-13T08:37:34.894284+0000","last_became_peered":"2024-09-13T08:37:34.894284+0000","last_unstale":"2024-09-13T08:37:37.084923+0000","last_undegraded":"2024-09-13T08:37:37.084923+0000","last_fullsized":"2024-09-13T08:37:37.084923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:28.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085351+0000","last_change":"2024-09-13T08:37:34.892338+0000","last_active":"2024-09-13T08:37:37.085351+0000","last_peered":"2024-09-13T08:37:37.085351+0000","last_clean":"2024-09-13T08:37:37.085351+0000","last_became_active":"2024-09-13T08:37:34.891790+0000","last_became_peered":"2024-09-13T08:37:34.891790+0000","last_unstale":"2024-09-13T08:37:37.085351+0000","last_undegraded":"2024-09-13T08:37:37.085351+0000","last_fullsized":"2024-09-13T08:37:37.085351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:24:06.938838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877706+0000","last_change":"2024-09-13T08:37:33.900929+0000","last_active":"2024-09-13T08:37:36.877706+0000","last_peered":"2024-09-13T08:37:36.877706+0000","last_clean":"2024-09-13T08:37:36.877706+0000","last_became_active":"2024-09-13T08:37:33.899776+0000","last_became_peered":"2024-09-13T08:37:33.899776+0000","last_unstale":"2024-09-13T08:37:36.877706+0000","last_undegraded":"2024-09-13T08:37:36.877706+0000","last_fullsized":"2024-09-13T08:37:36.877706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:27.711361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892184+0000","last_change":"2024-09-13T08:37:34.893833+0000","last_active":"2024-09-13T08:37:35.892184+0000","last_peered":"2024-09-13T08:37:35.892184+0000","last_clean":"2024-09-13T08:37:35.892184+0000","last_became_active":"2024-09-13T08:37:34.893512+0000","last_became_peered":"2024-09-13T08:37:34.893512+0000","last_unstale":"2024-09-13T08:37:35.892184+0000","last_undegraded":"2024-09-13T08:37:35.892184+0000","last_fullsized":"2024-09-13T08:37:35.892184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877847+0000","last_change":"2024-09-13T08:37:33.894985+0000","last_active":"2024-09-13T08:37:36.877847+0000","last_peered":"2024-09-13T08:37:36.877847+0000","last_clean":"2024-09-13T08:37:36.877847+0000","last_became_active":"2024-09-13T08:37:33.894087+0000","last_became_peered":"2024-09-13T08:37:33.894087+0000","last_unstale":"2024-09-13T08:37:36.877847+0000","last_undegraded":"2024-09-13T08:37:36.877847+0000","last_fullsized":"2024-09-13T08:37:36.877847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:51.688613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896328+0000","last_change":"2024-09-13T08:37:34.901345+0000","last_active":"2024-09-13T08:37:35.896328+0000","last_peered":"2024-09-13T08:37:35.896328+0000","last_clean":"2024-09-13T08:37:35.896328+0000","last_became_active":"2024-09-13T08:37:34.900290+0000","last_became_peered":"2024-09-13T08:37:34.900290+0000","last_unstale":"2024-09-13T08:37:35.896328+0000","last_undegraded":"2024-09-13T08:37:35.896328+0000","last_fullsized":"2024-09-13T08:37:35.896328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:08.493271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896451+0000","last_change":"2024-09-13T08:37:33.887848+0000","last_active":"2024-09-13T08:37:35.896451+0000","last_peered":"2024-09-13T08:37:35.896451+0000","last_clean":"2024-09-13T08:37:35.896451+0000","last_became_active":"2024-09-13T08:37:33.887508+0000","last_became_peered":"2024-09-13T08:37:33.887508+0000","last_unstale":"2024-09-13T08:37:35.896451+0000","last_undegraded":"2024-09-13T08:37:35.896451+0000","last_fullsized":"2024-09-13T08:37:35.896451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:27.615101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896328+0000","last_change":"2024-09-13T08:37:34.907107+0000","last_active":"2024-09-13T08:37:35.896328+0000","last_peered":"2024-09-13T08:37:35.896328+0000","last_clean":"2024-09-13T08:37:35.896328+0000","last_became_active":"2024-09-13T08:37:34.904160+0000","last_became_peered":"2024-09-13T08:37:34.904160+0000","last_unstale":"2024-09-13T08:37:35.896328+0000","last_undegraded":"2024-09-13T08:37:35.896328+0000","last_fullsized":"2024-09-13T08:37:35.896328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:55.775775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586114+0000","last_change":"2024-09-13T08:37:33.894082+0000","last_active":"2024-09-13T08:37:36.586114+0000","last_peered":"2024-09-13T08:37:36.586114+0000","last_clean":"2024-09-13T08:37:36.586114+0000","last_became_active":"2024-09-13T08:37:33.893820+0000","last_became_peered":"2024-09-13T08:37:33.893820+0000","last_unstale":"2024-09-13T08:37:36.586114+0000","last_undegraded":"2024-09-13T08:37:36.586114+0000","last_fullsized":"2024-09-13T08:37:36.586114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:48.992381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:37:48.827 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:37:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.007+0000 7fd83fa00640 1 -- 172.21.15.44:0/811221820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8380d28c0 msgr2=0x7fd8380cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.007+0000 7fd83fa00640 1 --2- 172.21.15.44:0/811221820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8380d28c0 0x7fd8380cea20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd838074290 tx=0x7fd82401c8c0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.007+0000 7fd83fa00640 1 -- 172.21.15.44:0/811221820 shutdown_connections 2024-09-13T08:37:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.007+0000 7fd83fa00640 1 --2- 172.21.15.44:0/811221820 >> v2:172.21.15.129:3300/0 conn(0x7fd8380b53f0 0x7fd8380b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.007+0000 7fd83fa00640 1 --2- 172.21.15.44:0/811221820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8380d28c0 0x7fd8380cea20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.007+0000 7fd83fa00640 1 --2- 172.21.15.44:0/811221820 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd83815be80 0x7fd8380ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.007+0000 7fd83fa00640 1 -- 172.21.15.44:0/811221820 >> 172.21.15.44:0/811221820 conn(0x7fd8380d1da0 msgr2=0x7fd8380d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.007+0000 7fd83fa00640 1 -- 172.21.15.44:0/811221820 shutdown_connections 2024-09-13T08:37:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.007+0000 7fd83fa00640 1 -- 172.21.15.44:0/811221820 wait complete. 2024-09-13T08:37:49.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83fa00640 1 Processor -- start 2024-09-13T08:37:49.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83fa00640 1 -- start start 2024-09-13T08:37:49.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83fa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8380b53f0 0x7fd8380c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83fa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd8380d28c0 0x7fd8381532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83fa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83815be80 0x7fd83806f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83fa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd8380ccf50 con 0x7fd83815be80 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83fa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8380ccdd0 con 0x7fd8380b53f0 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83fa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd8380cd0d0 con 0x7fd8380d28c0 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83815be80 0x7fd83806f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83815be80 0x7fd83806f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36778/0 (socket says 172.21.15.44:36778) 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83da00640 1 -- 172.21.15.44:0/2398069289 learned_addr learned my addr 172.21.15.44:0/2398069289 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd837e00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd8380d28c0 0x7fd8381532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83d000640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8380b53f0 0x7fd8380c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83da00640 1 -- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd8380d28c0 msgr2=0x7fd8381532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83da00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd8380d28c0 0x7fd8381532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83da00640 1 -- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8380b53f0 msgr2=0x7fd8380c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83da00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8380b53f0 0x7fd8380c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83da00640 1 -- 172.21.15.44:0/2398069289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8381faa10 con 0x7fd83815be80 2024-09-13T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83da00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83815be80 0x7fd83806f7b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd83000cd10 tx=0x7fd83000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd834c00640 1 -- 172.21.15.44:0/2398069289 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83000f450 con 0x7fd83815be80 2024-09-13T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd834c00640 1 -- 172.21.15.44:0/2398069289 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd83000f6e0 con 0x7fd83815be80 2024-09-13T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd834c00640 1 -- 172.21.15.44:0/2398069289 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83000fb30 con 0x7fd83815be80 2024-09-13T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83fa00640 1 -- 172.21.15.44:0/2398069289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8381facf0 con 0x7fd83815be80 2024-09-13T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.011+0000 7fd83fa00640 1 -- 172.21.15.44:0/2398069289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8381fb2b0 con 0x7fd83815be80 2024-09-13T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.015+0000 7fd83fa00640 1 -- 172.21.15.44:0/2398069289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8380b5880 con 0x7fd83815be80 2024-09-13T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.015+0000 7fd834c00640 1 -- 172.21.15.44:0/2398069289 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fd83000fcd0 con 0x7fd83815be80 2024-09-13T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.015+0000 7fd834c00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd818072ec0 0x7fd818075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.015+0000 7fd834c00640 1 -- 172.21.15.44:0/2398069289 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd830094960 con 0x7fd83815be80 2024-09-13T08:37:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.015+0000 7fd83d000640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd818072ec0 0x7fd818075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.015+0000 7fd83d000640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd818072ec0 0x7fd818075380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00aac0 tx=0x7fd82c00a250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.019+0000 7fd834c00640 1 -- 172.21.15.44:0/2398069289 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd830014030 con 0x7fd83815be80 2024-09-13T08:37:49.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.191+0000 7fd83fa00640 1 -- 172.21.15.44:0/2398069289 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8380b7610 con 0x7fd818072ec0 2024-09-13T08:37:49.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.195+0000 7fd834c00640 1 -- 172.21.15.44:0/2398069289 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7fd8380b7610 con 0x7fd818072ec0 2024-09-13T08:37:49.202 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:49.203 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:37:49.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 -- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd818072ec0 msgr2=0x7fd818075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd818072ec0 0x7fd818075380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00aac0 tx=0x7fd82c00a250 comp rx=0 tx=0).stop 2024-09-13T08:37:49.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 -- 172.21.15.44:0/2398069289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83815be80 msgr2=0x7fd83806f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83815be80 0x7fd83806f7b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd83000cd10 tx=0x7fd83000a620 comp rx=0 tx=0).stop 2024-09-13T08:37:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 -- 172.21.15.44:0/2398069289 shutdown_connections 2024-09-13T08:37:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd818072ec0 0x7fd818075380 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83815be80 0x7fd83806f7b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd8380d28c0 0x7fd8381532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 --2- 172.21.15.44:0/2398069289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8380b53f0 0x7fd8380c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 -- 172.21.15.44:0/2398069289 >> 172.21.15.44:0/2398069289 conn(0x7fd8380d1da0 msgr2=0x7fd838071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 -- 172.21.15.44:0/2398069289 shutdown_connections 2024-09-13T08:37:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.199+0000 7fd83fa00640 1 -- 172.21.15.44:0/2398069289 wait complete. 2024-09-13T08:37:49.222 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-13T08:37:48.514218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219224,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755496,"statfs":{"total":773094113280,"available":772869627904,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003268"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877562+0000","last_change":"2024-09-13T08:37:34.901083+0000","last_active":"2024-09-13T08:37:36.877562+0000","last_peered":"2024-09-13T08:37:36.877562+0000","last_clean":"2024-09-13T08:37:36.877562+0000","last_became_active":"2024-09-13T08:37:34.900414+0000","last_became_peered":"2024-09-13T08:37:34.900414+0000","last_unstale":"2024-09-13T08:37:36.877562+0000","last_undegraded":"2024-09-13T08:37:36.877562+0000","last_fullsized":"2024-09-13T08:37:36.877562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:28:50.949101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896264+0000","last_change":"2024-09-13T08:37:34.900607+0000","last_active":"2024-09-13T08:37:35.896264+0000","last_peered":"2024-09-13T08:37:35.896264+0000","last_clean":"2024-09-13T08:37:35.896264+0000","last_became_active":"2024-09-13T08:37:34.900222+0000","last_became_peered":"2024-09-13T08:37:34.900222+0000","last_unstale":"2024-09-13T08:37:35.896264+0000","last_undegraded":"2024-09-13T08:37:35.896264+0000","last_fullsized":"2024-09-13T08:37:35.896264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:47.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895933+0000","last_change":"2024-09-13T08:37:34.906335+0000","last_active":"2024-09-13T08:37:35.895933+0000","last_peered":"2024-09-13T08:37:35.895933+0000","last_clean":"2024-09-13T08:37:35.895933+0000","last_became_active":"2024-09-13T08:37:34.905961+0000","last_became_peered":"2024-09-13T08:37:34.905961+0000","last_unstale":"2024-09-13T08:37:35.895933+0000","last_undegraded":"2024-09-13T08:37:35.895933+0000","last_fullsized":"2024-09-13T08:37:35.895933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:54.979663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510756+0000","last_change":"2024-09-13T08:37:34.904837+0000","last_active":"2024-09-13T08:37:36.510756+0000","last_peered":"2024-09-13T08:37:36.510756+0000","last_clean":"2024-09-13T08:37:36.510756+0000","last_became_active":"2024-09-13T08:37:34.898571+0000","last_became_peered":"2024-09-13T08:37:34.898571+0000","last_unstale":"2024-09-13T08:37:36.510756+0000","last_undegraded":"2024-09-13T08:37:36.510756+0000","last_fullsized":"2024-09-13T08:37:36.510756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877279+0000","last_change":"2024-09-13T08:37:34.888564+0000","last_active":"2024-09-13T08:37:36.877279+0000","last_peered":"2024-09-13T08:37:36.877279+0000","last_clean":"2024-09-13T08:37:36.877279+0000","last_became_active":"2024-09-13T08:37:34.888280+0000","last_became_peered":"2024-09-13T08:37:34.888280+0000","last_unstale":"2024-09-13T08:37:36.877279+0000","last_undegraded":"2024-09-13T08:37:36.877279+0000","last_fullsized":"2024-09-13T08:37:36.877279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:27:32.330596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510582+0000","last_change":"2024-09-13T08:37:34.895044+0000","last_active":"2024-09-13T08:37:36.510582+0000","last_peered":"2024-09-13T08:37:36.510582+0000","last_clean":"2024-09-13T08:37:36.510582+0000","last_became_active":"2024-09-13T08:37:34.894777+0000","last_became_peered":"2024-09-13T08:37:34.894777+0000","last_unstale":"2024-09-13T08:37:36.510582+0000","last_undegraded":"2024-09-13T08:37:36.510582+0000","last_fullsized":"2024-09-13T08:37:36.510582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877014+0000","last_change":"2024-09-13T08:37:34.888796+0000","last_active":"2024-09-13T08:37:36.877014+0000","last_peered":"2024-09-13T08:37:36.877014+0000","last_clean":"2024-09-13T08:37:36.877014+0000","last_became_active":"2024-09-13T08:37:34.888488+0000","last_became_peered":"2024-09-13T08:37:34.888488+0000","last_unstale":"2024-09-13T08:37:36.877014+0000","last_undegraded":"2024-09-13T08:37:36.877014+0000","last_fullsized":"2024-09-13T08:37:36.877014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894878+0000","last_change":"2024-09-13T08:37:34.910054+0000","last_active":"2024-09-13T08:37:35.894878+0000","last_peered":"2024-09-13T08:37:35.894878+0000","last_clean":"2024-09-13T08:37:35.894878+0000","last_became_active":"2024-09-13T08:37:34.909596+0000","last_became_peered":"2024-09-13T08:37:34.909596+0000","last_unstale":"2024-09-13T08:37:35.894878+0000","last_undegraded":"2024-09-13T08:37:35.894878+0000","last_fullsized":"2024-09-13T08:37:35.894878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:55.615355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892431+0000","last_change":"2024-09-13T08:37:34.885156+0000","last_active":"2024-09-13T08:37:35.892431+0000","last_peered":"2024-09-13T08:37:35.892431+0000","last_clean":"2024-09-13T08:37:35.892431+0000","last_became_active":"2024-09-13T08:37:34.884851+0000","last_became_peered":"2024-09-13T08:37:34.884851+0000","last_unstale":"2024-09-13T08:37:35.892431+0000","last_undegraded":"2024-09-13T08:37:35.892431+0000","last_fullsized":"2024-09-13T08:37:35.892431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:06.047971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877522+0000","last_change":"2024-09-13T08:37:34.909843+0000","last_active":"2024-09-13T08:37:36.877522+0000","last_peered":"2024-09-13T08:37:36.877522+0000","last_clean":"2024-09-13T08:37:36.877522+0000","last_became_active":"2024-09-13T08:37:34.909552+0000","last_became_peered":"2024-09-13T08:37:34.909552+0000","last_unstale":"2024-09-13T08:37:36.877522+0000","last_undegraded":"2024-09-13T08:37:36.877522+0000","last_fullsized":"2024-09-13T08:37:36.877522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:08.493271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877666+0000","last_change":"2024-09-13T08:37:34.902054+0000","last_active":"2024-09-13T08:37:36.877666+0000","last_peered":"2024-09-13T08:37:36.877666+0000","last_clean":"2024-09-13T08:37:36.877666+0000","last_became_active":"2024-09-13T08:37:34.901788+0000","last_became_peered":"2024-09-13T08:37:34.901788+0000","last_unstale":"2024-09-13T08:37:36.877666+0000","last_undegraded":"2024-09-13T08:37:36.877666+0000","last_fullsized":"2024-09-13T08:37:36.877666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:14.002956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877261+0000","last_change":"2024-09-13T08:37:34.892602+0000","last_active":"2024-09-13T08:37:36.877261+0000","last_peered":"2024-09-13T08:37:36.877261+0000","last_clean":"2024-09-13T08:37:36.877261+0000","last_became_active":"2024-09-13T08:37:34.890000+0000","last_became_peered":"2024-09-13T08:37:34.890000+0000","last_unstale":"2024-09-13T08:37:36.877261+0000","last_undegraded":"2024-09-13T08:37:36.877261+0000","last_fullsized":"2024-09-13T08:37:36.877261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:06.530173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894980+0000","last_change":"2024-09-13T08:37:34.903037+0000","last_active":"2024-09-13T08:37:35.894980+0000","last_peered":"2024-09-13T08:37:35.894980+0000","last_clean":"2024-09-13T08:37:35.894980+0000","last_became_active":"2024-09-13T08:37:34.894805+0000","last_became_peered":"2024-09-13T08:37:34.894805+0000","last_unstale":"2024-09-13T08:37:35.894980+0000","last_undegraded":"2024-09-13T08:37:35.894980+0000","last_fullsized":"2024-09-13T08:37:35.894980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:49.712241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894573+0000","last_change":"2024-09-13T08:37:34.901603+0000","last_active":"2024-09-13T08:37:35.894573+0000","last_peered":"2024-09-13T08:37:35.894573+0000","last_clean":"2024-09-13T08:37:35.894573+0000","last_became_active":"2024-09-13T08:37:34.899884+0000","last_became_peered":"2024-09-13T08:37:34.899884+0000","last_unstale":"2024-09-13T08:37:35.894573+0000","last_undegraded":"2024-09-13T08:37:35.894573+0000","last_fullsized":"2024-09-13T08:37:35.894573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:05.260600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895761+0000","last_change":"2024-09-13T08:37:34.896921+0000","last_active":"2024-09-13T08:37:35.895761+0000","last_peered":"2024-09-13T08:37:35.895761+0000","last_clean":"2024-09-13T08:37:35.895761+0000","last_became_active":"2024-09-13T08:37:34.896593+0000","last_became_peered":"2024-09-13T08:37:34.896593+0000","last_unstale":"2024-09-13T08:37:35.895761+0000","last_undegraded":"2024-09-13T08:37:35.895761+0000","last_fullsized":"2024-09-13T08:37:35.895761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:06.530173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084616+0000","last_change":"2024-09-13T08:37:34.908031+0000","last_active":"2024-09-13T08:37:37.084616+0000","last_peered":"2024-09-13T08:37:37.084616+0000","last_clean":"2024-09-13T08:37:37.084616+0000","last_became_active":"2024-09-13T08:37:34.907734+0000","last_became_peered":"2024-09-13T08:37:34.907734+0000","last_unstale":"2024-09-13T08:37:37.084616+0000","last_undegraded":"2024-09-13T08:37:37.084616+0000","last_fullsized":"2024-09-13T08:37:37.084616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:14.002956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085576+0000","last_change":"2024-09-13T08:37:34.911572+0000","last_active":"2024-09-13T08:37:37.085576+0000","last_peered":"2024-09-13T08:37:37.085576+0000","last_clean":"2024-09-13T08:37:37.085576+0000","last_became_active":"2024-09-13T08:37:34.911105+0000","last_became_peered":"2024-09-13T08:37:34.911105+0000","last_unstale":"2024-09-13T08:37:37.085576+0000","last_undegraded":"2024-09-13T08:37:37.085576+0000","last_fullsized":"2024-09-13T08:37:37.085576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:49:35.223067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.530220+0000","last_change":"2024-09-13T08:37:34.907767+0000","last_active":"2024-09-13T08:37:36.530220+0000","last_peered":"2024-09-13T08:37:36.530220+0000","last_clean":"2024-09-13T08:37:36.530220+0000","last_became_active":"2024-09-13T08:37:34.907100+0000","last_became_peered":"2024-09-13T08:37:34.907100+0000","last_unstale":"2024-09-13T08:37:36.530220+0000","last_undegraded":"2024-09-13T08:37:36.530220+0000","last_fullsized":"2024-09-13T08:37:36.530220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085543+0000","last_change":"2024-09-13T08:37:34.895822+0000","last_active":"2024-09-13T08:37:37.085543+0000","last_peered":"2024-09-13T08:37:37.085543+0000","last_clean":"2024-09-13T08:37:37.085543+0000","last_became_active":"2024-09-13T08:37:34.895430+0000","last_became_peered":"2024-09-13T08:37:34.895430+0000","last_unstale":"2024-09-13T08:37:37.085543+0000","last_undegraded":"2024-09-13T08:37:37.085543+0000","last_fullsized":"2024-09-13T08:37:37.085543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:33:54.897556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877407+0000","last_change":"2024-09-13T08:37:34.891580+0000","last_active":"2024-09-13T08:37:36.877407+0000","last_peered":"2024-09-13T08:37:36.877407+0000","last_clean":"2024-09-13T08:37:36.877407+0000","last_became_active":"2024-09-13T08:37:34.889710+0000","last_became_peered":"2024-09-13T08:37:34.889710+0000","last_unstale":"2024-09-13T08:37:36.877407+0000","last_undegraded":"2024-09-13T08:37:36.877407+0000","last_fullsized":"2024-09-13T08:37:36.877407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:47.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585799+0000","last_change":"2024-09-13T08:37:34.894615+0000","last_active":"2024-09-13T08:37:36.585799+0000","last_peered":"2024-09-13T08:37:36.585799+0000","last_clean":"2024-09-13T08:37:36.585799+0000","last_became_active":"2024-09-13T08:37:34.894327+0000","last_became_peered":"2024-09-13T08:37:34.894327+0000","last_unstale":"2024-09-13T08:37:36.585799+0000","last_undegraded":"2024-09-13T08:37:36.585799+0000","last_fullsized":"2024-09-13T08:37:36.585799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:55.615355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891945+0000","last_change":"2024-09-13T08:37:34.883272+0000","last_active":"2024-09-13T08:37:35.891945+0000","last_peered":"2024-09-13T08:37:35.891945+0000","last_clean":"2024-09-13T08:37:35.891945+0000","last_became_active":"2024-09-13T08:37:34.882941+0000","last_became_peered":"2024-09-13T08:37:34.882941+0000","last_unstale":"2024-09-13T08:37:35.891945+0000","last_undegraded":"2024-09-13T08:37:35.891945+0000","last_fullsized":"2024-09-13T08:37:35.891945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895545+0000","last_change":"2024-09-13T08:37:34.903123+0000","last_active":"2024-09-13T08:37:35.895545+0000","last_peered":"2024-09-13T08:37:35.895545+0000","last_clean":"2024-09-13T08:37:35.895545+0000","last_became_active":"2024-09-13T08:37:34.901167+0000","last_became_peered":"2024-09-13T08:37:34.901167+0000","last_unstale":"2024-09-13T08:37:35.895545+0000","last_undegraded":"2024-09-13T08:37:35.895545+0000","last_fullsized":"2024-09-13T08:37:35.895545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084614+0000","last_change":"2024-09-13T08:37:34.891584+0000","last_active":"2024-09-13T08:37:37.084614+0000","last_peered":"2024-09-13T08:37:37.084614+0000","last_clean":"2024-09-13T08:37:37.084614+0000","last_became_active":"2024-09-13T08:37:34.891268+0000","last_became_peered":"2024-09-13T08:37:34.891268+0000","last_unstale":"2024-09-13T08:37:37.084614+0000","last_undegraded":"2024-09-13T08:37:37.084614+0000","last_fullsized":"2024-09-13T08:37:37.084614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:59.406808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896325+0000","last_change":"2024-09-13T08:37:34.902079+0000","last_active":"2024-09-13T08:37:35.896325+0000","last_peered":"2024-09-13T08:37:35.896325+0000","last_clean":"2024-09-13T08:37:35.896325+0000","last_became_active":"2024-09-13T08:37:34.900499+0000","last_became_peered":"2024-09-13T08:37:34.900499+0000","last_unstale":"2024-09-13T08:37:35.896325+0000","last_undegraded":"2024-09-13T08:37:35.896325+0000","last_fullsized":"2024-09-13T08:37:35.896325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:45.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511137+0000","last_change":"2024-09-13T08:37:34.888787+0000","last_active":"2024-09-13T08:37:36.511137+0000","last_peered":"2024-09-13T08:37:36.511137+0000","last_clean":"2024-09-13T08:37:36.511137+0000","last_became_active":"2024-09-13T08:37:34.888486+0000","last_became_peered":"2024-09-13T08:37:34.888486+0000","last_unstale":"2024-09-13T08:37:36.511137+0000","last_undegraded":"2024-09-13T08:37:36.511137+0000","last_fullsized":"2024-09-13T08:37:36.511137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:56.908418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895541+0000","last_change":"2024-09-13T08:37:34.902546+0000","last_active":"2024-09-13T08:37:35.895541+0000","last_peered":"2024-09-13T08:37:35.895541+0000","last_clean":"2024-09-13T08:37:35.895541+0000","last_became_active":"2024-09-13T08:37:34.901841+0000","last_became_peered":"2024-09-13T08:37:34.901841+0000","last_unstale":"2024-09-13T08:37:35.895541+0000","last_undegraded":"2024-09-13T08:37:35.895541+0000","last_fullsized":"2024-09-13T08:37:35.895541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:40.635073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895078+0000","last_change":"2024-09-13T08:37:34.907277+0000","last_active":"2024-09-13T08:37:35.895078+0000","last_peered":"2024-09-13T08:37:35.895078+0000","last_clean":"2024-09-13T08:37:35.895078+0000","last_became_active":"2024-09-13T08:37:34.906984+0000","last_became_peered":"2024-09-13T08:37:34.906984+0000","last_unstale":"2024-09-13T08:37:35.895078+0000","last_undegraded":"2024-09-13T08:37:35.895078+0000","last_fullsized":"2024-09-13T08:37:35.895078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:23.664559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895840+0000","last_change":"2024-09-13T08:37:34.902627+0000","last_active":"2024-09-13T08:37:35.895840+0000","last_peered":"2024-09-13T08:37:35.895840+0000","last_clean":"2024-09-13T08:37:35.895840+0000","last_became_active":"2024-09-13T08:37:34.901894+0000","last_became_peered":"2024-09-13T08:37:34.901894+0000","last_unstale":"2024-09-13T08:37:35.895840+0000","last_undegraded":"2024-09-13T08:37:35.895840+0000","last_fullsized":"2024-09-13T08:37:35.895840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:06.723541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084874+0000","last_change":"2024-09-13T08:37:34.891317+0000","last_active":"2024-09-13T08:37:37.084874+0000","last_peered":"2024-09-13T08:37:37.084874+0000","last_clean":"2024-09-13T08:37:37.084874+0000","last_became_active":"2024-09-13T08:37:34.891030+0000","last_became_peered":"2024-09-13T08:37:34.891030+0000","last_unstale":"2024-09-13T08:37:37.084874+0000","last_undegraded":"2024-09-13T08:37:37.084874+0000","last_fullsized":"2024-09-13T08:37:37.084874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:46.738660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529933+0000","last_change":"2024-09-13T08:37:34.894275+0000","last_active":"2024-09-13T08:37:36.529933+0000","last_peered":"2024-09-13T08:37:36.529933+0000","last_clean":"2024-09-13T08:37:36.529933+0000","last_became_active":"2024-09-13T08:37:34.893030+0000","last_became_peered":"2024-09-13T08:37:34.893030+0000","last_unstale":"2024-09-13T08:37:36.529933+0000","last_undegraded":"2024-09-13T08:37:36.529933+0000","last_fullsized":"2024-09-13T08:37:36.529933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510646+0000","last_change":"2024-09-13T08:37:34.900318+0000","last_active":"2024-09-13T08:37:36.510646+0000","last_peered":"2024-09-13T08:37:36.510646+0000","last_clean":"2024-09-13T08:37:36.510646+0000","last_became_active":"2024-09-13T08:37:34.899949+0000","last_became_peered":"2024-09-13T08:37:34.899949+0000","last_unstale":"2024-09-13T08:37:36.510646+0000","last_undegraded":"2024-09-13T08:37:36.510646+0000","last_fullsized":"2024-09-13T08:37:36.510646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892117+0000","last_change":"2024-09-13T08:37:34.889876+0000","last_active":"2024-09-13T08:37:35.892117+0000","last_peered":"2024-09-13T08:37:35.892117+0000","last_clean":"2024-09-13T08:37:35.892117+0000","last_became_active":"2024-09-13T08:37:34.889622+0000","last_became_peered":"2024-09-13T08:37:34.889622+0000","last_unstale":"2024-09-13T08:37:35.892117+0000","last_undegraded":"2024-09-13T08:37:35.892117+0000","last_fullsized":"2024-09-13T08:37:35.892117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:56.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895867+0000","last_change":"2024-09-13T08:37:33.888852+0000","last_active":"2024-09-13T08:37:35.895867+0000","last_peered":"2024-09-13T08:37:35.895867+0000","last_clean":"2024-09-13T08:37:35.895867+0000","last_became_active":"2024-09-13T08:37:33.888409+0000","last_became_peered":"2024-09-13T08:37:33.888409+0000","last_unstale":"2024-09-13T08:37:35.895867+0000","last_undegraded":"2024-09-13T08:37:35.895867+0000","last_fullsized":"2024-09-13T08:37:35.895867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585747+0000","last_change":"2024-09-13T08:37:34.890809+0000","last_active":"2024-09-13T08:37:36.585747+0000","last_peered":"2024-09-13T08:37:36.585747+0000","last_clean":"2024-09-13T08:37:36.585747+0000","last_became_active":"2024-09-13T08:37:34.887213+0000","last_became_peered":"2024-09-13T08:37:34.887213+0000","last_unstale":"2024-09-13T08:37:36.585747+0000","last_undegraded":"2024-09-13T08:37:36.585747+0000","last_fullsized":"2024-09-13T08:37:36.585747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:19:06.310722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085055+0000","last_change":"2024-09-13T08:37:33.892308+0000","last_active":"2024-09-13T08:37:37.085055+0000","last_peered":"2024-09-13T08:37:37.085055+0000","last_clean":"2024-09-13T08:37:37.085055+0000","last_became_active":"2024-09-13T08:37:33.891875+0000","last_became_peered":"2024-09-13T08:37:33.891875+0000","last_unstale":"2024-09-13T08:37:37.085055+0000","last_undegraded":"2024-09-13T08:37:37.085055+0000","last_fullsized":"2024-09-13T08:37:37.085055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:05.039750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894251+0000","last_change":"2024-09-13T08:37:33.896561+0000","last_active":"2024-09-13T08:37:35.894251+0000","last_peered":"2024-09-13T08:37:35.894251+0000","last_clean":"2024-09-13T08:37:35.894251+0000","last_became_active":"2024-09-13T08:37:33.896314+0000","last_became_peered":"2024-09-13T08:37:33.896314+0000","last_unstale":"2024-09-13T08:37:35.894251+0000","last_undegraded":"2024-09-13T08:37:35.894251+0000","last_fullsized":"2024-09-13T08:37:35.894251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:28.728254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895122+0000","last_change":"2024-09-13T08:37:34.900565+0000","last_active":"2024-09-13T08:37:35.895122+0000","last_peered":"2024-09-13T08:37:35.895122+0000","last_clean":"2024-09-13T08:37:35.895122+0000","last_became_active":"2024-09-13T08:37:34.900257+0000","last_became_peered":"2024-09-13T08:37:34.900257+0000","last_unstale":"2024-09-13T08:37:35.895122+0000","last_undegraded":"2024-09-13T08:37:35.895122+0000","last_fullsized":"2024-09-13T08:37:35.895122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:34.291551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084825+0000","last_change":"2024-09-13T08:37:33.883156+0000","last_active":"2024-09-13T08:37:37.084825+0000","last_peered":"2024-09-13T08:37:37.084825+0000","last_clean":"2024-09-13T08:37:37.084825+0000","last_became_active":"2024-09-13T08:37:33.882800+0000","last_became_peered":"2024-09-13T08:37:33.882800+0000","last_unstale":"2024-09-13T08:37:37.084825+0000","last_undegraded":"2024-09-13T08:37:37.084825+0000","last_fullsized":"2024-09-13T08:37:37.084825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:10.413313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585694+0000","last_change":"2024-09-13T08:37:34.888188+0000","last_active":"2024-09-13T08:37:36.585694+0000","last_peered":"2024-09-13T08:37:36.585694+0000","last_clean":"2024-09-13T08:37:36.585694+0000","last_became_active":"2024-09-13T08:37:34.887475+0000","last_became_peered":"2024-09-13T08:37:34.887475+0000","last_unstale":"2024-09-13T08:37:36.585694+0000","last_undegraded":"2024-09-13T08:37:36.585694+0000","last_fullsized":"2024-09-13T08:37:36.585694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:14.743221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892396+0000","last_change":"2024-09-13T08:37:33.891385+0000","last_active":"2024-09-13T08:37:35.892396+0000","last_peered":"2024-09-13T08:37:35.892396+0000","last_clean":"2024-09-13T08:37:35.892396+0000","last_became_active":"2024-09-13T08:37:33.891112+0000","last_became_peered":"2024-09-13T08:37:33.891112+0000","last_unstale":"2024-09-13T08:37:35.892396+0000","last_undegraded":"2024-09-13T08:37:35.892396+0000","last_fullsized":"2024-09-13T08:37:35.892396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511114+0000","last_change":"2024-09-13T08:37:34.899512+0000","last_active":"2024-09-13T08:37:36.511114+0000","last_peered":"2024-09-13T08:37:36.511114+0000","last_clean":"2024-09-13T08:37:36.511114+0000","last_became_active":"2024-09-13T08:37:34.887883+0000","last_became_peered":"2024-09-13T08:37:34.887883+0000","last_unstale":"2024-09-13T08:37:36.511114+0000","last_undegraded":"2024-09-13T08:37:36.511114+0000","last_fullsized":"2024-09-13T08:37:36.511114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:56.149863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877483+0000","last_change":"2024-09-13T08:37:33.899147+0000","last_active":"2024-09-13T08:37:36.877483+0000","last_peered":"2024-09-13T08:37:36.877483+0000","last_clean":"2024-09-13T08:37:36.877483+0000","last_became_active":"2024-09-13T08:37:33.896929+0000","last_became_peered":"2024-09-13T08:37:33.896929+0000","last_unstale":"2024-09-13T08:37:36.877483+0000","last_undegraded":"2024-09-13T08:37:36.877483+0000","last_fullsized":"2024-09-13T08:37:36.877483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:08:15.453880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896504+0000","last_change":"2024-09-13T08:37:34.900790+0000","last_active":"2024-09-13T08:37:35.896504+0000","last_peered":"2024-09-13T08:37:35.896504+0000","last_clean":"2024-09-13T08:37:35.896504+0000","last_became_active":"2024-09-13T08:37:34.898939+0000","last_became_peered":"2024-09-13T08:37:34.898939+0000","last_unstale":"2024-09-13T08:37:35.896504+0000","last_undegraded":"2024-09-13T08:37:35.896504+0000","last_fullsized":"2024-09-13T08:37:35.896504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:07.812549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.530096+0000","last_change":"2024-09-13T08:37:33.892872+0000","last_active":"2024-09-13T08:37:36.530096+0000","last_peered":"2024-09-13T08:37:36.530096+0000","last_clean":"2024-09-13T08:37:36.530096+0000","last_became_active":"2024-09-13T08:37:33.892579+0000","last_became_peered":"2024-09-13T08:37:33.892579+0000","last_unstale":"2024-09-13T08:37:36.530096+0000","last_undegraded":"2024-09-13T08:37:36.530096+0000","last_fullsized":"2024-09-13T08:37:36.530096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:57.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085280+0000","last_change":"2024-09-13T08:37:34.887215+0000","last_active":"2024-09-13T08:37:37.085280+0000","last_peered":"2024-09-13T08:37:37.085280+0000","last_clean":"2024-09-13T08:37:37.085280+0000","last_became_active":"2024-09-13T08:37:34.886889+0000","last_became_peered":"2024-09-13T08:37:34.886889+0000","last_unstale":"2024-09-13T08:37:37.085280+0000","last_undegraded":"2024-09-13T08:37:37.085280+0000","last_fullsized":"2024-09-13T08:37:37.085280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:57.113272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510921+0000","last_change":"2024-09-13T08:37:33.891105+0000","last_active":"2024-09-13T08:37:36.510921+0000","last_peered":"2024-09-13T08:37:36.510921+0000","last_clean":"2024-09-13T08:37:36.510921+0000","last_became_active":"2024-09-13T08:37:33.890783+0000","last_became_peered":"2024-09-13T08:37:33.890783+0000","last_unstale":"2024-09-13T08:37:36.510921+0000","last_undegraded":"2024-09-13T08:37:36.510921+0000","last_fullsized":"2024-09-13T08:37:36.510921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:30:45.112157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877636+0000","last_change":"2024-09-13T08:37:34.893268+0000","last_active":"2024-09-13T08:37:36.877636+0000","last_peered":"2024-09-13T08:37:36.877636+0000","last_clean":"2024-09-13T08:37:36.877636+0000","last_became_active":"2024-09-13T08:37:34.892944+0000","last_became_peered":"2024-09-13T08:37:34.892944+0000","last_unstale":"2024-09-13T08:37:36.877636+0000","last_undegraded":"2024-09-13T08:37:36.877636+0000","last_fullsized":"2024-09-13T08:37:36.877636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:45.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529953+0000","last_change":"2024-09-13T08:37:33.902610+0000","last_active":"2024-09-13T08:37:36.529953+0000","last_peered":"2024-09-13T08:37:36.529953+0000","last_clean":"2024-09-13T08:37:36.529953+0000","last_became_active":"2024-09-13T08:37:33.902275+0000","last_became_peered":"2024-09-13T08:37:33.902275+0000","last_unstale":"2024-09-13T08:37:36.529953+0000","last_undegraded":"2024-09-13T08:37:36.529953+0000","last_fullsized":"2024-09-13T08:37:36.529953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:57.838131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894679+0000","last_change":"2024-09-13T08:37:34.904983+0000","last_active":"2024-09-13T08:37:35.894679+0000","last_peered":"2024-09-13T08:37:35.894679+0000","last_clean":"2024-09-13T08:37:35.894679+0000","last_became_active":"2024-09-13T08:37:34.904618+0000","last_became_peered":"2024-09-13T08:37:34.904618+0000","last_unstale":"2024-09-13T08:37:35.894679+0000","last_undegraded":"2024-09-13T08:37:35.894679+0000","last_fullsized":"2024-09-13T08:37:35.894679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:30.070140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085096+0000","last_change":"2024-09-13T08:37:33.888874+0000","last_active":"2024-09-13T08:37:37.085096+0000","last_peered":"2024-09-13T08:37:37.085096+0000","last_clean":"2024-09-13T08:37:37.085096+0000","last_became_active":"2024-09-13T08:37:33.888111+0000","last_became_peered":"2024-09-13T08:37:33.888111+0000","last_unstale":"2024-09-13T08:37:37.085096+0000","last_undegraded":"2024-09-13T08:37:37.085096+0000","last_fullsized":"2024-09-13T08:37:37.085096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:55.141642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084977+0000","last_change":"2024-09-13T08:37:34.893432+0000","last_active":"2024-09-13T08:37:37.084977+0000","last_peered":"2024-09-13T08:37:37.084977+0000","last_clean":"2024-09-13T08:37:37.084977+0000","last_became_active":"2024-09-13T08:37:34.893160+0000","last_became_peered":"2024-09-13T08:37:34.893160+0000","last_unstale":"2024-09-13T08:37:37.084977+0000","last_undegraded":"2024-09-13T08:37:37.084977+0000","last_fullsized":"2024-09-13T08:37:37.084977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:46.900298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.876999+0000","last_change":"2024-09-13T08:37:33.892008+0000","last_active":"2024-09-13T08:37:36.876999+0000","last_peered":"2024-09-13T08:37:36.876999+0000","last_clean":"2024-09-13T08:37:36.876999+0000","last_became_active":"2024-09-13T08:37:33.891758+0000","last_became_peered":"2024-09-13T08:37:33.891758+0000","last_unstale":"2024-09-13T08:37:36.876999+0000","last_undegraded":"2024-09-13T08:37:36.876999+0000","last_fullsized":"2024-09-13T08:37:36.876999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:27.615101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891708+0000","last_change":"2024-09-13T08:37:34.889786+0000","last_active":"2024-09-13T08:37:35.891708+0000","last_peered":"2024-09-13T08:37:35.891708+0000","last_clean":"2024-09-13T08:37:35.891708+0000","last_became_active":"2024-09-13T08:37:34.889505+0000","last_became_peered":"2024-09-13T08:37:34.889505+0000","last_unstale":"2024-09-13T08:37:35.891708+0000","last_undegraded":"2024-09-13T08:37:35.891708+0000","last_fullsized":"2024-09-13T08:37:35.891708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:50.000602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585415+0000","last_change":"2024-09-13T08:37:33.888556+0000","last_active":"2024-09-13T08:37:36.585415+0000","last_peered":"2024-09-13T08:37:36.585415+0000","last_clean":"2024-09-13T08:37:36.585415+0000","last_became_active":"2024-09-13T08:37:33.887997+0000","last_became_peered":"2024-09-13T08:37:33.887997+0000","last_unstale":"2024-09-13T08:37:36.585415+0000","last_undegraded":"2024-09-13T08:37:36.585415+0000","last_fullsized":"2024-09-13T08:37:36.585415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084670+0000","last_change":"2024-09-13T08:37:34.908412+0000","last_active":"2024-09-13T08:37:37.084670+0000","last_peered":"2024-09-13T08:37:37.084670+0000","last_clean":"2024-09-13T08:37:37.084670+0000","last_became_active":"2024-09-13T08:37:34.908135+0000","last_became_peered":"2024-09-13T08:37:34.908135+0000","last_unstale":"2024-09-13T08:37:37.084670+0000","last_undegraded":"2024-09-13T08:37:37.084670+0000","last_fullsized":"2024-09-13T08:37:37.084670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:15.303677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877915+0000","last_change":"2024-09-13T08:37:33.895134+0000","last_active":"2024-09-13T08:37:36.877915+0000","last_peered":"2024-09-13T08:37:36.877915+0000","last_clean":"2024-09-13T08:37:36.877915+0000","last_became_active":"2024-09-13T08:37:33.894893+0000","last_became_peered":"2024-09-13T08:37:33.894893+0000","last_unstale":"2024-09-13T08:37:36.877915+0000","last_undegraded":"2024-09-13T08:37:36.877915+0000","last_fullsized":"2024-09-13T08:37:36.877915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:03:36.459606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.511239+0000","last_change":"2024-09-13T08:37:34.915800+0000","last_active":"2024-09-13T08:37:36.511239+0000","last_peered":"2024-09-13T08:37:36.511239+0000","last_clean":"2024-09-13T08:37:36.511239+0000","last_became_active":"2024-09-13T08:37:34.915478+0000","last_became_peered":"2024-09-13T08:37:34.915478+0000","last_unstale":"2024-09-13T08:37:36.511239+0000","last_undegraded":"2024-09-13T08:37:36.511239+0000","last_fullsized":"2024-09-13T08:37:36.511239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:55.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586002+0000","last_change":"2024-09-13T08:37:33.888156+0000","last_active":"2024-09-13T08:37:36.586002+0000","last_peered":"2024-09-13T08:37:36.586002+0000","last_clean":"2024-09-13T08:37:36.586002+0000","last_became_active":"2024-09-13T08:37:33.886607+0000","last_became_peered":"2024-09-13T08:37:33.886607+0000","last_unstale":"2024-09-13T08:37:36.586002+0000","last_undegraded":"2024-09-13T08:37:36.586002+0000","last_fullsized":"2024-09-13T08:37:36.586002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:36.540294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877863+0000","last_change":"2024-09-13T08:37:34.891196+0000","last_active":"2024-09-13T08:37:36.877863+0000","last_peered":"2024-09-13T08:37:36.877863+0000","last_clean":"2024-09-13T08:37:36.877863+0000","last_became_active":"2024-09-13T08:37:34.890848+0000","last_became_peered":"2024-09-13T08:37:34.890848+0000","last_unstale":"2024-09-13T08:37:36.877863+0000","last_undegraded":"2024-09-13T08:37:36.877863+0000","last_fullsized":"2024-09-13T08:37:36.877863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:07.683461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896137+0000","last_change":"2024-09-13T08:37:33.885405+0000","last_active":"2024-09-13T08:37:35.896137+0000","last_peered":"2024-09-13T08:37:35.896137+0000","last_clean":"2024-09-13T08:37:35.896137+0000","last_became_active":"2024-09-13T08:37:33.885048+0000","last_became_peered":"2024-09-13T08:37:33.885048+0000","last_unstale":"2024-09-13T08:37:35.896137+0000","last_undegraded":"2024-09-13T08:37:35.896137+0000","last_fullsized":"2024-09-13T08:37:35.896137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:33:47.115618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877736+0000","last_change":"2024-09-13T08:37:34.898475+0000","last_active":"2024-09-13T08:37:36.877736+0000","last_peered":"2024-09-13T08:37:36.877736+0000","last_clean":"2024-09-13T08:37:36.877736+0000","last_became_active":"2024-09-13T08:37:34.897854+0000","last_became_peered":"2024-09-13T08:37:34.897854+0000","last_unstale":"2024-09-13T08:37:36.877736+0000","last_undegraded":"2024-09-13T08:37:36.877736+0000","last_fullsized":"2024-09-13T08:37:36.877736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:15.303677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895446+0000","last_change":"2024-09-13T08:37:33.902815+0000","last_active":"2024-09-13T08:37:35.895446+0000","last_peered":"2024-09-13T08:37:35.895446+0000","last_clean":"2024-09-13T08:37:35.895446+0000","last_became_active":"2024-09-13T08:37:33.902490+0000","last_became_peered":"2024-09-13T08:37:33.902490+0000","last_unstale":"2024-09-13T08:37:35.895446+0000","last_undegraded":"2024-09-13T08:37:35.895446+0000","last_fullsized":"2024-09-13T08:37:35.895446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:32.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085416+0000","last_change":"2024-09-13T08:37:34.891378+0000","last_active":"2024-09-13T08:37:37.085416+0000","last_peered":"2024-09-13T08:37:37.085416+0000","last_clean":"2024-09-13T08:37:37.085416+0000","last_became_active":"2024-09-13T08:37:34.885687+0000","last_became_peered":"2024-09-13T08:37:34.885687+0000","last_unstale":"2024-09-13T08:37:37.085416+0000","last_undegraded":"2024-09-13T08:37:37.085416+0000","last_fullsized":"2024-09-13T08:37:37.085416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:42.639292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.891693+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:37:35.891693+0000","last_peered":"2024-09-13T08:37:35.891693+0000","last_clean":"2024-09-13T08:37:35.891693+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:37:35.891693+0000","last_undegraded":"2024-09-13T08:37:35.891693+0000","last_fullsized":"2024-09-13T08:37:35.891693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085218+0000","last_change":"2024-09-13T08:37:33.883826+0000","last_active":"2024-09-13T08:37:37.085218+0000","last_peered":"2024-09-13T08:37:37.085218+0000","last_clean":"2024-09-13T08:37:37.085218+0000","last_became_active":"2024-09-13T08:37:33.883511+0000","last_became_peered":"2024-09-13T08:37:33.883511+0000","last_unstale":"2024-09-13T08:37:37.085218+0000","last_undegraded":"2024-09-13T08:37:37.085218+0000","last_fullsized":"2024-09-13T08:37:37.085218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:39:48.743211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585719+0000","last_change":"2024-09-13T08:37:34.901372+0000","last_active":"2024-09-13T08:37:36.585719+0000","last_peered":"2024-09-13T08:37:36.585719+0000","last_clean":"2024-09-13T08:37:36.585719+0000","last_became_active":"2024-09-13T08:37:34.900931+0000","last_became_peered":"2024-09-13T08:37:34.900931+0000","last_unstale":"2024-09-13T08:37:36.585719+0000","last_undegraded":"2024-09-13T08:37:36.585719+0000","last_fullsized":"2024-09-13T08:37:36.585719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:16.328343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084832+0000","last_change":"2024-09-13T08:37:33.885219+0000","last_active":"2024-09-13T08:37:37.084832+0000","last_peered":"2024-09-13T08:37:37.084832+0000","last_clean":"2024-09-13T08:37:37.084832+0000","last_became_active":"2024-09-13T08:37:33.884889+0000","last_became_peered":"2024-09-13T08:37:33.884889+0000","last_unstale":"2024-09-13T08:37:37.084832+0000","last_undegraded":"2024-09-13T08:37:37.084832+0000","last_fullsized":"2024-09-13T08:37:37.084832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:55.495409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894451+0000","last_change":"2024-09-13T08:37:34.908186+0000","last_active":"2024-09-13T08:37:35.894451+0000","last_peered":"2024-09-13T08:37:35.894451+0000","last_clean":"2024-09-13T08:37:35.894451+0000","last_became_active":"2024-09-13T08:37:34.907886+0000","last_became_peered":"2024-09-13T08:37:34.907886+0000","last_unstale":"2024-09-13T08:37:35.894451+0000","last_undegraded":"2024-09-13T08:37:35.894451+0000","last_fullsized":"2024-09-13T08:37:35.894451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:11:14.712673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529782+0000","last_change":"2024-09-13T08:37:33.890556+0000","last_active":"2024-09-13T08:37:36.529782+0000","last_peered":"2024-09-13T08:37:36.529782+0000","last_clean":"2024-09-13T08:37:36.529782+0000","last_became_active":"2024-09-13T08:37:33.890252+0000","last_became_peered":"2024-09-13T08:37:33.890252+0000","last_unstale":"2024-09-13T08:37:36.529782+0000","last_undegraded":"2024-09-13T08:37:36.529782+0000","last_fullsized":"2024-09-13T08:37:36.529782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:11.519779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894861+0000","last_change":"2024-09-13T08:37:34.901633+0000","last_active":"2024-09-13T08:37:35.894861+0000","last_peered":"2024-09-13T08:37:35.894861+0000","last_clean":"2024-09-13T08:37:35.894861+0000","last_became_active":"2024-09-13T08:37:34.901323+0000","last_became_peered":"2024-09-13T08:37:34.901323+0000","last_unstale":"2024-09-13T08:37:35.894861+0000","last_undegraded":"2024-09-13T08:37:35.894861+0000","last_fullsized":"2024-09-13T08:37:35.894861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:37.744328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085159+0000","last_change":"2024-09-13T08:37:33.889192+0000","last_active":"2024-09-13T08:37:37.085159+0000","last_peered":"2024-09-13T08:37:37.085159+0000","last_clean":"2024-09-13T08:37:37.085159+0000","last_became_active":"2024-09-13T08:37:33.888556+0000","last_became_peered":"2024-09-13T08:37:33.888556+0000","last_unstale":"2024-09-13T08:37:37.085159+0000","last_undegraded":"2024-09-13T08:37:37.085159+0000","last_fullsized":"2024-09-13T08:37:37.085159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:55.900198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877366+0000","last_change":"2024-09-13T08:37:34.905109+0000","last_active":"2024-09-13T08:37:36.877366+0000","last_peered":"2024-09-13T08:37:36.877366+0000","last_clean":"2024-09-13T08:37:36.877366+0000","last_became_active":"2024-09-13T08:37:34.904766+0000","last_became_peered":"2024-09-13T08:37:34.904766+0000","last_unstale":"2024-09-13T08:37:36.877366+0000","last_undegraded":"2024-09-13T08:37:36.877366+0000","last_fullsized":"2024-09-13T08:37:36.877366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:54.979663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895896+0000","last_change":"2024-09-13T08:37:33.889699+0000","last_active":"2024-09-13T08:37:35.895896+0000","last_peered":"2024-09-13T08:37:35.895896+0000","last_clean":"2024-09-13T08:37:35.895896+0000","last_became_active":"2024-09-13T08:37:33.889224+0000","last_became_peered":"2024-09-13T08:37:33.889224+0000","last_unstale":"2024-09-13T08:37:35.895896+0000","last_undegraded":"2024-09-13T08:37:35.895896+0000","last_fullsized":"2024-09-13T08:37:35.895896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:49:24.413843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.510786+0000","last_change":"2024-09-13T08:37:34.901224+0000","last_active":"2024-09-13T08:37:36.510786+0000","last_peered":"2024-09-13T08:37:36.510786+0000","last_clean":"2024-09-13T08:37:36.510786+0000","last_became_active":"2024-09-13T08:37:34.900890+0000","last_became_peered":"2024-09-13T08:37:34.900890+0000","last_unstale":"2024-09-13T08:37:36.510786+0000","last_undegraded":"2024-09-13T08:37:36.510786+0000","last_fullsized":"2024-09-13T08:37:36.510786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:06.047971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085567+0000","last_change":"2024-09-13T08:37:33.893542+0000","last_active":"2024-09-13T08:37:37.085567+0000","last_peered":"2024-09-13T08:37:37.085567+0000","last_clean":"2024-09-13T08:37:37.085567+0000","last_became_active":"2024-09-13T08:37:33.893231+0000","last_became_peered":"2024-09-13T08:37:33.893231+0000","last_unstale":"2024-09-13T08:37:37.085567+0000","last_undegraded":"2024-09-13T08:37:37.085567+0000","last_fullsized":"2024-09-13T08:37:37.085567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:48.992381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.895292+0000","last_change":"2024-09-13T08:37:34.901999+0000","last_active":"2024-09-13T08:37:35.895292+0000","last_peered":"2024-09-13T08:37:35.895292+0000","last_clean":"2024-09-13T08:37:35.895292+0000","last_became_active":"2024-09-13T08:37:34.901765+0000","last_became_peered":"2024-09-13T08:37:34.901765+0000","last_unstale":"2024-09-13T08:37:35.895292+0000","last_undegraded":"2024-09-13T08:37:35.895292+0000","last_fullsized":"2024-09-13T08:37:35.895292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:12.422693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586075+0000","last_change":"2024-09-13T08:37:33.898217+0000","last_active":"2024-09-13T08:37:36.586075+0000","last_peered":"2024-09-13T08:37:36.586075+0000","last_clean":"2024-09-13T08:37:36.586075+0000","last_became_active":"2024-09-13T08:37:33.897954+0000","last_became_peered":"2024-09-13T08:37:33.897954+0000","last_unstale":"2024-09-13T08:37:36.586075+0000","last_undegraded":"2024-09-13T08:37:36.586075+0000","last_fullsized":"2024-09-13T08:37:36.586075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:05.995257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586135+0000","last_change":"2024-09-13T08:37:34.897630+0000","last_active":"2024-09-13T08:37:36.586135+0000","last_peered":"2024-09-13T08:37:36.586135+0000","last_clean":"2024-09-13T08:37:36.586135+0000","last_became_active":"2024-09-13T08:37:34.896603+0000","last_became_peered":"2024-09-13T08:37:34.896603+0000","last_unstale":"2024-09-13T08:37:36.586135+0000","last_undegraded":"2024-09-13T08:37:36.586135+0000","last_fullsized":"2024-09-13T08:37:36.586135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:13.488345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085624+0000","last_change":"2024-09-13T08:37:33.897139+0000","last_active":"2024-09-13T08:37:37.085624+0000","last_peered":"2024-09-13T08:37:37.085624+0000","last_clean":"2024-09-13T08:37:37.085624+0000","last_became_active":"2024-09-13T08:37:33.896576+0000","last_became_peered":"2024-09-13T08:37:33.896576+0000","last_unstale":"2024-09-13T08:37:37.085624+0000","last_undegraded":"2024-09-13T08:37:37.085624+0000","last_fullsized":"2024-09-13T08:37:37.085624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:54.815810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585844+0000","last_change":"2024-09-13T08:37:34.896701+0000","last_active":"2024-09-13T08:37:36.585844+0000","last_peered":"2024-09-13T08:37:36.585844+0000","last_clean":"2024-09-13T08:37:36.585844+0000","last_became_active":"2024-09-13T08:37:34.893772+0000","last_became_peered":"2024-09-13T08:37:34.893772+0000","last_unstale":"2024-09-13T08:37:36.585844+0000","last_undegraded":"2024-09-13T08:37:36.585844+0000","last_fullsized":"2024-09-13T08:37:36.585844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:34.291551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.876764+0000","last_change":"2024-09-13T08:37:33.893088+0000","last_active":"2024-09-13T08:37:36.876764+0000","last_peered":"2024-09-13T08:37:36.876764+0000","last_clean":"2024-09-13T08:37:36.876764+0000","last_became_active":"2024-09-13T08:37:33.892731+0000","last_became_peered":"2024-09-13T08:37:33.892731+0000","last_unstale":"2024-09-13T08:37:36.876764+0000","last_undegraded":"2024-09-13T08:37:36.876764+0000","last_fullsized":"2024-09-13T08:37:36.876764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084725+0000","last_change":"2024-09-13T08:37:34.895132+0000","last_active":"2024-09-13T08:37:37.084725+0000","last_peered":"2024-09-13T08:37:37.084725+0000","last_clean":"2024-09-13T08:37:37.084725+0000","last_became_active":"2024-09-13T08:37:34.894785+0000","last_became_peered":"2024-09-13T08:37:34.894785+0000","last_unstale":"2024-09-13T08:37:37.084725+0000","last_undegraded":"2024-09-13T08:37:37.084725+0000","last_fullsized":"2024-09-13T08:37:37.084725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:07.683461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585602+0000","last_change":"2024-09-13T08:37:33.898275+0000","last_active":"2024-09-13T08:37:36.585602+0000","last_peered":"2024-09-13T08:37:36.585602+0000","last_clean":"2024-09-13T08:37:36.585602+0000","last_became_active":"2024-09-13T08:37:33.897960+0000","last_became_peered":"2024-09-13T08:37:33.897960+0000","last_unstale":"2024-09-13T08:37:36.585602+0000","last_undegraded":"2024-09-13T08:37:36.585602+0000","last_fullsized":"2024-09-13T08:37:36.585602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:22.150995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.585465+0000","last_change":"2024-09-13T08:37:34.891285+0000","last_active":"2024-09-13T08:37:36.585465+0000","last_peered":"2024-09-13T08:37:36.585465+0000","last_clean":"2024-09-13T08:37:36.585465+0000","last_became_active":"2024-09-13T08:37:34.890996+0000","last_became_peered":"2024-09-13T08:37:34.890996+0000","last_unstale":"2024-09-13T08:37:36.585465+0000","last_undegraded":"2024-09-13T08:37:36.585465+0000","last_fullsized":"2024-09-13T08:37:36.585465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:47.564367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.894655+0000","last_change":"2024-09-13T08:37:33.891510+0000","last_active":"2024-09-13T08:37:35.894655+0000","last_peered":"2024-09-13T08:37:35.894655+0000","last_clean":"2024-09-13T08:37:35.894655+0000","last_became_active":"2024-09-13T08:37:33.890774+0000","last_became_peered":"2024-09-13T08:37:33.890774+0000","last_unstale":"2024-09-13T08:37:35.894655+0000","last_undegraded":"2024-09-13T08:37:35.894655+0000","last_fullsized":"2024-09-13T08:37:35.894655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:25.370931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.529890+0000","last_change":"2024-09-13T08:37:34.895928+0000","last_active":"2024-09-13T08:37:36.529890+0000","last_peered":"2024-09-13T08:37:36.529890+0000","last_clean":"2024-09-13T08:37:36.529890+0000","last_became_active":"2024-09-13T08:37:34.895612+0000","last_became_peered":"2024-09-13T08:37:34.895612+0000","last_unstale":"2024-09-13T08:37:36.529890+0000","last_undegraded":"2024-09-13T08:37:36.529890+0000","last_fullsized":"2024-09-13T08:37:36.529890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:11.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085214+0000","last_change":"2024-09-13T08:37:33.887243+0000","last_active":"2024-09-13T08:37:37.085214+0000","last_peered":"2024-09-13T08:37:37.085214+0000","last_clean":"2024-09-13T08:37:37.085214+0000","last_became_active":"2024-09-13T08:37:33.886882+0000","last_became_peered":"2024-09-13T08:37:33.886882+0000","last_unstale":"2024-09-13T08:37:37.085214+0000","last_undegraded":"2024-09-13T08:37:37.085214+0000","last_fullsized":"2024-09-13T08:37:37.085214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:31.044508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.084923+0000","last_change":"2024-09-13T08:37:34.894740+0000","last_active":"2024-09-13T08:37:37.084923+0000","last_peered":"2024-09-13T08:37:37.084923+0000","last_clean":"2024-09-13T08:37:37.084923+0000","last_became_active":"2024-09-13T08:37:34.894284+0000","last_became_peered":"2024-09-13T08:37:34.894284+0000","last_unstale":"2024-09-13T08:37:37.084923+0000","last_undegraded":"2024-09-13T08:37:37.084923+0000","last_fullsized":"2024-09-13T08:37:37.084923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:28.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:37.085351+0000","last_change":"2024-09-13T08:37:34.892338+0000","last_active":"2024-09-13T08:37:37.085351+0000","last_peered":"2024-09-13T08:37:37.085351+0000","last_clean":"2024-09-13T08:37:37.085351+0000","last_became_active":"2024-09-13T08:37:34.891790+0000","last_became_peered":"2024-09-13T08:37:34.891790+0000","last_unstale":"2024-09-13T08:37:37.085351+0000","last_undegraded":"2024-09-13T08:37:37.085351+0000","last_fullsized":"2024-09-13T08:37:37.085351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:24:06.938838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877706+0000","last_change":"2024-09-13T08:37:33.900929+0000","last_active":"2024-09-13T08:37:36.877706+0000","last_peered":"2024-09-13T08:37:36.877706+0000","last_clean":"2024-09-13T08:37:36.877706+0000","last_became_active":"2024-09-13T08:37:33.899776+0000","last_became_peered":"2024-09-13T08:37:33.899776+0000","last_unstale":"2024-09-13T08:37:36.877706+0000","last_undegraded":"2024-09-13T08:37:36.877706+0000","last_fullsized":"2024-09-13T08:37:36.877706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:27.711361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.892184+0000","last_change":"2024-09-13T08:37:34.893833+0000","last_active":"2024-09-13T08:37:35.892184+0000","last_peered":"2024-09-13T08:37:35.892184+0000","last_clean":"2024-09-13T08:37:35.892184+0000","last_became_active":"2024-09-13T08:37:34.893512+0000","last_became_peered":"2024-09-13T08:37:34.893512+0000","last_unstale":"2024-09-13T08:37:35.892184+0000","last_undegraded":"2024-09-13T08:37:35.892184+0000","last_fullsized":"2024-09-13T08:37:35.892184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:32.052729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.877847+0000","last_change":"2024-09-13T08:37:33.894985+0000","last_active":"2024-09-13T08:37:36.877847+0000","last_peered":"2024-09-13T08:37:36.877847+0000","last_clean":"2024-09-13T08:37:36.877847+0000","last_became_active":"2024-09-13T08:37:33.894087+0000","last_became_peered":"2024-09-13T08:37:33.894087+0000","last_unstale":"2024-09-13T08:37:36.877847+0000","last_undegraded":"2024-09-13T08:37:36.877847+0000","last_fullsized":"2024-09-13T08:37:36.877847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:51.688613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896328+0000","last_change":"2024-09-13T08:37:34.901345+0000","last_active":"2024-09-13T08:37:35.896328+0000","last_peered":"2024-09-13T08:37:35.896328+0000","last_clean":"2024-09-13T08:37:35.896328+0000","last_became_active":"2024-09-13T08:37:34.900290+0000","last_became_peered":"2024-09-13T08:37:34.900290+0000","last_unstale":"2024-09-13T08:37:35.896328+0000","last_undegraded":"2024-09-13T08:37:35.896328+0000","last_fullsized":"2024-09-13T08:37:35.896328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:08.493271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896451+0000","last_change":"2024-09-13T08:37:33.887848+0000","last_active":"2024-09-13T08:37:35.896451+0000","last_peered":"2024-09-13T08:37:35.896451+0000","last_clean":"2024-09-13T08:37:35.896451+0000","last_became_active":"2024-09-13T08:37:33.887508+0000","last_became_peered":"2024-09-13T08:37:33.887508+0000","last_unstale":"2024-09-13T08:37:35.896451+0000","last_undegraded":"2024-09-13T08:37:35.896451+0000","last_fullsized":"2024-09-13T08:37:35.896451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:27.615101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:35.896328+0000","last_change":"2024-09-13T08:37:34.907107+0000","last_active":"2024-09-13T08:37:35.896328+0000","last_peered":"2024-09-13T08:37:35.896328+0000","last_clean":"2024-09-13T08:37:35.896328+0000","last_became_active":"2024-09-13T08:37:34.904160+0000","last_became_peered":"2024-09-13T08:37:34.904160+0000","last_unstale":"2024-09-13T08:37:35.896328+0000","last_undegraded":"2024-09-13T08:37:35.896328+0000","last_fullsized":"2024-09-13T08:37:35.896328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:33.854563+0000","last_clean_scrub_stamp":"2024-09-13T08:37:33.854563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:55.775775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T08:37:36.586114+0000","last_change":"2024-09-13T08:37:33.894082+0000","last_active":"2024-09-13T08:37:36.586114+0000","last_peered":"2024-09-13T08:37:36.586114+0000","last_clean":"2024-09-13T08:37:36.586114+0000","last_became_active":"2024-09-13T08:37:33.893820+0000","last_became_peered":"2024-09-13T08:37:33.893820+0000","last_unstale":"2024-09-13T08:37:36.586114+0000","last_undegraded":"2024-09-13T08:37:36.586114+0000","last_fullsized":"2024-09-13T08:37:36.586114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:32.846342+0000","last_clean_scrub_stamp":"2024-09-13T08:37:32.846342+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:49:48.992381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:37:49.226 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:37:49.226 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T08:37:49.226 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T08:37:49.226 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T08:37:49.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 -- 172.21.15.44:0/210465535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43700b2d00 msgr2=0x7f43701605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 --2- 172.21.15.44:0/210465535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43700b2d00 0x7f43701605d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4368009c80 tx=0x7f436801cd10 comp rx=0 tx=0).stop 2024-09-13T08:37:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 -- 172.21.15.44:0/210465535 shutdown_connections 2024-09-13T08:37:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 --2- 172.21.15.44:0/210465535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43700b2d00 0x7f43701605d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 --2- 172.21.15.44:0/210465535 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43700b2760 0x7f43700d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 -- 172.21.15.44:0/210465535 >> 172.21.15.44:0/210465535 conn(0x7f43700d1e40 msgr2=0x7f43700d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 -- 172.21.15.44:0/210465535 shutdown_connections 2024-09-13T08:37:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 -- 172.21.15.44:0/210465535 wait complete. 2024-09-13T08:37:49.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 Processor -- start 2024-09-13T08:37:49.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 -- start start 2024-09-13T08:37:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43700b2760 0x7f4370151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700b2d00 0x7f43700c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43700cfe80 0x7f4370152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43700b09a0 con 0x7f43700b2760 2024-09-13T08:37:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43700b0820 con 0x7f43700b2d00 2024-09-13T08:37:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4375e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f43700b0b20 con 0x7f43700cfe80 2024-09-13T08:37:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f436ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43700b2760 0x7f4370151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4367e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700b2d00 0x7f43700c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f436ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43700b2760 0x7f4370151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36802/0 (socket says 172.21.15.44:36802) 2024-09-13T08:37:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f436f400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43700cfe80 0x7f4370152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f436ea00640 1 -- 172.21.15.44:0/3926828093 learned_addr learned my addr 172.21.15.44:0/3926828093 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.407+0000 7f4367e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700b2d00 0x7f43700c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:52872/0 (socket says 172.21.15.44:52872) 2024-09-13T08:37:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f436ea00640 1 -- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43700cfe80 msgr2=0x7f4370152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f436ea00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43700cfe80 0x7f4370152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f436ea00640 1 -- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700b2d00 msgr2=0x7f43700c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f436ea00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700b2d00 0x7f43700c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f436ea00640 1 -- 172.21.15.44:0/3926828093 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43701535f0 con 0x7f43700b2760 2024-09-13T08:37:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f436ea00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43700b2760 0x7f4370151880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f436000c3f0 tx=0x7f436000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f4367400640 1 -- 172.21.15.44:0/3926828093 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360017070 con 0x7f43700b2760 2024-09-13T08:37:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f4367400640 1 -- 172.21.15.44:0/3926828093 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43600053c0 con 0x7f43700b2760 2024-09-13T08:37:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f4375e00640 1 -- 172.21.15.44:0/3926828093 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43700c7520 con 0x7f43700b2760 2024-09-13T08:37:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f4367400640 1 -- 172.21.15.44:0/3926828093 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360003860 con 0x7f43700b2760 2024-09-13T08:37:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f4375e00640 1 -- 172.21.15.44:0/3926828093 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4370152320 con 0x7f43700b2760 2024-09-13T08:37:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f4375e00640 1 -- 172.21.15.44:0/3926828093 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43700d0260 con 0x7f43700b2760 2024-09-13T08:37:49.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f4367400640 1 -- 172.21.15.44:0/3926828093 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f436001a070 con 0x7f43700b2760 2024-09-13T08:37:49.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.411+0000 7f4367400640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f434c072ec0 0x7f434c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.415+0000 7f4367400640 1 -- 172.21.15.44:0/3926828093 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4360093100 con 0x7f43700b2760 2024-09-13T08:37:49.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.415+0000 7f4367e00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f434c072ec0 0x7f434c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.415+0000 7f4367e00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f434c072ec0 0x7f434c075380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43700aefb0 tx=0x7f435800b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:49.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.415+0000 7f4367400640 1 -- 172.21.15.44:0/3926828093 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f43700d0c80 con 0x7f43700b2760 2024-09-13T08:37:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 -- 172.21.15.44:0/3926828093 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f43700aed00 con 0x7f43700b2760 2024-09-13T08:37:49.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4367400640 1 -- 172.21.15.44:0/3926828093 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f436005dc60 con 0x7f43700b2760 2024-09-13T08:37:49.634 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:49.635 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T08:37:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 -- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f434c072ec0 msgr2=0x7f434c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f434c072ec0 0x7f434c075380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43700aefb0 tx=0x7f435800b390 comp rx=0 tx=0).stop 2024-09-13T08:37:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 -- 172.21.15.44:0/3926828093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43700b2760 msgr2=0x7f4370151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43700b2760 0x7f4370151880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f436000c3f0 tx=0x7f436000c8b0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 -- 172.21.15.44:0/3926828093 shutdown_connections 2024-09-13T08:37:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f434c072ec0 0x7f434c075380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43700cfe80 0x7f4370152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700b2d00 0x7f43700c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 --2- 172.21.15.44:0/3926828093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43700b2760 0x7f4370151880 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 -- 172.21.15.44:0/3926828093 >> 172.21.15.44:0/3926828093 conn(0x7f43700d1e40 msgr2=0x7f43700cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 -- 172.21.15.44:0/3926828093 shutdown_connections 2024-09-13T08:37:49.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.631+0000 7f4375e00640 1 -- 172.21.15.44:0/3926828093 wait complete. 2024-09-13T08:37:49.657 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T08:37:49.657 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:37:49.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.799+0000 7f6e4b600640 1 -- 172.21.15.44:0/372751083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4415c1c0 msgr2=0x7f6e44071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.799+0000 7f6e4b600640 1 --2- 172.21.15.44:0/372751083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4415c1c0 0x7f6e44071740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009bf0 tx=0x7f6e3c01cba0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.799+0000 7f6e4b600640 1 -- 172.21.15.44:0/372751083 shutdown_connections 2024-09-13T08:37:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.799+0000 7f6e4b600640 1 --2- 172.21.15.44:0/372751083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4415c1c0 0x7f6e44071740 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.799+0000 7f6e4b600640 1 -- 172.21.15.44:0/372751083 >> 172.21.15.44:0/372751083 conn(0x7f6e440d0c90 msgr2=0x7f6e440d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.799+0000 7f6e4b600640 1 -- 172.21.15.44:0/372751083 shutdown_connections 2024-09-13T08:37:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.799+0000 7f6e4b600640 1 -- 172.21.15.44:0/372751083 wait complete. 2024-09-13T08:37:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 Processor -- start 2024-09-13T08:37:49.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 -- start start 2024-09-13T08:37:49.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4406cc20 0x7f6e44155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e440737c0 0x7f6e440b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4415c1c0 0x7f6e440b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4416a990 con 0x7f6e4415c1c0 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6e44156670 con 0x7f6e4406cc20 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4416ab10 con 0x7f6e440737c0 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e49600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4415c1c0 0x7f6e440b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e49600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4415c1c0 0x7f6e440b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36828/0 (socket says 172.21.15.44:36828) 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e49600640 1 -- 172.21.15.44:0/2938433399 learned_addr learned my addr 172.21.15.44:0/2938433399 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e48c00640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4406cc20 0x7f6e44155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e3be00640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e440737c0 0x7f6e440b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e49600640 1 -- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e440737c0 msgr2=0x7f6e440b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e49600640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e440737c0 0x7f6e440b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e49600640 1 -- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4406cc20 msgr2=0x7f6e44155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e49600640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4406cc20 0x7f6e44155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e49600640 1 -- 172.21.15.44:0/2938433399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e440b4000 con 0x7f6e4415c1c0 2024-09-13T08:37:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e3be00640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e440737c0 0x7f6e440b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:49.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e48c00640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4406cc20 0x7f6e44155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:49.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e49600640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4415c1c0 0x7f6e440b6450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009b10 tx=0x7f6e3c0a5ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:49.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e41600640 1 -- 172.21.15.44:0/2938433399 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0bb070 con 0x7f6e4415c1c0 2024-09-13T08:37:49.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e41600640 1 -- 172.21.15.44:0/2938433399 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e3c004d40 con 0x7f6e4415c1c0 2024-09-13T08:37:49.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e41600640 1 -- 172.21.15.44:0/2938433399 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0b2040 con 0x7f6e4415c1c0 2024-09-13T08:37:49.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 -- 172.21.15.44:0/2938433399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e441f8850 con 0x7f6e4415c1c0 2024-09-13T08:37:49.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 -- 172.21.15.44:0/2938433399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e441f8af0 con 0x7f6e4415c1c0 2024-09-13T08:37:49.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.803+0000 7f6e4b600640 1 -- 172.21.15.44:0/2938433399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e4406d000 con 0x7f6e4415c1c0 2024-09-13T08:37:49.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.807+0000 7f6e41600640 1 -- 172.21.15.44:0/2938433399 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f6e3c0be070 con 0x7f6e4415c1c0 2024-09-13T08:37:49.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.807+0000 7f6e41600640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e20072ec0 0x7f6e20075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:49.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.807+0000 7f6e41600640 1 -- 172.21.15.44:0/2938433399 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6e3c12dac0 con 0x7f6e4415c1c0 2024-09-13T08:37:49.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.807+0000 7f6e48c00640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e20072ec0 0x7f6e20075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:49.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.807+0000 7f6e48c00640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e20072ec0 0x7f6e20075380 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6e440b1a50 tx=0x7f6e34009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:49.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:49.811+0000 7f6e41600640 1 -- 172.21.15.44:0/2938433399 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6e440b4000 con 0x7f6e4415c1c0 2024-09-13T08:37:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.027+0000 7f6e4b600640 1 -- 172.21.15.44:0/2938433399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e441f8cc0 con 0x7f6e4415c1c0 2024-09-13T08:37:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.027+0000 7f6e41600640 1 -- 172.21.15.44:0/2938433399 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f6e3c0a5e20 con 0x7f6e4415c1c0 2024-09-13T08:37:50.032 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:50.032 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"btime":"2024-09-13T08:37:37:896662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2931832597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2931832597},{"type":"v1","addr":"172.21.15.129:6837","nonce":2931832597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4335,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/853789824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":853789824},{"type":"v1","addr":"172.21.15.44:6834","nonce":853789824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/686971411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":686971411},{"type":"v1","addr":"172.21.15.44:6835","nonce":686971411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:37:34.290379+0000","modified":"2024-09-13T08:37:37.298049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4360":{"gid":4360,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6835/1614710769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1614710769},{"type":"v1","addr":"172.21.15.129:6835","nonce":1614710769}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4360,"qdb_cluster":[4360]},"id":1}]} 2024-09-13T08:37:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-09-13T08:37:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 -- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e20072ec0 msgr2=0x7f6e20075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e20072ec0 0x7f6e20075380 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6e440b1a50 tx=0x7f6e34009040 comp rx=0 tx=0).stop 2024-09-13T08:37:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 -- 172.21.15.44:0/2938433399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4415c1c0 msgr2=0x7f6e440b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4415c1c0 0x7f6e440b6450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009b10 tx=0x7f6e3c0a5ae0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 -- 172.21.15.44:0/2938433399 shutdown_connections 2024-09-13T08:37:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e20072ec0 0x7f6e20075380 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4415c1c0 0x7f6e440b6450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e440737c0 0x7f6e440b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 --2- 172.21.15.44:0/2938433399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4406cc20 0x7f6e44155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 -- 172.21.15.44:0/2938433399 >> 172.21.15.44:0/2938433399 conn(0x7f6e440d0c90 msgr2=0x7f6e44155540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 -- 172.21.15.44:0/2938433399 shutdown_connections 2024-09-13T08:37:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.031+0000 7f6e4b600640 1 -- 172.21.15.44:0/2938433399 wait complete. 2024-09-13T08:37:50.056 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:37:50.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.199+0000 7f9029600640 1 -- 172.21.15.44:0/3222330406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902415bea0 msgr2=0x7f90240ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.199+0000 7f9029600640 1 --2- 172.21.15.44:0/3222330406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902415bea0 0x7f90240ce670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f90240cc3a0 tx=0x7f901401c770 comp rx=0 tx=0).stop 2024-09-13T08:37:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.199+0000 7f9029600640 1 -- 172.21.15.44:0/3222330406 shutdown_connections 2024-09-13T08:37:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.199+0000 7f9029600640 1 --2- 172.21.15.44:0/3222330406 >> v2:172.21.15.129:3300/0 conn(0x7f90240d28e0 0x7f90240cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.199+0000 7f9029600640 1 --2- 172.21.15.44:0/3222330406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902415bea0 0x7f90240ce670 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.199+0000 7f9029600640 1 -- 172.21.15.44:0/3222330406 >> 172.21.15.44:0/3222330406 conn(0x7f90240d1dc0 msgr2=0x7f90240d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.199+0000 7f9029600640 1 -- 172.21.15.44:0/3222330406 shutdown_connections 2024-09-13T08:37:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.199+0000 7f9029600640 1 -- 172.21.15.44:0/3222330406 wait complete. 2024-09-13T08:37:50.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9029600640 1 Processor -- start 2024-09-13T08:37:50.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9029600640 1 -- start start 2024-09-13T08:37:50.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9029600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90240cdb30 0x7f9024156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9029600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90240d28e0 0x7f9024159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9029600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902415bea0 0x7f90240b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9029600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9024168fc0 con 0x7f902415bea0 2024-09-13T08:37:50.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9029600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9024168e40 con 0x7f90240cdb30 2024-09-13T08:37:50.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9029600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9024169140 con 0x7f90240d28e0 2024-09-13T08:37:50.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9023400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902415bea0 0x7f90240b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:50.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9023400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902415bea0 0x7f90240b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36840/0 (socket says 172.21.15.44:36840) 2024-09-13T08:37:50.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9023400640 1 -- 172.21.15.44:0/2094037750 learned_addr learned my addr 172.21.15.44:0/2094037750 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:50.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9022a00640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90240cdb30 0x7f9024156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:50.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f901a000640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90240d28e0 0x7f9024159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9023400640 1 -- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90240d28e0 msgr2=0x7f9024159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9023400640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90240d28e0 0x7f9024159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9023400640 1 -- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90240cdb30 msgr2=0x7f9024156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9023400640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90240cdb30 0x7f9024156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9023400640 1 -- 172.21.15.44:0/2094037750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902406de60 con 0x7f902415bea0 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9023400640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902415bea0 0x7f90240b56a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f901c00eb50 tx=0x7f901c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f901b400640 1 -- 172.21.15.44:0/2094037750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c004d30 con 0x7f902415bea0 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f901b400640 1 -- 172.21.15.44:0/2094037750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f901c00ee50 con 0x7f902415bea0 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9029600640 1 -- 172.21.15.44:0/2094037750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902406e140 con 0x7f902415bea0 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f901b400640 1 -- 172.21.15.44:0/2094037750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c005280 con 0x7f902415bea0 2024-09-13T08:37:50.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.203+0000 7f9029600640 1 -- 172.21.15.44:0/2094037750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f902406e700 con 0x7f902415bea0 2024-09-13T08:37:50.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.207+0000 7f9029600640 1 -- 172.21.15.44:0/2094037750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90240cc9d0 con 0x7f902415bea0 2024-09-13T08:37:50.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.207+0000 7f901b400640 1 -- 172.21.15.44:0/2094037750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f901c005420 con 0x7f902415bea0 2024-09-13T08:37:50.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.207+0000 7f901b400640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9000072ef0 0x7f90000753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.207+0000 7f901b400640 1 -- 172.21.15.44:0/2094037750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f901c094910 con 0x7f902415bea0 2024-09-13T08:37:50.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.207+0000 7f9022a00640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9000072ef0 0x7f90000753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:50.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.211+0000 7f9022a00640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9000072ef0 0x7f90000753b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f90140af000 tx=0x7f901401afb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:50.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.211+0000 7f901b400640 1 -- 172.21.15.44:0/2094037750 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f901c05f5a0 con 0x7f902415bea0 2024-09-13T08:37:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.427+0000 7f9029600640 1 -- 172.21.15.44:0/2094037750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9024072440 con 0x7f902415bea0 2024-09-13T08:37:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.427+0000 7f901b400640 1 -- 172.21.15.44:0/2094037750 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f901c063fd0 con 0x7f902415bea0 2024-09-13T08:37:50.434 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:50.434 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"btime":"2024-09-13T08:37:37:896662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2931832597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2931832597},{"type":"v1","addr":"172.21.15.129:6837","nonce":2931832597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4335,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/853789824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":853789824},{"type":"v1","addr":"172.21.15.44:6834","nonce":853789824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/686971411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":686971411},{"type":"v1","addr":"172.21.15.44:6835","nonce":686971411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:37:34.290379+0000","modified":"2024-09-13T08:37:37.298049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4360":{"gid":4360,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6835/1614710769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1614710769},{"type":"v1","addr":"172.21.15.129:6835","nonce":1614710769}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4360,"qdb_cluster":[4360]},"id":1}]} 2024-09-13T08:37:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-09-13T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 -- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9000072ef0 msgr2=0x7f90000753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9000072ef0 0x7f90000753b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f90140af000 tx=0x7f901401afb0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 -- 172.21.15.44:0/2094037750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902415bea0 msgr2=0x7f90240b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902415bea0 0x7f90240b56a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f901c00eb50 tx=0x7f901c00c6a0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 -- 172.21.15.44:0/2094037750 shutdown_connections 2024-09-13T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9000072ef0 0x7f90000753b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902415bea0 0x7f90240b56a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90240d28e0 0x7f9024159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 --2- 172.21.15.44:0/2094037750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90240cdb30 0x7f9024156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 -- 172.21.15.44:0/2094037750 >> 172.21.15.44:0/2094037750 conn(0x7f90240d1dc0 msgr2=0x7f902415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 -- 172.21.15.44:0/2094037750 shutdown_connections 2024-09-13T08:37:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.431+0000 7f9029600640 1 -- 172.21.15.44:0/2094037750 wait complete. 2024-09-13T08:37:50.456 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:37:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 -- 172.21.15.44:0/671311138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77640ced80 msgr2=0x7f77640ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 --2- 172.21.15.44:0/671311138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77640ced80 0x7f77640ae9b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f774c009840 tx=0x7f774c01c820 comp rx=0 tx=0).stop 2024-09-13T08:37:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 -- 172.21.15.44:0/671311138 shutdown_connections 2024-09-13T08:37:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 --2- 172.21.15.44:0/671311138 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77640cdb30 0x7f77640aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 --2- 172.21.15.44:0/671311138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77640ced80 0x7f77640ae9b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 -- 172.21.15.44:0/671311138 >> 172.21.15.44:0/671311138 conn(0x7f77640d1dc0 msgr2=0x7f77640d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 -- 172.21.15.44:0/671311138 shutdown_connections 2024-09-13T08:37:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 -- 172.21.15.44:0/671311138 wait complete. 2024-09-13T08:37:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 Processor -- start 2024-09-13T08:37:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 -- start start 2024-09-13T08:37:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77640cdb30 0x7f7764152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77640ced80 0x7f77641fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f776415bea0 0x7f7764158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7764169340 con 0x7f77640ced80 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77641691c0 con 0x7f77640cdb30 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f7769e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f77641694c0 con 0x7f776415bea0 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f775be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77640ced80 0x7f77641fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f775be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77640ced80 0x7f77641fbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36852/0 (socket says 172.21.15.44:36852) 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.599+0000 7f775be00640 1 -- 172.21.15.44:0/2099247243 learned_addr learned my addr 172.21.15.44:0/2099247243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775be00640 1 -- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f776415bea0 msgr2=0x7f7764158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775be00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f776415bea0 0x7f7764158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775be00640 1 -- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77640cdb30 msgr2=0x7f7764152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775be00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77640cdb30 0x7f7764152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775be00640 1 -- 172.21.15.44:0/2099247243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7764152ab0 con 0x7f77640ced80 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775be00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77640ced80 0x7f77641fbaf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f774c009790 tx=0x7f774c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775b400640 1 -- 172.21.15.44:0/2099247243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0b2040 con 0x7f77640ced80 2024-09-13T08:37:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775b400640 1 -- 172.21.15.44:0/2099247243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f774c004320 con 0x7f77640ced80 2024-09-13T08:37:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775b400640 1 -- 172.21.15.44:0/2099247243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0044f0 con 0x7f77640ced80 2024-09-13T08:37:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f7769e00640 1 -- 172.21.15.44:0/2099247243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77641fc030 con 0x7f77640ced80 2024-09-13T08:37:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f7769e00640 1 -- 172.21.15.44:0/2099247243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77640ca7a0 con 0x7f77640ced80 2024-09-13T08:37:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f7769e00640 1 -- 172.21.15.44:0/2099247243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77640ce670 con 0x7f77640ced80 2024-09-13T08:37:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775b400640 1 -- 172.21.15.44:0/2099247243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f774c0bd060 con 0x7f77640ced80 2024-09-13T08:37:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775b400640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7740072ec0 0x7f7740075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.603+0000 7f775b400640 1 -- 172.21.15.44:0/2099247243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f774c12d520 con 0x7f77640ced80 2024-09-13T08:37:50.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.607+0000 7f7762a00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7740072ec0 0x7f7740075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:50.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.607+0000 7f7762a00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7740072ec0 0x7f7740075380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f77540046c0 tx=0x7f7754007910 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.611+0000 7f775b400640 1 -- 172.21.15.44:0/2099247243 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f77641fc030 con 0x7f77640ced80 2024-09-13T08:37:50.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.791+0000 7f7769e00640 1 -- 172.21.15.44:0/2099247243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77640d21a0 con 0x7f77640ced80 2024-09-13T08:37:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.791+0000 7f775b400640 1 -- 172.21.15.44:0/2099247243 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16681 (secure 0 0 0) 0x7f774c016410 con 0x7f77640ced80 2024-09-13T08:37:50.796 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:50.796 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":22,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:35.883263+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:37:32.247896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T08:37:33.241202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 -- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7740072ec0 msgr2=0x7f7740075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7740072ec0 0x7f7740075380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f77540046c0 tx=0x7f7754007910 comp rx=0 tx=0).stop 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 -- 172.21.15.44:0/2099247243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77640ced80 msgr2=0x7f77641fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77640ced80 0x7f77641fbaf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f774c009790 tx=0x7f774c005dc0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 -- 172.21.15.44:0/2099247243 shutdown_connections 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7740072ec0 0x7f7740075380 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f776415bea0 0x7f7764158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77640ced80 0x7f77641fbaf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 --2- 172.21.15.44:0/2099247243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77640cdb30 0x7f7764152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 -- 172.21.15.44:0/2099247243 >> 172.21.15.44:0/2099247243 conn(0x7f77640d1dc0 msgr2=0x7f776406f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 -- 172.21.15.44:0/2099247243 shutdown_connections 2024-09-13T08:37:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.795+0000 7f7769e00640 1 -- 172.21.15.44:0/2099247243 wait complete. 2024-09-13T08:37:50.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:37:50.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.967+0000 7f2949a00640 1 -- 172.21.15.44:0/2510134423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29280039f0 msgr2=0x7f2928003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.967+0000 7f2949a00640 1 --2- 172.21.15.44:0/2510134423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29280039f0 0x7f2928003dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2934001ec0 tx=0x7f293401cf60 comp rx=0 tx=0).stop 2024-09-13T08:37:50.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.967+0000 7f2943400640 1 -- 172.21.15.44:0/2510134423 reap_dead start 2024-09-13T08:37:50.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.967+0000 7f2949a00640 1 -- 172.21.15.44:0/2510134423 shutdown_connections 2024-09-13T08:37:50.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.967+0000 7f2949a00640 1 -- 172.21.15.44:0/2510134423 >> 172.21.15.44:0/2510134423 conn(0x7f29440d1dc0 msgr2=0x7f29440d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:50.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.967+0000 7f2949a00640 1 -- 172.21.15.44:0/2510134423 shutdown_connections 2024-09-13T08:37:50.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.967+0000 7f2949a00640 1 -- 172.21.15.44:0/2510134423 wait complete. 2024-09-13T08:37:50.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 Processor -- start 2024-09-13T08:37:50.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 -- start start 2024-09-13T08:37:50.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f29280039f0 0x7f2944156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29441f7a30 0x7f2944155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f294406ea80 0x7f294406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2944167f10 con 0x7f294406ea80 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2944167d90 con 0x7f29441f7a30 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2944168090 con 0x7f29280039f0 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2943400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f294406ea80 0x7f294406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f293be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29441f7a30 0x7f2944155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2942a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f29280039f0 0x7f2944156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2943400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f294406ea80 0x7f294406ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36876/0 (socket says 172.21.15.44:36876) 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2943400640 1 -- 172.21.15.44:0/3004149773 learned_addr learned my addr 172.21.15.44:0/3004149773 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f293be00640 1 -- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f29280039f0 msgr2=0x7f2944156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f293be00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f29280039f0 0x7f2944156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f293be00640 1 -- 172.21.15.44:0/3004149773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f294406ea80 msgr2=0x7f294406ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f293be00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f294406ea80 0x7f294406ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f293be00640 1 -- 172.21.15.44:0/3004149773 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2944156150 con 0x7f29441f7a30 2024-09-13T08:37:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2943400640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f294406ea80 0x7f294406ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:37:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f293be00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29441f7a30 0x7f2944155c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f292c006fd0 tx=0x7f292c001720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f293aa00640 1 -- 172.21.15.44:0/3004149773 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c015070 con 0x7f29441f7a30 2024-09-13T08:37:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f293aa00640 1 -- 172.21.15.44:0/3004149773 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f292c011840 con 0x7f29441f7a30 2024-09-13T08:37:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 -- 172.21.15.44:0/3004149773 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29441f8080 con 0x7f29441f7a30 2024-09-13T08:37:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f293aa00640 1 -- 172.21.15.44:0/3004149773 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c011c90 con 0x7f29441f7a30 2024-09-13T08:37:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 -- 172.21.15.44:0/3004149773 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29441f8640 con 0x7f29441f7a30 2024-09-13T08:37:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.971+0000 7f2949a00640 1 -- 172.21.15.44:0/3004149773 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29440ae9b0 con 0x7f29441f7a30 2024-09-13T08:37:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.975+0000 7f293aa00640 1 -- 172.21.15.44:0/3004149773 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f292c018070 con 0x7f29441f7a30 2024-09-13T08:37:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.975+0000 7f293aa00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2920072ec0 0x7f2920075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.975+0000 7f293aa00640 1 -- 172.21.15.44:0/3004149773 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f292c0949e0 con 0x7f29441f7a30 2024-09-13T08:37:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.975+0000 7f2942a00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2920072ec0 0x7f2920075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.975+0000 7f2942a00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2920072ec0 0x7f2920075380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2934001ec0 tx=0x7f29340a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:50.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:50.979+0000 7f293aa00640 1 -- 172.21.15.44:0/3004149773 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2944156150 con 0x7f29441f7a30 2024-09-13T08:37:51.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.191+0000 7f2949a00640 1 -- 172.21.15.44:0/3004149773 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f294415a7f0 con 0x7f29441f7a30 2024-09-13T08:37:51.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.191+0000 7f293aa00640 1 -- 172.21.15.44:0/3004149773 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f292c05f540 con 0x7f29441f7a30 2024-09-13T08:37:51.197 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:37:51.198 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"btime":"2024-09-13T08:37:37:896662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2931832597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2931832597},{"type":"v1","addr":"172.21.15.129:6837","nonce":2931832597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4335,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/853789824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":853789824},{"type":"v1","addr":"172.21.15.44:6834","nonce":853789824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/686971411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":686971411},{"type":"v1","addr":"172.21.15.44:6835","nonce":686971411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:37:34.290379+0000","modified":"2024-09-13T08:37:37.298049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4360":{"gid":4360,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6835/1614710769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1614710769},{"type":"v1","addr":"172.21.15.129:6835","nonce":1614710769}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4360,"qdb_cluster":[4360]},"id":1}]} 2024-09-13T08:37:51.198 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-09-13T08:37:51.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 -- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2920072ec0 msgr2=0x7f2920075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2920072ec0 0x7f2920075380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2934001ec0 tx=0x7f29340a7040 comp rx=0 tx=0).stop 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 -- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29441f7a30 msgr2=0x7f2944155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29441f7a30 0x7f2944155c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f292c006fd0 tx=0x7f292c001720 comp rx=0 tx=0).stop 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 -- 172.21.15.44:0/3004149773 shutdown_connections 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2920072ec0 0x7f2920075380 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f294406ea80 0x7f294406ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29441f7a30 0x7f2944155c10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 --2- 172.21.15.44:0/3004149773 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f29280039f0 0x7f2944156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 -- 172.21.15.44:0/3004149773 >> 172.21.15.44:0/3004149773 conn(0x7f29440d1dc0 msgr2=0x7f294415ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 -- 172.21.15.44:0/3004149773 shutdown_connections 2024-09-13T08:37:51.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.195+0000 7f2949a00640 1 -- 172.21.15.44:0/3004149773 wait complete. 2024-09-13T08:37:51.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T08:37:34.290379+0000', 'modified': '2024-09-13T08:37:37.298049+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4360}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4360': {'gid': 4360, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.129:6835/1614710769', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 1614710769}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 1614710769}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4360, 'qdb_cluster': [4360]} 2024-09-13T08:37:51.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T08:37:51.623 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-13T08:37:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.767+0000 7fa6fb400640 1 -- 172.21.15.44:0/1479870775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f40b49e0 msgr2=0x7fa6f406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.767+0000 7fa6fb400640 1 --2- 172.21.15.44:0/1479870775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f40b49e0 0x7fa6f406f7b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa6f406aab0 tx=0x7fa6ec01ca60 comp rx=0 tx=0).stop 2024-09-13T08:37:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.767+0000 7fa6fb400640 1 -- 172.21.15.44:0/1479870775 shutdown_connections 2024-09-13T08:37:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.767+0000 7fa6fb400640 1 --2- 172.21.15.44:0/1479870775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f40b49e0 0x7fa6f406f7b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.767+0000 7fa6fb400640 1 --2- 172.21.15.44:0/1479870775 >> v2:172.21.15.129:3300/0 conn(0x7fa6f415dae0 0x7fa6f40acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.767+0000 7fa6fb400640 1 -- 172.21.15.44:0/1479870775 >> 172.21.15.44:0/1479870775 conn(0x7fa6f40b18f0 msgr2=0x7fa6f40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.767+0000 7fa6fb400640 1 -- 172.21.15.44:0/1479870775 shutdown_connections 2024-09-13T08:37:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.767+0000 7fa6fb400640 1 -- 172.21.15.44:0/1479870775 wait complete. 2024-09-13T08:37:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6fb400640 1 Processor -- start 2024-09-13T08:37:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6fb400640 1 -- start start 2024-09-13T08:37:51.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6fb400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6f40b49e0 0x7fa6f415cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6fb400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6f40cd100 0x7fa6f40af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6fb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f415dae0 0x7fa6f409adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6fb400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f41674b0 con 0x7fa6f415dae0 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6fb400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4167330 con 0x7fa6f40cd100 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6fb400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f4167630 con 0x7fa6f40b49e0 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f415dae0 0x7fa6f409adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f415dae0 0x7fa6f409adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36896/0 (socket says 172.21.15.44:36896) 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f9400640 1 -- 172.21.15.44:0/1039537905 learned_addr learned my addr 172.21.15.44:0/1039537905 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6f40b49e0 0x7fa6f415cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6f40cd100 0x7fa6f40af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f9400640 1 -- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6f40b49e0 msgr2=0x7fa6f415cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f9400640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6f40b49e0 0x7fa6f415cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f9400640 1 -- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6f40cd100 msgr2=0x7fa6f40af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f9400640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6f40cd100 0x7fa6f40af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f9400640 1 -- 172.21.15.44:0/1039537905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f41f7bc0 con 0x7fa6f415dae0 2024-09-13T08:37:51.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f9400640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f415dae0 0x7fa6f409adc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009b10 tx=0x7fa6ec0a5910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:51.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f0c00640 1 -- 172.21.15.44:0/1039537905 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0bb070 con 0x7fa6f415dae0 2024-09-13T08:37:51.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f0c00640 1 -- 172.21.15.44:0/1039537905 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6ec004e00 con 0x7fa6f415dae0 2024-09-13T08:37:51.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6f0c00640 1 -- 172.21.15.44:0/1039537905 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0b2040 con 0x7fa6f415dae0 2024-09-13T08:37:51.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6fb400640 1 -- 172.21.15.44:0/1039537905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f41f7ea0 con 0x7fa6f415dae0 2024-09-13T08:37:51.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.771+0000 7fa6fb400640 1 -- 172.21.15.44:0/1039537905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f41f8460 con 0x7fa6f415dae0 2024-09-13T08:37:51.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f0c00640 1 -- 172.21.15.44:0/1039537905 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fa6ec0be070 con 0x7fa6f415dae0 2024-09-13T08:37:51.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f0c00640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6d4072ef0 0x7fa6d40753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f0c00640 1 -- 172.21.15.44:0/1039537905 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa6ec12ddf0 con 0x7fa6f415dae0 2024-09-13T08:37:51.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6fe7c4000 1 Processor -- start 2024-09-13T08:37:51.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6d4072ef0 0x7fa6d40753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6fe7c4000 1 -- start start 2024-09-13T08:37:51.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6d4072ef0 0x7fa6d40753b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8006fd0 tx=0x7fa6e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:51.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03e9d0 0x559c1d03edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03f2f0 0x559c1cff28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1cff31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559c1cf133f0 con 0x559c1cff2e10 2024-09-13T08:37:51.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559c1ce50b20 con 0x559c1d03e9d0 2024-09-13T08:37:51.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x559c1ce3ff50 con 0x559c1d03f2f0 2024-09-13T08:37:51.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1cff31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f8a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03e9d0 0x559c1d03edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1cff31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36898/0 (socket says 172.21.15.44:36898) 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f9400640 1 -- 172.21.15.44:0/279541914 learned_addr learned my addr 172.21.15.44:0/279541914 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/279541914 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03f2f0 0x559c1cff28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f9400640 1 -- 172.21.15.44:0/279541914 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03f2f0 msgr2=0x559c1cff28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f9400640 1 --2- 172.21.15.44:0/279541914 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03f2f0 0x559c1cff28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f9400640 1 -- 172.21.15.44:0/279541914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03e9d0 msgr2=0x559c1d03edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f9400640 1 --2- 172.21.15.44:0/279541914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03e9d0 0x559c1d03edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.775+0000 7fa6f9400640 1 -- 172.21.15.44:0/279541914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c1ce07590 con 0x559c1cff2e10 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f9400640 1 --2- 172.21.15.44:0/279541914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1cff31f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0f8630 tx=0x7fa6ec0f8280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd64cd05d414814 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6e5600640 1 -- 172.21.15.44:0/279541914 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0fd1e0 con 0x559c1cff2e10 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6e5600640 1 -- 172.21.15.44:0/279541914 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6ec0fd030 con 0x559c1cff2e10 2024-09-13T08:37:51.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6e5600640 1 -- 172.21.15.44:0/279541914 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0f9030 con 0x559c1cff2e10 2024-09-13T08:37:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/279541914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 msgr2=0x559c1cff31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/279541914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1cff31f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0f8630 tx=0x7fa6ec0f8280 comp rx=0 tx=0).stop 2024-09-13T08:37:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/279541914 shutdown_connections 2024-09-13T08:37:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/279541914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1cff31f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/279541914 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03f2f0 0x559c1cff28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/279541914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03e9d0 0x559c1d03edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/279541914 >> 172.21.15.44:0/279541914 conn(0x559c1cedc130 msgr2=0x559c1d021e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/279541914 shutdown_connections 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/279541914 wait complete. 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 -- start start 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 Processor -- start 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1cff2e10 0x559c1d01f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03e9d0 0x559c1d01fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 0x559c1d0203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559c1cdda5d0 con 0x559c1d03e9d0 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559c1ce43d60 con 0x559c1d03f2f0 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x559c1cde6500 con 0x559c1cff2e10 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6fe7c4000 10 client.0 _ll_register_callbacks cb 0x559c1cdea810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:37:51.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03e9d0 0x559c1d01fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03e9d0 0x559c1d01fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36912/0 (socket says 172.21.15.44:36912) 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f3e00640 1 -- 172.21.15.44:0/2613400976 learned_addr learned my addr 172.21.15.44:0/2613400976 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1cff2e10 0x559c1d01f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f9400640 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 0x559c1d0203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f3e00640 1 -- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1cff2e10 msgr2=0x559c1d01f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1cff2e10 0x559c1d01f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f3e00640 1 -- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 msgr2=0x559c1d0203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 0x559c1d0203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.779+0000 7fa6f3e00640 1 -- 172.21.15.44:0/2613400976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c1cdde170 con 0x559c1d03e9d0 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03e9d0 0x559c1d01fe70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0011280 tx=0x7fa6e0011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2a6625c28a6942 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:51.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6da000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6da000640 1 -- 172.21.15.44:0/2613400976 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0019020 con 0x559c1d03e9d0 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6da000640 1 -- 172.21.15.44:0/2613400976 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa6e0005260 con 0x559c1d03e9d0 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2613400976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c1cdd80f0 con 0x559c1d03e9d0 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6da000640 1 -- 172.21.15.44:0/2613400976 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fa6e0023050 con 0x559c1d03e9d0 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 10 client.4680 fetch_fsmap learned FSMap version 8 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2613400976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x559c1cec4ef0 con 0x559c1d03e9d0 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6da000640 1 -- 172.21.15.44:0/2613400976 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559c1cdd80f0 con 0x559c1d03e9d0 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6da000640 1 -- 172.21.15.44:0/2613400976 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fa6e0026020 con 0x559c1d03e9d0 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 10 client.4680 fetch_fsmap finished waiting for FSMap version 8 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 10 client.4680 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 20 client.4680 populate_metadata read hostname 'smithi044' 2024-09-13T08:37:51.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:6834/1614710769,v1:172.21.15.129:6835/1614710769] conn(0x559c1d0210f0 0x559c1d050c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 4 client.4680 mds_command: new command op to 4360 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T08:37:51.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2613400976 --> [v2:172.21.15.129:6834/1614710769,v1:172.21.15.129:6835/1614710769] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c1ce52a70 con 0x559c1d0210f0 2024-09-13T08:37:51.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:6834/1614710769,v1:172.21.15.129:6835/1614710769] conn(0x559c1d0210f0 0x559c1d050c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6da000640 10 client.4680 ms_handle_connect on v2:172.21.15.129:6834/1614710769 2024-09-13T08:37:51.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:6834/1614710769,v1:172.21.15.129:6835/1614710769] conn(0x559c1d0210f0 0x559c1d050c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c45caece30414b74 server_cookie=78b49fa1d1e7cee3 in_seq=0 out_seq=0 2024-09-13T08:37:51.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6da000640 1 -- 172.21.15.44:0/2613400976 <== mds.0 v2:172.21.15.129:6834/1614710769 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c1ce52a70 con 0x559c1d0210f0 2024-09-13T08:37:51.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6da000640 10 client.4680 handle_command_reply: tid=0 2024-09-13T08:37:51.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 client.4680 shutdown 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2613400976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03e9d0 msgr2=0x559c1d01fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03e9d0 0x559c1d01fe70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0011280 tx=0x7fa6e0011740 comp rx=0 tx=0).stop 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2613400976 shutdown_connections 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:6834/1614710769,v1:172.21.15.129:6835/1614710769] conn(0x559c1d0210f0 0x559c1d050c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 0x559c1d0203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03e9d0 0x559c1d01fe70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2613400976 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1cff2e10 0x559c1d01f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2613400976 >> 172.21.15.44:0/2613400976 conn(0x559c1cedc130 msgr2=0x559c1cff1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6da000640 0 client.4680 ms_handle_reset on v2:172.21.15.129:6834/1614710769 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2613400976 shutdown_connections 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2613400976 wait complete. 2024-09-13T08:37:51.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.783+0000 7fa6fe7c4000 20 client.4680 trim_cache size 0 max 16384 2024-09-13T08:37:51.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6fe7c4000 1 Processor -- start 2024-09-13T08:37:51.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6fe7c4000 1 -- start start 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1d0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03e9d0 0x559c1d0b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 0x559c1d0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4203fa0 con 0x559c1cff2e10 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4203040 con 0x559c1d03f2f0 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f4159c20 con 0x559c1d03e9d0 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1d0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1d0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36918/0 (socket says 172.21.15.44:36918) 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f8a00640 1 -- 172.21.15.44:0/4164488506 learned_addr learned my addr 172.21.15.44:0/4164488506 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03e9d0 0x559c1d0b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f9400640 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 0x559c1d0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f8a00640 1 -- 172.21.15.44:0/4164488506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03e9d0 msgr2=0x559c1d0b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03e9d0 0x559c1d0b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f8a00640 1 -- 172.21.15.44:0/4164488506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 msgr2=0x559c1d0b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 0x559c1d0b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f8a00640 1 -- 172.21.15.44:0/4164488506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c1ce9bd90 con 0x559c1cff2e10 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03e9d0 0x559c1d0b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.803+0000 7fa6f9400640 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 0x559c1d0b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1d0b0bc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800b630 tx=0x7fa6e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad390d0a442ca1c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6e4c00640 1 -- 172.21.15.44:0/4164488506 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e802e070 con 0x559c1cff2e10 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6e4c00640 1 -- 172.21.15.44:0/4164488506 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e800dda0 con 0x559c1cff2e10 2024-09-13T08:37:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6e4c00640 1 -- 172.21.15.44:0/4164488506 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e801a040 con 0x559c1cff2e10 2024-09-13T08:37:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/4164488506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 msgr2=0x559c1d0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1d0b0bc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800b630 tx=0x7fa6e8014000 comp rx=0 tx=0).stop 2024-09-13T08:37:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/4164488506 shutdown_connections 2024-09-13T08:37:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03f2f0 0x559c1d0b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1d03e9d0 0x559c1d0b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/4164488506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1cff2e10 0x559c1d0b0bc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/4164488506 >> 172.21.15.44:0/4164488506 conn(0x559c1cedc130 msgr2=0x559c1d04bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/4164488506 shutdown_connections 2024-09-13T08:37:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/4164488506 wait complete. 2024-09-13T08:37:51.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 -- start start 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 Processor -- start 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1cff2e10 0x559c1d0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03e9d0 0x559c1d0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03f2f0 0x559c1d0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f415a270 con 0x559c1d03f2f0 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559c1d055ce0 con 0x559c1d03e9d0 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x559c1ce0bec0 con 0x559c1cff2e10 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 10 client.0 _ll_register_callbacks cb 0x7fa6e0001270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03f2f0 0x559c1d0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03f2f0 0x559c1d0b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36926/0 (socket says 172.21.15.44:36926) 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f9400640 1 -- 172.21.15.44:0/2285793003 learned_addr learned my addr 172.21.15.44:0/2285793003 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1cff2e10 0x559c1d0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03e9d0 0x559c1d0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f9400640 1 -- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1cff2e10 msgr2=0x559c1d0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f9400640 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1cff2e10 0x559c1d0b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f9400640 1 -- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03e9d0 msgr2=0x559c1d0b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f9400640 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03e9d0 0x559c1d0b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f9400640 1 -- 172.21.15.44:0/2285793003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c1ce06a90 con 0x559c1d03f2f0 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6f9400640 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03f2f0 0x559c1d0b5470 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0fc040 tx=0x7fa6ec0f50b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a333f1e1c1cd0447 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6da000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6da000640 1 -- 172.21.15.44:0/2285793003 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0f9030 con 0x559c1d03f2f0 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6da000640 1 -- 172.21.15.44:0/2285793003 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa6ec101e30 con 0x559c1d03f2f0 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.807+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2285793003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6e800dda0 con 0x559c1d03f2f0 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6da000640 1 -- 172.21.15.44:0/2285793003 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fa6ec0f8930 con 0x559c1d03f2f0 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 10 client.4686 fetch_fsmap learned FSMap version 8 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2285793003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x559c1cde5050 con 0x559c1d03f2f0 2024-09-13T08:37:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6da000640 1 -- 172.21.15.44:0/2285793003 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e800dda0 con 0x559c1d03f2f0 2024-09-13T08:37:51.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6da000640 1 -- 172.21.15.44:0/2285793003 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fa6ec0b5030 con 0x559c1d03f2f0 2024-09-13T08:37:51.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 10 client.4686 fetch_fsmap finished waiting for FSMap version 8 2024-09-13T08:37:51.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 10 client.4686 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-13T08:37:51.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 20 client.4686 populate_metadata read hostname 'smithi044' 2024-09-13T08:37:51.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:6834/1614710769,v1:172.21.15.129:6835/1614710769] conn(0x559c1d052c10 0x559c1d052ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:51.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 4 client.4686 mds_command: new command op to 4360 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T08:37:51.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2285793003 --> [v2:172.21.15.129:6834/1614710769,v1:172.21.15.129:6835/1614710769] -- command(tid 0: {"prefix": "status"}) -- 0x559c1ce4e520 con 0x559c1d052c10 2024-09-13T08:37:51.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:6834/1614710769,v1:172.21.15.129:6835/1614710769] conn(0x559c1d052c10 0x559c1d052ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6f8a00640 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:6834/1614710769,v1:172.21.15.129:6835/1614710769] conn(0x559c1d052c10 0x559c1d052ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f1e088bdd271c5 server_cookie=562a056568e9ded2 in_seq=0 out_seq=0 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6da000640 10 client.4686 ms_handle_connect on v2:172.21.15.129:6834/1614710769 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6da000640 1 -- 172.21.15.44:0/2285793003 <== mds.0 v2:172.21.15.129:6834/1614710769 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x559c1ce4e520 con 0x559c1d052c10 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6da000640 10 client.4686 handle_command_reply: tid=0 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 client.4686 shutdown 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2285793003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03f2f0 msgr2=0x559c1d0b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03f2f0 0x559c1d0b5470 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0fc040 tx=0x7fa6ec0f50b0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2285793003 shutdown_connections 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:6834/1614710769,v1:172.21.15.129:6835/1614710769] conn(0x559c1d052c10 0x559c1d052ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c1d03f2f0 0x559c1d0b5470 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c1d03e9d0 0x559c1d0b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 --2- 172.21.15.44:0/2285793003 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559c1cff2e10 0x559c1d0b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6da000640 0 client.4686 ms_handle_reset on v2:172.21.15.129:6834/1614710769 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2285793003 >> 172.21.15.44:0/2285793003 conn(0x559c1cedc130 msgr2=0x559c1cffaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2285793003 shutdown_connections 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 1 -- 172.21.15.44:0/2285793003 wait complete. 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fe7c4000 20 client.4686 trim_cache size 0 max 16384 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4360, 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T08:37:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T08:37:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T08:37:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 16.947891694999999, 2024-09-13T08:37:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 8, 2024-09-13T08:37:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 22, 2024-09-13T08:37:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 22, 2024-09-13T08:37:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 22.949058064999999 2024-09-13T08:37:51.823 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:37:51.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 -- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6d4072ef0 msgr2=0x7fa6d40753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6d4072ef0 0x7fa6d40753b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8006fd0 tx=0x7fa6e8008040 comp rx=0 tx=0).stop 2024-09-13T08:37:51.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 -- 172.21.15.44:0/1039537905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f415dae0 msgr2=0x7fa6f409adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:51.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f415dae0 0x7fa6f409adc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009b10 tx=0x7fa6ec0a5910 comp rx=0 tx=0).stop 2024-09-13T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 -- 172.21.15.44:0/1039537905 shutdown_connections 2024-09-13T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6d4072ef0 0x7fa6d40753b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f415dae0 0x7fa6f409adc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6f40cd100 0x7fa6f40af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 --2- 172.21.15.44:0/1039537905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6f40b49e0 0x7fa6f415cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 -- 172.21.15.44:0/1039537905 >> 172.21.15.44:0/1039537905 conn(0x7fa6f40b18f0 msgr2=0x7fa6f40b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 -- 172.21.15.44:0/1039537905 shutdown_connections 2024-09-13T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:51.811+0000 7fa6fb400640 1 -- 172.21.15.44:0/1039537905 wait complete. 2024-09-13T08:37:51.835 INFO:teuthology.run_tasks:Running task kclient... 2024-09-13T08:37:51.861 INFO:tasks.kclient:Mounting kernel clients... 2024-09-13T08:37:51.861 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-13T08:37:51.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:51.861 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:37:51.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:51.871 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-09-13T08:37:51.935 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2024-09-13T08:37:51.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:37:51.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:51.938 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:37:51.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:51.990 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-09-13T08:37:52.055 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2024-09-13T08:37:52.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:37:52.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:52.057 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-09-13T08:37:52.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:52.067 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-09-13T08:37:52.133 INFO:teuthology.orchestra.run.smithi129.stderr:Cannot find device "ceph-brx" 2024-09-13T08:37:52.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:37:52.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:52.135 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-09-13T08:37:52.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:52.186 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-09-13T08:37:52.253 INFO:teuthology.orchestra.run.smithi129.stderr:Cannot find device "ceph-brx" 2024-09-13T08:37:52.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:37:52.255 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-13T08:37:52.255 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T08:37:52.255 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:37:52.255 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:37:52.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.399+0000 7f80d6600640 1 -- 172.21.15.44:0/1317016239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80b4003ab0 msgr2=0x7f80b4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:52.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.399+0000 7f80d6600640 1 --2- 172.21.15.44:0/1317016239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80b4003ab0 0x7f80b4003ed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f80c0006f90 tx=0x7f80c001ce60 comp rx=0 tx=0).stop 2024-09-13T08:37:52.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.399+0000 7f80d6600640 1 -- 172.21.15.44:0/1317016239 shutdown_connections 2024-09-13T08:37:52.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.399+0000 7f80d6600640 1 --2- 172.21.15.44:0/1317016239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80b4003ab0 0x7f80b4003ed0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:52.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.399+0000 7f80d6600640 1 -- 172.21.15.44:0/1317016239 >> 172.21.15.44:0/1317016239 conn(0x7f80d00d1dc0 msgr2=0x7f80d00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:52.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.399+0000 7f80d6600640 1 -- 172.21.15.44:0/1317016239 shutdown_connections 2024-09-13T08:37:52.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.399+0000 7f80d6600640 1 -- 172.21.15.44:0/1317016239 wait complete. 2024-09-13T08:37:52.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.399+0000 7f80d6600640 1 Processor -- start 2024-09-13T08:37:52.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80d6600640 1 -- start start 2024-09-13T08:37:52.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80d6600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80b4003ab0 0x7f80d0155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:52.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80d6600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d0155730 0x7f80d0155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:52.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80d6600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d0152650 0x7f80d00b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:52.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80d6600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f80d00cea80 con 0x7f80d0152650 2024-09-13T08:37:52.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80d6600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80d00ce900 con 0x7f80d0155730 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80d6600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f80d00cec00 con 0x7f80b4003ab0 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d0152650 0x7f80d00b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d0152650 0x7f80d00b3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56732/0 (socket says 172.21.15.44:56732) 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80cfe00640 1 -- 172.21.15.44:0/2324418644 learned_addr learned my addr 172.21.15.44:0/2324418644 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80cf400640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80b4003ab0 0x7f80d0155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80c7e00640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d0155730 0x7f80d0155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80cfe00640 1 -- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80b4003ab0 msgr2=0x7f80d0155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80cfe00640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80b4003ab0 0x7f80d0155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80cfe00640 1 -- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d0155730 msgr2=0x7f80d0155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80cfe00640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d0155730 0x7f80d0155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80cfe00640 1 -- 172.21.15.44:0/2324418644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d0152b00 con 0x7f80d0152650 2024-09-13T08:37:52.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80c7e00640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d0155730 0x7f80d0155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:37:52.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80cfe00640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d0152650 0x7f80d00b3920 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80d00732d0 tx=0x7f80c800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:52.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80c7400640 1 -- 172.21.15.44:0/2324418644 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8004b30 con 0x7f80d0152650 2024-09-13T08:37:52.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80c7400640 1 -- 172.21.15.44:0/2324418644 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80c8004cd0 con 0x7f80d0152650 2024-09-13T08:37:52.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80c7400640 1 -- 172.21.15.44:0/2324418644 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8003be0 con 0x7f80d0152650 2024-09-13T08:37:52.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80d6600640 1 -- 172.21.15.44:0/2324418644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d01540b0 con 0x7f80d0152650 2024-09-13T08:37:52.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80d6600640 1 -- 172.21.15.44:0/2324418644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d0154670 con 0x7f80d0152650 2024-09-13T08:37:52.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.403+0000 7f80d6600640 1 -- 172.21.15.44:0/2324418644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d00cd4f0 con 0x7f80d0152650 2024-09-13T08:37:52.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.407+0000 7f80c7400640 1 -- 172.21.15.44:0/2324418644 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f80c8004030 con 0x7f80d0152650 2024-09-13T08:37:52.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.407+0000 7f80c7400640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80ac072ef0 0x7f80ac0753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:52.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.407+0000 7f80c7400640 1 -- 172.21.15.44:0/2324418644 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f80c80940c0 con 0x7f80d0152650 2024-09-13T08:37:52.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.407+0000 7f80cf400640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80ac072ef0 0x7f80ac0753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:52.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.407+0000 7f80cf400640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80ac072ef0 0x7f80ac0753b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f80c00ab000 tx=0x7f80c0019ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:52.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.411+0000 7f80c7400640 1 -- 172.21.15.44:0/2324418644 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f80c805ecb0 con 0x7f80d0152650 2024-09-13T08:37:52.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.623+0000 7f80d6600640 1 -- 172.21.15.44:0/2324418644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f80d0071f60 con 0x7f80d0152650 2024-09-13T08:37:52.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.623+0000 7f80c7400640 1 -- 172.21.15.44:0/2324418644 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f80c80636e0 con 0x7f80d0152650 2024-09-13T08:37:52.628 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:37:52.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 -- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80ac072ef0 msgr2=0x7f80ac0753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:52.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80ac072ef0 0x7f80ac0753b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f80c00ab000 tx=0x7f80c0019ca0 comp rx=0 tx=0).stop 2024-09-13T08:37:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 -- 172.21.15.44:0/2324418644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d0152650 msgr2=0x7f80d00b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d0152650 0x7f80d00b3920 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80d00732d0 tx=0x7f80c800f3c0 comp rx=0 tx=0).stop 2024-09-13T08:37:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 -- 172.21.15.44:0/2324418644 shutdown_connections 2024-09-13T08:37:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80ac072ef0 0x7f80ac0753b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d0152650 0x7f80d00b3920 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d0155730 0x7f80d0155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 --2- 172.21.15.44:0/2324418644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80b4003ab0 0x7f80d0155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 -- 172.21.15.44:0/2324418644 >> 172.21.15.44:0/2324418644 conn(0x7f80d00d1dc0 msgr2=0x7f80d00c8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 -- 172.21.15.44:0/2324418644 shutdown_connections 2024-09-13T08:37:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52.627+0000 7f80d6600640 1 -- 172.21.15.44:0/2324418644 wait complete. 2024-09-13T08:37:52.649 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T08:37:52.650 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:37:52.650 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:37:52.650 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T08:37:52.650 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:37:52.650 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T08:37:52.650 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T08:37:52.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:37:52.651 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:37:52.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:52.651 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:37:52.661 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:37:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:37:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42410sec preferred_lft 42410sec 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:37:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:52.664 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T08:37:52.664 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:52.664 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:37:52.664 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-09-13T08:37:52.664 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-09-13T08:37:52.664 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-09-13T08:37:52.664 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T08:37:52.664 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:37:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:52.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:52.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:52.820 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:37:52.879 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:37:52.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:52.882 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:37:52.955 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:37:52.955 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:37:52.955 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:37:52.955 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:37:52.956 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:37:52.956 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:37:52.956 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:52.956 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:37:52.956 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:37:52.956 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:37:52.956 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:37:52.956 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:37:53.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:53.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:53.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:53.098 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:37:53.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:53.107 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-09-13T08:37:53.159 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:53.159 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:37:53.159 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:37:53.159 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T08:37:53.159 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:37:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:53.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:53.290 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T08:37:53.291 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:53.291 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:37:53.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T08:37:53.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T08:37:53.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T08:37:53.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T08:37:53.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T08:37:53.291 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:37:53.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:53.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:53.512 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:53.512 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:37:53.512 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T08:37:53.512 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T08:37:53.512 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:37:53.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:53.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:53.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:37:53.600 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T08:37:53.609 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:37:53.609 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:37:53.609 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:37:53.657 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:37:53.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:53.657 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:37:53.750 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:37:53.750 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:37:53.750 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:37:54.429 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:37:54.429 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:37:54.429 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:37:54.429 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:37:54.429 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:37:54.442 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:37:54.443 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:37:54.443 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:37:54.443 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:37:54.443 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:37:54.443 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:37:54.443 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:37:54.443 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:37:54.446 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:37:54.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:54.447 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:37:54.468 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:37:54.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:37:54.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:54.599 INFO:teuthology.orchestra.run.smithi044.stdout:client4654 2024-09-13T08:37:54.599 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:54.599 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:37:54.609 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:37:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:37:54.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:54.726 INFO:teuthology.orchestra.run.smithi044.stdout:client4654 2024-09-13T08:37:54.727 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:37:54.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:37:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:54.796 INFO:teuthology.orchestra.run.smithi044.stdout:client4654 2024-09-13T08:37:54.796 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d85be7ac-239d-467b-bce8-dbc7fa81939e.client4654/status') 2024-09-13T08:37:54.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:54.865 INFO:teuthology.orchestra.run.smithi044.stdout:instance: client.4654 (3)172.21.15.44:0/2485189731 2024-09-13T08:37:54.865 INFO:teuthology.orchestra.run.smithi044.stdout:blocklisted: false 2024-09-13T08:37:54.865 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2024-09-13T08:37:54.865 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2024-09-13T08:37:54.865 INFO:teuthology.orchestra.run.smithi044.stderr:70 bytes copied, 5.2721e-05 s, 1.3 MB/s 2024-09-13T08:37:54.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:54.874 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-13T08:37:54.874 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T08:37:54.874 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-13T08:37:54.874 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:37:55.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.015+0000 7fc19de00640 1 -- 172.21.15.44:0/1479071508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1980d28e0 msgr2=0x7fc1980cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:55.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.015+0000 7fc19de00640 1 --2- 172.21.15.44:0/1479071508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1980d28e0 0x7fc1980cebb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc1980745d0 tx=0x7fc19001caf0 comp rx=0 tx=0).stop 2024-09-13T08:37:55.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.015+0000 7fc19de00640 1 -- 172.21.15.44:0/1479071508 shutdown_connections 2024-09-13T08:37:55.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.015+0000 7fc19de00640 1 --2- 172.21.15.44:0/1479071508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1980d28e0 0x7fc1980cebb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:55.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.015+0000 7fc19de00640 1 --2- 172.21.15.44:0/1479071508 >> v2:172.21.15.129:3300/0 conn(0x7fc19815bea0 0x7fc1980ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:55.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.015+0000 7fc19de00640 1 -- 172.21.15.44:0/1479071508 >> 172.21.15.44:0/1479071508 conn(0x7fc1980d1dc0 msgr2=0x7fc1980d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:55.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.015+0000 7fc19de00640 1 -- 172.21.15.44:0/1479071508 shutdown_connections 2024-09-13T08:37:55.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc19de00640 1 -- 172.21.15.44:0/1479071508 wait complete. 2024-09-13T08:37:55.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc19de00640 1 Processor -- start 2024-09-13T08:37:55.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc19de00640 1 -- start start 2024-09-13T08:37:55.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc19de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1980cdb30 0x7fc19814b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:55.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc19de00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc1980d28e0 0x7fc19806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:55.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc19de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc19815bea0 0x7fc19806e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:55.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc19de00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc198168fc0 con 0x7fc1980cdb30 2024-09-13T08:37:55.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc19de00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc198168e40 con 0x7fc19815bea0 2024-09-13T08:37:55.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc19de00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc198169140 con 0x7fc1980d28e0 2024-09-13T08:37:55.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc196a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1980cdb30 0x7fc19814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:55.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc196a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1980cdb30 0x7fc19814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56744/0 (socket says 172.21.15.44:56744) 2024-09-13T08:37:55.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc196a00640 1 -- 172.21.15.44:0/1280390627 learned_addr learned my addr 172.21.15.44:0/1280390627 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:55.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc197400640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc19815bea0 0x7fc19806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:55.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc18fe00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc1980d28e0 0x7fc19806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:55.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc196a00640 1 -- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc1980d28e0 msgr2=0x7fc19806e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:55.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc196a00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc1980d28e0 0x7fc19806e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:55.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc196a00640 1 -- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc19815bea0 msgr2=0x7fc19806e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:55.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc196a00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc19815bea0 0x7fc19806e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:55.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc196a00640 1 -- 172.21.15.44:0/1280390627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1981f87e0 con 0x7fc1980cdb30 2024-09-13T08:37:55.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc196a00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1980cdb30 0x7fc19814b150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc18800d950 tx=0x7fc18800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:55.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc18f400640 1 -- 172.21.15.44:0/1280390627 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188017070 con 0x7fc1980cdb30 2024-09-13T08:37:55.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc18f400640 1 -- 172.21.15.44:0/1280390627 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc188009d20 con 0x7fc1980cdb30 2024-09-13T08:37:55.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc18f400640 1 -- 172.21.15.44:0/1280390627 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1880106c0 con 0x7fc1980cdb30 2024-09-13T08:37:55.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.019+0000 7fc19de00640 1 -- 172.21.15.44:0/1280390627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1981f7a20 con 0x7fc1980cdb30 2024-09-13T08:37:55.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.023+0000 7fc19de00640 1 -- 172.21.15.44:0/1280390627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981f7fe0 con 0x7fc1980cdb30 2024-09-13T08:37:55.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.023+0000 7fc19de00640 1 -- 172.21.15.44:0/1280390627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1980c8e10 con 0x7fc1980cdb30 2024-09-13T08:37:55.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.023+0000 7fc18f400640 1 -- 172.21.15.44:0/1280390627 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fc18801a070 con 0x7fc1980cdb30 2024-09-13T08:37:55.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.023+0000 7fc18f400640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc174072ec0 0x7fc174075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:55.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.023+0000 7fc18f400640 1 -- 172.21.15.44:0/1280390627 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc188093f60 con 0x7fc1980cdb30 2024-09-13T08:37:55.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.023+0000 7fc18fe00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc174072ec0 0x7fc174075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:55.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.027+0000 7fc18fe00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc174072ec0 0x7fc174075380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc1800046c0 tx=0x7fc180007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:55.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.027+0000 7fc18f400640 1 -- 172.21.15.44:0/1280390627 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc18805eac0 con 0x7fc1980cdb30 2024-09-13T08:37:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.247+0000 7fc19de00640 1 -- 172.21.15.44:0/1280390627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc1980cc9d0 con 0x7fc1980cdb30 2024-09-13T08:37:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.247+0000 7fc18f400640 1 -- 172.21.15.44:0/1280390627 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fc1880634f0 con 0x7fc1980cdb30 2024-09-13T08:37:55.252 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:37:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 -- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc174072ec0 msgr2=0x7fc174075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc174072ec0 0x7fc174075380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc1800046c0 tx=0x7fc180007ea0 comp rx=0 tx=0).stop 2024-09-13T08:37:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 -- 172.21.15.44:0/1280390627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1980cdb30 msgr2=0x7fc19814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1980cdb30 0x7fc19814b150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc18800d950 tx=0x7fc18800de10 comp rx=0 tx=0).stop 2024-09-13T08:37:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 -- 172.21.15.44:0/1280390627 shutdown_connections 2024-09-13T08:37:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc174072ec0 0x7fc174075380 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc19815bea0 0x7fc19806e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc1980d28e0 0x7fc19806e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 --2- 172.21.15.44:0/1280390627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1980cdb30 0x7fc19814b150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 -- 172.21.15.44:0/1280390627 >> 172.21.15.44:0/1280390627 conn(0x7fc1980d1dc0 msgr2=0x7fc19815b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 -- 172.21.15.44:0/1280390627 shutdown_connections 2024-09-13T08:37:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55.251+0000 7fc19de00640 1 -- 172.21.15.44:0/1280390627 wait complete. 2024-09-13T08:37:55.273 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T08:37:55.273 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:37:55.273 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:37:55.274 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-13T08:37:55.274 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-13T08:37:55.274 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T08:37:55.274 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T08:37:55.274 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:37:55.274 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-13T08:37:55.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:55.274 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:37:55.284 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:37:55.284 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:37:55.284 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:37:55.284 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.284 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:37:55.284 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.284 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:55.284 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:55.284 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:37:55.284 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42407sec preferred_lft 42407sec 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:55.285 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:55.287 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:37:55.343 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:37:55.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:55.345 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:37:55.398 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:37:55.398 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:37:55.399 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:37:55.399 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:37:55.399 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:37:55.399 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:37:55.399 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:55.399 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:37:55.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:37:55.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:37:55.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:37:55.399 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:37:55.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:55.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:55.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:55.537 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:37:55.546 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:37:55.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:55.547 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-09-13T08:37:55.601 INFO:teuthology.orchestra.run.smithi044.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-13T08:37:55.602 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:55.602 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:37:55.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:37:55.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-13T08:37:55.602 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:37:55.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:55.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:55.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:55.734 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-13T08:37:55.754 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:37:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:37:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:37:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:37:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.755 INFO:teuthology.orchestra.run.smithi044.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:37:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 3a:18:0f:f7:2d:0d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T08:37:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T08:37:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::3818:fff:fef7:2d0d/64 scope link 2024-09-13T08:37:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:55.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-13T08:37:55.817 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:37:55.817 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:37:55.817 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:37:55.817 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.817 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:37:55.818 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.818 INFO:teuthology.orchestra.run.smithi044.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:37:55.818 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 3a:18:0f:f7:2d:0d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T08:37:55.818 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T08:37:55.818 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.818 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::3818:fff:fef7:2d0d/64 scope link 2024-09-13T08:37:55.818 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:55.819 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-13T08:37:55.819 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:55.820 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:37:55.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-13T08:37:55.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-13T08:37:55.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-13T08:37:55.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-13T08:37:55.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-13T08:37:55.820 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:37:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:55.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:56.000 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:56.000 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:37:56.000 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.1 up 2024-09-13T08:37:56.000 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.1 master ceph-brx 2024-09-13T08:37:56.000 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:37:56.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:56.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:56.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:37:56.085 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-13T08:37:56.094 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-13T08:37:56.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:37:56.094 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-13T08:37:56.144 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:37:56.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:56.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-13T08:37:56.238 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-13T08:37:56.238 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-13T08:37:56.239 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:37:56.254 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-13T08:37:56.254 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-13T08:37:56.254 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:37:56.254 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:37:56.254 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:37:56.264 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-13T08:37:56.264 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-13T08:37:56.264 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:37:56.265 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:37:56.265 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:37:56.265 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:37:56.265 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:37:56.265 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:37:56.268 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:37:56.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:56.268 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-13T08:37:56.287 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:37:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:37:56.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:56.387 INFO:teuthology.orchestra.run.smithi044.stdout:client4589 2024-09-13T08:37:56.388 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:37:56.388 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:37:56.396 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:37:56.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:37:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:56.498 INFO:teuthology.orchestra.run.smithi044.stdout:client4589 2024-09-13T08:37:56.498 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:37:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:37:56.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:56.555 INFO:teuthology.orchestra.run.smithi044.stdout:client4589 2024-09-13T08:37:56.556 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d85be7ac-239d-467b-bce8-dbc7fa81939e.client4589/status') 2024-09-13T08:37:56.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:56.616 INFO:teuthology.orchestra.run.smithi044.stdout:instance: client.4589 (3)172.21.15.44:0/3953545790 2024-09-13T08:37:56.616 INFO:teuthology.orchestra.run.smithi044.stdout:blocklisted: false 2024-09-13T08:37:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2024-09-13T08:37:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2024-09-13T08:37:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:70 bytes copied, 5.1678e-05 s, 1.4 MB/s 2024-09-13T08:37:56.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:56.624 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-13T08:37:56.624 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T08:37:56.624 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T08:37:56.624 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:37:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.767+0000 7fdd52a00640 1 -- 172.21.15.44:0/696219627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdd4c0b11d0 msgr2=0x7fdd4c06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.767+0000 7fdd52a00640 1 --2- 172.21.15.44:0/696219627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdd4c0b11d0 0x7fdd4c06ec80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009bc0 tx=0x7fdd4401cb10 comp rx=0 tx=0).stop 2024-09-13T08:37:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.767+0000 7fdd52a00640 1 -- 172.21.15.44:0/696219627 shutdown_connections 2024-09-13T08:37:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.767+0000 7fdd52a00640 1 --2- 172.21.15.44:0/696219627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdd4c0b11d0 0x7fdd4c06ec80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.767+0000 7fdd52a00640 1 --2- 172.21.15.44:0/696219627 >> v2:172.21.15.129:3300/0 conn(0x7fdd4c15df50 0x7fdd4c06e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.767+0000 7fdd52a00640 1 -- 172.21.15.44:0/696219627 >> 172.21.15.44:0/696219627 conn(0x7fdd4c0d1dc0 msgr2=0x7fdd4c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.767+0000 7fdd52a00640 1 -- 172.21.15.44:0/696219627 shutdown_connections 2024-09-13T08:37:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.767+0000 7fdd52a00640 1 -- 172.21.15.44:0/696219627 wait complete. 2024-09-13T08:37:56.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.767+0000 7fdd52a00640 1 Processor -- start 2024-09-13T08:37:56.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.767+0000 7fdd52a00640 1 -- start start 2024-09-13T08:37:56.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd52a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdd4c0b11d0 0x7fdd4c09e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:56.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd52a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd4c15df50 0x7fdd4c14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:56.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd52a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd4c15e330 0x7fdd4c0b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:56.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd52a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c0d08d0 con 0x7fdd4c15e330 2024-09-13T08:37:56.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd52a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c0d0750 con 0x7fdd4c15df50 2024-09-13T08:37:56.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd52a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4c0d0a50 con 0x7fdd4c0b11d0 2024-09-13T08:37:56.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd50a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd4c15e330 0x7fdd4c0b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:56.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd50a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd4c15e330 0x7fdd4c0b6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56760/0 (socket says 172.21.15.44:56760) 2024-09-13T08:37:56.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd50a00640 1 -- 172.21.15.44:0/2194272235 learned_addr learned my addr 172.21.15.44:0/2194272235 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:56.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd4be00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdd4c0b11d0 0x7fdd4c09e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:56.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd43400640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd4c15df50 0x7fdd4c14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:56.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd50a00640 1 -- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdd4c0b11d0 msgr2=0x7fdd4c09e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd50a00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdd4c0b11d0 0x7fdd4c09e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd50a00640 1 -- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd4c15df50 msgr2=0x7fdd4c14d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd50a00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd4c15df50 0x7fdd4c14d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd50a00640 1 -- 172.21.15.44:0/2194272235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c09edb0 con 0x7fdd4c15e330 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd50a00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd4c15e330 0x7fdd4c0b6dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd4401a740 tx=0x7fdd44003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd48c00640 1 -- 172.21.15.44:0/2194272235 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd440bc070 con 0x7fdd4c15e330 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd48c00640 1 -- 172.21.15.44:0/2194272235 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd440042c0 con 0x7fdd4c15e330 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd52a00640 1 -- 172.21.15.44:0/2194272235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4c09f090 con 0x7fdd4c15e330 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd48c00640 1 -- 172.21.15.44:0/2194272235 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd440b2040 con 0x7fdd4c15e330 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd52a00640 1 -- 172.21.15.44:0/2194272235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4c09f650 con 0x7fdd4c15e330 2024-09-13T08:37:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.771+0000 7fdd52a00640 1 -- 172.21.15.44:0/2194272235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4c0afcd0 con 0x7fdd4c15e330 2024-09-13T08:37:56.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.775+0000 7fdd48c00640 1 -- 172.21.15.44:0/2194272235 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fdd440046b0 con 0x7fdd4c15e330 2024-09-13T08:37:56.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.775+0000 7fdd48c00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd28072ec0 0x7fdd28075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:56.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.775+0000 7fdd48c00640 1 -- 172.21.15.44:0/2194272235 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdd4412db30 con 0x7fdd4c15e330 2024-09-13T08:37:56.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.775+0000 7fdd4be00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd28072ec0 0x7fdd28075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:56.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.775+0000 7fdd4be00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd28072ec0 0x7fdd28075380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0d19d0 tx=0x7fdd3c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:56.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.779+0000 7fdd48c00640 1 -- 172.21.15.44:0/2194272235 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdd440f3060 con 0x7fdd4c15e330 2024-09-13T08:37:56.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.991+0000 7fdd52a00640 1 -- 172.21.15.44:0/2194272235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdd4c0c5e90 con 0x7fdd4c15e330 2024-09-13T08:37:56.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.991+0000 7fdd48c00640 1 -- 172.21.15.44:0/2194272235 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fdd44016790 con 0x7fdd4c15e330 2024-09-13T08:37:56.996 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:37:56.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 -- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd28072ec0 msgr2=0x7fdd28075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:56.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd28072ec0 0x7fdd28075380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0d19d0 tx=0x7fdd3c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:37:56.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 -- 172.21.15.44:0/2194272235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd4c15e330 msgr2=0x7fdd4c0b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:56.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd4c15e330 0x7fdd4c0b6dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd4401a740 tx=0x7fdd44003f60 comp rx=0 tx=0).stop 2024-09-13T08:37:56.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 -- 172.21.15.44:0/2194272235 shutdown_connections 2024-09-13T08:37:56.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd28072ec0 0x7fdd28075380 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd4c15e330 0x7fdd4c0b6dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd4c15df50 0x7fdd4c14d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 --2- 172.21.15.44:0/2194272235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdd4c0b11d0 0x7fdd4c09e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 -- 172.21.15.44:0/2194272235 >> 172.21.15.44:0/2194272235 conn(0x7fdd4c0d1dc0 msgr2=0x7fdd4c06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 -- 172.21.15.44:0/2194272235 shutdown_connections 2024-09-13T08:37:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:56.995+0000 7fdd52a00640 1 -- 172.21.15.44:0/2194272235 wait complete. 2024-09-13T08:37:57.018 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T08:37:57.018 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:37:57.018 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi129.front.sepia.ceph.com 2024-09-13T08:37:57.018 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T08:37:57.018 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T08:37:57.018 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T08:37:57.018 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T08:37:57.018 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:37:57.018 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T08:37:57.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:57.019 DEBUG:teuthology.orchestra.run.smithi129:> ip addr 2024-09-13T08:37:57.028 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:38 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f0 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:39 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f1 2024-09-13T08:37:57.029 INFO:teuthology.orchestra.run.smithi129.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:60 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:61 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.21.15.129/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft 42408sec preferred_lft 42408sec 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::ec4:7aff:fe88:7261/64 scope link 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 02:42:59:46:40:81 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:37:57.030 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:57.031 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T08:37:57.031 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:57.031 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-13T08:37:57.031 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add name ceph-brx type bridge 2024-09-13T08:37:57.031 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr flush dev ceph-brx 2024-09-13T08:37:57.032 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set ceph-brx up 2024-09-13T08:37:57.032 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T08:37:57.032 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-13T08:37:57.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:57.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:57.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:57.197 DEBUG:teuthology.orchestra.run.smithi129:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:37:57.215 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-09-13T08:37:57.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:57.217 DEBUG:teuthology.orchestra.run.smithi129:> route 2024-09-13T08:37:57.314 INFO:teuthology.orchestra.run.smithi129.stdout:Kernel IP routing table 2024-09-13T08:37:57.315 INFO:teuthology.orchestra.run.smithi129.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:37:57.315 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T08:37:57.315 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T08:37:57.315 INFO:teuthology.orchestra.run.smithi129.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T08:37:57.315 INFO:teuthology.orchestra.run.smithi129.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:37:57.315 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T08:37:57.315 INFO:teuthology.orchestra.run.smithi129.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T08:37:57.315 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T08:37:57.315 INFO:teuthology.orchestra.run.smithi129.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T08:37:57.315 INFO:teuthology.orchestra.run.smithi129.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:37:57.316 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:57.316 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-13T08:37:57.316 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T08:37:57.316 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T08:37:57.316 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T08:37:57.316 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-13T08:37:57.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:57.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:57.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:57.409 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-09-13T08:37:57.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:57.418 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list-id 2024-09-13T08:37:57.473 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:57.473 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-13T08:37:57.474 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T08:37:57.474 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-13T08:37:57.474 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-13T08:37:57.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:57.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:57.600 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-13T08:37:57.600 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:57.600 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-13T08:37:57.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-13T08:37:57.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T08:37:57.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-13T08:37:57.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-13T08:37:57.601 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-13T08:37:57.601 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-13T08:37:57.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:57.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:57.768 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:57.768 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-13T08:37:57.768 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set brx.0 up 2024-09-13T08:37:57.768 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T08:37:57.768 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-13T08:37:57.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:57.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:57.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:37:57.897 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T08:37:57.905 INFO:teuthology.orchestra.run.smithi129.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T08:37:57.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:37:57.906 DEBUG:teuthology.orchestra.run.smithi129:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T08:37:57.961 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:37:57.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:57.961 DEBUG:teuthology.orchestra.run.smithi129:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-13T08:37:58.056 INFO:teuthology.orchestra.run.smithi129.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-13T08:37:58.057 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-13T08:37:58.057 INFO:teuthology.orchestra.run.smithi129.stdout:invalid new device string format 2024-09-13T08:37:58.741 INFO:teuthology.orchestra.run.smithi129.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-13T08:37:58.741 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-13T08:37:58.741 INFO:teuthology.orchestra.run.smithi129.stdout:invalid new device string format 2024-09-13T08:37:58.741 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:37:58.741 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:37:58.753 INFO:teuthology.orchestra.run.smithi129.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-13T08:37:58.754 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-13T08:37:58.754 INFO:teuthology.orchestra.run.smithi129.stdout:invalid new device string format 2024-09-13T08:37:58.754 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:37:58.754 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:37:58.754 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:37:58.754 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:37:58.754 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:37:58.758 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:37:58.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:58.758 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T08:37:58.820 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:37:58.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:37:58.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:58.932 INFO:teuthology.orchestra.run.smithi129.stdout:client4598 2024-09-13T08:37:58.932 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:37:58.932 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:37:58.941 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:37:59.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:37:59.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:59.036 INFO:teuthology.orchestra.run.smithi129.stdout:client4598 2024-09-13T08:37:59.037 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:37:59.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:37:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:59.132 INFO:teuthology.orchestra.run.smithi129.stdout:client4598 2024-09-13T08:37:59.133 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d85be7ac-239d-467b-bce8-dbc7fa81939e.client4598/status') 2024-09-13T08:37:59.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:59.191 INFO:teuthology.orchestra.run.smithi129.stdout:instance: client.4598 (3)192.168.0.1:0/3685374393 2024-09-13T08:37:59.191 INFO:teuthology.orchestra.run.smithi129.stdout:blocklisted: false 2024-09-13T08:37:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:0+1 records in 2024-09-13T08:37:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:0+1 records out 2024-09-13T08:37:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:69 bytes copied, 4.931e-05 s, 1.4 MB/s 2024-09-13T08:37:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:59.199 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-13T08:37:59.199 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T08:37:59.199 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-13T08:37:59.199 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:37:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 -- 172.21.15.44:0/2394825787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe31c003c90 msgr2=0x7fe31c004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 --2- 172.21.15.44:0/2394825787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe31c003c90 0x7fe31c004090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe328009b60 tx=0x7fe32801d240 comp rx=0 tx=0).stop 2024-09-13T08:37:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 -- 172.21.15.44:0/2394825787 shutdown_connections 2024-09-13T08:37:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 --2- 172.21.15.44:0/2394825787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe31c003c90 0x7fe31c004090 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 -- 172.21.15.44:0/2394825787 >> 172.21.15.44:0/2394825787 conn(0x7fe33409aa10 msgr2=0x7fe33409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 -- 172.21.15.44:0/2394825787 shutdown_connections 2024-09-13T08:37:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 -- 172.21.15.44:0/2394825787 wait complete. 2024-09-13T08:37:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 Processor -- start 2024-09-13T08:37:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 -- start start 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe31c003c90 0x7fe3340cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3340b5ee0 0x7fe33414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3340c78c0 0x7fe3340ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3340cffd0 con 0x7fe3340b5ee0 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe3340cfe50 con 0x7fe3340c78c0 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe33a800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe3340d0150 con 0x7fe31c003c90 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe338a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3340b5ee0 0x7fe33414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe338a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3340b5ee0 0x7fe33414ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56776/0 (socket says 172.21.15.44:56776) 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe338a00640 1 -- 172.21.15.44:0/2309512304 learned_addr learned my addr 172.21.15.44:0/2309512304 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe339400640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe31c003c90 0x7fe3340cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.359+0000 7fe339e00640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3340c78c0 0x7fe3340ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe338a00640 1 -- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe31c003c90 msgr2=0x7fe3340cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:59.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe338a00640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe31c003c90 0x7fe3340cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:59.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe338a00640 1 -- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3340c78c0 msgr2=0x7fe3340ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:59.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe338a00640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3340c78c0 0x7fe3340ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:59.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe338a00640 1 -- 172.21.15.44:0/2309512304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3340c7e50 con 0x7fe3340b5ee0 2024-09-13T08:37:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe338a00640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3340b5ee0 0x7fe33414ffd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe320006450 tx=0x7fe3200039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe331600640 1 -- 172.21.15.44:0/2309512304 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320015070 con 0x7fe3340b5ee0 2024-09-13T08:37:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe331600640 1 -- 172.21.15.44:0/2309512304 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe320011920 con 0x7fe3340b5ee0 2024-09-13T08:37:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe33a800640 1 -- 172.21.15.44:0/2309512304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3340c8130 con 0x7fe3340b5ee0 2024-09-13T08:37:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe331600640 1 -- 172.21.15.44:0/2309512304 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320011d70 con 0x7fe3340b5ee0 2024-09-13T08:37:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe33a800640 1 -- 172.21.15.44:0/2309512304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3340c86f0 con 0x7fe3340b5ee0 2024-09-13T08:37:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe33a800640 1 -- 172.21.15.44:0/2309512304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe334161930 con 0x7fe3340b5ee0 2024-09-13T08:37:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe331600640 1 -- 172.21.15.44:0/2309512304 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fe320018070 con 0x7fe3340b5ee0 2024-09-13T08:37:59.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe331600640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe314072ef0 0x7fe3140753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:37:59.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.363+0000 7fe331600640 1 -- 172.21.15.44:0/2309512304 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe320094ad0 con 0x7fe3340b5ee0 2024-09-13T08:37:59.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.367+0000 7fe339400640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe314072ef0 0x7fe3140753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:37:59.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.367+0000 7fe339400640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe314072ef0 0x7fe3140753b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3280ae000 tx=0x7fe328019fe0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:37:59.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.371+0000 7fe331600640 1 -- 172.21.15.44:0/2309512304 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe32005f630 con 0x7fe3340b5ee0 2024-09-13T08:37:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 -- 172.21.15.44:0/2309512304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe3340d1940 con 0x7fe3340b5ee0 2024-09-13T08:37:59.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe331600640 1 -- 172.21.15.44:0/2309512304 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fe320064060 con 0x7fe3340b5ee0 2024-09-13T08:37:59.591 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:37:59.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 -- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe314072ef0 msgr2=0x7fe3140753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:59.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe314072ef0 0x7fe3140753b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3280ae000 tx=0x7fe328019fe0 comp rx=0 tx=0).stop 2024-09-13T08:37:59.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 -- 172.21.15.44:0/2309512304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3340b5ee0 msgr2=0x7fe33414ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:37:59.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3340b5ee0 0x7fe33414ffd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe320006450 tx=0x7fe3200039d0 comp rx=0 tx=0).stop 2024-09-13T08:37:59.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 -- 172.21.15.44:0/2309512304 shutdown_connections 2024-09-13T08:37:59.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe314072ef0 0x7fe3140753b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:59.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3340c78c0 0x7fe3340ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:59.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3340b5ee0 0x7fe33414ffd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:59.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 --2- 172.21.15.44:0/2309512304 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe31c003c90 0x7fe3340cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:37:59.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 -- 172.21.15.44:0/2309512304 >> 172.21.15.44:0/2309512304 conn(0x7fe33409aa10 msgr2=0x7fe3340b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:37:59.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 -- 172.21.15.44:0/2309512304 shutdown_connections 2024-09-13T08:37:59.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:37:59.587+0000 7fe33a800640 1 -- 172.21.15.44:0/2309512304 wait complete. 2024-09-13T08:37:59.612 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T08:37:59.613 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:37:59.613 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi129.front.sepia.ceph.com 2024-09-13T08:37:59.613 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-13T08:37:59.613 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-13T08:37:59.613 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T08:37:59.613 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T08:37:59.613 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:37:59.613 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-13T08:37:59.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:59.613 DEBUG:teuthology.orchestra.run.smithi129:> ip addr 2024-09-13T08:37:59.622 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:37:59.622 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:37:59.622 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:37:59.622 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:59.622 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-09-13T08:37:59.622 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:38 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f0 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:39 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f1 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:60 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:61 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.21.15.129/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft 42406sec preferred_lft 42406sec 2024-09-13T08:37:59.623 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::ec4:7aff:fe88:7261/64 scope link 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 02:42:59:46:40:81 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:59.624 INFO:teuthology.orchestra.run.smithi129.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:37:59.625 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T08:37:59.625 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-09-13T08:37:59.625 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:37:59.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:59.626 DEBUG:teuthology.orchestra.run.smithi129:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:37:59.687 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-09-13T08:37:59.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:59.692 DEBUG:teuthology.orchestra.run.smithi129:> route 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:Kernel IP routing table 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T08:37:59.741 INFO:teuthology.orchestra.run.smithi129.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:37:59.742 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:59.742 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-13T08:37:59.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T08:37:59.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T08:37:59.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T08:37:59.742 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-13T08:37:59.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:37:59.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:37:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:37:59.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:59.880 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-09-13T08:37:59.889 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-13T08:37:59.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:37:59.890 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list-id 2024-09-13T08:37:59.941 INFO:teuthology.orchestra.run.smithi129.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-13T08:37:59.942 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:37:59.942 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-13T08:37:59.942 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-13T08:37:59.942 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-13T08:37:59.942 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-13T08:38:00.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:38:00.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:38:00.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:00.072 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-13T08:38:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:38:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:38:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:38:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-09-13T08:38:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:38:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 1e:28:6a:0b:37:33 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T08:38:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T08:38:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::1c28:6aff:fe0b:3733/64 scope link 2024-09-13T08:38:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:00.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:00.094 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-13T08:38:00.157 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:38:00.157 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:38:00.157 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:38:00.157 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:00.157 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-09-13T08:38:00.157 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:00.157 INFO:teuthology.orchestra.run.smithi129.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:38:00.157 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 1e:28:6a:0b:37:33 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T08:38:00.157 INFO:teuthology.orchestra.run.smithi129.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T08:38:00.157 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:00.158 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::1c28:6aff:fe0b:3733/64 scope link 2024-09-13T08:38:00.158 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:00.159 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-13T08:38:00.159 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:38:00.159 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-13T08:38:00.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-13T08:38:00.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-13T08:38:00.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-13T08:38:00.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-13T08:38:00.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-13T08:38:00.159 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-13T08:38:00.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:38:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:38:00.338 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:38:00.338 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-13T08:38:00.338 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set brx.1 up 2024-09-13T08:38:00.338 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set dev brx.1 master ceph-brx 2024-09-13T08:38:00.338 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-13T08:38:00.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:38:00.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:38:00.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:38:00.467 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-13T08:38:00.476 INFO:teuthology.orchestra.run.smithi129.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-13T08:38:00.477 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:38:00.477 DEBUG:teuthology.orchestra.run.smithi129:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-13T08:38:00.525 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:38:00.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:00.525 DEBUG:teuthology.orchestra.run.smithi129:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-13T08:38:00.616 INFO:teuthology.orchestra.run.smithi129.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-13T08:38:00.617 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-13T08:38:00.617 INFO:teuthology.orchestra.run.smithi129.stdout:invalid new device string format 2024-09-13T08:38:00.631 INFO:teuthology.orchestra.run.smithi129.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-13T08:38:00.632 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-13T08:38:00.632 INFO:teuthology.orchestra.run.smithi129.stdout:invalid new device string format 2024-09-13T08:38:00.632 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:38:00.632 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:38:00.642 INFO:teuthology.orchestra.run.smithi129.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-13T08:38:00.642 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-13T08:38:00.642 INFO:teuthology.orchestra.run.smithi129.stdout:invalid new device string format 2024-09-13T08:38:00.642 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:38:00.642 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:38:00.642 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:38:00.642 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:38:00.642 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:38:00.646 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:38:00.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:00.646 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-13T08:38:00.664 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:38:00.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:38:00.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:38:00.760 INFO:teuthology.orchestra.run.smithi129.stdout:client4713 2024-09-13T08:38:00.761 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:38:00.761 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:00.810 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:38:00.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:38:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:38:00.912 INFO:teuthology.orchestra.run.smithi129.stdout:client4713 2024-09-13T08:38:00.912 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:38:00.933 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:38:00.936 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:38:00.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:38:00.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:38:00.968 INFO:teuthology.orchestra.run.smithi129.stdout:client4713 2024-09-13T08:38:00.969 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d85be7ac-239d-467b-bce8-dbc7fa81939e.client4713/status') 2024-09-13T08:38:01.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:38:01.102 INFO:teuthology.orchestra.run.smithi129.stdout:instance: client.4713 (3)172.21.15.129:0/3727365680 2024-09-13T08:38:01.102 INFO:teuthology.orchestra.run.smithi129.stdout:blocklisted: false 2024-09-13T08:38:01.102 INFO:teuthology.orchestra.run.smithi129.stderr:0+1 records in 2024-09-13T08:38:01.102 INFO:teuthology.orchestra.run.smithi129.stderr:0+1 records out 2024-09-13T08:38:01.102 INFO:teuthology.orchestra.run.smithi129.stderr:71 bytes copied, 7.1223e-05 s, 997 kB/s 2024-09-13T08:38:01.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T08:38:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:38:01.110 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-13T08:38:01.207 INFO:tasks.cephfs_test_runner:Starting test: test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Initiate cloning where clone subvolume and source subvolume are located 2024-09-13T08:38:01.207 INFO:tasks.cephfs_test_runner:test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-13T08:38:01.207 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:38:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 -- 172.21.15.44:0/2058785316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac15ade0 msgr2=0x7f6aac0cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 --2- 172.21.15.44:0/2058785316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac15ade0 0x7f6aac0cebf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6aac076920 tx=0x7f6a9c01c5d0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 -- 172.21.15.44:0/2058785316 shutdown_connections 2024-09-13T08:38:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 --2- 172.21.15.44:0/2058785316 >> v2:172.21.15.129:3300/0 conn(0x7f6aac0b4bd0 0x7f6aac15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 --2- 172.21.15.44:0/2058785316 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6aac06e810 0x7f6aac0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 --2- 172.21.15.44:0/2058785316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac15ade0 0x7f6aac0cebf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 -- 172.21.15.44:0/2058785316 >> 172.21.15.44:0/2058785316 conn(0x7f6aac0b18f0 msgr2=0x7f6aac0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 -- 172.21.15.44:0/2058785316 shutdown_connections 2024-09-13T08:38:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 -- 172.21.15.44:0/2058785316 wait complete. 2024-09-13T08:38:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 Processor -- start 2024-09-13T08:38:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 -- start start 2024-09-13T08:38:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac06e810 0x7f6aac160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6aac0b4bd0 0x7f6aac1623b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6aac15ade0 0x7f6aac1f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac06f150 con 0x7f6aac06e810 2024-09-13T08:38:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac06efd0 con 0x7f6aac15ade0 2024-09-13T08:38:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6ab1400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6aac06f2d0 con 0x7f6aac0b4bd0 2024-09-13T08:38:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6aaa800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac06e810 0x7f6aac160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6aaa800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac06e810 0x7f6aac160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56792/0 (socket says 172.21.15.44:56792) 2024-09-13T08:38:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.363+0000 7f6aaa800640 1 -- 172.21.15.44:0/3206088650 learned_addr learned my addr 172.21.15.44:0/3206088650 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aa3e00640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6aac0b4bd0 0x7f6aac1623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aab200640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6aac15ade0 0x7f6aac1f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aaa800640 1 -- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6aac0b4bd0 msgr2=0x7f6aac1623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aaa800640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6aac0b4bd0 0x7f6aac1623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aaa800640 1 -- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6aac15ade0 msgr2=0x7f6aac1f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aaa800640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6aac15ade0 0x7f6aac1f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aaa800640 1 -- 172.21.15.44:0/3206088650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac1f99d0 con 0x7f6aac06e810 2024-09-13T08:38:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aaa800640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac06e810 0x7f6aac160b10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c01cab0 tx=0x7f6a9c005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aa2a00640 1 -- 172.21.15.44:0/3206088650 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0ad070 con 0x7f6aac06e810 2024-09-13T08:38:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aa2a00640 1 -- 172.21.15.44:0/3206088650 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a9c0a9cc0 con 0x7f6aac06e810 2024-09-13T08:38:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aa2a00640 1 -- 172.21.15.44:0/3206088650 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c005290 con 0x7f6aac06e810 2024-09-13T08:38:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6ab1400640 1 -- 172.21.15.44:0/3206088650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac1f9cb0 con 0x7f6aac06e810 2024-09-13T08:38:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6ab1400640 1 -- 172.21.15.44:0/3206088650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac09c920 con 0x7f6aac06e810 2024-09-13T08:38:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6ab1400640 1 -- 172.21.15.44:0/3206088650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac069410 con 0x7f6aac06e810 2024-09-13T08:38:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.367+0000 7f6aa2a00640 1 -- 172.21.15.44:0/3206088650 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f6a9c0b0070 con 0x7f6aac06e810 2024-09-13T08:38:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.371+0000 7f6aa2a00640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a88072ec0 0x7f6a88075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.371+0000 7f6aa2a00640 1 -- 172.21.15.44:0/3206088650 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6a9c12e250 con 0x7f6aac06e810 2024-09-13T08:38:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.371+0000 7f6aa3e00640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a88072ec0 0x7f6a88075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.371+0000 7f6aa3e00640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a88072ec0 0x7f6a88075380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6aac09bc50 tx=0x7f6a940091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.371+0000 7f6aa2a00640 1 -- 172.21.15.44:0/3206088650 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6a9c0164b0 con 0x7f6aac06e810 2024-09-13T08:38:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.547+0000 7f6ab1400640 1 -- 172.21.15.44:0/3206088650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6aac0cf720 con 0x7f6aac06e810 2024-09-13T08:38:01.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6aa2a00640 1 -- 172.21.15.44:0/3206088650 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16681 (secure 0 0 0) 0x7f6a9c0f8e40 con 0x7f6aac06e810 2024-09-13T08:38:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:01.554 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":22,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:35.883263+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:37:32.247896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T08:37:33.241202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:38:01.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 -- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a88072ec0 msgr2=0x7f6a88075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:01.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a88072ec0 0x7f6a88075380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6aac09bc50 tx=0x7f6a940091e0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 -- 172.21.15.44:0/3206088650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac06e810 msgr2=0x7f6aac160b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:01.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac06e810 0x7f6aac160b10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c01cab0 tx=0x7f6a9c005e20 comp rx=0 tx=0).stop 2024-09-13T08:38:01.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 -- 172.21.15.44:0/3206088650 shutdown_connections 2024-09-13T08:38:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a88072ec0 0x7f6a88075380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6aac15ade0 0x7f6aac1f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6aac0b4bd0 0x7f6aac1623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 --2- 172.21.15.44:0/3206088650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac06e810 0x7f6aac160b10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 -- 172.21.15.44:0/3206088650 >> 172.21.15.44:0/3206088650 conn(0x7f6aac0b18f0 msgr2=0x7f6aac0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 -- 172.21.15.44:0/3206088650 shutdown_connections 2024-09-13T08:38:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.551+0000 7f6ab1400640 1 -- 172.21.15.44:0/3206088650 wait complete. 2024-09-13T08:38:01.575 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T08:37:24.563344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T08:37:32.247896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T08:37:33.241202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T08:38:01.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T08:38:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 -- 172.21.15.44:0/652316460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c80cfd80 msgr2=0x7f06c80d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 --2- 172.21.15.44:0/652316460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c80cfd80 0x7f06c80d1bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009840 tx=0x7f06b001c820 comp rx=0 tx=0).stop 2024-09-13T08:38:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 -- 172.21.15.44:0/652316460 shutdown_connections 2024-09-13T08:38:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 --2- 172.21.15.44:0/652316460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c80cfd80 0x7f06c80d1bf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 --2- 172.21.15.44:0/652316460 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06c815df10 0x7f06c806eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 -- 172.21.15.44:0/652316460 >> 172.21.15.44:0/652316460 conn(0x7f06c809aa10 msgr2=0x7f06c809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 -- 172.21.15.44:0/652316460 shutdown_connections 2024-09-13T08:38:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 -- 172.21.15.44:0/652316460 wait complete. 2024-09-13T08:38:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 Processor -- start 2024-09-13T08:38:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 -- start start 2024-09-13T08:38:01.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06c80cfd80 0x7f06c80c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:01.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c80d2130 0x7f06c80b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:01.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c815df10 0x7f06c81fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:01.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06c816b160 con 0x7f06c80d2130 2024-09-13T08:38:01.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06c816afe0 con 0x7f06c815df10 2024-09-13T08:38:01.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06cce00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f06c816b2e0 con 0x7f06c80cfd80 2024-09-13T08:38:01.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06c7400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06c80cfd80 0x7f06c80c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:01.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c80d2130 0x7f06c80b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:01.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06c7400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06c80cfd80 0x7f06c80c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:50096/0 (socket says 172.21.15.44:50096) 2024-09-13T08:38:01.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.723+0000 7f06c7400640 1 -- 172.21.15.44:0/1379084483 learned_addr learned my addr 172.21.15.44:0/1379084483 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:01.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06c7e00640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c815df10 0x7f06c81fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06c7400640 1 -- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c815df10 msgr2=0x7f06c81fb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06c7400640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c815df10 0x7f06c81fb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06c7400640 1 -- 172.21.15.44:0/1379084483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c80d2130 msgr2=0x7f06c80b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06c7400640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c80d2130 0x7f06c80b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06c7400640 1 -- 172.21.15.44:0/1379084483 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c80b09d0 con 0x7f06c80cfd80 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06c7400640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06c80cfd80 0x7f06c80c8d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f06b800e7d0 tx=0x7f06b800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06bf400640 1 -- 172.21.15.44:0/1379084483 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b800ccc0 con 0x7f06c80cfd80 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06bf400640 1 -- 172.21.15.44:0/1379084483 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06b8010430 con 0x7f06c80cfd80 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06bf400640 1 -- 172.21.15.44:0/1379084483 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8010860 con 0x7f06c80cfd80 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06cce00640 1 -- 172.21.15.44:0/1379084483 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c80b0cb0 con 0x7f06c80cfd80 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06cce00640 1 -- 172.21.15.44:0/1379084483 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c80aef50 con 0x7f06c80cfd80 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06cce00640 1 -- 172.21.15.44:0/1379084483 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c80d1bf0 con 0x7f06c80cfd80 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.727+0000 7f06bf400640 1 -- 172.21.15.44:0/1379084483 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f06b8010a00 con 0x7f06c80cfd80 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.731+0000 7f06bf400640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06a4072ec0 0x7f06a4075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.731+0000 7f06bf400640 1 -- 172.21.15.44:0/1379084483 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f06b8094740 con 0x7f06c80cfd80 2024-09-13T08:38:01.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.731+0000 7f06bfe00640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06a4072ec0 0x7f06a4075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:01.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.731+0000 7f06bfe00640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06a4072ec0 0x7f06a4075380 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f06b00af000 tx=0x7f06b001b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.735+0000 7f06bf400640 1 -- 172.21.15.44:0/1379084483 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f06b805f330 con 0x7f06c80cfd80 2024-09-13T08:38:01.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.907+0000 7f06cce00640 1 -- 172.21.15.44:0/1379084483 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f06c806ea70 con 0x7f06c80cfd80 2024-09-13T08:38:01.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.907+0000 7f06bf400640 1 -- 172.21.15.44:0/1379084483 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f06b8063d60 con 0x7f06c80cfd80 2024-09-13T08:38:01.914 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T08:38:01.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 -- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06a4072ec0 msgr2=0x7f06a4075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:01.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06a4072ec0 0x7f06a4075380 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f06b00af000 tx=0x7f06b001b060 comp rx=0 tx=0).stop 2024-09-13T08:38:01.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 -- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06c80cfd80 msgr2=0x7f06c80c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:01.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06c80cfd80 0x7f06c80c8d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f06b800e7d0 tx=0x7f06b800ec90 comp rx=0 tx=0).stop 2024-09-13T08:38:01.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 -- 172.21.15.44:0/1379084483 shutdown_connections 2024-09-13T08:38:01.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06a4072ec0 0x7f06a4075380 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c815df10 0x7f06c81fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c80d2130 0x7f06c80b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 --2- 172.21.15.44:0/1379084483 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06c80cfd80 0x7f06c80c8d30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:01.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 -- 172.21.15.44:0/1379084483 >> 172.21.15.44:0/1379084483 conn(0x7f06c809aa10 msgr2=0x7f06c80707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:01.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 -- 172.21.15.44:0/1379084483 shutdown_connections 2024-09-13T08:38:01.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:01.911+0000 7f06cce00640 1 -- 172.21.15.44:0/1379084483 wait complete. 2024-09-13T08:38:01.935 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T08:38:02.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.079+0000 7f7115600640 1 -- 172.21.15.44:0/2628009082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100cdf20 msgr2=0x7f71100b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.079+0000 7f7115600640 1 --2- 172.21.15.44:0/2628009082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100cdf20 0x7f71100b1a30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7108009bc0 tx=0x7f710801cb70 comp rx=0 tx=0).stop 2024-09-13T08:38:02.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.079+0000 7f7115600640 1 -- 172.21.15.44:0/2628009082 shutdown_connections 2024-09-13T08:38:02.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.079+0000 7f7115600640 1 --2- 172.21.15.44:0/2628009082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100cdf20 0x7f71100b1a30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.079+0000 7f7115600640 1 -- 172.21.15.44:0/2628009082 >> 172.21.15.44:0/2628009082 conn(0x7f71100d1e00 msgr2=0x7f71100d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:02.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.079+0000 7f7115600640 1 -- 172.21.15.44:0/2628009082 shutdown_connections 2024-09-13T08:38:02.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.079+0000 7f7115600640 1 -- 172.21.15.44:0/2628009082 wait complete. 2024-09-13T08:38:02.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7115600640 1 Processor -- start 2024-09-13T08:38:02.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7115600640 1 -- start start 2024-09-13T08:38:02.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7115600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b3bb0 0x7f711015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7115600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f71100cdf20 0x7f71100b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7115600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f711015f340 0x7f71101f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7115600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f711016a3b0 con 0x7f71100b3bb0 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7115600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f711016a230 con 0x7f711015f340 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7115600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f711016a530 con 0x7f71100cdf20 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f710ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b3bb0 0x7f711015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f710ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b3bb0 0x7f711015b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55924/0 (socket says 172.21.15.44:55924) 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f710ea00640 1 -- 172.21.15.44:0/3966072116 learned_addr learned my addr 172.21.15.44:0/3966072116 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7107e00640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f71100cdf20 0x7f71100b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f710f400640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f711015f340 0x7f71101f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f710ea00640 1 -- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f71100cdf20 msgr2=0x7f71100b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f710ea00640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f71100cdf20 0x7f71100b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f710ea00640 1 -- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f711015f340 msgr2=0x7f71101f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f710ea00640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f711015f340 0x7f71101f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f710ea00640 1 -- 172.21.15.44:0/3966072116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71100b7a00 con 0x7f71100b3bb0 2024-09-13T08:38:02.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f710ea00640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b3bb0 0x7f711015b0f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f710000c3c0 tx=0x7f710000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:02.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7106a00640 1 -- 172.21.15.44:0/3966072116 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100017070 con 0x7f71100b3bb0 2024-09-13T08:38:02.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7106a00640 1 -- 172.21.15.44:0/3966072116 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7100003600 con 0x7f71100b3bb0 2024-09-13T08:38:02.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7115600640 1 -- 172.21.15.44:0/3966072116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71100c7de0 con 0x7f71100b3bb0 2024-09-13T08:38:02.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7106a00640 1 -- 172.21.15.44:0/3966072116 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100003a50 con 0x7f71100b3bb0 2024-09-13T08:38:02.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.083+0000 7f7115600640 1 -- 172.21.15.44:0/3966072116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71100c83a0 con 0x7f71100b3bb0 2024-09-13T08:38:02.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.087+0000 7f7115600640 1 -- 172.21.15.44:0/3966072116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71100c85e0 con 0x7f71100b3bb0 2024-09-13T08:38:02.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.087+0000 7f7106a00640 1 -- 172.21.15.44:0/3966072116 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f710001a070 con 0x7f71100b3bb0 2024-09-13T08:38:02.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.087+0000 7f7106a00640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70ec072ef0 0x7f70ec0753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.087+0000 7f7106a00640 1 -- 172.21.15.44:0/3966072116 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7100092f70 con 0x7f71100b3bb0 2024-09-13T08:38:02.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.087+0000 7f7107e00640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70ec072ef0 0x7f70ec0753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.087+0000 7f7107e00640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70ec072ef0 0x7f70ec0753b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f711015ed20 tx=0x7f70f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:02.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.091+0000 7f7106a00640 1 -- 172.21.15.44:0/3966072116 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f71100c85e0 con 0x7f71100b3bb0 2024-09-13T08:38:02.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 -- 172.21.15.44:0/3966072116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71100b34a0 con 0x7f71100b3bb0 2024-09-13T08:38:02.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7106a00640 1 -- 172.21.15.44:0/3966072116 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f71100d2c90 con 0x7f71100b3bb0 2024-09-13T08:38:02.271 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-09-13T08:38:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 -- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70ec072ef0 msgr2=0x7f70ec0753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70ec072ef0 0x7f70ec0753b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f711015ed20 tx=0x7f70f8009040 comp rx=0 tx=0).stop 2024-09-13T08:38:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 -- 172.21.15.44:0/3966072116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b3bb0 msgr2=0x7f711015b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b3bb0 0x7f711015b0f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f710000c3c0 tx=0x7f710000c880 comp rx=0 tx=0).stop 2024-09-13T08:38:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 -- 172.21.15.44:0/3966072116 shutdown_connections 2024-09-13T08:38:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70ec072ef0 0x7f70ec0753b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f711015f340 0x7f71101f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f71100cdf20 0x7f71100b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 --2- 172.21.15.44:0/3966072116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b3bb0 0x7f711015b0f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 -- 172.21.15.44:0/3966072116 >> 172.21.15.44:0/3966072116 conn(0x7f71100d1e00 msgr2=0x7f71100ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 -- 172.21.15.44:0/3966072116 shutdown_connections 2024-09-13T08:38:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.267+0000 7f7115600640 1 -- 172.21.15.44:0/3966072116 wait complete. 2024-09-13T08:38:02.292 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T08:38:02.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa617c00640 1 -- 172.21.15.44:0/3434244186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5f8004810 msgr2=0x7fa5f8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa617c00640 1 --2- 172.21.15.44:0/3434244186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5f8004810 0x7fa5f8004c90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009d30 tx=0x7fa5fc01d410 comp rx=0 tx=0).stop 2024-09-13T08:38:02.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa615c00640 1 -- 172.21.15.44:0/3434244186 reap_dead start 2024-09-13T08:38:02.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa617c00640 1 -- 172.21.15.44:0/3434244186 shutdown_connections 2024-09-13T08:38:02.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa617c00640 1 -- 172.21.15.44:0/3434244186 >> 172.21.15.44:0/3434244186 conn(0x7fa6100d1dd0 msgr2=0x7fa6100d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:02.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa617c00640 1 -- 172.21.15.44:0/3434244186 shutdown_connections 2024-09-13T08:38:02.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa617c00640 1 -- 172.21.15.44:0/3434244186 wait complete. 2024-09-13T08:38:02.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa617c00640 1 Processor -- start 2024-09-13T08:38:02.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa617c00640 1 -- start start 2024-09-13T08:38:02.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa617c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f8004810 0x7fa61015e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.439+0000 7fa617c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6100c8360 0x7fa6101f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa617c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100c87c0 0x7fa6101f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa617c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6100b0e50 con 0x7fa6100c87c0 2024-09-13T08:38:02.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa617c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6100b0cd0 con 0x7fa5f8004810 2024-09-13T08:38:02.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa617c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa6100b1220 con 0x7fa6100c8360 2024-09-13T08:38:02.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa615c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100c87c0 0x7fa6101f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa615200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f8004810 0x7fa61015e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa615c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100c87c0 0x7fa6101f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55944/0 (socket says 172.21.15.44:55944) 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa615c00640 1 -- 172.21.15.44:0/2535785163 learned_addr learned my addr 172.21.15.44:0/2535785163 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa60fe00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6100c8360 0x7fa6101f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa615c00640 1 -- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6100c8360 msgr2=0x7fa6101f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa615c00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6100c8360 0x7fa6101f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa615c00640 1 -- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f8004810 msgr2=0x7fa61015e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa615c00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f8004810 0x7fa61015e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa615c00640 1 -- 172.21.15.44:0/2535785163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6100ca090 con 0x7fa6100c87c0 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa615c00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100c87c0 0x7fa6101f63f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa6100744b0 tx=0x7fa60800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa60cc00640 1 -- 172.21.15.44:0/2535785163 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608004b30 con 0x7fa6100c87c0 2024-09-13T08:38:02.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa60cc00640 1 -- 172.21.15.44:0/2535785163 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa608004cd0 con 0x7fa6100c87c0 2024-09-13T08:38:02.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa617c00640 1 -- 172.21.15.44:0/2535785163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61014e9d0 con 0x7fa6100c87c0 2024-09-13T08:38:02.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa60cc00640 1 -- 172.21.15.44:0/2535785163 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608003be0 con 0x7fa6100c87c0 2024-09-13T08:38:02.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa617c00640 1 -- 172.21.15.44:0/2535785163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61014dce0 con 0x7fa6100c87c0 2024-09-13T08:38:02.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa617c00640 1 -- 172.21.15.44:0/2535785163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6100b05e0 con 0x7fa6100c87c0 2024-09-13T08:38:02.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.443+0000 7fa60cc00640 1 -- 172.21.15.44:0/2535785163 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fa608004030 con 0x7fa6100c87c0 2024-09-13T08:38:02.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.447+0000 7fa60cc00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5f0072ef0 0x7fa5f00753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.447+0000 7fa60cc00640 1 -- 172.21.15.44:0/2535785163 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa6080949a0 con 0x7fa6100c87c0 2024-09-13T08:38:02.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.447+0000 7fa615200640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5f0072ef0 0x7fa5f00753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.447+0000 7fa615200640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5f0072ef0 0x7fa5f00753b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa604006bb0 tx=0x7fa604006b40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:02.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.451+0000 7fa60cc00640 1 -- 172.21.15.44:0/2535785163 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa60801f060 con 0x7fa6100c87c0 2024-09-13T08:38:02.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 -- 172.21.15.44:0/2535785163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6100d1080 con 0x7fa6100c87c0 2024-09-13T08:38:02.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa60cc00640 1 -- 172.21.15.44:0/2535785163 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fa60805f500 con 0x7fa6100c87c0 2024-09-13T08:38:02.631 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-09-13T08:38:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 -- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5f0072ef0 msgr2=0x7fa5f00753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5f0072ef0 0x7fa5f00753b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa604006bb0 tx=0x7fa604006b40 comp rx=0 tx=0).stop 2024-09-13T08:38:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 -- 172.21.15.44:0/2535785163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100c87c0 msgr2=0x7fa6101f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100c87c0 0x7fa6101f63f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa6100744b0 tx=0x7fa60800f3c0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 -- 172.21.15.44:0/2535785163 shutdown_connections 2024-09-13T08:38:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5f0072ef0 0x7fa5f00753b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100c87c0 0x7fa6101f63f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6100c8360 0x7fa6101f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 --2- 172.21.15.44:0/2535785163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f8004810 0x7fa61015e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 -- 172.21.15.44:0/2535785163 >> 172.21.15.44:0/2535785163 conn(0x7fa6100d1dd0 msgr2=0x7fa6100b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 -- 172.21.15.44:0/2535785163 shutdown_connections 2024-09-13T08:38:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.627+0000 7fa617c00640 1 -- 172.21.15.44:0/2535785163 wait complete. 2024-09-13T08:38:02.653 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-13T08:38:02.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 -- 172.21.15.44:0/2732106957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d040052f0 msgr2=0x7f7d0400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 --2- 172.21.15.44:0/2732106957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d040052f0 0x7f7d0400b9d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d18006fd0 tx=0x7f7d1801f6a0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 -- 172.21.15.44:0/2732106957 shutdown_connections 2024-09-13T08:38:02.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 --2- 172.21.15.44:0/2732106957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d040052f0 0x7f7d0400b9d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 -- 172.21.15.44:0/2732106957 >> 172.21.15.44:0/2732106957 conn(0x7f7d200d1dc0 msgr2=0x7f7d200d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:02.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 -- 172.21.15.44:0/2732106957 shutdown_connections 2024-09-13T08:38:02.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 -- 172.21.15.44:0/2732106957 wait complete. 2024-09-13T08:38:02.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 Processor -- start 2024-09-13T08:38:02.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 -- start start 2024-09-13T08:38:02.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d040052f0 0x7f7d200c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d201f4810 0x7f7d2006f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d201e78f0 0x7f7d200b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d200ce7f0 con 0x7f7d040052f0 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d200ce670 con 0x7f7d201e78f0 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d27200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7d200ce970 con 0x7f7d201f4810 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d040052f0 0x7f7d200c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d040052f0 0x7f7d200c4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55972/0 (socket says 172.21.15.44:55972) 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d1fe00640 1 -- 172.21.15.44:0/2632899863 learned_addr learned my addr 172.21.15.44:0/2632899863 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d17400640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d201f4810 0x7f7d2006f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d25200640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d201e78f0 0x7f7d200b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d1fe00640 1 -- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d201f4810 msgr2=0x7f7d2006f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d1fe00640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d201f4810 0x7f7d2006f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d1fe00640 1 -- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d201e78f0 msgr2=0x7f7d200b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:02.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d1fe00640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d201e78f0 0x7f7d200b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:02.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.799+0000 7f7d1fe00640 1 -- 172.21.15.44:0/2632899863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d20155f70 con 0x7f7d040052f0 2024-09-13T08:38:02.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.803+0000 7f7d1fe00640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d040052f0 0x7f7d200c4c60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d10011ae0 tx=0x7f7d10009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:02.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.803+0000 7f7d1cc00640 1 -- 172.21.15.44:0/2632899863 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1001e070 con 0x7f7d040052f0 2024-09-13T08:38:02.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.803+0000 7f7d1cc00640 1 -- 172.21.15.44:0/2632899863 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d10004460 con 0x7f7d040052f0 2024-09-13T08:38:02.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.803+0000 7f7d1cc00640 1 -- 172.21.15.44:0/2632899863 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000fbe0 con 0x7f7d040052f0 2024-09-13T08:38:02.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.803+0000 7f7d27200640 1 -- 172.21.15.44:0/2632899863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2006fcd0 con 0x7f7d040052f0 2024-09-13T08:38:02.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.803+0000 7f7d27200640 1 -- 172.21.15.44:0/2632899863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d201e6e10 con 0x7f7d040052f0 2024-09-13T08:38:02.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.803+0000 7f7d27200640 1 -- 172.21.15.44:0/2632899863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d201f4bf0 con 0x7f7d040052f0 2024-09-13T08:38:02.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.803+0000 7f7d1cc00640 1 -- 172.21.15.44:0/2632899863 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f7d10021070 con 0x7f7d040052f0 2024-09-13T08:38:02.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.803+0000 7f7d1cc00640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7cfc072ef0 0x7f7cfc0753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:02.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.803+0000 7f7d1cc00640 1 -- 172.21.15.44:0/2632899863 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7d10094840 con 0x7f7d040052f0 2024-09-13T08:38:02.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.807+0000 7f7d17400640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7cfc072ef0 0x7f7cfc0753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:02.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.807+0000 7f7d17400640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7cfc072ef0 0x7f7cfc0753b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08006f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:02.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:02.807+0000 7f7d1cc00640 1 -- 172.21.15.44:0/2632899863 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7d1005f3a0 con 0x7f7d040052f0 2024-09-13T08:38:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.051+0000 7f7d27200640 1 -- 172.21.15.44:0/2632899863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f7d200ceed0 con 0x7f7d040052f0 2024-09-13T08:38:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.987+0000 7f7d1cc00640 1 -- 172.21.15.44:0/2632899863 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v50) ==== 187+0+0 (secure 0 0 0) 0x7f7d10063dd0 con 0x7f7d040052f0 2024-09-13T08:38:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 -- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7cfc072ef0 msgr2=0x7f7cfc0753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7cfc072ef0 0x7f7cfc0753b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08006f00 comp rx=0 tx=0).stop 2024-09-13T08:38:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 -- 172.21.15.44:0/2632899863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d040052f0 msgr2=0x7f7d200c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d040052f0 0x7f7d200c4c60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d10011ae0 tx=0x7f7d10009e50 comp rx=0 tx=0).stop 2024-09-13T08:38:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 -- 172.21.15.44:0/2632899863 shutdown_connections 2024-09-13T08:38:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7cfc072ef0 0x7f7cfc0753b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d201e78f0 0x7f7d200b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d201f4810 0x7f7d2006f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 --2- 172.21.15.44:0/2632899863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d040052f0 0x7f7d200c4c60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 -- 172.21.15.44:0/2632899863 >> 172.21.15.44:0/2632899863 conn(0x7f7d200d1dc0 msgr2=0x7f7d200af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 -- 172.21.15.44:0/2632899863 shutdown_connections 2024-09-13T08:38:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:03.991+0000 7f7d27200640 1 -- 172.21.15.44:0/2632899863 wait complete. 2024-09-13T08:38:04.019 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-13T08:38:04.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 -- 172.21.15.44:0/1010080517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21880b10a0 msgr2=0x7f2188074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:04.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 --2- 172.21.15.44:0/1010080517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21880b10a0 0x7f2188074380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21880c7480 tx=0x7f217001c8c0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 -- 172.21.15.44:0/1010080517 shutdown_connections 2024-09-13T08:38:04.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 --2- 172.21.15.44:0/1010080517 >> v2:172.21.15.129:3300/0 conn(0x7f21880748c0 0x7f21880cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 --2- 172.21.15.44:0/1010080517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21880b10a0 0x7f2188074380 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 -- 172.21.15.44:0/1010080517 >> 172.21.15.44:0/1010080517 conn(0x7f21880b1f10 msgr2=0x7f21880b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 -- 172.21.15.44:0/1010080517 shutdown_connections 2024-09-13T08:38:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 -- 172.21.15.44:0/1010080517 wait complete. 2024-09-13T08:38:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 Processor -- start 2024-09-13T08:38:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 -- start start 2024-09-13T08:38:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21880748c0 0x7f21880c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21880b10a0 0x7f21881558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f218815bea0 0x7f2188156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2188168ee0 con 0x7f218815bea0 2024-09-13T08:38:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2188168d60 con 0x7f21880b10a0 2024-09-13T08:38:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.167+0000 7f218d000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2188169060 con 0x7f21880748c0 2024-09-13T08:38:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f2186800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f218815bea0 0x7f2188156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f2186800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f218815bea0 0x7f2188156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55980/0 (socket says 172.21.15.44:55980) 2024-09-13T08:38:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f2186800640 1 -- 172.21.15.44:0/2007161740 learned_addr learned my addr 172.21.15.44:0/2007161740 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f2185e00640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21880748c0 0x7f21880c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:04.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f217fe00640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21880b10a0 0x7f21881558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f2186800640 1 -- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21880748c0 msgr2=0x7f21880c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f2186800640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21880748c0 0x7f21880c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f2186800640 1 -- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21880b10a0 msgr2=0x7f21881558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f2186800640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21880b10a0 0x7f21881558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f2186800640 1 -- 172.21.15.44:0/2007161740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2188156da0 con 0x7f218815bea0 2024-09-13T08:38:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f2186800640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f218815bea0 0x7f2188156860 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f218000eb50 tx=0x7f218000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f217e000640 1 -- 172.21.15.44:0/2007161740 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180004d30 con 0x7f218815bea0 2024-09-13T08:38:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f217e000640 1 -- 172.21.15.44:0/2007161740 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f218000ee50 con 0x7f218815bea0 2024-09-13T08:38:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f217e000640 1 -- 172.21.15.44:0/2007161740 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180005280 con 0x7f218815bea0 2024-09-13T08:38:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f218d000640 1 -- 172.21.15.44:0/2007161740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21881f8bc0 con 0x7f218815bea0 2024-09-13T08:38:04.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f218d000640 1 -- 172.21.15.44:0/2007161740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21881f9140 con 0x7f218815bea0 2024-09-13T08:38:04.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f218d000640 1 -- 172.21.15.44:0/2007161740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21880cbe30 con 0x7f218815bea0 2024-09-13T08:38:04.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.171+0000 7f217e000640 1 -- 172.21.15.44:0/2007161740 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f2180005420 con 0x7f218815bea0 2024-09-13T08:38:04.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.175+0000 7f217e000640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2164072ec0 0x7f2164075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:04.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.175+0000 7f217e000640 1 -- 172.21.15.44:0/2007161740 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2180094a40 con 0x7f218815bea0 2024-09-13T08:38:04.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.175+0000 7f2185e00640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2164072ec0 0x7f2164075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:04.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.175+0000 7f2185e00640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2164072ec0 0x7f2164075380 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2178007fd0 tx=0x7f2178009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:04.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.179+0000 7f217e000640 1 -- 172.21.15.44:0/2007161740 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2188156da0 con 0x7f218815bea0 2024-09-13T08:38:04.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.359+0000 7f218d000640 1 -- 172.21.15.44:0/2007161740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f21880b62a0 con 0x7f218815bea0 2024-09-13T08:38:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.359+0000 7f217e000640 1 -- 172.21.15.44:0/2007161740 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7f218005f630 con 0x7f218815bea0 2024-09-13T08:38:04.364 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-13T08:38:04.364 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-09-13T08:38:04.364 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-13T08:38:04.364 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-13T08:38:04.364 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4094.620852 2024-09-13T08:38:04.364 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3964.014083 2024-09-13T08:38:04.364 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3978.263337 2024-09-13T08:38:04.364 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4002.298162 2024-09-13T08:38:04.364 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4667.000623 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4664.046790 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6403.016405 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4696.908546 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 2/2 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-09-13T08:38:04.365 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-13T08:38:04.366 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-09-13T08:38:04.366 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-09-13T08:38:04.366 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-09-13T08:38:04.366 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-09-13T08:38:04.366 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-09-13T08:38:04.366 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-09-13T08:38:04.366 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-09-13T08:38:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 -- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2164072ec0 msgr2=0x7f2164075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2164072ec0 0x7f2164075380 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2178007fd0 tx=0x7f2178009040 comp rx=0 tx=0).stop 2024-09-13T08:38:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 -- 172.21.15.44:0/2007161740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f218815bea0 msgr2=0x7f2188156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f218815bea0 0x7f2188156860 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f218000eb50 tx=0x7f218000c6a0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 -- 172.21.15.44:0/2007161740 shutdown_connections 2024-09-13T08:38:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2164072ec0 0x7f2164075380 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f218815bea0 0x7f2188156860 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21880b10a0 0x7f21881558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 --2- 172.21.15.44:0/2007161740 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21880748c0 0x7f21880c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 -- 172.21.15.44:0/2007161740 >> 172.21.15.44:0/2007161740 conn(0x7f21880b1f10 msgr2=0x7f2188068960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 -- 172.21.15.44:0/2007161740 shutdown_connections 2024-09-13T08:38:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.363+0000 7f218d000640 1 -- 172.21.15.44:0/2007161740 wait complete. 2024-09-13T08:38:04.386 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:38:04.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 -- 172.21.15.44:0/1831243177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97900b3190 msgr2=0x7f97900b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:04.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 --2- 172.21.15.44:0/1831243177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97900b3190 0x7f97900b1450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9790073990 tx=0x7f978801cc30 comp rx=0 tx=0).stop 2024-09-13T08:38:04.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 -- 172.21.15.44:0/1831243177 shutdown_connections 2024-09-13T08:38:04.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 --2- 172.21.15.44:0/1831243177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97900b3190 0x7f97900b1450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 --2- 172.21.15.44:0/1831243177 >> v2:172.21.15.129:3300/0 conn(0x7f97901613c0 0x7f9790161bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 -- 172.21.15.44:0/1831243177 >> 172.21.15.44:0/1831243177 conn(0x7f97900d1dd0 msgr2=0x7f97900d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:04.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 -- 172.21.15.44:0/1831243177 shutdown_connections 2024-09-13T08:38:04.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 -- 172.21.15.44:0/1831243177 wait complete. 2024-09-13T08:38:04.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 Processor -- start 2024-09-13T08:38:04.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 -- start start 2024-09-13T08:38:04.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97900b3190 0x7f979015c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:04.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97901613c0 0x7f9790153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:04.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f97901617a0 0x7f979015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:04.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f979016b0b0 con 0x7f97901613c0 2024-09-13T08:38:04.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f979016af30 con 0x7f97900b3190 2024-09-13T08:38:04.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.531+0000 7f9798400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f979016b230 con 0x7f97901617a0 2024-09-13T08:38:04.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9795000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97901613c0 0x7f9790153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:04.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9795000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97901613c0 0x7f9790153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55986/0 (socket says 172.21.15.44:55986) 2024-09-13T08:38:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9795000640 1 -- 172.21.15.44:0/3237815424 learned_addr learned my addr 172.21.15.44:0/3237815424 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9795a00640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97900b3190 0x7f979015c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9796400640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f97901617a0 0x7f979015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9795000640 1 -- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f97901617a0 msgr2=0x7f979015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9795000640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f97901617a0 0x7f979015d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9795000640 1 -- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97900b3190 msgr2=0x7f979015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:04.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9795000640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97900b3190 0x7f979015c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9795000640 1 -- 172.21.15.44:0/3237815424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97901fb480 con 0x7f97901613c0 2024-09-13T08:38:04.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9795000640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97901613c0 0x7f9790153190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c3c0 tx=0x7f977c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:04.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f978d600640 1 -- 172.21.15.44:0/3237815424 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c017070 con 0x7f97901613c0 2024-09-13T08:38:04.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f978d600640 1 -- 172.21.15.44:0/3237815424 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f977c0053c0 con 0x7f97901613c0 2024-09-13T08:38:04.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9798400640 1 -- 172.21.15.44:0/3237815424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97901fb760 con 0x7f97901613c0 2024-09-13T08:38:04.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f978d600640 1 -- 172.21.15.44:0/3237815424 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c003860 con 0x7f97901613c0 2024-09-13T08:38:04.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9798400640 1 -- 172.21.15.44:0/3237815424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97901fbd20 con 0x7f97901613c0 2024-09-13T08:38:04.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f9798400640 1 -- 172.21.15.44:0/3237815424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97900b1450 con 0x7f97901613c0 2024-09-13T08:38:04.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.535+0000 7f978d600640 1 -- 172.21.15.44:0/3237815424 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f977c01a070 con 0x7f97901613c0 2024-09-13T08:38:04.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.539+0000 7f978d600640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9770072ec0 0x7f9770075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:04.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.539+0000 7f978d600640 1 -- 172.21.15.44:0/3237815424 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f977c093070 con 0x7f97901613c0 2024-09-13T08:38:04.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.539+0000 7f9795a00640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9770072ec0 0x7f9770075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:04.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.539+0000 7f9795a00640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9770072ec0 0x7f9770075380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9784004630 tx=0x7f978400b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:04.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.543+0000 7f978d600640 1 -- 172.21.15.44:0/3237815424 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f977c05dc60 con 0x7f97901613c0 2024-09-13T08:38:04.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.719+0000 7f9798400640 1 -- 172.21.15.44:0/3237815424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f979015b200 con 0x7f97901613c0 2024-09-13T08:38:04.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.723+0000 7f978d600640 1 -- 172.21.15.44:0/3237815424 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f977c062690 con 0x7f97901613c0 2024-09-13T08:38:04.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 -- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9770072ec0 msgr2=0x7f9770075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:04.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9770072ec0 0x7f9770075380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9784004630 tx=0x7f978400b390 comp rx=0 tx=0).stop 2024-09-13T08:38:04.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 -- 172.21.15.44:0/3237815424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97901613c0 msgr2=0x7f9790153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:04.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97901613c0 0x7f9790153190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c3c0 tx=0x7f977c00c880 comp rx=0 tx=0).stop 2024-09-13T08:38:04.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 -- 172.21.15.44:0/3237815424 shutdown_connections 2024-09-13T08:38:04.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9770072ec0 0x7f9770075380 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f97901617a0 0x7f979015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97901613c0 0x7f9790153190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 --2- 172.21.15.44:0/3237815424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97900b3190 0x7f979015c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:04.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 -- 172.21.15.44:0/3237815424 >> 172.21.15.44:0/3237815424 conn(0x7f97900d1dd0 msgr2=0x7f9790161b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:04.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 -- 172.21.15.44:0/3237815424 shutdown_connections 2024-09-13T08:38:04.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:04.727+0000 7f9798400640 1 -- 172.21.15.44:0/3237815424 wait complete. 2024-09-13T08:38:04.749 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:38:04.749 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:38:04.824 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:38:04.905 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:38:04.905 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:04.905 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:04.914 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:04.914 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:04.969 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:04.970 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:05.022 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:38:05.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.211+0000 7f9e8fe00640 1 -- 172.21.15.44:0/852884712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e9015f000 msgr2=0x7f9e900d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:05.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.211+0000 7f9e8fe00640 1 --2- 172.21.15.44:0/852884712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e9015f000 0x7f9e900d01b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009840 tx=0x7f9e8001c700 comp rx=0 tx=0).stop 2024-09-13T08:38:05.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.211+0000 7f9e8fe00640 1 -- 172.21.15.44:0/852884712 shutdown_connections 2024-09-13T08:38:05.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.211+0000 7f9e8fe00640 1 --2- 172.21.15.44:0/852884712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e9015f000 0x7f9e900d01b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.211+0000 7f9e8fe00640 1 -- 172.21.15.44:0/852884712 >> 172.21.15.44:0/852884712 conn(0x7f9e9009aa10 msgr2=0x7f9e9009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:05.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.211+0000 7f9e8fe00640 1 -- 172.21.15.44:0/852884712 shutdown_connections 2024-09-13T08:38:05.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.211+0000 7f9e8fe00640 1 -- 172.21.15.44:0/852884712 wait complete. 2024-09-13T08:38:05.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8fe00640 1 Processor -- start 2024-09-13T08:38:05.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8fe00640 1 -- start start 2024-09-13T08:38:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e900b2590 0x7f9e90159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e900cefb0 0x7f9e90166440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e9015f000 0x7f9e900b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9016a110 con 0x7f9e900b2590 2024-09-13T08:38:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9e90169f90 con 0x7f9e900cefb0 2024-09-13T08:38:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9016a290 con 0x7f9e9015f000 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e900b2590 0x7f9e90159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e900b2590 0x7f9e90159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55994/0 (socket says 172.21.15.44:55994) 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8ea00640 1 -- 172.21.15.44:0/1969971545 learned_addr learned my addr 172.21.15.44:0/1969971545 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e87e00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e900cefb0 0x7f9e90166440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8f400640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e9015f000 0x7f9e900b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8ea00640 1 -- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e9015f000 msgr2=0x7f9e900b6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8ea00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e9015f000 0x7f9e900b6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8ea00640 1 -- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e900cefb0 msgr2=0x7f9e90166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8ea00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e900cefb0 0x7f9e90166440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8ea00640 1 -- 172.21.15.44:0/1969971545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9009b5b0 con 0x7f9e900b2590 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8f400640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e9015f000 0x7f9e900b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8ea00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e900b2590 0x7f9e90159c90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009790 tx=0x7f9e80005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e86a00640 1 -- 172.21.15.44:0/1969971545 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e800a85f0 con 0x7f9e900b2590 2024-09-13T08:38:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e86a00640 1 -- 172.21.15.44:0/1969971545 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e800a8a70 con 0x7f9e900b2590 2024-09-13T08:38:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8fe00640 1 -- 172.21.15.44:0/1969971545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e900ce940 con 0x7f9e900b2590 2024-09-13T08:38:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e86a00640 1 -- 172.21.15.44:0/1969971545 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e800a8e60 con 0x7f9e900b2590 2024-09-13T08:38:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.215+0000 7f9e8fe00640 1 -- 172.21.15.44:0/1969971545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e901f80e0 con 0x7f9e900b2590 2024-09-13T08:38:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.219+0000 7f9e8fe00640 1 -- 172.21.15.44:0/1969971545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e9015a1d0 con 0x7f9e900b2590 2024-09-13T08:38:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.219+0000 7f9e86a00640 1 -- 172.21.15.44:0/1969971545 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f9e80004030 con 0x7f9e900b2590 2024-09-13T08:38:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.219+0000 7f9e86a00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e6c072ec0 0x7f9e6c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.219+0000 7f9e86a00640 1 -- 172.21.15.44:0/1969971545 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9e8012e350 con 0x7f9e900b2590 2024-09-13T08:38:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.219+0000 7f9e87e00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e6c072ec0 0x7f9e6c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.219+0000 7f9e87e00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e6c072ec0 0x7f9e6c075380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e900cc200 tx=0x7f9e78009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.223+0000 7f9e86a00640 1 -- 172.21.15.44:0/1969971545 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9e800ac030 con 0x7f9e900b2590 2024-09-13T08:38:05.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.411+0000 7f9e8fe00640 1 -- 172.21.15.44:0/1969971545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9e900ce200 con 0x7f9e900b2590 2024-09-13T08:38:05.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.411+0000 7f9e86a00640 1 -- 172.21.15.44:0/1969971545 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f9e80016410 con 0x7f9e900b2590 2024-09-13T08:38:05.416 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:38:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 -- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e6c072ec0 msgr2=0x7f9e6c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e6c072ec0 0x7f9e6c075380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e900cc200 tx=0x7f9e78009040 comp rx=0 tx=0).stop 2024-09-13T08:38:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 -- 172.21.15.44:0/1969971545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e900b2590 msgr2=0x7f9e90159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e900b2590 0x7f9e90159c90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009790 tx=0x7f9e80005d30 comp rx=0 tx=0).stop 2024-09-13T08:38:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 -- 172.21.15.44:0/1969971545 shutdown_connections 2024-09-13T08:38:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e6c072ec0 0x7f9e6c075380 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e9015f000 0x7f9e900b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e900cefb0 0x7f9e90166440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 --2- 172.21.15.44:0/1969971545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e900b2590 0x7f9e90159c90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 -- 172.21.15.44:0/1969971545 >> 172.21.15.44:0/1969971545 conn(0x7f9e9009aa10 msgr2=0x7f9e9006dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 -- 172.21.15.44:0/1969971545 shutdown_connections 2024-09-13T08:38:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.415+0000 7f9e8fe00640 1 -- 172.21.15.44:0/1969971545 wait complete. 2024-09-13T08:38:05.437 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T08:38:05.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:05.437 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-13T08:38:05.471 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:38:05.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:05.472 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T08:38:05.524 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:38:05.524 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:05.534 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:38:05.534 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:05.590 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:38:05.590 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:05.642 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:38:05.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.787+0000 7f1164200640 1 -- 172.21.15.44:0/4114888823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11440051d0 msgr2=0x7f114400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:05.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.787+0000 7f1164200640 1 --2- 172.21.15.44:0/4114888823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11440051d0 0x7f114400b9d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f115c0aff30 tx=0x7f115401caf0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.787+0000 7f1164200640 1 -- 172.21.15.44:0/4114888823 shutdown_connections 2024-09-13T08:38:05.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.787+0000 7f1164200640 1 --2- 172.21.15.44:0/4114888823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11440051d0 0x7f114400b9d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.787+0000 7f1164200640 1 -- 172.21.15.44:0/4114888823 >> 172.21.15.44:0/4114888823 conn(0x7f115c0b2240 msgr2=0x7f115c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:05.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.787+0000 7f1164200640 1 -- 172.21.15.44:0/4114888823 shutdown_connections 2024-09-13T08:38:05.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.787+0000 7f1164200640 1 -- 172.21.15.44:0/4114888823 wait complete. 2024-09-13T08:38:05.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.787+0000 7f1164200640 1 Processor -- start 2024-09-13T08:38:05.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.787+0000 7f1164200640 1 -- start start 2024-09-13T08:38:05.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1164200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11440051d0 0x7f115c1f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1164200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f115c0aedd0 0x7f115c15ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1164200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c153f10 0x7f115c154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1164200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f115c0732f0 con 0x7f115c153f10 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1164200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f115c073170 con 0x7f11440051d0 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1164200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f115c06f1d0 con 0x7f115c0aedd0 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1162200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c153f10 0x7f115c154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1161800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11440051d0 0x7f115c1f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1161800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11440051d0 0x7f115c1f38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:38204/0 (socket says 172.21.15.44:38204) 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1161800640 1 -- 172.21.15.44:0/1910446250 learned_addr learned my addr 172.21.15.44:0/1910446250 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1160e00640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f115c0aedd0 0x7f115c15ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1161800640 1 -- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f115c0aedd0 msgr2=0x7f115c15ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1161800640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f115c0aedd0 0x7f115c15ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1161800640 1 -- 172.21.15.44:0/1910446250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c153f10 msgr2=0x7f115c154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1161800640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c153f10 0x7f115c154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1161800640 1 -- 172.21.15.44:0/1910446250 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115c1f2ae0 con 0x7f11440051d0 2024-09-13T08:38:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1162200640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c153f10 0x7f115c154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:38:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1161800640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11440051d0 0x7f115c1f38c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1150006660 tx=0x7f11500039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1159600640 1 -- 172.21.15.44:0/1910446250 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150015070 con 0x7f11440051d0 2024-09-13T08:38:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1159600640 1 -- 172.21.15.44:0/1910446250 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1150011900 con 0x7f11440051d0 2024-09-13T08:38:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1164200640 1 -- 172.21.15.44:0/1910446250 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115c1f2dc0 con 0x7f11440051d0 2024-09-13T08:38:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1159600640 1 -- 172.21.15.44:0/1910446250 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150011d50 con 0x7f11440051d0 2024-09-13T08:38:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1164200640 1 -- 172.21.15.44:0/1910446250 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115c1f3380 con 0x7f11440051d0 2024-09-13T08:38:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.791+0000 7f1164200640 1 -- 172.21.15.44:0/1910446250 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f115c0d0500 con 0x7f11440051d0 2024-09-13T08:38:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.795+0000 7f1159600640 1 -- 172.21.15.44:0/1910446250 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f1150018070 con 0x7f11440051d0 2024-09-13T08:38:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.795+0000 7f1159600640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f113c072ec0 0x7f113c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.795+0000 7f1159600640 1 -- 172.21.15.44:0/1910446250 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1150094bc0 con 0x7f11440051d0 2024-09-13T08:38:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.795+0000 7f1160e00640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f113c072ec0 0x7f113c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.795+0000 7f1160e00640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f113c072ec0 0x7f113c075380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f115c0b4830 tx=0x7f1148005f40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.799+0000 7f1159600640 1 -- 172.21.15.44:0/1910446250 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f115005f8a0 con 0x7f11440051d0 2024-09-13T08:38:05.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.987+0000 7f1164200640 1 -- 172.21.15.44:0/1910446250 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f115c151af0 con 0x7f11440051d0 2024-09-13T08:38:05.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.987+0000 7f1159600640 1 -- 172.21.15.44:0/1910446250 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f11500642d0 con 0x7f11440051d0 2024-09-13T08:38:05.992 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:38:05.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 -- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f113c072ec0 msgr2=0x7f113c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:05.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f113c072ec0 0x7f113c075380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f115c0b4830 tx=0x7f1148005f40 comp rx=0 tx=0).stop 2024-09-13T08:38:05.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 -- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11440051d0 msgr2=0x7f115c1f38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:05.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11440051d0 0x7f115c1f38c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1150006660 tx=0x7f11500039d0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 -- 172.21.15.44:0/1910446250 shutdown_connections 2024-09-13T08:38:05.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f113c072ec0 0x7f113c075380 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f115c153f10 0x7f115c154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f115c0aedd0 0x7f115c15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 --2- 172.21.15.44:0/1910446250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11440051d0 0x7f115c1f38c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:05.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 -- 172.21.15.44:0/1910446250 >> 172.21.15.44:0/1910446250 conn(0x7f115c0b2240 msgr2=0x7f115c073470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:05.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 -- 172.21.15.44:0/1910446250 shutdown_connections 2024-09-13T08:38:05.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:05.991+0000 7f1164200640 1 -- 172.21.15.44:0/1910446250 wait complete. 2024-09-13T08:38:06.015 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-13T08:38:06.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:06.015 DEBUG:teuthology.orchestra.run.smithi129:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-13T08:38:06.054 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:38:06.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:06.054 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T08:38:06.123 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:06.123 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:06.132 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:06.132 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:06.181 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:06.182 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:06.233 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:38:06.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 -- 172.21.15.44:0/145241795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f186c06e810 msgr2=0x7f186c0cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:06.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 --2- 172.21.15.44:0/145241795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f186c06e810 0x7f186c0cf130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f186c06e730 tx=0x7f185801c5d0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 -- 172.21.15.44:0/145241795 shutdown_connections 2024-09-13T08:38:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 --2- 172.21.15.44:0/145241795 >> v2:172.21.15.129:3300/0 conn(0x7f186c0b4bd0 0x7f186c15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 --2- 172.21.15.44:0/145241795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f186c06e810 0x7f186c0cf130 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 --2- 172.21.15.44:0/145241795 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f186c15ade0 0x7f186c0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 -- 172.21.15.44:0/145241795 >> 172.21.15.44:0/145241795 conn(0x7f186c0b18f0 msgr2=0x7f186c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 -- 172.21.15.44:0/145241795 shutdown_connections 2024-09-13T08:38:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 -- 172.21.15.44:0/145241795 wait complete. 2024-09-13T08:38:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 Processor -- start 2024-09-13T08:38:06.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 -- start start 2024-09-13T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f186c06e810 0x7f186c1526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f186c0b4bd0 0x7f186c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f186c15ade0 0x7f186c15c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f186c06f040 con 0x7f186c0b4bd0 2024-09-13T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f186c06eec0 con 0x7f186c06e810 2024-09-13T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1873800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f186c06f1c0 con 0x7f186c15ade0 2024-09-13T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f186be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f186c0b4bd0 0x7f186c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f186be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f186c0b4bd0 0x7f186c15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56018/0 (socket says 172.21.15.44:56018) 2024-09-13T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f186be00640 1 -- 172.21.15.44:0/3283788876 learned_addr learned my addr 172.21.15.44:0/3283788876 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1870e00640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f186c06e810 0x7f186c1526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f186be00640 1 -- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f186c15ade0 msgr2=0x7f186c15c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1871800640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f186c15ade0 0x7f186c15c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f186be00640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f186c15ade0 0x7f186c15c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f186be00640 1 -- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f186c06e810 msgr2=0x7f186c1526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f186be00640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f186c06e810 0x7f186c1526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f186be00640 1 -- 172.21.15.44:0/3283788876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c165890 con 0x7f186c0b4bd0 2024-09-13T08:38:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1870e00640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f186c06e810 0x7f186c1526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.423+0000 7f1871800640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f186c15ade0 0x7f186c15c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f186be00640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f186c0b4bd0 0x7f186c15bff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1858009850 tx=0x7f18580a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f1868c00640 1 -- 172.21.15.44:0/3283788876 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18580ac070 con 0x7f186c0b4bd0 2024-09-13T08:38:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f1868c00640 1 -- 172.21.15.44:0/3283788876 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1858004340 con 0x7f186c0b4bd0 2024-09-13T08:38:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f1868c00640 1 -- 172.21.15.44:0/3283788876 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18580053a0 con 0x7f186c0b4bd0 2024-09-13T08:38:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f1873800640 1 -- 172.21.15.44:0/3283788876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c09c8f0 con 0x7f186c0b4bd0 2024-09-13T08:38:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f1873800640 1 -- 172.21.15.44:0/3283788876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c09ceb0 con 0x7f186c0b4bd0 2024-09-13T08:38:06.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f1873800640 1 -- 172.21.15.44:0/3283788876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c15e900 con 0x7f186c0b4bd0 2024-09-13T08:38:06.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f1868c00640 1 -- 172.21.15.44:0/3283788876 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f18580af070 con 0x7f186c0b4bd0 2024-09-13T08:38:06.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f1868c00640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f184c072ec0 0x7f184c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:06.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f1868c00640 1 -- 172.21.15.44:0/3283788876 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f185812e330 con 0x7f186c0b4bd0 2024-09-13T08:38:06.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.427+0000 7f1870e00640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f184c072ec0 0x7f184c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:06.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.431+0000 7f1870e00640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f184c072ec0 0x7f184c075380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f186000a870 tx=0x7f1860008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:06.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.431+0000 7f1868c00640 1 -- 172.21.15.44:0/3283788876 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f18580164b0 con 0x7f186c0b4bd0 2024-09-13T08:38:06.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.619+0000 7f1873800640 1 -- 172.21.15.44:0/3283788876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f186c0b25e0 con 0x7f186c0b4bd0 2024-09-13T08:38:06.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.619+0000 7f1868c00640 1 -- 172.21.15.44:0/3283788876 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f18580f9570 con 0x7f186c0b4bd0 2024-09-13T08:38:06.625 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:38:06.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 -- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f184c072ec0 msgr2=0x7f184c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:06.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f184c072ec0 0x7f184c075380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f186000a870 tx=0x7f1860008040 comp rx=0 tx=0).stop 2024-09-13T08:38:06.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 -- 172.21.15.44:0/3283788876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f186c0b4bd0 msgr2=0x7f186c15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:06.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f186c0b4bd0 0x7f186c15bff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1858009850 tx=0x7f18580a6570 comp rx=0 tx=0).stop 2024-09-13T08:38:06.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 -- 172.21.15.44:0/3283788876 shutdown_connections 2024-09-13T08:38:06.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f184c072ec0 0x7f184c075380 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f186c15ade0 0x7f186c15c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f186c0b4bd0 0x7f186c15bff0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 --2- 172.21.15.44:0/3283788876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f186c06e810 0x7f186c1526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 -- 172.21.15.44:0/3283788876 >> 172.21.15.44:0/3283788876 conn(0x7f186c0b18f0 msgr2=0x7f186c072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:06.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 -- 172.21.15.44:0/3283788876 shutdown_connections 2024-09-13T08:38:06.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.623+0000 7f1873800640 1 -- 172.21.15.44:0/3283788876 wait complete. 2024-09-13T08:38:06.646 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-13T08:38:06.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:06.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-13T08:38:06.670 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:38:06.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:06.670 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-13T08:38:06.726 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:38:06.727 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:06.736 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:38:06.737 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:06.786 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:38:06.786 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:06.838 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:38:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.983+0000 7fcd1f400640 1 -- 172.21.15.44:0/2722730491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd1815bd50 msgr2=0x7fcd180ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.983+0000 7fcd1f400640 1 --2- 172.21.15.44:0/2722730491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd1815bd50 0x7fcd180ce5e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009840 tx=0x7fcd0c01c700 comp rx=0 tx=0).stop 2024-09-13T08:38:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.983+0000 7fcd1f400640 1 -- 172.21.15.44:0/2722730491 shutdown_connections 2024-09-13T08:38:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.983+0000 7fcd1f400640 1 --2- 172.21.15.44:0/2722730491 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcd1815c130 0x7fcd180ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.983+0000 7fcd1f400640 1 --2- 172.21.15.44:0/2722730491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd1815bd50 0x7fcd180ce5e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.983+0000 7fcd1f400640 1 -- 172.21.15.44:0/2722730491 >> 172.21.15.44:0/2722730491 conn(0x7fcd180d1ea0 msgr2=0x7fcd180d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1f400640 1 -- 172.21.15.44:0/2722730491 shutdown_connections 2024-09-13T08:38:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1f400640 1 -- 172.21.15.44:0/2722730491 wait complete. 2024-09-13T08:38:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1f400640 1 Processor -- start 2024-09-13T08:38:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1f400640 1 -- start start 2024-09-13T08:38:06.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd180cf4e0 0x7fcd180ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:06.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd1815bd50 0x7fcd1814fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:06.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1f400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcd1815c130 0x7fcd180c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1f400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18168e50 con 0x7fcd180cf4e0 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1f400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18168cd0 con 0x7fcd1815bd50 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1f400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcd18168fd0 con 0x7fcd1815c130 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd180cf4e0 0x7fcd180ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd180cf4e0 0x7fcd180ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56032/0 (socket says 172.21.15.44:56032) 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1ca00640 1 -- 172.21.15.44:0/1268384323 learned_addr learned my addr 172.21.15.44:0/1268384323 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd17e00640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd1815bd50 0x7fcd1814fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1d400640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcd1815c130 0x7fcd180c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1ca00640 1 -- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcd1815c130 msgr2=0x7fcd180c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1ca00640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcd1815c130 0x7fcd180c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1ca00640 1 -- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd1815bd50 msgr2=0x7fcd1814fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1ca00640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd1815bd50 0x7fcd1814fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1ca00640 1 -- 172.21.15.44:0/1268384323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd180ca790 con 0x7fcd180cf4e0 2024-09-13T08:38:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1d400640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcd1815c130 0x7fcd180c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.987+0000 7fcd1ca00640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd180cf4e0 0x7fcd180ca0d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009790 tx=0x7fcd0c0a6770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.991+0000 7fcd14c00640 1 -- 172.21.15.44:0/1268384323 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0bf020 con 0x7fcd180cf4e0 2024-09-13T08:38:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.991+0000 7fcd14c00640 1 -- 172.21.15.44:0/1268384323 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd0c005020 con 0x7fcd180cf4e0 2024-09-13T08:38:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.991+0000 7fcd14c00640 1 -- 172.21.15.44:0/1268384323 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c005430 con 0x7fcd180cf4e0 2024-09-13T08:38:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.991+0000 7fcd1f400640 1 -- 172.21.15.44:0/1268384323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd180caa70 con 0x7fcd180cf4e0 2024-09-13T08:38:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.991+0000 7fcd1f400640 1 -- 172.21.15.44:0/1268384323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd180cb030 con 0x7fcd180cf4e0 2024-09-13T08:38:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.991+0000 7fcd1f400640 1 -- 172.21.15.44:0/1268384323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd180c5dd0 con 0x7fcd180cf4e0 2024-09-13T08:38:06.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.991+0000 7fcd14c00640 1 -- 172.21.15.44:0/1268384323 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fcd0c0c2070 con 0x7fcd180cf4e0 2024-09-13T08:38:06.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.991+0000 7fcd14c00640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fccf8072ec0 0x7fccf8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:06.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.991+0000 7fcd14c00640 1 -- 172.21.15.44:0/1268384323 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcd0c12e760 con 0x7fcd180cf4e0 2024-09-13T08:38:06.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.991+0000 7fcd17e00640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fccf8072ec0 0x7fccf8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:06.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.995+0000 7fcd17e00640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fccf8072ec0 0x7fccf8075380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcd180d2460 tx=0x7fcd0400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:07.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:06.995+0000 7fcd14c00640 1 -- 172.21.15.44:0/1268384323 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcd0c016410 con 0x7fcd180cf4e0 2024-09-13T08:38:07.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.183+0000 7fcd1f400640 1 -- 172.21.15.44:0/1268384323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcd180cc2d0 con 0x7fcd180cf4e0 2024-09-13T08:38:07.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.183+0000 7fcd14c00640 1 -- 172.21.15.44:0/1268384323 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fcd0c0f92c0 con 0x7fcd180cf4e0 2024-09-13T08:38:07.188 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:38:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 -- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fccf8072ec0 msgr2=0x7fccf8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fccf8072ec0 0x7fccf8075380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcd180d2460 tx=0x7fcd0400b2e0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 -- 172.21.15.44:0/1268384323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd180cf4e0 msgr2=0x7fcd180ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd180cf4e0 0x7fcd180ca0d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009790 tx=0x7fcd0c0a6770 comp rx=0 tx=0).stop 2024-09-13T08:38:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 -- 172.21.15.44:0/1268384323 shutdown_connections 2024-09-13T08:38:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fccf8072ec0 0x7fccf8075380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcd1815c130 0x7fcd180c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd1815bd50 0x7fcd1814fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 --2- 172.21.15.44:0/1268384323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd180cf4e0 0x7fcd180ca0d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 -- 172.21.15.44:0/1268384323 >> 172.21.15.44:0/1268384323 conn(0x7fcd180d1ea0 msgr2=0x7fcd180b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:07.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 -- 172.21.15.44:0/1268384323 shutdown_connections 2024-09-13T08:38:07.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.187+0000 7fcd1f400640 1 -- 172.21.15.44:0/1268384323 wait complete. 2024-09-13T08:38:07.211 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-13T08:38:07.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:07.211 DEBUG:teuthology.orchestra.run.smithi129:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-13T08:38:07.238 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:38:07.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:07.238 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-13T08:38:07.291 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T08:38:07.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:38:07.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.431+0000 7f950ec00640 1 -- 172.21.15.44:0/3739315700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f950815c130 msgr2=0x7f95080ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:07.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.431+0000 7f950ec00640 1 --2- 172.21.15.44:0/3739315700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f950815c130 0x7f95080ceb20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9508073a20 tx=0x7f94f001c7a0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.431+0000 7f950ec00640 1 -- 172.21.15.44:0/3739315700 shutdown_connections 2024-09-13T08:38:07.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.431+0000 7f950ec00640 1 --2- 172.21.15.44:0/3739315700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f950815c130 0x7f95080ceb20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.431+0000 7f950ec00640 1 --2- 172.21.15.44:0/3739315700 >> v2:172.21.15.129:3300/0 conn(0x7f950815bd50 0x7f95080ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.431+0000 7f950ec00640 1 -- 172.21.15.44:0/3739315700 >> 172.21.15.44:0/3739315700 conn(0x7f95080d1ea0 msgr2=0x7f95080d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:07.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.431+0000 7f950ec00640 1 -- 172.21.15.44:0/3739315700 shutdown_connections 2024-09-13T08:38:07.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.431+0000 7f950ec00640 1 -- 172.21.15.44:0/3739315700 wait complete. 2024-09-13T08:38:07.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950ec00640 1 Processor -- start 2024-09-13T08:38:07.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950ec00640 1 -- start start 2024-09-13T08:38:07.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95080cf4e0 0x7f95080ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:07.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950ec00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950815bd50 0x7f9508158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:07.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950ec00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950815c130 0x7f95081587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:07.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9508168e50 con 0x7f95080cf4e0 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950ec00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9508168cd0 con 0x7f950815c130 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950ec00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9508168fd0 con 0x7f950815bd50 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95080cf4e0 0x7f95080ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95080cf4e0 0x7f95080ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56048/0 (socket says 172.21.15.44:56048) 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9507e00640 1 -- 172.21.15.44:0/424688633 learned_addr learned my addr 172.21.15.44:0/424688633 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f94ffe00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950815bd50 0x7f9508158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950cc00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950815c130 0x7f95081587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9507e00640 1 -- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950815bd50 msgr2=0x7f9508158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9507e00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950815bd50 0x7f9508158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9507e00640 1 -- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950815c130 msgr2=0x7f95081587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9507e00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950815c130 0x7f95081587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9507e00640 1 -- 172.21.15.44:0/424688633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9508144320 con 0x7f95080cf4e0 2024-09-13T08:38:07.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9507e00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95080cf4e0 0x7f95080ca860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f94f800c970 tx=0x7f94f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:07.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9504c00640 1 -- 172.21.15.44:0/424688633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8007bb0 con 0x7f95080cf4e0 2024-09-13T08:38:07.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9504c00640 1 -- 172.21.15.44:0/424688633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94f8007e40 con 0x7f95080cf4e0 2024-09-13T08:38:07.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f9504c00640 1 -- 172.21.15.44:0/424688633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8005260 con 0x7f95080cf4e0 2024-09-13T08:38:07.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950ec00640 1 -- 172.21.15.44:0/424688633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95080cb0e0 con 0x7f95080cf4e0 2024-09-13T08:38:07.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.435+0000 7f950ec00640 1 -- 172.21.15.44:0/424688633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95080cb6a0 con 0x7f95080cf4e0 2024-09-13T08:38:07.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.439+0000 7f950ec00640 1 -- 172.21.15.44:0/424688633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95081444f0 con 0x7f95080cf4e0 2024-09-13T08:38:07.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.439+0000 7f9504c00640 1 -- 172.21.15.44:0/424688633 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f94f8005400 con 0x7f95080cf4e0 2024-09-13T08:38:07.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.439+0000 7f9504c00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e4072ec0 0x7f94e4075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:07.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.439+0000 7f9504c00640 1 -- 172.21.15.44:0/424688633 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f94f8094400 con 0x7f95080cf4e0 2024-09-13T08:38:07.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.439+0000 7f94ffe00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e4072ec0 0x7f94e4075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:07.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.439+0000 7f94ffe00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e4072ec0 0x7f94e4075380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94f00af000 tx=0x7f94f001afe0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:07.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.443+0000 7f9504c00640 1 -- 172.21.15.44:0/424688633 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f94f8014030 con 0x7f95080cf4e0 2024-09-13T08:38:07.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.655+0000 7f950ec00640 1 -- 172.21.15.44:0/424688633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95080c5dd0 con 0x7f95080cf4e0 2024-09-13T08:38:07.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.655+0000 7f9504c00640 1 -- 172.21.15.44:0/424688633 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f94f805ef60 con 0x7f95080cf4e0 2024-09-13T08:38:07.660 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:07.660 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"btime":"2024-09-13T08:37:37:896662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2931832597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2931832597},{"type":"v1","addr":"172.21.15.129:6837","nonce":2931832597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4335,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/853789824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":853789824},{"type":"v1","addr":"172.21.15.44:6834","nonce":853789824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/686971411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":686971411},{"type":"v1","addr":"172.21.15.44:6835","nonce":686971411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:37:34.290379+0000","modified":"2024-09-13T08:37:37.298049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4360":{"gid":4360,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6835/1614710769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1614710769},{"type":"v1","addr":"172.21.15.129:6835","nonce":1614710769}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4360,"qdb_cluster":[4360]},"id":1}]} 2024-09-13T08:38:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-09-13T08:38:07.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 -- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e4072ec0 msgr2=0x7f94e4075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:07.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e4072ec0 0x7f94e4075380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94f00af000 tx=0x7f94f001afe0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 -- 172.21.15.44:0/424688633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95080cf4e0 msgr2=0x7f95080ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:07.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95080cf4e0 0x7f95080ca860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f94f800c970 tx=0x7f94f800ce30 comp rx=0 tx=0).stop 2024-09-13T08:38:07.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 -- 172.21.15.44:0/424688633 shutdown_connections 2024-09-13T08:38:07.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e4072ec0 0x7f94e4075380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950815c130 0x7f95081587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950815bd50 0x7f9508158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 --2- 172.21.15.44:0/424688633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95080cf4e0 0x7f95080ca860 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 -- 172.21.15.44:0/424688633 >> 172.21.15.44:0/424688633 conn(0x7f95080d1ea0 msgr2=0x7f95080b6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:07.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 -- 172.21.15.44:0/424688633 shutdown_connections 2024-09-13T08:38:07.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.659+0000 7f950ec00640 1 -- 172.21.15.44:0/424688633 wait complete. 2024-09-13T08:38:07.682 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:38:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.823+0000 7f5019000640 1 -- 172.21.15.44:0/996152821 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50140cdb30 msgr2=0x7f50140cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.823+0000 7f5019000640 1 --2- 172.21.15.44:0/996152821 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50140cdb30 0x7f50140cebb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009900 tx=0x7f4ffc01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.823+0000 7f5019000640 1 -- 172.21.15.44:0/996152821 shutdown_connections 2024-09-13T08:38:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.823+0000 7f5019000640 1 --2- 172.21.15.44:0/996152821 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50140cdb30 0x7f50140cebb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.823+0000 7f5019000640 1 --2- 172.21.15.44:0/996152821 >> v2:172.21.15.129:3300/0 conn(0x7f501415bea0 0x7f50140ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.823+0000 7f5019000640 1 -- 172.21.15.44:0/996152821 >> 172.21.15.44:0/996152821 conn(0x7f50140d1dc0 msgr2=0x7f50140d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.823+0000 7f5019000640 1 -- 172.21.15.44:0/996152821 shutdown_connections 2024-09-13T08:38:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.823+0000 7f5019000640 1 -- 172.21.15.44:0/996152821 wait complete. 2024-09-13T08:38:07.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5019000640 1 Processor -- start 2024-09-13T08:38:07.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5019000640 1 -- start start 2024-09-13T08:38:07.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5019000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140cdb30 0x7f50141f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:07.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5019000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50140d28e0 0x7f501414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:07.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5019000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f501415bea0 0x7f50140cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:07.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5019000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5014168f10 con 0x7f501415bea0 2024-09-13T08:38:07.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5019000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5014168d90 con 0x7f50140cdb30 2024-09-13T08:38:07.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5019000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5014169090 con 0x7f50140d28e0 2024-09-13T08:38:07.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5012800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f501415bea0 0x7f50140cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:07.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5012800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f501415bea0 0x7f50140cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56054/0 (socket says 172.21.15.44:56054) 2024-09-13T08:38:07.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5012800640 1 -- 172.21.15.44:0/2100150976 learned_addr learned my addr 172.21.15.44:0/2100150976 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:07.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f500be00640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50140d28e0 0x7f501414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:07.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5011e00640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140cdb30 0x7f50141f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:07.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5012800640 1 -- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50140d28e0 msgr2=0x7f501414a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:07.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5012800640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50140d28e0 0x7f501414a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5012800640 1 -- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140cdb30 msgr2=0x7f50141f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:07.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5012800640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140cdb30 0x7f50141f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:07.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5012800640 1 -- 172.21.15.44:0/2100150976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50140b2b80 con 0x7f501415bea0 2024-09-13T08:38:07.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5012800640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f501415bea0 0x7f50140cba50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f500c00c7f0 tx=0x7f500c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:07.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f500a000640 1 -- 172.21.15.44:0/2100150976 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c003b80 con 0x7f501415bea0 2024-09-13T08:38:07.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f500a000640 1 -- 172.21.15.44:0/2100150976 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f500c003d20 con 0x7f501415bea0 2024-09-13T08:38:07.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5019000640 1 -- 172.21.15.44:0/2100150976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50140b2e60 con 0x7f501415bea0 2024-09-13T08:38:07.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f500a000640 1 -- 172.21.15.44:0/2100150976 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c00aab0 con 0x7f501415bea0 2024-09-13T08:38:07.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.827+0000 7f5019000640 1 -- 172.21.15.44:0/2100150976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50140cbf90 con 0x7f501415bea0 2024-09-13T08:38:07.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.831+0000 7f5019000640 1 -- 172.21.15.44:0/2100150976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50140cc9d0 con 0x7f501415bea0 2024-09-13T08:38:07.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.831+0000 7f500a000640 1 -- 172.21.15.44:0/2100150976 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f500c00ac50 con 0x7f501415bea0 2024-09-13T08:38:07.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.831+0000 7f500a000640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ff0072ec0 0x7f4ff0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:07.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.831+0000 7f500a000640 1 -- 172.21.15.44:0/2100150976 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f500c093c30 con 0x7f501415bea0 2024-09-13T08:38:07.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.831+0000 7f5011e00640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ff0072ec0 0x7f4ff0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.835+0000 7f5011e00640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ff0072ec0 0x7f4ff0075380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5004009b40 tx=0x7f50040091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:07.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:07.835+0000 7f500a000640 1 -- 172.21.15.44:0/2100150976 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f500c05e700 con 0x7f501415bea0 2024-09-13T08:38:08.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.047+0000 7f5019000640 1 -- 172.21.15.44:0/2100150976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50140b3030 con 0x7f501415bea0 2024-09-13T08:38:08.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.047+0000 7f500a000640 1 -- 172.21.15.44:0/2100150976 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f500c063130 con 0x7f501415bea0 2024-09-13T08:38:08.053 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:08.054 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"btime":"2024-09-13T08:37:37:896662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2931832597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2931832597},{"type":"v1","addr":"172.21.15.129:6837","nonce":2931832597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4335,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/853789824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":853789824},{"type":"v1","addr":"172.21.15.44:6834","nonce":853789824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/686971411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":686971411},{"type":"v1","addr":"172.21.15.44:6835","nonce":686971411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:37:34.290379+0000","modified":"2024-09-13T08:37:37.298049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4360":{"gid":4360,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6835/1614710769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1614710769},{"type":"v1","addr":"172.21.15.129:6835","nonce":1614710769}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4360,"qdb_cluster":[4360]},"id":1}]} 2024-09-13T08:38:08.054 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-09-13T08:38:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 -- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ff0072ec0 msgr2=0x7f4ff0075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ff0072ec0 0x7f4ff0075380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5004009b40 tx=0x7f50040091e0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 -- 172.21.15.44:0/2100150976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f501415bea0 msgr2=0x7f50140cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f501415bea0 0x7f50140cba50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f500c00c7f0 tx=0x7f500c00ccb0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 -- 172.21.15.44:0/2100150976 shutdown_connections 2024-09-13T08:38:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ff0072ec0 0x7f4ff0075380 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f501415bea0 0x7f50140cba50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50140d28e0 0x7f501414a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 --2- 172.21.15.44:0/2100150976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140cdb30 0x7f50141f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 -- 172.21.15.44:0/2100150976 >> 172.21.15.44:0/2100150976 conn(0x7f50140d1dc0 msgr2=0x7f50140d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:08.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 -- 172.21.15.44:0/2100150976 shutdown_connections 2024-09-13T08:38:08.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.051+0000 7f5019000640 1 -- 172.21.15.44:0/2100150976 wait complete. 2024-09-13T08:38:08.075 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:38:08.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.219+0000 7fc01ce00640 1 -- 172.21.15.44:0/528339552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbffc003c70 msgr2=0x7fbffc004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.219+0000 7fc01ce00640 1 --2- 172.21.15.44:0/528339552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbffc003c70 0x7fbffc004070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc008009b60 tx=0x7fc00801d240 comp rx=0 tx=0).stop 2024-09-13T08:38:08.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.219+0000 7fc016800640 1 -- 172.21.15.44:0/528339552 reap_dead start 2024-09-13T08:38:08.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.219+0000 7fc01ce00640 1 -- 172.21.15.44:0/528339552 shutdown_connections 2024-09-13T08:38:08.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.219+0000 7fc01ce00640 1 -- 172.21.15.44:0/528339552 >> 172.21.15.44:0/528339552 conn(0x7fc0180d0c90 msgr2=0x7fc0180d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:08.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.219+0000 7fc01ce00640 1 -- 172.21.15.44:0/528339552 shutdown_connections 2024-09-13T08:38:08.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.219+0000 7fc01ce00640 1 -- 172.21.15.44:0/528339552 wait complete. 2024-09-13T08:38:08.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.219+0000 7fc01ce00640 1 Processor -- start 2024-09-13T08:38:08.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.219+0000 7fc01ce00640 1 -- start start 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.219+0000 7fc01ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbffc003c70 0x7fc0180712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc01ce00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0180c73a0 0x7fc0180c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc01ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0181560b0 0x7fc0180b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc01ce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc018166b40 con 0x7fc0181560b0 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc01ce00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc018070b70 con 0x7fbffc003c70 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc01ce00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc018166cc0 con 0x7fc0180c73a0 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc016800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0181560b0 0x7fc0180b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc016800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0181560b0 0x7fc0180b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56060/0 (socket says 172.21.15.44:56060) 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc016800640 1 -- 172.21.15.44:0/3520397658 learned_addr learned my addr 172.21.15.44:0/3520397658 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc015e00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbffc003c70 0x7fc0180712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc016800640 1 -- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0180c73a0 msgr2=0x7fc0180c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc00fe00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0180c73a0 0x7fc0180c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc016800640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0180c73a0 0x7fc0180c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc016800640 1 -- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbffc003c70 msgr2=0x7fc0180712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc016800640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbffc003c70 0x7fc0180712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc016800640 1 -- 172.21.15.44:0/3520397658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0180c7910 con 0x7fc0181560b0 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc015e00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbffc003c70 0x7fc0180712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc00fe00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0180c73a0 0x7fc0180c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc016800640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0181560b0 0x7fc0180b5c90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc018074710 tx=0x7fc010001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc00e000640 1 -- 172.21.15.44:0/3520397658 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010010890 con 0x7fc0181560b0 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc00e000640 1 -- 172.21.15.44:0/3520397658 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc010010a70 con 0x7fc0181560b0 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc00e000640 1 -- 172.21.15.44:0/3520397658 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010011460 con 0x7fc0181560b0 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc01ce00640 1 -- 172.21.15.44:0/3520397658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0180c5d60 con 0x7fc0181560b0 2024-09-13T08:38:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc01ce00640 1 -- 172.21.15.44:0/3520397658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0180c6320 con 0x7fc0181560b0 2024-09-13T08:38:08.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc01ce00640 1 -- 172.21.15.44:0/3520397658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc018073c20 con 0x7fc0181560b0 2024-09-13T08:38:08.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.223+0000 7fc00e000640 1 -- 172.21.15.44:0/3520397658 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fc01001a400 con 0x7fc0181560b0 2024-09-13T08:38:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.227+0000 7fc00e000640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbff4072ec0 0x7fbff4075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.227+0000 7fc00e000640 1 -- 172.21.15.44:0/3520397658 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc0100944e0 con 0x7fc0181560b0 2024-09-13T08:38:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.227+0000 7fc015e00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbff4072ec0 0x7fbff4075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.227+0000 7fc015e00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbff4072ec0 0x7fbff4075380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0080ae000 tx=0x7fc008019fe0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:08.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.231+0000 7fc00e000640 1 -- 172.21.15.44:0/3520397658 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc01005f720 con 0x7fc0181560b0 2024-09-13T08:38:08.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.403+0000 7fc01ce00640 1 -- 172.21.15.44:0/3520397658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc018165b00 con 0x7fc0181560b0 2024-09-13T08:38:08.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc00e000640 1 -- 172.21.15.44:0/3520397658 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16681 (secure 0 0 0) 0x7fc01005f020 con 0x7fc0181560b0 2024-09-13T08:38:08.410 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:08.410 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":22,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:35.883263+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:37:32.247896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T08:37:33.241202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:38:08.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 -- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbff4072ec0 msgr2=0x7fbff4075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbff4072ec0 0x7fbff4075380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0080ae000 tx=0x7fc008019fe0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 -- 172.21.15.44:0/3520397658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0181560b0 msgr2=0x7fc0180b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0181560b0 0x7fc0180b5c90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc018074710 tx=0x7fc010001b90 comp rx=0 tx=0).stop 2024-09-13T08:38:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 -- 172.21.15.44:0/3520397658 shutdown_connections 2024-09-13T08:38:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbff4072ec0 0x7fbff4075380 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0181560b0 0x7fc0180b5c90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0180c73a0 0x7fc0180c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 --2- 172.21.15.44:0/3520397658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbffc003c70 0x7fc0180712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 -- 172.21.15.44:0/3520397658 >> 172.21.15.44:0/3520397658 conn(0x7fc0180d0c90 msgr2=0x7fc0180c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 -- 172.21.15.44:0/3520397658 shutdown_connections 2024-09-13T08:38:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.407+0000 7fc01ce00640 1 -- 172.21.15.44:0/3520397658 wait complete. 2024-09-13T08:38:08.431 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:38:08.432 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:38:08.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 -- 172.21.15.44:0/58238429 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5aec0cefb0 msgr2=0x7f5aec0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 --2- 172.21.15.44:0/58238429 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5aec0cefb0 0x7f5aec0cc5b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009840 tx=0x7f5ad401c790 comp rx=0 tx=0).stop 2024-09-13T08:38:08.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 -- 172.21.15.44:0/58238429 shutdown_connections 2024-09-13T08:38:08.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 --2- 172.21.15.44:0/58238429 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5aec0cefb0 0x7f5aec0cc5b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 --2- 172.21.15.44:0/58238429 >> v2:172.21.15.129:3300/0 conn(0x7f5aec15f000 0x7f5aec0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 -- 172.21.15.44:0/58238429 >> 172.21.15.44:0/58238429 conn(0x7f5aec09aa10 msgr2=0x7f5aec09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:08.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 -- 172.21.15.44:0/58238429 shutdown_connections 2024-09-13T08:38:08.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 -- 172.21.15.44:0/58238429 wait complete. 2024-09-13T08:38:08.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 Processor -- start 2024-09-13T08:38:08.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 -- start start 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5aec0ccaf0 0x7f5aec14f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aec0cefb0 0x7f5aec0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5aec15f000 0x7f5aec0cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec169270 con 0x7f5aec0cefb0 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec1690f0 con 0x7f5aec0ccaf0 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5aec1693f0 con 0x7f5aec15f000 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5ae3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aec0cefb0 0x7f5aec0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5ae3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aec0cefb0 0x7f5aec0ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56074/0 (socket says 172.21.15.44:56074) 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5ae3400640 1 -- 172.21.15.44:0/805404374 learned_addr learned my addr 172.21.15.44:0/805404374 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5aebe00640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5aec0ccaf0 0x7f5aec14f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.575+0000 7f5af1200640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5aec15f000 0x7f5aec0cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5ae3400640 1 -- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5aec15f000 msgr2=0x7f5aec0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5ae3400640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5aec15f000 0x7f5aec0cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5ae3400640 1 -- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5aec0ccaf0 msgr2=0x7f5aec14f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5ae3400640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5aec0ccaf0 0x7f5aec14f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5ae3400640 1 -- 172.21.15.44:0/805404374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aec150170 con 0x7f5aec0cefb0 2024-09-13T08:38:08.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5ae3400640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aec0cefb0 0x7f5aec0ccf90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4000c10 tx=0x7f5ad40a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:08.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5ae8c00640 1 -- 172.21.15.44:0/805404374 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40ac070 con 0x7f5aec0cefb0 2024-09-13T08:38:08.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5ae8c00640 1 -- 172.21.15.44:0/805404374 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ad4004310 con 0x7f5aec0cefb0 2024-09-13T08:38:08.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5ae8c00640 1 -- 172.21.15.44:0/805404374 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4005320 con 0x7f5aec0cefb0 2024-09-13T08:38:08.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5af1c00640 1 -- 172.21.15.44:0/805404374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aec150450 con 0x7f5aec0cefb0 2024-09-13T08:38:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5af1c00640 1 -- 172.21.15.44:0/805404374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aec150a10 con 0x7f5aec0cefb0 2024-09-13T08:38:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5af1c00640 1 -- 172.21.15.44:0/805404374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aec0cc5b0 con 0x7f5aec0cefb0 2024-09-13T08:38:08.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.579+0000 7f5ae8c00640 1 -- 172.21.15.44:0/805404374 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f5ad40af070 con 0x7f5aec0cefb0 2024-09-13T08:38:08.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.583+0000 7f5ae8c00640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ac8072ef0 0x7f5ac80753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.583+0000 7f5ae8c00640 1 -- 172.21.15.44:0/805404374 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5ad412e240 con 0x7f5aec0cefb0 2024-09-13T08:38:08.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.583+0000 7f5aebe00640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ac8072ef0 0x7f5ac80753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.583+0000 7f5aebe00640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ac8072ef0 0x7f5ac80753b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5aec1f8c40 tx=0x7f5adc0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:08.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.587+0000 7f5ae8c00640 1 -- 172.21.15.44:0/805404374 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5ad40b6070 con 0x7f5aec0cefb0 2024-09-13T08:38:08.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.799+0000 7f5af1c00640 1 -- 172.21.15.44:0/805404374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5aec076210 con 0x7f5aec0cefb0 2024-09-13T08:38:08.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.799+0000 7f5ae8c00640 1 -- 172.21.15.44:0/805404374 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f5ad4016410 con 0x7f5aec0cefb0 2024-09-13T08:38:08.804 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:08.804 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"btime":"2024-09-13T08:37:37:896662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2931832597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2931832597},{"type":"v1","addr":"172.21.15.129:6837","nonce":2931832597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4335,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/853789824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":853789824},{"type":"v1","addr":"172.21.15.44:6834","nonce":853789824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/686971411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":686971411},{"type":"v1","addr":"172.21.15.44:6835","nonce":686971411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:37:34.290379+0000","modified":"2024-09-13T08:37:37.298049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4360":{"gid":4360,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6835/1614710769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1614710769},{"type":"v1","addr":"172.21.15.129:6835","nonce":1614710769}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4360,"qdb_cluster":[4360]},"id":1}]} 2024-09-13T08:38:08.805 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-09-13T08:38:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 -- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ac8072ef0 msgr2=0x7f5ac80753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ac8072ef0 0x7f5ac80753b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5aec1f8c40 tx=0x7f5adc0091e0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 -- 172.21.15.44:0/805404374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aec0cefb0 msgr2=0x7f5aec0ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aec0cefb0 0x7f5aec0ccf90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4000c10 tx=0x7f5ad40a64e0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 -- 172.21.15.44:0/805404374 shutdown_connections 2024-09-13T08:38:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ac8072ef0 0x7f5ac80753b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5aec15f000 0x7f5aec0cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aec0cefb0 0x7f5aec0ccf90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 --2- 172.21.15.44:0/805404374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5aec0ccaf0 0x7f5aec14f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 -- 172.21.15.44:0/805404374 >> 172.21.15.44:0/805404374 conn(0x7f5aec09aa10 msgr2=0x7f5aec15bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:08.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 -- 172.21.15.44:0/805404374 shutdown_connections 2024-09-13T08:38:08.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.803+0000 7f5af1c00640 1 -- 172.21.15.44:0/805404374 wait complete. 2024-09-13T08:38:08.826 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:38:08.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.967+0000 7f0b73200640 1 -- 172.21.15.44:0/2217348186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c06f9a0 msgr2=0x7f0b6c06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.967+0000 7f0b73200640 1 --2- 172.21.15.44:0/2217348186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c06f9a0 0x7f0b6c06fe00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009bc0 tx=0x7f0b6401cb70 comp rx=0 tx=0).stop 2024-09-13T08:38:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.967+0000 7f0b73200640 1 -- 172.21.15.44:0/2217348186 shutdown_connections 2024-09-13T08:38:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.967+0000 7f0b73200640 1 --2- 172.21.15.44:0/2217348186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c06f9a0 0x7f0b6c06fe00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.967+0000 7f0b73200640 1 --2- 172.21.15.44:0/2217348186 >> v2:172.21.15.129:3300/0 conn(0x7f0b6c0cffc0 0x7f0b6c073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.967+0000 7f0b73200640 1 -- 172.21.15.44:0/2217348186 >> 172.21.15.44:0/2217348186 conn(0x7f0b6c0d0c90 msgr2=0x7f0b6c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.967+0000 7f0b73200640 1 -- 172.21.15.44:0/2217348186 shutdown_connections 2024-09-13T08:38:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.967+0000 7f0b73200640 1 -- 172.21.15.44:0/2217348186 wait complete. 2024-09-13T08:38:08.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b73200640 1 Processor -- start 2024-09-13T08:38:08.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b73200640 1 -- start start 2024-09-13T08:38:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b73200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c06f9a0 0x7f0b6c1fa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b73200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0cffc0 0x7f0b6c0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b73200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15af60 0x7f0b6c0a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b73200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c167cc0 con 0x7f0b6c0cffc0 2024-09-13T08:38:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b73200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c167b40 con 0x7f0b6c15af60 2024-09-13T08:38:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b73200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c167e40 con 0x7f0b6c06f9a0 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0cffc0 0x7f0b6c0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0cffc0 0x7f0b6c0b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56078/0 (socket says 172.21.15.44:56078) 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b63e00640 1 -- 172.21.15.44:0/2125776499 learned_addr learned my addr 172.21.15.44:0/2125776499 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b71200640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15af60 0x7f0b6c0a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b6be00640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c06f9a0 0x7f0b6c1fa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b63e00640 1 -- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c06f9a0 msgr2=0x7f0b6c1fa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b63e00640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c06f9a0 0x7f0b6c1fa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b63e00640 1 -- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15af60 msgr2=0x7f0b6c0a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b63e00640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15af60 0x7f0b6c0a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b63e00640 1 -- 172.21.15.44:0/2125776499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c0a1630 con 0x7f0b6c0cffc0 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b71200640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15af60 0x7f0b6c0a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b63e00640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0cffc0 0x7f0b6c0b06f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400e9f0 tx=0x7f0b5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b68c00640 1 -- 172.21.15.44:0/2125776499 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5400cd20 con 0x7f0b6c0cffc0 2024-09-13T08:38:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b68c00640 1 -- 172.21.15.44:0/2125776499 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b54010430 con 0x7f0b6c0cffc0 2024-09-13T08:38:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b73200640 1 -- 172.21.15.44:0/2125776499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c0a18d0 con 0x7f0b6c0cffc0 2024-09-13T08:38:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b68c00640 1 -- 172.21.15.44:0/2125776499 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b54010860 con 0x7f0b6c0cffc0 2024-09-13T08:38:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.971+0000 7f0b73200640 1 -- 172.21.15.44:0/2125776499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c0a01b0 con 0x7f0b6c0cffc0 2024-09-13T08:38:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.975+0000 7f0b73200640 1 -- 172.21.15.44:0/2125776499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6c06fe00 con 0x7f0b6c0cffc0 2024-09-13T08:38:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.975+0000 7f0b68c00640 1 -- 172.21.15.44:0/2125776499 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f0b54010a00 con 0x7f0b6c0cffc0 2024-09-13T08:38:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.975+0000 7f0b68c00640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48072ec0 0x7f0b48075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.975+0000 7f0b68c00640 1 -- 172.21.15.44:0/2125776499 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0b540949d0 con 0x7f0b6c0cffc0 2024-09-13T08:38:08.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.975+0000 7f0b6be00640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48072ec0 0x7f0b48075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:08.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.979+0000 7f0b6be00640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48072ec0 0x7f0b48075380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c001640 tx=0x7f0b5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:08.979+0000 7f0b68c00640 1 -- 172.21.15.44:0/2125776499 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0b5405fc40 con 0x7f0b6c0cffc0 2024-09-13T08:38:09.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.195+0000 7f0b73200640 1 -- 172.21.15.44:0/2125776499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b6c0b0c30 con 0x7f0b6c0cffc0 2024-09-13T08:38:09.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.195+0000 7f0b68c00640 1 -- 172.21.15.44:0/2125776499 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f0b54066070 con 0x7f0b6c0cffc0 2024-09-13T08:38:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:09.200 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"btime":"2024-09-13T08:37:37:896662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2931832597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2931832597},{"type":"v1","addr":"172.21.15.129:6837","nonce":2931832597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4335,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/853789824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":853789824},{"type":"v1","addr":"172.21.15.44:6834","nonce":853789824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/686971411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":686971411},{"type":"v1","addr":"172.21.15.44:6835","nonce":686971411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:37:34.290379+0000","modified":"2024-09-13T08:37:37.298049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4360":{"gid":4360,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6835/1614710769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1614710769},{"type":"v1","addr":"172.21.15.129:6835","nonce":1614710769}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4360,"qdb_cluster":[4360]},"id":1}]} 2024-09-13T08:38:09.201 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-09-13T08:38:09.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 -- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48072ec0 msgr2=0x7f0b48075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48072ec0 0x7f0b48075380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c001640 tx=0x7f0b5c009040 comp rx=0 tx=0).stop 2024-09-13T08:38:09.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 -- 172.21.15.44:0/2125776499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0cffc0 msgr2=0x7f0b6c0b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0cffc0 0x7f0b6c0b06f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400e9f0 tx=0x7f0b5400eeb0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 -- 172.21.15.44:0/2125776499 shutdown_connections 2024-09-13T08:38:09.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48072ec0 0x7f0b48075380 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15af60 0x7f0b6c0a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0cffc0 0x7f0b6c0b06f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 --2- 172.21.15.44:0/2125776499 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c06f9a0 0x7f0b6c1fa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 -- 172.21.15.44:0/2125776499 >> 172.21.15.44:0/2125776499 conn(0x7f0b6c0d0c90 msgr2=0x7f0b6c1577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:09.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 -- 172.21.15.44:0/2125776499 shutdown_connections 2024-09-13T08:38:09.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.199+0000 7f0b73200640 1 -- 172.21.15.44:0/2125776499 wait complete. 2024-09-13T08:38:09.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:38:09.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.363+0000 7f88e0600640 1 -- 172.21.15.44:0/725236237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d80d28e0 msgr2=0x7f88d80cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.363+0000 7f88e0600640 1 --2- 172.21.15.44:0/725236237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d80d28e0 0x7f88d80cc9d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f88d80745d0 tx=0x7f88d001cd80 comp rx=0 tx=0).stop 2024-09-13T08:38:09.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.363+0000 7f88e0600640 1 -- 172.21.15.44:0/725236237 shutdown_connections 2024-09-13T08:38:09.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.363+0000 7f88e0600640 1 --2- 172.21.15.44:0/725236237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d80d28e0 0x7f88d80cc9d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.363+0000 7f88e0600640 1 --2- 172.21.15.44:0/725236237 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d80cdb30 0x7f88d80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.363+0000 7f88e0600640 1 --2- 172.21.15.44:0/725236237 >> v2:172.21.15.129:3300/0 conn(0x7f88d815bea0 0x7f88d80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.363+0000 7f88e0600640 1 -- 172.21.15.44:0/725236237 >> 172.21.15.44:0/725236237 conn(0x7f88d80d1dc0 msgr2=0x7f88d80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:09.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.363+0000 7f88e0600640 1 -- 172.21.15.44:0/725236237 shutdown_connections 2024-09-13T08:38:09.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.363+0000 7f88e0600640 1 -- 172.21.15.44:0/725236237 wait complete. 2024-09-13T08:38:09.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 Processor -- start 2024-09-13T08:38:09.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 -- start start 2024-09-13T08:38:09.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d80cdb30 0x7f88d81662b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:09.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d80d28e0 0x7f88d8071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d815bea0 0x7f88d8154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f88d80af120 con 0x7f88d80cdb30 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88d80aefa0 con 0x7f88d815bea0 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f88d80af2a0 con 0x7f88d80d28e0 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88ddc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d80cdb30 0x7f88d81662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88ddc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d80cdb30 0x7f88d81662b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56096/0 (socket says 172.21.15.44:56096) 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88ddc00640 1 -- 172.21.15.44:0/556375462 learned_addr learned my addr 172.21.15.44:0/556375462 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88dd200640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d80d28e0 0x7f88d8071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88de600640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d815bea0 0x7f88d8154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88ddc00640 1 -- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d80d28e0 msgr2=0x7f88d8071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88ddc00640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d80d28e0 0x7f88d8071fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88ddc00640 1 -- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d815bea0 msgr2=0x7f88d8154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88ddc00640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d815bea0 0x7f88d8154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88ddc00640 1 -- 172.21.15.44:0/556375462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d81faf30 con 0x7f88d80cdb30 2024-09-13T08:38:09.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88dd200640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d80d28e0 0x7f88d8071fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:09.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88ddc00640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d80cdb30 0x7f88d81662b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00d950 tx=0x7f88cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:09.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88d5600640 1 -- 172.21.15.44:0/556375462 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc017070 con 0x7f88d80cdb30 2024-09-13T08:38:09.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88d5600640 1 -- 172.21.15.44:0/556375462 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88cc009d20 con 0x7f88d80cdb30 2024-09-13T08:38:09.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 -- 172.21.15.44:0/556375462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d81fb210 con 0x7f88d80cdb30 2024-09-13T08:38:09.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88d5600640 1 -- 172.21.15.44:0/556375462 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0106c0 con 0x7f88d80cdb30 2024-09-13T08:38:09.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 -- 172.21.15.44:0/556375462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d81fb7d0 con 0x7f88d80cdb30 2024-09-13T08:38:09.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.367+0000 7f88e0600640 1 -- 172.21.15.44:0/556375462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d80af830 con 0x7f88d80cdb30 2024-09-13T08:38:09.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.371+0000 7f88d5600640 1 -- 172.21.15.44:0/556375462 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f88cc01a070 con 0x7f88d80cdb30 2024-09-13T08:38:09.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.371+0000 7f88d5600640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f88b8072ec0 0x7f88b8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:09.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.371+0000 7f88d5600640 1 -- 172.21.15.44:0/556375462 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f88cc093ee0 con 0x7f88d80cdb30 2024-09-13T08:38:09.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.371+0000 7f88dd200640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f88b8072ec0 0x7f88b8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:09.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.371+0000 7f88dd200640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f88b8072ec0 0x7f88b8075380 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88c4005fd0 tx=0x7f88c4005f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:09.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.375+0000 7f88d5600640 1 -- 172.21.15.44:0/556375462 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f88cc05ead0 con 0x7f88d80cdb30 2024-09-13T08:38:09.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.547+0000 7f88e0600640 1 -- 172.21.15.44:0/556375462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88d80b6360 con 0x7f88d80cdb30 2024-09-13T08:38:09.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88d5600640 1 -- 172.21.15.44:0/556375462 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16681 (secure 0 0 0) 0x7f88cc063500 con 0x7f88d80cdb30 2024-09-13T08:38:09.555 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:09.555 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":22,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:37:35.883263+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:37:32.247896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T08:37:33.241202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 -- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f88b8072ec0 msgr2=0x7f88b8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f88b8072ec0 0x7f88b8075380 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88c4005fd0 tx=0x7f88c4005f00 comp rx=0 tx=0).stop 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 -- 172.21.15.44:0/556375462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d80cdb30 msgr2=0x7f88d81662b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d80cdb30 0x7f88d81662b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00d950 tx=0x7f88cc00de10 comp rx=0 tx=0).stop 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 -- 172.21.15.44:0/556375462 shutdown_connections 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f88b8072ec0 0x7f88b8075380 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d815bea0 0x7f88d8154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88d80d28e0 0x7f88d8071fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 --2- 172.21.15.44:0/556375462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d80cdb30 0x7f88d81662b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 -- 172.21.15.44:0/556375462 >> 172.21.15.44:0/556375462 conn(0x7f88d80d1dc0 msgr2=0x7f88d80b6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:09.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 -- 172.21.15.44:0/556375462 shutdown_connections 2024-09-13T08:38:09.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.551+0000 7f88e0600640 1 -- 172.21.15.44:0/556375462 wait complete. 2024-09-13T08:38:09.577 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T08:38:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.719+0000 7fa580400640 1 -- 172.21.15.44:0/2614227596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa57815e280 msgr2=0x7fa5780af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.719+0000 7fa580400640 1 --2- 172.21.15.44:0/2614227596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa57815e280 0x7fa5780af0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa564009840 tx=0x7fa56401c720 comp rx=0 tx=0).stop 2024-09-13T08:38:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.719+0000 7fa580400640 1 -- 172.21.15.44:0/2614227596 shutdown_connections 2024-09-13T08:38:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.719+0000 7fa580400640 1 --2- 172.21.15.44:0/2614227596 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5780b3150 0x7fa5780cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.719+0000 7fa580400640 1 --2- 172.21.15.44:0/2614227596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa57815e280 0x7fa5780af0a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.719+0000 7fa580400640 1 -- 172.21.15.44:0/2614227596 >> 172.21.15.44:0/2614227596 conn(0x7fa5780d1da0 msgr2=0x7fa5780d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.719+0000 7fa580400640 1 -- 172.21.15.44:0/2614227596 shutdown_connections 2024-09-13T08:38:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.719+0000 7fa580400640 1 -- 172.21.15.44:0/2614227596 wait complete. 2024-09-13T08:38:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa580400640 1 Processor -- start 2024-09-13T08:38:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa580400640 1 -- start start 2024-09-13T08:38:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa580400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5780b3150 0x7fa5781491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa580400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa57815dea0 0x7fa5781faba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa580400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa57815e280 0x7fa5780c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa580400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa57816ad70 con 0x7fa5780b3150 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa580400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa57816abf0 con 0x7fa57815dea0 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa580400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa57816aef0 con 0x7fa57815e280 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5780b3150 0x7fa5781491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5780b3150 0x7fa5781491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56112/0 (socket says 172.21.15.44:56112) 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57da00640 1 -- 172.21.15.44:0/2783284509 learned_addr learned my addr 172.21.15.44:0/2783284509 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57e400640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa57815e280 0x7fa5780c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57d000640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa57815dea0 0x7fa5781faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57da00640 1 -- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa57815e280 msgr2=0x7fa5780c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57da00640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa57815e280 0x7fa5780c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57da00640 1 -- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa57815dea0 msgr2=0x7fa5781faba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57da00640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa57815dea0 0x7fa5781faba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57da00640 1 -- 172.21.15.44:0/2783284509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5781496e0 con 0x7fa5780b3150 2024-09-13T08:38:09.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa57da00640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5780b3150 0x7fa5781491a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00c730 tx=0x7fa56c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:09.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa575600640 1 -- 172.21.15.44:0/2783284509 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c016020 con 0x7fa5780b3150 2024-09-13T08:38:09.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa575600640 1 -- 172.21.15.44:0/2783284509 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa56c00a8f0 con 0x7fa5780b3150 2024-09-13T08:38:09.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa580400640 1 -- 172.21.15.44:0/2783284509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5781fb3d0 con 0x7fa5780b3150 2024-09-13T08:38:09.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa575600640 1 -- 172.21.15.44:0/2783284509 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00ad40 con 0x7fa5780b3150 2024-09-13T08:38:09.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.723+0000 7fa580400640 1 -- 172.21.15.44:0/2783284509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5781fb990 con 0x7fa5780b3150 2024-09-13T08:38:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.727+0000 7fa580400640 1 -- 172.21.15.44:0/2783284509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5780cffe0 con 0x7fa5780b3150 2024-09-13T08:38:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.727+0000 7fa575600640 1 -- 172.21.15.44:0/2783284509 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fa56c019020 con 0x7fa5780b3150 2024-09-13T08:38:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.727+0000 7fa575600640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa558072ec0 0x7fa558075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.727+0000 7fa575600640 1 -- 172.21.15.44:0/2783284509 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa56c093f90 con 0x7fa5780b3150 2024-09-13T08:38:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.727+0000 7fa57d000640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa558072ec0 0x7fa558075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:09.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.731+0000 7fa57d000640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa558072ec0 0x7fa558075380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa5640ae000 tx=0x7fa5640aa040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:09.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.731+0000 7fa575600640 1 -- 172.21.15.44:0/2783284509 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa56c020020 con 0x7fa5780b3150 2024-09-13T08:38:09.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.943+0000 7fa580400640 1 -- 172.21.15.44:0/2783284509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa5780b4ff0 con 0x7fa5780b3150 2024-09-13T08:38:09.948 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T08:38:09.943+0000 7f03b3c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T08:38:09.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa575600640 1 -- 172.21.15.44:0/2783284509 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fa56c05f140 con 0x7fa5780b3150 2024-09-13T08:38:09.950 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T08:38:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 -- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa558072ec0 msgr2=0x7fa558075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa558072ec0 0x7fa558075380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa5640ae000 tx=0x7fa5640aa040 comp rx=0 tx=0).stop 2024-09-13T08:38:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 -- 172.21.15.44:0/2783284509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5780b3150 msgr2=0x7fa5781491a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5780b3150 0x7fa5781491a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00c730 tx=0x7fa56c00cbf0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 -- 172.21.15.44:0/2783284509 shutdown_connections 2024-09-13T08:38:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa558072ec0 0x7fa558075380 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa57815e280 0x7fa5780c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa57815dea0 0x7fa5781faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 --2- 172.21.15.44:0/2783284509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5780b3150 0x7fa5781491a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 -- 172.21.15.44:0/2783284509 >> 172.21.15.44:0/2783284509 conn(0x7fa5780d1da0 msgr2=0x7fa5780720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 -- 172.21.15.44:0/2783284509 shutdown_connections 2024-09-13T08:38:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:09.947+0000 7fa580400640 1 -- 172.21.15.44:0/2783284509 wait complete. 2024-09-13T08:38:09.967 INFO:tasks.ceph.mds.d.smithi129.stderr: -7062> 2024-09-13T08:37:39.842+0000 7ff048c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:38:09.973 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T08:38:10.020 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:38:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.119+0000 7f48efe00640 1 -- 172.21.15.44:0/1192439817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48e80cf710 msgr2=0x7f48e80ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.119+0000 7f48efe00640 1 --2- 172.21.15.44:0/1192439817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48e80cf710 0x7f48e80ce3b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009900 tx=0x7f48d401c850 comp rx=0 tx=0).stop 2024-09-13T08:38:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.119+0000 7f48efe00640 1 -- 172.21.15.44:0/1192439817 shutdown_connections 2024-09-13T08:38:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.119+0000 7f48efe00640 1 --2- 172.21.15.44:0/1192439817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48e80cf710 0x7f48e80ce3b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.119+0000 7f48efe00640 1 --2- 172.21.15.44:0/1192439817 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48e8151410 0x7f48e80cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:10.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.119+0000 7f48efe00640 1 -- 172.21.15.44:0/1192439817 >> 172.21.15.44:0/1192439817 conn(0x7f48e80cfc20 msgr2=0x7f48e80d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:10.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.119+0000 7f48efe00640 1 -- 172.21.15.44:0/1192439817 shutdown_connections 2024-09-13T08:38:10.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.119+0000 7f48efe00640 1 -- 172.21.15.44:0/1192439817 wait complete. 2024-09-13T08:38:10.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.119+0000 7f48efe00640 1 Processor -- start 2024-09-13T08:38:10.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.119+0000 7f48efe00640 1 -- start start 2024-09-13T08:38:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48efe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48e80ce8f0 0x7f48e81fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48efe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48e80cf710 0x7f48e8073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48e8151410 0x7f48e8073fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48efe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48e80ad500 con 0x7f48e8151410 2024-09-13T08:38:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48efe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48e80ad380 con 0x7f48e80cf710 2024-09-13T08:38:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48efe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f48e80ad680 con 0x7f48e80ce8f0 2024-09-13T08:38:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ede00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48e8151410 0x7f48e8073fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ed400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48e80ce8f0 0x7f48e81fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ed400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48e80ce8f0 0x7f48e81fbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:56894/0 (socket says 172.21.15.44:56894) 2024-09-13T08:38:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ed400640 1 -- 172.21.15.44:0/2839023990 learned_addr learned my addr 172.21.15.44:0/2839023990 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ede00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48e8151410 0x7f48e8073fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56132/0 (socket says 172.21.15.44:56132) 2024-09-13T08:38:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48eca00640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48e80cf710 0x7f48e8073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ede00640 1 -- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48e80ce8f0 msgr2=0x7f48e81fbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ede00640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48e80ce8f0 0x7f48e81fbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ede00640 1 -- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48e80cf710 msgr2=0x7f48e8073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ede00640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48e80cf710 0x7f48e8073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ede00640 1 -- 172.21.15.44:0/2839023990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e81525a0 con 0x7f48e8151410 2024-09-13T08:38:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48ede00640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48e8151410 0x7f48e8073fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f48e000c730 tx=0x7f48e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48e5600640 1 -- 172.21.15.44:0/2839023990 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0003b80 con 0x7f48e8151410 2024-09-13T08:38:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48e5600640 1 -- 172.21.15.44:0/2839023990 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48e0003d20 con 0x7f48e8151410 2024-09-13T08:38:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48efe00640 1 -- 172.21.15.44:0/2839023990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e8152880 con 0x7f48e8151410 2024-09-13T08:38:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48e5600640 1 -- 172.21.15.44:0/2839023990 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0005330 con 0x7f48e8151410 2024-09-13T08:38:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48efe00640 1 -- 172.21.15.44:0/2839023990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e8152e40 con 0x7f48e8151410 2024-09-13T08:38:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.123+0000 7f48efe00640 1 -- 172.21.15.44:0/2839023990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e80ce3b0 con 0x7f48e8151410 2024-09-13T08:38:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.127+0000 7f48e5600640 1 -- 172.21.15.44:0/2839023990 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f48e00056e0 con 0x7f48e8151410 2024-09-13T08:38:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.127+0000 7f48e5600640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48c8072ec0 0x7f48c8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.127+0000 7f48e5600640 1 -- 172.21.15.44:0/2839023990 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7f48e0093d00 con 0x7f48e8151410 2024-09-13T08:38:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.127+0000 7f48ed400640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48c8072ec0 0x7f48c8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.127+0000 7f48ed400640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48c8072ec0 0x7f48c8075380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00ac70 tx=0x7f48dc010040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.131+0000 7f48e5600640 1 -- 172.21.15.44:0/2839023990 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f48e005e800 con 0x7f48e8151410 2024-09-13T08:38:10.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:10.343+0000 7f48efe00640 1 -- 172.21.15.44:0/2839023990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f48e80ae230 con 0x7f48e8151410 2024-09-13T08:38:11.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48e5600640 1 -- 172.21.15.44:0/2839023990 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f48e0063230 con 0x7f48e8151410 2024-09-13T08:38:11.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 -- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48c8072ec0 msgr2=0x7f48c8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:11.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48c8072ec0 0x7f48c8075380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00ac70 tx=0x7f48dc010040 comp rx=0 tx=0).stop 2024-09-13T08:38:11.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 -- 172.21.15.44:0/2839023990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48e8151410 msgr2=0x7f48e8073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:11.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48e8151410 0x7f48e8073fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f48e000c730 tx=0x7f48e000cbf0 comp rx=0 tx=0).stop 2024-09-13T08:38:11.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 -- 172.21.15.44:0/2839023990 shutdown_connections 2024-09-13T08:38:11.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48c8072ec0 0x7f48c8075380 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:11.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48e8151410 0x7f48e8073fe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:11.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48e80cf710 0x7f48e8073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:11.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 --2- 172.21.15.44:0/2839023990 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48e80ce8f0 0x7f48e81fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:11.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 -- 172.21.15.44:0/2839023990 >> 172.21.15.44:0/2839023990 conn(0x7f48e80cfc20 msgr2=0x7f48e80ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:11.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 -- 172.21.15.44:0/2839023990 shutdown_connections 2024-09-13T08:38:11.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.019+0000 7f48efe00640 1 -- 172.21.15.44:0/2839023990 wait complete. 2024-09-13T08:38:11.049 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T08:38:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.199+0000 7f6eb3e00640 1 -- 172.21.15.44:0/285752983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e98003ab0 msgr2=0x7f6e98003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.199+0000 7f6eb3e00640 1 --2- 172.21.15.44:0/285752983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e98003ab0 0x7f6e98003ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400ac70 tx=0x7f6ea401e090 comp rx=0 tx=0).stop 2024-09-13T08:38:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.199+0000 7f6eb3e00640 1 -- 172.21.15.44:0/285752983 shutdown_connections 2024-09-13T08:38:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.199+0000 7f6eb3e00640 1 --2- 172.21.15.44:0/285752983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e98003ab0 0x7f6e98003ed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.199+0000 7f6eb3e00640 1 -- 172.21.15.44:0/285752983 >> 172.21.15.44:0/285752983 conn(0x7f6eb40cfa80 msgr2=0x7f6eb40cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:11.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.199+0000 7f6eb3e00640 1 -- 172.21.15.44:0/285752983 shutdown_connections 2024-09-13T08:38:11.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.199+0000 7f6eb3e00640 1 -- 172.21.15.44:0/285752983 wait complete. 2024-09-13T08:38:11.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3e00640 1 Processor -- start 2024-09-13T08:38:11.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3e00640 1 -- start start 2024-09-13T08:38:11.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e98003ab0 0x7f6eb41fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:11.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6eb4150b60 0x7f6eb41f9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41f96d0 0x7f6eb40c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb406f230 con 0x7f6eb41f96d0 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb406f0b0 con 0x7f6e98003ab0 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb40cd650 con 0x7f6eb4150b60 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41f96d0 0x7f6eb40c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41f96d0 0x7f6eb40c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56148/0 (socket says 172.21.15.44:56148) 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3400640 1 -- 172.21.15.44:0/2693534884 learned_addr learned my addr 172.21.15.44:0/2693534884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb2a00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e98003ab0 0x7f6eb41fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3400640 1 -- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6eb4150b60 msgr2=0x7f6eb41f9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb2000640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6eb4150b60 0x7f6eb41f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3400640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6eb4150b60 0x7f6eb41f9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3400640 1 -- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e98003ab0 msgr2=0x7f6eb41fbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3400640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e98003ab0 0x7f6eb41fbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3400640 1 -- 172.21.15.44:0/2693534884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb41fc1e0 con 0x7f6eb41f96d0 2024-09-13T08:38:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb2a00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e98003ab0 0x7f6eb41fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb2000640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6eb4150b60 0x7f6eb41f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3400640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41f96d0 0x7f6eb40c5430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000f90 tx=0x7f6eac00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:11.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eaaa00640 1 -- 172.21.15.44:0/2693534884 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00cb40 con 0x7f6eb41f96d0 2024-09-13T08:38:11.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eaaa00640 1 -- 172.21.15.44:0/2693534884 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6eac017070 con 0x7f6eb41f96d0 2024-09-13T08:38:11.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3e00640 1 -- 172.21.15.44:0/2693534884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb40b4630 con 0x7f6eb41f96d0 2024-09-13T08:38:11.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eaaa00640 1 -- 172.21.15.44:0/2693534884 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac011460 con 0x7f6eb41f96d0 2024-09-13T08:38:11.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.203+0000 7f6eb3e00640 1 -- 172.21.15.44:0/2693534884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb40c4c30 con 0x7f6eb41f96d0 2024-09-13T08:38:11.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.207+0000 7f6eb3e00640 1 -- 172.21.15.44:0/2693534884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb40cdb50 con 0x7f6eb41f96d0 2024-09-13T08:38:11.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.207+0000 7f6eaaa00640 1 -- 172.21.15.44:0/2693534884 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f6eac011620 con 0x7f6eb41f96d0 2024-09-13T08:38:11.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.207+0000 7f6eaaa00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e94072ec0 0x7f6e94075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:11.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.207+0000 7f6eaaa00640 1 -- 172.21.15.44:0/2693534884 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7f6eac094390 con 0x7f6eb41f96d0 2024-09-13T08:38:11.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.207+0000 7f6eb2a00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e94072ec0 0x7f6e94075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:11.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.207+0000 7f6eb2a00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e94072ec0 0x7f6e94075380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40b0000 tx=0x7f6ea401ae30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:11.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.211+0000 7f6eaaa00640 1 -- 172.21.15.44:0/2693534884 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6eb41fc1e0 con 0x7f6eb41f96d0 2024-09-13T08:38:11.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:11.391+0000 7f6eb3e00640 1 -- 172.21.15.44:0/2693534884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6eb40c6e00 con 0x7f6eb41f96d0 2024-09-13T08:38:12.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.019+0000 7f6eaaa00640 1 -- 172.21.15.44:0/2693534884 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f6eac05f4e0 con 0x7f6eb41f96d0 2024-09-13T08:38:12.024 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T08:38:12.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 -- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e94072ec0 msgr2=0x7f6e94075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e94072ec0 0x7f6e94075380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40b0000 tx=0x7f6ea401ae30 comp rx=0 tx=0).stop 2024-09-13T08:38:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 -- 172.21.15.44:0/2693534884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41f96d0 msgr2=0x7f6eb40c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41f96d0 0x7f6eb40c5430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000f90 tx=0x7f6eac00ecf0 comp rx=0 tx=0).stop 2024-09-13T08:38:12.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 -- 172.21.15.44:0/2693534884 shutdown_connections 2024-09-13T08:38:12.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e94072ec0 0x7f6e94075380 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:12.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41f96d0 0x7f6eb40c5430 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:12.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6eb4150b60 0x7f6eb41f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:12.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 --2- 172.21.15.44:0/2693534884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e98003ab0 0x7f6eb41fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:12.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 -- 172.21.15.44:0/2693534884 >> 172.21.15.44:0/2693534884 conn(0x7f6eb40cfa80 msgr2=0x7f6eb415e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:12.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 -- 172.21.15.44:0/2693534884 shutdown_connections 2024-09-13T08:38:12.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.023+0000 7f6eb3e00640 1 -- 172.21.15.44:0/2693534884 wait complete. 2024-09-13T08:38:12.056 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T08:38:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.207+0000 7f50f6a00640 1 -- 172.21.15.44:0/1590414615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d4004810 msgr2=0x7f50d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.207+0000 7f50f6a00640 1 --2- 172.21.15.44:0/1590414615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d4004810 0x7f50d4004c90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f50d80016f0 tx=0x7f50d801c7b0 comp rx=0 tx=0).stop 2024-09-13T08:38:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.207+0000 7f50f4a00640 1 -- 172.21.15.44:0/1590414615 reap_dead start 2024-09-13T08:38:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.207+0000 7f50f6a00640 1 -- 172.21.15.44:0/1590414615 shutdown_connections 2024-09-13T08:38:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.207+0000 7f50f6a00640 1 -- 172.21.15.44:0/1590414615 >> 172.21.15.44:0/1590414615 conn(0x7f50f00cce90 msgr2=0x7f50f00cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.207+0000 7f50f6a00640 1 -- 172.21.15.44:0/1590414615 shutdown_connections 2024-09-13T08:38:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.207+0000 7f50f6a00640 1 -- 172.21.15.44:0/1590414615 wait complete. 2024-09-13T08:38:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.207+0000 7f50f6a00640 1 Processor -- start 2024-09-13T08:38:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.207+0000 7f50f6a00640 1 -- start start 2024-09-13T08:38:12.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f6a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50d4004810 0x7f50f01f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:12.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f6a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50f014f8f0 0x7f50f014f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:12.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50f00b0900 0x7f50f00b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:12.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f6a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50f0068960 con 0x7f50f00b0900 2024-09-13T08:38:12.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f6a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50f015ee10 con 0x7f50d4004810 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f6a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f50f0068ae0 con 0x7f50f014f8f0 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50f00b0900 0x7f50f00b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50f00b0900 0x7f50f00b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40382/0 (socket says 172.21.15.44:40382) 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f4a00640 1 -- 172.21.15.44:0/2698710269 learned_addr learned my addr 172.21.15.44:0/2698710269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50e7e00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50f014f8f0 0x7f50f014f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50efe00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50d4004810 0x7f50f01f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f4a00640 1 -- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50f014f8f0 msgr2=0x7f50f014f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f4a00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50f014f8f0 0x7f50f014f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f4a00640 1 -- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50d4004810 msgr2=0x7f50f01f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f4a00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50d4004810 0x7f50f01f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f4a00640 1 -- 172.21.15.44:0/2698710269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f01f5430 con 0x7f50f00b0900 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f4a00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50f00b0900 0x7f50f00b0d80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f50f00c6790 tx=0x7f50e800d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50ecc00640 1 -- 172.21.15.44:0/2698710269 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8003980 con 0x7f50f00b0900 2024-09-13T08:38:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50ecc00640 1 -- 172.21.15.44:0/2698710269 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50e8003b40 con 0x7f50f00b0900 2024-09-13T08:38:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f6a00640 1 -- 172.21.15.44:0/2698710269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f01f56b0 con 0x7f50f00b0900 2024-09-13T08:38:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50ecc00640 1 -- 172.21.15.44:0/2698710269 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8004030 con 0x7f50f00b0900 2024-09-13T08:38:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f6a00640 1 -- 172.21.15.44:0/2698710269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f01f5c70 con 0x7f50f00b0900 2024-09-13T08:38:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50f6a00640 1 -- 172.21.15.44:0/2698710269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f01f5eb0 con 0x7f50f00b0900 2024-09-13T08:38:12.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.211+0000 7f50ecc00640 1 -- 172.21.15.44:0/2698710269 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f50e80041d0 con 0x7f50f00b0900 2024-09-13T08:38:12.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.215+0000 7f50ecc00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50cc072ec0 0x7f50cc075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:12.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.215+0000 7f50ecc00640 1 -- 172.21.15.44:0/2698710269 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5357+0+0 (secure 0 0 0) 0x7f50e8094540 con 0x7f50f00b0900 2024-09-13T08:38:12.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.215+0000 7f50efe00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50cc072ec0 0x7f50cc075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:12.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.215+0000 7f50efe00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50cc072ec0 0x7f50cc075380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f50f015dd30 tx=0x7f50e00011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:12.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.219+0000 7f50ecc00640 1 -- 172.21.15.44:0/2698710269 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f50f01f5eb0 con 0x7f50f00b0900 2024-09-13T08:38:12.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:12.395+0000 7f50f6a00640 1 -- 172.21.15.44:0/2698710269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f50f015aa80 con 0x7f50f00b0900 2024-09-13T08:38:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.031+0000 7f50ecc00640 1 -- 172.21.15.44:0/2698710269 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f50f01f5430 con 0x7f50f00b0900 2024-09-13T08:38:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T08:38:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 -- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50cc072ec0 msgr2=0x7f50cc075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50cc072ec0 0x7f50cc075380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f50f015dd30 tx=0x7f50e00011a0 comp rx=0 tx=0).stop 2024-09-13T08:38:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 -- 172.21.15.44:0/2698710269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50f00b0900 msgr2=0x7f50f00b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50f00b0900 0x7f50f00b0d80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f50f00c6790 tx=0x7f50e800d7b0 comp rx=0 tx=0).stop 2024-09-13T08:38:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 -- 172.21.15.44:0/2698710269 shutdown_connections 2024-09-13T08:38:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50cc072ec0 0x7f50cc075380 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50f00b0900 0x7f50f00b0d80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50f014f8f0 0x7f50f014f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 --2- 172.21.15.44:0/2698710269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50d4004810 0x7f50f01f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:13.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 -- 172.21.15.44:0/2698710269 >> 172.21.15.44:0/2698710269 conn(0x7f50f00cce90 msgr2=0x7f50f0158610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:13.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 -- 172.21.15.44:0/2698710269 shutdown_connections 2024-09-13T08:38:13.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.035+0000 7f50f6a00640 1 -- 172.21.15.44:0/2698710269 wait complete. 2024-09-13T08:38:13.069 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T08:38:13.069 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T08:38:13.070 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T08:38:13.070 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:38:13.070 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T08:38:13.070 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T08:38:13.071 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T08:38:13.071 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:38:13.071 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T08:38:13.072 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T08:38:13.072 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T08:38:13.072 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:38:13.072 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T08:38:13.073 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T08:38:13.073 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T08:38:13.073 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:38:13.084 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T08:38:13.084 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T08:38:13.086 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T08:38:13.086 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T08:38:13.088 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T08:38:13.088 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T08:38:13.089 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T08:38:13.090 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T08:38:13.132 INFO:tasks.ceph.mds.a:Started 2024-09-13T08:38:13.133 DEBUG:teuthology.parallel:result is None 2024-09-13T08:38:13.133 INFO:tasks.ceph.mds.b:Started 2024-09-13T08:38:13.134 DEBUG:teuthology.parallel:result is None 2024-09-13T08:38:13.134 INFO:tasks.ceph.mds.c:Started 2024-09-13T08:38:13.134 DEBUG:teuthology.parallel:result is None 2024-09-13T08:38:13.135 INFO:tasks.ceph.mds.d:Started 2024-09-13T08:38:13.135 DEBUG:teuthology.parallel:result is None 2024-09-13T08:38:13.135 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T08:38:13.222 INFO:tasks.ceph.mds.b.smithi129.stdout:starting mds.b at 2024-09-13T08:38:13.223 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:38:13.223 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:38:13.223 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:38:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.299+0000 7f655f200640 1 -- 172.21.15.44:0/2599438547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6560074ee0 msgr2=0x7f65600cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:13.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.299+0000 7f655f200640 1 --2- 172.21.15.44:0/2599438547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6560074ee0 0x7f65600cc030 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f654c009900 tx=0x7f654c01c850 comp rx=0 tx=0).stop 2024-09-13T08:38:13.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.299+0000 7f655f200640 1 -- 172.21.15.44:0/2599438547 shutdown_connections 2024-09-13T08:38:13.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.299+0000 7f655f200640 1 --2- 172.21.15.44:0/2599438547 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65600b01c0 0x7f65600cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:13.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.299+0000 7f655f200640 1 --2- 172.21.15.44:0/2599438547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6560074ee0 0x7f65600cc030 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:13.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.299+0000 7f655f200640 1 -- 172.21.15.44:0/2599438547 >> 172.21.15.44:0/2599438547 conn(0x7f6560074310 msgr2=0x7f65600746f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:13.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.299+0000 7f655f200640 1 -- 172.21.15.44:0/2599438547 shutdown_connections 2024-09-13T08:38:13.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.299+0000 7f655f200640 1 -- 172.21.15.44:0/2599438547 wait complete. 2024-09-13T08:38:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.299+0000 7f655f200640 1 Processor -- start 2024-09-13T08:38:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.299+0000 7f655f200640 1 -- start start 2024-09-13T08:38:13.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6560074ee0 0x7f6560099950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:13.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655f200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65600b01c0 0x7f65600cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f656015f250 0x7f65600cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6560168db0 con 0x7f6560074ee0 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6560168c30 con 0x7f656015f250 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655f200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6560168f30 con 0x7f65600b01c0 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6560074ee0 0x7f6560099950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6560074ee0 0x7f6560099950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40442/0 (socket says 172.21.15.44:40442) 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655de00640 1 -- 172.21.15.44:0/2386041622 learned_addr learned my addr 172.21.15.44:0/2386041622 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655e800640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f656015f250 0x7f65600cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655d400640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65600b01c0 0x7f65600cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655de00640 1 -- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65600b01c0 msgr2=0x7f65600cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655de00640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65600b01c0 0x7f65600cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655de00640 1 -- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f656015f250 msgr2=0x7f65600cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655de00640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f656015f250 0x7f65600cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655de00640 1 -- 172.21.15.44:0/2386041622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6560099e90 con 0x7f6560074ee0 2024-09-13T08:38:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655de00640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6560074ee0 0x7f6560099950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f655000c3c0 tx=0x7f655000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f6556000640 1 -- 172.21.15.44:0/2386041622 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550017070 con 0x7f6560074ee0 2024-09-13T08:38:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f6556000640 1 -- 172.21.15.44:0/2386041622 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65500053c0 con 0x7f6560074ee0 2024-09-13T08:38:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f6556000640 1 -- 172.21.15.44:0/2386041622 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550003860 con 0x7f6560074ee0 2024-09-13T08:38:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655f200640 1 -- 172.21.15.44:0/2386041622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65600ce640 con 0x7f6560074ee0 2024-09-13T08:38:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655f200640 1 -- 172.21.15.44:0/2386041622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65600cec00 con 0x7f6560074ee0 2024-09-13T08:38:13.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f655f200640 1 -- 172.21.15.44:0/2386041622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65600b0710 con 0x7f6560074ee0 2024-09-13T08:38:13.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.303+0000 7f6556000640 1 -- 172.21.15.44:0/2386041622 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f655001a070 con 0x7f6560074ee0 2024-09-13T08:38:13.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.307+0000 7f6556000640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6540072ec0 0x7f6540075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:13.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.307+0000 7f6556000640 1 -- 172.21.15.44:0/2386041622 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f6550092e50 con 0x7f6560074ee0 2024-09-13T08:38:13.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.307+0000 7f655d400640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6540072ec0 0x7f6540075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.307+0000 7f655d400640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6540072ec0 0x7f6540075380 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f654c0af000 tx=0x7f654c01b090 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:13.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.311+0000 7f6556000640 1 -- 172.21.15.44:0/2386041622 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f655005dcb0 con 0x7f6560074ee0 2024-09-13T08:38:13.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:13.495+0000 7f655f200640 1 -- 172.21.15.44:0/2386041622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f656015fa10 con 0x7f6560074ee0 2024-09-13T08:38:14.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f6556000640 1 -- 172.21.15.44:0/2386041622 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f65500626e0 con 0x7f6560074ee0 2024-09-13T08:38:14.055 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-09-13T08:38:14.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 -- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6540072ec0 msgr2=0x7f6540075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6540072ec0 0x7f6540075380 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f654c0af000 tx=0x7f654c01b090 comp rx=0 tx=0).stop 2024-09-13T08:38:14.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 -- 172.21.15.44:0/2386041622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6560074ee0 msgr2=0x7f6560099950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6560074ee0 0x7f6560099950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f655000c3c0 tx=0x7f655000c880 comp rx=0 tx=0).stop 2024-09-13T08:38:14.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 -- 172.21.15.44:0/2386041622 shutdown_connections 2024-09-13T08:38:14.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6540072ec0 0x7f6540075380 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f656015f250 0x7f65600cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65600b01c0 0x7f65600cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 --2- 172.21.15.44:0/2386041622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6560074ee0 0x7f6560099950 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 -- 172.21.15.44:0/2386041622 >> 172.21.15.44:0/2386041622 conn(0x7f6560074310 msgr2=0x7f65600ada00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:14.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 -- 172.21.15.44:0/2386041622 shutdown_connections 2024-09-13T08:38:14.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.051+0000 7f655f200640 1 -- 172.21.15.44:0/2386041622 wait complete. 2024-09-13T08:38:14.085 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T08:38:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.231+0000 7fd9c6e00640 1 -- 172.21.15.44:0/1284286681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9c00b3dc0 msgr2=0x7fd9c00d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.231+0000 7fd9c6e00640 1 --2- 172.21.15.44:0/1284286681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9c00b3dc0 0x7fd9c00d13b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009bc0 tx=0x7fd9b801cba0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.231+0000 7fd9c6e00640 1 -- 172.21.15.44:0/1284286681 shutdown_connections 2024-09-13T08:38:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.231+0000 7fd9c6e00640 1 --2- 172.21.15.44:0/1284286681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9c00b3dc0 0x7fd9c00d13b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.231+0000 7fd9c6e00640 1 -- 172.21.15.44:0/1284286681 >> 172.21.15.44:0/1284286681 conn(0x7fd9c00b2890 msgr2=0x7fd9c00b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.231+0000 7fd9c6e00640 1 -- 172.21.15.44:0/1284286681 shutdown_connections 2024-09-13T08:38:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.231+0000 7fd9c6e00640 1 -- 172.21.15.44:0/1284286681 wait complete. 2024-09-13T08:38:14.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.231+0000 7fd9c6e00640 1 Processor -- start 2024-09-13T08:38:14.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.231+0000 7fd9c6e00640 1 -- start start 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9c6e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9c00b3dc0 0x7fd9c0150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9c6e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c00b4db0 0x7fd9c0152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9c6e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9c015a980 0x7fd9c014f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9c6e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0168c00 con 0x7fd9c00b4db0 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9c6e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0168a80 con 0x7fd9c00b3dc0 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9c6e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c0168d80 con 0x7fd9c015a980 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c00b4db0 0x7fd9c0152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c00b4db0 0x7fd9c0152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40454/0 (socket says 172.21.15.44:40454) 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9b7e00640 1 -- 172.21.15.44:0/115352152 learned_addr learned my addr 172.21.15.44:0/115352152 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9bfe00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9c00b3dc0 0x7fd9c0150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9c4e00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9c015a980 0x7fd9c014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9b7e00640 1 -- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9c015a980 msgr2=0x7fd9c014f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9b7e00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9c015a980 0x7fd9c014f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9b7e00640 1 -- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9c00b3dc0 msgr2=0x7fd9c0150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9b7e00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9c00b3dc0 0x7fd9c0150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9b7e00640 1 -- 172.21.15.44:0/115352152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c0150750 con 0x7fd9c00b4db0 2024-09-13T08:38:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9b7e00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c00b4db0 0x7fd9c0152650 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800c4b0 tx=0x7fd9a800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9bcc00640 1 -- 172.21.15.44:0/115352152 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80052d0 con 0x7fd9c00b4db0 2024-09-13T08:38:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9bcc00640 1 -- 172.21.15.44:0/115352152 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9a8005470 con 0x7fd9c00b4db0 2024-09-13T08:38:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9bcc00640 1 -- 172.21.15.44:0/115352152 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8012870 con 0x7fd9c00b4db0 2024-09-13T08:38:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9c6e00640 1 -- 172.21.15.44:0/115352152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c0150a30 con 0x7fd9c00b4db0 2024-09-13T08:38:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9c6e00640 1 -- 172.21.15.44:0/115352152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c0150ff0 con 0x7fd9c00b4db0 2024-09-13T08:38:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9c6e00640 1 -- 172.21.15.44:0/115352152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c00d13b0 con 0x7fd9c00b4db0 2024-09-13T08:38:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.235+0000 7fd9bcc00640 1 -- 172.21.15.44:0/115352152 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fd9a8003600 con 0x7fd9c00b4db0 2024-09-13T08:38:14.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.239+0000 7fd9bcc00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd99c072ef0 0x7fd99c0753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:14.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.239+0000 7fd9bcc00640 1 -- 172.21.15.44:0/115352152 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fd9a8093560 con 0x7fd9c00b4db0 2024-09-13T08:38:14.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.239+0000 7fd9bfe00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd99c072ef0 0x7fd99c0753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:14.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.239+0000 7fd9bfe00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd99c072ef0 0x7fd99c0753b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00046c0 tx=0x7fd9b0007cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.243+0000 7fd9bcc00640 1 -- 172.21.15.44:0/115352152 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd9a805e410 con 0x7fd9c00b4db0 2024-09-13T08:38:14.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.475+0000 7fd9c6e00640 1 -- 172.21.15.44:0/115352152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd9c00d0f00 con 0x7fd9c00b4db0 2024-09-13T08:38:14.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.475+0000 7fd9bcc00640 1 -- 172.21.15.44:0/115352152 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fd9a8062e40 con 0x7fd9c00b4db0 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw==", 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ==", 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA==", 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.483 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw==", 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:38:14.484 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg==", 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-09-13T08:38:14.485 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg==", 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw==", 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:38:14.486 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig==", 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.487 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ==", 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g==", 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.488 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg==", 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.489 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg==", 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-09-13T08:38:14.490 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA==", 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w==", 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:38:14.491 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA==", 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:38:14.492 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w==", 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.493 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmDD2LJxAA3s0jEb2EfPsLcu0w145G3Q==", 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-mds", 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQC1+eNmjuRmDxAAKRAeRWBLeylKWEA/X8BboQ==", 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-mds" 2024-09-13T08:38:14.494 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-mgr", 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQC1+eNmTe9mDxAAR0FCPBGhNFlH/7AmfSz2Tg==", 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-osd", 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQC1+eNm+PdmDxAAz/XSOPXZw9sGdy5Yben2aw==", 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.495 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-osd" 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-rbd", 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQC1+eNmLABnDxAAxwFeVaV2+4tW5WCN5ZCg+g==", 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQC1+eNmnQhnDxAAQIbzpMzibxq8xobkWwS5Uw==", 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.496 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-rgw", 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQC1+eNm7RJnDxAAwrAnMEhlWoT1wTZNf4U1UQ==", 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ==", 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg==", 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T08:38:14.499 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:38:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 -- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd99c072ef0 msgr2=0x7fd99c0753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd99c072ef0 0x7fd99c0753b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00046c0 tx=0x7fd9b0007cd0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 -- 172.21.15.44:0/115352152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c00b4db0 msgr2=0x7fd9c0152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c00b4db0 0x7fd9c0152650 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800c4b0 tx=0x7fd9a800c970 comp rx=0 tx=0).stop 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 -- 172.21.15.44:0/115352152 shutdown_connections 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd99c072ef0 0x7fd99c0753b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9c015a980 0x7fd9c014f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c00b4db0 0x7fd9c0152650 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 --2- 172.21.15.44:0/115352152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9c00b3dc0 0x7fd9c0150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 -- 172.21.15.44:0/115352152 >> 172.21.15.44:0/115352152 conn(0x7fd9c00b2890 msgr2=0x7fd9c00b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 -- 172.21.15.44:0/115352152 shutdown_connections 2024-09-13T08:38:14.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.479+0000 7fd9c6e00640 1 -- 172.21.15.44:0/115352152 wait complete. 2024-09-13T08:38:14.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-13T08:38:14.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.647+0000 7ff78f800640 1 -- 172.21.15.44:0/1147886701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff770005220 msgr2=0x7ff77000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.647+0000 7ff78f800640 1 --2- 172.21.15.44:0/1147886701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff770005220 0x7ff77000b9d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff780001dc0 tx=0x7ff78001caf0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.647+0000 7ff78d800640 1 -- 172.21.15.44:0/1147886701 reap_dead start 2024-09-13T08:38:14.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.647+0000 7ff78f800640 1 -- 172.21.15.44:0/1147886701 shutdown_connections 2024-09-13T08:38:14.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.647+0000 7ff78f800640 1 -- 172.21.15.44:0/1147886701 >> 172.21.15.44:0/1147886701 conn(0x7ff7880d1dc0 msgr2=0x7ff7880d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:14.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.647+0000 7ff78f800640 1 -- 172.21.15.44:0/1147886701 shutdown_connections 2024-09-13T08:38:14.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.647+0000 7ff78f800640 1 -- 172.21.15.44:0/1147886701 wait complete. 2024-09-13T08:38:14.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 Processor -- start 2024-09-13T08:38:14.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 -- start start 2024-09-13T08:38:14.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff770005220 0x7ff7880b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff78815b5e0 0x7ff7881e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff7880b3740 0x7ff7880b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff78815c400 con 0x7ff770005220 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff78815c280 con 0x7ff78815b5e0 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff7880ce670 con 0x7ff7880b3740 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff770005220 0x7ff7880b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff770005220 0x7ff7880b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40478/0 (socket says 172.21.15.44:40478) 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78ce00640 1 -- 172.21.15.44:0/2664684095 learned_addr learned my addr 172.21.15.44:0/2664684095 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78ce00640 1 -- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff7880b3740 msgr2=0x7ff7880b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78d800640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff7880b3740 0x7ff7880b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff787e00640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff78815b5e0 0x7ff7881e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78ce00640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff7880b3740 0x7ff7880b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78ce00640 1 -- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff78815b5e0 msgr2=0x7ff7881e9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78ce00640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff78815b5e0 0x7ff7881e9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78ce00640 1 -- 172.21.15.44:0/2664684095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7881eaf70 con 0x7ff770005220 2024-09-13T08:38:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78d800640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff7880b3740 0x7ff7880b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff787e00640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff78815b5e0 0x7ff7881e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78ce00640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff770005220 0x7ff7880b71a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0107f0 tx=0x7ff77c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff784c00640 1 -- 172.21.15.44:0/2664684095 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c010ea0 con 0x7ff770005220 2024-09-13T08:38:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff784c00640 1 -- 172.21.15.44:0/2664684095 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff77c00ece0 con 0x7ff770005220 2024-09-13T08:38:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 -- 172.21.15.44:0/2664684095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7881ea6d0 con 0x7ff770005220 2024-09-13T08:38:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff784c00640 1 -- 172.21.15.44:0/2664684095 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c01c6e0 con 0x7ff770005220 2024-09-13T08:38:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 -- 172.21.15.44:0/2664684095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7881eac90 con 0x7ff770005220 2024-09-13T08:38:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.651+0000 7ff78f800640 1 -- 172.21.15.44:0/2664684095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7880b4100 con 0x7ff770005220 2024-09-13T08:38:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.655+0000 7ff784c00640 1 -- 172.21.15.44:0/2664684095 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7ff77c01c880 con 0x7ff770005220 2024-09-13T08:38:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.655+0000 7ff784c00640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff768072ec0 0x7ff768075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.655+0000 7ff784c00640 1 -- 172.21.15.44:0/2664684095 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7ff77c094840 con 0x7ff770005220 2024-09-13T08:38:14.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.655+0000 7ff787e00640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff768072ec0 0x7ff768075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:14.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.655+0000 7ff787e00640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff768072ec0 0x7ff768075380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff774001870 tx=0x7ff774001120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:14.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.659+0000 7ff784c00640 1 -- 172.21.15.44:0/2664684095 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff77c05fe70 con 0x7ff770005220 2024-09-13T08:38:14.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.895+0000 7ff78f800640 1 -- 172.21.15.44:0/2664684095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7ff7880ceb90 con 0x7ff770005220 2024-09-13T08:38:14.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff784c00640 1 -- 172.21.15.44:0/2664684095 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7ff77c05f770 con 0x7ff770005220 2024-09-13T08:38:14.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 -- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff768072ec0 msgr2=0x7ff768075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff768072ec0 0x7ff768075380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff774001870 tx=0x7ff774001120 comp rx=0 tx=0).stop 2024-09-13T08:38:14.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 -- 172.21.15.44:0/2664684095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff770005220 msgr2=0x7ff7880b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:14.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff770005220 0x7ff7880b71a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0107f0 tx=0x7ff77c010cb0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 -- 172.21.15.44:0/2664684095 shutdown_connections 2024-09-13T08:38:14.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff768072ec0 0x7ff768075380 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff7880b3740 0x7ff7880b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff78815b5e0 0x7ff7881e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 --2- 172.21.15.44:0/2664684095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff770005220 0x7ff7880b71a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:14.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 -- 172.21.15.44:0/2664684095 >> 172.21.15.44:0/2664684095 conn(0x7ff7880d1dc0 msgr2=0x7ff7880cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:14.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 -- 172.21.15.44:0/2664684095 shutdown_connections 2024-09-13T08:38:14.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:14.899+0000 7ff78f800640 1 -- 172.21.15.44:0/2664684095 wait complete. 2024-09-13T08:38:14.924 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-13T08:38:15.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 -- 172.21.15.44:0/1769693753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300003c20 msgr2=0x7f6300004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 --2- 172.21.15.44:0/1769693753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300003c20 0x7f6300004040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f630c001860 tx=0x7f630c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631d800640 1 -- 172.21.15.44:0/1769693753 reap_dead start 2024-09-13T08:38:15.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 -- 172.21.15.44:0/1769693753 shutdown_connections 2024-09-13T08:38:15.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 -- 172.21.15.44:0/1769693753 >> 172.21.15.44:0/1769693753 conn(0x7f63180d1dc0 msgr2=0x7f63180d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:15.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 -- 172.21.15.44:0/1769693753 shutdown_connections 2024-09-13T08:38:15.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 -- 172.21.15.44:0/1769693753 wait complete. 2024-09-13T08:38:15.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 Processor -- start 2024-09-13T08:38:15.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 -- start start 2024-09-13T08:38:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6300003c20 0x7f631814b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f63180ce740 0x7f631815d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63181591b0 0x7f631815d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f63180d1550 con 0x7f63181591b0 2024-09-13T08:38:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63180d13d0 con 0x7f6300003c20 2024-09-13T08:38:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631f800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f631816a520 con 0x7f63180ce740 2024-09-13T08:38:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63181591b0 0x7f631815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63181591b0 0x7f631815d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40488/0 (socket says 172.21.15.44:40488) 2024-09-13T08:38:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6300003c20 0x7f631814b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631d800640 1 -- 172.21.15.44:0/1286843846 learned_addr learned my addr 172.21.15.44:0/1286843846 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f6317e00640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f63180ce740 0x7f631815d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631d800640 1 -- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f63180ce740 msgr2=0x7f631815d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631d800640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f63180ce740 0x7f631815d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631d800640 1 -- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6300003c20 msgr2=0x7f631814b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631d800640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6300003c20 0x7f631814b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.071+0000 7f631d800640 1 -- 172.21.15.44:0/1286843846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63180b69c0 con 0x7f63181591b0 2024-09-13T08:38:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.075+0000 7f631d800640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63181591b0 0x7f631815d540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6318069e80 tx=0x7f6310006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:15.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.075+0000 7f6314c00640 1 -- 172.21.15.44:0/1286843846 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631000fa20 con 0x7f63181591b0 2024-09-13T08:38:15.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.075+0000 7f6314c00640 1 -- 172.21.15.44:0/1286843846 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6310017070 con 0x7f63181591b0 2024-09-13T08:38:15.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.075+0000 7f6314c00640 1 -- 172.21.15.44:0/1286843846 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631000fea0 con 0x7f63181591b0 2024-09-13T08:38:15.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.075+0000 7f631f800640 1 -- 172.21.15.44:0/1286843846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63180b6ca0 con 0x7f63181591b0 2024-09-13T08:38:15.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.075+0000 7f631f800640 1 -- 172.21.15.44:0/1286843846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63180b7260 con 0x7f63181591b0 2024-09-13T08:38:15.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.075+0000 7f631f800640 1 -- 172.21.15.44:0/1286843846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6318071c60 con 0x7f63181591b0 2024-09-13T08:38:15.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.075+0000 7f6314c00640 1 -- 172.21.15.44:0/1286843846 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f631001c460 con 0x7f63181591b0 2024-09-13T08:38:15.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.079+0000 7f6314c00640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62f8072ec0 0x7f62f8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.079+0000 7f6314c00640 1 -- 172.21.15.44:0/1286843846 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f6310094130 con 0x7f63181591b0 2024-09-13T08:38:15.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.079+0000 7f631ce00640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62f8072ec0 0x7f62f8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.079+0000 7f631ce00640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62f8072ec0 0x7f62f8075380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f630c001830 tx=0x7f630c019720 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:15.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.083+0000 7f6314c00640 1 -- 172.21.15.44:0/1286843846 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f631005f630 con 0x7f63181591b0 2024-09-13T08:38:15.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.319+0000 7f631f800640 1 -- 172.21.15.44:0/1286843846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f631806e6c0 con 0x7f63181591b0 2024-09-13T08:38:15.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.319+0000 7f6314c00640 1 -- 172.21.15.44:0/1286843846 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f631005ef30 con 0x7f63181591b0 2024-09-13T08:38:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 -- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62f8072ec0 msgr2=0x7f62f8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62f8072ec0 0x7f62f8075380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f630c001830 tx=0x7f630c019720 comp rx=0 tx=0).stop 2024-09-13T08:38:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 -- 172.21.15.44:0/1286843846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63181591b0 msgr2=0x7f631815d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63181591b0 0x7f631815d540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6318069e80 tx=0x7f6310006660 comp rx=0 tx=0).stop 2024-09-13T08:38:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 -- 172.21.15.44:0/1286843846 shutdown_connections 2024-09-13T08:38:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f62f8072ec0 0x7f62f8075380 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63181591b0 0x7f631815d540 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f63180ce740 0x7f631815d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 --2- 172.21.15.44:0/1286843846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6300003c20 0x7f631814b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 -- 172.21.15.44:0/1286843846 >> 172.21.15.44:0/1286843846 conn(0x7f63180d1dc0 msgr2=0x7f6318167d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 -- 172.21.15.44:0/1286843846 shutdown_connections 2024-09-13T08:38:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.323+0000 7f631f800640 1 -- 172.21.15.44:0/1286843846 wait complete. 2024-09-13T08:38:15.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-13T08:38:15.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 -- 172.21.15.44:0/3872458660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f956c15bea0 msgr2=0x7f956c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 --2- 172.21.15.44:0/3872458660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f956c15bea0 0x7f956c0ce670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f956c0cc3a0 tx=0x7f955c01c830 comp rx=0 tx=0).stop 2024-09-13T08:38:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 -- 172.21.15.44:0/3872458660 shutdown_connections 2024-09-13T08:38:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 --2- 172.21.15.44:0/3872458660 >> v2:172.21.15.129:3300/0 conn(0x7f956c0d28e0 0x7f956c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 --2- 172.21.15.44:0/3872458660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f956c15bea0 0x7f956c0ce670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 -- 172.21.15.44:0/3872458660 >> 172.21.15.44:0/3872458660 conn(0x7f956c0d1dc0 msgr2=0x7f956c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 -- 172.21.15.44:0/3872458660 shutdown_connections 2024-09-13T08:38:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 -- 172.21.15.44:0/3872458660 wait complete. 2024-09-13T08:38:15.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 Processor -- start 2024-09-13T08:38:15.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 -- start start 2024-09-13T08:38:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f956c0cdb30 0x7f956c1f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f956c0d28e0 0x7f956c06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f956c15bea0 0x7f956c0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f956c168fc0 con 0x7f956c15bea0 2024-09-13T08:38:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f956c168e40 con 0x7f956c0d28e0 2024-09-13T08:38:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9570c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f956c169140 con 0x7f956c0cdb30 2024-09-13T08:38:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f956a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f956c15bea0 0x7f956c0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f956a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f956c15bea0 0x7f956c0719d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40512/0 (socket says 172.21.15.44:40512) 2024-09-13T08:38:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f956a800640 1 -- 172.21.15.44:0/449748938 learned_addr learned my addr 172.21.15.44:0/449748938 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9569e00640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f956c0cdb30 0x7f956c1f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f9563e00640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f956c0d28e0 0x7f956c06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.491+0000 7f956a800640 1 -- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f956c0cdb30 msgr2=0x7f956c1f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f956a800640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f956c0cdb30 0x7f956c1f7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f956a800640 1 -- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f956c0d28e0 msgr2=0x7f956c06f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f956a800640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f956c0d28e0 0x7f956c06f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f956a800640 1 -- 172.21.15.44:0/449748938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c1f7e80 con 0x7f956c15bea0 2024-09-13T08:38:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f956a800640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f956c15bea0 0x7f956c0719d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f956400b9a0 tx=0x7f956400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f9562000640 1 -- 172.21.15.44:0/449748938 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564019070 con 0x7f956c15bea0 2024-09-13T08:38:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f9562000640 1 -- 172.21.15.44:0/449748938 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f956400ccb0 con 0x7f956c15bea0 2024-09-13T08:38:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f9562000640 1 -- 172.21.15.44:0/449748938 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640146a0 con 0x7f956c15bea0 2024-09-13T08:38:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f9570c00640 1 -- 172.21.15.44:0/449748938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956c14dd60 con 0x7f956c15bea0 2024-09-13T08:38:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f9570c00640 1 -- 172.21.15.44:0/449748938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956c14e320 con 0x7f956c15bea0 2024-09-13T08:38:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f9570c00640 1 -- 172.21.15.44:0/449748938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f956c0cc9d0 con 0x7f956c15bea0 2024-09-13T08:38:15.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.495+0000 7f9562000640 1 -- 172.21.15.44:0/449748938 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f956401c070 con 0x7f956c15bea0 2024-09-13T08:38:15.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.499+0000 7f9562000640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9548072ef0 0x7f95480753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.499+0000 7f9562000640 1 -- 172.21.15.44:0/449748938 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f9564093710 con 0x7f956c15bea0 2024-09-13T08:38:15.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.499+0000 7f9569e00640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9548072ef0 0x7f95480753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.499+0000 7f9569e00640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9548072ef0 0x7f95480753b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f955c0af000 tx=0x7f955c01b070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:15.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.503+0000 7f9562000640 1 -- 172.21.15.44:0/449748938 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f956405e650 con 0x7f956c15bea0 2024-09-13T08:38:15.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.739+0000 7f9570c00640 1 -- 172.21.15.44:0/449748938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f956c0cebb0 con 0x7f956c15bea0 2024-09-13T08:38:15.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9562000640 1 -- 172.21.15.44:0/449748938 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f9564063080 con 0x7f956c15bea0 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 -- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9548072ef0 msgr2=0x7f95480753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9548072ef0 0x7f95480753b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f955c0af000 tx=0x7f955c01b070 comp rx=0 tx=0).stop 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 -- 172.21.15.44:0/449748938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f956c15bea0 msgr2=0x7f956c0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f956c15bea0 0x7f956c0719d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f956400b9a0 tx=0x7f956400be60 comp rx=0 tx=0).stop 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 -- 172.21.15.44:0/449748938 shutdown_connections 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9548072ef0 0x7f95480753b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f956c15bea0 0x7f956c0719d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f956c0d28e0 0x7f956c06f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 --2- 172.21.15.44:0/449748938 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f956c0cdb30 0x7f956c1f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 -- 172.21.15.44:0/449748938 >> 172.21.15.44:0/449748938 conn(0x7f956c0d1dc0 msgr2=0x7f956c155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 -- 172.21.15.44:0/449748938 shutdown_connections 2024-09-13T08:38:15.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.743+0000 7f9570c00640 1 -- 172.21.15.44:0/449748938 wait complete. 2024-09-13T08:38:15.768 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-13T08:38:15.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.911+0000 7fbc71e00640 1 -- 172.21.15.44:0/1918887163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc6c0cdb30 msgr2=0x7fbc6c0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.911+0000 7fbc71e00640 1 --2- 172.21.15.44:0/1918887163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc6c0cdb30 0x7fbc6c0aeef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009bc0 tx=0x7fbc6401cba0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.911+0000 7fbc71e00640 1 -- 172.21.15.44:0/1918887163 shutdown_connections 2024-09-13T08:38:15.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.911+0000 7fbc71e00640 1 --2- 172.21.15.44:0/1918887163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc6c0cdb30 0x7fbc6c0aeef0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.911+0000 7fbc71e00640 1 --2- 172.21.15.44:0/1918887163 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc6c0ced80 0x7fbc6c0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.911+0000 7fbc71e00640 1 -- 172.21.15.44:0/1918887163 >> 172.21.15.44:0/1918887163 conn(0x7fbc6c0d1dc0 msgr2=0x7fbc6c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:15.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.911+0000 7fbc71e00640 1 -- 172.21.15.44:0/1918887163 shutdown_connections 2024-09-13T08:38:15.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.911+0000 7fbc71e00640 1 -- 172.21.15.44:0/1918887163 wait complete. 2024-09-13T08:38:15.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.911+0000 7fbc71e00640 1 Processor -- start 2024-09-13T08:38:15.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.911+0000 7fbc71e00640 1 -- start start 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc71e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc6c0cdb30 0x7fbc6c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc71e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc6c0ced80 0x7fbc6c158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc71e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc6c15bea0 0x7fbc6c06e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc71e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c169340 con 0x7fbc6c0ced80 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc71e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1691c0 con 0x7fbc6c15bea0 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc71e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6c1694c0 con 0x7fbc6c0cdb30 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc6c0ced80 0x7fbc6c158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc6c0ced80 0x7fbc6c158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40542/0 (socket says 172.21.15.44:40542) 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63e00640 1 -- 172.21.15.44:0/1601327665 learned_addr learned my addr 172.21.15.44:0/1601327665 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc6aa00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc6c0cdb30 0x7fbc6c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc6b400640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc6c15bea0 0x7fbc6c06e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63e00640 1 -- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc6c0cdb30 msgr2=0x7fbc6c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63e00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc6c0cdb30 0x7fbc6c06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63e00640 1 -- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc6c15bea0 msgr2=0x7fbc6c06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:15.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63e00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc6c15bea0 0x7fbc6c06e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:15.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63e00640 1 -- 172.21.15.44:0/1601327665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c0c56a0 con 0x7fbc6c0ced80 2024-09-13T08:38:15.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc6b400640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc6c15bea0 0x7fbc6c06e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:15.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc6aa00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc6c0cdb30 0x7fbc6c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:15.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63e00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc6c0ced80 0x7fbc6c158c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400d950 tx=0x7fbc5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:15.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63400640 1 -- 172.21.15.44:0/1601327665 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54017070 con 0x7fbc6c0ced80 2024-09-13T08:38:15.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63400640 1 -- 172.21.15.44:0/1601327665 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc54009d20 con 0x7fbc6c0ced80 2024-09-13T08:38:15.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63400640 1 -- 172.21.15.44:0/1601327665 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc540106c0 con 0x7fbc6c0ced80 2024-09-13T08:38:15.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc71e00640 1 -- 172.21.15.44:0/1601327665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c06dd30 con 0x7fbc6c0ced80 2024-09-13T08:38:15.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc71e00640 1 -- 172.21.15.44:0/1601327665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c06dfd0 con 0x7fbc6c0ced80 2024-09-13T08:38:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc71e00640 1 -- 172.21.15.44:0/1601327665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6c0ce670 con 0x7fbc6c0ced80 2024-09-13T08:38:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.915+0000 7fbc63400640 1 -- 172.21.15.44:0/1601327665 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fbc54010860 con 0x7fbc6c0ced80 2024-09-13T08:38:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.919+0000 7fbc63400640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc48072ec0 0x7fbc48075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.919+0000 7fbc63400640 1 -- 172.21.15.44:0/1601327665 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fbc54093eb0 con 0x7fbc6c0ced80 2024-09-13T08:38:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.919+0000 7fbc6aa00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc48072ec0 0x7fbc48075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.919+0000 7fbc6aa00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc48072ec0 0x7fbc48075380 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c001630 tx=0x7fbc5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:15.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:15.923+0000 7fbc63400640 1 -- 172.21.15.44:0/1601327665 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fbc5405f440 con 0x7fbc6c0ced80 2024-09-13T08:38:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.155+0000 7fbc71e00640 1 -- 172.21.15.44:0/1601327665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fbc6c0b3f00 con 0x7fbc6c0ced80 2024-09-13T08:38:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.159+0000 7fbc63400640 1 -- 172.21.15.44:0/1601327665 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fbc5405ed40 con 0x7fbc6c0ced80 2024-09-13T08:38:16.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 -- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc48072ec0 msgr2=0x7fbc48075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:16.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc48072ec0 0x7fbc48075380 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c001630 tx=0x7fbc5c009040 comp rx=0 tx=0).stop 2024-09-13T08:38:16.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 -- 172.21.15.44:0/1601327665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc6c0ced80 msgr2=0x7fbc6c158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:16.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc6c0ced80 0x7fbc6c158c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400d950 tx=0x7fbc5400de10 comp rx=0 tx=0).stop 2024-09-13T08:38:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 -- 172.21.15.44:0/1601327665 shutdown_connections 2024-09-13T08:38:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc48072ec0 0x7fbc48075380 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc6c15bea0 0x7fbc6c06e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc6c0ced80 0x7fbc6c158c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 --2- 172.21.15.44:0/1601327665 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc6c0cdb30 0x7fbc6c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 -- 172.21.15.44:0/1601327665 >> 172.21.15.44:0/1601327665 conn(0x7fbc6c0d1dc0 msgr2=0x7fbc6c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 -- 172.21.15.44:0/1601327665 shutdown_connections 2024-09-13T08:38:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.163+0000 7fbc71e00640 1 -- 172.21.15.44:0/1601327665 wait complete. 2024-09-13T08:38:16.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-13T08:38:16.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 -- 172.21.15.44:0/961610020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa948004810 msgr2=0x7fa948004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:16.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 --2- 172.21.15.44:0/961610020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa948004810 0x7fa948004c90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa94c006f90 tx=0x7fa94c01ceb0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa965800640 1 -- 172.21.15.44:0/961610020 reap_dead start 2024-09-13T08:38:16.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 -- 172.21.15.44:0/961610020 shutdown_connections 2024-09-13T08:38:16.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 -- 172.21.15.44:0/961610020 >> 172.21.15.44:0/961610020 conn(0x7fa9600d1e30 msgr2=0x7fa9600d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:16.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 -- 172.21.15.44:0/961610020 shutdown_connections 2024-09-13T08:38:16.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 -- 172.21.15.44:0/961610020 wait complete. 2024-09-13T08:38:16.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 Processor -- start 2024-09-13T08:38:16.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 -- start start 2024-09-13T08:38:16.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa948004810 0x7fa96006f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:16.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9601fb7b0 0x7fa96014d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:16.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa96015ce90 0x7fa96015d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:16.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600b08c0 con 0x7fa948004810 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600b2320 con 0x7fa9601fb7b0 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa967800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa9600b0a40 con 0x7fa96015ce90 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa964e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa948004810 0x7fa96006f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa964e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa948004810 0x7fa96006f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40562/0 (socket says 172.21.15.44:40562) 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa964e00640 1 -- 172.21.15.44:0/402920430 learned_addr learned my addr 172.21.15.44:0/402920430 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa965800640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa96015ce90 0x7fa96015d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa95fe00640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9601fb7b0 0x7fa96014d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa964e00640 1 -- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa96015ce90 msgr2=0x7fa96015d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa964e00640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa96015ce90 0x7fa96015d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa964e00640 1 -- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9601fb7b0 msgr2=0x7fa96014d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa964e00640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9601fb7b0 0x7fa96014d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.330+0000 7fa964e00640 1 -- 172.21.15.44:0/402920430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96006dd30 con 0x7fa948004810 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.334+0000 7fa964e00640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa948004810 0x7fa96006f0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa954011ab0 tx=0x7fa954011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.334+0000 7fa95cc00640 1 -- 172.21.15.44:0/402920430 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95400fe10 con 0x7fa948004810 2024-09-13T08:38:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.334+0000 7fa95cc00640 1 -- 172.21.15.44:0/402920430 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa954016070 con 0x7fa948004810 2024-09-13T08:38:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.334+0000 7fa95cc00640 1 -- 172.21.15.44:0/402920430 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9540126b0 con 0x7fa948004810 2024-09-13T08:38:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.334+0000 7fa967800640 1 -- 172.21.15.44:0/402920430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96006f6f0 con 0x7fa948004810 2024-09-13T08:38:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.334+0000 7fa967800640 1 -- 172.21.15.44:0/402920430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96006e2b0 con 0x7fa948004810 2024-09-13T08:38:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.334+0000 7fa967800640 1 -- 172.21.15.44:0/402920430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9600b14c0 con 0x7fa948004810 2024-09-13T08:38:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.334+0000 7fa95cc00640 1 -- 172.21.15.44:0/402920430 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fa954018030 con 0x7fa948004810 2024-09-13T08:38:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.338+0000 7fa95cc00640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa940072ef0 0x7fa9400753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.338+0000 7fa95cc00640 1 -- 172.21.15.44:0/402920430 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fa954094350 con 0x7fa948004810 2024-09-13T08:38:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.338+0000 7fa95fe00640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa940072ef0 0x7fa9400753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.338+0000 7fa95fe00640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa940072ef0 0x7fa9400753b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0ac000 tx=0x7fa94c019cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.342+0000 7fa95cc00640 1 -- 172.21.15.44:0/402920430 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa95405f850 con 0x7fa948004810 2024-09-13T08:38:16.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.582+0000 7fa967800640 1 -- 172.21.15.44:0/402920430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fa9600cf370 con 0x7fa948004810 2024-09-13T08:38:16.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa95cc00640 1 -- 172.21.15.44:0/402920430 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fa95405f150 con 0x7fa948004810 2024-09-13T08:38:16.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 -- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa940072ef0 msgr2=0x7fa9400753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:16.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa940072ef0 0x7fa9400753b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0ac000 tx=0x7fa94c019cf0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 -- 172.21.15.44:0/402920430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa948004810 msgr2=0x7fa96006f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:16.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa948004810 0x7fa96006f0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa954011ab0 tx=0x7fa954011f70 comp rx=0 tx=0).stop 2024-09-13T08:38:16.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 -- 172.21.15.44:0/402920430 shutdown_connections 2024-09-13T08:38:16.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa940072ef0 0x7fa9400753b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa96015ce90 0x7fa96015d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9601fb7b0 0x7fa96014d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 --2- 172.21.15.44:0/402920430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa948004810 0x7fa96006f0a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 -- 172.21.15.44:0/402920430 >> 172.21.15.44:0/402920430 conn(0x7fa9600d1e30 msgr2=0x7fa9600cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:16.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 -- 172.21.15.44:0/402920430 shutdown_connections 2024-09-13T08:38:16.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.586+0000 7fa967800640 1 -- 172.21.15.44:0/402920430 wait complete. 2024-09-13T08:38:16.613 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-13T08:38:16.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.754+0000 7f6866000640 1 -- 172.21.15.44:0/2991652040 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f686015be80 msgr2=0x7f68600ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:16.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.754+0000 7f6866000640 1 --2- 172.21.15.44:0/2991652040 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f686015be80 0x7f68600ce4e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6850009840 tx=0x7f685001c820 comp rx=0 tx=0).stop 2024-09-13T08:38:16.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.754+0000 7f6866000640 1 -- 172.21.15.44:0/2991652040 shutdown_connections 2024-09-13T08:38:16.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.754+0000 7f6866000640 1 --2- 172.21.15.44:0/2991652040 >> v2:172.21.15.129:3300/0 conn(0x7f68600cd8c0 0x7f68600cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.754+0000 7f6866000640 1 --2- 172.21.15.44:0/2991652040 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f686015be80 0x7f68600ce4e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.754+0000 7f6866000640 1 -- 172.21.15.44:0/2991652040 >> 172.21.15.44:0/2991652040 conn(0x7f68600d1da0 msgr2=0x7f68600d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:16.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.754+0000 7f6866000640 1 -- 172.21.15.44:0/2991652040 shutdown_connections 2024-09-13T08:38:16.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.754+0000 7f6866000640 1 -- 172.21.15.44:0/2991652040 wait complete. 2024-09-13T08:38:16.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.754+0000 7f6866000640 1 Processor -- start 2024-09-13T08:38:16.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6866000640 1 -- start start 2024-09-13T08:38:16.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6866000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68600cd8c0 0x7f68600cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:16.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6866000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68600d28c0 0x7f6860158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:16.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6866000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f686015be80 0x7f68601f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:16.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6866000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6860168de0 con 0x7f686015be80 2024-09-13T08:38:16.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6866000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6860168c60 con 0x7f68600d28c0 2024-09-13T08:38:16.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6866000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6860168f60 con 0x7f68600cd8c0 2024-09-13T08:38:16.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f686015be80 0x7f68601f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:16.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f686015be80 0x7f68601f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40578/0 (socket says 172.21.15.44:40578) 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685fe00640 1 -- 172.21.15.44:0/98776743 learned_addr learned my addr 172.21.15.44:0/98776743 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685fe00640 1 -- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68600cd8c0 msgr2=0x7f68600cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685f400640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68600cd8c0 0x7f68600cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6857e00640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68600d28c0 0x7f6860158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685fe00640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68600cd8c0 0x7f68600cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685fe00640 1 -- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68600d28c0 msgr2=0x7f6860158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685fe00640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68600d28c0 0x7f6860158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685fe00640 1 -- 172.21.15.44:0/98776743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68600cbac0 con 0x7f686015be80 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685f400640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68600cd8c0 0x7f68600cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6857e00640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68600d28c0 0x7f6860158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f685fe00640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f686015be80 0x7f68601f8c50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f685800eb20 tx=0x7f685800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6857400640 1 -- 172.21.15.44:0/98776743 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858004b30 con 0x7f686015be80 2024-09-13T08:38:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6857400640 1 -- 172.21.15.44:0/98776743 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6858004cf0 con 0x7f686015be80 2024-09-13T08:38:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6857400640 1 -- 172.21.15.44:0/98776743 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858005280 con 0x7f686015be80 2024-09-13T08:38:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6866000640 1 -- 172.21.15.44:0/98776743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68600c6020 con 0x7f686015be80 2024-09-13T08:38:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6866000640 1 -- 172.21.15.44:0/98776743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686014f380 con 0x7f686015be80 2024-09-13T08:38:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.758+0000 7f6866000640 1 -- 172.21.15.44:0/98776743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68600b53f0 con 0x7f686015be80 2024-09-13T08:38:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.762+0000 7f6857400640 1 -- 172.21.15.44:0/98776743 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f6858005420 con 0x7f686015be80 2024-09-13T08:38:16.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.762+0000 7f6857400640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f683c072ec0 0x7f683c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:16.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.762+0000 7f6857400640 1 -- 172.21.15.44:0/98776743 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f6858094810 con 0x7f686015be80 2024-09-13T08:38:16.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.762+0000 7f685f400640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f683c072ec0 0x7f683c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:16.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.762+0000 7f685f400640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f683c072ec0 0x7f683c075380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68500af000 tx=0x7f685001b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:16.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:16.766+0000 7f6857400640 1 -- 172.21.15.44:0/98776743 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f68600cbac0 con 0x7f686015be80 2024-09-13T08:38:17.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.002+0000 7f6866000640 1 -- 172.21.15.44:0/98776743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f6860155bd0 con 0x7f686015be80 2024-09-13T08:38:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.002+0000 7f6857400640 1 -- 172.21.15.44:0/98776743 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f685805fda0 con 0x7f686015be80 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 -- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f683c072ec0 msgr2=0x7f683c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f683c072ec0 0x7f683c075380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68500af000 tx=0x7f685001b060 comp rx=0 tx=0).stop 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 -- 172.21.15.44:0/98776743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f686015be80 msgr2=0x7f68601f8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f686015be80 0x7f68601f8c50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f685800eb20 tx=0x7f685800c2d0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 -- 172.21.15.44:0/98776743 shutdown_connections 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f683c072ec0 0x7f683c075380 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f686015be80 0x7f68601f8c50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68600d28c0 0x7f6860158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 --2- 172.21.15.44:0/98776743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68600cd8c0 0x7f68600cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 -- 172.21.15.44:0/98776743 >> 172.21.15.44:0/98776743 conn(0x7f68600d1da0 msgr2=0x7f6860071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 -- 172.21.15.44:0/98776743 shutdown_connections 2024-09-13T08:38:17.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.006+0000 7f6866000640 1 -- 172.21.15.44:0/98776743 wait complete. 2024-09-13T08:38:17.032 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T08:38:17.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 -- 172.21.15.44:0/3691669879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01000cdb30 msgr2=0x7f01000aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:17.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 --2- 172.21.15.44:0/3691669879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01000cdb30 0x7f01000aeef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f01000cc3a0 tx=0x7f00f801cd80 comp rx=0 tx=0).stop 2024-09-13T08:38:17.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 -- 172.21.15.44:0/3691669879 shutdown_connections 2024-09-13T08:38:17.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 --2- 172.21.15.44:0/3691669879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01000cdb30 0x7f01000aeef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 --2- 172.21.15.44:0/3691669879 >> v2:172.21.15.129:3300/0 conn(0x7f01000ced80 0x7f01000ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 -- 172.21.15.44:0/3691669879 >> 172.21.15.44:0/3691669879 conn(0x7f01000d1dc0 msgr2=0x7f01000d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:17.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 -- 172.21.15.44:0/3691669879 shutdown_connections 2024-09-13T08:38:17.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 -- 172.21.15.44:0/3691669879 wait complete. 2024-09-13T08:38:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 Processor -- start 2024-09-13T08:38:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 -- start start 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01000cdb30 0x7f0100157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01000ced80 0x7f01000b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f010015bea0 0x7f01000b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01001692f0 con 0x7f01000ced80 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0100169170 con 0x7f010015bea0 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f0106e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0100169470 con 0x7f01000cdb30 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01000ced80 0x7f01000b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01000ced80 0x7f01000b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40598/0 (socket says 172.21.15.44:40598) 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.174+0000 7f00f7e00640 1 -- 172.21.15.44:0/627560144 learned_addr learned my addr 172.21.15.44:0/627560144 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f0104e00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f010015bea0 0x7f01000b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00ffe00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01000cdb30 0x7f0100157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:17.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00f7e00640 1 -- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01000cdb30 msgr2=0x7f0100157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:17.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00f7e00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01000cdb30 0x7f0100157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00f7e00640 1 -- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f010015bea0 msgr2=0x7f01000b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:17.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00f7e00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f010015bea0 0x7f01000b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00f7e00640 1 -- 172.21.15.44:0/627560144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01000b24f0 con 0x7f01000ced80 2024-09-13T08:38:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00f7e00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01000ced80 0x7f01000b3d30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00e800d950 tx=0x7f00e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00fcc00640 1 -- 172.21.15.44:0/627560144 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8017070 con 0x7f01000ced80 2024-09-13T08:38:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00fcc00640 1 -- 172.21.15.44:0/627560144 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00e8009d20 con 0x7f01000ced80 2024-09-13T08:38:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00fcc00640 1 -- 172.21.15.44:0/627560144 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e80106c0 con 0x7f01000ced80 2024-09-13T08:38:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f0106e00640 1 -- 172.21.15.44:0/627560144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01000b2790 con 0x7f01000ced80 2024-09-13T08:38:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f0106e00640 1 -- 172.21.15.44:0/627560144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01000b2d50 con 0x7f01000ced80 2024-09-13T08:38:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f0106e00640 1 -- 172.21.15.44:0/627560144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01000ce670 con 0x7f01000ced80 2024-09-13T08:38:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.178+0000 7f00fcc00640 1 -- 172.21.15.44:0/627560144 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f00e8010860 con 0x7f01000ced80 2024-09-13T08:38:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.182+0000 7f00fcc00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00dc072ec0 0x7f00dc075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.182+0000 7f00fcc00640 1 -- 172.21.15.44:0/627560144 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f00e8093d60 con 0x7f01000ced80 2024-09-13T08:38:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.182+0000 7f00ffe00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00dc072ec0 0x7f00dc075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.182+0000 7f00ffe00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00dc072ec0 0x7f00dc075380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00f00015d0 tx=0x7f00f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:17.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.186+0000 7f00fcc00640 1 -- 172.21.15.44:0/627560144 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f00e805ed40 con 0x7f01000ced80 2024-09-13T08:38:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.362+0000 7f0106e00640 1 -- 172.21.15.44:0/627560144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f01000b5c50 con 0x7f01000ced80 2024-09-13T08:38:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.362+0000 7f00fcc00640 1 -- 172.21.15.44:0/627560144 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f00e8063770 con 0x7f01000ced80 2024-09-13T08:38:17.368 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:17.369 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:38:17.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:17.369 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T08:38:17.369 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T08:38:17.369 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:17.369 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:38:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 -- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00dc072ec0 msgr2=0x7f00dc075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00dc072ec0 0x7f00dc075380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00f00015d0 tx=0x7f00f0009040 comp rx=0 tx=0).stop 2024-09-13T08:38:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 -- 172.21.15.44:0/627560144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01000ced80 msgr2=0x7f01000b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01000ced80 0x7f01000b3d30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00e800d950 tx=0x7f00e800de10 comp rx=0 tx=0).stop 2024-09-13T08:38:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 -- 172.21.15.44:0/627560144 shutdown_connections 2024-09-13T08:38:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00dc072ec0 0x7f00dc075380 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f010015bea0 0x7f01000b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01000ced80 0x7f01000b3d30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 --2- 172.21.15.44:0/627560144 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01000cdb30 0x7f0100157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 -- 172.21.15.44:0/627560144 >> 172.21.15.44:0/627560144 conn(0x7f01000d1dc0 msgr2=0x7f01000b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 -- 172.21.15.44:0/627560144 shutdown_connections 2024-09-13T08:38:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.366+0000 7f0106e00640 1 -- 172.21.15.44:0/627560144 wait complete. 2024-09-13T08:38:17.390 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T08:38:17.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T08:38:17.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.534+0000 7f6b18a00640 1 -- 172.21.15.44:0/439587571 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b140d2540 msgr2=0x7f6b140d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:17.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.534+0000 7f6b18a00640 1 --2- 172.21.15.44:0/439587571 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b140d2540 0x7f6b140d29b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009c80 tx=0x7f6b0c01cd70 comp rx=0 tx=0).stop 2024-09-13T08:38:17.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.534+0000 7f6b18a00640 1 -- 172.21.15.44:0/439587571 shutdown_connections 2024-09-13T08:38:17.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.534+0000 7f6b18a00640 1 --2- 172.21.15.44:0/439587571 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b140d2540 0x7f6b140d29b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.534+0000 7f6b18a00640 1 --2- 172.21.15.44:0/439587571 >> v2:172.21.15.129:3300/0 conn(0x7f6b1415de40 0x7f6b140b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.534+0000 7f6b18a00640 1 -- 172.21.15.44:0/439587571 >> 172.21.15.44:0/439587571 conn(0x7f6b140d3f90 msgr2=0x7f6b140d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:17.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.534+0000 7f6b18a00640 1 -- 172.21.15.44:0/439587571 shutdown_connections 2024-09-13T08:38:17.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.534+0000 7f6b18a00640 1 -- 172.21.15.44:0/439587571 wait complete. 2024-09-13T08:38:17.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b18a00640 1 Processor -- start 2024-09-13T08:38:17.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b18a00640 1 -- start start 2024-09-13T08:38:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b18a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b140d2540 0x7f6b140cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b18a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1415de40 0x7f6b140cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b18a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1415e220 0x7f6b14150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b18a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1416b0e0 con 0x7f6b140d2540 2024-09-13T08:38:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b18a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1416af60 con 0x7f6b1415e220 2024-09-13T08:38:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b18a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1416b260 con 0x7f6b1415de40 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b11e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b140d2540 0x7f6b140cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b11e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b140d2540 0x7f6b140cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40614/0 (socket says 172.21.15.44:40614) 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b11e00640 1 -- 172.21.15.44:0/2043483344 learned_addr learned my addr 172.21.15.44:0/2043483344 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b0be00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1415de40 0x7f6b140cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b12800640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1415e220 0x7f6b14150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b11e00640 1 -- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1415de40 msgr2=0x7f6b140cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b11e00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1415de40 0x7f6b140cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b11e00640 1 -- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1415e220 msgr2=0x7f6b14150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b11e00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1415e220 0x7f6b14150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b11e00640 1 -- 172.21.15.44:0/2043483344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b140b7070 con 0x7f6b140d2540 2024-09-13T08:38:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b11e00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b140d2540 0x7f6b140cb060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400ea40 tx=0x7f6b0400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:17.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b0a000640 1 -- 172.21.15.44:0/2043483344 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400ce40 con 0x7f6b140d2540 2024-09-13T08:38:17.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b0a000640 1 -- 172.21.15.44:0/2043483344 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b04004020 con 0x7f6b140d2540 2024-09-13T08:38:17.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b18a00640 1 -- 172.21.15.44:0/2043483344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b140b7350 con 0x7f6b140d2540 2024-09-13T08:38:17.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.538+0000 7f6b0a000640 1 -- 172.21.15.44:0/2043483344 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04004470 con 0x7f6b140d2540 2024-09-13T08:38:17.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.542+0000 7f6b18a00640 1 -- 172.21.15.44:0/2043483344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b140cb640 con 0x7f6b140d2540 2024-09-13T08:38:17.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.542+0000 7f6b18a00640 1 -- 172.21.15.44:0/2043483344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b140d29b0 con 0x7f6b140d2540 2024-09-13T08:38:17.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.542+0000 7f6b0a000640 1 -- 172.21.15.44:0/2043483344 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f6b04003600 con 0x7f6b140d2540 2024-09-13T08:38:17.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.542+0000 7f6b0a000640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af0072ec0 0x7f6af0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:17.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.542+0000 7f6b0a000640 1 -- 172.21.15.44:0/2043483344 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f6b040944b0 con 0x7f6b140d2540 2024-09-13T08:38:17.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.542+0000 7f6b0be00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af0072ec0 0x7f6af0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:17.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.546+0000 7f6b0be00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af0072ec0 0x7f6af0075380 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b140b6470 tx=0x7f6afc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:17.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.546+0000 7f6b0a000640 1 -- 172.21.15.44:0/2043483344 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6b0405f2e0 con 0x7f6b140d2540 2024-09-13T08:38:17.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:17.722+0000 7f6b18a00640 1 -- 172.21.15.44:0/2043483344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6b14162a60 con 0x7f6b140d2540 2024-09-13T08:38:18.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.166+0000 7f6b0a000640 1 -- 172.21.15.44:0/2043483344 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f6b04063d10 con 0x7f6b140d2540 2024-09-13T08:38:18.173 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T08:38:18.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 -- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af0072ec0 msgr2=0x7f6af0075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:18.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af0072ec0 0x7f6af0075380 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b140b6470 tx=0x7f6afc009040 comp rx=0 tx=0).stop 2024-09-13T08:38:18.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 -- 172.21.15.44:0/2043483344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b140d2540 msgr2=0x7f6b140cb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:18.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b140d2540 0x7f6b140cb060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400ea40 tx=0x7f6b0400ef00 comp rx=0 tx=0).stop 2024-09-13T08:38:18.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 -- 172.21.15.44:0/2043483344 shutdown_connections 2024-09-13T08:38:18.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af0072ec0 0x7f6af0075380 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:18.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1415e220 0x7f6b14150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:18.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1415de40 0x7f6b140cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:18.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 --2- 172.21.15.44:0/2043483344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b140d2540 0x7f6b140cb060 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:18.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 -- 172.21.15.44:0/2043483344 >> 172.21.15.44:0/2043483344 conn(0x7f6b140d3f90 msgr2=0x7f6b1406e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:18.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 -- 172.21.15.44:0/2043483344 shutdown_connections 2024-09-13T08:38:18.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.170+0000 7f6b18a00640 1 -- 172.21.15.44:0/2043483344 wait complete. 2024-09-13T08:38:18.202 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T08:38:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 -- 172.21.15.44:0/2623235888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fece8004810 msgr2=0x7fece8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 --2- 172.21.15.44:0/2623235888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fece8004810 0x7fece8004c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fecec00c2e0 tx=0x7fecec01f6e0 comp rx=0 tx=0).stop 2024-09-13T08:38:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed05a00640 1 -- 172.21.15.44:0/2623235888 reap_dead start 2024-09-13T08:38:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 -- 172.21.15.44:0/2623235888 shutdown_connections 2024-09-13T08:38:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 -- 172.21.15.44:0/2623235888 >> 172.21.15.44:0/2623235888 conn(0x7fed000d1dd0 msgr2=0x7fed000d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 -- 172.21.15.44:0/2623235888 shutdown_connections 2024-09-13T08:38:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 -- 172.21.15.44:0/2623235888 wait complete. 2024-09-13T08:38:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 Processor -- start 2024-09-13T08:38:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 -- start start 2024-09-13T08:38:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8004810 0x7fed001f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed001f6890 0x7fed000b7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed000b7a60 0x7fed0015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.354+0000 7fed07a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed000b0cd0 con 0x7fed000b7a60 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed07a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed000b06d0 con 0x7fece8004810 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed07a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fed000b0e50 con 0x7fed001f6890 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed05a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed000b7a60 0x7fed0015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed05000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8004810 0x7fed001f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed05a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed000b7a60 0x7fed0015b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40640/0 (socket says 172.21.15.44:40640) 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed05a00640 1 -- 172.21.15.44:0/2317130847 learned_addr learned my addr 172.21.15.44:0/2317130847 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fecffe00640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed001f6890 0x7fed000b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed05000640 1 -- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed001f6890 msgr2=0x7fed000b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed05000640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed001f6890 0x7fed000b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed05000640 1 -- 172.21.15.44:0/2317130847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed000b7a60 msgr2=0x7fed0015b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed05000640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed000b7a60 0x7fed0015b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed05000640 1 -- 172.21.15.44:0/2317130847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed000b4d30 con 0x7fece8004810 2024-09-13T08:38:18.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed05000640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8004810 0x7fed001f5ee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fecf40118b0 tx=0x7fecf4011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:18.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fecfcc00640 1 -- 172.21.15.44:0/2317130847 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf401e070 con 0x7fece8004810 2024-09-13T08:38:18.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fecfcc00640 1 -- 172.21.15.44:0/2317130847 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecf400f9a0 con 0x7fece8004810 2024-09-13T08:38:18.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed07a00640 1 -- 172.21.15.44:0/2317130847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed000b5070 con 0x7fece8004810 2024-09-13T08:38:18.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fecfcc00640 1 -- 172.21.15.44:0/2317130847 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf400fdf0 con 0x7fece8004810 2024-09-13T08:38:18.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed07a00640 1 -- 172.21.15.44:0/2317130847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed000b58b0 con 0x7fece8004810 2024-09-13T08:38:18.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fed07a00640 1 -- 172.21.15.44:0/2317130847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed000affc0 con 0x7fece8004810 2024-09-13T08:38:18.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.358+0000 7fecfcc00640 1 -- 172.21.15.44:0/2317130847 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fecf4021070 con 0x7fece8004810 2024-09-13T08:38:18.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.362+0000 7fecfcc00640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0072ec0 0x7fece0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.362+0000 7fecfcc00640 1 -- 172.21.15.44:0/2317130847 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7fecf4094eb0 con 0x7fece8004810 2024-09-13T08:38:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.362+0000 7fecffe00640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0072ec0 0x7fece0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.362+0000 7fecffe00640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0072ec0 0x7fece0075380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fecec0b0000 tx=0x7fecec01c520 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.366+0000 7fecfcc00640 1 -- 172.21.15.44:0/2317130847 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fecf40c4060 con 0x7fece8004810 2024-09-13T08:38:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:18.542+0000 7fed07a00640 1 -- 172.21.15.44:0/2317130847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fed000d1080 con 0x7fece8004810 2024-09-13T08:38:19.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.182+0000 7fecfcc00640 1 -- 172.21.15.44:0/2317130847 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7fed000b5070 con 0x7fece8004810 2024-09-13T08:38:19.189 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T08:38:19.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 -- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0072ec0 msgr2=0x7fece0075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:19.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0072ec0 0x7fece0075380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fecec0b0000 tx=0x7fecec01c520 comp rx=0 tx=0).stop 2024-09-13T08:38:19.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 -- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8004810 msgr2=0x7fed001f5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:19.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8004810 0x7fed001f5ee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fecf40118b0 tx=0x7fecf4011d70 comp rx=0 tx=0).stop 2024-09-13T08:38:19.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 -- 172.21.15.44:0/2317130847 shutdown_connections 2024-09-13T08:38:19.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0072ec0 0x7fece0075380 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:19.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed000b7a60 0x7fed0015b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:19.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed001f6890 0x7fed000b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:19.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 --2- 172.21.15.44:0/2317130847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8004810 0x7fed001f5ee0 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fecf40118b0 tx=0x7fecf4011d70 comp rx=0 tx=0).stop 2024-09-13T08:38:19.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 -- 172.21.15.44:0/2317130847 >> 172.21.15.44:0/2317130847 conn(0x7fed000d1dd0 msgr2=0x7fed000cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:19.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.190+0000 7fed07a00640 1 -- 172.21.15.44:0/2317130847 shutdown_connections 2024-09-13T08:38:19.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.194+0000 7fed07a00640 1 -- 172.21.15.44:0/2317130847 wait complete. 2024-09-13T08:38:19.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T08:38:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.378+0000 7f6657000640 1 -- 172.21.15.44:0/4248998602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6634003fe0 msgr2=0x7f6634004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.378+0000 7f6657000640 1 --2- 172.21.15.44:0/4248998602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6634003fe0 0x7f6634004810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f663c00c4d0 tx=0x7f663c01f720 comp rx=0 tx=0).stop 2024-09-13T08:38:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.378+0000 7f6655000640 1 -- 172.21.15.44:0/4248998602 reap_dead start 2024-09-13T08:38:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.378+0000 7f6657000640 1 -- 172.21.15.44:0/4248998602 shutdown_connections 2024-09-13T08:38:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.378+0000 7f6657000640 1 -- 172.21.15.44:0/4248998602 >> 172.21.15.44:0/4248998602 conn(0x7f66500d1e00 msgr2=0x7f66500d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.378+0000 7f6657000640 1 -- 172.21.15.44:0/4248998602 shutdown_connections 2024-09-13T08:38:19.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.378+0000 7f6657000640 1 -- 172.21.15.44:0/4248998602 wait complete. 2024-09-13T08:38:19.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6657000640 1 Processor -- start 2024-09-13T08:38:19.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6657000640 1 -- start start 2024-09-13T08:38:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6657000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6634003fe0 0x7f66500aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6657000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500b1830 0x7f66500c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6657000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66500b1cb0 0x7f665015e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6657000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66500b24b0 con 0x7f66500b1cb0 2024-09-13T08:38:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6657000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66500b2330 con 0x7f66500b1830 2024-09-13T08:38:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6657000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f66500b2630 con 0x7f6634003fe0 2024-09-13T08:38:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6655000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66500b1cb0 0x7f665015e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6655000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66500b1cb0 0x7f665015e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40654/0 (socket says 172.21.15.44:40654) 2024-09-13T08:38:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6655000640 1 -- 172.21.15.44:0/866809847 learned_addr learned my addr 172.21.15.44:0/866809847 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:19.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6655000640 1 -- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6634003fe0 msgr2=0x7f66500aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6655000640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6634003fe0 0x7f66500aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6655000640 1 -- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500b1830 msgr2=0x7f66500c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6655000640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500b1830 0x7f66500c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6655000640 1 -- 172.21.15.44:0/866809847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66500b2150 con 0x7f66500b1cb0 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6655000640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66500b1cb0 0x7f665015e0d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6650073a90 tx=0x7f6648010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6647e00640 1 -- 172.21.15.44:0/866809847 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664800ee10 con 0x7f66500b1cb0 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6647e00640 1 -- 172.21.15.44:0/866809847 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6648015070 con 0x7f66500b1cb0 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6647e00640 1 -- 172.21.15.44:0/866809847 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648003d10 con 0x7f66500b1cb0 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6657000640 1 -- 172.21.15.44:0/866809847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66500c8f70 con 0x7f66500b1cb0 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.382+0000 7f6657000640 1 -- 172.21.15.44:0/866809847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66500c9530 con 0x7f66500b1cb0 2024-09-13T08:38:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.386+0000 7f6657000640 1 -- 172.21.15.44:0/866809847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66500c9960 con 0x7f66500b1cb0 2024-09-13T08:38:19.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.386+0000 7f6647e00640 1 -- 172.21.15.44:0/866809847 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f6648004030 con 0x7f66500b1cb0 2024-09-13T08:38:19.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.386+0000 7f6647e00640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6630072ec0 0x7f6630075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:19.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.386+0000 7f6647e00640 1 -- 172.21.15.44:0/866809847 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f66480943c0 con 0x7f66500b1cb0 2024-09-13T08:38:19.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.386+0000 7f664fe00640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6630072ec0 0x7f6630075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:19.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.386+0000 7f664fe00640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6630072ec0 0x7f6630075380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6640009bd0 tx=0x7f6640009690 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:19.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.390+0000 7f6647e00640 1 -- 172.21.15.44:0/866809847 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f66500b2150 con 0x7f66500b1cb0 2024-09-13T08:38:19.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:19.602+0000 7f6657000640 1 -- 172.21.15.44:0/866809847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f66500cc350 con 0x7f66500b1cb0 2024-09-13T08:38:20.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.178+0000 7f6647e00640 1 -- 172.21.15.44:0/866809847 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f664805f660 con 0x7f66500b1cb0 2024-09-13T08:38:20.183 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 4 and data pool 5 2024-09-13T08:38:20.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 -- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6630072ec0 msgr2=0x7f6630075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6630072ec0 0x7f6630075380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6640009bd0 tx=0x7f6640009690 comp rx=0 tx=0).stop 2024-09-13T08:38:20.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 -- 172.21.15.44:0/866809847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66500b1cb0 msgr2=0x7f665015e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66500b1cb0 0x7f665015e0d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6650073a90 tx=0x7f6648010e90 comp rx=0 tx=0).stop 2024-09-13T08:38:20.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 -- 172.21.15.44:0/866809847 shutdown_connections 2024-09-13T08:38:20.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6630072ec0 0x7f6630075380 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66500b1cb0 0x7f665015e0d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500b1830 0x7f66500c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 --2- 172.21.15.44:0/866809847 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6634003fe0 0x7f66500aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 -- 172.21.15.44:0/866809847 >> 172.21.15.44:0/866809847 conn(0x7f66500d1e00 msgr2=0x7f665006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:20.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.182+0000 7f6657000640 1 -- 172.21.15.44:0/866809847 shutdown_connections 2024-09-13T08:38:20.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.186+0000 7f6657000640 1 -- 172.21.15.44:0/866809847 wait complete. 2024-09-13T08:38:20.240 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:38:20.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.382+0000 7f03d2a00640 1 -- 172.21.15.44:0/3426096194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03cc15bea0 msgr2=0x7f03cc0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.382+0000 7f03d2a00640 1 --2- 172.21.15.44:0/3426096194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03cc15bea0 0x7f03cc0ce670 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f03cc073d10 tx=0x7f03bc01c830 comp rx=0 tx=0).stop 2024-09-13T08:38:20.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.382+0000 7f03d2a00640 1 -- 172.21.15.44:0/3426096194 shutdown_connections 2024-09-13T08:38:20.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.382+0000 7f03d2a00640 1 --2- 172.21.15.44:0/3426096194 >> v2:172.21.15.129:3300/0 conn(0x7f03cc0cdb30 0x7f03cc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.382+0000 7f03d2a00640 1 --2- 172.21.15.44:0/3426096194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03cc15bea0 0x7f03cc0ce670 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.382+0000 7f03d2a00640 1 -- 172.21.15.44:0/3426096194 >> 172.21.15.44:0/3426096194 conn(0x7f03cc0d1dc0 msgr2=0x7f03cc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:20.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.382+0000 7f03d2a00640 1 -- 172.21.15.44:0/3426096194 shutdown_connections 2024-09-13T08:38:20.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.382+0000 7f03d2a00640 1 -- 172.21.15.44:0/3426096194 wait complete. 2024-09-13T08:38:20.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 Processor -- start 2024-09-13T08:38:20.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 -- start start 2024-09-13T08:38:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03cc0cdb30 0x7f03cc15a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc0d28e0 0x7f03cc144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f03cc15bea0 0x7f03cc0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc168fc0 con 0x7f03cc0cdb30 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc168e40 con 0x7f03cc0d28e0 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f03cc169140 con 0x7f03cc15bea0 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03cc0cdb30 0x7f03cc15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03cc0cdb30 0x7f03cc15a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40670/0 (socket says 172.21.15.44:40670) 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03cbe00640 1 -- 172.21.15.44:0/2731021750 learned_addr learned my addr 172.21.15.44:0/2731021750 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03c3e00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc0d28e0 0x7f03cc144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d0a00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f03cc15bea0 0x7f03cc0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03cbe00640 1 -- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f03cc15bea0 msgr2=0x7f03cc0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03cbe00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f03cc15bea0 0x7f03cc0b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03cbe00640 1 -- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc0d28e0 msgr2=0x7f03cc144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03cbe00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc0d28e0 0x7f03cc144d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03cbe00640 1 -- 172.21.15.44:0/2731021750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc15a8a0 con 0x7f03cc0cdb30 2024-09-13T08:38:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03c3e00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc0d28e0 0x7f03cc144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03cbe00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03cc0cdb30 0x7f03cc15a360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f03bc019440 tx=0x7f03bc0a5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03c8c00640 1 -- 172.21.15.44:0/2731021750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0b2040 con 0x7f03cc0cdb30 2024-09-13T08:38:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03c8c00640 1 -- 172.21.15.44:0/2731021750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03bc0042c0 con 0x7f03cc0cdb30 2024-09-13T08:38:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 -- 172.21.15.44:0/2731021750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc147970 con 0x7f03cc0cdb30 2024-09-13T08:38:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03c8c00640 1 -- 172.21.15.44:0/2731021750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc005290 con 0x7f03cc0cdb30 2024-09-13T08:38:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 -- 172.21.15.44:0/2731021750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc147f30 con 0x7f03cc0cdb30 2024-09-13T08:38:20.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.386+0000 7f03d2a00640 1 -- 172.21.15.44:0/2731021750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03cc1485f0 con 0x7f03cc0cdb30 2024-09-13T08:38:20.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.390+0000 7f03c8c00640 1 -- 172.21.15.44:0/2731021750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f03bc005430 con 0x7f03cc0cdb30 2024-09-13T08:38:20.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.390+0000 7f03c8c00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03a8072ec0 0x7f03a8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.390+0000 7f03c8c00640 1 -- 172.21.15.44:0/2731021750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f03bc12dd60 con 0x7f03cc0cdb30 2024-09-13T08:38:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.390+0000 7f03c3e00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03a8072ec0 0x7f03a8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.390+0000 7f03c3e00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03a8072ec0 0x7f03a8075380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f03b40058d0 tx=0x7f03b4005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:20.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.394+0000 7f03c8c00640 1 -- 172.21.15.44:0/2731021750 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f03cc1485f0 con 0x7f03cc0cdb30 2024-09-13T08:38:20.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 -- 172.21.15.44:0/2731021750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03cc0cc9d0 con 0x7f03cc0cdb30 2024-09-13T08:38:20.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03c8c00640 1 -- 172.21.15.44:0/2731021750 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16694 (secure 0 0 0) 0x7f03bc0c4060 con 0x7f03cc0cdb30 2024-09-13T08:38:20.579 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:20.579 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":32,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:38:19.608834+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:38:17.738159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T08:38:18.557711+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:38:20.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 -- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03a8072ec0 msgr2=0x7f03a8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03a8072ec0 0x7f03a8075380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f03b40058d0 tx=0x7f03b4005860 comp rx=0 tx=0).stop 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 -- 172.21.15.44:0/2731021750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03cc0cdb30 msgr2=0x7f03cc15a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03cc0cdb30 0x7f03cc15a360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f03bc019440 tx=0x7f03bc0a5680 comp rx=0 tx=0).stop 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 -- 172.21.15.44:0/2731021750 shutdown_connections 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03a8072ec0 0x7f03a8075380 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f03cc15bea0 0x7f03cc0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc0d28e0 0x7f03cc144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 --2- 172.21.15.44:0/2731021750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03cc0cdb30 0x7f03cc15a360 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 -- 172.21.15.44:0/2731021750 >> 172.21.15.44:0/2731021750 conn(0x7f03cc0d1dc0 msgr2=0x7f03cc06f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 -- 172.21.15.44:0/2731021750 shutdown_connections 2024-09-13T08:38:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.574+0000 7f03d2a00640 1 -- 172.21.15.44:0/2731021750 wait complete. 2024-09-13T08:38:20.601 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:38:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.742+0000 7f22a6200640 1 -- 172.21.15.44:0/3921148644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22a00ced80 msgr2=0x7f22a00ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.742+0000 7f22a6200640 1 --2- 172.21.15.44:0/3921148644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22a00ced80 0x7f22a00ae9b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2288009870 tx=0x7f228801c860 comp rx=0 tx=0).stop 2024-09-13T08:38:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.742+0000 7f22a6200640 1 -- 172.21.15.44:0/3921148644 shutdown_connections 2024-09-13T08:38:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.742+0000 7f22a6200640 1 --2- 172.21.15.44:0/3921148644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22a00ced80 0x7f22a00ae9b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.742+0000 7f22a6200640 1 -- 172.21.15.44:0/3921148644 >> 172.21.15.44:0/3921148644 conn(0x7f22a00d1dc0 msgr2=0x7f22a00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.742+0000 7f22a6200640 1 -- 172.21.15.44:0/3921148644 shutdown_connections 2024-09-13T08:38:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.742+0000 7f22a6200640 1 -- 172.21.15.44:0/3921148644 wait complete. 2024-09-13T08:38:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.742+0000 7f22a6200640 1 Processor -- start 2024-09-13T08:38:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f22a6200640 1 -- start start 2024-09-13T08:38:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f22a6200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22a00cdb30 0x7f22a01fb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f22a6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a00ced80 0x7f22a014d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f22a6200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22a015bea0 0x7f22a01568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f22a6200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22a016a8a0 con 0x7f22a00ced80 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f22a6200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f22a016a720 con 0x7f22a015bea0 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f22a6200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f22a016aa20 con 0x7f22a00cdb30 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a00ced80 0x7f22a014d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a00ced80 0x7f22a014d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40674/0 (socket says 172.21.15.44:40674) 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297e00640 1 -- 172.21.15.44:0/504659494 learned_addr learned my addr 172.21.15.44:0/504659494 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f229f400640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22a00cdb30 0x7f22a01fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f229fe00640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22a015bea0 0x7f22a01568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297e00640 1 -- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22a00cdb30 msgr2=0x7f22a01fb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297e00640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22a00cdb30 0x7f22a01fb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297e00640 1 -- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22a015bea0 msgr2=0x7f22a01568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297e00640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22a015bea0 0x7f22a01568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297e00640 1 -- 172.21.15.44:0/504659494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a01fbb40 con 0x7f22a00ced80 2024-09-13T08:38:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297e00640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a00ced80 0x7f22a014d930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2288000c10 tx=0x7f22880a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297400640 1 -- 172.21.15.44:0/504659494 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22880bb070 con 0x7f22a00ced80 2024-09-13T08:38:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297400640 1 -- 172.21.15.44:0/504659494 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2288004470 con 0x7f22a00ced80 2024-09-13T08:38:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f2297400640 1 -- 172.21.15.44:0/504659494 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22880b2040 con 0x7f22a00ced80 2024-09-13T08:38:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f22a6200640 1 -- 172.21.15.44:0/504659494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a00b68f0 con 0x7f22a00ced80 2024-09-13T08:38:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f22a6200640 1 -- 172.21.15.44:0/504659494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a00b6eb0 con 0x7f22a00ced80 2024-09-13T08:38:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.746+0000 7f22a6200640 1 -- 172.21.15.44:0/504659494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a00ce670 con 0x7f22a00ced80 2024-09-13T08:38:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.750+0000 7f2297400640 1 -- 172.21.15.44:0/504659494 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f2288005290 con 0x7f22a00ced80 2024-09-13T08:38:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.750+0000 7f2297400640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f227c072f20 0x7f227c0753e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.750+0000 7f2297400640 1 -- 172.21.15.44:0/504659494 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f228812d6c0 con 0x7f22a00ced80 2024-09-13T08:38:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.750+0000 7f229f400640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f227c072f20 0x7f227c0753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.750+0000 7f229f400640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f227c072f20 0x7f227c0753e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2290001640 tx=0x7f2290009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:20.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.754+0000 7f2297400640 1 -- 172.21.15.44:0/504659494 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2288016450 con 0x7f22a00ced80 2024-09-13T08:38:20.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.930+0000 7f22a6200640 1 -- 172.21.15.44:0/504659494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22a0072780 con 0x7f22a00ced80 2024-09-13T08:38:20.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.930+0000 7f2297400640 1 -- 172.21.15.44:0/504659494 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16694 (secure 0 0 0) 0x7f22880f8920 con 0x7f22a00ced80 2024-09-13T08:38:20.938 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:20.938 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":32,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:38:19.608834+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:38:17.738159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T08:38:18.557711+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:38:20.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 -- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f227c072f20 msgr2=0x7f227c0753e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f227c072f20 0x7f227c0753e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2290001640 tx=0x7f2290009040 comp rx=0 tx=0).stop 2024-09-13T08:38:20.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 -- 172.21.15.44:0/504659494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a00ced80 msgr2=0x7f22a014d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:20.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a00ced80 0x7f22a014d930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2288000c10 tx=0x7f22880a56b0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 -- 172.21.15.44:0/504659494 shutdown_connections 2024-09-13T08:38:20.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f227c072f20 0x7f227c0753e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22a015bea0 0x7f22a01568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a00ced80 0x7f22a014d930 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 --2- 172.21.15.44:0/504659494 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22a00cdb30 0x7f22a01fb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:20.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 -- 172.21.15.44:0/504659494 >> 172.21.15.44:0/504659494 conn(0x7f22a00d1dc0 msgr2=0x7f22a00d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:20.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 -- 172.21.15.44:0/504659494 shutdown_connections 2024-09-13T08:38:20.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:20.934+0000 7f22a6200640 1 -- 172.21.15.44:0/504659494 wait complete. 2024-09-13T08:38:20.961 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T08:38:21.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.106+0000 7fcf0ce00640 1 -- 172.21.15.44:0/1777304408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf080afb10 msgr2=0x7fcf080affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:21.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.106+0000 7fcf0ce00640 1 --2- 172.21.15.44:0/1777304408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf080afb10 0x7fcf080affc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcf08073810 tx=0x7fcf0001cb80 comp rx=0 tx=0).stop 2024-09-13T08:38:21.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.106+0000 7fcf0ce00640 1 -- 172.21.15.44:0/1777304408 shutdown_connections 2024-09-13T08:38:21.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.106+0000 7fcf0ce00640 1 --2- 172.21.15.44:0/1777304408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf080afb10 0x7fcf080affc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:21.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.106+0000 7fcf0ce00640 1 --2- 172.21.15.44:0/1777304408 >> v2:172.21.15.129:3300/0 conn(0x7fcf081605d0 0x7fcf080d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:21.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.106+0000 7fcf0ce00640 1 -- 172.21.15.44:0/1777304408 >> 172.21.15.44:0/1777304408 conn(0x7fcf080d1dd0 msgr2=0x7fcf080d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:21.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.106+0000 7fcf0ce00640 1 -- 172.21.15.44:0/1777304408 shutdown_connections 2024-09-13T08:38:21.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.106+0000 7fcf0ce00640 1 -- 172.21.15.44:0/1777304408 wait complete. 2024-09-13T08:38:21.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 Processor -- start 2024-09-13T08:38:21.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 -- start start 2024-09-13T08:38:21.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf080afb10 0x7fcf08151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:21.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcf080b1650 0x7fcf081f9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:21.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf081605d0 0x7fcf0814c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcf08169540 con 0x7fcf080afb10 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf080b1430 con 0x7fcf081605d0 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcf081696c0 con 0x7fcf080b1650 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf05e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf080afb10 0x7fcf08151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf05e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf080afb10 0x7fcf08151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40700/0 (socket says 172.21.15.44:40700) 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf05e00640 1 -- 172.21.15.44:0/2292366174 learned_addr learned my addr 172.21.15.44:0/2292366174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf06800640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf081605d0 0x7fcf0814c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcefd400640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcf080b1650 0x7fcf081f9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf05e00640 1 -- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcf080b1650 msgr2=0x7fcf081f9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf05e00640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcf080b1650 0x7fcf081f9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf05e00640 1 -- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf081605d0 msgr2=0x7fcf0814c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf05e00640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf081605d0 0x7fcf0814c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:21.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf05e00640 1 -- 172.21.15.44:0/2292366174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0814ce70 con 0x7fcf080afb10 2024-09-13T08:38:21.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf06800640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf081605d0 0x7fcf0814c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:21.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf05e00640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf080afb10 0x7fcf08151540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcef800d950 tx=0x7fcef800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:21.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcefea00640 1 -- 172.21.15.44:0/2292366174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8017070 con 0x7fcf080afb10 2024-09-13T08:38:21.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcefea00640 1 -- 172.21.15.44:0/2292366174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcef8009d20 con 0x7fcf080afb10 2024-09-13T08:38:21.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 -- 172.21.15.44:0/2292366174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf08150780 con 0x7fcf080afb10 2024-09-13T08:38:21.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcefea00640 1 -- 172.21.15.44:0/2292366174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef80106c0 con 0x7fcf080afb10 2024-09-13T08:38:21.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 -- 172.21.15.44:0/2292366174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf08150d40 con 0x7fcf080afb10 2024-09-13T08:38:21.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.110+0000 7fcf0ce00640 1 -- 172.21.15.44:0/2292366174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf080affc0 con 0x7fcf080afb10 2024-09-13T08:38:21.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.114+0000 7fcefea00640 1 -- 172.21.15.44:0/2292366174 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fcef801a070 con 0x7fcf080afb10 2024-09-13T08:38:21.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.114+0000 7fcefea00640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcee4072ef0 0x7fcee40753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:21.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.114+0000 7fcefea00640 1 -- 172.21.15.44:0/2292366174 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fcef8093ff0 con 0x7fcf080afb10 2024-09-13T08:38:21.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.114+0000 7fcefd400640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcee4072ef0 0x7fcee40753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:21.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.114+0000 7fcefd400640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcee4072ef0 0x7fcee40753b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcf08151de0 tx=0x7fcef0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:21.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.118+0000 7fcefea00640 1 -- 172.21.15.44:0/2292366174 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcef805f140 con 0x7fcf080afb10 2024-09-13T08:38:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:21.342+0000 7fcf0ce00640 1 -- 172.21.15.44:0/2292366174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcf080d1080 con 0x7fcf080afb10 2024-09-13T08:38:22.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.206+0000 7fcefea00640 1 -- 172.21.15.44:0/2292366174 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fcef805ea40 con 0x7fcf080afb10 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 -- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcee4072ef0 msgr2=0x7fcee40753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcee4072ef0 0x7fcee40753b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcf08151de0 tx=0x7fcef0009040 comp rx=0 tx=0).stop 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 -- 172.21.15.44:0/2292366174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf080afb10 msgr2=0x7fcf08151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf080afb10 0x7fcf08151540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcef800d950 tx=0x7fcef800de10 comp rx=0 tx=0).stop 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 -- 172.21.15.44:0/2292366174 shutdown_connections 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcee4072ef0 0x7fcee40753b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf081605d0 0x7fcf0814c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcf080b1650 0x7fcf081f9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 --2- 172.21.15.44:0/2292366174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf080afb10 0x7fcf08151540 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 -- 172.21.15.44:0/2292366174 >> 172.21.15.44:0/2292366174 conn(0x7fcf080d1dd0 msgr2=0x7fcf08070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 -- 172.21.15.44:0/2292366174 shutdown_connections 2024-09-13T08:38:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.210+0000 7fcf0ce00640 1 -- 172.21.15.44:0/2292366174 wait complete. 2024-09-13T08:38:22.240 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T08:38:22.241 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T08:38:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 -- 172.21.15.44:0/2455855850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401415bd50 msgr2=0x7f40140ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 --2- 172.21.15.44:0/2455855850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401415bd50 0x7f40140ce5e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4004009900 tx=0x7f400401c7c0 comp rx=0 tx=0).stop 2024-09-13T08:38:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 -- 172.21.15.44:0/2455855850 shutdown_connections 2024-09-13T08:38:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 --2- 172.21.15.44:0/2455855850 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f401415c130 0x7f40140ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 --2- 172.21.15.44:0/2455855850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401415bd50 0x7f40140ce5e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 -- 172.21.15.44:0/2455855850 >> 172.21.15.44:0/2455855850 conn(0x7f40140d1ea0 msgr2=0x7f40140d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:22.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 -- 172.21.15.44:0/2455855850 shutdown_connections 2024-09-13T08:38:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 -- 172.21.15.44:0/2455855850 wait complete. 2024-09-13T08:38:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 Processor -- start 2024-09-13T08:38:22.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 -- start start 2024-09-13T08:38:22.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f40140cf4e0 0x7f40141578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:22.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f401415bd50 0x7f40140cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:22.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401415c130 0x7f40140cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:22.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4014168e50 con 0x7f401415c130 2024-09-13T08:38:22.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4014168cd0 con 0x7f401415bd50 2024-09-13T08:38:22.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f401b200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4014168fd0 con 0x7f40140cf4e0 2024-09-13T08:38:22.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f4019200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401415c130 0x7f40140cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:22.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f4019200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401415c130 0x7f40140cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55934/0 (socket says 172.21.15.44:55934) 2024-09-13T08:38:22.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f4019200640 1 -- 172.21.15.44:0/1008731832 learned_addr learned my addr 172.21.15.44:0/1008731832 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:22.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f4013e00640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f40140cf4e0 0x7f40141578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:22.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.390+0000 7f4013400640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f401415bd50 0x7f40140cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:22.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f4019200640 1 -- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f40140cf4e0 msgr2=0x7f40141578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:22.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f4019200640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f40140cf4e0 0x7f40141578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:22.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f4019200640 1 -- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f401415bd50 msgr2=0x7f40140cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:22.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f4019200640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f401415bd50 0x7f40140cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:22.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f4019200640 1 -- 172.21.15.44:0/1008731832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4014159590 con 0x7f401415c130 2024-09-13T08:38:22.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f4013400640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f401415bd50 0x7f40140cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:22.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f4019200640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401415c130 0x7f40140cba60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f400c00c7c0 tx=0x7f400c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:22.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f400be00640 1 -- 172.21.15.44:0/1008731832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c003b80 con 0x7f401415c130 2024-09-13T08:38:22.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f400be00640 1 -- 172.21.15.44:0/1008731832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f400c003d20 con 0x7f401415c130 2024-09-13T08:38:22.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f400be00640 1 -- 172.21.15.44:0/1008731832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c00aab0 con 0x7f401415c130 2024-09-13T08:38:22.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f401b200640 1 -- 172.21.15.44:0/1008731832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4014157e10 con 0x7f401415c130 2024-09-13T08:38:22.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f401b200640 1 -- 172.21.15.44:0/1008731832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40140b5e90 con 0x7f401415c130 2024-09-13T08:38:22.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f401b200640 1 -- 172.21.15.44:0/1008731832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40140c5dd0 con 0x7f401415c130 2024-09-13T08:38:22.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f400be00640 1 -- 172.21.15.44:0/1008731832 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f400c00ac50 con 0x7f401415c130 2024-09-13T08:38:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.394+0000 7f400be00640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ff4072ec0 0x7f3ff4075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.398+0000 7f400be00640 1 -- 172.21.15.44:0/1008731832 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f400c093c70 con 0x7f401415c130 2024-09-13T08:38:22.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.398+0000 7f4013e00640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ff4072ec0 0x7f3ff4075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:22.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.398+0000 7f4013e00640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ff4072ec0 0x7f3ff4075380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f40040af000 tx=0x7f400401b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:22.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.402+0000 7f400be00640 1 -- 172.21.15.44:0/1008731832 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f400c05eeb0 con 0x7f401415c130 2024-09-13T08:38:22.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:22.614+0000 7f401b200640 1 -- 172.21.15.44:0/1008731832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4014076360 con 0x7f401415c130 2024-09-13T08:38:23.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f400be00640 1 -- 172.21.15.44:0/1008731832 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f400c05e7b0 con 0x7f401415c130 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 -- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ff4072ec0 msgr2=0x7f3ff4075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ff4072ec0 0x7f3ff4075380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f40040af000 tx=0x7f400401b000 comp rx=0 tx=0).stop 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 -- 172.21.15.44:0/1008731832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401415c130 msgr2=0x7f40140cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401415c130 0x7f40140cba60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f400c00c7c0 tx=0x7f400c00cc80 comp rx=0 tx=0).stop 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 -- 172.21.15.44:0/1008731832 shutdown_connections 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ff4072ec0 0x7f3ff4075380 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401415c130 0x7f40140cba60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f401415bd50 0x7f40140cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 --2- 172.21.15.44:0/1008731832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f40140cf4e0 0x7f40141578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 -- 172.21.15.44:0/1008731832 >> 172.21.15.44:0/1008731832 conn(0x7f40140d1ea0 msgr2=0x7f4014071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 -- 172.21.15.44:0/1008731832 shutdown_connections 2024-09-13T08:38:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.214+0000 7f401b200640 1 -- 172.21.15.44:0/1008731832 wait complete. 2024-09-13T08:38:23.244 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T08:38:23.245 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:38:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.386+0000 7fe629e00640 1 -- 172.21.15.44:0/1199172020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6240ced80 msgr2=0x7fe6240ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.386+0000 7fe629e00640 1 --2- 172.21.15.44:0/1199172020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6240ced80 0x7fe6240ae9b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009900 tx=0x7fe60c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 -- 172.21.15.44:0/1199172020 shutdown_connections 2024-09-13T08:38:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 --2- 172.21.15.44:0/1199172020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6240ced80 0x7fe6240ae9b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 -- 172.21.15.44:0/1199172020 >> 172.21.15.44:0/1199172020 conn(0x7fe6240d1dc0 msgr2=0x7fe6240d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:23.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 -- 172.21.15.44:0/1199172020 shutdown_connections 2024-09-13T08:38:23.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 -- 172.21.15.44:0/1199172020 wait complete. 2024-09-13T08:38:23.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 Processor -- start 2024-09-13T08:38:23.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 -- start start 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6240cdb30 0x7fe6241548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6240ced80 0x7fe624158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe62415bea0 0x7fe624154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe62416a8a0 con 0x7fe6240cdb30 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe62416a720 con 0x7fe6240ced80 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe62416aa20 con 0x7fe62415bea0 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe622a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6240cdb30 0x7fe6241548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe622a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6240cdb30 0x7fe6241548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55956/0 (socket says 172.21.15.44:55956) 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe622a00640 1 -- 172.21.15.44:0/3605772855 learned_addr learned my addr 172.21.15.44:0/3605772855 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe61be00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6240ced80 0x7fe624158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe623400640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe62415bea0 0x7fe624154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe622a00640 1 -- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe62415bea0 msgr2=0x7fe624154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe622a00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe62415bea0 0x7fe624154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe622a00640 1 -- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6240ced80 msgr2=0x7fe624158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe622a00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6240ced80 0x7fe624158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe622a00640 1 -- 172.21.15.44:0/3605772855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6240b1520 con 0x7fe6240cdb30 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe623400640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe62415bea0 0x7fe624154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe622a00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6240cdb30 0x7fe6241548f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c3c0 tx=0x7fe61400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe61b400640 1 -- 172.21.15.44:0/3605772855 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614017070 con 0x7fe6240cdb30 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe61b400640 1 -- 172.21.15.44:0/3605772855 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe614003600 con 0x7fe6240cdb30 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe61b400640 1 -- 172.21.15.44:0/3605772855 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614003a50 con 0x7fe6240cdb30 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.390+0000 7fe629e00640 1 -- 172.21.15.44:0/3605772855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6240b1800 con 0x7fe6240cdb30 2024-09-13T08:38:23.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.394+0000 7fe629e00640 1 -- 172.21.15.44:0/3605772855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6240b1dc0 con 0x7fe6240cdb30 2024-09-13T08:38:23.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.394+0000 7fe629e00640 1 -- 172.21.15.44:0/3605772855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6240ce670 con 0x7fe6240cdb30 2024-09-13T08:38:23.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.394+0000 7fe61b400640 1 -- 172.21.15.44:0/3605772855 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fe61401a070 con 0x7fe6240cdb30 2024-09-13T08:38:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.394+0000 7fe61b400640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe600072ec0 0x7fe600075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.394+0000 7fe61b400640 1 -- 172.21.15.44:0/3605772855 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe614093090 con 0x7fe6240cdb30 2024-09-13T08:38:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.394+0000 7fe61be00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe600072ec0 0x7fe600075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:23.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.398+0000 7fe61be00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe600072ec0 0x7fe600075380 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0af000 tx=0x7fe60c01b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.398+0000 7fe61b400640 1 -- 172.21.15.44:0/3605772855 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe61405db60 con 0x7fe6240cdb30 2024-09-13T08:38:23.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 -- 172.21.15.44:0/3605772855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6240d2240 con 0x7fe6240cdb30 2024-09-13T08:38:23.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe61b400640 1 -- 172.21.15.44:0/3605772855 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7fe614062590 con 0x7fe6240cdb30 2024-09-13T08:38:23.616 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:23.616 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"btime":"2024-09-13T08:38:23:214240+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4828,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/238336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":238336703},{"type":"v1","addr":"172.21.15.129:6836","nonce":238336703}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2440287716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2440287716},{"type":"v1","addr":"172.21.15.44:6834","nonce":2440287716}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3117701056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":3117701056},{"type":"v1","addr":"172.21.15.129:6837","nonce":3117701056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:38:19.608778+0000","modified":"2024-09-13T08:38:22.619955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4866":{"gid":4866,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3820157165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3820157165},{"type":"v1","addr":"172.21.15.44:6835","nonce":3820157165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4866,"qdb_cluster":[4866]},"id":2}]} 2024-09-13T08:38:23.616 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-09-13T08:38:23.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 -- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe600072ec0 msgr2=0x7fe600075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe600072ec0 0x7fe600075380 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0af000 tx=0x7fe60c01b120 comp rx=0 tx=0).stop 2024-09-13T08:38:23.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 -- 172.21.15.44:0/3605772855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6240cdb30 msgr2=0x7fe6241548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6240cdb30 0x7fe6241548f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c3c0 tx=0x7fe61400c880 comp rx=0 tx=0).stop 2024-09-13T08:38:23.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 -- 172.21.15.44:0/3605772855 shutdown_connections 2024-09-13T08:38:23.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe600072ec0 0x7fe600075380 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe62415bea0 0x7fe624154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6240ced80 0x7fe624158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 --2- 172.21.15.44:0/3605772855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6240cdb30 0x7fe6241548f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 -- 172.21.15.44:0/3605772855 >> 172.21.15.44:0/3605772855 conn(0x7fe6240d1dc0 msgr2=0x7fe62406e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:23.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 -- 172.21.15.44:0/3605772855 shutdown_connections 2024-09-13T08:38:23.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.610+0000 7fe629e00640 1 -- 172.21.15.44:0/3605772855 wait complete. 2024-09-13T08:38:23.638 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:38:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.778+0000 7fad6a200640 1 -- 172.21.15.44:0/2135029497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad640cdb30 msgr2=0x7fad640cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.778+0000 7fad6a200640 1 --2- 172.21.15.44:0/2135029497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad640cdb30 0x7fad640cebb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009900 tx=0x7fad4c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.778+0000 7fad6a200640 1 -- 172.21.15.44:0/2135029497 shutdown_connections 2024-09-13T08:38:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.778+0000 7fad6a200640 1 --2- 172.21.15.44:0/2135029497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad640cdb30 0x7fad640cebb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.778+0000 7fad6a200640 1 --2- 172.21.15.44:0/2135029497 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad6415bea0 0x7fad640ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.778+0000 7fad6a200640 1 -- 172.21.15.44:0/2135029497 >> 172.21.15.44:0/2135029497 conn(0x7fad640d1dc0 msgr2=0x7fad640d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.778+0000 7fad6a200640 1 -- 172.21.15.44:0/2135029497 shutdown_connections 2024-09-13T08:38:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.778+0000 7fad6a200640 1 -- 172.21.15.44:0/2135029497 wait complete. 2024-09-13T08:38:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad6a200640 1 Processor -- start 2024-09-13T08:38:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad6a200640 1 -- start start 2024-09-13T08:38:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad6a200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad640cdb30 0x7fad640c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad6a200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad640d28e0 0x7fad64150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad6a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6415bea0 0x7fad641f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad6a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad64168fc0 con 0x7fad6415bea0 2024-09-13T08:38:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad6a200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad64168e40 con 0x7fad640d28e0 2024-09-13T08:38:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad6a200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fad64169140 con 0x7fad640cdb30 2024-09-13T08:38:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6415bea0 0x7fad641f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad5be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad640d28e0 0x7fad64150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6415bea0 0x7fad641f7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55958/0 (socket says 172.21.15.44:55958) 2024-09-13T08:38:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad63e00640 1 -- 172.21.15.44:0/1379768766 learned_addr learned my addr 172.21.15.44:0/1379768766 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad63400640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad640cdb30 0x7fad640c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:23.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad63e00640 1 -- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad640cdb30 msgr2=0x7fad640c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad63e00640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad640cdb30 0x7fad640c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad63e00640 1 -- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad640d28e0 msgr2=0x7fad64150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad63e00640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad640d28e0 0x7fad64150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad63e00640 1 -- 172.21.15.44:0/1379768766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad641f7640 con 0x7fad6415bea0 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad5be00640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad640d28e0 0x7fad64150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad63e00640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6415bea0 0x7fad641f7dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00c7a0 tx=0x7fad5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad5b400640 1 -- 172.21.15.44:0/1379768766 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c003b80 con 0x7fad6415bea0 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad5b400640 1 -- 172.21.15.44:0/1379768766 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad5c003d40 con 0x7fad6415bea0 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad5b400640 1 -- 172.21.15.44:0/1379768766 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c00aab0 con 0x7fad6415bea0 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad6a200640 1 -- 172.21.15.44:0/1379768766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad640c9bf0 con 0x7fad6415bea0 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.782+0000 7fad6a200640 1 -- 172.21.15.44:0/1379768766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad640ca1b0 con 0x7fad6415bea0 2024-09-13T08:38:23.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.786+0000 7fad6a200640 1 -- 172.21.15.44:0/1379768766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad640cc9d0 con 0x7fad6415bea0 2024-09-13T08:38:23.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.786+0000 7fad5b400640 1 -- 172.21.15.44:0/1379768766 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fad5c00ac50 con 0x7fad6415bea0 2024-09-13T08:38:23.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.786+0000 7fad5b400640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad40072ec0 0x7fad40075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:23.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.786+0000 7fad5b400640 1 -- 172.21.15.44:0/1379768766 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fad5c093ad0 con 0x7fad6415bea0 2024-09-13T08:38:23.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.786+0000 7fad63400640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad40072ec0 0x7fad40075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:23.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.786+0000 7fad63400640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad40072ec0 0x7fad40075380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fad54009b20 tx=0x7fad540091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:23.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.790+0000 7fad5b400640 1 -- 172.21.15.44:0/1379768766 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fad5c05e6c0 con 0x7fad6415bea0 2024-09-13T08:38:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.966+0000 7fad6a200640 1 -- 172.21.15.44:0/1379768766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad64151bc0 con 0x7fad6415bea0 2024-09-13T08:38:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.966+0000 7fad5b400640 1 -- 172.21.15.44:0/1379768766 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7fad5c0630f0 con 0x7fad6415bea0 2024-09-13T08:38:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:23.973 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":33,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:38:21.196650+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:38:17.738159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T08:38:18.557711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:38:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 -- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad40072ec0 msgr2=0x7fad40075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad40072ec0 0x7fad40075380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fad54009b20 tx=0x7fad540091e0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 -- 172.21.15.44:0/1379768766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6415bea0 msgr2=0x7fad641f7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6415bea0 0x7fad641f7dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00c7a0 tx=0x7fad5c00cc60 comp rx=0 tx=0).stop 2024-09-13T08:38:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 -- 172.21.15.44:0/1379768766 shutdown_connections 2024-09-13T08:38:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad40072ec0 0x7fad40075380 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6415bea0 0x7fad641f7dd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad640d28e0 0x7fad64150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 --2- 172.21.15.44:0/1379768766 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad640cdb30 0x7fad640c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 -- 172.21.15.44:0/1379768766 >> 172.21.15.44:0/1379768766 conn(0x7fad640d1dc0 msgr2=0x7fad640b7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 -- 172.21.15.44:0/1379768766 shutdown_connections 2024-09-13T08:38:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:23.970+0000 7fad6a200640 1 -- 172.21.15.44:0/1379768766 wait complete. 2024-09-13T08:38:23.995 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:38:23.996 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:38:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 -- 172.21.15.44:0/2454965129 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f725c0cdb30 msgr2=0x7f725c0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 --2- 172.21.15.44:0/2454965129 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f725c0cdb30 0x7f725c0aeef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7254009c80 tx=0x7f725401cda0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 -- 172.21.15.44:0/2454965129 shutdown_connections 2024-09-13T08:38:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 --2- 172.21.15.44:0/2454965129 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f725c0cdb30 0x7f725c0aeef0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 --2- 172.21.15.44:0/2454965129 >> v2:172.21.15.129:3300/0 conn(0x7f725c0ced80 0x7f725c0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 -- 172.21.15.44:0/2454965129 >> 172.21.15.44:0/2454965129 conn(0x7f725c0d1dc0 msgr2=0x7f725c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 -- 172.21.15.44:0/2454965129 shutdown_connections 2024-09-13T08:38:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 -- 172.21.15.44:0/2454965129 wait complete. 2024-09-13T08:38:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 Processor -- start 2024-09-13T08:38:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 -- start start 2024-09-13T08:38:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f725c0cdb30 0x7f725c1fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f725c0ced80 0x7f725c071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f725c15bea0 0x7f725c1f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f725c169340 con 0x7f725c0cdb30 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1691c0 con 0x7f725c0ced80 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.138+0000 7f7262200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f725c1694c0 con 0x7f725c15bea0 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f725b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f725c0cdb30 0x7f725c1fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f725b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f725c0cdb30 0x7f725c1fb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55964/0 (socket says 172.21.15.44:55964) 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f725b400640 1 -- 172.21.15.44:0/3692733615 learned_addr learned my addr 172.21.15.44:0/3692733615 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f7253e00640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f725c0ced80 0x7f725c071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f725be00640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f725c15bea0 0x7f725c1f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f725b400640 1 -- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f725c15bea0 msgr2=0x7f725c1f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f725b400640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f725c15bea0 0x7f725c1f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f725b400640 1 -- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f725c0ced80 msgr2=0x7f725c071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f725b400640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f725c0ced80 0x7f725c071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f725b400640 1 -- 172.21.15.44:0/3692733615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c0719a0 con 0x7f725c0cdb30 2024-09-13T08:38:24.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f725b400640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f725c0cdb30 0x7f725c1fb780 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c3c0 tx=0x7f724c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:24.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f7253400640 1 -- 172.21.15.44:0/3692733615 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c017070 con 0x7f725c0cdb30 2024-09-13T08:38:24.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f7253400640 1 -- 172.21.15.44:0/3692733615 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f724c0053c0 con 0x7f725c0cdb30 2024-09-13T08:38:24.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f7262200640 1 -- 172.21.15.44:0/3692733615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c0b16a0 con 0x7f725c0cdb30 2024-09-13T08:38:24.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f7253400640 1 -- 172.21.15.44:0/3692733615 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c003860 con 0x7f725c0cdb30 2024-09-13T08:38:24.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f7262200640 1 -- 172.21.15.44:0/3692733615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c0b1c60 con 0x7f725c0cdb30 2024-09-13T08:38:24.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f7262200640 1 -- 172.21.15.44:0/3692733615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725c0ce670 con 0x7f725c0cdb30 2024-09-13T08:38:24.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.142+0000 7f7253400640 1 -- 172.21.15.44:0/3692733615 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f724c01a070 con 0x7f725c0cdb30 2024-09-13T08:38:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.146+0000 7f7253400640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7238072ec0 0x7f7238075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.146+0000 7f7253400640 1 -- 172.21.15.44:0/3692733615 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f724c093070 con 0x7f725c0cdb30 2024-09-13T08:38:24.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.146+0000 7f7253e00640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7238072ec0 0x7f7238075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.146+0000 7f7253e00640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7238072ec0 0x7f7238075380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7244009600 tx=0x7f72440091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:24.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.150+0000 7f7253400640 1 -- 172.21.15.44:0/3692733615 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f724c05dc60 con 0x7f725c0cdb30 2024-09-13T08:38:24.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.322+0000 7f7262200640 1 -- 172.21.15.44:0/3692733615 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f725c0762f0 con 0x7f7238072ec0 2024-09-13T08:38:24.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7253400640 1 -- 172.21.15.44:0/3692733615 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239662 (secure 0 0 0) 0x7f725c0762f0 con 0x7f7238072ec0 2024-09-13T08:38:24.337 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:24.338 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:38:24.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 -- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7238072ec0 msgr2=0x7f7238075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7238072ec0 0x7f7238075380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7244009600 tx=0x7f72440091e0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 -- 172.21.15.44:0/3692733615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f725c0cdb30 msgr2=0x7f725c1fb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f725c0cdb30 0x7f725c1fb780 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c3c0 tx=0x7f724c00c880 comp rx=0 tx=0).stop 2024-09-13T08:38:24.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 -- 172.21.15.44:0/3692733615 shutdown_connections 2024-09-13T08:38:24.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7238072ec0 0x7f7238075380 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f725c15bea0 0x7f725c1f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f725c0ced80 0x7f725c071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 --2- 172.21.15.44:0/3692733615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f725c0cdb30 0x7f725c1fb780 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 -- 172.21.15.44:0/3692733615 >> 172.21.15.44:0/3692733615 conn(0x7f725c0d1dc0 msgr2=0x7f725c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:24.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.330+0000 7f7262200640 1 -- 172.21.15.44:0/3692733615 shutdown_connections 2024-09-13T08:38:24.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.334+0000 7f7262200640 1 -- 172.21.15.44:0/3692733615 wait complete. 2024-09-13T08:38:24.358 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-13T08:38:22.521391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219944,"kb_used_data":3696,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754776,"statfs":{"total":773094113280,"available":772868890624,"internally_reserved":0,"allocated":3784704,"data_stored":2130174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.344479"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.180408+0000","last_change":"2024-09-13T08:38:18.180408+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.180408+0000","last_undegraded":"2024-09-13T08:38:18.180408+0000","last_fullsized":"2024-09-13T08:38:18.180408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.179468+0000","last_change":"2024-09-13T08:38:18.179468+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.179468+0000","last_undegraded":"2024-09-13T08:38:18.179468+0000","last_fullsized":"2024-09-13T08:38:18.179468+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.184406+0000","last_change":"2024-09-13T08:38:18.184406+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.184406+0000","last_undegraded":"2024-09-13T08:38:18.184406+0000","last_fullsized":"2024-09-13T08:38:18.184406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.179802+0000","last_change":"2024-09-13T08:38:18.179802+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.179802+0000","last_undegraded":"2024-09-13T08:38:18.179802+0000","last_fullsized":"2024-09-13T08:38:18.179802+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.177863+0000","last_change":"2024-09-13T08:38:18.177863+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.177863+0000","last_undegraded":"2024-09-13T08:38:18.177863+0000","last_fullsized":"2024-09-13T08:38:18.177863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.177732+0000","last_change":"2024-09-13T08:38:18.177732+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.177732+0000","last_undegraded":"2024-09-13T08:38:18.177732+0000","last_fullsized":"2024-09-13T08:38:18.177732+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.184184+0000","last_change":"2024-09-13T08:38:18.184184+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.184184+0000","last_undegraded":"2024-09-13T08:38:18.184184+0000","last_fullsized":"2024-09-13T08:38:18.184184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.177135+0000","last_change":"2024-09-13T08:38:18.177135+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.177135+0000","last_undegraded":"2024-09-13T08:38:18.177135+0000","last_fullsized":"2024-09-13T08:38:18.177135+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.180399+0000","last_change":"2024-09-13T08:38:18.180399+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.180399+0000","last_undegraded":"2024-09-13T08:38:18.180399+0000","last_fullsized":"2024-09-13T08:38:18.180399+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.180381+0000","last_change":"2024-09-13T08:38:18.180381+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.180381+0000","last_undegraded":"2024-09-13T08:38:18.180381+0000","last_fullsized":"2024-09-13T08:38:18.180381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.181536+0000","last_change":"2024-09-13T08:38:18.181536+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.181536+0000","last_undegraded":"2024-09-13T08:38:18.181536+0000","last_fullsized":"2024-09-13T08:38:18.181536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:18.166960+0000","last_change":"2024-09-13T08:38:18.166960+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.166960+0000","last_undegraded":"2024-09-13T08:38:18.166960+0000","last_fullsized":"2024-09-13T08:38:18.166960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.177029+0000","last_change":"2024-09-13T08:38:18.177029+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.177029+0000","last_undegraded":"2024-09-13T08:38:18.177029+0000","last_fullsized":"2024-09-13T08:38:18.177029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.179467+0000","last_change":"2024-09-13T08:38:18.179467+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.179467+0000","last_undegraded":"2024-09-13T08:38:18.179467+0000","last_fullsized":"2024-09-13T08:38:18.179467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.179751+0000","last_change":"2024-09-13T08:38:18.179751+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.179751+0000","last_undegraded":"2024-09-13T08:38:18.179751+0000","last_fullsized":"2024-09-13T08:38:18.179751+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.183754+0000","last_change":"2024-09-13T08:38:18.183754+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.183754+0000","last_undegraded":"2024-09-13T08:38:18.183754+0000","last_fullsized":"2024-09-13T08:38:18.183754+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.184976+0000","last_change":"2024-09-13T08:38:18.184976+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.184976+0000","last_undegraded":"2024-09-13T08:38:18.184976+0000","last_fullsized":"2024-09-13T08:38:18.184976+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.176961+0000","last_change":"2024-09-13T08:38:18.176961+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.176961+0000","last_undegraded":"2024-09-13T08:38:18.176961+0000","last_fullsized":"2024-09-13T08:38:18.176961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.180988+0000","last_change":"2024-09-13T08:38:18.180988+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.180988+0000","last_undegraded":"2024-09-13T08:38:18.180988+0000","last_fullsized":"2024-09-13T08:38:18.180988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.183211+0000","last_change":"2024-09-13T08:38:18.183211+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.183211+0000","last_undegraded":"2024-09-13T08:38:18.183211+0000","last_fullsized":"2024-09-13T08:38:18.183211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.179863+0000","last_change":"2024-09-13T08:38:18.179863+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.179863+0000","last_undegraded":"2024-09-13T08:38:18.179863+0000","last_fullsized":"2024-09-13T08:38:18.179863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.177338+0000","last_change":"2024-09-13T08:38:18.177338+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.177338+0000","last_undegraded":"2024-09-13T08:38:18.177338+0000","last_fullsized":"2024-09-13T08:38:18.177338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.184648+0000","last_change":"2024-09-13T08:38:18.184648+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.184648+0000","last_undegraded":"2024-09-13T08:38:18.184648+0000","last_fullsized":"2024-09-13T08:38:18.184648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.180026+0000","last_change":"2024-09-13T08:38:18.180026+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.180026+0000","last_undegraded":"2024-09-13T08:38:18.180026+0000","last_fullsized":"2024-09-13T08:38:18.180026+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:18.166960+0000","last_change":"2024-09-13T08:38:18.166960+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.166960+0000","last_undegraded":"2024-09-13T08:38:18.166960+0000","last_fullsized":"2024-09-13T08:38:18.166960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.179720+0000","last_change":"2024-09-13T08:38:18.179720+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.179720+0000","last_undegraded":"2024-09-13T08:38:18.179720+0000","last_fullsized":"2024-09-13T08:38:18.179720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:18.166960+0000","last_change":"2024-09-13T08:38:18.166960+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.166960+0000","last_undegraded":"2024-09-13T08:38:18.166960+0000","last_fullsized":"2024-09-13T08:38:18.166960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:18.166960+0000","last_change":"2024-09-13T08:38:18.166960+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.166960+0000","last_undegraded":"2024-09-13T08:38:18.166960+0000","last_fullsized":"2024-09-13T08:38:18.166960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:18.166960+0000","last_change":"2024-09-13T08:38:18.166960+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.166960+0000","last_undegraded":"2024-09-13T08:38:18.166960+0000","last_fullsized":"2024-09-13T08:38:18.166960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.176961+0000","last_change":"2024-09-13T08:38:18.176961+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.176961+0000","last_undegraded":"2024-09-13T08:38:18.176961+0000","last_fullsized":"2024-09-13T08:38:18.176961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.179371+0000","last_change":"2024-09-13T08:38:18.179371+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.179371+0000","last_undegraded":"2024-09-13T08:38:18.179371+0000","last_fullsized":"2024-09-13T08:38:18.179371+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:38:19.174381+0000","last_change":"2024-09-13T08:38:19.174381+0000","last_active":"2024-09-13T08:38:19.174381+0000","last_peered":"2024-09-13T08:38:19.174381+0000","last_clean":"2024-09-13T08:38:19.174381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:19.174381+0000","last_undegraded":"2024-09-13T08:38:19.174381+0000","last_fullsized":"2024-09-13T08:38:19.174381+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T08:38:18.180358+0000","last_change":"2024-09-13T08:38:18.180358+0000","last_active":"2024-09-13T08:38:18.166960+0000","last_peered":"2024-09-13T08:38:18.166960+0000","last_clean":"2024-09-13T08:38:18.166960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:38:18.180358+0000","last_undegraded":"2024-09-13T08:38:18.180358+0000","last_fullsized":"2024-09-13T08:38:18.180358+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":124,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T08:38:18.187056+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:38:18.187056+0000","last_peered":"2024-09-13T08:38:18.187056+0000","last_clean":"2024-09-13T08:38:18.187056+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:38:18.187056+0000","last_undegraded":"2024-09-13T08:38:18.187056+0000","last_fullsized":"2024-09-13T08:38:18.187056+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":120134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":909312,"data_stored":704685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":120134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":120134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":120134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":921600,"data_stored":710502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":120134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27344,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":315392,"data_stored":114317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:38:24.362 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:38:24.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.546+0000 7f7879c00640 1 -- 172.21.15.44:0/685739950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7858004d50 msgr2=0x7f78580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.546+0000 7f7879c00640 1 --2- 172.21.15.44:0/685739950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7858004d50 0x7f78580051d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f786c00c640 tx=0x7f786c01d280 comp rx=0 tx=0).stop 2024-09-13T08:38:24.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.546+0000 7f7879200640 1 -- 172.21.15.44:0/685739950 reap_dead start 2024-09-13T08:38:24.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.546+0000 7f7879c00640 1 -- 172.21.15.44:0/685739950 shutdown_connections 2024-09-13T08:38:24.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.546+0000 7f7879c00640 1 -- 172.21.15.44:0/685739950 >> 172.21.15.44:0/685739950 conn(0x7f787409aa10 msgr2=0x7f787409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:24.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.546+0000 7f7879c00640 1 -- 172.21.15.44:0/685739950 shutdown_connections 2024-09-13T08:38:24.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.546+0000 7f7879c00640 1 -- 172.21.15.44:0/685739950 wait complete. 2024-09-13T08:38:24.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879c00640 1 Processor -- start 2024-09-13T08:38:24.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879c00640 1 -- start start 2024-09-13T08:38:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7858004d50 0x7f7874154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7874158720 0x7f787409ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f787409d200 0x7f787415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78740b1530 con 0x7f787409d200 2024-09-13T08:38:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78740b13b0 con 0x7f7858004d50 2024-09-13T08:38:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7874071060 con 0x7f7874158720 2024-09-13T08:38:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f787409d200 0x7f787415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f787409d200 0x7f787415a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55992/0 (socket says 172.21.15.44:55992) 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879200640 1 -- 172.21.15.44:0/2230992392 learned_addr learned my addr 172.21.15.44:0/2230992392 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7873400640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7874158720 0x7f787409ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7873e00640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7858004d50 0x7f7874154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879200640 1 -- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7874158720 msgr2=0x7f787409ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879200640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7874158720 0x7f787409ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879200640 1 -- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7858004d50 msgr2=0x7f7874154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879200640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7858004d50 0x7f7874154840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879200640 1 -- 172.21.15.44:0/2230992392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78741f7b20 con 0x7f787409d200 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7873400640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7874158720 0x7f787409ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879200640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f787409d200 0x7f787415a6f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f786c01cd40 tx=0x7f786c01d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f786be00640 1 -- 172.21.15.44:0/2230992392 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c003d90 con 0x7f787409d200 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f786be00640 1 -- 172.21.15.44:0/2230992392 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f786c004e40 con 0x7f787409d200 2024-09-13T08:38:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f786be00640 1 -- 172.21.15.44:0/2230992392 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c006290 con 0x7f787409d200 2024-09-13T08:38:24.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879c00640 1 -- 172.21.15.44:0/2230992392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78741f7e00 con 0x7f787409d200 2024-09-13T08:38:24.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.550+0000 7f7879c00640 1 -- 172.21.15.44:0/2230992392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78741f83c0 con 0x7f787409d200 2024-09-13T08:38:24.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.554+0000 7f7879c00640 1 -- 172.21.15.44:0/2230992392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78740b5160 con 0x7f787409d200 2024-09-13T08:38:24.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.554+0000 7f786be00640 1 -- 172.21.15.44:0/2230992392 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f786c0ac020 con 0x7f787409d200 2024-09-13T08:38:24.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.554+0000 7f786be00640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7854072f20 0x7f78540753e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.554+0000 7f786be00640 1 -- 172.21.15.44:0/2230992392 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f786c12dfa0 con 0x7f787409d200 2024-09-13T08:38:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.554+0000 7f7873e00640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7854072f20 0x7f78540753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.554+0000 7f7873e00640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7854072f20 0x7f78540753e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7864009ba0 tx=0x7f78640091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:24.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.558+0000 7f786be00640 1 -- 172.21.15.44:0/2230992392 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f786c017030 con 0x7f787409d200 2024-09-13T08:38:24.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.730+0000 7f7879c00640 1 -- 172.21.15.44:0/2230992392 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78740ce0c0 con 0x7f7854072f20 2024-09-13T08:38:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.734+0000 7f786be00640 1 -- 172.21.15.44:0/2230992392 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f78740ce0c0 con 0x7f7854072f20 2024-09-13T08:38:24.744 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:38:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 -- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7854072f20 msgr2=0x7f78540753e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7854072f20 0x7f78540753e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7864009ba0 tx=0x7f78640091e0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 -- 172.21.15.44:0/2230992392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f787409d200 msgr2=0x7f787415a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f787409d200 0x7f787415a6f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f786c01cd40 tx=0x7f786c01d4d0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 -- 172.21.15.44:0/2230992392 shutdown_connections 2024-09-13T08:38:24.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7854072f20 0x7f78540753e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f787409d200 0x7f787415a6f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7874158720 0x7f787409ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 --2- 172.21.15.44:0/2230992392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7858004d50 0x7f7874154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 -- 172.21.15.44:0/2230992392 >> 172.21.15.44:0/2230992392 conn(0x7f787409aa10 msgr2=0x7f78740b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:24.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 -- 172.21.15.44:0/2230992392 shutdown_connections 2024-09-13T08:38:24.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.738+0000 7f7879c00640 1 -- 172.21.15.44:0/2230992392 wait complete. 2024-09-13T08:38:24.765 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-13T08:38:24.521848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220336,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754384,"statfs":{"total":773094113280,"available":772868489216,"internally_reserved":0,"allocated":4169728,"data_stored":2301428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001368"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362693+0000","last_change":"2024-09-13T08:38:20.222645+0000","last_active":"2024-09-13T08:38:21.362693+0000","last_peered":"2024-09-13T08:38:21.362693+0000","last_clean":"2024-09-13T08:38:21.362693+0000","last_became_active":"2024-09-13T08:38:20.218359+0000","last_became_peered":"2024-09-13T08:38:20.218359+0000","last_unstale":"2024-09-13T08:38:21.362693+0000","last_undegraded":"2024-09-13T08:38:21.362693+0000","last_fullsized":"2024-09-13T08:38:21.362693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:07:09.933022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.209618+0000","last_change":"2024-09-13T08:38:20.199946+0000","last_active":"2024-09-13T08:38:21.209618+0000","last_peered":"2024-09-13T08:38:21.209618+0000","last_clean":"2024-09-13T08:38:21.209618+0000","last_became_active":"2024-09-13T08:38:20.199277+0000","last_became_peered":"2024-09-13T08:38:20.199277+0000","last_unstale":"2024-09-13T08:38:21.209618+0000","last_undegraded":"2024-09-13T08:38:21.209618+0000","last_fullsized":"2024-09-13T08:38:21.209618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:43:07.335452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363367+0000","last_change":"2024-09-13T08:38:20.221851+0000","last_active":"2024-09-13T08:38:21.363367+0000","last_peered":"2024-09-13T08:38:21.363367+0000","last_clean":"2024-09-13T08:38:21.363367+0000","last_became_active":"2024-09-13T08:38:20.221538+0000","last_became_peered":"2024-09-13T08:38:20.221538+0000","last_unstale":"2024-09-13T08:38:21.363367+0000","last_undegraded":"2024-09-13T08:38:21.363367+0000","last_fullsized":"2024-09-13T08:38:21.363367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:41.512174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210910+0000","last_change":"2024-09-13T08:38:20.226985+0000","last_active":"2024-09-13T08:38:21.210910+0000","last_peered":"2024-09-13T08:38:21.210910+0000","last_clean":"2024-09-13T08:38:21.210910+0000","last_became_active":"2024-09-13T08:38:20.226748+0000","last_became_peered":"2024-09-13T08:38:20.226748+0000","last_unstale":"2024-09-13T08:38:21.210910+0000","last_undegraded":"2024-09-13T08:38:21.210910+0000","last_fullsized":"2024-09-13T08:38:21.210910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:43.789970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.209810+0000","last_change":"2024-09-13T08:38:20.210714+0000","last_active":"2024-09-13T08:38:21.209810+0000","last_peered":"2024-09-13T08:38:21.209810+0000","last_clean":"2024-09-13T08:38:21.209810+0000","last_became_active":"2024-09-13T08:38:20.210440+0000","last_became_peered":"2024-09-13T08:38:20.210440+0000","last_unstale":"2024-09-13T08:38:21.209810+0000","last_undegraded":"2024-09-13T08:38:21.209810+0000","last_fullsized":"2024-09-13T08:38:21.209810+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:22.746997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362904+0000","last_change":"2024-09-13T08:38:20.217303+0000","last_active":"2024-09-13T08:38:21.362904+0000","last_peered":"2024-09-13T08:38:21.362904+0000","last_clean":"2024-09-13T08:38:21.362904+0000","last_became_active":"2024-09-13T08:38:20.216962+0000","last_became_peered":"2024-09-13T08:38:20.216962+0000","last_unstale":"2024-09-13T08:38:21.362904+0000","last_undegraded":"2024-09-13T08:38:21.362904+0000","last_fullsized":"2024-09-13T08:38:21.362904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:34:13.968313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362405+0000","last_change":"2024-09-13T08:38:20.222376+0000","last_active":"2024-09-13T08:38:21.362405+0000","last_peered":"2024-09-13T08:38:21.362405+0000","last_clean":"2024-09-13T08:38:21.362405+0000","last_became_active":"2024-09-13T08:38:20.222083+0000","last_became_peered":"2024-09-13T08:38:20.222083+0000","last_unstale":"2024-09-13T08:38:21.362405+0000","last_undegraded":"2024-09-13T08:38:21.362405+0000","last_fullsized":"2024-09-13T08:38:21.362405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:55:08.703956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362026+0000","last_change":"2024-09-13T08:38:20.221721+0000","last_active":"2024-09-13T08:38:21.362026+0000","last_peered":"2024-09-13T08:38:21.362026+0000","last_clean":"2024-09-13T08:38:21.362026+0000","last_became_active":"2024-09-13T08:38:20.218212+0000","last_became_peered":"2024-09-13T08:38:20.218212+0000","last_unstale":"2024-09-13T08:38:21.362026+0000","last_undegraded":"2024-09-13T08:38:21.362026+0000","last_fullsized":"2024-09-13T08:38:21.362026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:04.442980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363625+0000","last_change":"2024-09-13T08:38:20.201723+0000","last_active":"2024-09-13T08:38:21.363625+0000","last_peered":"2024-09-13T08:38:21.363625+0000","last_clean":"2024-09-13T08:38:21.363625+0000","last_became_active":"2024-09-13T08:38:20.200397+0000","last_became_peered":"2024-09-13T08:38:20.200397+0000","last_unstale":"2024-09-13T08:38:21.363625+0000","last_undegraded":"2024-09-13T08:38:21.363625+0000","last_fullsized":"2024-09-13T08:38:21.363625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:17:35.729105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.211049+0000","last_change":"2024-09-13T08:38:20.199791+0000","last_active":"2024-09-13T08:38:21.211049+0000","last_peered":"2024-09-13T08:38:21.211049+0000","last_clean":"2024-09-13T08:38:21.211049+0000","last_became_active":"2024-09-13T08:38:20.199333+0000","last_became_peered":"2024-09-13T08:38:20.199333+0000","last_unstale":"2024-09-13T08:38:21.211049+0000","last_undegraded":"2024-09-13T08:38:21.211049+0000","last_fullsized":"2024-09-13T08:38:21.211049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:53.317016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.206077+0000","last_change":"2024-09-13T08:38:20.211410+0000","last_active":"2024-09-13T08:38:21.206077+0000","last_peered":"2024-09-13T08:38:21.206077+0000","last_clean":"2024-09-13T08:38:21.206077+0000","last_became_active":"2024-09-13T08:38:20.208553+0000","last_became_peered":"2024-09-13T08:38:20.208553+0000","last_unstale":"2024-09-13T08:38:21.206077+0000","last_undegraded":"2024-09-13T08:38:21.206077+0000","last_fullsized":"2024-09-13T08:38:21.206077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:13:46.465923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363187+0000","last_change":"2024-09-13T08:38:20.217454+0000","last_active":"2024-09-13T08:38:21.363187+0000","last_peered":"2024-09-13T08:38:21.363187+0000","last_clean":"2024-09-13T08:38:21.363187+0000","last_became_active":"2024-09-13T08:38:20.217032+0000","last_became_peered":"2024-09-13T08:38:20.217032+0000","last_unstale":"2024-09-13T08:38:21.363187+0000","last_undegraded":"2024-09-13T08:38:21.363187+0000","last_fullsized":"2024-09-13T08:38:21.363187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:06.842151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363024+0000","last_change":"2024-09-13T08:38:20.205943+0000","last_active":"2024-09-13T08:38:21.363024+0000","last_peered":"2024-09-13T08:38:21.363024+0000","last_clean":"2024-09-13T08:38:21.363024+0000","last_became_active":"2024-09-13T08:38:20.205643+0000","last_became_peered":"2024-09-13T08:38:20.205643+0000","last_unstale":"2024-09-13T08:38:21.363024+0000","last_undegraded":"2024-09-13T08:38:21.363024+0000","last_fullsized":"2024-09-13T08:38:21.363024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:54.886011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210731+0000","last_change":"2024-09-13T08:38:20.200569+0000","last_active":"2024-09-13T08:38:21.210731+0000","last_peered":"2024-09-13T08:38:21.210731+0000","last_clean":"2024-09-13T08:38:21.210731+0000","last_became_active":"2024-09-13T08:38:20.200283+0000","last_became_peered":"2024-09-13T08:38:20.200283+0000","last_unstale":"2024-09-13T08:38:21.210731+0000","last_undegraded":"2024-09-13T08:38:21.210731+0000","last_fullsized":"2024-09-13T08:38:21.210731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:22.746997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362116+0000","last_change":"2024-09-13T08:38:20.223553+0000","last_active":"2024-09-13T08:38:21.362116+0000","last_peered":"2024-09-13T08:38:21.362116+0000","last_clean":"2024-09-13T08:38:21.362116+0000","last_became_active":"2024-09-13T08:38:20.223356+0000","last_became_peered":"2024-09-13T08:38:20.223356+0000","last_unstale":"2024-09-13T08:38:21.362116+0000","last_undegraded":"2024-09-13T08:38:21.362116+0000","last_fullsized":"2024-09-13T08:38:21.362116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:32.554222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362150+0000","last_change":"2024-09-13T08:38:20.202113+0000","last_active":"2024-09-13T08:38:21.362150+0000","last_peered":"2024-09-13T08:38:21.362150+0000","last_clean":"2024-09-13T08:38:21.362150+0000","last_became_active":"2024-09-13T08:38:20.201902+0000","last_became_peered":"2024-09-13T08:38:20.201902+0000","last_unstale":"2024-09-13T08:38:21.362150+0000","last_undegraded":"2024-09-13T08:38:21.362150+0000","last_fullsized":"2024-09-13T08:38:21.362150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:56.671782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.206271+0000","last_change":"2024-09-13T08:38:20.224424+0000","last_active":"2024-09-13T08:38:21.206271+0000","last_peered":"2024-09-13T08:38:21.206271+0000","last_clean":"2024-09-13T08:38:21.206271+0000","last_became_active":"2024-09-13T08:38:20.224127+0000","last_became_peered":"2024-09-13T08:38:20.224127+0000","last_unstale":"2024-09-13T08:38:21.206271+0000","last_undegraded":"2024-09-13T08:38:21.206271+0000","last_fullsized":"2024-09-13T08:38:21.206271+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:38.791879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363054+0000","last_change":"2024-09-13T08:38:20.214421+0000","last_active":"2024-09-13T08:38:21.363054+0000","last_peered":"2024-09-13T08:38:21.363054+0000","last_clean":"2024-09-13T08:38:21.363054+0000","last_became_active":"2024-09-13T08:38:20.207047+0000","last_became_peered":"2024-09-13T08:38:20.207047+0000","last_unstale":"2024-09-13T08:38:21.363054+0000","last_undegraded":"2024-09-13T08:38:21.363054+0000","last_fullsized":"2024-09-13T08:38:21.363054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:32.220116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363246+0000","last_change":"2024-09-13T08:38:20.211968+0000","last_active":"2024-09-13T08:38:21.363246+0000","last_peered":"2024-09-13T08:38:21.363246+0000","last_clean":"2024-09-13T08:38:21.363246+0000","last_became_active":"2024-09-13T08:38:20.211670+0000","last_became_peered":"2024-09-13T08:38:20.211670+0000","last_unstale":"2024-09-13T08:38:21.363246+0000","last_undegraded":"2024-09-13T08:38:21.363246+0000","last_fullsized":"2024-09-13T08:38:21.363246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:58:42.433091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363165+0000","last_change":"2024-09-13T08:38:20.211824+0000","last_active":"2024-09-13T08:38:21.363165+0000","last_peered":"2024-09-13T08:38:21.363165+0000","last_clean":"2024-09-13T08:38:21.363165+0000","last_became_active":"2024-09-13T08:38:20.209693+0000","last_became_peered":"2024-09-13T08:38:20.209693+0000","last_unstale":"2024-09-13T08:38:21.363165+0000","last_undegraded":"2024-09-13T08:38:21.363165+0000","last_fullsized":"2024-09-13T08:38:21.363165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:48:30.790593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362627+0000","last_change":"2024-09-13T08:38:20.205236+0000","last_active":"2024-09-13T08:38:21.362627+0000","last_peered":"2024-09-13T08:38:21.362627+0000","last_clean":"2024-09-13T08:38:21.362627+0000","last_became_active":"2024-09-13T08:38:20.204904+0000","last_became_peered":"2024-09-13T08:38:20.204904+0000","last_unstale":"2024-09-13T08:38:21.362627+0000","last_undegraded":"2024-09-13T08:38:21.362627+0000","last_fullsized":"2024-09-13T08:38:21.362627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:23.200220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.361796+0000","last_change":"2024-09-13T08:38:20.217780+0000","last_active":"2024-09-13T08:38:21.361796+0000","last_peered":"2024-09-13T08:38:21.361796+0000","last_clean":"2024-09-13T08:38:21.361796+0000","last_became_active":"2024-09-13T08:38:20.217447+0000","last_became_peered":"2024-09-13T08:38:20.217447+0000","last_unstale":"2024-09-13T08:38:21.361796+0000","last_undegraded":"2024-09-13T08:38:21.361796+0000","last_fullsized":"2024-09-13T08:38:21.361796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:04:27.348133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363031+0000","last_change":"2024-09-13T08:38:20.218959+0000","last_active":"2024-09-13T08:38:21.363031+0000","last_peered":"2024-09-13T08:38:21.363031+0000","last_clean":"2024-09-13T08:38:21.363031+0000","last_became_active":"2024-09-13T08:38:20.218642+0000","last_became_peered":"2024-09-13T08:38:20.218642+0000","last_unstale":"2024-09-13T08:38:21.363031+0000","last_undegraded":"2024-09-13T08:38:21.363031+0000","last_fullsized":"2024-09-13T08:38:21.363031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:53.213641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362222+0000","last_change":"2024-09-13T08:38:20.217832+0000","last_active":"2024-09-13T08:38:21.362222+0000","last_peered":"2024-09-13T08:38:21.362222+0000","last_clean":"2024-09-13T08:38:21.362222+0000","last_became_active":"2024-09-13T08:38:20.217343+0000","last_became_peered":"2024-09-13T08:38:20.217343+0000","last_unstale":"2024-09-13T08:38:21.362222+0000","last_undegraded":"2024-09-13T08:38:21.362222+0000","last_fullsized":"2024-09-13T08:38:21.362222+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:31.984929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.211252+0000","last_change":"2024-09-13T08:38:20.220721+0000","last_active":"2024-09-13T08:38:21.211252+0000","last_peered":"2024-09-13T08:38:21.211252+0000","last_clean":"2024-09-13T08:38:21.211252+0000","last_became_active":"2024-09-13T08:38:20.220447+0000","last_became_peered":"2024-09-13T08:38:20.220447+0000","last_unstale":"2024-09-13T08:38:21.211252+0000","last_undegraded":"2024-09-13T08:38:21.211252+0000","last_fullsized":"2024-09-13T08:38:21.211252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:07:09.933022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.211301+0000","last_change":"2024-09-13T08:38:20.205875+0000","last_active":"2024-09-13T08:38:21.211301+0000","last_peered":"2024-09-13T08:38:21.211301+0000","last_clean":"2024-09-13T08:38:21.211301+0000","last_became_active":"2024-09-13T08:38:20.205599+0000","last_became_peered":"2024-09-13T08:38:20.205599+0000","last_unstale":"2024-09-13T08:38:21.211301+0000","last_undegraded":"2024-09-13T08:38:21.211301+0000","last_fullsized":"2024-09-13T08:38:21.211301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:53:47.201495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.209547+0000","last_change":"2024-09-13T08:38:20.196721+0000","last_active":"2024-09-13T08:38:21.209547+0000","last_peered":"2024-09-13T08:38:21.209547+0000","last_clean":"2024-09-13T08:38:21.209547+0000","last_became_active":"2024-09-13T08:38:20.196440+0000","last_became_peered":"2024-09-13T08:38:20.196440+0000","last_unstale":"2024-09-13T08:38:21.209547+0000","last_undegraded":"2024-09-13T08:38:21.209547+0000","last_fullsized":"2024-09-13T08:38:21.209547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:23.200220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.361891+0000","last_change":"2024-09-13T08:38:20.224529+0000","last_active":"2024-09-13T08:38:21.361891+0000","last_peered":"2024-09-13T08:38:21.361891+0000","last_clean":"2024-09-13T08:38:21.361891+0000","last_became_active":"2024-09-13T08:38:20.222028+0000","last_became_peered":"2024-09-13T08:38:20.222028+0000","last_unstale":"2024-09-13T08:38:21.361891+0000","last_undegraded":"2024-09-13T08:38:21.361891+0000","last_fullsized":"2024-09-13T08:38:21.361891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:58:09.717497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362820+0000","last_change":"2024-09-13T08:38:20.199669+0000","last_active":"2024-09-13T08:38:21.362820+0000","last_peered":"2024-09-13T08:38:21.362820+0000","last_clean":"2024-09-13T08:38:21.362820+0000","last_became_active":"2024-09-13T08:38:20.196530+0000","last_became_peered":"2024-09-13T08:38:20.196530+0000","last_unstale":"2024-09-13T08:38:21.362820+0000","last_undegraded":"2024-09-13T08:38:21.362820+0000","last_fullsized":"2024-09-13T08:38:21.362820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:43:07.335452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362510+0000","last_change":"2024-09-13T08:38:20.212946+0000","last_active":"2024-09-13T08:38:21.362510+0000","last_peered":"2024-09-13T08:38:21.362510+0000","last_clean":"2024-09-13T08:38:21.362510+0000","last_became_active":"2024-09-13T08:38:20.212564+0000","last_became_peered":"2024-09-13T08:38:20.212564+0000","last_unstale":"2024-09-13T08:38:21.362510+0000","last_undegraded":"2024-09-13T08:38:21.362510+0000","last_fullsized":"2024-09-13T08:38:21.362510+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:00:41.184271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362609+0000","last_change":"2024-09-13T08:38:20.212845+0000","last_active":"2024-09-13T08:38:21.362609+0000","last_peered":"2024-09-13T08:38:21.362609+0000","last_clean":"2024-09-13T08:38:21.362609+0000","last_became_active":"2024-09-13T08:38:20.212550+0000","last_became_peered":"2024-09-13T08:38:20.212550+0000","last_unstale":"2024-09-13T08:38:21.362609+0000","last_undegraded":"2024-09-13T08:38:21.362609+0000","last_fullsized":"2024-09-13T08:38:21.362609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:53.317016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.205534+0000","last_change":"2024-09-13T08:38:20.223870+0000","last_active":"2024-09-13T08:38:21.205534+0000","last_peered":"2024-09-13T08:38:21.205534+0000","last_clean":"2024-09-13T08:38:21.205534+0000","last_became_active":"2024-09-13T08:38:20.223586+0000","last_became_peered":"2024-09-13T08:38:20.223586+0000","last_unstale":"2024-09-13T08:38:21.205534+0000","last_undegraded":"2024-09-13T08:38:21.205534+0000","last_fullsized":"2024-09-13T08:38:21.205534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:24.274711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363763+0000","last_change":"2024-09-13T08:38:20.214162+0000","last_active":"2024-09-13T08:38:21.363763+0000","last_peered":"2024-09-13T08:38:21.363763+0000","last_clean":"2024-09-13T08:38:21.363763+0000","last_became_active":"2024-09-13T08:38:20.213827+0000","last_became_peered":"2024-09-13T08:38:20.213827+0000","last_unstale":"2024-09-13T08:38:21.363763+0000","last_undegraded":"2024-09-13T08:38:21.363763+0000","last_fullsized":"2024-09-13T08:38:21.363763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:55.647844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.206298+0000","last_change":"2024-09-13T08:38:19.210428+0000","last_active":"2024-09-13T08:38:21.206298+0000","last_peered":"2024-09-13T08:38:21.206298+0000","last_clean":"2024-09-13T08:38:21.206298+0000","last_became_active":"2024-09-13T08:38:19.210013+0000","last_became_peered":"2024-09-13T08:38:19.210013+0000","last_unstale":"2024-09-13T08:38:21.206298+0000","last_undegraded":"2024-09-13T08:38:21.206298+0000","last_fullsized":"2024-09-13T08:38:21.206298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:08:56.730182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362603+0000","last_change":"2024-09-13T08:38:20.225452+0000","last_active":"2024-09-13T08:38:21.362603+0000","last_peered":"2024-09-13T08:38:21.362603+0000","last_clean":"2024-09-13T08:38:21.362603+0000","last_became_active":"2024-09-13T08:38:20.218535+0000","last_became_peered":"2024-09-13T08:38:20.218535+0000","last_unstale":"2024-09-13T08:38:21.362603+0000","last_undegraded":"2024-09-13T08:38:21.362603+0000","last_fullsized":"2024-09-13T08:38:21.362603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:08:57.737604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.211365+0000","last_change":"2024-09-13T08:38:19.208474+0000","last_active":"2024-09-13T08:38:21.211365+0000","last_peered":"2024-09-13T08:38:21.211365+0000","last_clean":"2024-09-13T08:38:21.211365+0000","last_became_active":"2024-09-13T08:38:19.208162+0000","last_became_peered":"2024-09-13T08:38:19.208162+0000","last_unstale":"2024-09-13T08:38:21.211365+0000","last_undegraded":"2024-09-13T08:38:21.211365+0000","last_fullsized":"2024-09-13T08:38:21.211365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:17:34.721683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362085+0000","last_change":"2024-09-13T08:38:20.227106+0000","last_active":"2024-09-13T08:38:21.362085+0000","last_peered":"2024-09-13T08:38:21.362085+0000","last_clean":"2024-09-13T08:38:21.362085+0000","last_became_active":"2024-09-13T08:38:20.218344+0000","last_became_peered":"2024-09-13T08:38:20.218344+0000","last_unstale":"2024-09-13T08:38:21.362085+0000","last_undegraded":"2024-09-13T08:38:21.362085+0000","last_fullsized":"2024-09-13T08:38:21.362085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:16.987112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363481+0000","last_change":"2024-09-13T08:38:19.207817+0000","last_active":"2024-09-13T08:38:21.363481+0000","last_peered":"2024-09-13T08:38:21.363481+0000","last_clean":"2024-09-13T08:38:21.363481+0000","last_became_active":"2024-09-13T08:38:19.207571+0000","last_became_peered":"2024-09-13T08:38:19.207571+0000","last_unstale":"2024-09-13T08:38:21.363481+0000","last_undegraded":"2024-09-13T08:38:21.363481+0000","last_fullsized":"2024-09-13T08:38:21.363481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:17.792852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.211325+0000","last_change":"2024-09-13T08:38:20.207578+0000","last_active":"2024-09-13T08:38:21.211325+0000","last_peered":"2024-09-13T08:38:21.211325+0000","last_clean":"2024-09-13T08:38:21.211325+0000","last_became_active":"2024-09-13T08:38:20.205456+0000","last_became_peered":"2024-09-13T08:38:20.205456+0000","last_unstale":"2024-09-13T08:38:21.211325+0000","last_undegraded":"2024-09-13T08:38:21.211325+0000","last_fullsized":"2024-09-13T08:38:21.211325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:46.950636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.211040+0000","last_change":"2024-09-13T08:38:19.209533+0000","last_active":"2024-09-13T08:38:21.211040+0000","last_peered":"2024-09-13T08:38:21.211040+0000","last_clean":"2024-09-13T08:38:21.211040+0000","last_became_active":"2024-09-13T08:38:19.209260+0000","last_became_peered":"2024-09-13T08:38:19.209260+0000","last_unstale":"2024-09-13T08:38:21.211040+0000","last_undegraded":"2024-09-13T08:38:21.211040+0000","last_fullsized":"2024-09-13T08:38:21.211040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:00:21.059431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363533+0000","last_change":"2024-09-13T08:38:20.204434+0000","last_active":"2024-09-13T08:38:21.363533+0000","last_peered":"2024-09-13T08:38:21.363533+0000","last_clean":"2024-09-13T08:38:21.363533+0000","last_became_active":"2024-09-13T08:38:20.204160+0000","last_became_peered":"2024-09-13T08:38:20.204160+0000","last_unstale":"2024-09-13T08:38:21.363533+0000","last_undegraded":"2024-09-13T08:38:21.363533+0000","last_fullsized":"2024-09-13T08:38:21.363533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:59:35.477060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.361772+0000","last_change":"2024-09-13T08:38:19.192666+0000","last_active":"2024-09-13T08:38:21.361772+0000","last_peered":"2024-09-13T08:38:21.361772+0000","last_clean":"2024-09-13T08:38:21.361772+0000","last_became_active":"2024-09-13T08:38:19.192375+0000","last_became_peered":"2024-09-13T08:38:19.192375+0000","last_unstale":"2024-09-13T08:38:21.361772+0000","last_undegraded":"2024-09-13T08:38:21.361772+0000","last_fullsized":"2024-09-13T08:38:21.361772+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:34.940822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.361713+0000","last_change":"2024-09-13T08:38:20.224760+0000","last_active":"2024-09-13T08:38:21.361713+0000","last_peered":"2024-09-13T08:38:21.361713+0000","last_clean":"2024-09-13T08:38:21.361713+0000","last_became_active":"2024-09-13T08:38:20.224455+0000","last_became_peered":"2024-09-13T08:38:20.224455+0000","last_unstale":"2024-09-13T08:38:21.361713+0000","last_undegraded":"2024-09-13T08:38:21.361713+0000","last_fullsized":"2024-09-13T08:38:21.361713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:39:24.410429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.361574+0000","last_change":"2024-09-13T08:38:19.203084+0000","last_active":"2024-09-13T08:38:21.361574+0000","last_peered":"2024-09-13T08:38:21.361574+0000","last_clean":"2024-09-13T08:38:21.361574+0000","last_became_active":"2024-09-13T08:38:19.202565+0000","last_became_peered":"2024-09-13T08:38:19.202565+0000","last_unstale":"2024-09-13T08:38:21.361574+0000","last_undegraded":"2024-09-13T08:38:21.361574+0000","last_fullsized":"2024-09-13T08:38:21.361574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:05.834729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.361808+0000","last_change":"2024-09-13T08:38:20.214838+0000","last_active":"2024-09-13T08:38:21.361808+0000","last_peered":"2024-09-13T08:38:21.361808+0000","last_clean":"2024-09-13T08:38:21.361808+0000","last_became_active":"2024-09-13T08:38:20.214400+0000","last_became_peered":"2024-09-13T08:38:20.214400+0000","last_unstale":"2024-09-13T08:38:21.361808+0000","last_undegraded":"2024-09-13T08:38:21.361808+0000","last_fullsized":"2024-09-13T08:38:21.361808+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:44.726627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363267+0000","last_change":"2024-09-13T08:38:19.207206+0000","last_active":"2024-09-13T08:38:21.363267+0000","last_peered":"2024-09-13T08:38:21.363267+0000","last_clean":"2024-09-13T08:38:21.363267+0000","last_became_active":"2024-09-13T08:38:19.206880+0000","last_became_peered":"2024-09-13T08:38:19.206880+0000","last_unstale":"2024-09-13T08:38:21.363267+0000","last_undegraded":"2024-09-13T08:38:21.363267+0000","last_fullsized":"2024-09-13T08:38:21.363267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:33.450088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210203+0000","last_change":"2024-09-13T08:38:20.214613+0000","last_active":"2024-09-13T08:38:21.210203+0000","last_peered":"2024-09-13T08:38:21.210203+0000","last_clean":"2024-09-13T08:38:21.210203+0000","last_became_active":"2024-09-13T08:38:20.214329+0000","last_became_peered":"2024-09-13T08:38:20.214329+0000","last_unstale":"2024-09-13T08:38:21.210203+0000","last_undegraded":"2024-09-13T08:38:21.210203+0000","last_fullsized":"2024-09-13T08:38:21.210203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:58:09.717497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362471+0000","last_change":"2024-09-13T08:38:19.199286+0000","last_active":"2024-09-13T08:38:21.362471+0000","last_peered":"2024-09-13T08:38:21.362471+0000","last_clean":"2024-09-13T08:38:21.362471+0000","last_became_active":"2024-09-13T08:38:19.198974+0000","last_became_peered":"2024-09-13T08:38:19.198974+0000","last_unstale":"2024-09-13T08:38:21.362471+0000","last_undegraded":"2024-09-13T08:38:21.362471+0000","last_fullsized":"2024-09-13T08:38:21.362471+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.765606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.205997+0000","last_change":"2024-09-13T08:38:20.215924+0000","last_active":"2024-09-13T08:38:21.205997+0000","last_peered":"2024-09-13T08:38:21.205997+0000","last_clean":"2024-09-13T08:38:21.205997+0000","last_became_active":"2024-09-13T08:38:20.215513+0000","last_became_peered":"2024-09-13T08:38:20.215513+0000","last_unstale":"2024-09-13T08:38:21.205997+0000","last_undegraded":"2024-09-13T08:38:21.205997+0000","last_fullsized":"2024-09-13T08:38:21.205997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:29.601905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.209649+0000","last_change":"2024-09-13T08:38:19.198010+0000","last_active":"2024-09-13T08:38:21.209649+0000","last_peered":"2024-09-13T08:38:21.209649+0000","last_clean":"2024-09-13T08:38:21.209649+0000","last_became_active":"2024-09-13T08:38:19.197701+0000","last_became_peered":"2024-09-13T08:38:19.197701+0000","last_unstale":"2024-09-13T08:38:21.209649+0000","last_undegraded":"2024-09-13T08:38:21.209649+0000","last_fullsized":"2024-09-13T08:38:21.209649+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:52.691227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.211471+0000","last_change":"2024-09-13T08:38:20.200488+0000","last_active":"2024-09-13T08:38:21.211471+0000","last_peered":"2024-09-13T08:38:21.211471+0000","last_clean":"2024-09-13T08:38:21.211471+0000","last_became_active":"2024-09-13T08:38:20.200233+0000","last_became_peered":"2024-09-13T08:38:20.200233+0000","last_unstale":"2024-09-13T08:38:21.211471+0000","last_undegraded":"2024-09-13T08:38:21.211471+0000","last_fullsized":"2024-09-13T08:38:21.211471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:29.066045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.209683+0000","last_change":"2024-09-13T08:38:19.202135+0000","last_active":"2024-09-13T08:38:21.209683+0000","last_peered":"2024-09-13T08:38:21.209683+0000","last_clean":"2024-09-13T08:38:21.209683+0000","last_became_active":"2024-09-13T08:38:19.201858+0000","last_became_peered":"2024-09-13T08:38:19.201858+0000","last_unstale":"2024-09-13T08:38:21.209683+0000","last_undegraded":"2024-09-13T08:38:21.209683+0000","last_fullsized":"2024-09-13T08:38:21.209683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:36.904676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210969+0000","last_change":"2024-09-13T08:38:20.227178+0000","last_active":"2024-09-13T08:38:21.210969+0000","last_peered":"2024-09-13T08:38:21.210969+0000","last_clean":"2024-09-13T08:38:21.210969+0000","last_became_active":"2024-09-13T08:38:20.226801+0000","last_became_peered":"2024-09-13T08:38:20.226801+0000","last_unstale":"2024-09-13T08:38:21.210969+0000","last_undegraded":"2024-09-13T08:38:21.210969+0000","last_fullsized":"2024-09-13T08:38:21.210969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:08:57.737604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.205815+0000","last_change":"2024-09-13T08:38:19.210979+0000","last_active":"2024-09-13T08:38:21.205815+0000","last_peered":"2024-09-13T08:38:21.205815+0000","last_clean":"2024-09-13T08:38:21.205815+0000","last_became_active":"2024-09-13T08:38:19.210532+0000","last_became_peered":"2024-09-13T08:38:19.210532+0000","last_unstale":"2024-09-13T08:38:21.205815+0000","last_undegraded":"2024-09-13T08:38:21.205815+0000","last_fullsized":"2024-09-13T08:38:21.205815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:30.977507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.205868+0000","last_change":"2024-09-13T08:38:20.220419+0000","last_active":"2024-09-13T08:38:21.205868+0000","last_peered":"2024-09-13T08:38:21.205868+0000","last_clean":"2024-09-13T08:38:21.205868+0000","last_became_active":"2024-09-13T08:38:20.216298+0000","last_became_peered":"2024-09-13T08:38:20.216298+0000","last_unstale":"2024-09-13T08:38:21.205868+0000","last_undegraded":"2024-09-13T08:38:21.205868+0000","last_fullsized":"2024-09-13T08:38:21.205868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:25.660057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363373+0000","last_change":"2024-09-13T08:38:19.207575+0000","last_active":"2024-09-13T08:38:21.363373+0000","last_peered":"2024-09-13T08:38:21.363373+0000","last_clean":"2024-09-13T08:38:21.363373+0000","last_became_active":"2024-09-13T08:38:19.206978+0000","last_became_peered":"2024-09-13T08:38:19.206978+0000","last_unstale":"2024-09-13T08:38:21.363373+0000","last_undegraded":"2024-09-13T08:38:21.363373+0000","last_fullsized":"2024-09-13T08:38:21.363373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:30:51.771065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362983+0000","last_change":"2024-09-13T08:38:20.218545+0000","last_active":"2024-09-13T08:38:21.362983+0000","last_peered":"2024-09-13T08:38:21.362983+0000","last_clean":"2024-09-13T08:38:21.362983+0000","last_became_active":"2024-09-13T08:38:20.218158+0000","last_became_peered":"2024-09-13T08:38:20.218158+0000","last_unstale":"2024-09-13T08:38:21.362983+0000","last_undegraded":"2024-09-13T08:38:21.362983+0000","last_fullsized":"2024-09-13T08:38:21.362983+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:54.204820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.361883+0000","last_change":"2024-09-13T08:38:19.206656+0000","last_active":"2024-09-13T08:38:21.361883+0000","last_peered":"2024-09-13T08:38:21.361883+0000","last_clean":"2024-09-13T08:38:21.361883+0000","last_became_active":"2024-09-13T08:38:19.206248+0000","last_became_peered":"2024-09-13T08:38:19.206248+0000","last_unstale":"2024-09-13T08:38:21.361883+0000","last_undegraded":"2024-09-13T08:38:21.361883+0000","last_fullsized":"2024-09-13T08:38:21.361883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:35:31.649682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210649+0000","last_change":"2024-09-13T08:38:20.215749+0000","last_active":"2024-09-13T08:38:21.210649+0000","last_peered":"2024-09-13T08:38:21.210649+0000","last_clean":"2024-09-13T08:38:21.210649+0000","last_became_active":"2024-09-13T08:38:20.215478+0000","last_became_peered":"2024-09-13T08:38:20.215478+0000","last_unstale":"2024-09-13T08:38:21.210649+0000","last_undegraded":"2024-09-13T08:38:21.210649+0000","last_fullsized":"2024-09-13T08:38:21.210649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:16.987112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210843+0000","last_change":"2024-09-13T08:38:19.211423+0000","last_active":"2024-09-13T08:38:21.210843+0000","last_peered":"2024-09-13T08:38:21.210843+0000","last_clean":"2024-09-13T08:38:21.210843+0000","last_became_active":"2024-09-13T08:38:19.211055+0000","last_became_peered":"2024-09-13T08:38:19.211055+0000","last_unstale":"2024-09-13T08:38:21.210843+0000","last_undegraded":"2024-09-13T08:38:21.210843+0000","last_fullsized":"2024-09-13T08:38:21.210843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:59:34.469638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362415+0000","last_change":"2024-09-13T08:38:20.217794+0000","last_active":"2024-09-13T08:38:21.362415+0000","last_peered":"2024-09-13T08:38:21.362415+0000","last_clean":"2024-09-13T08:38:21.362415+0000","last_became_active":"2024-09-13T08:38:20.217170+0000","last_became_peered":"2024-09-13T08:38:20.217170+0000","last_unstale":"2024-09-13T08:38:21.362415+0000","last_undegraded":"2024-09-13T08:38:21.362415+0000","last_fullsized":"2024-09-13T08:38:21.362415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:43.789970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210280+0000","last_change":"2024-09-13T08:38:19.194451+0000","last_active":"2024-09-13T08:38:21.210280+0000","last_peered":"2024-09-13T08:38:21.210280+0000","last_clean":"2024-09-13T08:38:21.210280+0000","last_became_active":"2024-09-13T08:38:19.194177+0000","last_became_peered":"2024-09-13T08:38:19.194177+0000","last_unstale":"2024-09-13T08:38:21.210280+0000","last_undegraded":"2024-09-13T08:38:21.210280+0000","last_fullsized":"2024-09-13T08:38:21.210280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:35:31.649682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.205607+0000","last_change":"2024-09-13T08:38:20.223841+0000","last_active":"2024-09-13T08:38:21.205607+0000","last_peered":"2024-09-13T08:38:21.205607+0000","last_clean":"2024-09-13T08:38:21.205607+0000","last_became_active":"2024-09-13T08:38:20.223587+0000","last_became_peered":"2024-09-13T08:38:20.223587+0000","last_unstale":"2024-09-13T08:38:21.205607+0000","last_undegraded":"2024-09-13T08:38:21.205607+0000","last_fullsized":"2024-09-13T08:38:21.205607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:22.327102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362449+0000","last_change":"2024-09-13T08:38:19.217660+0000","last_active":"2024-09-13T08:38:21.362449+0000","last_peered":"2024-09-13T08:38:21.362449+0000","last_clean":"2024-09-13T08:38:21.362449+0000","last_became_active":"2024-09-13T08:38:19.217347+0000","last_became_peered":"2024-09-13T08:38:19.217347+0000","last_unstale":"2024-09-13T08:38:21.362449+0000","last_undegraded":"2024-09-13T08:38:21.362449+0000","last_fullsized":"2024-09-13T08:38:21.362449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:28:16.642992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.206110+0000","last_change":"2024-09-13T08:38:20.209860+0000","last_active":"2024-09-13T08:38:21.206110+0000","last_peered":"2024-09-13T08:38:21.206110+0000","last_clean":"2024-09-13T08:38:21.206110+0000","last_became_active":"2024-09-13T08:38:20.208411+0000","last_became_peered":"2024-09-13T08:38:20.208411+0000","last_unstale":"2024-09-13T08:38:21.206110+0000","last_undegraded":"2024-09-13T08:38:21.206110+0000","last_fullsized":"2024-09-13T08:38:21.206110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:49.415502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363470+0000","last_change":"2024-09-13T08:38:19.206523+0000","last_active":"2024-09-13T08:38:21.363470+0000","last_peered":"2024-09-13T08:38:21.363470+0000","last_clean":"2024-09-13T08:38:21.363470+0000","last_became_active":"2024-09-13T08:38:19.205280+0000","last_became_peered":"2024-09-13T08:38:19.205280+0000","last_unstale":"2024-09-13T08:38:21.363470+0000","last_undegraded":"2024-09-13T08:38:21.363470+0000","last_fullsized":"2024-09-13T08:38:21.363470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:41:20.612280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363570+0000","last_change":"2024-09-13T08:38:20.214920+0000","last_active":"2024-09-13T08:38:21.363570+0000","last_peered":"2024-09-13T08:38:21.363570+0000","last_clean":"2024-09-13T08:38:21.363570+0000","last_became_active":"2024-09-13T08:38:20.214641+0000","last_became_peered":"2024-09-13T08:38:20.214641+0000","last_unstale":"2024-09-13T08:38:21.363570+0000","last_undegraded":"2024-09-13T08:38:21.363570+0000","last_fullsized":"2024-09-13T08:38:21.363570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:35.948244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362176+0000","last_change":"2024-09-13T08:38:19.215027+0000","last_active":"2024-09-13T08:38:21.362176+0000","last_peered":"2024-09-13T08:38:21.362176+0000","last_clean":"2024-09-13T08:38:21.362176+0000","last_became_active":"2024-09-13T08:38:19.214776+0000","last_became_peered":"2024-09-13T08:38:19.214776+0000","last_unstale":"2024-09-13T08:38:21.362176+0000","last_undegraded":"2024-09-13T08:38:21.362176+0000","last_fullsized":"2024-09-13T08:38:21.362176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:31.212694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.209467+0000","last_change":"2024-09-13T08:38:20.208392+0000","last_active":"2024-09-13T08:38:21.209467+0000","last_peered":"2024-09-13T08:38:21.209467+0000","last_clean":"2024-09-13T08:38:21.209467+0000","last_became_active":"2024-09-13T08:38:20.208122+0000","last_became_peered":"2024-09-13T08:38:20.208122+0000","last_unstale":"2024-09-13T08:38:21.209467+0000","last_undegraded":"2024-09-13T08:38:21.209467+0000","last_fullsized":"2024-09-13T08:38:21.209467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:43:43.902857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.205941+0000","last_change":"2024-09-13T08:38:19.210824+0000","last_active":"2024-09-13T08:38:21.205941+0000","last_peered":"2024-09-13T08:38:21.205941+0000","last_clean":"2024-09-13T08:38:21.205941+0000","last_became_active":"2024-09-13T08:38:19.210450+0000","last_became_peered":"2024-09-13T08:38:19.210450+0000","last_unstale":"2024-09-13T08:38:21.205941+0000","last_undegraded":"2024-09-13T08:38:21.205941+0000","last_fullsized":"2024-09-13T08:38:21.205941+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:55:07.696534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363371+0000","last_change":"2024-09-13T08:38:20.208374+0000","last_active":"2024-09-13T08:38:21.363371+0000","last_peered":"2024-09-13T08:38:21.363371+0000","last_clean":"2024-09-13T08:38:21.363371+0000","last_became_active":"2024-09-13T08:38:20.208046+0000","last_became_peered":"2024-09-13T08:38:20.208046+0000","last_unstale":"2024-09-13T08:38:21.363371+0000","last_undegraded":"2024-09-13T08:38:21.363371+0000","last_fullsized":"2024-09-13T08:38:21.363371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:24:52.258656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363178+0000","last_change":"2024-09-13T08:38:19.197102+0000","last_active":"2024-09-13T08:38:21.363178+0000","last_peered":"2024-09-13T08:38:21.363178+0000","last_clean":"2024-09-13T08:38:21.363178+0000","last_became_active":"2024-09-13T08:38:19.196612+0000","last_became_peered":"2024-09-13T08:38:19.196612+0000","last_unstale":"2024-09-13T08:38:21.363178+0000","last_undegraded":"2024-09-13T08:38:21.363178+0000","last_fullsized":"2024-09-13T08:38:21.363178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:39.209953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363981+0000","last_change":"2024-09-13T08:38:20.209471+0000","last_active":"2024-09-13T08:38:21.363981+0000","last_peered":"2024-09-13T08:38:21.363981+0000","last_clean":"2024-09-13T08:38:21.363981+0000","last_became_active":"2024-09-13T08:38:20.209246+0000","last_became_peered":"2024-09-13T08:38:20.209246+0000","last_unstale":"2024-09-13T08:38:21.363981+0000","last_undegraded":"2024-09-13T08:38:21.363981+0000","last_fullsized":"2024-09-13T08:38:21.363981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:31.609393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210091+0000","last_change":"2024-09-13T08:38:19.198905+0000","last_active":"2024-09-13T08:38:21.210091+0000","last_peered":"2024-09-13T08:38:21.210091+0000","last_clean":"2024-09-13T08:38:21.210091+0000","last_became_active":"2024-09-13T08:38:19.198615+0000","last_became_peered":"2024-09-13T08:38:19.198615+0000","last_unstale":"2024-09-13T08:38:21.210091+0000","last_undegraded":"2024-09-13T08:38:21.210091+0000","last_fullsized":"2024-09-13T08:38:21.210091+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:38:49.934138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210024+0000","last_change":"2024-09-13T08:38:20.204192+0000","last_active":"2024-09-13T08:38:21.210024+0000","last_peered":"2024-09-13T08:38:21.210024+0000","last_clean":"2024-09-13T08:38:21.210024+0000","last_became_active":"2024-09-13T08:38:20.203911+0000","last_became_peered":"2024-09-13T08:38:20.203911+0000","last_unstale":"2024-09-13T08:38:21.210024+0000","last_undegraded":"2024-09-13T08:38:21.210024+0000","last_fullsized":"2024-09-13T08:38:21.210024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:13.638616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.361686+0000","last_change":"2024-09-13T08:38:19.211567+0000","last_active":"2024-09-13T08:38:21.361686+0000","last_peered":"2024-09-13T08:38:21.361686+0000","last_clean":"2024-09-13T08:38:21.361686+0000","last_became_active":"2024-09-13T08:38:19.210135+0000","last_became_peered":"2024-09-13T08:38:19.210135+0000","last_unstale":"2024-09-13T08:38:21.361686+0000","last_undegraded":"2024-09-13T08:38:21.361686+0000","last_fullsized":"2024-09-13T08:38:21.361686+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:59:34.469638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362756+0000","last_change":"2024-09-13T08:38:20.200329+0000","last_active":"2024-09-13T08:38:21.362756+0000","last_peered":"2024-09-13T08:38:21.362756+0000","last_clean":"2024-09-13T08:38:21.362756+0000","last_became_active":"2024-09-13T08:38:20.199962+0000","last_became_peered":"2024-09-13T08:38:20.199962+0000","last_unstale":"2024-09-13T08:38:21.362756+0000","last_undegraded":"2024-09-13T08:38:21.362756+0000","last_fullsized":"2024-09-13T08:38:21.362756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:33:57.264520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363000+0000","last_change":"2024-09-13T08:38:19.201974+0000","last_active":"2024-09-13T08:38:21.363000+0000","last_peered":"2024-09-13T08:38:21.363000+0000","last_clean":"2024-09-13T08:38:21.363000+0000","last_became_active":"2024-09-13T08:38:19.201665+0000","last_became_peered":"2024-09-13T08:38:19.201665+0000","last_unstale":"2024-09-13T08:38:21.363000+0000","last_undegraded":"2024-09-13T08:38:21.363000+0000","last_fullsized":"2024-09-13T08:38:21.363000+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:18.603948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363076+0000","last_change":"2024-09-13T08:38:20.207761+0000","last_active":"2024-09-13T08:38:21.363076+0000","last_peered":"2024-09-13T08:38:21.363076+0000","last_clean":"2024-09-13T08:38:21.363076+0000","last_became_active":"2024-09-13T08:38:20.207408+0000","last_became_peered":"2024-09-13T08:38:20.207408+0000","last_unstale":"2024-09-13T08:38:21.363076+0000","last_undegraded":"2024-09-13T08:38:21.363076+0000","last_fullsized":"2024-09-13T08:38:21.363076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:34.773027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.205438+0000","last_change":"2024-09-13T08:38:19.209858+0000","last_active":"2024-09-13T08:38:21.205438+0000","last_peered":"2024-09-13T08:38:21.205438+0000","last_clean":"2024-09-13T08:38:21.205438+0000","last_became_active":"2024-09-13T08:38:19.209580+0000","last_became_peered":"2024-09-13T08:38:19.209580+0000","last_unstale":"2024-09-13T08:38:21.205438+0000","last_undegraded":"2024-09-13T08:38:21.205438+0000","last_fullsized":"2024-09-13T08:38:21.205438+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:15.979690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363480+0000","last_change":"2024-09-13T08:38:20.207753+0000","last_active":"2024-09-13T08:38:21.363480+0000","last_peered":"2024-09-13T08:38:21.363480+0000","last_clean":"2024-09-13T08:38:21.363480+0000","last_became_active":"2024-09-13T08:38:20.207450+0000","last_became_peered":"2024-09-13T08:38:20.207450+0000","last_unstale":"2024-09-13T08:38:21.363480+0000","last_undegraded":"2024-09-13T08:38:21.363480+0000","last_fullsized":"2024-09-13T08:38:21.363480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:50:20.542886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363892+0000","last_change":"2024-09-13T08:38:19.205074+0000","last_active":"2024-09-13T08:38:21.363892+0000","last_peered":"2024-09-13T08:38:21.363892+0000","last_clean":"2024-09-13T08:38:21.363892+0000","last_became_active":"2024-09-13T08:38:19.204207+0000","last_became_peered":"2024-09-13T08:38:19.204207+0000","last_unstale":"2024-09-13T08:38:21.363892+0000","last_undegraded":"2024-09-13T08:38:21.363892+0000","last_fullsized":"2024-09-13T08:38:21.363892+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:50:29.374728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.206163+0000","last_change":"2024-09-13T08:38:20.225113+0000","last_active":"2024-09-13T08:38:21.206163+0000","last_peered":"2024-09-13T08:38:21.206163+0000","last_clean":"2024-09-13T08:38:21.206163+0000","last_became_active":"2024-09-13T08:38:20.224832+0000","last_became_peered":"2024-09-13T08:38:20.224832+0000","last_unstale":"2024-09-13T08:38:21.206163+0000","last_undegraded":"2024-09-13T08:38:21.206163+0000","last_fullsized":"2024-09-13T08:38:21.206163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:05.466047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.209816+0000","last_change":"2024-09-13T08:38:19.201973+0000","last_active":"2024-09-13T08:38:21.209816+0000","last_peered":"2024-09-13T08:38:21.209816+0000","last_clean":"2024-09-13T08:38:21.209816+0000","last_became_active":"2024-09-13T08:38:19.201721+0000","last_became_peered":"2024-09-13T08:38:19.201721+0000","last_unstale":"2024-09-13T08:38:21.209816+0000","last_undegraded":"2024-09-13T08:38:21.209816+0000","last_fullsized":"2024-09-13T08:38:21.209816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:02.207647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363455+0000","last_change":"2024-09-13T08:38:20.212698+0000","last_active":"2024-09-13T08:38:21.363455+0000","last_peered":"2024-09-13T08:38:21.363455+0000","last_clean":"2024-09-13T08:38:21.363455+0000","last_became_active":"2024-09-13T08:38:20.212436+0000","last_became_peered":"2024-09-13T08:38:20.212436+0000","last_unstale":"2024-09-13T08:38:21.363455+0000","last_undegraded":"2024-09-13T08:38:21.363455+0000","last_fullsized":"2024-09-13T08:38:21.363455+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:19:27.959111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363485+0000","last_change":"2024-09-13T08:38:19.207256+0000","last_active":"2024-09-13T08:38:21.363485+0000","last_peered":"2024-09-13T08:38:21.363485+0000","last_clean":"2024-09-13T08:38:21.363485+0000","last_became_active":"2024-09-13T08:38:19.206641+0000","last_became_peered":"2024-09-13T08:38:19.206641+0000","last_unstale":"2024-09-13T08:38:21.363485+0000","last_undegraded":"2024-09-13T08:38:21.363485+0000","last_fullsized":"2024-09-13T08:38:21.363485+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:52.987289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362529+0000","last_change":"2024-09-13T08:38:20.202851+0000","last_active":"2024-09-13T08:38:21.362529+0000","last_peered":"2024-09-13T08:38:21.362529+0000","last_clean":"2024-09-13T08:38:21.362529+0000","last_became_active":"2024-09-13T08:38:20.202497+0000","last_became_peered":"2024-09-13T08:38:20.202497+0000","last_unstale":"2024-09-13T08:38:21.362529+0000","last_undegraded":"2024-09-13T08:38:21.362529+0000","last_fullsized":"2024-09-13T08:38:21.362529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:43:43.902857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363245+0000","last_change":"2024-09-13T08:38:19.203651+0000","last_active":"2024-09-13T08:38:21.363245+0000","last_peered":"2024-09-13T08:38:21.363245+0000","last_clean":"2024-09-13T08:38:21.363245+0000","last_became_active":"2024-09-13T08:38:19.203345+0000","last_became_peered":"2024-09-13T08:38:19.203345+0000","last_unstale":"2024-09-13T08:38:21.363245+0000","last_undegraded":"2024-09-13T08:38:21.363245+0000","last_fullsized":"2024-09-13T08:38:21.363245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:48:55.146479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210732+0000","last_change":"2024-09-13T08:38:20.215583+0000","last_active":"2024-09-13T08:38:21.210732+0000","last_peered":"2024-09-13T08:38:21.210732+0000","last_clean":"2024-09-13T08:38:21.210732+0000","last_became_active":"2024-09-13T08:38:20.215113+0000","last_became_peered":"2024-09-13T08:38:20.215113+0000","last_unstale":"2024-09-13T08:38:21.210732+0000","last_undegraded":"2024-09-13T08:38:21.210732+0000","last_fullsized":"2024-09-13T08:38:21.210732+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:31.984929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363133+0000","last_change":"2024-09-13T08:38:19.214900+0000","last_active":"2024-09-13T08:38:21.363133+0000","last_peered":"2024-09-13T08:38:21.363133+0000","last_clean":"2024-09-13T08:38:21.363133+0000","last_became_active":"2024-09-13T08:38:19.214649+0000","last_became_peered":"2024-09-13T08:38:19.214649+0000","last_unstale":"2024-09-13T08:38:21.363133+0000","last_undegraded":"2024-09-13T08:38:21.363133+0000","last_fullsized":"2024-09-13T08:38:21.363133+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:57.875378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210223+0000","last_change":"2024-09-13T08:38:20.220182+0000","last_active":"2024-09-13T08:38:21.210223+0000","last_peered":"2024-09-13T08:38:21.210223+0000","last_clean":"2024-09-13T08:38:21.210223+0000","last_became_active":"2024-09-13T08:38:20.219879+0000","last_became_peered":"2024-09-13T08:38:20.219879+0000","last_unstale":"2024-09-13T08:38:21.210223+0000","last_undegraded":"2024-09-13T08:38:21.210223+0000","last_fullsized":"2024-09-13T08:38:21.210223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:04.442980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.362665+0000","last_change":"2024-09-13T08:38:19.211753+0000","last_active":"2024-09-13T08:38:21.362665+0000","last_peered":"2024-09-13T08:38:21.362665+0000","last_clean":"2024-09-13T08:38:21.362665+0000","last_became_active":"2024-09-13T08:38:19.211462+0000","last_became_peered":"2024-09-13T08:38:19.211462+0000","last_unstale":"2024-09-13T08:38:21.362665+0000","last_undegraded":"2024-09-13T08:38:21.362665+0000","last_fullsized":"2024-09-13T08:38:21.362665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:40.504752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.205619+0000","last_change":"2024-09-13T08:38:20.211933+0000","last_active":"2024-09-13T08:38:21.205619+0000","last_peered":"2024-09-13T08:38:21.205619+0000","last_clean":"2024-09-13T08:38:21.205619+0000","last_became_active":"2024-09-13T08:38:20.211568+0000","last_became_peered":"2024-09-13T08:38:20.211568+0000","last_unstale":"2024-09-13T08:38:21.205619+0000","last_undegraded":"2024-09-13T08:38:21.205619+0000","last_fullsized":"2024-09-13T08:38:21.205619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:25:44.913210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.210200+0000","last_change":"2024-09-13T08:38:19.205325+0000","last_active":"2024-09-13T08:38:21.210200+0000","last_peered":"2024-09-13T08:38:21.210200+0000","last_clean":"2024-09-13T08:38:21.210200+0000","last_became_active":"2024-09-13T08:38:19.205088+0000","last_became_peered":"2024-09-13T08:38:19.205088+0000","last_unstale":"2024-09-13T08:38:21.210200+0000","last_undegraded":"2024-09-13T08:38:21.210200+0000","last_fullsized":"2024-09-13T08:38:21.210200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:34.940822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.363124+0000","last_change":"2024-09-13T08:38:20.203495+0000","last_active":"2024-09-13T08:38:21.363124+0000","last_peered":"2024-09-13T08:38:21.363124+0000","last_clean":"2024-09-13T08:38:21.363124+0000","last_became_active":"2024-09-13T08:38:20.203210+0000","last_became_peered":"2024-09-13T08:38:20.203210+0000","last_unstale":"2024-09-13T08:38:21.363124+0000","last_undegraded":"2024-09-13T08:38:21.363124+0000","last_fullsized":"2024-09-13T08:38:21.363124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:19.174381+0000","last_clean_scrub_stamp":"2024-09-13T08:38:19.174381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:22.746997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.206057+0000","last_change":"2024-09-13T08:38:19.210722+0000","last_active":"2024-09-13T08:38:21.206057+0000","last_peered":"2024-09-13T08:38:21.206057+0000","last_clean":"2024-09-13T08:38:21.206057+0000","last_became_active":"2024-09-13T08:38:19.210286+0000","last_became_peered":"2024-09-13T08:38:19.210286+0000","last_unstale":"2024-09-13T08:38:21.206057+0000","last_undegraded":"2024-09-13T08:38:21.206057+0000","last_fullsized":"2024-09-13T08:38:21.206057+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:38:18.166960+0000","last_clean_scrub_stamp":"2024-09-13T08:38:18.166960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:42.782548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":130,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T08:38:21.361866+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:38:21.361866+0000","last_peered":"2024-09-13T08:38:21.361866+0000","last_clean":"2024-09-13T08:38:21.361866+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:38:21.361866+0000","last_undegraded":"2024-09-13T08:38:21.361866+0000","last_fullsized":"2024-09-13T08:38:21.361866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":962560,"data_stored":730434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":380928,"data_stored":141118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":380928,"data_stored":140260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":962560,"data_stored":729922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542160,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":141096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:38:24.769 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:38:24.769 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T08:38:24.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 -- 172.21.15.44:0/3432673280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c50003ab0 msgr2=0x7f8c50003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 --2- 172.21.15.44:0/3432673280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c50003ab0 0x7f8c50003ed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0016c0 tx=0x7f8c5c01c730 comp rx=0 tx=0).stop 2024-09-13T08:38:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 -- 172.21.15.44:0/3432673280 shutdown_connections 2024-09-13T08:38:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 --2- 172.21.15.44:0/3432673280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c50003ab0 0x7f8c50003ed0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 -- 172.21.15.44:0/3432673280 >> 172.21.15.44:0/3432673280 conn(0x7f8c680d1dd0 msgr2=0x7f8c680d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 -- 172.21.15.44:0/3432673280 shutdown_connections 2024-09-13T08:38:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 -- 172.21.15.44:0/3432673280 wait complete. 2024-09-13T08:38:24.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 Processor -- start 2024-09-13T08:38:24.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 -- start start 2024-09-13T08:38:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c50003ab0 0x7f8c6814b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c681f72a0 0x7f8c6814add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8c6814b310 0x7f8c68147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.958+0000 7f8c6f400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c680b0e50 con 0x7f8c681f72a0 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c6f400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c680b0cd0 con 0x7f8c50003ab0 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c6f400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8c680b1220 con 0x7f8c6814b310 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c67e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c681f72a0 0x7f8c6814add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c67e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c681f72a0 0x7f8c6814add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56026/0 (socket says 172.21.15.44:56026) 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c67e00640 1 -- 172.21.15.44:0/2509610136 learned_addr learned my addr 172.21.15.44:0/2509610136 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c6ca00640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c50003ab0 0x7f8c6814b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c6d400640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8c6814b310 0x7f8c68147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c67e00640 1 -- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8c6814b310 msgr2=0x7f8c68147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c67e00640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8c6814b310 0x7f8c68147950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c67e00640 1 -- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c50003ab0 msgr2=0x7f8c6814b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c67e00640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c50003ab0 0x7f8c6814b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c67e00640 1 -- 172.21.15.44:0/2509610136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c681fab10 con 0x7f8c681f72a0 2024-09-13T08:38:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c6ca00640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c50003ab0 0x7f8c6814b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c67e00640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c681f72a0 0x7f8c6814add0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8c54006fb0 tx=0x7f8c540039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c64c00640 1 -- 172.21.15.44:0/2509610136 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54015070 con 0x7f8c681f72a0 2024-09-13T08:38:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c64c00640 1 -- 172.21.15.44:0/2509610136 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c54011920 con 0x7f8c681f72a0 2024-09-13T08:38:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c6f400640 1 -- 172.21.15.44:0/2509610136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6815cbd0 con 0x7f8c681f72a0 2024-09-13T08:38:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c64c00640 1 -- 172.21.15.44:0/2509610136 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54011d70 con 0x7f8c681f72a0 2024-09-13T08:38:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c6f400640 1 -- 172.21.15.44:0/2509610136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6815ce70 con 0x7f8c681f72a0 2024-09-13T08:38:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c6f400640 1 -- 172.21.15.44:0/2509610136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c680aeb00 con 0x7f8c681f72a0 2024-09-13T08:38:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.962+0000 7f8c64c00640 1 -- 172.21.15.44:0/2509610136 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f8c54018070 con 0x7f8c681f72a0 2024-09-13T08:38:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.966+0000 7f8c64c00640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8c48072ef0 0x7f8c480753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.966+0000 7f8c64c00640 1 -- 172.21.15.44:0/2509610136 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8c54094c70 con 0x7f8c681f72a0 2024-09-13T08:38:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.966+0000 7f8c6ca00640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8c48072ef0 0x7f8c480753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.966+0000 7f8c6ca00640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8c48072ef0 0x7f8c480753b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8c50004790 tx=0x7f8c5c019520 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:24.970+0000 7f8c64c00640 1 -- 172.21.15.44:0/2509610136 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8c5405fe30 con 0x7f8c681f72a0 2024-09-13T08:38:25.063 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-09-13T08:38:25.058+0000 7f37f6e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:38:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.206+0000 7f8c6f400640 1 -- 172.21.15.44:0/2509610136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8c680cc180 con 0x7f8c681f72a0 2024-09-13T08:38:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c64c00640 1 -- 172.21.15.44:0/2509610136 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f8c5405f730 con 0x7f8c681f72a0 2024-09-13T08:38:25.215 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T08:38:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:38:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-09-13T08:38:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-09-13T08:38:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-13T08:38:25.215 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-09-13T08:38:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 -- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8c48072ef0 msgr2=0x7f8c480753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8c48072ef0 0x7f8c480753b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8c50004790 tx=0x7f8c5c019520 comp rx=0 tx=0).stop 2024-09-13T08:38:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 -- 172.21.15.44:0/2509610136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c681f72a0 msgr2=0x7f8c6814add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:25.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c681f72a0 0x7f8c6814add0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8c54006fb0 tx=0x7f8c540039d0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 -- 172.21.15.44:0/2509610136 shutdown_connections 2024-09-13T08:38:25.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8c48072ef0 0x7f8c480753b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8c6814b310 0x7f8c68147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c681f72a0 0x7f8c6814add0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 --2- 172.21.15.44:0/2509610136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c50003ab0 0x7f8c6814b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 -- 172.21.15.44:0/2509610136 >> 172.21.15.44:0/2509610136 conn(0x7f8c680d1dd0 msgr2=0x7f8c68072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:25.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 -- 172.21.15.44:0/2509610136 shutdown_connections 2024-09-13T08:38:25.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.210+0000 7f8c6f400640 1 -- 172.21.15.44:0/2509610136 wait complete. 2024-09-13T08:38:25.237 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:38:25.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.378+0000 7f466f200640 1 -- 172.21.15.44:0/1402603271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c004810 msgr2=0x7f464c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:25.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.378+0000 7f466f200640 1 --2- 172.21.15.44:0/1402603271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c004810 0x7f464c004c90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f46500016c0 tx=0x7f465001c770 comp rx=0 tx=0).stop 2024-09-13T08:38:25.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.378+0000 7f466f200640 1 -- 172.21.15.44:0/1402603271 shutdown_connections 2024-09-13T08:38:25.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.378+0000 7f466f200640 1 --2- 172.21.15.44:0/1402603271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c004810 0x7f464c004c90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.378+0000 7f466f200640 1 -- 172.21.15.44:0/1402603271 >> 172.21.15.44:0/1402603271 conn(0x7f46680d1dc0 msgr2=0x7f46680d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:25.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.378+0000 7f466f200640 1 -- 172.21.15.44:0/1402603271 shutdown_connections 2024-09-13T08:38:25.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.378+0000 7f466f200640 1 -- 172.21.15.44:0/1402603271 wait complete. 2024-09-13T08:38:25.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466f200640 1 Processor -- start 2024-09-13T08:38:25.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466f200640 1 -- start start 2024-09-13T08:38:25.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f464c004810 0x7f4668152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:25.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46681f8820 0x7f46681f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:25.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466f200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f46681fadb0 0x7f46680b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:25.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46680ceab0 con 0x7f46681f8820 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46680ce930 con 0x7f464c004810 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466f200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f46680cec30 con 0x7f46681fadb0 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f465fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46681f8820 0x7f46681f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f465fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46681f8820 0x7f46681f7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56056/0 (socket says 172.21.15.44:56056) 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f465fe00640 1 -- 172.21.15.44:0/491658956 learned_addr learned my addr 172.21.15.44:0/491658956 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f4667e00640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f464c004810 0x7f4668152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466d200640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f46681fadb0 0x7f46680b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f465fe00640 1 -- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f46681fadb0 msgr2=0x7f46680b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f465fe00640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f46681fadb0 0x7f46680b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f465fe00640 1 -- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f464c004810 msgr2=0x7f4668152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f465fe00640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f464c004810 0x7f4668152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f465fe00640 1 -- 172.21.15.44:0/491658956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4668152bb0 con 0x7f46681f8820 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f465fe00640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46681f8820 0x7f46681f7790 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4650006040 tx=0x7f4650006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:25.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f4664c00640 1 -- 172.21.15.44:0/491658956 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650003680 con 0x7f46681f8820 2024-09-13T08:38:25.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f4664c00640 1 -- 172.21.15.44:0/491658956 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4650003910 con 0x7f46681f8820 2024-09-13T08:38:25.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466f200640 1 -- 172.21.15.44:0/491658956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46681f7cd0 con 0x7f46681f8820 2024-09-13T08:38:25.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f4664c00640 1 -- 172.21.15.44:0/491658956 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650003d60 con 0x7f46681f8820 2024-09-13T08:38:25.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.382+0000 7f466f200640 1 -- 172.21.15.44:0/491658956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46681f8260 con 0x7f46681f8820 2024-09-13T08:38:25.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.386+0000 7f466f200640 1 -- 172.21.15.44:0/491658956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46680cd420 con 0x7f46681f8820 2024-09-13T08:38:25.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.386+0000 7f4664c00640 1 -- 172.21.15.44:0/491658956 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f46500a5550 con 0x7f46681f8820 2024-09-13T08:38:25.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.386+0000 7f4664c00640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4644072ec0 0x7f4644075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:25.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.386+0000 7f4664c00640 1 -- 172.21.15.44:0/491658956 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f465012da60 con 0x7f46681f8820 2024-09-13T08:38:25.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.386+0000 7f4667e00640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4644072ec0 0x7f4644075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:25.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.386+0000 7f4667e00640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4644072ec0 0x7f4644075380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f466806d110 tx=0x7f46580091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.390+0000 7f4664c00640 1 -- 172.21.15.44:0/491658956 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f46500164d0 con 0x7f46681f8820 2024-09-13T08:38:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 -- 172.21.15.44:0/491658956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46680cd910 con 0x7f46681f8820 2024-09-13T08:38:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f4664c00640 1 -- 172.21.15.44:0/491658956 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7f46500f8ca0 con 0x7f46681f8820 2024-09-13T08:38:25.608 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:38:25.608 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"btime":"2024-09-13T08:38:23:214240+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4828,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/238336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":238336703},{"type":"v1","addr":"172.21.15.129:6836","nonce":238336703}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2440287716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2440287716},{"type":"v1","addr":"172.21.15.44:6834","nonce":2440287716}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3117701056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":3117701056},{"type":"v1","addr":"172.21.15.129:6837","nonce":3117701056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:38:19.608778+0000","modified":"2024-09-13T08:38:22.619955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4866":{"gid":4866,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3820157165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3820157165},{"type":"v1","addr":"172.21.15.44:6835","nonce":3820157165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4866,"qdb_cluster":[4866]},"id":2}]} 2024-09-13T08:38:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-09-13T08:38:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 -- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4644072ec0 msgr2=0x7f4644075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4644072ec0 0x7f4644075380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f466806d110 tx=0x7f46580091e0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 -- 172.21.15.44:0/491658956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46681f8820 msgr2=0x7f46681f7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46681f8820 0x7f46681f7790 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4650006040 tx=0x7f4650006950 comp rx=0 tx=0).stop 2024-09-13T08:38:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 -- 172.21.15.44:0/491658956 shutdown_connections 2024-09-13T08:38:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4644072ec0 0x7f4644075380 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f46681fadb0 0x7f46680b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46681f8820 0x7f46681f7790 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 --2- 172.21.15.44:0/491658956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f464c004810 0x7f4668152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 -- 172.21.15.44:0/491658956 >> 172.21.15.44:0/491658956 conn(0x7f46680d1dc0 msgr2=0x7f46680c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 -- 172.21.15.44:0/491658956 shutdown_connections 2024-09-13T08:38:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:25.602+0000 7f466f200640 1 -- 172.21.15.44:0/491658956 wait complete. 2024-09-13T08:38:26.032 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T08:38:19.608778+0000', 'modified': '2024-09-13T08:38:22.619955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4866}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4866': {'gid': 4866, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/3820157165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 3820157165}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 3820157165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4866, 'qdb_cluster': [4866]} 2024-09-13T08:38:26.033 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T08:38:26.033 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T08:38:26.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 -- 172.21.15.44:0/201366908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c200b2e80 msgr2=0x7f5c200b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 --2- 172.21.15.44:0/201366908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c200b2e80 0x7f5c200b1a00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5c20074c80 tx=0x7f5c0c01c770 comp rx=0 tx=0).stop 2024-09-13T08:38:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 -- 172.21.15.44:0/201366908 shutdown_connections 2024-09-13T08:38:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 --2- 172.21.15.44:0/201366908 >> v2:172.21.15.129:3300/0 conn(0x7f5c200b1f40 0x7f5c200cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 --2- 172.21.15.44:0/201366908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c200b2e80 0x7f5c200b1a00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 --2- 172.21.15.44:0/201366908 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c2015f490 0x7f5c200b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 -- 172.21.15.44:0/201366908 >> 172.21.15.44:0/201366908 conn(0x7f5c200d1e30 msgr2=0x7f5c200d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 -- 172.21.15.44:0/201366908 shutdown_connections 2024-09-13T08:38:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 -- 172.21.15.44:0/201366908 wait complete. 2024-09-13T08:38:26.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 Processor -- start 2024-09-13T08:38:26.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 -- start start 2024-09-13T08:38:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c200b1f40 0x7f5c201557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c200b2e80 0x7f5c200b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c2015f490 0x7f5c200b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c200b0010 con 0x7f5c200b1f40 2024-09-13T08:38:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c200afe90 con 0x7f5c200b2e80 2024-09-13T08:38:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.178+0000 7f5c25400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5c200b0190 con 0x7f5c2015f490 2024-09-13T08:38:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c200b1f40 0x7f5c201557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c200b1f40 0x7f5c201557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56088/0 (socket says 172.21.15.44:56088) 2024-09-13T08:38:26.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1e800640 1 -- 172.21.15.44:0/3480493340 learned_addr learned my addr 172.21.15.44:0/3480493340 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:26.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1f200640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c2015f490 0x7f5c200b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1de00640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c200b2e80 0x7f5c200b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1e800640 1 -- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c2015f490 msgr2=0x7f5c200b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1e800640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c2015f490 0x7f5c200b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1e800640 1 -- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c200b2e80 msgr2=0x7f5c200b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1e800640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c200b2e80 0x7f5c200b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1e800640 1 -- 172.21.15.44:0/3480493340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c20155d10 con 0x7f5c200b1f40 2024-09-13T08:38:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c1e800640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c200b1f40 0x7f5c201557d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000d780 tx=0x7f5c1000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c16a00640 1 -- 172.21.15.44:0/3480493340 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c10017070 con 0x7f5c200b1f40 2024-09-13T08:38:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c16a00640 1 -- 172.21.15.44:0/3480493340 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c1000de90 con 0x7f5c200b1f40 2024-09-13T08:38:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c25400640 1 -- 172.21.15.44:0/3480493340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c200cd240 con 0x7f5c200b1f40 2024-09-13T08:38:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c16a00640 1 -- 172.21.15.44:0/3480493340 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c10004020 con 0x7f5c200b1f40 2024-09-13T08:38:26.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c25400640 1 -- 172.21.15.44:0/3480493340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c200b74c0 con 0x7f5c200b1f40 2024-09-13T08:38:26.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.182+0000 7f5c16a00640 1 -- 172.21.15.44:0/3480493340 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f5c100041c0 con 0x7f5c200b1f40 2024-09-13T08:38:26.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c16a00640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5c00072ec0 0x7f5c00075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c16a00640 1 -- 172.21.15.44:0/3480493340 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5c10094130 con 0x7f5c200b1f40 2024-09-13T08:38:26.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c28777000 1 Processor -- start 2024-09-13T08:38:26.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1de00640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5c00072ec0 0x7f5c00075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c28777000 1 -- start start 2024-09-13T08:38:26.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1de00640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5c00072ec0 0x7f5c00075380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0af000 tx=0x7f5c0c01afb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c28777000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4ffdc0 0x55a42f5001a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c28777000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f5006e0 0x55a42f4b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c28777000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4b4220 0x55a42f4b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c28777000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a42f3d03f0 con 0x55a42f4b4220 2024-09-13T08:38:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c28777000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a42f30db20 con 0x55a42f4ffdc0 2024-09-13T08:38:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c28777000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55a42f2fcf50 con 0x55a42f5006e0 2024-09-13T08:38:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4b4220 0x55a42f4b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4b4220 0x55a42f4b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56090/0 (socket says 172.21.15.44:56090) 2024-09-13T08:38:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1f200640 1 -- 172.21.15.44:0/76764400 learned_addr learned my addr 172.21.15.44:0/76764400 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:26.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1e800640 1 --2- 172.21.15.44:0/76764400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4ffdc0 0x55a42f5001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1de00640 1 --2- 172.21.15.44:0/76764400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f5006e0 0x55a42f4b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1f200640 1 -- 172.21.15.44:0/76764400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f5006e0 msgr2=0x55a42f4b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:38:26.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1f200640 1 --2- 172.21.15.44:0/76764400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f5006e0 0x55a42f4b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1f200640 1 -- 172.21.15.44:0/76764400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4ffdc0 msgr2=0x55a42f5001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1f200640 1 --2- 172.21.15.44:0/76764400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4ffdc0 0x55a42f5001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1f200640 1 -- 172.21.15.44:0/76764400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42f2c4590 con 0x55a42f4b4220 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c1f200640 1 --2- 172.21.15.44:0/76764400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4b4220 0x55a42f4b4600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009eb0 tx=0x7f5c180117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879fa525e5addb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c0b400640 1 -- 172.21.15.44:0/76764400 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18004d30 con 0x55a42f4b4220 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c0b400640 1 -- 172.21.15.44:0/76764400 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c18003980 con 0x55a42f4b4220 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.186+0000 7f5c0b400640 1 -- 172.21.15.44:0/76764400 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18003d70 con 0x55a42f4b4220 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 -- 172.21.15.44:0/76764400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4b4220 msgr2=0x55a42f4b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 --2- 172.21.15.44:0/76764400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4b4220 0x55a42f4b4600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009eb0 tx=0x7f5c180117b0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 -- 172.21.15.44:0/76764400 shutdown_connections 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 --2- 172.21.15.44:0/76764400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4b4220 0x55a42f4b4600 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 --2- 172.21.15.44:0/76764400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f5006e0 0x55a42f4b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 --2- 172.21.15.44:0/76764400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4ffdc0 0x55a42f5001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 -- 172.21.15.44:0/76764400 >> 172.21.15.44:0/76764400 conn(0x55a42f399130 msgr2=0x55a42f4dee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:26.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 -- 172.21.15.44:0/76764400 shutdown_connections 2024-09-13T08:38:26.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 -- 172.21.15.44:0/76764400 wait complete. 2024-09-13T08:38:26.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 -- start start 2024-09-13T08:38:26.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 Processor -- start 2024-09-13T08:38:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 0x55a42f4ea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f4ea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f5006e0 0x55a42f4eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a42f2975d0 con 0x55a42f4ffdc0 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a42f300d60 con 0x55a42f4b4220 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55a42f2a3500 con 0x55a42f5006e0 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c28777000 10 client.0 _ll_register_callbacks cb 0x55a42f2a7810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f4ea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1e800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 0x55a42f4ea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1f200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f5006e0 0x55a42f4eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1e800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 0x55a42f4ea360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:45458/0 (socket says 172.21.15.44:45458) 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1e800640 1 -- 172.21.15.44:0/3471694875 learned_addr learned my addr 172.21.15.44:0/3471694875 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f4ea8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56098/0 (socket says 172.21.15.44:56098) 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1e800640 1 -- 172.21.15.44:0/3471694875 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f5006e0 msgr2=0x55a42f4eade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1e800640 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f5006e0 0x55a42f4eade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1e800640 1 -- 172.21.15.44:0/3471694875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 msgr2=0x55a42f4ea8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1e800640 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f4ea8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1e800640 1 -- 172.21.15.44:0/3471694875 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42f29b170 con 0x55a42f4b4220 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5c1e800640 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 0x55a42f4ea360 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5c100925c0 tx=0x7f5c100045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fff86caa1d74a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.190+0000 7f5bffe00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5bffe00640 1 -- 172.21.15.44:0/3471694875 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c10084070 con 0x55a42f4b4220 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5bffe00640 1 -- 172.21.15.44:0/3471694875 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5c1004c170 con 0x55a42f4b4220 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 -- 172.21.15.44:0/3471694875 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a42f2950f0 con 0x55a42f4b4220 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5bffe00640 1 -- 172.21.15.44:0/3471694875 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5c100636c0 con 0x55a42f4b4220 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 10 client.4975 fetch_fsmap learned FSMap version 17 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 -- 172.21.15.44:0/3471694875 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a42f381ef0 con 0x55a42f4b4220 2024-09-13T08:38:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5bffe00640 1 -- 172.21.15.44:0/3471694875 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a42f2950f0 con 0x55a42f4b4220 2024-09-13T08:38:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5bffe00640 1 -- 172.21.15.44:0/3471694875 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f5c10074b60 con 0x55a42f4b4220 2024-09-13T08:38:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 10 client.4975 fetch_fsmap finished waiting for FSMap version 17 2024-09-13T08:38:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 10 client.4975 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:38:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 20 client.4975 populate_metadata read hostname 'smithi044' 2024-09-13T08:38:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.44:6833/3820157165,v1:172.21.15.44:6835/3820157165] conn(0x55a42f4eb7f0 0x55a42f511070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 4 client.4975 mds_command: new command op to 4866 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T08:38:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 -- 172.21.15.44:0/3471694875 --> [v2:172.21.15.44:6833/3820157165,v1:172.21.15.44:6835/3820157165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a42f30fa70 con 0x55a42f4eb7f0 2024-09-13T08:38:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c1de00640 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.44:6833/3820157165,v1:172.21.15.44:6835/3820157165] conn(0x55a42f4eb7f0 0x55a42f511070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c1de00640 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.44:6833/3820157165,v1:172.21.15.44:6835/3820157165] conn(0x55a42f4eb7f0 0x55a42f511070 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248b1d83405139a6 server_cookie=1479a62fab34aee2 in_seq=0 out_seq=0 2024-09-13T08:38:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5bffe00640 10 client.4975 ms_handle_connect on v2:172.21.15.44:6833/3820157165 2024-09-13T08:38:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5bffe00640 1 -- 172.21.15.44:0/3471694875 <== mds.0 v2:172.21.15.44:6833/3820157165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a42f30fa70 con 0x55a42f4eb7f0 2024-09-13T08:38:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5bffe00640 10 client.4975 handle_command_reply: tid=0 2024-09-13T08:38:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 client.4975 shutdown 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 -- 172.21.15.44:0/3471694875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 msgr2=0x55a42f4ea360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 0x55a42f4ea360 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5c100925c0 tx=0x7f5c100045e0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 -- 172.21.15.44:0/3471694875 shutdown_connections 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.44:6833/3820157165,v1:172.21.15.44:6835/3820157165] conn(0x55a42f4eb7f0 0x55a42f511070 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f5006e0 0x55a42f4eade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f4ea8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5bffe00640 0 client.4975 ms_handle_reset on v2:172.21.15.44:6833/3820157165 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 --2- 172.21.15.44:0/3471694875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 0x55a42f4ea360 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 -- 172.21.15.44:0/3471694875 >> 172.21.15.44:0/3471694875 conn(0x55a42f399130 msgr2=0x55a42f4ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 -- 172.21.15.44:0/3471694875 shutdown_connections 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 1 -- 172.21.15.44:0/3471694875 wait complete. 2024-09-13T08:38:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.194+0000 7f5c28777000 20 client.4975 trim_cache size 0 max 16384 2024-09-13T08:38:26.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c28777000 1 Processor -- start 2024-09-13T08:38:26.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c28777000 1 -- start start 2024-09-13T08:38:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c28777000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f399130 0x55a42f4dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c28777000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f4b4220 0x55a42f4dc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c28777000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c28777000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2006f340 con 0x55a42f4ffdc0 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c28777000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a42f305390 con 0x55a42f399130 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c28777000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55a42f3bc730 con 0x55a42f4b4220 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56110/0 (socket says 172.21.15.44:56110) 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1f200640 1 -- 172.21.15.44:0/1068449246 learned_addr learned my addr 172.21.15.44:0/1068449246 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1e800640 1 --2- 172.21.15.44:0/1068449246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f399130 0x55a42f4dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1de00640 1 --2- 172.21.15.44:0/1068449246 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f4b4220 0x55a42f4dc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1f200640 1 -- 172.21.15.44:0/1068449246 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f4b4220 msgr2=0x55a42f4dc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1f200640 1 --2- 172.21.15.44:0/1068449246 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f4b4220 0x55a42f4dc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1f200640 1 -- 172.21.15.44:0/1068449246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f399130 msgr2=0x55a42f4dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1f200640 1 --2- 172.21.15.44:0/1068449246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f399130 0x55a42f4dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1f200640 1 -- 172.21.15.44:0/1068449246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42f358d90 con 0x55a42f4ffdc0 2024-09-13T08:38:26.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.214+0000 7f5c1f200640 1 --2- 172.21.15.44:0/1068449246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5579f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5c18004000 tx=0x7f5c18004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de701a26a9ccf8ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c0aa00640 1 -- 172.21.15.44:0/1068449246 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18003b80 con 0x55a42f4ffdc0 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c0aa00640 1 -- 172.21.15.44:0/1068449246 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c18003d40 con 0x55a42f4ffdc0 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c0aa00640 1 -- 172.21.15.44:0/1068449246 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18004290 con 0x55a42f4ffdc0 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 -- 172.21.15.44:0/1068449246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 msgr2=0x55a42f5579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 --2- 172.21.15.44:0/1068449246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5579f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5c18004000 tx=0x7f5c18004ac0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 -- 172.21.15.44:0/1068449246 shutdown_connections 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 --2- 172.21.15.44:0/1068449246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5579f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 --2- 172.21.15.44:0/1068449246 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f4b4220 0x55a42f4dc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 --2- 172.21.15.44:0/1068449246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f399130 0x55a42f4dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 -- 172.21.15.44:0/1068449246 >> 172.21.15.44:0/1068449246 conn(0x55a42f4b6f60 msgr2=0x55a42f4bd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 -- 172.21.15.44:0/1068449246 shutdown_connections 2024-09-13T08:38:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 -- 172.21.15.44:0/1068449246 wait complete. 2024-09-13T08:38:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 -- start start 2024-09-13T08:38:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 Processor -- start 2024-09-13T08:38:26.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f399130 0x55a42f576c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 0x55a42f577190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5776d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a42f30c8f0 con 0x55a42f4ffdc0 2024-09-13T08:38:26.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a42f381a60 con 0x55a42f4b4220 2024-09-13T08:38:26.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55a42f300070 con 0x55a42f399130 2024-09-13T08:38:26.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 10 client.0 _ll_register_callbacks cb 0x7f5c1005e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5776d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56116/0 (socket says 172.21.15.44:56116) 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1f200640 1 -- 172.21.15.44:0/1086743451 learned_addr learned my addr 172.21.15.44:0/1086743451 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1e800640 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f399130 0x55a42f576c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1de00640 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 0x55a42f577190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1f200640 1 -- 172.21.15.44:0/1086743451 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f399130 msgr2=0x55a42f576c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1f200640 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f399130 0x55a42f576c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1f200640 1 -- 172.21.15.44:0/1086743451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 msgr2=0x55a42f577190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1f200640 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 0x55a42f577190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1f200640 1 -- 172.21.15.44:0/1086743451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42f2c3a90 con 0x55a42f4ffdc0 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c1f200640 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5776d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009e80 tx=0x7f5c18003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9a598cf67167b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5bffe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:38:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5bffe00640 1 -- 172.21.15.44:0/1086743451 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18004d30 con 0x55a42f4ffdc0 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5bffe00640 1 -- 172.21.15.44:0/1086743451 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5c18004340 con 0x55a42f4ffdc0 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.218+0000 7f5c28777000 1 -- 172.21.15.44:0/1086743451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c18003d40 con 0x55a42f4ffdc0 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5bffe00640 1 -- 172.21.15.44:0/1086743451 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5c18038050 con 0x55a42f4ffdc0 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 10 client.5022 fetch_fsmap learned FSMap version 17 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 -- 172.21.15.44:0/1086743451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a42f2a2050 con 0x55a42f4ffdc0 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5bffe00640 1 -- 172.21.15.44:0/1086743451 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c18003d40 con 0x55a42f4ffdc0 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5bffe00640 1 -- 172.21.15.44:0/1086743451 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f5c1803b020 con 0x55a42f4ffdc0 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 10 client.5022 fetch_fsmap finished waiting for FSMap version 17 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 10 client.5022 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 20 client.5022 populate_metadata read hostname 'smithi044' 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.44:6833/3820157165,v1:172.21.15.44:6835/3820157165] conn(0x55a42f5006e0 0x55a42f514860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 4 client.5022 mds_command: new command op to 4866 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T08:38:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 -- 172.21.15.44:0/1086743451 --> [v2:172.21.15.44:6833/3820157165,v1:172.21.15.44:6835/3820157165] -- command(tid 0: {"prefix": "status"}) -- 0x55a42f30b520 con 0x55a42f5006e0 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c1e800640 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.44:6833/3820157165,v1:172.21.15.44:6835/3820157165] conn(0x55a42f5006e0 0x55a42f514860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c1e800640 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.44:6833/3820157165,v1:172.21.15.44:6835/3820157165] conn(0x55a42f5006e0 0x55a42f514860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace3f6db213a2ad6 server_cookie=328ebce4a7ec3eb5 in_seq=0 out_seq=0 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5bffe00640 10 client.5022 ms_handle_connect on v2:172.21.15.44:6833/3820157165 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5bffe00640 1 -- 172.21.15.44:0/1086743451 <== mds.0 v2:172.21.15.44:6833/3820157165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55a42f30b520 con 0x55a42f5006e0 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5bffe00640 10 client.5022 handle_command_reply: tid=0 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 client.5022 shutdown 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 -- 172.21.15.44:0/1086743451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 msgr2=0x55a42f5776d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5776d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009e80 tx=0x7f5c18003bb0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 -- 172.21.15.44:0/1086743451 shutdown_connections 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.44:6833/3820157165,v1:172.21.15.44:6835/3820157165] conn(0x55a42f5006e0 0x55a42f514860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a42f4ffdc0 0x55a42f5776d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a42f4b4220 0x55a42f577190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 --2- 172.21.15.44:0/1086743451 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55a42f399130 0x55a42f576c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 -- 172.21.15.44:0/1086743451 >> 172.21.15.44:0/1086743451 conn(0x55a42f4b6f60 msgr2=0x55a42f4daf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5bffe00640 0 client.5022 ms_handle_reset on v2:172.21.15.44:6833/3820157165 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 -- 172.21.15.44:0/1086743451 shutdown_connections 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 1 -- 172.21.15.44:0/1086743451 wait complete. 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c28777000 20 client.5022 trim_cache size 0 max 16384 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4866, 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T08:38:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T08:38:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T08:38:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.0440313980000004, 2024-09-13T08:38:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-09-13T08:38:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 33, 2024-09-13T08:38:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 33, 2024-09-13T08:38:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 12.998962126 2024-09-13T08:38:26.233 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 -- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5c00072ec0 msgr2=0x7f5c00075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5c00072ec0 0x7f5c00075380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0af000 tx=0x7f5c0c01afb0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 -- 172.21.15.44:0/3480493340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c200b1f40 msgr2=0x7f5c201557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c200b1f40 0x7f5c201557d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000d780 tx=0x7f5c1000dc40 comp rx=0 tx=0).stop 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 -- 172.21.15.44:0/3480493340 shutdown_connections 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5c00072ec0 0x7f5c00075380 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c2015f490 0x7f5c200b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c200b2e80 0x7f5c200b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 --2- 172.21.15.44:0/3480493340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c200b1f40 0x7f5c201557d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 -- 172.21.15.44:0/3480493340 >> 172.21.15.44:0/3480493340 conn(0x7f5c200d1e30 msgr2=0x7f5c200ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 -- 172.21.15.44:0/3480493340 shutdown_connections 2024-09-13T08:38:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.222+0000 7f5c25400640 1 -- 172.21.15.44:0/3480493340 wait complete. 2024-09-13T08:38:26.249 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:38:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.390+0000 7fd402400640 1 -- 172.21.15.44:0/3854510853 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3fc15f490 msgr2=0x7fd3fc0b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.390+0000 7fd402400640 1 --2- 172.21.15.44:0/3854510853 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3fc15f490 0x7fd3fc0b14c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009870 tx=0x7fd3ec01c710 comp rx=0 tx=0).stop 2024-09-13T08:38:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.390+0000 7fd402400640 1 -- 172.21.15.44:0/3854510853 shutdown_connections 2024-09-13T08:38:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.390+0000 7fd402400640 1 --2- 172.21.15.44:0/3854510853 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3fc15f490 0x7fd3fc0b14c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.390+0000 7fd402400640 1 -- 172.21.15.44:0/3854510853 >> 172.21.15.44:0/3854510853 conn(0x7fd3fc0d1e30 msgr2=0x7fd3fc0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.390+0000 7fd402400640 1 -- 172.21.15.44:0/3854510853 shutdown_connections 2024-09-13T08:38:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.390+0000 7fd402400640 1 -- 172.21.15.44:0/3854510853 wait complete. 2024-09-13T08:38:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.390+0000 7fd402400640 1 Processor -- start 2024-09-13T08:38:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.390+0000 7fd402400640 1 -- start start 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd402400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3fc0afca0 0x7fd3fc14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd402400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3fc0b2e80 0x7fd3fc14b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd402400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3fc15f490 0x7fd3fc15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd402400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc0b2290 con 0x7fd3fc15f490 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd402400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc0b2110 con 0x7fd3fc0afca0 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd402400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd3fc16a4a0 con 0x7fd3fc0b2e80 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3fc15f490 0x7fd3fc15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3fc15f490 0x7fd3fc15ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56130/0 (socket says 172.21.15.44:56130) 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3fbe00640 1 -- 172.21.15.44:0/4118988521 learned_addr learned my addr 172.21.15.44:0/4118988521 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3fb400640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3fc0afca0 0x7fd3fc14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3faa00640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3fc0b2e80 0x7fd3fc14b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3fbe00640 1 -- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3fc0b2e80 msgr2=0x7fd3fc14b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3fbe00640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3fc0b2e80 0x7fd3fc14b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3fbe00640 1 -- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3fc0afca0 msgr2=0x7fd3fc14e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3fbe00640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3fc0afca0 0x7fd3fc14e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3fbe00640 1 -- 172.21.15.44:0/4118988521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc0ce160 con 0x7fd3fc15f490 2024-09-13T08:38:26.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3fbe00640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3fc15f490 0x7fd3fc15ace0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400cbc0 tx=0x7fd3f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:26.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3f3400640 1 -- 172.21.15.44:0/4118988521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4016020 con 0x7fd3fc15f490 2024-09-13T08:38:26.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3f3400640 1 -- 172.21.15.44:0/4118988521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3f400ab00 con 0x7fd3fc15f490 2024-09-13T08:38:26.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd3f3400640 1 -- 172.21.15.44:0/4118988521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4012420 con 0x7fd3fc15f490 2024-09-13T08:38:26.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd402400640 1 -- 172.21.15.44:0/4118988521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc0c7ab0 con 0x7fd3fc15f490 2024-09-13T08:38:26.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd402400640 1 -- 172.21.15.44:0/4118988521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc0c8070 con 0x7fd3fc15f490 2024-09-13T08:38:26.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.394+0000 7fd402400640 1 -- 172.21.15.44:0/4118988521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3fc0b0130 con 0x7fd3fc15f490 2024-09-13T08:38:26.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.398+0000 7fd3f3400640 1 -- 172.21.15.44:0/4118988521 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fd3f4019020 con 0x7fd3fc15f490 2024-09-13T08:38:26.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.398+0000 7fd3f3400640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd3dc072ec0 0x7fd3dc075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:26.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.398+0000 7fd3f3400640 1 -- 172.21.15.44:0/4118988521 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd3f4094380 con 0x7fd3fc15f490 2024-09-13T08:38:26.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.398+0000 7fd3fb400640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd3dc072ec0 0x7fd3dc075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:26.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.398+0000 7fd3fb400640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd3dc072ec0 0x7fd3dc075380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0af000 tx=0x7fd3ec01af50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.402+0000 7fd3f3400640 1 -- 172.21.15.44:0/4118988521 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd3f4020020 con 0x7fd3fc15f490 2024-09-13T08:38:26.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.610+0000 7fd402400640 1 -- 172.21.15.44:0/4118988521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd3fc0cf390 con 0x7fd3fc15f490 2024-09-13T08:38:26.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd3f3400640 1 -- 172.21.15.44:0/4118988521 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fd3f405f0f0 con 0x7fd3fc15f490 2024-09-13T08:38:26.619 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:38:26.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 -- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd3dc072ec0 msgr2=0x7fd3dc075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd3dc072ec0 0x7fd3dc075380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0af000 tx=0x7fd3ec01af50 comp rx=0 tx=0).stop 2024-09-13T08:38:26.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 -- 172.21.15.44:0/4118988521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3fc15f490 msgr2=0x7fd3fc15ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3fc15f490 0x7fd3fc15ace0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400cbc0 tx=0x7fd3f400ccb0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 -- 172.21.15.44:0/4118988521 shutdown_connections 2024-09-13T08:38:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd3dc072ec0 0x7fd3dc075380 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3fc15f490 0x7fd3fc15ace0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3fc0b2e80 0x7fd3fc14b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 --2- 172.21.15.44:0/4118988521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3fc0afca0 0x7fd3fc14e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 -- 172.21.15.44:0/4118988521 >> 172.21.15.44:0/4118988521 conn(0x7fd3fc0d1e30 msgr2=0x7fd3fc0ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 -- 172.21.15.44:0/4118988521 shutdown_connections 2024-09-13T08:38:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26.614+0000 7fd402400640 1 -- 172.21.15.44:0/4118988521 wait complete. 2024-09-13T08:38:26.640 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T08:38:26.640 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:38:26.640 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:38:26.640 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T08:38:26.641 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:38:26.641 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T08:38:26.641 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T08:38:26.641 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:38:26.641 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:38:26.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:26.641 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:38:26.650 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:38:26.650 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:38:26.651 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42376sec preferred_lft 42376sec 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:38:26.652 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:26.653 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:38:26.654 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:38:26.654 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T08:38:26.654 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:38:26.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:26.655 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:38:26.719 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:38:26.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:26.722 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:38:26.779 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:38:26.779 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:38:26.780 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:38:26.780 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:38:26.780 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:38:26.780 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:38:26.780 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:38:26.780 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:38:26.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:38:26.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:38:26.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:38:26.780 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:38:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:38:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:38:26.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:26.923 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:38:26.932 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:38:26.932 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T08:38:26.932 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:38:26.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:26.932 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T08:38:26.993 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:38:26.993 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T08:38:27.046 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:38:27.046 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:38:27.046 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:38:27.097 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:38:27.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:27.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:38:27.186 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:38:27.186 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:38:27.186 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:38:27.202 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:38:27.202 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:38:27.202 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:38:27.202 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:38:27.202 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:38:27.212 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:38:27.212 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:38:27.212 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:38:27.212 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:38:27.212 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:38:27.213 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:38:27.213 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:38:27.213 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:38:27.216 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:38:27.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:27.216 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:38:27.237 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:38:27.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:38:27.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:38:27.346 INFO:teuthology.orchestra.run.smithi044.stdout:client4835 2024-09-13T08:38:27.347 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:27.347 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:27.356 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-13T08:38:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.546+0000 7f8257000640 1 -- 172.21.15.44:0/2008348774 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f825015eff0 msgr2=0x7f825015f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.546+0000 7f8257000640 1 --2- 172.21.15.44:0/2008348774 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f825015eff0 0x7f825015f460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8248009c80 tx=0x7f824801cd70 comp rx=0 tx=0).stop 2024-09-13T08:38:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.546+0000 7f8257000640 1 -- 172.21.15.44:0/2008348774 shutdown_connections 2024-09-13T08:38:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.546+0000 7f8257000640 1 --2- 172.21.15.44:0/2008348774 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f825015eff0 0x7f825015f460 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.546+0000 7f8257000640 1 --2- 172.21.15.44:0/2008348774 >> v2:172.21.15.129:3300/0 conn(0x7f82500d09f0 0x7f82500d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.546+0000 7f8257000640 1 -- 172.21.15.44:0/2008348774 >> 172.21.15.44:0/2008348774 conn(0x7f82500d1da0 msgr2=0x7f82500d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.546+0000 7f8257000640 1 -- 172.21.15.44:0/2008348774 shutdown_connections 2024-09-13T08:38:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.546+0000 7f8257000640 1 -- 172.21.15.44:0/2008348774 wait complete. 2024-09-13T08:38:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 Processor -- start 2024-09-13T08:38:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 -- start start 2024-09-13T08:38:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82500cff00 0x7f82500b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825015eff0 0x7f8250145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82500b7950 0x7f82500cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f82500b0ca0 con 0x7f825015eff0 2024-09-13T08:38:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f82500b0b20 con 0x7f82500cff00 2024-09-13T08:38:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8250168bf0 con 0x7f82500b7950 2024-09-13T08:38:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8255200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825015eff0 0x7f8250145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8255200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825015eff0 0x7f8250145140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56134/0 (socket says 172.21.15.44:56134) 2024-09-13T08:38:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8255200640 1 -- 172.21.15.44:0/411144349 learned_addr learned my addr 172.21.15.44:0/411144349 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8255c00640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82500cff00 0x7f82500b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8256600640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82500b7950 0x7f82500cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8255200640 1 -- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82500b7950 msgr2=0x7f82500cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8255200640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82500b7950 0x7f82500cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8255200640 1 -- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82500cff00 msgr2=0x7f82500b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8255200640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82500cff00 0x7f82500b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8255200640 1 -- 172.21.15.44:0/411144349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82500cce20 con 0x7f825015eff0 2024-09-13T08:38:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8255200640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825015eff0 0x7f8250145140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f823c00ea20 tx=0x7f823c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f824d600640 1 -- 172.21.15.44:0/411144349 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c00cd80 con 0x7f825015eff0 2024-09-13T08:38:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f824d600640 1 -- 172.21.15.44:0/411144349 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f823c010430 con 0x7f825015eff0 2024-09-13T08:38:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 -- 172.21.15.44:0/411144349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82500cc880 con 0x7f825015eff0 2024-09-13T08:38:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f824d600640 1 -- 172.21.15.44:0/411144349 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c010860 con 0x7f825015eff0 2024-09-13T08:38:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 -- 172.21.15.44:0/411144349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82500b7150 con 0x7f825015eff0 2024-09-13T08:38:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.550+0000 7f8257000640 1 -- 172.21.15.44:0/411144349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82500d28c0 con 0x7f825015eff0 2024-09-13T08:38:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.554+0000 7f824d600640 1 -- 172.21.15.44:0/411144349 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f823c010a00 con 0x7f825015eff0 2024-09-13T08:38:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.554+0000 7f824d600640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8230072ef0 0x7f82300753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.554+0000 7f824d600640 1 -- 172.21.15.44:0/411144349 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f823c094990 con 0x7f825015eff0 2024-09-13T08:38:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.554+0000 7f8255c00640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8230072ef0 0x7f82300753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.554+0000 7f8255c00640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8230072ef0 0x7f82300753b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8244007fd0 tx=0x7f8244009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.558+0000 7f824d600640 1 -- 172.21.15.44:0/411144349 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f823c05f4f0 con 0x7f825015eff0 2024-09-13T08:38:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:27.770+0000 7f8257000640 1 -- 172.21.15.44:0/411144349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f82500b2f20 con 0x7f825015eff0 2024-09-13T08:38:28.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.230+0000 7f824d600640 1 -- 172.21.15.44:0/411144349 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7f823c063f20 con 0x7f825015eff0 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 -- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8230072ef0 msgr2=0x7f82300753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8230072ef0 0x7f82300753b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8244007fd0 tx=0x7f8244009040 comp rx=0 tx=0).stop 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 -- 172.21.15.44:0/411144349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825015eff0 msgr2=0x7f8250145140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825015eff0 0x7f8250145140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f823c00ea20 tx=0x7f823c00eee0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 -- 172.21.15.44:0/411144349 shutdown_connections 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8230072ef0 0x7f82300753b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82500b7950 0x7f82500cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825015eff0 0x7f8250145140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 --2- 172.21.15.44:0/411144349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82500cff00 0x7f82500b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 -- 172.21.15.44:0/411144349 >> 172.21.15.44:0/411144349 conn(0x7f82500d1da0 msgr2=0x7f8250071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 -- 172.21.15.44:0/411144349 shutdown_connections 2024-09-13T08:38:28.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.234+0000 7f8257000640 1 -- 172.21.15.44:0/411144349 wait complete. 2024-09-13T08:38:28.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:38:28.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.402+0000 7fa5c3200640 1 -- 172.21.15.44:0/428257240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c40cde90 msgr2=0x7fa5c40cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.402+0000 7fa5c3200640 1 --2- 172.21.15.44:0/428257240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c40cde90 0x7fa5c40cbfa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009900 tx=0x7fa5ac01c880 comp rx=0 tx=0).stop 2024-09-13T08:38:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.402+0000 7fa5c3200640 1 -- 172.21.15.44:0/428257240 shutdown_connections 2024-09-13T08:38:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.402+0000 7fa5c3200640 1 --2- 172.21.15.44:0/428257240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c40cde90 0x7fa5c40cbfa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.402+0000 7fa5c3200640 1 --2- 172.21.15.44:0/428257240 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5c415a420 0x7fa5c40af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.402+0000 7fa5c3200640 1 -- 172.21.15.44:0/428257240 >> 172.21.15.44:0/428257240 conn(0x7fa5c4073490 msgr2=0x7fa5c40d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.402+0000 7fa5c3200640 1 -- 172.21.15.44:0/428257240 shutdown_connections 2024-09-13T08:38:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.402+0000 7fa5c3200640 1 -- 172.21.15.44:0/428257240 wait complete. 2024-09-13T08:38:28.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c3200640 1 Processor -- start 2024-09-13T08:38:28.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c3200640 1 -- start start 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c40ccca0 0x7fa5c40b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c3200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5c40cde90 0x7fa5c40b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c3200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c415a420 0x7fa5c41faf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4165200 con 0x7fa5c40ccca0 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c3200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4165080 con 0x7fa5c415a420 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c3200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c4165380 con 0x7fa5c40cde90 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c40ccca0 0x7fa5c40b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c40ccca0 0x7fa5c40b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56136/0 (socket says 172.21.15.44:56136) 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c1e00640 1 -- 172.21.15.44:0/4138608310 learned_addr learned my addr 172.21.15.44:0/4138608310 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5bbe00640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5c40cde90 0x7fa5c40b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c1e00640 1 -- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5c40cde90 msgr2=0x7fa5c40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c1e00640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5c40cde90 0x7fa5c40b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c1e00640 1 -- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c415a420 msgr2=0x7fa5c41faf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c2800640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c415a420 0x7fa5c41faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c1e00640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c415a420 0x7fa5c41faf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c1e00640 1 -- 172.21.15.44:0/4138608310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c40b36f0 con 0x7fa5c40ccca0 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5bbe00640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5c40cde90 0x7fa5c40b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c2800640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c415a420 0x7fa5c41faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c1e00640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c40ccca0 0x7fa5c40b3110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400e970 tx=0x7fa5b400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5ba000640 1 -- 172.21.15.44:0/4138608310 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400cd80 con 0x7fa5c40ccca0 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5ba000640 1 -- 172.21.15.44:0/4138608310 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5b4004020 con 0x7fa5c40ccca0 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c3200640 1 -- 172.21.15.44:0/4138608310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c40b39d0 con 0x7fa5c40ccca0 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5ba000640 1 -- 172.21.15.44:0/4138608310 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4004470 con 0x7fa5c40ccca0 2024-09-13T08:38:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.406+0000 7fa5c3200640 1 -- 172.21.15.44:0/4138608310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c40b3f90 con 0x7fa5c40ccca0 2024-09-13T08:38:28.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.410+0000 7fa5c3200640 1 -- 172.21.15.44:0/4138608310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c40cbfa0 con 0x7fa5c40ccca0 2024-09-13T08:38:28.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.410+0000 7fa5ba000640 1 -- 172.21.15.44:0/4138608310 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fa5b4003600 con 0x7fa5c40ccca0 2024-09-13T08:38:28.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.410+0000 7fa5ba000640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5a0072ec0 0x7fa5a0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:28.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.410+0000 7fa5ba000640 1 -- 172.21.15.44:0/4138608310 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa5b4094800 con 0x7fa5c40ccca0 2024-09-13T08:38:28.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.410+0000 7fa5bbe00640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5a0072ec0 0x7fa5a0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:28.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.410+0000 7fa5bbe00640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5a0072ec0 0x7fa5a0075380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0af000 tx=0x7fa5ac01b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.414+0000 7fa5ba000640 1 -- 172.21.15.44:0/4138608310 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa5b405f9b0 con 0x7fa5c40ccca0 2024-09-13T08:38:28.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.610+0000 7fa5c3200640 1 -- 172.21.15.44:0/4138608310 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa5c40c3b90 con 0x7fa5a0072ec0 2024-09-13T08:38:28.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.614+0000 7fa5ba000640 1 -- 172.21.15.44:0/4138608310 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa5c40c3b90 con 0x7fa5a0072ec0 2024-09-13T08:38:28.620 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:38:28.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:38:28.620 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:38:28.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:38:28.621 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:38:28.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 -- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5a0072ec0 msgr2=0x7fa5a0075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5a0072ec0 0x7fa5a0075380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0af000 tx=0x7fa5ac01b120 comp rx=0 tx=0).stop 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 -- 172.21.15.44:0/4138608310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c40ccca0 msgr2=0x7fa5c40b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c40ccca0 0x7fa5c40b3110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400e970 tx=0x7fa5b400ee30 comp rx=0 tx=0).stop 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 -- 172.21.15.44:0/4138608310 shutdown_connections 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5a0072ec0 0x7fa5a0075380 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c415a420 0x7fa5c41faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5c40cde90 0x7fa5c40b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 --2- 172.21.15.44:0/4138608310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c40ccca0 0x7fa5c40b3110 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 -- 172.21.15.44:0/4138608310 >> 172.21.15.44:0/4138608310 conn(0x7fa5c4073490 msgr2=0x7fa5c41593c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 -- 172.21.15.44:0/4138608310 shutdown_connections 2024-09-13T08:38:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.618+0000 7fa5c3200640 1 -- 172.21.15.44:0/4138608310 wait complete. 2024-09-13T08:38:28.641 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:38:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.786+0000 7f8422000640 1 -- 172.21.15.44:0/2450229613 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f841c0afb10 msgr2=0x7f841c0affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.786+0000 7f8422000640 1 --2- 172.21.15.44:0/2450229613 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f841c0afb10 0x7f841c0affc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8414009b70 tx=0x7f841401c930 comp rx=0 tx=0).stop 2024-09-13T08:38:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.786+0000 7f8422000640 1 -- 172.21.15.44:0/2450229613 shutdown_connections 2024-09-13T08:38:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.786+0000 7f8422000640 1 --2- 172.21.15.44:0/2450229613 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f841c0afb10 0x7f841c0affc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.786+0000 7f8422000640 1 -- 172.21.15.44:0/2450229613 >> 172.21.15.44:0/2450229613 conn(0x7f841c0d1dd0 msgr2=0x7f841c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.786+0000 7f8422000640 1 -- 172.21.15.44:0/2450229613 shutdown_connections 2024-09-13T08:38:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.786+0000 7f8422000640 1 -- 172.21.15.44:0/2450229613 wait complete. 2024-09-13T08:38:28.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.786+0000 7f8422000640 1 Processor -- start 2024-09-13T08:38:28.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.786+0000 7f8422000640 1 -- start start 2024-09-13T08:38:28.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8422000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f841c0afb10 0x7f841c0b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:28.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8422000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f841c0b1650 0x7f841c15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:28.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8422000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f841c1605d0 0x7f841c14de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:28.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8422000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f841c0b1220 con 0x7f841c0b1650 2024-09-13T08:38:28.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8422000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f841c0b0ee0 con 0x7f841c1605d0 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8422000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f841c0b13a0 con 0x7f841c0afb10 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f841c0b1650 0x7f841c15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f841c0b1650 0x7f841c15cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56142/0 (socket says 172.21.15.44:56142) 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413e00640 1 -- 172.21.15.44:0/2395330565 learned_addr learned my addr 172.21.15.44:0/2395330565 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f841b400640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f841c0afb10 0x7f841c0b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f841be00640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f841c1605d0 0x7f841c14de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413e00640 1 -- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f841c0afb10 msgr2=0x7f841c0b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413e00640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f841c0afb10 0x7f841c0b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413e00640 1 -- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f841c1605d0 msgr2=0x7f841c14de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413e00640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f841c1605d0 0x7f841c14de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413e00640 1 -- 172.21.15.44:0/2395330565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c0c9d90 con 0x7f841c0b1650 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f841be00640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f841c1605d0 0x7f841c14de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413e00640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f841c0b1650 0x7f841c15cd00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8404009e00 tx=0x7f840400c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:28.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413400640 1 -- 172.21.15.44:0/2395330565 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840400ce10 con 0x7f841c0b1650 2024-09-13T08:38:28.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413400640 1 -- 172.21.15.44:0/2395330565 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8404004130 con 0x7f841c0b1650 2024-09-13T08:38:28.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8413400640 1 -- 172.21.15.44:0/2395330565 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84040087f0 con 0x7f841c0b1650 2024-09-13T08:38:28.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8422000640 1 -- 172.21.15.44:0/2395330565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841c0ca070 con 0x7f841c0b1650 2024-09-13T08:38:28.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8422000640 1 -- 172.21.15.44:0/2395330565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841c0ca630 con 0x7f841c0b1650 2024-09-13T08:38:28.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.790+0000 7f8422000640 1 -- 172.21.15.44:0/2395330565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841c0b0730 con 0x7f841c0b1650 2024-09-13T08:38:28.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.794+0000 7f8413400640 1 -- 172.21.15.44:0/2395330565 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f840401b020 con 0x7f841c0b1650 2024-09-13T08:38:28.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.794+0000 7f8413400640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83f8072ec0 0x7f83f8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:28.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.794+0000 7f8413400640 1 -- 172.21.15.44:0/2395330565 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8404095e50 con 0x7f841c0b1650 2024-09-13T08:38:28.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.794+0000 7f841b400640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83f8072ec0 0x7f83f8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:28.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.794+0000 7f841b400640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83f8072ec0 0x7f83f8075380 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f840c001ec0 tx=0x7f840c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.798+0000 7f8413400640 1 -- 172.21.15.44:0/2395330565 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8404061090 con 0x7f841c0b1650 2024-09-13T08:38:28.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.982+0000 7f8422000640 1 -- 172.21.15.44:0/2395330565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f841c0d1080 con 0x7f841c0b1650 2024-09-13T08:38:28.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.982+0000 7f8413400640 1 -- 172.21.15.44:0/2395330565 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f8404060990 con 0x7f841c0b1650 2024-09-13T08:38:28.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 -- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83f8072ec0 msgr2=0x7f83f8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83f8072ec0 0x7f83f8075380 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f840c001ec0 tx=0x7f840c009040 comp rx=0 tx=0).stop 2024-09-13T08:38:28.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 -- 172.21.15.44:0/2395330565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f841c0b1650 msgr2=0x7f841c15cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:28.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f841c0b1650 0x7f841c15cd00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8404009e00 tx=0x7f840400c430 comp rx=0 tx=0).stop 2024-09-13T08:38:28.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 -- 172.21.15.44:0/2395330565 shutdown_connections 2024-09-13T08:38:28.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83f8072ec0 0x7f83f8075380 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f841c1605d0 0x7f841c14de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f841c0b1650 0x7f841c15cd00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 --2- 172.21.15.44:0/2395330565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f841c0afb10 0x7f841c0b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:28.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 -- 172.21.15.44:0/2395330565 >> 172.21.15.44:0/2395330565 conn(0x7f841c0d1dd0 msgr2=0x7f841c071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:28.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 -- 172.21.15.44:0/2395330565 shutdown_connections 2024-09-13T08:38:28.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:28.986+0000 7f8422000640 1 -- 172.21.15.44:0/2395330565 wait complete. 2024-09-13T08:38:29.011 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-13T08:38:29.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.154+0000 7fd328c00640 1 -- 172.21.15.44:0/1153512855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3240ce7b0 msgr2=0x7fd3240cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.154+0000 7fd328c00640 1 --2- 172.21.15.44:0/1153512855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3240ce7b0 0x7fd3240cf9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd3240b7d20 tx=0x7fd31c01cb50 comp rx=0 tx=0).stop 2024-09-13T08:38:29.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.154+0000 7fd328c00640 1 -- 172.21.15.44:0/1153512855 shutdown_connections 2024-09-13T08:38:29.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.154+0000 7fd328c00640 1 --2- 172.21.15.44:0/1153512855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3240ce7b0 0x7fd3240cf9c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.154+0000 7fd328c00640 1 --2- 172.21.15.44:0/1153512855 >> v2:172.21.15.129:3300/0 conn(0x7fd3240b1190 0x7fd3240ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.154+0000 7fd328c00640 1 -- 172.21.15.44:0/1153512855 >> 172.21.15.44:0/1153512855 conn(0x7fd3240d1df0 msgr2=0x7fd3240d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:29.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.154+0000 7fd328c00640 1 -- 172.21.15.44:0/1153512855 shutdown_connections 2024-09-13T08:38:29.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.154+0000 7fd328c00640 1 -- 172.21.15.44:0/1153512855 wait complete. 2024-09-13T08:38:29.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.154+0000 7fd328c00640 1 Processor -- start 2024-09-13T08:38:29.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.154+0000 7fd328c00640 1 -- start start 2024-09-13T08:38:29.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd328c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3240b1190 0x7fd32406f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd328c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3240ce7b0 0x7fd3240b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd328c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3241603f0 0x7fd324158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd328c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3240b3290 con 0x7fd3241603f0 2024-09-13T08:38:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd328c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3240b2b80 con 0x7fd3240b1190 2024-09-13T08:38:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd328c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd3240b3410 con 0x7fd3240ce7b0 2024-09-13T08:38:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd322800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3241603f0 0x7fd324158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd322800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3241603f0 0x7fd324158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56162/0 (socket says 172.21.15.44:56162) 2024-09-13T08:38:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd322800640 1 -- 172.21.15.44:0/2034047134 learned_addr learned my addr 172.21.15.44:0/2034047134 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:29.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd321e00640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3240b1190 0x7fd32406f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd319400640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3240ce7b0 0x7fd3240b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd322800640 1 -- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3240ce7b0 msgr2=0x7fd3240b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd322800640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3240ce7b0 0x7fd3240b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd322800640 1 -- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3240b1190 msgr2=0x7fd32406f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd322800640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3240b1190 0x7fd32406f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd322800640 1 -- 172.21.15.44:0/2034047134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3240b5f30 con 0x7fd3241603f0 2024-09-13T08:38:29.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd319400640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3240ce7b0 0x7fd3240b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:29.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd322800640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3241603f0 0x7fd324158810 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009b10 tx=0x7fd31c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:29.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd31aa00640 1 -- 172.21.15.44:0/2034047134 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0bb070 con 0x7fd3241603f0 2024-09-13T08:38:29.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd31aa00640 1 -- 172.21.15.44:0/2034047134 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd31c004d40 con 0x7fd3241603f0 2024-09-13T08:38:29.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd328c00640 1 -- 172.21.15.44:0/2034047134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3240b6210 con 0x7fd3241603f0 2024-09-13T08:38:29.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd31aa00640 1 -- 172.21.15.44:0/2034047134 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0b2040 con 0x7fd3241603f0 2024-09-13T08:38:29.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd328c00640 1 -- 172.21.15.44:0/2034047134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3240b67d0 con 0x7fd3241603f0 2024-09-13T08:38:29.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.158+0000 7fd328c00640 1 -- 172.21.15.44:0/2034047134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3240ce270 con 0x7fd3241603f0 2024-09-13T08:38:29.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.162+0000 7fd31aa00640 1 -- 172.21.15.44:0/2034047134 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fd31c0be070 con 0x7fd3241603f0 2024-09-13T08:38:29.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.162+0000 7fd31aa00640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd300072ef0 0x7fd3000753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.162+0000 7fd31aa00640 1 -- 172.21.15.44:0/2034047134 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd31c12da10 con 0x7fd3241603f0 2024-09-13T08:38:29.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.162+0000 7fd321e00640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd300072ef0 0x7fd3000753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.162+0000 7fd321e00640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd300072ef0 0x7fd3000753b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd314001400 tx=0x7fd314009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:29.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.166+0000 7fd31aa00640 1 -- 172.21.15.44:0/2034047134 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd31c016770 con 0x7fd3241603f0 2024-09-13T08:38:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 -- 172.21.15.44:0/2034047134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fd3240b7d70 con 0x7fd3241603f0 2024-09-13T08:38:29.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd31aa00640 1 -- 172.21.15.44:0/2034047134 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v28) ==== 116+0+2 (secure 0 0 0) 0x7fd31c0f8c50 con 0x7fd3241603f0 2024-09-13T08:38:29.351 INFO:teuthology.orchestra.run.smithi044.stdout:4 2024-09-13T08:38:29.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 -- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd300072ef0 msgr2=0x7fd3000753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd300072ef0 0x7fd3000753b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd314001400 tx=0x7fd314009040 comp rx=0 tx=0).stop 2024-09-13T08:38:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 -- 172.21.15.44:0/2034047134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3241603f0 msgr2=0x7fd324158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3241603f0 0x7fd324158810 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009b10 tx=0x7fd31c009300 comp rx=0 tx=0).stop 2024-09-13T08:38:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 -- 172.21.15.44:0/2034047134 shutdown_connections 2024-09-13T08:38:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd300072ef0 0x7fd3000753b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3241603f0 0x7fd324158810 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3240ce7b0 0x7fd3240b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 --2- 172.21.15.44:0/2034047134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3240b1190 0x7fd32406f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 -- 172.21.15.44:0/2034047134 >> 172.21.15.44:0/2034047134 conn(0x7fd3240d1df0 msgr2=0x7fd3240d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 -- 172.21.15.44:0/2034047134 shutdown_connections 2024-09-13T08:38:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.346+0000 7fd328c00640 1 -- 172.21.15.44:0/2034047134 wait complete. 2024-09-13T08:38:29.373 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:38:29.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.518+0000 7f0d15c00640 1 -- 172.21.15.44:0/3838818194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf4005220 msgr2=0x7f0cf400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.518+0000 7f0d15c00640 1 --2- 172.21.15.44:0/3838818194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf4005220 0x7f0cf400b9d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c560 tx=0x7f0d0801f9c0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.518+0000 7f0d15c00640 1 -- 172.21.15.44:0/3838818194 shutdown_connections 2024-09-13T08:38:29.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.518+0000 7f0d15c00640 1 --2- 172.21.15.44:0/3838818194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf4005220 0x7f0cf400b9d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.518+0000 7f0d15c00640 1 -- 172.21.15.44:0/3838818194 >> 172.21.15.44:0/3838818194 conn(0x7f0d100cce90 msgr2=0x7f0d100cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:29.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.518+0000 7f0d15c00640 1 -- 172.21.15.44:0/3838818194 shutdown_connections 2024-09-13T08:38:29.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.518+0000 7f0d15c00640 1 -- 172.21.15.44:0/3838818194 wait complete. 2024-09-13T08:38:29.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.518+0000 7f0d15c00640 1 Processor -- start 2024-09-13T08:38:29.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.518+0000 7f0d15c00640 1 -- start start 2024-09-13T08:38:29.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d15c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cf4005220 0x7f0d1014f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d15c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d101f5ac0 0x7f0d100b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d15c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d100b2c10 0x7f0d100cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d15c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1006f420 con 0x7f0d100b2c10 2024-09-13T08:38:29.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d15c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1006f2a0 con 0x7f0cf4005220 2024-09-13T08:38:29.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d15c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1006d630 con 0x7f0d101f5ac0 2024-09-13T08:38:29.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d0f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d100b2c10 0x7f0d100cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d0f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d100b2c10 0x7f0d100cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56182/0 (socket says 172.21.15.44:56182) 2024-09-13T08:38:29.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d0ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cf4005220 0x7f0d1014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d07e00640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d101f5ac0 0x7f0d100b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d0f400640 1 -- 172.21.15.44:0/126077912 learned_addr learned my addr 172.21.15.44:0/126077912 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:29.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d0f400640 1 -- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d101f5ac0 msgr2=0x7f0d100b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d0f400640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d101f5ac0 0x7f0d100b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d0f400640 1 -- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cf4005220 msgr2=0x7f0d1014f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d0f400640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cf4005220 0x7f0d1014f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d0f400640 1 -- 172.21.15.44:0/126077912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d101f8a50 con 0x7f0d100b2c10 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d0f400640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d100b2c10 0x7f0d100cba00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0d08006040 tx=0x7f0d08001fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d06a00640 1 -- 172.21.15.44:0/126077912 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08003b80 con 0x7f0d100b2c10 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d06a00640 1 -- 172.21.15.44:0/126077912 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d08003e10 con 0x7f0d100b2c10 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d15c00640 1 -- 172.21.15.44:0/126077912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d101f7ed0 con 0x7f0d100b2c10 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d06a00640 1 -- 172.21.15.44:0/126077912 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08006330 con 0x7f0d100b2c10 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d15c00640 1 -- 172.21.15.44:0/126077912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d101f8490 con 0x7f0d100b2c10 2024-09-13T08:38:29.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.522+0000 7f0d15c00640 1 -- 172.21.15.44:0/126077912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1015f840 con 0x7f0d100b2c10 2024-09-13T08:38:29.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.526+0000 7f0d06a00640 1 -- 172.21.15.44:0/126077912 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f0d080064d0 con 0x7f0d100b2c10 2024-09-13T08:38:29.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.526+0000 7f0d06a00640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cec072ef0 0x7f0cec0753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.526+0000 7f0d06a00640 1 -- 172.21.15.44:0/126077912 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0d08130ff0 con 0x7f0d100b2c10 2024-09-13T08:38:29.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.526+0000 7f0d0ea00640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cec072ef0 0x7f0cec0753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.526+0000 7f0d0ea00640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cec072ef0 0x7f0cec0753b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0d00000c10 tx=0x7f0d00006b20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:29.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.530+0000 7f0d06a00640 1 -- 172.21.15.44:0/126077912 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0d08019770 con 0x7f0d100b2c10 2024-09-13T08:38:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.702+0000 7f0d15c00640 1 -- 172.21.15.44:0/126077912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f0d101507f0 con 0x7f0d100b2c10 2024-09-13T08:38:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.706+0000 7f0d06a00640 1 -- 172.21.15.44:0/126077912 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v29)=0 v29) ==== 131+0+0 (secure 0 0 0) 0x7f0d080fc440 con 0x7f0d100b2c10 2024-09-13T08:38:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 -- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cec072ef0 msgr2=0x7f0cec0753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cec072ef0 0x7f0cec0753b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0d00000c10 tx=0x7f0d00006b20 comp rx=0 tx=0).stop 2024-09-13T08:38:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 -- 172.21.15.44:0/126077912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d100b2c10 msgr2=0x7f0d100cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d100b2c10 0x7f0d100cba00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0d08006040 tx=0x7f0d08001fc0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 -- 172.21.15.44:0/126077912 shutdown_connections 2024-09-13T08:38:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cec072ef0 0x7f0cec0753b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d100b2c10 0x7f0d100cba00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d101f5ac0 0x7f0d100b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 --2- 172.21.15.44:0/126077912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cf4005220 0x7f0d1014f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 -- 172.21.15.44:0/126077912 >> 172.21.15.44:0/126077912 conn(0x7f0d100cce90 msgr2=0x7f0d1014f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:29.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 -- 172.21.15.44:0/126077912 shutdown_connections 2024-09-13T08:38:29.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.710+0000 7f0d15c00640 1 -- 172.21.15.44:0/126077912 wait complete. 2024-09-13T08:38:29.735 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group1 2024-09-13T08:38:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 -- 172.21.15.44:0/379366527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f00b1650 msgr2=0x7f77f00b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 --2- 172.21.15.44:0/379366527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f00b1650 0x7f77f00b1a30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009900 tx=0x7f77dc01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 -- 172.21.15.44:0/379366527 shutdown_connections 2024-09-13T08:38:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 --2- 172.21.15.44:0/379366527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77f00afb10 0x7f77f00affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 --2- 172.21.15.44:0/379366527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f00b1650 0x7f77f00b1a30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 -- 172.21.15.44:0/379366527 >> 172.21.15.44:0/379366527 conn(0x7f77f00d1dd0 msgr2=0x7f77f00d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 -- 172.21.15.44:0/379366527 shutdown_connections 2024-09-13T08:38:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 -- 172.21.15.44:0/379366527 wait complete. 2024-09-13T08:38:29.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 Processor -- start 2024-09-13T08:38:29.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 -- start start 2024-09-13T08:38:29.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77f00afb10 0x7f77f014ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77f00b1650 0x7f77f014c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f01605d0 0x7f77f00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77f00b0ec0 con 0x7f77f01605d0 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77f00b1430 con 0x7f77f00afb10 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f8200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f77f0169540 con 0x7f77f00b1650 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f01605d0 0x7f77f00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f01605d0 0x7f77f00cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56200/0 (socket says 172.21.15.44:56200) 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f6200640 1 -- 172.21.15.44:0/2920224893 learned_addr learned my addr 172.21.15.44:0/2920224893 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f4e00640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77f00b1650 0x7f77f014c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f5800640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77f00afb10 0x7f77f014ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f6200640 1 -- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77f00b1650 msgr2=0x7f77f014c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f6200640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77f00b1650 0x7f77f014c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f6200640 1 -- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77f00afb10 msgr2=0x7f77f014ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:29.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f6200640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77f00afb10 0x7f77f014ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:29.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f6200640 1 -- 172.21.15.44:0/2920224893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f014d560 con 0x7f77f01605d0 2024-09-13T08:38:29.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.882+0000 7f77f5800640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77f00afb10 0x7f77f014ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:38:29.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77f6200640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f01605d0 0x7f77f00cd8b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f77e800daa0 tx=0x7f77e800df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:29.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77ed600640 1 -- 172.21.15.44:0/2920224893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8017070 con 0x7f77f01605d0 2024-09-13T08:38:29.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77ed600640 1 -- 172.21.15.44:0/2920224893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77e8004b30 con 0x7f77f01605d0 2024-09-13T08:38:29.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77ed600640 1 -- 172.21.15.44:0/2920224893 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8005020 con 0x7f77f01605d0 2024-09-13T08:38:29.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77f8200640 1 -- 172.21.15.44:0/2920224893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f014d840 con 0x7f77f01605d0 2024-09-13T08:38:29.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77f8200640 1 -- 172.21.15.44:0/2920224893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f014de00 con 0x7f77f01605d0 2024-09-13T08:38:29.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77f8200640 1 -- 172.21.15.44:0/2920224893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f00affc0 con 0x7f77f01605d0 2024-09-13T08:38:29.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77ed600640 1 -- 172.21.15.44:0/2920224893 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7f77e80051c0 con 0x7f77f01605d0 2024-09-13T08:38:29.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77ed600640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77d0072ec0 0x7f77d0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:29.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77ed600640 1 -- 172.21.15.44:0/2920224893 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f77e8094510 con 0x7f77f01605d0 2024-09-13T08:38:29.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.886+0000 7f77f5800640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77d0072ec0 0x7f77d0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:29.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.890+0000 7f77f5800640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77d0072ec0 0x7f77d0075380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77e4000f70 tx=0x7f77e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:29.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:29.890+0000 7f77ed600640 1 -- 172.21.15.44:0/2920224893 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f77e8020070 con 0x7f77f01605d0 2024-09-13T08:38:30.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.094+0000 7f77f8200640 1 -- 172.21.15.44:0/2920224893 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f77f00d1080 con 0x7f77d0072ec0 2024-09-13T08:38:30.107 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:38:30.104+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:38:30.108 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:38:30.104+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:38:30.108 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:38:30.104+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:38:30.108 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:38:30.104+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:38:30.108 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:38:30.104+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:38:30.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77ed600640 1 -- 172.21.15.44:0/2920224893 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f77f00d1080 con 0x7f77d0072ec0 2024-09-13T08:38:30.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 -- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77d0072ec0 msgr2=0x7f77d0075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77d0072ec0 0x7f77d0075380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77e4000f70 tx=0x7f77e4009040 comp rx=0 tx=0).stop 2024-09-13T08:38:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 -- 172.21.15.44:0/2920224893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f01605d0 msgr2=0x7f77f00cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f01605d0 0x7f77f00cd8b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f77e800daa0 tx=0x7f77e800df60 comp rx=0 tx=0).stop 2024-09-13T08:38:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 -- 172.21.15.44:0/2920224893 shutdown_connections 2024-09-13T08:38:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77d0072ec0 0x7f77d0075380 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f01605d0 0x7f77f00cd8b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77f00b1650 0x7f77f014c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 --2- 172.21.15.44:0/2920224893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77f00afb10 0x7f77f014ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 -- 172.21.15.44:0/2920224893 >> 172.21.15.44:0/2920224893 conn(0x7f77f00d1dd0 msgr2=0x7f77f0155600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 -- 172.21.15.44:0/2920224893 shutdown_connections 2024-09-13T08:38:30.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.122+0000 7f77f8200640 1 -- 172.21.15.44:0/2920224893 wait complete. 2024-09-13T08:38:30.149 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 group1 --mode=777 2024-09-13T08:38:30.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.294+0000 7fe102600640 1 -- 172.21.15.44:0/3023018278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0fc0d1180 msgr2=0x7fe0fc0b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:30.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.294+0000 7fe102600640 1 --2- 172.21.15.44:0/3023018278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0fc0d1180 0x7fe0fc0b48f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009c80 tx=0x7fe0f401cda0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.294+0000 7fe102600640 1 -- 172.21.15.44:0/3023018278 shutdown_connections 2024-09-13T08:38:30.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.294+0000 7fe102600640 1 --2- 172.21.15.44:0/3023018278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0fc0d1180 0x7fe0fc0b48f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.294+0000 7fe102600640 1 --2- 172.21.15.44:0/3023018278 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe0fc15abc0 0x7fe0fc0d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.294+0000 7fe102600640 1 -- 172.21.15.44:0/3023018278 >> 172.21.15.44:0/3023018278 conn(0x7fe0fc06dd30 msgr2=0x7fe0fc06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:30.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.294+0000 7fe102600640 1 -- 172.21.15.44:0/3023018278 shutdown_connections 2024-09-13T08:38:30.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.294+0000 7fe102600640 1 -- 172.21.15.44:0/3023018278 wait complete. 2024-09-13T08:38:30.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe102600640 1 Processor -- start 2024-09-13T08:38:30.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe102600640 1 -- start start 2024-09-13T08:38:30.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe102600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0fc0d1180 0x7fe0fc1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:30.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe102600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0fc15a7e0 0x7fe0fc15c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:30.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe102600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe0fc15abc0 0x7fe0fc1f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:30.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe102600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc167ea0 con 0x7fe0fc0d1180 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe102600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc167d20 con 0x7fe0fc15a7e0 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe102600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe0fc168020 con 0x7fe0fc15abc0 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0fb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0fc0d1180 0x7fe0fc1472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0fb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0fc0d1180 0x7fe0fc1472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56220/0 (socket says 172.21.15.44:56220) 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0fb400640 1 -- 172.21.15.44:0/1418256767 learned_addr learned my addr 172.21.15.44:0/1418256767 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0f2a00640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0fc15a7e0 0x7fe0fc15c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0fbe00640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe0fc15abc0 0x7fe0fc1f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0fb400640 1 -- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe0fc15abc0 msgr2=0x7fe0fc1f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0fb400640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe0fc15abc0 0x7fe0fc1f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0fb400640 1 -- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0fc15a7e0 msgr2=0x7fe0fc15c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0fb400640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0fc15a7e0 0x7fe0fc15c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0fb400640 1 -- 172.21.15.44:0/1418256767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc15cb70 con 0x7fe0fc0d1180 2024-09-13T08:38:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0f2a00640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0fc15a7e0 0x7fe0fc15c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:38:30.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0fb400640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0fc0d1180 0x7fe0fc1472f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00c500 tx=0x7fe0ec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:30.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0f3e00640 1 -- 172.21.15.44:0/1418256767 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec017070 con 0x7fe0fc0d1180 2024-09-13T08:38:30.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0f3e00640 1 -- 172.21.15.44:0/1418256767 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0ec0053c0 con 0x7fe0fc0d1180 2024-09-13T08:38:30.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe0f3e00640 1 -- 172.21.15.44:0/1418256767 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec012890 con 0x7fe0fc0d1180 2024-09-13T08:38:30.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe102600640 1 -- 172.21.15.44:0/1418256767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc1f7670 con 0x7fe0fc0d1180 2024-09-13T08:38:30.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.298+0000 7fe102600640 1 -- 172.21.15.44:0/1418256767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc1f7c30 con 0x7fe0fc0d1180 2024-09-13T08:38:30.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.302+0000 7fe102600640 1 -- 172.21.15.44:0/1418256767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0fc0b48f0 con 0x7fe0fc0d1180 2024-09-13T08:38:30.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.302+0000 7fe0f3e00640 1 -- 172.21.15.44:0/1418256767 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fe0ec01a070 con 0x7fe0fc0d1180 2024-09-13T08:38:30.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.302+0000 7fe0f3e00640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe0d8072ec0 0x7fe0d8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:30.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.302+0000 7fe0f3e00640 1 -- 172.21.15.44:0/1418256767 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe0ec0931c0 con 0x7fe0fc0d1180 2024-09-13T08:38:30.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.302+0000 7fe0f2a00640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe0d8072ec0 0x7fe0d8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.302+0000 7fe0f2a00640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe0d8072ec0 0x7fe0d8075380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc155a10 tx=0x7fe0e400b5b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.306+0000 7fe0f3e00640 1 -- 172.21.15.44:0/1418256767 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe0fc15cb70 con 0x7fe0fc0d1180 2024-09-13T08:38:30.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.506+0000 7fe102600640 1 -- 172.21.15.44:0/1418256767 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fe0fc1649b0 con 0x7fe0d8072ec0 2024-09-13T08:38:30.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe0f3e00640 1 -- 172.21.15.44:0/1418256767 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe0fc1649b0 con 0x7fe0d8072ec0 2024-09-13T08:38:30.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 -- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe0d8072ec0 msgr2=0x7fe0d8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:30.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe0d8072ec0 0x7fe0d8075380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc155a10 tx=0x7fe0e400b5b0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 -- 172.21.15.44:0/1418256767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0fc0d1180 msgr2=0x7fe0fc1472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:30.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0fc0d1180 0x7fe0fc1472f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00c500 tx=0x7fe0ec00c9c0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 -- 172.21.15.44:0/1418256767 shutdown_connections 2024-09-13T08:38:30.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe0d8072ec0 0x7fe0d8075380 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe0fc15abc0 0x7fe0fc1f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0fc15a7e0 0x7fe0fc15c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 --2- 172.21.15.44:0/1418256767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0fc0d1180 0x7fe0fc1472f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 -- 172.21.15.44:0/1418256767 >> 172.21.15.44:0/1418256767 conn(0x7fe0fc06dd30 msgr2=0x7fe0fc0ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:30.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 -- 172.21.15.44:0/1418256767 shutdown_connections 2024-09-13T08:38:30.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.642+0000 7fe102600640 1 -- 172.21.15.44:0/1418256767 wait complete. 2024-09-13T08:38:30.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-13T08:38:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.814+0000 7fa3f1e00640 1 -- 172.21.15.44:0/1810490175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d00051d0 msgr2=0x7fa3d000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.814+0000 7fa3f1e00640 1 --2- 172.21.15.44:0/1810490175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d00051d0 0x7fa3d000b9d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009ee0 tx=0x7fa3e401d630 comp rx=0 tx=0).stop 2024-09-13T08:38:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.814+0000 7fa3f1e00640 1 -- 172.21.15.44:0/1810490175 shutdown_connections 2024-09-13T08:38:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.814+0000 7fa3f1e00640 1 --2- 172.21.15.44:0/1810490175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d00051d0 0x7fa3d000b9d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.814+0000 7fa3f1e00640 1 -- 172.21.15.44:0/1810490175 >> 172.21.15.44:0/1810490175 conn(0x7fa3ec0d1da0 msgr2=0x7fa3ec0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:30.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.814+0000 7fa3f1e00640 1 -- 172.21.15.44:0/1810490175 shutdown_connections 2024-09-13T08:38:30.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.814+0000 7fa3f1e00640 1 -- 172.21.15.44:0/1810490175 wait complete. 2024-09-13T08:38:30.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.814+0000 7fa3f1e00640 1 Processor -- start 2024-09-13T08:38:30.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.814+0000 7fa3f1e00640 1 -- start start 2024-09-13T08:38:30.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d00051d0 0x7fa3ec0c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3ec0b2890 0x7fa3ec06f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa3ec0b2cf0 0x7fa3ec14f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3f1e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec0cea90 con 0x7fa3d00051d0 2024-09-13T08:38:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3f1e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec0ce910 con 0x7fa3ec0b2890 2024-09-13T08:38:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3f1e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ec0cec10 con 0x7fa3ec0b2cf0 2024-09-13T08:38:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3eaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d00051d0 0x7fa3ec0c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3eaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d00051d0 0x7fa3ec0c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56238/0 (socket says 172.21.15.44:56238) 2024-09-13T08:38:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3eaa00640 1 -- 172.21.15.44:0/3602913330 learned_addr learned my addr 172.21.15.44:0/3602913330 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:38:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3e3e00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3ec0b2890 0x7fa3ec06f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3eb400640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa3ec0b2cf0 0x7fa3ec14f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3eaa00640 1 -- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa3ec0b2cf0 msgr2=0x7fa3ec14f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3eaa00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa3ec0b2cf0 0x7fa3ec14f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3eaa00640 1 -- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3ec0b2890 msgr2=0x7fa3ec06f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3eaa00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3ec0b2890 0x7fa3ec06f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3eaa00640 1 -- 172.21.15.44:0/3602913330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec0ca160 con 0x7fa3d00051d0 2024-09-13T08:38:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3eaa00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d00051d0 0x7fa3ec0c9c20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc003eb0 tx=0x7fa3dc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3e3400640 1 -- 172.21.15.44:0/3602913330 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc015070 con 0x7fa3d00051d0 2024-09-13T08:38:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3e3400640 1 -- 172.21.15.44:0/3602913330 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3dc011840 con 0x7fa3d00051d0 2024-09-13T08:38:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3e3400640 1 -- 172.21.15.44:0/3602913330 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc011c50 con 0x7fa3d00051d0 2024-09-13T08:38:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3f1e00640 1 -- 172.21.15.44:0/3602913330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ec0c81f0 con 0x7fa3d00051d0 2024-09-13T08:38:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3f1e00640 1 -- 172.21.15.44:0/3602913330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ec0c7b30 con 0x7fa3d00051d0 2024-09-13T08:38:30.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.818+0000 7fa3f1e00640 1 -- 172.21.15.44:0/3602913330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ec0cf170 con 0x7fa3d00051d0 2024-09-13T08:38:30.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.822+0000 7fa3e3400640 1 -- 172.21.15.44:0/3602913330 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 97080+0+0 (secure 0 0 0) 0x7fa3dc018070 con 0x7fa3d00051d0 2024-09-13T08:38:30.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.822+0000 7fa3e3400640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3c8072ef0 0x7fa3c80753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:38:30.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.822+0000 7fa3e3400640 1 -- 172.21.15.44:0/3602913330 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa3dc0949c0 con 0x7fa3d00051d0 2024-09-13T08:38:30.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.822+0000 7fa3e3e00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3c8072ef0 0x7fa3c80753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:38:30.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.822+0000 7fa3e3e00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3c8072ef0 0x7fa3c80753b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0c99b0 tx=0x7fa3d40067b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:38:30.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:30.826+0000 7fa3e3400640 1 -- 172.21.15.44:0/3602913330 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa3dc05f5a0 con 0x7fa3d00051d0 2024-09-13T08:38:31.006 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:38:31.009 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:38:31.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.030+0000 7fa3f1e00640 1 -- 172.21.15.44:0/3602913330 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fa3ec0c55e0 con 0x7fa3c8072ef0 2024-09-13T08:38:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.034+0000 7fa3e3400640 1 -- 172.21.15.44:0/3602913330 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7fa3ec0c55e0 con 0x7fa3c8072ef0 2024-09-13T08:38:31.041 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367 2024-09-13T08:38:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 -- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3c8072ef0 msgr2=0x7fa3c80753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3c8072ef0 0x7fa3c80753b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0c99b0 tx=0x7fa3d40067b0 comp rx=0 tx=0).stop 2024-09-13T08:38:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 -- 172.21.15.44:0/3602913330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d00051d0 msgr2=0x7fa3ec0c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:38:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d00051d0 0x7fa3ec0c9c20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc003eb0 tx=0x7fa3dc001640 comp rx=0 tx=0).stop 2024-09-13T08:38:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 -- 172.21.15.44:0/3602913330 shutdown_connections 2024-09-13T08:38:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3c8072ef0 0x7fa3c80753b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa3ec0b2cf0 0x7fa3ec14f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3ec0b2890 0x7fa3ec06f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 --2- 172.21.15.44:0/3602913330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d00051d0 0x7fa3ec0c9c20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:38:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 -- 172.21.15.44:0/3602913330 >> 172.21.15.44:0/3602913330 conn(0x7fa3ec0d1da0 msgr2=0x7fa3ec0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:38:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 -- 172.21.15.44:0/3602913330 shutdown_connections 2024-09-13T08:38:31.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:38:31.038+0000 7fa3f1e00640 1 -- 172.21.15.44:0/3602913330 wait complete. 2024-09-13T08:38:31.063 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367 2024-09-13T08:38:31.063 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:31.063 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:31.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:31.114 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:38:40.290 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:38:40.300 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:38:40.300 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.11222 s, 118 MB/s 2024-09-13T08:38:40.301 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:40.301 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:40.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:40.309 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:38:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:38:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:38:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.0303 s, 119 MB/s 2024-09-13T08:38:49.392 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:49.392 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:49.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:49.402 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:38:59.141 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:38:59.142 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:38:59.142 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.69025 s, 111 MB/s 2024-09-13T08:38:59.143 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:38:59.143 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:38:59.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:38:59.152 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:39:01.064 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:39:01.069 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:39:08.572 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:39:08.572 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:39:08.573 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.36484 s, 115 MB/s 2024-09-13T08:39:08.574 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:39:08.574 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:39:08.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:39:08.583 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:39:18.044 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:39:18.044 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:39:18.044 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.40499 s, 114 MB/s 2024-09-13T08:39:18.045 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:39:18.046 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:39:18.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:39:18.055 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:39:27.512 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:39:27.513 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:39:27.513 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.40473 s, 114 MB/s 2024-09-13T08:39:27.514 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:39:27.514 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:39:27.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:39:27.523 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:39:31.127 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:39:31.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:39:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:39:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:39:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.08579 s, 118 MB/s 2024-09-13T08:39:36.665 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:39:36.666 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:39:36.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:39:36.675 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:39:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:39:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:39:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.35277 s, 115 MB/s 2024-09-13T08:39:46.082 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:39:46.082 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:39:46.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:39:46.094 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:39:55.683 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:39:55.684 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:39:55.684 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.53579 s, 113 MB/s 2024-09-13T08:39:55.686 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:39:55.686 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:39:55.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:39:55.695 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:40:01.185 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:40:01.189 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:40:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:40:05.117 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:40:05.117 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.36971 s, 115 MB/s 2024-09-13T08:40:05.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 group1 2024-09-13T08:40:05.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.269+0000 7faf05000640 1 -- 172.21.15.44:0/2462301137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf0015c130 msgr2=0x7faf000ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:05.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.269+0000 7faf05000640 1 --2- 172.21.15.44:0/2462301137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf0015c130 0x7faf000ceb20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faeec009900 tx=0x7faeec01c7c0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.269+0000 7faf05000640 1 -- 172.21.15.44:0/2462301137 shutdown_connections 2024-09-13T08:40:05.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.269+0000 7faf05000640 1 --2- 172.21.15.44:0/2462301137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf0015c130 0x7faf000ceb20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.269+0000 7faf05000640 1 --2- 172.21.15.44:0/2462301137 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf0015bd50 0x7faf000ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.269+0000 7faf05000640 1 -- 172.21.15.44:0/2462301137 >> 172.21.15.44:0/2462301137 conn(0x7faf000d1ea0 msgr2=0x7faf000d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:05.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.269+0000 7faf05000640 1 -- 172.21.15.44:0/2462301137 shutdown_connections 2024-09-13T08:40:05.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.269+0000 7faf05000640 1 -- 172.21.15.44:0/2462301137 wait complete. 2024-09-13T08:40:05.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.269+0000 7faf05000640 1 Processor -- start 2024-09-13T08:40:05.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faf05000640 1 -- start start 2024-09-13T08:40:05.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faf05000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf000cf4e0 0x7faf001570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:05.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faf05000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf0015bd50 0x7faf00156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:05.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faf05000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf0015c130 0x7faf000ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:05.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faf05000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf00168e50 con 0x7faf0015c130 2024-09-13T08:40:05.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faf05000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf00168cd0 con 0x7faf0015bd50 2024-09-13T08:40:05.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faf05000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7faf00168fd0 con 0x7faf000cf4e0 2024-09-13T08:40:05.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefe800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf0015c130 0x7faf000ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:05.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefde00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf000cf4e0 0x7faf001570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefe800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf0015c130 0x7faf000ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43400/0 (socket says 172.21.15.44:43400) 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefe800640 1 -- 172.21.15.44:0/3384205746 learned_addr learned my addr 172.21.15.44:0/3384205746 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefde00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf000cf4e0 0x7faf001570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:50106/0 (socket says 172.21.15.44:50106) 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefd400640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf0015bd50 0x7faf00156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefe800640 1 -- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf000cf4e0 msgr2=0x7faf001570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefe800640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf000cf4e0 0x7faf001570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefe800640 1 -- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf0015bd50 msgr2=0x7faf00156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefe800640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf0015bd50 0x7faf00156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefe800640 1 -- 172.21.15.44:0/3384205746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf000b6ba0 con 0x7faf0015c130 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faefe800640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf0015c130 0x7faf000ca730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faef800c7f0 tx=0x7faef800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faef6000640 1 -- 172.21.15.44:0/3384205746 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8003b80 con 0x7faf0015c130 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faef6000640 1 -- 172.21.15.44:0/3384205746 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef8003d20 con 0x7faf0015c130 2024-09-13T08:40:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faf05000640 1 -- 172.21.15.44:0/3384205746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf000b6e80 con 0x7faf0015c130 2024-09-13T08:40:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faef6000640 1 -- 172.21.15.44:0/3384205746 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef800aab0 con 0x7faf0015c130 2024-09-13T08:40:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faf05000640 1 -- 172.21.15.44:0/3384205746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf00156e70 con 0x7faf0015c130 2024-09-13T08:40:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.273+0000 7faf05000640 1 -- 172.21.15.44:0/3384205746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf000c5dd0 con 0x7faf0015c130 2024-09-13T08:40:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.277+0000 7faef6000640 1 -- 172.21.15.44:0/3384205746 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 7) ==== 97080+0+0 (secure 0 0 0) 0x7faef800ac50 con 0x7faf0015c130 2024-09-13T08:40:05.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.277+0000 7faef6000640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee0072ec0 0x7faee0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:05.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.277+0000 7faef6000640 1 -- 172.21.15.44:0/3384205746 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7faef8093b90 con 0x7faf0015c130 2024-09-13T08:40:05.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.277+0000 7faefde00640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee0072ec0 0x7faee0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:05.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.277+0000 7faefde00640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee0072ec0 0x7faee0075380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faef0004630 tx=0x7faef000b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:05.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.281+0000 7faef6000640 1 -- 172.21.15.44:0/3384205746 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7faef805e700 con 0x7faf0015c130 2024-09-13T08:40:05.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.485+0000 7faf05000640 1 -- 172.21.15.44:0/3384205746 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7faf000c9fc0 con 0x7faee0072ec0 2024-09-13T08:40:05.493 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:05.489+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:05.493 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:05.489+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:05.493 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:05.489+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:05.493 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:05.489+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:05.494 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:05.489+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:05.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faef6000640 1 -- 172.21.15.44:0/3384205746 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf000c9fc0 con 0x7faee0072ec0 2024-09-13T08:40:05.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 -- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee0072ec0 msgr2=0x7faee0075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:05.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee0072ec0 0x7faee0075380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faef0004630 tx=0x7faef000b390 comp rx=0 tx=0).stop 2024-09-13T08:40:05.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 -- 172.21.15.44:0/3384205746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf0015c130 msgr2=0x7faf000ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:05.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf0015c130 0x7faf000ca730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faef800c7f0 tx=0x7faef800ccb0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 -- 172.21.15.44:0/3384205746 shutdown_connections 2024-09-13T08:40:05.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee0072ec0 0x7faee0075380 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf0015c130 0x7faf000ca730 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf0015bd50 0x7faf00156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 --2- 172.21.15.44:0/3384205746 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf000cf4e0 0x7faf001570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 -- 172.21.15.44:0/3384205746 >> 172.21.15.44:0/3384205746 conn(0x7faf000d1ea0 msgr2=0x7faf000d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:05.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 -- 172.21.15.44:0/3384205746 shutdown_connections 2024-09-13T08:40:05.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.513+0000 7faf05000640 1 -- 172.21.15.44:0/3384205746 wait complete. 2024-09-13T08:40:05.535 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-13T08:40:05.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.681+0000 7f8e5ee00640 1 -- 172.21.15.44:0/1928326185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e580b0b60 msgr2=0x7f8e580ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:05.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.681+0000 7f8e5ee00640 1 --2- 172.21.15.44:0/1928326185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e580b0b60 0x7f8e580ccbd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009bc0 tx=0x7f8e5001f3b0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 -- 172.21.15.44:0/1928326185 shutdown_connections 2024-09-13T08:40:05.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 --2- 172.21.15.44:0/1928326185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e580b0b60 0x7f8e580ccbd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 --2- 172.21.15.44:0/1928326185 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e5815c1f0 0x7f8e580cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 -- 172.21.15.44:0/1928326185 >> 172.21.15.44:0/1928326185 conn(0x7f8e580d1da0 msgr2=0x7f8e580d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:05.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 -- 172.21.15.44:0/1928326185 shutdown_connections 2024-09-13T08:40:05.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 -- 172.21.15.44:0/1928326185 wait complete. 2024-09-13T08:40:05.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 Processor -- start 2024-09-13T08:40:05.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 -- start start 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e580b0b60 0x7f8e5815e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5815be10 0x7f8e5814f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5815c1f0 0x7f8e5814fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8e580cd320 con 0x7f8e5815c1f0 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8e580cf590 con 0x7f8e5815be10 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ee00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8e580cd4a0 con 0x7f8e580b0b60 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5815c1f0 0x7f8e5814fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5815c1f0 0x7f8e5814fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43414/0 (socket says 172.21.15.44:43414) 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ce00640 1 -- 172.21.15.44:0/3613413719 learned_addr learned my addr 172.21.15.44:0/3613413719 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e4fe00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5815be10 0x7f8e5814f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:05.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e57e00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e580b0b60 0x7f8e5815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:05.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ce00640 1 -- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e580b0b60 msgr2=0x7f8e5815e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:05.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ce00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e580b0b60 0x7f8e5815e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ce00640 1 -- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5815be10 msgr2=0x7f8e5814f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:05.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ce00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5815be10 0x7f8e5814f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ce00640 1 -- 172.21.15.44:0/3613413719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5815ebe0 con 0x7f8e5815c1f0 2024-09-13T08:40:05.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e5ce00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5815c1f0 0x7f8e5814fe70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8e5001cfe0 tx=0x7f8e50009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:05.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.685+0000 7f8e54c00640 1 -- 172.21.15.44:0/3613413719 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e500b4040 con 0x7f8e5815c1f0 2024-09-13T08:40:05.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.689+0000 7f8e54c00640 1 -- 172.21.15.44:0/3613413719 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e50004b40 con 0x7f8e5815c1f0 2024-09-13T08:40:05.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.689+0000 7f8e54c00640 1 -- 172.21.15.44:0/3613413719 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e500af8a0 con 0x7f8e5815c1f0 2024-09-13T08:40:05.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.689+0000 7f8e5ee00640 1 -- 172.21.15.44:0/3613413719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5815eec0 con 0x7f8e5815c1f0 2024-09-13T08:40:05.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.689+0000 7f8e5ee00640 1 -- 172.21.15.44:0/3613413719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5815f480 con 0x7f8e5815c1f0 2024-09-13T08:40:05.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.689+0000 7f8e5ee00640 1 -- 172.21.15.44:0/3613413719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e580ccbd0 con 0x7f8e5815c1f0 2024-09-13T08:40:05.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.689+0000 7f8e54c00640 1 -- 172.21.15.44:0/3613413719 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 7) ==== 97080+0+0 (secure 0 0 0) 0x7f8e500c1020 con 0x7f8e5815c1f0 2024-09-13T08:40:05.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.689+0000 7f8e54c00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e34072ec0 0x7f8e34075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:05.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.689+0000 7f8e54c00640 1 -- 172.21.15.44:0/3613413719 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f8e50131880 con 0x7f8e5815c1f0 2024-09-13T08:40:05.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.689+0000 7f8e57e00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e34072ec0 0x7f8e34075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:05.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.693+0000 7f8e57e00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e34072ec0 0x7f8e34075380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e48000f70 tx=0x7f8e48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:05.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.693+0000 7f8e54c00640 1 -- 172.21.15.44:0/3613413719 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8e50018f10 con 0x7f8e5815c1f0 2024-09-13T08:40:05.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.901+0000 7f8e5ee00640 1 -- 172.21.15.44:0/3613413719 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f8e580cf130 con 0x7f8e34072ec0 2024-09-13T08:40:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.905+0000 7f8e54c00640 1 -- 172.21.15.44:0/3613413719 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f8e580cf130 con 0x7f8e34072ec0 2024-09-13T08:40:05.908 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367 2024-09-13T08:40:05.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 -- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e34072ec0 msgr2=0x7f8e34075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:05.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e34072ec0 0x7f8e34075380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e48000f70 tx=0x7f8e48009040 comp rx=0 tx=0).stop 2024-09-13T08:40:05.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 -- 172.21.15.44:0/3613413719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5815c1f0 msgr2=0x7f8e5814fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:05.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5815c1f0 0x7f8e5814fe70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8e5001cfe0 tx=0x7f8e50009300 comp rx=0 tx=0).stop 2024-09-13T08:40:05.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 -- 172.21.15.44:0/3613413719 shutdown_connections 2024-09-13T08:40:05.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e34072ec0 0x7f8e34075380 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5815c1f0 0x7f8e5814fe70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5815be10 0x7f8e5814f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 --2- 172.21.15.44:0/3613413719 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e580b0b60 0x7f8e5815e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:05.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 -- 172.21.15.44:0/3613413719 >> 172.21.15.44:0/3613413719 conn(0x7f8e580d1da0 msgr2=0x7f8e58070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:05.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 -- 172.21.15.44:0/3613413719 shutdown_connections 2024-09-13T08:40:05.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:05.909+0000 7f8e5ee00640 1 -- 172.21.15.44:0/3613413719 wait complete. 2024-09-13T08:40:05.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:05.930 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367 2024-09-13T08:40:05.930 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:40:05.939 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367 2024-09-13T08:40:05.939 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="10262032384" 2024-09-13T08:40:05.939 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:05.940 INFO:tasks.cephfs.test_volumes:r_size = 10262032384 exp_size = 10737418240 2024-09-13T08:40:07.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:07.941 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367 2024-09-13T08:40:07.941 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:40:07.952 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367 2024-09-13T08:40:07.952 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="10262032384" 2024-09-13T08:40:07.952 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:07.953 INFO:tasks.cephfs.test_volumes:r_size = 10262032384 exp_size = 10737418240 2024-09-13T08:40:09.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:09.954 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367 2024-09-13T08:40:09.954 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:40:09.965 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/group1/sv1/fef4a3af-ce1e-4fa9-af70-2b1f585f5367 2024-09-13T08:40:09.965 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="10737418240" 2024-09-13T08:40:09.965 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:09.966 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-13T08:40:09.966 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 --group-name group1 2024-09-13T08:40:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.173+0000 7f800a800640 1 -- 172.21.15.44:0/3475693744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe8004810 msgr2=0x7f7fe8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.173+0000 7f800a800640 1 --2- 172.21.15.44:0/3475693744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe8004810 0x7f7fe8004c90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000c2e0 tx=0x7f7ff001f660 comp rx=0 tx=0).stop 2024-09-13T08:40:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.173+0000 7f800a800640 1 -- 172.21.15.44:0/3475693744 shutdown_connections 2024-09-13T08:40:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.173+0000 7f800a800640 1 --2- 172.21.15.44:0/3475693744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe8004810 0x7f7fe8004c90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.173+0000 7f800a800640 1 -- 172.21.15.44:0/3475693744 >> 172.21.15.44:0/3475693744 conn(0x7f800406d020 msgr2=0x7f800406d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.173+0000 7f800a800640 1 -- 172.21.15.44:0/3475693744 shutdown_connections 2024-09-13T08:40:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.173+0000 7f800a800640 1 -- 172.21.15.44:0/3475693744 wait complete. 2024-09-13T08:40:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f800a800640 1 Processor -- start 2024-09-13T08:40:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f800a800640 1 -- start start 2024-09-13T08:40:10.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f800a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe8004810 0x7f80040b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:10.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f800a800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80040aed10 0x7f8004157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:10.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f800a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80040acd40 0x7f80040ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f800a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f80040d0580 con 0x7f7fe8004810 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f800a800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80040d0ac0 con 0x7f80040acd40 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f800a800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f80040d0700 con 0x7f80040aed10 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe8004810 0x7f80040b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe8004810 0x7f80040b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43436/0 (socket says 172.21.15.44:43436) 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003400640 1 -- 172.21.15.44:0/42016323 learned_addr learned my addr 172.21.15.44:0/42016323 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8002a00640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80040aed10 0x7f8004157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003e00640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80040acd40 0x7f80040ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003400640 1 -- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80040aed10 msgr2=0x7f8004157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003400640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80040aed10 0x7f8004157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003400640 1 -- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80040acd40 msgr2=0x7f80040ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003400640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80040acd40 0x7f80040ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003400640 1 -- 172.21.15.44:0/42016323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80040ca210 con 0x7f7fe8004810 2024-09-13T08:40:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003e00640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80040acd40 0x7f80040ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f8003400640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe8004810 0x7f80040b1d10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4003eb0 tx=0x7f7ff4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f7ffb400640 1 -- 172.21.15.44:0/42016323 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4015070 con 0x7f7fe8004810 2024-09-13T08:40:10.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f7ffb400640 1 -- 172.21.15.44:0/42016323 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ff4011840 con 0x7f7fe8004810 2024-09-13T08:40:10.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f7ffb400640 1 -- 172.21.15.44:0/42016323 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4011c50 con 0x7f7fe8004810 2024-09-13T08:40:10.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f800a800640 1 -- 172.21.15.44:0/42016323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80040ca4f0 con 0x7f7fe8004810 2024-09-13T08:40:10.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.177+0000 7f800a800640 1 -- 172.21.15.44:0/42016323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8004157c40 con 0x7f7fe8004810 2024-09-13T08:40:10.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.181+0000 7f800a800640 1 -- 172.21.15.44:0/42016323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80040ad6e0 con 0x7f7fe8004810 2024-09-13T08:40:10.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.181+0000 7f7ffb400640 1 -- 172.21.15.44:0/42016323 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 97131+0+0 (secure 0 0 0) 0x7f7ff4018070 con 0x7f7fe8004810 2024-09-13T08:40:10.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.181+0000 7f7ffb400640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fe4072f50 0x7f7fe4075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:10.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.181+0000 7f7ffb400640 1 -- 172.21.15.44:0/42016323 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f7ff4094c20 con 0x7f7fe8004810 2024-09-13T08:40:10.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.181+0000 7f8002a00640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fe4072f50 0x7f7fe4075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:10.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.185+0000 7f8002a00640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fe4072f50 0x7f7fe4075410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00b0000 tx=0x7f7ff001c4a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:10.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.185+0000 7f7ffb400640 1 -- 172.21.15.44:0/42016323 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7ff405f5a0 con 0x7f7fe8004810 2024-09-13T08:40:10.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.397+0000 7f800a800640 1 -- 172.21.15.44:0/42016323 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f800415bc60 con 0x7f7fe4072f50 2024-09-13T08:40:10.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.477+0000 7f7ffb400640 1 -- 172.21.15.44:0/42016323 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f800415bc60 con 0x7f7fe4072f50 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 -- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fe4072f50 msgr2=0x7f7fe4075410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fe4072f50 0x7f7fe4075410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00b0000 tx=0x7f7ff001c4a0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 -- 172.21.15.44:0/42016323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe8004810 msgr2=0x7f80040b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe8004810 0x7f80040b1d10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4003eb0 tx=0x7f7ff4001640 comp rx=0 tx=0).stop 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 -- 172.21.15.44:0/42016323 shutdown_connections 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fe4072f50 0x7f7fe4075410 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80040acd40 0x7f80040ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80040aed10 0x7f8004157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 --2- 172.21.15.44:0/42016323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe8004810 0x7f80040b1d10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 -- 172.21.15.44:0/42016323 >> 172.21.15.44:0/42016323 conn(0x7f800406d020 msgr2=0x7f800415d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:10.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 -- 172.21.15.44:0/42016323 shutdown_connections 2024-09-13T08:40:10.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.481+0000 7f800a800640 1 -- 172.21.15.44:0/42016323 wait complete. 2024-09-13T08:40:10.484 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:10.481+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:10.484 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:10.481+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:10.484 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:10.481+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:10.484 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:10.481+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:10.484 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:10.481+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:10.503 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:10.497+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:10.503 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:10.497+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:10.503 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:10.497+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:10.503 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:10.497+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:10.503 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:10.497+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:10.505 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:40:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.653+0000 7f811f000640 1 -- 172.21.15.44:0/1413122717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81180cfe80 msgr2=0x7f81180d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.653+0000 7f811f000640 1 --2- 172.21.15.44:0/1413122717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81180cfe80 0x7f81180d0260 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8108009840 tx=0x7f810801c790 comp rx=0 tx=0).stop 2024-09-13T08:40:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.653+0000 7f811f000640 1 -- 172.21.15.44:0/1413122717 shutdown_connections 2024-09-13T08:40:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.653+0000 7f811f000640 1 --2- 172.21.15.44:0/1413122717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81180cfe80 0x7f81180d0260 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.653+0000 7f811f000640 1 -- 172.21.15.44:0/1413122717 >> 172.21.15.44:0/1413122717 conn(0x7f81180d1e40 msgr2=0x7f81180d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.653+0000 7f811f000640 1 -- 172.21.15.44:0/1413122717 shutdown_connections 2024-09-13T08:40:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.653+0000 7f811f000640 1 -- 172.21.15.44:0/1413122717 wait complete. 2024-09-13T08:40:10.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.653+0000 7f811f000640 1 Processor -- start 2024-09-13T08:40:10.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.653+0000 7f811f000640 1 -- start start 2024-09-13T08:40:10.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811f000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81180b2760 0x7f81180c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:10.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811f000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81180b2d00 0x7f81180b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:10.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811f000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81180cfe80 0x7f81180c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:10.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811f000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f811816a340 con 0x7f81180cfe80 2024-09-13T08:40:10.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811f000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f811816a1c0 con 0x7f81180b2d00 2024-09-13T08:40:10.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811f000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f811816a4c0 con 0x7f81180b2760 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81180cfe80 0x7f81180c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81180cfe80 0x7f81180c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43454/0 (socket says 172.21.15.44:43454) 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811d000640 1 -- 172.21.15.44:0/647318677 learned_addr learned my addr 172.21.15.44:0/647318677 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f8117400640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81180b2d00 0x7f81180b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f8117e00640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81180b2760 0x7f81180c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811d000640 1 -- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81180b2760 msgr2=0x7f81180c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811d000640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81180b2760 0x7f81180c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811d000640 1 -- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81180b2d00 msgr2=0x7f81180b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811d000640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81180b2d00 0x7f81180b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811d000640 1 -- 172.21.15.44:0/647318677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811815e9d0 con 0x7f81180cfe80 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f8117400640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81180b2d00 0x7f81180b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:10.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811d000640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81180cfe80 0x7f81180c7280 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f811000c700 tx=0x7f811000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f810fe00640 1 -- 172.21.15.44:0/647318677 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110003db0 con 0x7f81180cfe80 2024-09-13T08:40:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f810fe00640 1 -- 172.21.15.44:0/647318677 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f811000ce10 con 0x7f81180cfe80 2024-09-13T08:40:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f810fe00640 1 -- 172.21.15.44:0/647318677 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81100052d0 con 0x7f81180cfe80 2024-09-13T08:40:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811f000640 1 -- 172.21.15.44:0/647318677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81180aecc0 con 0x7f81180cfe80 2024-09-13T08:40:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811f000640 1 -- 172.21.15.44:0/647318677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81180c6700 con 0x7f81180cfe80 2024-09-13T08:40:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.657+0000 7f811f000640 1 -- 172.21.15.44:0/647318677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81180d0260 con 0x7f81180cfe80 2024-09-13T08:40:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.661+0000 7f810fe00640 1 -- 172.21.15.44:0/647318677 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 97131+0+0 (secure 0 0 0) 0x7f8110005470 con 0x7f81180cfe80 2024-09-13T08:40:10.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.661+0000 7f810fe00640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80f8072f50 0x7f80f8075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:10.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.661+0000 7f810fe00640 1 -- 172.21.15.44:0/647318677 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f8110093ed0 con 0x7f81180cfe80 2024-09-13T08:40:10.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.661+0000 7f8117e00640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80f8072f50 0x7f80f8075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:10.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.661+0000 7f8117e00640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80f8072f50 0x7f80f8075410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f81080af000 tx=0x7f810801afd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.665+0000 7f810fe00640 1 -- 172.21.15.44:0/647318677 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f81180d0c80 con 0x7f81180cfe80 2024-09-13T08:40:10.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.769+0000 7f810fe00640 1 -- 172.21.15.44:0/647318677 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 9) ==== 97233+0+0 (secure 0 0 0) 0x7f811005aaa0 con 0x7f81180cfe80 2024-09-13T08:40:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.905+0000 7f811f000640 1 -- 172.21.15.44:0/647318677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f811815fa50 con 0x7f81180cfe80 2024-09-13T08:40:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.905+0000 7f810fe00640 1 -- 172.21.15.44:0/647318677 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1912 (secure 0 0 0) 0x7f811005ef30 con 0x7f81180cfe80 2024-09-13T08:40:10.909 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:10.909 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:10.910 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 181, 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 35, 2024-09-13T08:40:10.911 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:10.912 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 2585, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 10738565698, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 21735788544, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 751358324736, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 85, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 118458060, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 0, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 29 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:40:10.913 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 18, 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:38:28:234730+0000", 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 2, 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 2, 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 4866 2024-09-13T08:40:10.914 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:10.915 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:40:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 7, 2024-09-13T08:40:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:39:26.537326+0000", 2024-09-13T08:40:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:40:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:10.916 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": {} 2024-09-13T08:40:10.916 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:40:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 -- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80f8072f50 msgr2=0x7f80f8075410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80f8072f50 0x7f80f8075410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f81080af000 tx=0x7f810801afd0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 -- 172.21.15.44:0/647318677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81180cfe80 msgr2=0x7f81180c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81180cfe80 0x7f81180c7280 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f811000c700 tx=0x7f811000cbc0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 -- 172.21.15.44:0/647318677 shutdown_connections 2024-09-13T08:40:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80f8072f50 0x7f80f8075410 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81180cfe80 0x7f81180c7280 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81180b2d00 0x7f81180b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 --2- 172.21.15.44:0/647318677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81180b2760 0x7f81180c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 -- 172.21.15.44:0/647318677 >> 172.21.15.44:0/647318677 conn(0x7f81180d1e40 msgr2=0x7f811815ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 -- 172.21.15.44:0/647318677 shutdown_connections 2024-09-13T08:40:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:10.909+0000 7f811f000640 1 -- 172.21.15.44:0/647318677 wait complete. 2024-09-13T08:40:11.490 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:11.485+0000 7f8ab7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:11.490 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:11.485+0000 7f8ab7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:11.490 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:11.485+0000 7f8ab7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:11.490 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:11.485+0000 7f8ab7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:11.491 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:11.485+0000 7f8ab7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:11.932 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:40:12.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.161+0000 7f012de00640 1 -- 172.21.15.44:0/213781178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c004810 msgr2=0x7f010c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:12.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.161+0000 7f012de00640 1 --2- 172.21.15.44:0/213781178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c004810 0x7f010c004c90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0114006fd0 tx=0x7f011401f770 comp rx=0 tx=0).stop 2024-09-13T08:40:12.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.161+0000 7f0127400640 1 -- 172.21.15.44:0/213781178 reap_dead start 2024-09-13T08:40:12.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.161+0000 7f012de00640 1 -- 172.21.15.44:0/213781178 shutdown_connections 2024-09-13T08:40:12.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.161+0000 7f012de00640 1 -- 172.21.15.44:0/213781178 >> 172.21.15.44:0/213781178 conn(0x7f01280d0cd0 msgr2=0x7f01280d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:12.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.161+0000 7f012de00640 1 -- 172.21.15.44:0/213781178 shutdown_connections 2024-09-13T08:40:12.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.161+0000 7f012de00640 1 -- 172.21.15.44:0/213781178 wait complete. 2024-09-13T08:40:12.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f012de00640 1 Processor -- start 2024-09-13T08:40:12.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f012de00640 1 -- start start 2024-09-13T08:40:12.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f012de00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f012814d170 0x7f01280c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:12.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f012de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01280c8620 0x7f0128159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:12.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f012de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f012814d550 0x7f0128158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:12.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f012de00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01280cc820 con 0x7f01280c8620 2024-09-13T08:40:12.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f012de00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01280cc6a0 con 0x7f012814d550 2024-09-13T08:40:12.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f012de00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f01280ccbc0 con 0x7f012814d170 2024-09-13T08:40:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f0126000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01280c8620 0x7f0128159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f0126000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01280c8620 0x7f0128159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44582/0 (socket says 172.21.15.44:44582) 2024-09-13T08:40:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.165+0000 7f0126000640 1 -- 172.21.15.44:0/126818705 learned_addr learned my addr 172.21.15.44:0/126818705 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f0126000640 1 -- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f012814d170 msgr2=0x7f01280c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f0126000640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f012814d170 0x7f01280c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f0126000640 1 -- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f012814d550 msgr2=0x7f0128158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f0126000640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f012814d550 0x7f0128158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f0126000640 1 -- 172.21.15.44:0/126818705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012815aa10 con 0x7f01280c8620 2024-09-13T08:40:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f0126000640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01280c8620 0x7f0128159fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0114020010 tx=0x7f0114006dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:12.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f011f400640 1 -- 172.21.15.44:0/126818705 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01140bd070 con 0x7f01280c8620 2024-09-13T08:40:12.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f011f400640 1 -- 172.21.15.44:0/126818705 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01140036e0 con 0x7f01280c8620 2024-09-13T08:40:12.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f012de00640 1 -- 172.21.15.44:0/126818705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012815a730 con 0x7f01280c8620 2024-09-13T08:40:12.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f011f400640 1 -- 172.21.15.44:0/126818705 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01140b2040 con 0x7f01280c8620 2024-09-13T08:40:12.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f012de00640 1 -- 172.21.15.44:0/126818705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01280c4b00 con 0x7f01280c8620 2024-09-13T08:40:12.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f012de00640 1 -- 172.21.15.44:0/126818705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01280cd090 con 0x7f01280c8620 2024-09-13T08:40:12.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.169+0000 7f011f400640 1 -- 172.21.15.44:0/126818705 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 9) ==== 97233+0+0 (secure 0 0 0) 0x7f01140c0070 con 0x7f01280c8620 2024-09-13T08:40:12.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.173+0000 7f011f400640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0108073070 0x7f0108075530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:12.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.173+0000 7f011f400640 1 -- 172.21.15.44:0/126818705 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f0114131420 con 0x7f01280c8620 2024-09-13T08:40:12.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.173+0000 7f0126a00640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0108073070 0x7f0108075530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:12.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.177+0000 7f0126a00640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0108073070 0x7f0108075530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f01280c6a90 tx=0x7f011800d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.177+0000 7f011f400640 1 -- 172.21.15.44:0/126818705 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f01140194d0 con 0x7f01280c8620 2024-09-13T08:40:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.521+0000 7f012de00640 1 -- 172.21.15.44:0/126818705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f01280b6ba0 con 0x7f01280c8620 2024-09-13T08:40:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.521+0000 7f011f400640 1 -- 172.21.15.44:0/126818705 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1912 (secure 0 0 0) 0x7f01140fbca0 con 0x7f01280c8620 2024-09-13T08:40:12.525 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:40:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 183, 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 35, 2024-09-13T08:40:12.527 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:40:12.528 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 2585, 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 10738565698, 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 21735788544, 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 751358324736, 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 85, 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 118458060, 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 0, 2024-09-13T08:40:12.529 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 29 2024-09-13T08:40:12.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:12.530 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:40:12.530 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 18, 2024-09-13T08:40:12.530 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:38:28:234730+0000", 2024-09-13T08:40:12.530 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 2, 2024-09-13T08:40:12.530 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:40:12.530 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:40:12.530 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:40:12.530 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 2, 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 4866 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:40:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 7, 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:39:26.537326+0000", 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": {} 2024-09-13T08:40:12.532 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:40:12.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 -- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0108073070 msgr2=0x7f0108075530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0108073070 0x7f0108075530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f01280c6a90 tx=0x7f011800d040 comp rx=0 tx=0).stop 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 -- 172.21.15.44:0/126818705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01280c8620 msgr2=0x7f0128159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01280c8620 0x7f0128159fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0114020010 tx=0x7f0114006dd0 comp rx=0 tx=0).stop 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 -- 172.21.15.44:0/126818705 shutdown_connections 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0108073070 0x7f0108075530 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f012814d550 0x7f0128158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01280c8620 0x7f0128159fe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 --2- 172.21.15.44:0/126818705 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f012814d170 0x7f01280c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 -- 172.21.15.44:0/126818705 >> 172.21.15.44:0/126818705 conn(0x7f01280d0cd0 msgr2=0x7f01280d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 -- 172.21.15.44:0/126818705 shutdown_connections 2024-09-13T08:40:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:12.525+0000 7f012de00640 1 -- 172.21.15.44:0/126818705 wait complete. 2024-09-13T08:40:13.563 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:40:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.777+0000 7f3a8ea00640 1 -- 172.21.15.44:0/1453673600 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3a6c003a60 msgr2=0x7f3a6c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.777+0000 7f3a8ea00640 1 --2- 172.21.15.44:0/1453673600 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3a6c003a60 0x7f3a6c003e80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a78006c40 tx=0x7f3a780218b0 comp rx=0 tx=0).stop 2024-09-13T08:40:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.777+0000 7f3a8ea00640 1 -- 172.21.15.44:0/1453673600 shutdown_connections 2024-09-13T08:40:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.777+0000 7f3a8ea00640 1 --2- 172.21.15.44:0/1453673600 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3a6c003a60 0x7f3a6c003e80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.777+0000 7f3a8ea00640 1 -- 172.21.15.44:0/1453673600 >> 172.21.15.44:0/1453673600 conn(0x7f3a8806cdf0 msgr2=0x7f3a8806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.777+0000 7f3a8ea00640 1 -- 172.21.15.44:0/1453673600 shutdown_connections 2024-09-13T08:40:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.777+0000 7f3a8ea00640 1 -- 172.21.15.44:0/1453673600 wait complete. 2024-09-13T08:40:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.777+0000 7f3a8ea00640 1 Processor -- start 2024-09-13T08:40:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.777+0000 7f3a8ea00640 1 -- start start 2024-09-13T08:40:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a8ea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3a6c003a60 0x7f3a880ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a8ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a880af1c0 0x7f3a881f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a8ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a881f7d70 0x7f3a881470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a8ea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8815ba10 con 0x7f3a880af1c0 2024-09-13T08:40:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a8ea00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3a880705c0 con 0x7f3a881f7d70 2024-09-13T08:40:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a8ea00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8815bb90 con 0x7f3a6c003a60 2024-09-13T08:40:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a87400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a880af1c0 0x7f3a881f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a87400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a880af1c0 0x7f3a881f4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44620/0 (socket says 172.21.15.44:44620) 2024-09-13T08:40:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a87400640 1 -- 172.21.15.44:0/822922631 learned_addr learned my addr 172.21.15.44:0/822922631 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a8ca00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a881f7d70 0x7f3a881470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a87e00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3a6c003a60 0x7f3a880ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a87400640 1 -- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3a6c003a60 msgr2=0x7f3a880ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a87400640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3a6c003a60 0x7f3a880ada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a87400640 1 -- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a881f7d70 msgr2=0x7f3a881470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a87400640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a881f7d70 0x7f3a881470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a87400640 1 -- 172.21.15.44:0/822922631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a881f54e0 con 0x7f3a880af1c0 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a87400640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a880af1c0 0x7f3a881f4fa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400c450 tx=0x7f3a7400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a7fe00640 1 -- 172.21.15.44:0/822922631 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74016020 con 0x7f3a880af1c0 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a8ea00640 1 -- 172.21.15.44:0/822922631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a880ae120 con 0x7f3a880af1c0 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a7fe00640 1 -- 172.21.15.44:0/822922631 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a74012920 con 0x7f3a880af1c0 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a7fe00640 1 -- 172.21.15.44:0/822922631 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74012d70 con 0x7f3a880af1c0 2024-09-13T08:40:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a8ea00640 1 -- 172.21.15.44:0/822922631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a880ae6e0 con 0x7f3a880af1c0 2024-09-13T08:40:13.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.781+0000 7f3a8ea00640 1 -- 172.21.15.44:0/822922631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8815e720 con 0x7f3a880af1c0 2024-09-13T08:40:13.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.785+0000 7f3a7fe00640 1 -- 172.21.15.44:0/822922631 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 97284+0+0 (secure 0 0 0) 0x7f3a74019020 con 0x7f3a880af1c0 2024-09-13T08:40:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.785+0000 7f3a7fe00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3a68073100 0x7f3a680755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.785+0000 7f3a7fe00640 1 -- 172.21.15.44:0/822922631 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f3a74093d00 con 0x7f3a880af1c0 2024-09-13T08:40:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.785+0000 7f3a87e00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3a68073100 0x7f3a680755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.785+0000 7f3a87e00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3a68073100 0x7f3a680755c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3a780b1000 tx=0x7f3a7801e6f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:13.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:13.793+0000 7f3a7fe00640 1 -- 172.21.15.44:0/822922631 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3a74020070 con 0x7f3a880af1c0 2024-09-13T08:40:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.169+0000 7f3a8ea00640 1 -- 172.21.15.44:0/822922631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3a8815d760 con 0x7f3a880af1c0 2024-09-13T08:40:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.169+0000 7f3a7fe00640 1 -- 172.21.15.44:0/822922631 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2137 (secure 0 0 0) 0x7f3a881f54e0 con 0x7f3a880af1c0 2024-09-13T08:40:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:14.174 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:40:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:40:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:40:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:40:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:40:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:40:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:14.174 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 184, 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:40:14.175 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 35, 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:40:14.176 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 2585, 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 10738565698, 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 21735788544, 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 751358324736, 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 85, 2024-09-13T08:40:14.177 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 89456681, 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 0, 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 22 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 18, 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:38:28:234730+0000", 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 2, 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:14.178 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 2, 2024-09-13T08:40:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:40:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:40:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:40:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 4866 2024-09-13T08:40:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:14.179 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 7, 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:39:26.537326+0000", 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": { 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-ongoing-clones": { 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "1 ongoing clones - average progress is 0.0% (0s)\n [............................] ", 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0, 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:14.181 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:40:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 -- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3a68073100 msgr2=0x7f3a680755c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3a68073100 0x7f3a680755c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3a780b1000 tx=0x7f3a7801e6f0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 -- 172.21.15.44:0/822922631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a880af1c0 msgr2=0x7f3a881f4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a880af1c0 0x7f3a881f4fa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400c450 tx=0x7f3a7400c910 comp rx=0 tx=0).stop 2024-09-13T08:40:14.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 -- 172.21.15.44:0/822922631 shutdown_connections 2024-09-13T08:40:14.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3a68073100 0x7f3a680755c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a881f7d70 0x7f3a881470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a880af1c0 0x7f3a881f4fa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 --2- 172.21.15.44:0/822922631 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3a6c003a60 0x7f3a880ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 -- 172.21.15.44:0/822922631 >> 172.21.15.44:0/822922631 conn(0x7f3a8806cdf0 msgr2=0x7f3a88159c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:14.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 -- 172.21.15.44:0/822922631 shutdown_connections 2024-09-13T08:40:14.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.177+0000 7f3a8ea00640 1 -- 172.21.15.44:0/822922631 wait complete. 2024-09-13T08:40:14.207 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-13T08:40:14.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 -- 172.21.15.44:0/3209925274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef9015ad60 msgr2=0x7fef90071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 --2- 172.21.15.44:0/3209925274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef9015ad60 0x7fef90071320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef80009900 tx=0x7fef8001c8e0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 -- 172.21.15.44:0/3209925274 shutdown_connections 2024-09-13T08:40:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 --2- 172.21.15.44:0/3209925274 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fef9015e830 0x7fef9015ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 --2- 172.21.15.44:0/3209925274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef9015ad60 0x7fef90071320 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 -- 172.21.15.44:0/3209925274 >> 172.21.15.44:0/3209925274 conn(0x7fef900b2860 msgr2=0x7fef900b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 -- 172.21.15.44:0/3209925274 shutdown_connections 2024-09-13T08:40:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 -- 172.21.15.44:0/3209925274 wait complete. 2024-09-13T08:40:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 Processor -- start 2024-09-13T08:40:14.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 -- start start 2024-09-13T08:40:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef90073820 0x7fef90165570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9015ad60 0x7fef9015b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fef9015e830 0x7fef900b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fef90168730 con 0x7fef9015ad60 2024-09-13T08:40:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef901685b0 con 0x7fef90073820 2024-09-13T08:40:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef95e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fef901688b0 con 0x7fef9015e830 2024-09-13T08:40:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef8e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9015ad60 0x7fef9015b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef8e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9015ad60 0x7fef9015b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44632/0 (socket says 172.21.15.44:44632) 2024-09-13T08:40:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef8e000640 1 -- 172.21.15.44:0/3422043247 learned_addr learned my addr 172.21.15.44:0/3422043247 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.457+0000 7fef8ea00640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef90073820 0x7fef90165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:14.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef8e000640 1 -- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fef9015e830 msgr2=0x7fef900b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:14.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef8e000640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fef9015e830 0x7fef900b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef8e000640 1 -- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef90073820 msgr2=0x7fef90165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:14.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef8e000640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef90073820 0x7fef90165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef8e000640 1 -- 172.21.15.44:0/3422043247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef900ccfd0 con 0x7fef9015ad60 2024-09-13T08:40:14.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef8e000640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9015ad60 0x7fef9015b8e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00b3f0 tx=0x7fef7c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef87400640 1 -- 172.21.15.44:0/3422043247 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c015020 con 0x7fef9015ad60 2024-09-13T08:40:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef87400640 1 -- 172.21.15.44:0/3422043247 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef7c011940 con 0x7fef9015ad60 2024-09-13T08:40:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef95e00640 1 -- 172.21.15.44:0/3422043247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef900b0a20 con 0x7fef9015ad60 2024-09-13T08:40:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef87400640 1 -- 172.21.15.44:0/3422043247 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c011d90 con 0x7fef9015ad60 2024-09-13T08:40:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef95e00640 1 -- 172.21.15.44:0/3422043247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef900b12c0 con 0x7fef9015ad60 2024-09-13T08:40:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef95e00640 1 -- 172.21.15.44:0/3422043247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef90071320 con 0x7fef9015ad60 2024-09-13T08:40:14.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.461+0000 7fef87400640 1 -- 172.21.15.44:0/3422043247 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 97284+0+0 (secure 0 0 0) 0x7fef7c018020 con 0x7fef9015ad60 2024-09-13T08:40:14.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.465+0000 7fef87400640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef70073130 0x7fef700755f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:14.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.465+0000 7fef87400640 1 -- 172.21.15.44:0/3422043247 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fef7c094290 con 0x7fef9015ad60 2024-09-13T08:40:14.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.469+0000 7fef87400640 1 -- 172.21.15.44:0/3422043247 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fef7c01f020 con 0x7fef9015ad60 2024-09-13T08:40:14.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.469+0000 7fef8ea00640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef70073130 0x7fef700755f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:14.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.473+0000 7fef8ea00640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef70073130 0x7fef700755f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef800af000 tx=0x7fef8001b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:14.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.797+0000 7fef95e00640 1 -- 172.21.15.44:0/3422043247 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fef900cd200 con 0x7fef70073130 2024-09-13T08:40:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.977+0000 7fef87400640 1 -- 172.21.15.44:0/3422043247 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fef900cd200 con 0x7fef70073130 2024-09-13T08:40:14.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 -- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef70073130 msgr2=0x7fef700755f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:14.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef70073130 0x7fef700755f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef800af000 tx=0x7fef8001b120 comp rx=0 tx=0).stop 2024-09-13T08:40:14.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 -- 172.21.15.44:0/3422043247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9015ad60 msgr2=0x7fef9015b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:14.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9015ad60 0x7fef9015b8e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00b3f0 tx=0x7fef7c00b8b0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 -- 172.21.15.44:0/3422043247 shutdown_connections 2024-09-13T08:40:14.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fef70073130 0x7fef700755f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fef9015e830 0x7fef900b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef9015ad60 0x7fef9015b8e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 --2- 172.21.15.44:0/3422043247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef90073820 0x7fef90165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:14.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 -- 172.21.15.44:0/3422043247 >> 172.21.15.44:0/3422043247 conn(0x7fef900b2860 msgr2=0x7fef900b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:14.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 -- 172.21.15.44:0/3422043247 shutdown_connections 2024-09-13T08:40:14.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:14.981+0000 7fef95e00640 1 -- 172.21.15.44:0/3422043247 wait complete. 2024-09-13T08:40:15.001 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:40:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.145+0000 7f573b200640 1 -- 172.21.15.44:0/1094115151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f573c15f000 msgr2=0x7f573c0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.145+0000 7f573b200640 1 --2- 172.21.15.44:0/1094115151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f573c15f000 0x7f573c0d01b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f572c009900 tx=0x7f572c01c850 comp rx=0 tx=0).stop 2024-09-13T08:40:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.145+0000 7f573b200640 1 -- 172.21.15.44:0/1094115151 shutdown_connections 2024-09-13T08:40:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.145+0000 7f573b200640 1 --2- 172.21.15.44:0/1094115151 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f573c0cefb0 0x7f573c0cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.145+0000 7f573b200640 1 --2- 172.21.15.44:0/1094115151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f573c15f000 0x7f573c0d01b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.145+0000 7f573b200640 1 -- 172.21.15.44:0/1094115151 >> 172.21.15.44:0/1094115151 conn(0x7f573c09aa10 msgr2=0x7f573c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573b200640 1 -- 172.21.15.44:0/1094115151 shutdown_connections 2024-09-13T08:40:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573b200640 1 -- 172.21.15.44:0/1094115151 wait complete. 2024-09-13T08:40:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573b200640 1 Processor -- start 2024-09-13T08:40:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573b200640 1 -- start start 2024-09-13T08:40:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f573c0ccaf0 0x7f573c159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573b200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f573c0cefb0 0x7f573c0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f573c15f000 0x7f573c0b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f573c169290 con 0x7f573c0ccaf0 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f573c169110 con 0x7f573c15f000 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573b200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f573c169410 con 0x7f573c0cefb0 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5739e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f573c0ccaf0 0x7f573c159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5739e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f573c0ccaf0 0x7f573c159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44650/0 (socket says 172.21.15.44:44650) 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5739e00640 1 -- 172.21.15.44:0/1361009023 learned_addr learned my addr 172.21.15.44:0/1361009023 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5733e00640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f573c0cefb0 0x7f573c0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573a800640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f573c15f000 0x7f573c0b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5739e00640 1 -- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f573c0cefb0 msgr2=0x7f573c0b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5739e00640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f573c0cefb0 0x7f573c0b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5739e00640 1 -- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f573c15f000 msgr2=0x7f573c0b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5739e00640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f573c15f000 0x7f573c0b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5739e00640 1 -- 172.21.15.44:0/1361009023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573c050f90 con 0x7f573c0ccaf0 2024-09-13T08:40:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f573a800640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f573c15f000 0x7f573c0b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5739e00640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f573c0ccaf0 0x7f573c159eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f572c009850 tx=0x7f572c0a5510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5732000640 1 -- 172.21.15.44:0/1361009023 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0b2040 con 0x7f573c0ccaf0 2024-09-13T08:40:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5732000640 1 -- 172.21.15.44:0/1361009023 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f572c0042c0 con 0x7f573c0ccaf0 2024-09-13T08:40:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.149+0000 7f5732000640 1 -- 172.21.15.44:0/1361009023 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c005020 con 0x7f573c0ccaf0 2024-09-13T08:40:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.153+0000 7f573b200640 1 -- 172.21.15.44:0/1361009023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573c1f8d50 con 0x7f573c0ccaf0 2024-09-13T08:40:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.153+0000 7f573b200640 1 -- 172.21.15.44:0/1361009023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573c1f8270 con 0x7f573c0ccaf0 2024-09-13T08:40:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.153+0000 7f573b200640 1 -- 172.21.15.44:0/1361009023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573c0cc5b0 con 0x7f573c0ccaf0 2024-09-13T08:40:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.153+0000 7f5732000640 1 -- 172.21.15.44:0/1361009023 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 97284+0+0 (secure 0 0 0) 0x7f572c0051e0 con 0x7f573c0ccaf0 2024-09-13T08:40:15.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.153+0000 7f5732000640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5718073100 0x7f57180755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.153+0000 7f5732000640 1 -- 172.21.15.44:0/1361009023 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f572c12ddb0 con 0x7f573c0ccaf0 2024-09-13T08:40:15.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.153+0000 7f5733e00640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5718073100 0x7f57180755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.157+0000 7f5733e00640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5718073100 0x7f57180755c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f573c15d0c0 tx=0x7f572400b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:15.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.157+0000 7f5732000640 1 -- 172.21.15.44:0/1361009023 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f572c132070 con 0x7f573c0ccaf0 2024-09-13T08:40:15.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.357+0000 7f573b200640 1 -- 172.21.15.44:0/1361009023 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f573c15e830 con 0x7f5718073100 2024-09-13T08:40:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.357+0000 7f5732000640 1 -- 172.21.15.44:0/1361009023 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f573c15e830 con 0x7f5718073100 2024-09-13T08:40:15.361 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:40:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:40:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:15.361 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:40:15.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 -- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5718073100 msgr2=0x7f57180755c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5718073100 0x7f57180755c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f573c15d0c0 tx=0x7f572400b390 comp rx=0 tx=0).stop 2024-09-13T08:40:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 -- 172.21.15.44:0/1361009023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f573c0ccaf0 msgr2=0x7f573c159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f573c0ccaf0 0x7f573c159eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f572c009850 tx=0x7f572c0a5510 comp rx=0 tx=0).stop 2024-09-13T08:40:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 -- 172.21.15.44:0/1361009023 shutdown_connections 2024-09-13T08:40:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5718073100 0x7f57180755c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f573c15f000 0x7f573c0b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f573c0cefb0 0x7f573c0b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 --2- 172.21.15.44:0/1361009023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f573c0ccaf0 0x7f573c159eb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 -- 172.21.15.44:0/1361009023 >> 172.21.15.44:0/1361009023 conn(0x7f573c09aa10 msgr2=0x7f573c15d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 -- 172.21.15.44:0/1361009023 shutdown_connections 2024-09-13T08:40:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.361+0000 7f573b200640 1 -- 172.21.15.44:0/1361009023 wait complete. 2024-09-13T08:40:15.383 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-13T08:40:15.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.529+0000 7f96ac000640 1 -- 172.21.15.44:0/596775124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f968c004810 msgr2=0x7f968c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.529+0000 7f96ac000640 1 --2- 172.21.15.44:0/596775124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f968c004810 0x7f968c004c90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9690009b60 tx=0x7f969001d270 comp rx=0 tx=0).stop 2024-09-13T08:40:15.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.529+0000 7f96ac000640 1 -- 172.21.15.44:0/596775124 shutdown_connections 2024-09-13T08:40:15.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.529+0000 7f96ac000640 1 --2- 172.21.15.44:0/596775124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f968c004810 0x7f968c004c90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.529+0000 7f96ac000640 1 -- 172.21.15.44:0/596775124 >> 172.21.15.44:0/596775124 conn(0x7f96a40d1e90 msgr2=0x7f96a40d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:15.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.529+0000 7f96ac000640 1 -- 172.21.15.44:0/596775124 shutdown_connections 2024-09-13T08:40:15.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.529+0000 7f96ac000640 1 -- 172.21.15.44:0/596775124 wait complete. 2024-09-13T08:40:15.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96ac000640 1 Processor -- start 2024-09-13T08:40:15.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96ac000640 1 -- start start 2024-09-13T08:40:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96ac000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c004810 0x7f96a4153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96ac000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96a415c780 0x7f96a41f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96ac000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a415dd20 0x7f96a415e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96ac000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96a40b1760 con 0x7f96a415dd20 2024-09-13T08:40:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96ac000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96a40b15e0 con 0x7f968c004810 2024-09-13T08:40:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96ac000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f96a40b18e0 con 0x7f96a415c780 2024-09-13T08:40:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96aa000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a415dd20 0x7f96a415e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96aa000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a415dd20 0x7f96a415e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44668/0 (socket says 172.21.15.44:44668) 2024-09-13T08:40:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96aa000640 1 -- 172.21.15.44:0/723251590 learned_addr learned my addr 172.21.15.44:0/723251590 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96a9600640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c004810 0x7f96a4153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96a8c00640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96a415c780 0x7f96a41f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96aa000640 1 -- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96a415c780 msgr2=0x7f96a41f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96aa000640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96a415c780 0x7f96a41f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96aa000640 1 -- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c004810 msgr2=0x7f96a4153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96aa000640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c004810 0x7f96a4153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96aa000640 1 -- 172.21.15.44:0/723251590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a41f32a0 con 0x7f96a415dd20 2024-09-13T08:40:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96aa000640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a415dd20 0x7f96a415e870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f969c0021d0 tx=0x7f969c002200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:15.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96a1600640 1 -- 172.21.15.44:0/723251590 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c011900 con 0x7f96a415dd20 2024-09-13T08:40:15.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96a1600640 1 -- 172.21.15.44:0/723251590 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f969c011ac0 con 0x7f96a415dd20 2024-09-13T08:40:15.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96ac000640 1 -- 172.21.15.44:0/723251590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a415cc80 con 0x7f96a415dd20 2024-09-13T08:40:15.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.533+0000 7f96a1600640 1 -- 172.21.15.44:0/723251590 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c00f430 con 0x7f96a415dd20 2024-09-13T08:40:15.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.537+0000 7f96ac000640 1 -- 172.21.15.44:0/723251590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a415d240 con 0x7f96a415dd20 2024-09-13T08:40:15.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.537+0000 7f96ac000640 1 -- 172.21.15.44:0/723251590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a40b1e40 con 0x7f96a415dd20 2024-09-13T08:40:15.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.537+0000 7f96a1600640 1 -- 172.21.15.44:0/723251590 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 97284+0+0 (secure 0 0 0) 0x7f969c00f5d0 con 0x7f96a415dd20 2024-09-13T08:40:15.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.537+0000 7f96a1600640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9684073130 0x7f96840755f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.537+0000 7f96a1600640 1 -- 172.21.15.44:0/723251590 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f969c094a70 con 0x7f96a415dd20 2024-09-13T08:40:15.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.537+0000 7f96a9600640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9684073130 0x7f96840755f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.537+0000 7f96a9600640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9684073130 0x7f96840755f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f96a4052940 tx=0x7f9698000bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:15.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.541+0000 7f96a1600640 1 -- 172.21.15.44:0/723251590 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f96a41f32a0 con 0x7f96a415dd20 2024-09-13T08:40:15.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.741+0000 7f96ac000640 1 -- 172.21.15.44:0/723251590 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f96a40b7780 con 0x7f9684073130 2024-09-13T08:40:15.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96a1600640 1 -- 172.21.15.44:0/723251590 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+44 (secure 0 0 0) 0x7f96a40b7780 con 0x7f9684073130 2024-09-13T08:40:15.747 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:15.747 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:40:15.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:15.747 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1clone1" 2024-09-13T08:40:15.747 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:15.747 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:40:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 -- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9684073130 msgr2=0x7f96840755f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9684073130 0x7f96840755f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f96a4052940 tx=0x7f9698000bd0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 -- 172.21.15.44:0/723251590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a415dd20 msgr2=0x7f96a415e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a415dd20 0x7f96a415e870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f969c0021d0 tx=0x7f969c002200 comp rx=0 tx=0).stop 2024-09-13T08:40:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 -- 172.21.15.44:0/723251590 shutdown_connections 2024-09-13T08:40:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9684073130 0x7f96840755f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96a415dd20 0x7f96a415e870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96a415c780 0x7f96a41f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 --2- 172.21.15.44:0/723251590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c004810 0x7f96a4153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 -- 172.21.15.44:0/723251590 >> 172.21.15.44:0/723251590 conn(0x7f96a40d1e90 msgr2=0x7f96a40b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 -- 172.21.15.44:0/723251590 shutdown_connections 2024-09-13T08:40:15.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.745+0000 7f96ac000640 1 -- 172.21.15.44:0/723251590 wait complete. 2024-09-13T08:40:15.767 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-13T08:40:15.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 -- 172.21.15.44:0/2835024168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e40af850 msgr2=0x7f04e40afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 --2- 172.21.15.44:0/2835024168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e40af850 0x7f04e40afce0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009c80 tx=0x7f04dc01cd70 comp rx=0 tx=0).stop 2024-09-13T08:40:15.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 -- 172.21.15.44:0/2835024168 shutdown_connections 2024-09-13T08:40:15.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 --2- 172.21.15.44:0/2835024168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e40af850 0x7f04e40afce0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 --2- 172.21.15.44:0/2835024168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04e40d28e0 0x7f04e40d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 -- 172.21.15.44:0/2835024168 >> 172.21.15.44:0/2835024168 conn(0x7f04e40d1dc0 msgr2=0x7f04e40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:15.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 -- 172.21.15.44:0/2835024168 shutdown_connections 2024-09-13T08:40:15.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 -- 172.21.15.44:0/2835024168 wait complete. 2024-09-13T08:40:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 Processor -- start 2024-09-13T08:40:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 -- start start 2024-09-13T08:40:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04e40af850 0x7f04e40b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e40d28e0 0x7f04e415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04e415f390 0x7f04e415c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4168ea0 con 0x7f04e40d28e0 2024-09-13T08:40:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4168d20 con 0x7f04e415f390 2024-09-13T08:40:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e9600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f04e4169020 con 0x7f04e40af850 2024-09-13T08:40:15.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e40d28e0 0x7f04e415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e40d28e0 0x7f04e415bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44684/0 (socket says 172.21.15.44:44684) 2024-09-13T08:40:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04dbe00640 1 -- 172.21.15.44:0/2408733424 learned_addr learned my addr 172.21.15.44:0/2408733424 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e3400640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04e415f390 0x7f04e415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e2a00640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04e40af850 0x7f04e40b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04dbe00640 1 -- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04e40af850 msgr2=0x7f04e40b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04dbe00640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04e40af850 0x7f04e40b7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04dbe00640 1 -- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04e415f390 msgr2=0x7f04e415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04dbe00640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04e415f390 0x7f04e415c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04dbe00640 1 -- 172.21.15.44:0/2408733424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e4146c10 con 0x7f04e40d28e0 2024-09-13T08:40:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.917+0000 7f04e2a00640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04e40af850 0x7f04e40b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04dbe00640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e40d28e0 0x7f04e415bb30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00c970 tx=0x7f04cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04daa00640 1 -- 172.21.15.44:0/2408733424 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc007c70 con 0x7f04e40d28e0 2024-09-13T08:40:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04daa00640 1 -- 172.21.15.44:0/2408733424 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04cc00f450 con 0x7f04e40d28e0 2024-09-13T08:40:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04daa00640 1 -- 172.21.15.44:0/2408733424 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00f8a0 con 0x7f04e40d28e0 2024-09-13T08:40:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04e9600640 1 -- 172.21.15.44:0/2408733424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e40cc900 con 0x7f04e40d28e0 2024-09-13T08:40:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04e9600640 1 -- 172.21.15.44:0/2408733424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e40cee50 con 0x7f04e40d28e0 2024-09-13T08:40:15.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04e9600640 1 -- 172.21.15.44:0/2408733424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e40cf140 con 0x7f04e40d28e0 2024-09-13T08:40:15.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04daa00640 1 -- 172.21.15.44:0/2408733424 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 97284+0+0 (secure 0 0 0) 0x7f04cc00fa40 con 0x7f04e40d28e0 2024-09-13T08:40:15.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04daa00640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f04c0073100 0x7f04c00755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:15.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04daa00640 1 -- 172.21.15.44:0/2408733424 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f04cc094c00 con 0x7f04e40d28e0 2024-09-13T08:40:15.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.921+0000 7f04e2a00640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f04c0073100 0x7f04c00755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:15.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.925+0000 7f04e2a00640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f04c0073100 0x7f04c00755c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f04d40098b0 tx=0x7f04d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:15.925+0000 7f04daa00640 1 -- 172.21.15.44:0/2408733424 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f04cc014030 con 0x7f04e40d28e0 2024-09-13T08:40:16.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:16.129+0000 7f04e9600640 1 -- 172.21.15.44:0/2408733424 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f04e4146ef0 con 0x7f04c0073100 2024-09-13T08:40:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.265+0000 7f04daa00640 1 -- 172.21.15.44:0/2408733424 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f04e4146ef0 con 0x7f04c0073100 2024-09-13T08:40:18.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 -- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f04c0073100 msgr2=0x7f04c00755c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:18.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f04c0073100 0x7f04c00755c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f04d40098b0 tx=0x7f04d4008040 comp rx=0 tx=0).stop 2024-09-13T08:40:18.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 -- 172.21.15.44:0/2408733424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e40d28e0 msgr2=0x7f04e415bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:18.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e40d28e0 0x7f04e415bb30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00c970 tx=0x7f04cc00ce30 comp rx=0 tx=0).stop 2024-09-13T08:40:18.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 -- 172.21.15.44:0/2408733424 shutdown_connections 2024-09-13T08:40:18.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f04c0073100 0x7f04c00755c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04e415f390 0x7f04e415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e40d28e0 0x7f04e415bb30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 --2- 172.21.15.44:0/2408733424 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f04e40af850 0x7f04e40b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 -- 172.21.15.44:0/2408733424 >> 172.21.15.44:0/2408733424 conn(0x7f04e40d1dc0 msgr2=0x7f04e40b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:18.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 -- 172.21.15.44:0/2408733424 shutdown_connections 2024-09-13T08:40:18.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.269+0000 7f04e9600640 1 -- 172.21.15.44:0/2408733424 wait complete. 2024-09-13T08:40:18.273 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:18.269+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:18.273 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:18.269+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:18.273 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:18.269+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:18.273 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:18.269+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:18.274 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:18.269+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:18.293 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:18.289+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:18.293 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:18.289+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:18.294 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:18.289+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:18.294 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:18.289+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:18.294 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:18.289+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:18.299 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-13T08:40:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:40:18.362 INFO:teuthology.orchestra.run.smithi044.stdout:2 2024-09-13T08:40:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:40:18.369 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:40:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.517+0000 7f231a600640 1 -- 172.21.15.44:0/684423844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f231415ff50 msgr2=0x7f231406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.517+0000 7f231a600640 1 --2- 172.21.15.44:0/684423844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f231415ff50 0x7f231406f090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009920 tx=0x7f22fc01c8c0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.517+0000 7f231a600640 1 -- 172.21.15.44:0/684423844 shutdown_connections 2024-09-13T08:40:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.517+0000 7f231a600640 1 --2- 172.21.15.44:0/684423844 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23140c6b00 0x7f231406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.517+0000 7f231a600640 1 --2- 172.21.15.44:0/684423844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f231415ff50 0x7f231406f090 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.517+0000 7f231a600640 1 -- 172.21.15.44:0/684423844 >> 172.21.15.44:0/684423844 conn(0x7f23140cce90 msgr2=0x7f23140cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.517+0000 7f231a600640 1 -- 172.21.15.44:0/684423844 shutdown_connections 2024-09-13T08:40:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.517+0000 7f231a600640 1 -- 172.21.15.44:0/684423844 wait complete. 2024-09-13T08:40:18.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.517+0000 7f231a600640 1 Processor -- start 2024-09-13T08:40:18.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.517+0000 7f231a600640 1 -- start start 2024-09-13T08:40:18.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f231a600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23140c6b00 0x7f23140b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:18.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f231a600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2314152b90 0x7f23140b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:18.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f231a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f231415ff50 0x7f23141410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:18.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f231a600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2314168bc0 con 0x7f231415ff50 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f231a600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2314073ff0 con 0x7f2314152b90 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f231a600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2314168d40 con 0x7f23140c6b00 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f2313e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f231415ff50 0x7f23141410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f2313e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f231415ff50 0x7f23141410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44698/0 (socket says 172.21.15.44:44698) 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f2313e00640 1 -- 172.21.15.44:0/731677693 learned_addr learned my addr 172.21.15.44:0/731677693 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f230be00640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2314152b90 0x7f23140b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f2313e00640 1 -- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23140c6b00 msgr2=0x7f23140b2b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f2313e00640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23140c6b00 0x7f23140b2b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f2313e00640 1 -- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2314152b90 msgr2=0x7f23140b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f2313e00640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2314152b90 0x7f23140b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f2313e00640 1 -- 172.21.15.44:0/731677693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23141fb960 con 0x7f231415ff50 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f2313e00640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f231415ff50 0x7f23141410e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f230c00e9f0 tx=0x7f230c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:18.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f230b400640 1 -- 172.21.15.44:0/731677693 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c00cd80 con 0x7f231415ff50 2024-09-13T08:40:18.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f230b400640 1 -- 172.21.15.44:0/731677693 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f230c004c20 con 0x7f231415ff50 2024-09-13T08:40:18.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f231a600640 1 -- 172.21.15.44:0/731677693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231414e3c0 con 0x7f231415ff50 2024-09-13T08:40:18.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f230b400640 1 -- 172.21.15.44:0/731677693 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c005020 con 0x7f231415ff50 2024-09-13T08:40:18.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f231a600640 1 -- 172.21.15.44:0/731677693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23141fae80 con 0x7f231415ff50 2024-09-13T08:40:18.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f231a600640 1 -- 172.21.15.44:0/731677693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231406d630 con 0x7f231415ff50 2024-09-13T08:40:18.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.521+0000 7f230b400640 1 -- 172.21.15.44:0/731677693 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 97284+0+0 (secure 0 0 0) 0x7f230c0051c0 con 0x7f231415ff50 2024-09-13T08:40:18.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.525+0000 7f230b400640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22f0073100 0x7f22f00755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:18.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.525+0000 7f230b400640 1 -- 172.21.15.44:0/731677693 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f230c094ca0 con 0x7f231415ff50 2024-09-13T08:40:18.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.525+0000 7f2313400640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22f0073100 0x7f22f00755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:18.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.525+0000 7f2313400640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22f0073100 0x7f22f00755c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2304001550 tx=0x7f23040079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:18.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.529+0000 7f230b400640 1 -- 172.21.15.44:0/731677693 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f230c016030 con 0x7f231415ff50 2024-09-13T08:40:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.705+0000 7f231a600640 1 -- 172.21.15.44:0/731677693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f23140b39f0 con 0x7f231415ff50 2024-09-13T08:40:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.705+0000 7f230b400640 1 -- 172.21.15.44:0/731677693 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v29)=0 v29) ==== 131+0+0 (secure 0 0 0) 0x7f230c05f3a0 con 0x7f231415ff50 2024-09-13T08:40:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 -- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22f0073100 msgr2=0x7f22f00755c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22f0073100 0x7f22f00755c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2304001550 tx=0x7f23040079c0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 -- 172.21.15.44:0/731677693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f231415ff50 msgr2=0x7f23141410e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f231415ff50 0x7f23141410e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f230c00e9f0 tx=0x7f230c00eeb0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 -- 172.21.15.44:0/731677693 shutdown_connections 2024-09-13T08:40:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22f0073100 0x7f22f00755c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f231415ff50 0x7f23141410e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2314152b90 0x7f23140b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 --2- 172.21.15.44:0/731677693 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23140c6b00 0x7f23140b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 -- 172.21.15.44:0/731677693 >> 172.21.15.44:0/731677693 conn(0x7f23140cce90 msgr2=0x7f231415b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 -- 172.21.15.44:0/731677693 shutdown_connections 2024-09-13T08:40:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:18.709+0000 7f231a600640 1 -- 172.21.15.44:0/731677693 wait complete. 2024-09-13T08:40:18.732 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:40:18.732 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:40:18.804 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:40:18.890 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:40:18.890 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:40:18.891 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:40:18.891 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:18.900 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:40:18.900 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:18.953 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:40:19.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.141+0000 7fddf9c00640 1 -- 172.21.15.44:0/3077731813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddf40b49e0 msgr2=0x7fddf406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:19.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.141+0000 7fddf9c00640 1 --2- 172.21.15.44:0/3077731813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddf40b49e0 0x7fddf406f7b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fddec009c80 tx=0x7fddec01cd10 comp rx=0 tx=0).stop 2024-09-13T08:40:19.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.141+0000 7fddf9c00640 1 -- 172.21.15.44:0/3077731813 shutdown_connections 2024-09-13T08:40:19.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.141+0000 7fddf9c00640 1 --2- 172.21.15.44:0/3077731813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddf40b49e0 0x7fddf406f7b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:19.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.141+0000 7fddf9c00640 1 --2- 172.21.15.44:0/3077731813 >> v2:172.21.15.129:3300/0 conn(0x7fddf415dae0 0x7fddf40acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:19.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.141+0000 7fddf9c00640 1 -- 172.21.15.44:0/3077731813 >> 172.21.15.44:0/3077731813 conn(0x7fddf40b18f0 msgr2=0x7fddf40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:19.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.141+0000 7fddf9c00640 1 -- 172.21.15.44:0/3077731813 shutdown_connections 2024-09-13T08:40:19.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 -- 172.21.15.44:0/3077731813 wait complete. 2024-09-13T08:40:19.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 Processor -- start 2024-09-13T08:40:19.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 -- start start 2024-09-13T08:40:19.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddf40b49e0 0x7fddf41552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:19.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddf40cd100 0x7fddf406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:19.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf415dae0 0x7fddf406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:19.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fddf40cf1c0 con 0x7fddf415dae0 2024-09-13T08:40:19.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fddf40cf040 con 0x7fddf40cd100 2024-09-13T08:40:19.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fddf40cf340 con 0x7fddf40b49e0 2024-09-13T08:40:19.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf415dae0 0x7fddf406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:19.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf415dae0 0x7fddf406eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44710/0 (socket says 172.21.15.44:44710) 2024-09-13T08:40:19.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf3400640 1 -- 172.21.15.44:0/363670925 learned_addr learned my addr 172.21.15.44:0/363670925 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:19.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf2a00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddf40b49e0 0x7fddf41552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:19.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddebe00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddf40cd100 0x7fddf406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:19.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf3400640 1 -- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddf40b49e0 msgr2=0x7fddf41552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:19.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf3400640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddf40b49e0 0x7fddf41552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:19.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf3400640 1 -- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddf40cd100 msgr2=0x7fddf406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:19.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf3400640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddf40cd100 0x7fddf406e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:19.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf3400640 1 -- 172.21.15.44:0/363670925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf409d7d0 con 0x7fddf415dae0 2024-09-13T08:40:19.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf3400640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf415dae0 0x7fddf406eb30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fddec009bd0 tx=0x7fddec0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:19.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddeaa00640 1 -- 172.21.15.44:0/363670925 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0bb070 con 0x7fddf415dae0 2024-09-13T08:40:19.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddeaa00640 1 -- 172.21.15.44:0/363670925 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddec004d40 con 0x7fddf415dae0 2024-09-13T08:40:19.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 -- 172.21.15.44:0/363670925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf41557f0 con 0x7fddf415dae0 2024-09-13T08:40:19.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddeaa00640 1 -- 172.21.15.44:0/363670925 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0b2040 con 0x7fddf415dae0 2024-09-13T08:40:19.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.145+0000 7fddf9c00640 1 -- 172.21.15.44:0/363670925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf41fb120 con 0x7fddf415dae0 2024-09-13T08:40:19.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.149+0000 7fddf9c00640 1 -- 172.21.15.44:0/363670925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf40acd40 con 0x7fddf415dae0 2024-09-13T08:40:19.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.149+0000 7fddeaa00640 1 -- 172.21.15.44:0/363670925 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fddec0be070 con 0x7fddf415dae0 2024-09-13T08:40:19.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.149+0000 7fddeaa00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fddd0073220 0x7fddd00756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:19.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.149+0000 7fddeaa00640 1 -- 172.21.15.44:0/363670925 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fddec12e160 con 0x7fddf415dae0 2024-09-13T08:40:19.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.149+0000 7fddf2a00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fddd0073220 0x7fddd00756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:19.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.153+0000 7fddf2a00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fddd0073220 0x7fddd00756e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009b40 tx=0x7fdde40091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:19.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.153+0000 7fddeaa00640 1 -- 172.21.15.44:0/363670925 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fddec016990 con 0x7fddf415dae0 2024-09-13T08:40:19.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 -- 172.21.15.44:0/363670925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fddf415b160 con 0x7fddf415dae0 2024-09-13T08:40:19.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddeaa00640 1 -- 172.21.15.44:0/363670925 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7fddec0f87e0 con 0x7fddf415dae0 2024-09-13T08:40:19.362 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:40:19.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 -- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fddd0073220 msgr2=0x7fddd00756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:19.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fddd0073220 0x7fddd00756e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009b40 tx=0x7fdde40091e0 comp rx=0 tx=0).stop 2024-09-13T08:40:19.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 -- 172.21.15.44:0/363670925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf415dae0 msgr2=0x7fddf406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:19.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf415dae0 0x7fddf406eb30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fddec009bd0 tx=0x7fddec0092e0 comp rx=0 tx=0).stop 2024-09-13T08:40:19.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 -- 172.21.15.44:0/363670925 shutdown_connections 2024-09-13T08:40:19.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fddd0073220 0x7fddd00756e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:19.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf415dae0 0x7fddf406eb30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:19.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddf40cd100 0x7fddf406e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:19.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 --2- 172.21.15.44:0/363670925 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddf40b49e0 0x7fddf41552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:19.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 -- 172.21.15.44:0/363670925 >> 172.21.15.44:0/363670925 conn(0x7fddf40b18f0 msgr2=0x7fddf415a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:19.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 -- 172.21.15.44:0/363670925 shutdown_connections 2024-09-13T08:40:19.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:19.361+0000 7fddf9c00640 1 -- 172.21.15.44:0/363670925 wait complete. 2024-09-13T08:40:19.384 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T08:40:19.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:19.384 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T08:40:21.290 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:40:21.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:21.291 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T08:40:21.309 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:40:21.310 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:40:21.310 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:21.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:40:21.319 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:40:21.319 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:40:21.319 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:21.366 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:40:21.366 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:40:21.366 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:40:21.366 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:21.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:40:21.375 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:40:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 -- 172.21.15.44:0/1683885144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71000b63f0 msgr2=0x7f7100166c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 --2- 172.21.15.44:0/1683885144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71000b63f0 0x7f7100166c90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009c80 tx=0x7f70f801cd70 comp rx=0 tx=0).stop 2024-09-13T08:40:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 -- 172.21.15.44:0/1683885144 shutdown_connections 2024-09-13T08:40:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 --2- 172.21.15.44:0/1683885144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71000b63f0 0x7f7100166c90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 -- 172.21.15.44:0/1683885144 >> 172.21.15.44:0/1683885144 conn(0x7f71000d1d90 msgr2=0x7f71000d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 -- 172.21.15.44:0/1683885144 shutdown_connections 2024-09-13T08:40:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 -- 172.21.15.44:0/1683885144 wait complete. 2024-09-13T08:40:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 Processor -- start 2024-09-13T08:40:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 -- start start 2024-09-13T08:40:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71000b63f0 0x7f71001551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71000d2940 0x7f7100153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f710015bb90 0x7f710014e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f710016a0b0 con 0x7f71000b63f0 2024-09-13T08:40:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7100169f30 con 0x7f71000d2940 2024-09-13T08:40:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7108000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f710016a230 con 0x7f710015bb90 2024-09-13T08:40:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7105600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71000b63f0 0x7f71001551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7105600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71000b63f0 0x7f71001551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44732/0 (socket says 172.21.15.44:44732) 2024-09-13T08:40:21.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.553+0000 7f7105600640 1 -- 172.21.15.44:0/1035758571 learned_addr learned my addr 172.21.15.44:0/1035758571 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:21.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7104c00640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71000d2940 0x7f7100153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:21.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7106000640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f710015bb90 0x7f710014e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:21.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7105600640 1 -- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f710015bb90 msgr2=0x7f710014e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:21.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7105600640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f710015bb90 0x7f710014e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7105600640 1 -- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71000d2940 msgr2=0x7f7100153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:21.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7105600640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71000d2940 0x7f7100153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7105600640 1 -- 172.21.15.44:0/1035758571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7100150870 con 0x7f71000b63f0 2024-09-13T08:40:21.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7104c00640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71000d2940 0x7f7100153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:21.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7105600640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71000b63f0 0x7f71001551d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f70f400c450 tx=0x7f70f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f70fd600640 1 -- 172.21.15.44:0/1035758571 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f40052d0 con 0x7f71000b63f0 2024-09-13T08:40:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f70fd600640 1 -- 172.21.15.44:0/1035758571 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70f4005470 con 0x7f71000b63f0 2024-09-13T08:40:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7108000640 1 -- 172.21.15.44:0/1035758571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7100150b50 con 0x7f71000b63f0 2024-09-13T08:40:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f70fd600640 1 -- 172.21.15.44:0/1035758571 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4012870 con 0x7f71000b63f0 2024-09-13T08:40:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7108000640 1 -- 172.21.15.44:0/1035758571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710015a400 con 0x7f71000b63f0 2024-09-13T08:40:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f7108000640 1 -- 172.21.15.44:0/1035758571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710015a600 con 0x7f71000b63f0 2024-09-13T08:40:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.557+0000 7f70fd600640 1 -- 172.21.15.44:0/1035758571 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f70f4003600 con 0x7f71000b63f0 2024-09-13T08:40:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.561+0000 7f70fd600640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70e0073220 0x7f70e00756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.561+0000 7f70fd600640 1 -- 172.21.15.44:0/1035758571 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f70f4093fd0 con 0x7f71000b63f0 2024-09-13T08:40:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.561+0000 7f7104c00640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70e0073220 0x7f70e00756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:21.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.561+0000 7f7104c00640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70e0073220 0x7f70e00756e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f71000cff80 tx=0x7f70ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:21.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.561+0000 7f70fd600640 1 -- 172.21.15.44:0/1035758571 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f710015a600 con 0x7f71000b63f0 2024-09-13T08:40:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.773+0000 7f7108000640 1 -- 172.21.15.44:0/1035758571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7100070270 con 0x7f71000b63f0 2024-09-13T08:40:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.773+0000 7f70fd600640 1 -- 172.21.15.44:0/1035758571 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7f70f405ebd0 con 0x7f71000b63f0 2024-09-13T08:40:21.777 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:21.777 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"btime":"2024-09-13T08:38:28:234730+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4828,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/238336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":238336703},{"type":"v1","addr":"172.21.15.129:6836","nonce":238336703}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2440287716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2440287716},{"type":"v1","addr":"172.21.15.44:6834","nonce":2440287716}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3117701056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":3117701056},{"type":"v1","addr":"172.21.15.129:6837","nonce":3117701056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:38:19.608778+0000","modified":"2024-09-13T08:38:22.619955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4866":{"gid":4866,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3820157165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3820157165},{"type":"v1","addr":"172.21.15.44:6835","nonce":3820157165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4866,"qdb_cluster":[4866]},"id":2}]} 2024-09-13T08:40:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 18 2024-09-13T08:40:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 -- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70e0073220 msgr2=0x7f70e00756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70e0073220 0x7f70e00756e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f71000cff80 tx=0x7f70ec009040 comp rx=0 tx=0).stop 2024-09-13T08:40:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 -- 172.21.15.44:0/1035758571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71000b63f0 msgr2=0x7f71001551d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71000b63f0 0x7f71001551d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f70f400c450 tx=0x7f70f400c910 comp rx=0 tx=0).stop 2024-09-13T08:40:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 -- 172.21.15.44:0/1035758571 shutdown_connections 2024-09-13T08:40:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70e0073220 0x7f70e00756e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f710015bb90 0x7f710014e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71000d2940 0x7f7100153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 --2- 172.21.15.44:0/1035758571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71000b63f0 0x7f71001551d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 -- 172.21.15.44:0/1035758571 >> 172.21.15.44:0/1035758571 conn(0x7f71000d1d90 msgr2=0x7f7100158060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 -- 172.21.15.44:0/1035758571 shutdown_connections 2024-09-13T08:40:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.777+0000 7f7108000640 1 -- 172.21.15.44:0/1035758571 wait complete. 2024-09-13T08:40:21.798 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:40:21.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.945+0000 7f56d0000640 1 -- 172.21.15.44:0/696782733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56c815bea0 msgr2=0x7f56c80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:21.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.945+0000 7f56d0000640 1 --2- 172.21.15.44:0/696782733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56c815bea0 0x7f56c80ce670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009840 tx=0x7f56bc01c820 comp rx=0 tx=0).stop 2024-09-13T08:40:21.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.945+0000 7f56d0000640 1 -- 172.21.15.44:0/696782733 shutdown_connections 2024-09-13T08:40:21.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.945+0000 7f56d0000640 1 --2- 172.21.15.44:0/696782733 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56c80d28e0 0x7f56c80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.945+0000 7f56d0000640 1 --2- 172.21.15.44:0/696782733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56c815bea0 0x7f56c80ce670 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.945+0000 7f56d0000640 1 -- 172.21.15.44:0/696782733 >> 172.21.15.44:0/696782733 conn(0x7f56c80d1dc0 msgr2=0x7f56c80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:21.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.945+0000 7f56d0000640 1 -- 172.21.15.44:0/696782733 shutdown_connections 2024-09-13T08:40:21.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.945+0000 7f56d0000640 1 -- 172.21.15.44:0/696782733 wait complete. 2024-09-13T08:40:21.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.945+0000 7f56d0000640 1 Processor -- start 2024-09-13T08:40:21.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.945+0000 7f56d0000640 1 -- start start 2024-09-13T08:40:21.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56d0000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c80cdb30 0x7f56c80c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:21.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56d0000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56c80d28e0 0x7f56c80cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:21.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56d0000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56c815bea0 0x7f56c80b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:21.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56d0000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8168fc0 con 0x7f56c80cdb30 2024-09-13T08:40:21.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56d0000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8168e40 con 0x7f56c815bea0 2024-09-13T08:40:21.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56d0000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f56c8169140 con 0x7f56c80d28e0 2024-09-13T08:40:21.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56cd600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c80cdb30 0x7f56c80c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:21.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56cd600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c80cdb30 0x7f56c80c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44748/0 (socket says 172.21.15.44:44748) 2024-09-13T08:40:21.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56ccc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56c80d28e0 0x7f56c80cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:21.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56cd600640 1 -- 172.21.15.44:0/3581901612 learned_addr learned my addr 172.21.15.44:0/3581901612 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:21.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56ce000640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56c815bea0 0x7f56c80b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:21.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56cd600640 1 -- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56c80d28e0 msgr2=0x7f56c80cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:21.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56cd600640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56c80d28e0 0x7f56c80cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56cd600640 1 -- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56c815bea0 msgr2=0x7f56c80b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:21.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56cd600640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56c815bea0 0x7f56c80b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:21.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56cd600640 1 -- 172.21.15.44:0/3581901612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c81452d0 con 0x7f56c80cdb30 2024-09-13T08:40:21.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56ccc00640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56c80d28e0 0x7f56c80cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:40:21.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56cd600640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c80cdb30 0x7f56c80c98a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f56bc01aed0 tx=0x7f56bc005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:21.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56c5600640 1 -- 172.21.15.44:0/3581901612 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0b2040 con 0x7f56c80cdb30 2024-09-13T08:40:21.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56c5600640 1 -- 172.21.15.44:0/3581901612 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56bc004320 con 0x7f56c80cdb30 2024-09-13T08:40:21.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56c5600640 1 -- 172.21.15.44:0/3581901612 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0052d0 con 0x7f56c80cdb30 2024-09-13T08:40:21.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56d0000640 1 -- 172.21.15.44:0/3581901612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c81455b0 con 0x7f56c80cdb30 2024-09-13T08:40:21.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56d0000640 1 -- 172.21.15.44:0/3581901612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c8145b70 con 0x7f56c80cdb30 2024-09-13T08:40:21.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.949+0000 7f56d0000640 1 -- 172.21.15.44:0/3581901612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c80cc9d0 con 0x7f56c80cdb30 2024-09-13T08:40:21.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.953+0000 7f56c5600640 1 -- 172.21.15.44:0/3581901612 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f56bc0a5770 con 0x7f56c80cdb30 2024-09-13T08:40:21.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.953+0000 7f56c5600640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56a8073220 0x7f56a80756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:21.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.953+0000 7f56c5600640 1 -- 172.21.15.44:0/3581901612 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f56bc12dd90 con 0x7f56c80cdb30 2024-09-13T08:40:21.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.953+0000 7f56ccc00640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56a8073220 0x7f56a80756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:21.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.953+0000 7f56ccc00640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56a8073220 0x7f56a80756e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f56b4007f30 tx=0x7f56b4007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:21.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:21.957+0000 7f56c5600640 1 -- 172.21.15.44:0/3581901612 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f56bc016410 con 0x7f56c80cdb30 2024-09-13T08:40:22.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56d0000640 1 -- 172.21.15.44:0/3581901612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56c80724b0 con 0x7f56c80cdb30 2024-09-13T08:40:22.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56c5600640 1 -- 172.21.15.44:0/3581901612 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7f56bc0f82b0 con 0x7f56c80cdb30 2024-09-13T08:40:22.172 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:22.172 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"btime":"2024-09-13T08:38:28:234730+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4828,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/238336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":238336703},{"type":"v1","addr":"172.21.15.129:6836","nonce":238336703}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2440287716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2440287716},{"type":"v1","addr":"172.21.15.44:6834","nonce":2440287716}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3117701056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":3117701056},{"type":"v1","addr":"172.21.15.129:6837","nonce":3117701056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:38:19.608778+0000","modified":"2024-09-13T08:38:22.619955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4866":{"gid":4866,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3820157165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3820157165},{"type":"v1","addr":"172.21.15.44:6835","nonce":3820157165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4866,"qdb_cluster":[4866]},"id":2}]} 2024-09-13T08:40:22.172 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 18 2024-09-13T08:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56d0000640 1 -- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56a8073220 msgr2=0x7f56a80756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56d0000640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56a8073220 0x7f56a80756e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f56b4007f30 tx=0x7f56b4007ec0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56d0000640 1 -- 172.21.15.44:0/3581901612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c80cdb30 msgr2=0x7f56c80c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56d0000640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c80cdb30 0x7f56c80c98a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f56bc01aed0 tx=0x7f56bc005d90 comp rx=0 tx=0).stop 2024-09-13T08:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56d0000640 1 -- 172.21.15.44:0/3581901612 shutdown_connections 2024-09-13T08:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56d0000640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56a8073220 0x7f56a80756e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56d0000640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56c815bea0 0x7f56c80b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56d0000640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56c80d28e0 0x7f56c80cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.169+0000 7f56d0000640 1 --2- 172.21.15.44:0/3581901612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c80cdb30 0x7f56c80c98a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.173+0000 7f56d0000640 1 -- 172.21.15.44:0/3581901612 >> 172.21.15.44:0/3581901612 conn(0x7f56c80d1dc0 msgr2=0x7f56c80b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:22.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.173+0000 7f56d0000640 1 -- 172.21.15.44:0/3581901612 shutdown_connections 2024-09-13T08:40:22.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.173+0000 7f56d0000640 1 -- 172.21.15.44:0/3581901612 wait complete. 2024-09-13T08:40:22.194 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:40:22.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.337+0000 7fd391c00640 1 -- 172.21.15.44:0/3287879500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38c15bea0 msgr2=0x7fd38c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.337+0000 7fd391c00640 1 --2- 172.21.15.44:0/3287879500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38c15bea0 0x7fd38c0ce670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009840 tx=0x7fd37c01c790 comp rx=0 tx=0).stop 2024-09-13T08:40:22.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.337+0000 7fd391c00640 1 -- 172.21.15.44:0/3287879500 shutdown_connections 2024-09-13T08:40:22.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.337+0000 7fd391c00640 1 --2- 172.21.15.44:0/3287879500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38c15bea0 0x7fd38c0ce670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.337+0000 7fd391c00640 1 -- 172.21.15.44:0/3287879500 >> 172.21.15.44:0/3287879500 conn(0x7fd38c0d1dc0 msgr2=0x7fd38c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:22.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.337+0000 7fd391c00640 1 -- 172.21.15.44:0/3287879500 shutdown_connections 2024-09-13T08:40:22.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.337+0000 7fd391c00640 1 -- 172.21.15.44:0/3287879500 wait complete. 2024-09-13T08:40:22.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd391c00640 1 Processor -- start 2024-09-13T08:40:22.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd391c00640 1 -- start start 2024-09-13T08:40:22.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd391c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38c0cdb30 0x7fd38c13d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:22.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd391c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd38c0d28e0 0x7fd38c150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:22.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd391c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c15bea0 0x7fd38c0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:22.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd391c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c16a930 con 0x7fd38c15bea0 2024-09-13T08:40:22.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd391c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c16a7b0 con 0x7fd38c0cdb30 2024-09-13T08:40:22.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd391c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd38c16aab0 con 0x7fd38c0d28e0 2024-09-13T08:40:22.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd38b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c15bea0 0x7fd38c0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:22.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd38b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c15bea0 0x7fd38c0c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57246/0 (socket says 172.21.15.44:57246) 2024-09-13T08:40:22.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd38b400640 1 -- 172.21.15.44:0/2915145149 learned_addr learned my addr 172.21.15.44:0/2915145149 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:22.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd38aa00640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38c0cdb30 0x7fd38c13d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:22.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd38b400640 1 -- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd38c0d28e0 msgr2=0x7fd38c150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd38b400640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd38c0d28e0 0x7fd38c150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd38b400640 1 -- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38c0cdb30 msgr2=0x7fd38c13d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd38b400640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38c0cdb30 0x7fd38c13d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd38b400640 1 -- 172.21.15.44:0/2915145149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c13da10 con 0x7fd38c15bea0 2024-09-13T08:40:22.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd38b400640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c15bea0 0x7fd38c0c6010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd38400c730 tx=0x7fd38400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:22.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd382a00640 1 -- 172.21.15.44:0/2915145149 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384003db0 con 0x7fd38c15bea0 2024-09-13T08:40:22.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd382a00640 1 -- 172.21.15.44:0/2915145149 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd38400ce40 con 0x7fd38c15bea0 2024-09-13T08:40:22.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd382a00640 1 -- 172.21.15.44:0/2915145149 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3840052d0 con 0x7fd38c15bea0 2024-09-13T08:40:22.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd391c00640 1 -- 172.21.15.44:0/2915145149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c15a410 con 0x7fd38c15bea0 2024-09-13T08:40:22.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.341+0000 7fd391c00640 1 -- 172.21.15.44:0/2915145149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c158a10 con 0x7fd38c15bea0 2024-09-13T08:40:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.345+0000 7fd391c00640 1 -- 172.21.15.44:0/2915145149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38c0b7a90 con 0x7fd38c15bea0 2024-09-13T08:40:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.345+0000 7fd382a00640 1 -- 172.21.15.44:0/2915145149 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fd384005470 con 0x7fd38c15bea0 2024-09-13T08:40:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.345+0000 7fd382a00640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd368073220 0x7fd3680756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.345+0000 7fd382a00640 1 -- 172.21.15.44:0/2915145149 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fd3840943c0 con 0x7fd38c15bea0 2024-09-13T08:40:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.345+0000 7fd38aa00640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd368073220 0x7fd3680756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:22.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.345+0000 7fd38aa00640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd368073220 0x7fd3680756e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0af000 tx=0x7fd37c0b2040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:22.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.349+0000 7fd382a00640 1 -- 172.21.15.44:0/2915145149 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd38c13da10 con 0x7fd38c15bea0 2024-09-13T08:40:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.525+0000 7fd391c00640 1 -- 172.21.15.44:0/2915145149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd38c0b77b0 con 0x7fd38c15bea0 2024-09-13T08:40:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.525+0000 7fd382a00640 1 -- 172.21.15.44:0/2915145149 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16930 (secure 0 0 0) 0x7fd38405efc0 con 0x7fd38c15bea0 2024-09-13T08:40:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:22.529 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":35,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:39:15.614386+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:38:17.738159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T08:38:18.557711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.17","mappings":[{"from":4,"to":3}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:40:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 -- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd368073220 msgr2=0x7fd3680756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd368073220 0x7fd3680756e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0af000 tx=0x7fd37c0b2040 comp rx=0 tx=0).stop 2024-09-13T08:40:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 -- 172.21.15.44:0/2915145149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c15bea0 msgr2=0x7fd38c0c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c15bea0 0x7fd38c0c6010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd38400c730 tx=0x7fd38400cbf0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 -- 172.21.15.44:0/2915145149 shutdown_connections 2024-09-13T08:40:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd368073220 0x7fd3680756e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c15bea0 0x7fd38c0c6010 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd38c0d28e0 0x7fd38c150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 --2- 172.21.15.44:0/2915145149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38c0cdb30 0x7fd38c13d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 -- 172.21.15.44:0/2915145149 >> 172.21.15.44:0/2915145149 conn(0x7fd38c0d1dc0 msgr2=0x7fd38c0b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 -- 172.21.15.44:0/2915145149 shutdown_connections 2024-09-13T08:40:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.529+0000 7fd391c00640 1 -- 172.21.15.44:0/2915145149 wait complete. 2024-09-13T08:40:22.550 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:40:22.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:40:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.693+0000 7efc57e00640 1 -- 172.21.15.44:0/1007586140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5015bea0 msgr2=0x7efc500ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.693+0000 7efc57e00640 1 --2- 172.21.15.44:0/1007586140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5015bea0 0x7efc500ce670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efc44009900 tx=0x7efc4401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.693+0000 7efc57e00640 1 -- 172.21.15.44:0/1007586140 shutdown_connections 2024-09-13T08:40:22.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.693+0000 7efc57e00640 1 --2- 172.21.15.44:0/1007586140 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc500d28e0 0x7efc500cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.693+0000 7efc57e00640 1 --2- 172.21.15.44:0/1007586140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5015bea0 0x7efc500ce670 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.693+0000 7efc57e00640 1 -- 172.21.15.44:0/1007586140 >> 172.21.15.44:0/1007586140 conn(0x7efc500d1dc0 msgr2=0x7efc500d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:22.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.693+0000 7efc57e00640 1 -- 172.21.15.44:0/1007586140 shutdown_connections 2024-09-13T08:40:22.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.693+0000 7efc57e00640 1 -- 172.21.15.44:0/1007586140 wait complete. 2024-09-13T08:40:22.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc57e00640 1 Processor -- start 2024-09-13T08:40:22.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc57e00640 1 -- start start 2024-09-13T08:40:22.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc57e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc500cdb30 0x7efc5014e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:22.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc57e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc500d28e0 0x7efc5014d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc57e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5015bea0 0x7efc500b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc57e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efc50169050 con 0x7efc5015bea0 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc57e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc50168ed0 con 0x7efc500d28e0 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc57e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7efc501691d0 con 0x7efc500cdb30 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc55e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5015bea0 0x7efc500b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc55e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5015bea0 0x7efc500b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57264/0 (socket says 172.21.15.44:57264) 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc55e00640 1 -- 172.21.15.44:0/582145479 learned_addr learned my addr 172.21.15.44:0/582145479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc55400640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc500cdb30 0x7efc5014e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc54a00640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc500d28e0 0x7efc5014d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc55e00640 1 -- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc500cdb30 msgr2=0x7efc5014e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc55e00640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc500cdb30 0x7efc5014e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc55e00640 1 -- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc500d28e0 msgr2=0x7efc5014d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc55e00640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc500d28e0 0x7efc5014d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc55e00640 1 -- 172.21.15.44:0/582145479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5014df20 con 0x7efc5015bea0 2024-09-13T08:40:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc55e00640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5015bea0 0x7efc500b30d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efc4800b9a0 tx=0x7efc4800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc4d600640 1 -- 172.21.15.44:0/582145479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48019070 con 0x7efc5015bea0 2024-09-13T08:40:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc4d600640 1 -- 172.21.15.44:0/582145479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc4800ccb0 con 0x7efc5015bea0 2024-09-13T08:40:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc57e00640 1 -- 172.21.15.44:0/582145479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc500b71a0 con 0x7efc5015bea0 2024-09-13T08:40:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc4d600640 1 -- 172.21.15.44:0/582145479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc480146a0 con 0x7efc5015bea0 2024-09-13T08:40:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.697+0000 7efc57e00640 1 -- 172.21.15.44:0/582145479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc500b51e0 con 0x7efc5015bea0 2024-09-13T08:40:22.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.701+0000 7efc57e00640 1 -- 172.21.15.44:0/582145479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc500cc9d0 con 0x7efc5015bea0 2024-09-13T08:40:22.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.701+0000 7efc4d600640 1 -- 172.21.15.44:0/582145479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7efc4801c070 con 0x7efc5015bea0 2024-09-13T08:40:22.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.701+0000 7efc4d600640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc30073220 0x7efc300756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:22.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.701+0000 7efc4d600640 1 -- 172.21.15.44:0/582145479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7efc48094190 con 0x7efc5015bea0 2024-09-13T08:40:22.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.701+0000 7efc55400640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc30073220 0x7efc300756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:22.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.701+0000 7efc55400640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc30073220 0x7efc300756e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc440af000 tx=0x7efc4401b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:22.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.705+0000 7efc4d600640 1 -- 172.21.15.44:0/582145479 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7efc4805e7b0 con 0x7efc5015bea0 2024-09-13T08:40:22.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 -- 172.21.15.44:0/582145479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc500b07d0 con 0x7efc5015bea0 2024-09-13T08:40:22.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc4d600640 1 -- 172.21.15.44:0/582145479 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7efc480631e0 con 0x7efc5015bea0 2024-09-13T08:40:22.920 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:22.920 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"btime":"2024-09-13T08:38:28:234730+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4828,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/238336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":238336703},{"type":"v1","addr":"172.21.15.129:6836","nonce":238336703}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2440287716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2440287716},{"type":"v1","addr":"172.21.15.44:6834","nonce":2440287716}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3117701056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":3117701056},{"type":"v1","addr":"172.21.15.129:6837","nonce":3117701056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:38:19.608778+0000","modified":"2024-09-13T08:38:22.619955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4866":{"gid":4866,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3820157165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3820157165},{"type":"v1","addr":"172.21.15.44:6835","nonce":3820157165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4866,"qdb_cluster":[4866]},"id":2}]} 2024-09-13T08:40:22.920 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 18 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 -- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc30073220 msgr2=0x7efc300756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc30073220 0x7efc300756e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc440af000 tx=0x7efc4401b120 comp rx=0 tx=0).stop 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 -- 172.21.15.44:0/582145479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5015bea0 msgr2=0x7efc500b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5015bea0 0x7efc500b30d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efc4800b9a0 tx=0x7efc4800be60 comp rx=0 tx=0).stop 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 -- 172.21.15.44:0/582145479 shutdown_connections 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc30073220 0x7efc300756e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5015bea0 0x7efc500b30d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc500d28e0 0x7efc5014d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 --2- 172.21.15.44:0/582145479 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc500cdb30 0x7efc5014e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.917+0000 7efc57e00640 1 -- 172.21.15.44:0/582145479 >> 172.21.15.44:0/582145479 conn(0x7efc500d1dc0 msgr2=0x7efc5006e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.921+0000 7efc57e00640 1 -- 172.21.15.44:0/582145479 shutdown_connections 2024-09-13T08:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:22.921+0000 7efc57e00640 1 -- 172.21.15.44:0/582145479 wait complete. 2024-09-13T08:40:22.942 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:40:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.085+0000 7f3f9f600640 1 -- 172.21.15.44:0/2811298182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f9815e330 msgr2=0x7f3f9806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.085+0000 7f3f9f600640 1 --2- 172.21.15.44:0/2811298182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f9815e330 0x7f3f9806ec80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3f980c58a0 tx=0x7f3f8001c7a0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.085+0000 7f3f9f600640 1 -- 172.21.15.44:0/2811298182 shutdown_connections 2024-09-13T08:40:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.085+0000 7f3f9f600640 1 --2- 172.21.15.44:0/2811298182 >> v2:172.21.15.129:3300/0 conn(0x7f3f980b11d0 0x7f3f980afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.085+0000 7f3f9f600640 1 --2- 172.21.15.44:0/2811298182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f9815e330 0x7f3f9806ec80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.085+0000 7f3f9f600640 1 --2- 172.21.15.44:0/2811298182 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3f9815df50 0x7f3f9806e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.085+0000 7f3f9f600640 1 -- 172.21.15.44:0/2811298182 >> 172.21.15.44:0/2811298182 conn(0x7f3f980d1dc0 msgr2=0x7f3f980d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.085+0000 7f3f9f600640 1 -- 172.21.15.44:0/2811298182 shutdown_connections 2024-09-13T08:40:23.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.085+0000 7f3f9f600640 1 -- 172.21.15.44:0/2811298182 wait complete. 2024-09-13T08:40:23.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 Processor -- start 2024-09-13T08:40:23.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 -- start start 2024-09-13T08:40:23.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f980b11d0 0x7f3f981530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f9815df50 0x7f3f980d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3f9815e330 0x7f3f980d0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f980b2ec0 con 0x7f3f980b11d0 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f980b2d40 con 0x7f3f9815df50 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3f980b3040 con 0x7f3f9815e330 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f980b11d0 0x7f3f981530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f980b11d0 0x7f3f981530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57284/0 (socket says 172.21.15.44:57284) 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9cc00640 1 -- 172.21.15.44:0/2186023796 learned_addr learned my addr 172.21.15.44:0/2186023796 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9d600640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3f9815e330 0x7f3f980d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f8fe00640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f9815df50 0x7f3f980d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9cc00640 1 -- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3f9815e330 msgr2=0x7f3f980d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9cc00640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3f9815e330 0x7f3f980d0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9cc00640 1 -- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f9815df50 msgr2=0x7f3f980d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9cc00640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f9815df50 0x7f3f980d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9cc00640 1 -- 172.21.15.44:0/2186023796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f981f7460 con 0x7f3f980b11d0 2024-09-13T08:40:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9cc00640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f980b11d0 0x7f3f981530a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800b580 tx=0x7f3f8800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f95600640 1 -- 172.21.15.44:0/2186023796 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88014070 con 0x7f3f980b11d0 2024-09-13T08:40:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f95600640 1 -- 172.21.15.44:0/2186023796 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f88010980 con 0x7f3f980b11d0 2024-09-13T08:40:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f95600640 1 -- 172.21.15.44:0/2186023796 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88010dd0 con 0x7f3f980b11d0 2024-09-13T08:40:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 -- 172.21.15.44:0/2186023796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f981f7720 con 0x7f3f980b11d0 2024-09-13T08:40:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 -- 172.21.15.44:0/2186023796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f981f7ce0 con 0x7f3f980b11d0 2024-09-13T08:40:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.089+0000 7f3f9f600640 1 -- 172.21.15.44:0/2186023796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f980b35a0 con 0x7f3f980b11d0 2024-09-13T08:40:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.093+0000 7f3f95600640 1 -- 172.21.15.44:0/2186023796 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f3f88017070 con 0x7f3f980b11d0 2024-09-13T08:40:23.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.093+0000 7f3f95600640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3f74073220 0x7f3f740756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.093+0000 7f3f95600640 1 -- 172.21.15.44:0/2186023796 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f3f88094450 con 0x7f3f980b11d0 2024-09-13T08:40:23.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.093+0000 7f3f8fe00640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3f74073220 0x7f3f740756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.093+0000 7f3f8fe00640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3f74073220 0x7f3f740756e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3f800af000 tx=0x7f3f8001afe0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.097+0000 7f3f95600640 1 -- 172.21.15.44:0/2186023796 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3f8805efd0 con 0x7f3f980b11d0 2024-09-13T08:40:23.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.309+0000 7f3f9f600640 1 -- 172.21.15.44:0/2186023796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f981535e0 con 0x7f3f980b11d0 2024-09-13T08:40:23.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.309+0000 7f3f95600640 1 -- 172.21.15.44:0/2186023796 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7f3f8805e8d0 con 0x7f3f980b11d0 2024-09-13T08:40:23.312 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:23.313 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"btime":"2024-09-13T08:38:28:234730+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4828,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/238336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":238336703},{"type":"v1","addr":"172.21.15.129:6836","nonce":238336703}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2440287716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2440287716},{"type":"v1","addr":"172.21.15.44:6834","nonce":2440287716}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3117701056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":3117701056},{"type":"v1","addr":"172.21.15.129:6837","nonce":3117701056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:38:19.608778+0000","modified":"2024-09-13T08:38:22.619955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4866":{"gid":4866,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3820157165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3820157165},{"type":"v1","addr":"172.21.15.44:6835","nonce":3820157165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4866,"qdb_cluster":[4866]},"id":2}]} 2024-09-13T08:40:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 18 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 -- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3f74073220 msgr2=0x7f3f740756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3f74073220 0x7f3f740756e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3f800af000 tx=0x7f3f8001afe0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 -- 172.21.15.44:0/2186023796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f980b11d0 msgr2=0x7f3f981530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f980b11d0 0x7f3f981530a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800b580 tx=0x7f3f8800ba40 comp rx=0 tx=0).stop 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 -- 172.21.15.44:0/2186023796 shutdown_connections 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3f74073220 0x7f3f740756e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3f9815e330 0x7f3f980d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f9815df50 0x7f3f980d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 --2- 172.21.15.44:0/2186023796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f980b11d0 0x7f3f981530a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 -- 172.21.15.44:0/2186023796 >> 172.21.15.44:0/2186023796 conn(0x7f3f980d1dc0 msgr2=0x7f3f98070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 -- 172.21.15.44:0/2186023796 shutdown_connections 2024-09-13T08:40:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.313+0000 7f3f9f600640 1 -- 172.21.15.44:0/2186023796 wait complete. 2024-09-13T08:40:23.335 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:40:23.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.477+0000 7fabbca00640 1 -- 172.21.15.44:0/2339293186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb80d28e0 msgr2=0x7fabb80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.477+0000 7fabbca00640 1 --2- 172.21.15.44:0/2339293186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb80d28e0 0x7fabb80cebb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fabb80745d0 tx=0x7fabb001ccf0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.477+0000 7fabbca00640 1 -- 172.21.15.44:0/2339293186 shutdown_connections 2024-09-13T08:40:23.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.477+0000 7fabbca00640 1 --2- 172.21.15.44:0/2339293186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb80d28e0 0x7fabb80cebb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.477+0000 7fabbca00640 1 --2- 172.21.15.44:0/2339293186 >> v2:172.21.15.129:3300/0 conn(0x7fabb815bea0 0x7fabb80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.477+0000 7fabbca00640 1 -- 172.21.15.44:0/2339293186 >> 172.21.15.44:0/2339293186 conn(0x7fabb80d1dc0 msgr2=0x7fabb80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:23.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.477+0000 7fabbca00640 1 -- 172.21.15.44:0/2339293186 shutdown_connections 2024-09-13T08:40:23.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.477+0000 7fabbca00640 1 -- 172.21.15.44:0/2339293186 wait complete. 2024-09-13T08:40:23.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabbca00640 1 Processor -- start 2024-09-13T08:40:23.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabbca00640 1 -- start start 2024-09-13T08:40:23.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabbca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb80cdb30 0x7fabb8148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabbca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabb80d28e0 0x7fabb806ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabbca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb815bea0 0x7fabb80c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabbca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8168fc0 con 0x7fabb80cdb30 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabbca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8168e40 con 0x7fabb815bea0 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabbca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fabb8169140 con 0x7fabb80d28e0 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabb5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb80cdb30 0x7fabb8148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabb5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb80cdb30 0x7fabb8148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57302/0 (socket says 172.21.15.44:57302) 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabb5e00640 1 -- 172.21.15.44:0/391048108 learned_addr learned my addr 172.21.15.44:0/391048108 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabb6800640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb815bea0 0x7fabb80c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabafe00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabb80d28e0 0x7fabb806ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabb5e00640 1 -- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabb80d28e0 msgr2=0x7fabb806ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabb5e00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabb80d28e0 0x7fabb806ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabb5e00640 1 -- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb815bea0 msgr2=0x7fabb80c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabb5e00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb815bea0 0x7fabb80c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabb5e00640 1 -- 172.21.15.44:0/391048108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb81490b0 con 0x7fabb80cdb30 2024-09-13T08:40:23.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabb5e00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb80cdb30 0x7fabb8148150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faba800d720 tx=0x7faba800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:23.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabae000640 1 -- 172.21.15.44:0/391048108 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8017070 con 0x7fabb80cdb30 2024-09-13T08:40:23.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabae000640 1 -- 172.21.15.44:0/391048108 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faba8009d20 con 0x7fabb80cdb30 2024-09-13T08:40:23.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabae000640 1 -- 172.21.15.44:0/391048108 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8005280 con 0x7fabb80cdb30 2024-09-13T08:40:23.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabbca00640 1 -- 172.21.15.44:0/391048108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb806f120 con 0x7fabb80cdb30 2024-09-13T08:40:23.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.481+0000 7fabbca00640 1 -- 172.21.15.44:0/391048108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb8148810 con 0x7fabb80cdb30 2024-09-13T08:40:23.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.485+0000 7fabbca00640 1 -- 172.21.15.44:0/391048108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb80cc9d0 con 0x7fabb80cdb30 2024-09-13T08:40:23.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.485+0000 7fabae000640 1 -- 172.21.15.44:0/391048108 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7faba8005420 con 0x7fabb80cdb30 2024-09-13T08:40:23.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.485+0000 7fabae000640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab94073220 0x7fab940756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.485+0000 7fabae000640 1 -- 172.21.15.44:0/391048108 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7faba8094660 con 0x7fabb80cdb30 2024-09-13T08:40:23.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.485+0000 7fabafe00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab94073220 0x7fab940756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.489+0000 7fabafe00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab94073220 0x7fab940756e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fabb8145f90 tx=0x7faba0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:23.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.489+0000 7fabae000640 1 -- 172.21.15.44:0/391048108 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fabb806f120 con 0x7fabb80cdb30 2024-09-13T08:40:23.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.669+0000 7fabbca00640 1 -- 172.21.15.44:0/391048108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabb80b0fa0 con 0x7fabb80cdb30 2024-09-13T08:40:23.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.669+0000 7fabae000640 1 -- 172.21.15.44:0/391048108 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16930 (secure 0 0 0) 0x7faba805ec10 con 0x7fabb80cdb30 2024-09-13T08:40:23.672 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:23.672 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":35,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:39:15.614386+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:38:17.738159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T08:38:18.557711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.17","mappings":[{"from":4,"to":3}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:40:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 -- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab94073220 msgr2=0x7fab940756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab94073220 0x7fab940756e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fabb8145f90 tx=0x7faba0009040 comp rx=0 tx=0).stop 2024-09-13T08:40:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 -- 172.21.15.44:0/391048108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb80cdb30 msgr2=0x7fabb8148150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb80cdb30 0x7fabb8148150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faba800d720 tx=0x7faba800dbe0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 -- 172.21.15.44:0/391048108 shutdown_connections 2024-09-13T08:40:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab94073220 0x7fab940756e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb815bea0 0x7fabb80c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fabb80d28e0 0x7fabb806ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 --2- 172.21.15.44:0/391048108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb80cdb30 0x7fabb8148150 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 -- 172.21.15.44:0/391048108 >> 172.21.15.44:0/391048108 conn(0x7fabb80d1dc0 msgr2=0x7fabb8071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:23.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 -- 172.21.15.44:0/391048108 shutdown_connections 2024-09-13T08:40:23.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.673+0000 7fabbca00640 1 -- 172.21.15.44:0/391048108 wait complete. 2024-09-13T08:40:23.694 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T08:40:23.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 -- 172.21.15.44:0/1601348168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f314c0cdb30 msgr2=0x7f314c0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 --2- 172.21.15.44:0/1601348168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f314c0cdb30 0x7f314c0aeef0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3144009bc0 tx=0x7f314401cba0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 -- 172.21.15.44:0/1601348168 shutdown_connections 2024-09-13T08:40:23.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 --2- 172.21.15.44:0/1601348168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f314c0cdb30 0x7f314c0aeef0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 --2- 172.21.15.44:0/1601348168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f314c0ced80 0x7f314c0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 -- 172.21.15.44:0/1601348168 >> 172.21.15.44:0/1601348168 conn(0x7f314c0d1dc0 msgr2=0x7f314c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:23.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 -- 172.21.15.44:0/1601348168 shutdown_connections 2024-09-13T08:40:23.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 -- 172.21.15.44:0/1601348168 wait complete. 2024-09-13T08:40:23.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 Processor -- start 2024-09-13T08:40:23.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 -- start start 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f314c0cdb30 0x7f314c159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f314c0ced80 0x7f314c159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f314c15bea0 0x7f314c0718a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f314c169290 con 0x7f314c15bea0 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f314c169110 con 0x7f314c0cdb30 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3153e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f314c169410 con 0x7f314c0ced80 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f314c15bea0 0x7f314c0718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f314c15bea0 0x7f314c0718a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57314/0 (socket says 172.21.15.44:57314) 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151e00640 1 -- 172.21.15.44:0/3543433036 learned_addr learned my addr 172.21.15.44:0/3543433036 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3150a00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f314c0ced80 0x7f314c159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151e00640 1 -- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f314c0ced80 msgr2=0x7f314c159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151e00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f314c0ced80 0x7f314c159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151400640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f314c0cdb30 0x7f314c159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151e00640 1 -- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f314c0cdb30 msgr2=0x7f314c159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151e00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f314c0cdb30 0x7f314c159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151e00640 1 -- 172.21.15.44:0/3543433036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c070b00 con 0x7f314c15bea0 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3150a00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f314c0ced80 0x7f314c159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151400640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f314c0cdb30 0x7f314c159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.841+0000 7f3151e00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f314c15bea0 0x7f314c0718a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f31440b0040 tx=0x7f3144005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.845+0000 7f3149600640 1 -- 172.21.15.44:0/3543433036 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31440b2040 con 0x7f314c15bea0 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.845+0000 7f3149600640 1 -- 172.21.15.44:0/3543433036 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3144004040 con 0x7f314c15bea0 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.845+0000 7f3153e00640 1 -- 172.21.15.44:0/3543433036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314c070dc0 con 0x7f314c15bea0 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.845+0000 7f3149600640 1 -- 172.21.15.44:0/3543433036 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144004430 con 0x7f314c15bea0 2024-09-13T08:40:23.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.845+0000 7f3153e00640 1 -- 172.21.15.44:0/3543433036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314c1574b0 con 0x7f314c15bea0 2024-09-13T08:40:23.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.845+0000 7f3153e00640 1 -- 172.21.15.44:0/3543433036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314c0ce670 con 0x7f314c15bea0 2024-09-13T08:40:23.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.845+0000 7f3149600640 1 -- 172.21.15.44:0/3543433036 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f31440bd030 con 0x7f314c15bea0 2024-09-13T08:40:23.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.845+0000 7f3149600640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f312c073220 0x7f312c0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:23.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.845+0000 7f3149600640 1 -- 172.21.15.44:0/3543433036 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f314412e490 con 0x7f314c15bea0 2024-09-13T08:40:23.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.845+0000 7f3151400640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f312c073220 0x7f312c0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:23.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.849+0000 7f3151400640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f312c073220 0x7f312c0756e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3140000f70 tx=0x7f3140009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:23.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:23.849+0000 7f3149600640 1 -- 172.21.15.44:0/3543433036 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f31440c4060 con 0x7f314c15bea0 2024-09-13T08:40:24.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.061+0000 7f3153e00640 1 -- 172.21.15.44:0/3543433036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f314c15b830 con 0x7f314c15bea0 2024-09-13T08:40:24.065 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T08:40:24.061+0000 7f03b3c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T08:40:24.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.065+0000 7f3149600640 1 -- 172.21.15.44:0/3543433036 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f3144016790 con 0x7f314c15bea0 2024-09-13T08:40:24.067 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T08:40:24.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 -- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f312c073220 msgr2=0x7f312c0756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:24.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f312c073220 0x7f312c0756e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3140000f70 tx=0x7f3140009040 comp rx=0 tx=0).stop 2024-09-13T08:40:24.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 -- 172.21.15.44:0/3543433036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f314c15bea0 msgr2=0x7f314c0718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:24.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f314c15bea0 0x7f314c0718a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f31440b0040 tx=0x7f3144005b00 comp rx=0 tx=0).stop 2024-09-13T08:40:24.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 -- 172.21.15.44:0/3543433036 shutdown_connections 2024-09-13T08:40:24.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f312c073220 0x7f312c0756e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:24.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f314c15bea0 0x7f314c0718a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:24.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f314c0ced80 0x7f314c159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:24.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 --2- 172.21.15.44:0/3543433036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f314c0cdb30 0x7f314c159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:24.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 -- 172.21.15.44:0/3543433036 >> 172.21.15.44:0/3543433036 conn(0x7f314c0d1dc0 msgr2=0x7f314c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:24.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 -- 172.21.15.44:0/3543433036 shutdown_connections 2024-09-13T08:40:24.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.069+0000 7f3153e00640 1 -- 172.21.15.44:0/3543433036 wait complete. 2024-09-13T08:40:24.092 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T08:40:24.142 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:40:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 -- 172.21.15.44:0/429856937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000d04d0 msgr2=0x7f1e000af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 --2- 172.21.15.44:0/429856937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000d04d0 0x7f1e000af200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009c80 tx=0x7f1df801f5a0 comp rx=0 tx=0).stop 2024-09-13T08:40:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 -- 172.21.15.44:0/429856937 shutdown_connections 2024-09-13T08:40:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 --2- 172.21.15.44:0/429856937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000d04d0 0x7f1e000af200 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 --2- 172.21.15.44:0/429856937 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1e000d00f0 0x7f1e000b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 -- 172.21.15.44:0/429856937 >> 172.21.15.44:0/429856937 conn(0x7f1e000d1e90 msgr2=0x7f1e000d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 -- 172.21.15.44:0/429856937 shutdown_connections 2024-09-13T08:40:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 -- 172.21.15.44:0/429856937 wait complete. 2024-09-13T08:40:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 Processor -- start 2024-09-13T08:40:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 -- start start 2024-09-13T08:40:24.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e000d00f0 0x7f1e000b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:24.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1e000d04d0 0x7f1e00152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:24.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000d29b0 0x7f1e001f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:24.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1e000b0ed0 con 0x7f1e000d29b0 2024-09-13T08:40:24.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e000b0d50 con 0x7f1e000d00f0 2024-09-13T08:40:24.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e07400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1e000af740 con 0x7f1e000d04d0 2024-09-13T08:40:24.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e05400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000d29b0 0x7f1e001f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:24.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e05400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000d29b0 0x7f1e001f4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57360/0 (socket says 172.21.15.44:57360) 2024-09-13T08:40:24.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e05400640 1 -- 172.21.15.44:0/3391321653 learned_addr learned my addr 172.21.15.44:0/3391321653 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:24.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1dffe00640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1e000d04d0 0x7f1e00152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:24.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.245+0000 7f1e04a00640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e000d00f0 0x7f1e000b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:24.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1e05400640 1 -- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1e000d04d0 msgr2=0x7f1e00152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:24.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1e05400640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1e000d04d0 0x7f1e00152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:24.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1e05400640 1 -- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e000d00f0 msgr2=0x7f1e000b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:24.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1e05400640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e000d00f0 0x7f1e000b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1e05400640 1 -- 172.21.15.44:0/3391321653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e000cc900 con 0x7f1e000d29b0 2024-09-13T08:40:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1e05400640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000d29b0 0x7f1e001f4960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009bd0 tx=0x7f1df80af4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1dfcc00640 1 -- 172.21.15.44:0/3391321653 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8004c50 con 0x7f1e000d29b0 2024-09-13T08:40:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1dfcc00640 1 -- 172.21.15.44:0/3391321653 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1df80af8a0 con 0x7f1e000d29b0 2024-09-13T08:40:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1dfcc00640 1 -- 172.21.15.44:0/3391321653 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80afcd0 con 0x7f1e000d29b0 2024-09-13T08:40:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1e07400640 1 -- 172.21.15.44:0/3391321653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e000b62d0 con 0x7f1e000d29b0 2024-09-13T08:40:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1e07400640 1 -- 172.21.15.44:0/3391321653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e001f5470 con 0x7f1e000d29b0 2024-09-13T08:40:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1e07400640 1 -- 172.21.15.44:0/3391321653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e000af200 con 0x7f1e000d29b0 2024-09-13T08:40:24.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.249+0000 7f1dfcc00640 1 -- 172.21.15.44:0/3391321653 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f1df8004000 con 0x7f1e000d29b0 2024-09-13T08:40:24.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.253+0000 7f1dfcc00640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1de0073220 0x7f1de00756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:24.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.253+0000 7f1dfcc00640 1 -- 172.21.15.44:0/3391321653 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5911+0+0 (secure 0 0 0) 0x7f1df81332e0 con 0x7f1e000d29b0 2024-09-13T08:40:24.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.253+0000 7f1e04a00640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1de0073220 0x7f1de00756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:24.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.253+0000 7f1e04a00640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1de0073220 0x7f1de00756e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1df400abe0 tx=0x7f1df400a2d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:24.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.257+0000 7f1dfcc00640 1 -- 172.21.15.44:0/3391321653 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1df80ac050 con 0x7f1e000d29b0 2024-09-13T08:40:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:24.465+0000 7f1e07400640 1 -- 172.21.15.44:0/3391321653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1e0015bc30 con 0x7f1e000d29b0 2024-09-13T08:40:25.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.069+0000 7f1dfcc00640 1 -- 172.21.15.44:0/3391321653 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f1df8019100 con 0x7f1e000d29b0 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 -- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1de0073220 msgr2=0x7f1de00756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1de0073220 0x7f1de00756e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1df400abe0 tx=0x7f1df400a2d0 comp rx=0 tx=0).stop 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 -- 172.21.15.44:0/3391321653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000d29b0 msgr2=0x7f1e001f4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000d29b0 0x7f1e001f4960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009bd0 tx=0x7f1df80af4a0 comp rx=0 tx=0).stop 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 -- 172.21.15.44:0/3391321653 shutdown_connections 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1de0073220 0x7f1de00756e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000d29b0 0x7f1e001f4960 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1e000d04d0 0x7f1e00152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 --2- 172.21.15.44:0/3391321653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e000d00f0 0x7f1e000b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 -- 172.21.15.44:0/3391321653 >> 172.21.15.44:0/3391321653 conn(0x7f1e000d1e90 msgr2=0x7f1e000d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 -- 172.21.15.44:0/3391321653 shutdown_connections 2024-09-13T08:40:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.073+0000 7f1e07400640 1 -- 172.21.15.44:0/3391321653 wait complete. 2024-09-13T08:40:25.099 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T08:40:25.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.249+0000 7f077da00640 1 -- 172.21.15.44:0/408864430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c004810 msgr2=0x7f075c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:25.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.249+0000 7f077da00640 1 --2- 172.21.15.44:0/408864430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c004810 0x7f075c004c90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0764009b60 tx=0x7f076401d270 comp rx=0 tx=0).stop 2024-09-13T08:40:25.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.249+0000 7f077da00640 1 -- 172.21.15.44:0/408864430 shutdown_connections 2024-09-13T08:40:25.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.249+0000 7f077da00640 1 --2- 172.21.15.44:0/408864430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c004810 0x7f075c004c90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:25.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.249+0000 7f077da00640 1 -- 172.21.15.44:0/408864430 >> 172.21.15.44:0/408864430 conn(0x7f07780d1da0 msgr2=0x7f07780d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:25.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.249+0000 7f077da00640 1 -- 172.21.15.44:0/408864430 shutdown_connections 2024-09-13T08:40:25.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.249+0000 7f077da00640 1 -- 172.21.15.44:0/408864430 wait complete. 2024-09-13T08:40:25.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 Processor -- start 2024-09-13T08:40:25.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 -- start start 2024-09-13T08:40:25.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f075c004810 0x7f07780cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778154fe0 0x7f0778154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cea50 0x7f07780ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07780d0460 con 0x7f0778154fe0 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07780d02e0 con 0x7f07780cea50 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f07780d05e0 con 0x7f075c004810 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f0777400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778154fe0 0x7f0778154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f0777400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778154fe0 0x7f0778154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57370/0 (socket says 172.21.15.44:57370) 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f0777400640 1 -- 172.21.15.44:0/3979347259 learned_addr learned my addr 172.21.15.44:0/3979347259 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f0777e00640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f075c004810 0x7f07780cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077d000640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cea50 0x7f07780ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f0777400640 1 -- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f075c004810 msgr2=0x7f07780cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f0777400640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f075c004810 0x7f07780cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f0777400640 1 -- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cea50 msgr2=0x7f07780ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f0777400640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cea50 0x7f07780ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f0777400640 1 -- 172.21.15.44:0/3979347259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07780b7870 con 0x7f0778154fe0 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077d000640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cea50 0x7f07780ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f0777400640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778154fe0 0x7f0778154710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f07640af040 tx=0x7f0764005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f076fe00640 1 -- 172.21.15.44:0/3979347259 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07640bd070 con 0x7f0778154fe0 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f076fe00640 1 -- 172.21.15.44:0/3979347259 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0764004120 con 0x7f0778154fe0 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f076fe00640 1 -- 172.21.15.44:0/3979347259 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07640b1040 con 0x7f0778154fe0 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 -- 172.21.15.44:0/3979347259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07780b5120 con 0x7f0778154fe0 2024-09-13T08:40:25.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 -- 172.21.15.44:0/3979347259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07780b7b10 con 0x7f0778154fe0 2024-09-13T08:40:25.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.253+0000 7f077da00640 1 -- 172.21.15.44:0/3979347259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077815f580 con 0x7f0778154fe0 2024-09-13T08:40:25.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.257+0000 7f076fe00640 1 -- 172.21.15.44:0/3979347259 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f07640046d0 con 0x7f0778154fe0 2024-09-13T08:40:25.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.257+0000 7f076fe00640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0758073250 0x7f0758075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:25.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.257+0000 7f076fe00640 1 -- 172.21.15.44:0/3979347259 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5911+0+0 (secure 0 0 0) 0x7f076412e8d0 con 0x7f0778154fe0 2024-09-13T08:40:25.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.257+0000 7f0777e00640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0758073250 0x7f0758075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:25.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.257+0000 7f0777e00640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0758073250 0x7f0758075710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0768001ec0 tx=0x7f0768006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:25.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.261+0000 7f076fe00640 1 -- 172.21.15.44:0/3979347259 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0764016f80 con 0x7f0778154fe0 2024-09-13T08:40:25.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:25.437+0000 7f077da00640 1 -- 172.21.15.44:0/3979347259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f07780b1000 con 0x7f0778154fe0 2024-09-13T08:40:26.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.073+0000 7f076fe00640 1 -- 172.21.15.44:0/3979347259 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7f07640f8ea0 con 0x7f0778154fe0 2024-09-13T08:40:26.075 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T08:40:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 -- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0758073250 msgr2=0x7f0758075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0758073250 0x7f0758075710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0768001ec0 tx=0x7f0768006040 comp rx=0 tx=0).stop 2024-09-13T08:40:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 -- 172.21.15.44:0/3979347259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778154fe0 msgr2=0x7f0778154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778154fe0 0x7f0778154710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f07640af040 tx=0x7f0764005520 comp rx=0 tx=0).stop 2024-09-13T08:40:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 -- 172.21.15.44:0/3979347259 shutdown_connections 2024-09-13T08:40:26.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0758073250 0x7f0758075710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:26.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cea50 0x7f07780ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:26.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778154fe0 0x7f0778154710 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:26.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 --2- 172.21.15.44:0/3979347259 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f075c004810 0x7f07780cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:26.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 -- 172.21.15.44:0/3979347259 >> 172.21.15.44:0/3979347259 conn(0x7f07780d1da0 msgr2=0x7f077806e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:26.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 -- 172.21.15.44:0/3979347259 shutdown_connections 2024-09-13T08:40:26.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.077+0000 7f077da00640 1 -- 172.21.15.44:0/3979347259 wait complete. 2024-09-13T08:40:26.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T08:40:26.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 -- 172.21.15.44:0/2397481625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6e8163d00 msgr2=0x7fc6e81660f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 --2- 172.21.15.44:0/2397481625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6e8163d00 0x7fc6e81660f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8073b30 tx=0x7fc6e001cb10 comp rx=0 tx=0).stop 2024-09-13T08:40:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 -- 172.21.15.44:0/2397481625 shutdown_connections 2024-09-13T08:40:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 --2- 172.21.15.44:0/2397481625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6e8163d00 0x7fc6e81660f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 -- 172.21.15.44:0/2397481625 >> 172.21.15.44:0/2397481625 conn(0x7fc6e80cfbc0 msgr2=0x7fc6e80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 -- 172.21.15.44:0/2397481625 shutdown_connections 2024-09-13T08:40:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 -- 172.21.15.44:0/2397481625 wait complete. 2024-09-13T08:40:26.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 Processor -- start 2024-09-13T08:40:26.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 -- start start 2024-09-13T08:40:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6e80aef60 0x7fc6e81f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6e815e360 0x7fc6e81424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6e8163d00 0x7fc6e80af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81691e0 con 0x7fc6e80aef60 2024-09-13T08:40:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8169060 con 0x7fc6e815e360 2024-09-13T08:40:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6ed800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e8169360 con 0x7fc6e8163d00 2024-09-13T08:40:26.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6e6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6e80aef60 0x7fc6e81f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6e6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6e80aef60 0x7fc6e81f8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57392/0 (socket says 172.21.15.44:57392) 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6e6a00640 1 -- 172.21.15.44:0/1072424442 learned_addr learned my addr 172.21.15.44:0/1072424442 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6dfe00640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6e815e360 0x7fc6e81424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6e7400640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6e8163d00 0x7fc6e80af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6e6a00640 1 -- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6e8163d00 msgr2=0x7fc6e80af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6e6a00640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6e8163d00 0x7fc6e80af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6e6a00640 1 -- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6e815e360 msgr2=0x7fc6e81424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6e6a00640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6e815e360 0x7fc6e81424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.261+0000 7fc6e6a00640 1 -- 172.21.15.44:0/1072424442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e8165010 con 0x7fc6e80aef60 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.265+0000 7fc6e6a00640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6e80aef60 0x7fc6e81f8490 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c4b0 tx=0x7fc6d800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.265+0000 7fc6dea00640 1 -- 172.21.15.44:0/1072424442 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d80052d0 con 0x7fc6e80aef60 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.265+0000 7fc6dea00640 1 -- 172.21.15.44:0/1072424442 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6d8005470 con 0x7fc6e80aef60 2024-09-13T08:40:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.265+0000 7fc6dea00640 1 -- 172.21.15.44:0/1072424442 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8012870 con 0x7fc6e80aef60 2024-09-13T08:40:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.265+0000 7fc6ed800640 1 -- 172.21.15.44:0/1072424442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e81652b0 con 0x7fc6e80aef60 2024-09-13T08:40:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.265+0000 7fc6ed800640 1 -- 172.21.15.44:0/1072424442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e8165870 con 0x7fc6e80aef60 2024-09-13T08:40:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.265+0000 7fc6ed800640 1 -- 172.21.15.44:0/1072424442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e80ae710 con 0x7fc6e80aef60 2024-09-13T08:40:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.265+0000 7fc6dea00640 1 -- 172.21.15.44:0/1072424442 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fc6d8003600 con 0x7fc6e80aef60 2024-09-13T08:40:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.265+0000 7fc6dea00640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6c4073280 0x7fc6c4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.265+0000 7fc6dea00640 1 -- 172.21.15.44:0/1072424442 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5473+0+0 (secure 0 0 0) 0x7fc6d8094000 con 0x7fc6e80aef60 2024-09-13T08:40:26.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.269+0000 7fc6dfe00640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6c4073280 0x7fc6c4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.269+0000 7fc6dfe00640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6c4073280 0x7fc6c4075740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8164e00 tx=0x7fc6d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:26.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.269+0000 7fc6dea00640 1 -- 172.21.15.44:0/1072424442 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc6d805ecd0 con 0x7fc6e80aef60 2024-09-13T08:40:26.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:26.445+0000 7fc6ed800640 1 -- 172.21.15.44:0/1072424442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc6e8156620 con 0x7fc6e80aef60 2024-09-13T08:40:27.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.077+0000 7fc6dea00640 1 -- 172.21.15.44:0/1072424442 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7fc6d805e5d0 con 0x7fc6e80aef60 2024-09-13T08:40:27.081 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T08:40:27.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 -- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6c4073280 msgr2=0x7fc6c4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6c4073280 0x7fc6c4075740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8164e00 tx=0x7fc6d0009040 comp rx=0 tx=0).stop 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 -- 172.21.15.44:0/1072424442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6e80aef60 msgr2=0x7fc6e81f8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6e80aef60 0x7fc6e81f8490 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c4b0 tx=0x7fc6d800c970 comp rx=0 tx=0).stop 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 -- 172.21.15.44:0/1072424442 shutdown_connections 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6c4073280 0x7fc6c4075740 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6e8163d00 0x7fc6e80af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6e815e360 0x7fc6e81424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 --2- 172.21.15.44:0/1072424442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6e80aef60 0x7fc6e81f8490 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 -- 172.21.15.44:0/1072424442 >> 172.21.15.44:0/1072424442 conn(0x7fc6e80cfbc0 msgr2=0x7fc6e80b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 -- 172.21.15.44:0/1072424442 shutdown_connections 2024-09-13T08:40:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.081+0000 7fc6ed800640 1 -- 172.21.15.44:0/1072424442 wait complete. 2024-09-13T08:40:27.122 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-13T08:40:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 -- 172.21.15.44:0/3464934893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f859415f130 msgr2=0x7f859415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 --2- 172.21.15.44:0/3464934893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f859415f130 0x7f859415b1c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f85940d1a40 tx=0x7f858001c850 comp rx=0 tx=0).stop 2024-09-13T08:40:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 -- 172.21.15.44:0/3464934893 shutdown_connections 2024-09-13T08:40:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 --2- 172.21.15.44:0/3464934893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f859415f130 0x7f859415b1c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 --2- 172.21.15.44:0/3464934893 >> v2:172.21.15.44:3300/0 conn(0x7f85941518b0 0x7f8594151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 -- 172.21.15.44:0/3464934893 >> 172.21.15.44:0/3464934893 conn(0x7f85940d0cf0 msgr2=0x7f85940d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 -- 172.21.15.44:0/3464934893 shutdown_connections 2024-09-13T08:40:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 -- 172.21.15.44:0/3464934893 wait complete. 2024-09-13T08:40:27.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 Processor -- start 2024-09-13T08:40:27.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 -- start start 2024-09-13T08:40:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85941518b0 0x7f85941486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f859415b700 0x7f8594168fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f859415f130 0x7f85941694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8594071340 con 0x7f859415f130 2024-09-13T08:40:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f85940711c0 con 0x7f859415b700 2024-09-13T08:40:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.385+0000 7f8598a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f85940cc460 con 0x7f85941518b0 2024-09-13T08:40:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f859415b700 0x7f8594168fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85941518b0 0x7f85941486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f859415b700 0x7f8594168fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:33546/0 (socket says 172.21.15.44:33546) 2024-09-13T08:40:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85941518b0 0x7f85941486d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:49784/0 (socket says 172.21.15.44:49784) 2024-09-13T08:40:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591400640 1 -- 172.21.15.44:0/2025639752 learned_addr learned my addr 172.21.15.44:0/2025639752 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:27.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591e00640 1 -- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f859415b700 msgr2=0x7f8594168fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:27.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591e00640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f859415b700 0x7f8594168fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591e00640 1 -- 172.21.15.44:0/2025639752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f859415f130 msgr2=0x7f85941694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:27.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591e00640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f859415f130 0x7f85941694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591e00640 1 -- 172.21.15.44:0/2025639752 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859415d780 con 0x7f85941518b0 2024-09-13T08:40:27.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8591e00640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85941518b0 0x7f85941486d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f858400d750 tx=0x7f858400dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:27.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f858a000640 1 -- 172.21.15.44:0/2025639752 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584017070 con 0x7f85941518b0 2024-09-13T08:40:27.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8598a00640 1 -- 172.21.15.44:0/2025639752 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85940c5da0 con 0x7f85941518b0 2024-09-13T08:40:27.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f858a000640 1 -- 172.21.15.44:0/2025639752 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8584009cc0 con 0x7f85941518b0 2024-09-13T08:40:27.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8598a00640 1 -- 172.21.15.44:0/2025639752 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85940c6360 con 0x7f85941518b0 2024-09-13T08:40:27.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f858a000640 1 -- 172.21.15.44:0/2025639752 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858400de60 con 0x7f85941518b0 2024-09-13T08:40:27.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.389+0000 7f8598a00640 1 -- 172.21.15.44:0/2025639752 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856c0052a0 con 0x7f85941518b0 2024-09-13T08:40:27.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.393+0000 7f858a000640 1 -- 172.21.15.44:0/2025639752 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f858401a070 con 0x7f85941518b0 2024-09-13T08:40:27.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.393+0000 7f858a000640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8574073250 0x7f8574075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:27.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.393+0000 7f858a000640 1 -- 172.21.15.44:0/2025639752 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f85840941c0 con 0x7f85941518b0 2024-09-13T08:40:27.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.393+0000 7f8591400640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8574073250 0x7f8574075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:27.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.393+0000 7f8591400640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8574073250 0x7f8574075710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8580009230 tx=0x7f858001b090 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:27.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.397+0000 7f858a000640 1 -- 172.21.15.44:0/2025639752 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f858405e9d0 con 0x7f85941518b0 2024-09-13T08:40:27.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.633+0000 7f8598a00640 1 -- 172.21.15.44:0/2025639752 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f856c005860 con 0x7f85941518b0 2024-09-13T08:40:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.645+0000 7f858a000640 1 -- 172.21.15.44:0/2025639752 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v30) ==== 71+0+0 (secure 0 0 0) 0x7f8584063400 con 0x7f85941518b0 2024-09-13T08:40:27.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 -- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8574073250 msgr2=0x7f8574075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:27.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8574073250 0x7f8574075710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8580009230 tx=0x7f858001b090 comp rx=0 tx=0).stop 2024-09-13T08:40:27.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 -- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85941518b0 msgr2=0x7f85941486d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:27.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85941518b0 0x7f85941486d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f858400d750 tx=0x7f858400dc10 comp rx=0 tx=0).stop 2024-09-13T08:40:27.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 -- 172.21.15.44:0/2025639752 shutdown_connections 2024-09-13T08:40:27.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8574073250 0x7f8574075710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f859415f130 0x7f85941694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f859415b700 0x7f8594168fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 --2- 172.21.15.44:0/2025639752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85941518b0 0x7f85941486d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 -- 172.21.15.44:0/2025639752 >> 172.21.15.44:0/2025639752 conn(0x7f85940d0cf0 msgr2=0x7f85940cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 -- 172.21.15.44:0/2025639752 shutdown_connections 2024-09-13T08:40:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.649+0000 7f8598a00640 1 -- 172.21.15.44:0/2025639752 wait complete. 2024-09-13T08:40:27.670 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-13T08:40:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 -- 172.21.15.44:0/1346585776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23bc004810 msgr2=0x7f23bc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 --2- 172.21.15.44:0/1346585776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23bc004810 0x7f23bc004c90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c4a0 tx=0x7f23c401cec0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 -- 172.21.15.44:0/1346585776 shutdown_connections 2024-09-13T08:40:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 --2- 172.21.15.44:0/1346585776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23bc004810 0x7f23bc004c90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 -- 172.21.15.44:0/1346585776 >> 172.21.15.44:0/1346585776 conn(0x7f23d80d1da0 msgr2=0x7f23d80d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:27.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 -- 172.21.15.44:0/1346585776 shutdown_connections 2024-09-13T08:40:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 -- 172.21.15.44:0/1346585776 wait complete. 2024-09-13T08:40:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 Processor -- start 2024-09-13T08:40:27.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 -- start start 2024-09-13T08:40:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23bc004810 0x7f23d81ea810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23d8157e90 0x7f23d80c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d8157240 0x7f23d81576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23d80ce870 con 0x7f23d8157240 2024-09-13T08:40:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23d80ce6f0 con 0x7f23bc004810 2024-09-13T08:40:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23df200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f23d80ce9f0 con 0x7f23d8157e90 2024-09-13T08:40:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23dd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d8157240 0x7f23d81576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23d7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23bc004810 0x7f23d81ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23dd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d8157240 0x7f23d81576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57422/0 (socket says 172.21.15.44:57422) 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23dd200640 1 -- 172.21.15.44:0/3492921238 learned_addr learned my addr 172.21.15.44:0/3492921238 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23d7400640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23d8157e90 0x7f23d80c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23dd200640 1 -- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23d8157e90 msgr2=0x7f23d80c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23dd200640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23d8157e90 0x7f23d80c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23dd200640 1 -- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23bc004810 msgr2=0x7f23d81ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23dd200640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23bc004810 0x7f23d81ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.817+0000 7f23dd200640 1 -- 172.21.15.44:0/3492921238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d81f8a10 con 0x7f23d8157240 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23dd200640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d8157240 0x7f23d81576c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f23d8074290 tx=0x7f23d000e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23cfe00640 1 -- 172.21.15.44:0/3492921238 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0003d90 con 0x7f23d8157240 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23cfe00640 1 -- 172.21.15.44:0/3492921238 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23d0006070 con 0x7f23d8157240 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23cfe00640 1 -- 172.21.15.44:0/3492921238 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d00064c0 con 0x7f23d8157240 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23df200640 1 -- 172.21.15.44:0/3492921238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d80cb970 con 0x7f23d8157240 2024-09-13T08:40:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23df200640 1 -- 172.21.15.44:0/3492921238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d8157c00 con 0x7f23d8157240 2024-09-13T08:40:27.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23df200640 1 -- 172.21.15.44:0/3492921238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d80cb790 con 0x7f23d8157240 2024-09-13T08:40:27.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23cfe00640 1 -- 172.21.15.44:0/3492921238 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f23d000d070 con 0x7f23d8157240 2024-09-13T08:40:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23cfe00640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23b8073220 0x7f23b80756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23cfe00640 1 -- 172.21.15.44:0/3492921238 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f23d0097790 con 0x7f23d8157240 2024-09-13T08:40:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.821+0000 7f23d7e00640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23b8073220 0x7f23b80756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:27.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.825+0000 7f23d7e00640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23b8073220 0x7f23b80756e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23d8052940 tx=0x7f23c80091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:27.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:27.825+0000 7f23cfe00640 1 -- 172.21.15.44:0/3492921238 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f23d80cb790 con 0x7f23d8157240 2024-09-13T08:40:28.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.065+0000 7f23df200640 1 -- 172.21.15.44:0/3492921238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f23d80c8f30 con 0x7f23d8157240 2024-09-13T08:40:28.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23cfe00640 1 -- 172.21.15.44:0/3492921238 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v146) ==== 184+0+0 (secure 0 0 0) 0x7f23d80c8f30 con 0x7f23d8157240 2024-09-13T08:40:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23df200640 1 -- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23b8073220 msgr2=0x7f23b80756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23df200640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23b8073220 0x7f23b80756e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23d8052940 tx=0x7f23c80091e0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23df200640 1 -- 172.21.15.44:0/3492921238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d8157240 msgr2=0x7f23d81576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23df200640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d8157240 0x7f23d81576c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f23d8074290 tx=0x7f23d000e6b0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23df200640 1 -- 172.21.15.44:0/3492921238 shutdown_connections 2024-09-13T08:40:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23df200640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23b8073220 0x7f23b80756e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23df200640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d8157240 0x7f23d81576c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23df200640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23d8157e90 0x7f23d80c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23df200640 1 --2- 172.21.15.44:0/3492921238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23bc004810 0x7f23d81ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.081+0000 7f23df200640 1 -- 172.21.15.44:0/3492921238 >> 172.21.15.44:0/3492921238 conn(0x7f23d80d1da0 msgr2=0x7f23d8071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.085+0000 7f23df200640 1 -- 172.21.15.44:0/3492921238 shutdown_connections 2024-09-13T08:40:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.085+0000 7f23df200640 1 -- 172.21.15.44:0/3492921238 wait complete. 2024-09-13T08:40:28.106 INFO:tasks.cephfs_test_runner:Initiate cloning where clone subvolume and source subvolume are located ... ok 2024-09-13T08:40:28.106 INFO:tasks.cephfs_test_runner:Starting test: test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-13T08:40:28.106 INFO:tasks.cephfs_test_runner:test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-13T08:40:28.107 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:40:28.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.249+0000 7f8e86400640 1 -- 172.21.15.44:0/750454677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e8015f300 msgr2=0x7f8e800cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.249+0000 7f8e86400640 1 --2- 172.21.15.44:0/750454677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e8015f300 0x7f8e800cfca0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009870 tx=0x7f8e7001f050 comp rx=0 tx=0).stop 2024-09-13T08:40:28.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.249+0000 7f8e86400640 1 -- 172.21.15.44:0/750454677 shutdown_connections 2024-09-13T08:40:28.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.249+0000 7f8e86400640 1 --2- 172.21.15.44:0/750454677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e8015f300 0x7f8e800cfca0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.249+0000 7f8e86400640 1 -- 172.21.15.44:0/750454677 >> 172.21.15.44:0/750454677 conn(0x7f8e800d1e30 msgr2=0x7f8e800d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:28.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.249+0000 7f8e86400640 1 -- 172.21.15.44:0/750454677 shutdown_connections 2024-09-13T08:40:28.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.249+0000 7f8e86400640 1 -- 172.21.15.44:0/750454677 wait complete. 2024-09-13T08:40:28.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e86400640 1 Processor -- start 2024-09-13T08:40:28.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e86400640 1 -- start start 2024-09-13T08:40:28.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e86400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e800b2d60 0x7f8e801f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e86400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e8015f300 0x7f8e8015e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e86400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e8015f6e0 0x7f8e801f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e86400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8e800b01b0 con 0x7f8e8015f6e0 2024-09-13T08:40:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e86400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8e800b0030 con 0x7f8e800b2d60 2024-09-13T08:40:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e86400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8e800b0330 con 0x7f8e8015f300 2024-09-13T08:40:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e7fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e8015f6e0 0x7f8e801f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e7fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e8015f6e0 0x7f8e801f8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57434/0 (socket says 172.21.15.44:57434) 2024-09-13T08:40:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e7fe00640 1 -- 172.21.15.44:0/396179388 learned_addr learned my addr 172.21.15.44:0/396179388 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e77e00640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e8015f300 0x7f8e8015e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e7f400640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e800b2d60 0x7f8e801f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e7fe00640 1 -- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e8015f300 msgr2=0x7f8e8015e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e7fe00640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e8015f300 0x7f8e8015e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e7fe00640 1 -- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e800b2d60 msgr2=0x7f8e801f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e7fe00640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e800b2d60 0x7f8e801f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e7fe00640 1 -- 172.21.15.44:0/396179388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e800ccca0 con 0x7f8e8015f6e0 2024-09-13T08:40:28.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e7fe00640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e8015f6e0 0x7f8e801f8490 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800cbc0 tx=0x7f8e7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:28.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e77400640 1 -- 172.21.15.44:0/396179388 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78016020 con 0x7f8e8015f6e0 2024-09-13T08:40:28.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e77400640 1 -- 172.21.15.44:0/396179388 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e7800ab00 con 0x7f8e8015f6e0 2024-09-13T08:40:28.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e86400640 1 -- 172.21.15.44:0/396179388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e800ccf80 con 0x7f8e8015f6e0 2024-09-13T08:40:28.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.253+0000 7f8e77400640 1 -- 172.21.15.44:0/396179388 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78012420 con 0x7f8e8015f6e0 2024-09-13T08:40:28.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.257+0000 7f8e86400640 1 -- 172.21.15.44:0/396179388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e801f4e20 con 0x7f8e8015f6e0 2024-09-13T08:40:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.257+0000 7f8e86400640 1 -- 172.21.15.44:0/396179388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e800b0750 con 0x7f8e8015f6e0 2024-09-13T08:40:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.257+0000 7f8e77400640 1 -- 172.21.15.44:0/396179388 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f8e78019020 con 0x7f8e8015f6e0 2024-09-13T08:40:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.257+0000 7f8e77400640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e5c073190 0x7f8e5c075650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.257+0000 7f8e77400640 1 -- 172.21.15.44:0/396179388 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f8e780943d0 con 0x7f8e8015f6e0 2024-09-13T08:40:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.257+0000 7f8e7f400640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e5c073190 0x7f8e5c075650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.261+0000 7f8e7f400640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e5c073190 0x7f8e5c075650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8e700b2000 tx=0x7f8e7001d890 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:28.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.261+0000 7f8e77400640 1 -- 172.21.15.44:0/396179388 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8e78020020 con 0x7f8e8015f6e0 2024-09-13T08:40:28.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.441+0000 7f8e86400640 1 -- 172.21.15.44:0/396179388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e801f4ff0 con 0x7f8e8015f6e0 2024-09-13T08:40:28.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.441+0000 7f8e77400640 1 -- 172.21.15.44:0/396179388 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13044 (secure 0 0 0) 0x7f8e7805ed90 con 0x7f8e8015f6e0 2024-09-13T08:40:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:28.446 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":38,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:40:27.075848+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/3820157165":"2024-09-14T08:40:24.065366+0000","172.21.15.44:6833/3820157165":"2024-09-14T08:40:24.065366+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:40:28.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 -- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e5c073190 msgr2=0x7f8e5c075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e5c073190 0x7f8e5c075650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8e700b2000 tx=0x7f8e7001d890 comp rx=0 tx=0).stop 2024-09-13T08:40:28.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 -- 172.21.15.44:0/396179388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e8015f6e0 msgr2=0x7f8e801f8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e8015f6e0 0x7f8e801f8490 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800cbc0 tx=0x7f8e7800ccb0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 -- 172.21.15.44:0/396179388 shutdown_connections 2024-09-13T08:40:28.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e5c073190 0x7f8e5c075650 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e8015f6e0 0x7f8e801f8490 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e8015f300 0x7f8e8015e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 --2- 172.21.15.44:0/396179388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e800b2d60 0x7f8e801f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 -- 172.21.15.44:0/396179388 >> 172.21.15.44:0/396179388 conn(0x7f8e800d1e30 msgr2=0x7f8e801ea3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:28.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 -- 172.21.15.44:0/396179388 shutdown_connections 2024-09-13T08:40:28.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.445+0000 7f8e86400640 1 -- 172.21.15.44:0/396179388 wait complete. 2024-09-13T08:40:28.468 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T08:37:24.563344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T08:40:28.468 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T08:40:28.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.617+0000 7f77a4e00640 1 -- 172.21.15.44:0/574456655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77840051d0 msgr2=0x7f778400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.617+0000 7f77a4e00640 1 --2- 172.21.15.44:0/574456655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77840051d0 0x7f778400b9d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7798001dc0 tx=0x7f779801cb30 comp rx=0 tx=0).stop 2024-09-13T08:40:28.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.617+0000 7f77a4e00640 1 -- 172.21.15.44:0/574456655 shutdown_connections 2024-09-13T08:40:28.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.617+0000 7f77a4e00640 1 --2- 172.21.15.44:0/574456655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77840051d0 0x7f778400b9d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.617+0000 7f77a4e00640 1 -- 172.21.15.44:0/574456655 >> 172.21.15.44:0/574456655 conn(0x7f77a00d1dd0 msgr2=0x7f77a00d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:28.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.617+0000 7f77a4e00640 1 -- 172.21.15.44:0/574456655 shutdown_connections 2024-09-13T08:40:28.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.617+0000 7f77a4e00640 1 -- 172.21.15.44:0/574456655 wait complete. 2024-09-13T08:40:28.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.617+0000 7f77a4e00640 1 Processor -- start 2024-09-13T08:40:28.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.617+0000 7f77a4e00640 1 -- start start 2024-09-13T08:40:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f77a4e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77840051d0 0x7f77a015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f77a4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a015d290 0x7f77a015f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f77a4e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77a00c58d0 0x7f77a00c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f77a4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77a00b1220 con 0x7f77a015d290 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f77a4e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77a00b0ee0 con 0x7f77840051d0 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f77a4e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f77a00b13a0 con 0x7f77a00c58d0 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7797e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a015d290 0x7f77a015f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7797e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a015d290 0x7f77a015f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57452/0 (socket says 172.21.15.44:57452) 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7797e00640 1 -- 172.21.15.44:0/3337782751 learned_addr learned my addr 172.21.15.44:0/3337782751 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f779de00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77840051d0 0x7f77a015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f779e800640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77a00c58d0 0x7f77a00c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7797e00640 1 -- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77a00c58d0 msgr2=0x7f77a00c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7797e00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77a00c58d0 0x7f77a00c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7797e00640 1 -- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77840051d0 msgr2=0x7f77a015aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7797e00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77840051d0 0x7f77a015aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7797e00640 1 -- 172.21.15.44:0/3337782751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a006e950 con 0x7f77a015d290 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f779de00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77840051d0 0x7f77a015aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f779e800640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77a00c58d0 0x7f77a00c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7797e00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a015d290 0x7f77a015f540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f778800e590 tx=0x7f7788001870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7796000640 1 -- 172.21.15.44:0/3337782751 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788015070 con 0x7f77a015d290 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7796000640 1 -- 172.21.15.44:0/3337782751 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7788010980 con 0x7f77a015d290 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7796000640 1 -- 172.21.15.44:0/3337782751 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788010dd0 con 0x7f77a015d290 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f77a4e00640 1 -- 172.21.15.44:0/3337782751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a006ec30 con 0x7f77a015d290 2024-09-13T08:40:28.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f77a4e00640 1 -- 172.21.15.44:0/3337782751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a006f1f0 con 0x7f77a015d290 2024-09-13T08:40:28.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f77a4e00640 1 -- 172.21.15.44:0/3337782751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a00d0690 con 0x7f77a015d290 2024-09-13T08:40:28.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.621+0000 7f7796000640 1 -- 172.21.15.44:0/3337782751 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f7788018070 con 0x7f77a015d290 2024-09-13T08:40:28.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.625+0000 7f7796000640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f777c073220 0x7f777c0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.625+0000 7f7796000640 1 -- 172.21.15.44:0/3337782751 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f7788094620 con 0x7f77a015d290 2024-09-13T08:40:28.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.625+0000 7f779de00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f777c073220 0x7f777c0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.625+0000 7f779de00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f777c073220 0x7f777c0756e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f77a00719d0 tx=0x7f77900091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:28.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.629+0000 7f7796000640 1 -- 172.21.15.44:0/3337782751 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f778805f550 con 0x7f77a015d290 2024-09-13T08:40:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.801+0000 7f77a4e00640 1 -- 172.21.15.44:0/3337782751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77a006f3c0 con 0x7f77a015d290 2024-09-13T08:40:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f7796000640 1 -- 172.21.15.44:0/3337782751 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f778805ee50 con 0x7f77a015d290 2024-09-13T08:40:28.807 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T08:40:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 -- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f777c073220 msgr2=0x7f777c0756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f777c073220 0x7f777c0756e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f77a00719d0 tx=0x7f77900091e0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 -- 172.21.15.44:0/3337782751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a015d290 msgr2=0x7f77a015f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a015d290 0x7f77a015f540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f778800e590 tx=0x7f7788001870 comp rx=0 tx=0).stop 2024-09-13T08:40:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 -- 172.21.15.44:0/3337782751 shutdown_connections 2024-09-13T08:40:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f777c073220 0x7f777c0756e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77a00c58d0 0x7f77a00c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a015d290 0x7f77a015f540 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 --2- 172.21.15.44:0/3337782751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77840051d0 0x7f77a015aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 -- 172.21.15.44:0/3337782751 >> 172.21.15.44:0/3337782751 conn(0x7f77a00d1dd0 msgr2=0x7f77a015c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 -- 172.21.15.44:0/3337782751 shutdown_connections 2024-09-13T08:40:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.805+0000 7f77a4e00640 1 -- 172.21.15.44:0/3337782751 wait complete. 2024-09-13T08:40:28.829 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-13T08:40:28.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 -- 172.21.15.44:0/4231556250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e40744c0 msgr2=0x7f53e4074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 --2- 172.21.15.44:0/4231556250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e40744c0 0x7f53e4074930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f53e40aea80 tx=0x7f53dc01cd30 comp rx=0 tx=0).stop 2024-09-13T08:40:28.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 -- 172.21.15.44:0/4231556250 shutdown_connections 2024-09-13T08:40:28.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 --2- 172.21.15.44:0/4231556250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e40744c0 0x7f53e4074930 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 --2- 172.21.15.44:0/4231556250 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e4072c90 0x7f53e415c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 --2- 172.21.15.44:0/4231556250 >> v2:172.21.15.129:3300/0 conn(0x7f53e415c5e0 0x7f53e406ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 -- 172.21.15.44:0/4231556250 >> 172.21.15.44:0/4231556250 conn(0x7f53e40b1c20 msgr2=0x7f53e40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:28.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 -- 172.21.15.44:0/4231556250 shutdown_connections 2024-09-13T08:40:28.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 -- 172.21.15.44:0/4231556250 wait complete. 2024-09-13T08:40:28.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 Processor -- start 2024-09-13T08:40:28.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.973+0000 7f53ea800640 1 -- start start 2024-09-13T08:40:28.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53ea800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e4072c90 0x7f53e415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53ea800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e40744c0 0x7f53e40b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53ea800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e415c5e0 0x7f53e414b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53ea800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f53e4152260 con 0x7f53e415c5e0 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53ea800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f53e40af8f0 con 0x7f53e40744c0 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53ea800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f53e41523e0 con 0x7f53e4072c90 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53e3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e415c5e0 0x7f53e414b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53e3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e415c5e0 0x7f53e414b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57472/0 (socket says 172.21.15.44:57472) 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53e3e00640 1 -- 172.21.15.44:0/1385632229 learned_addr learned my addr 172.21.15.44:0/1385632229 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53daa00640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e40744c0 0x7f53e40b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53e3400640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e4072c90 0x7f53e415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53e3e00640 1 -- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e4072c90 msgr2=0x7f53e415ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53e3e00640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e4072c90 0x7f53e415ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53e3e00640 1 -- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e40744c0 msgr2=0x7f53e40b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:28.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53e3e00640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e40744c0 0x7f53e40b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:28.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53e3e00640 1 -- 172.21.15.44:0/1385632229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e41faa30 con 0x7f53e415c5e0 2024-09-13T08:40:28.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53daa00640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e40744c0 0x7f53e40b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:28.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53e3e00640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e415c5e0 0x7f53e414b720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f53dc01b3e0 tx=0x7f53dc003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:28.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53dbe00640 1 -- 172.21.15.44:0/1385632229 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0bb070 con 0x7f53e415c5e0 2024-09-13T08:40:28.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53dbe00640 1 -- 172.21.15.44:0/1385632229 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53dc004e00 con 0x7f53e415c5e0 2024-09-13T08:40:28.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53dbe00640 1 -- 172.21.15.44:0/1385632229 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0b2040 con 0x7f53e415c5e0 2024-09-13T08:40:28.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53ea800640 1 -- 172.21.15.44:0/1385632229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e41facf0 con 0x7f53e415c5e0 2024-09-13T08:40:28.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53ea800640 1 -- 172.21.15.44:0/1385632229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e41fb2b0 con 0x7f53e415c5e0 2024-09-13T08:40:28.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53ea800640 1 -- 172.21.15.44:0/1385632229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e4074930 con 0x7f53e415c5e0 2024-09-13T08:40:28.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.977+0000 7f53dbe00640 1 -- 172.21.15.44:0/1385632229 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f53dc0be070 con 0x7f53e415c5e0 2024-09-13T08:40:28.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.981+0000 7f53dbe00640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c0073220 0x7f53c00756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:28.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.981+0000 7f53dbe00640 1 -- 172.21.15.44:0/1385632229 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f53dc12e0d0 con 0x7f53e415c5e0 2024-09-13T08:40:28.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.981+0000 7f53e3400640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c0073220 0x7f53c00756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:28.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.981+0000 7f53e3400640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c0073220 0x7f53c00756e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f53d40058d0 tx=0x7f53d4005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:28.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:28.985+0000 7f53dbe00640 1 -- 172.21.15.44:0/1385632229 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f53dc016950 con 0x7f53e415c5e0 2024-09-13T08:40:29.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:29.225+0000 7f53ea800640 1 -- 172.21.15.44:0/1385632229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7f53e406a330 con 0x7f53e415c5e0 2024-09-13T08:40:30.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53dbe00640 1 -- 172.21.15.44:0/1385632229 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v148) ==== 169+0+0 (secure 0 0 0) 0x7f53dc0f9090 con 0x7f53e415c5e0 2024-09-13T08:40:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 -- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c0073220 msgr2=0x7f53c00756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c0073220 0x7f53c00756e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f53d40058d0 tx=0x7f53d4005860 comp rx=0 tx=0).stop 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 -- 172.21.15.44:0/1385632229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e415c5e0 msgr2=0x7f53e414b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e415c5e0 0x7f53e414b720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f53dc01b3e0 tx=0x7f53dc003f60 comp rx=0 tx=0).stop 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 -- 172.21.15.44:0/1385632229 shutdown_connections 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c0073220 0x7f53c00756e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e415c5e0 0x7f53e414b720 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e40744c0 0x7f53e40b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 --2- 172.21.15.44:0/1385632229 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e4072c90 0x7f53e415ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 -- 172.21.15.44:0/1385632229 >> 172.21.15.44:0/1385632229 conn(0x7f53e40b1c20 msgr2=0x7f53e409b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 -- 172.21.15.44:0/1385632229 shutdown_connections 2024-09-13T08:40:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.089+0000 7f53ea800640 1 -- 172.21.15.44:0/1385632229 wait complete. 2024-09-13T08:40:30.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-13T08:40:30.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.257+0000 7fa85d000640 1 -- 172.21.15.44:0/515469354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa85815c130 msgr2=0x7fa8580cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.257+0000 7fa85d000640 1 --2- 172.21.15.44:0/515469354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa85815c130 0x7fa8580cce80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa85806d870 tx=0x7fa85001cc60 comp rx=0 tx=0).stop 2024-09-13T08:40:30.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.257+0000 7fa85d000640 1 -- 172.21.15.44:0/515469354 shutdown_connections 2024-09-13T08:40:30.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.257+0000 7fa85d000640 1 --2- 172.21.15.44:0/515469354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa85815c130 0x7fa8580cce80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.257+0000 7fa85d000640 1 --2- 172.21.15.44:0/515469354 >> v2:172.21.15.129:3300/0 conn(0x7fa8580cecf0 0x7fa8580cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.257+0000 7fa85d000640 1 -- 172.21.15.44:0/515469354 >> 172.21.15.44:0/515469354 conn(0x7fa8580d1ea0 msgr2=0x7fa8580d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:30.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.257+0000 7fa85d000640 1 -- 172.21.15.44:0/515469354 shutdown_connections 2024-09-13T08:40:30.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.257+0000 7fa85d000640 1 -- 172.21.15.44:0/515469354 wait complete. 2024-09-13T08:40:30.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa85d000640 1 Processor -- start 2024-09-13T08:40:30.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa85d000640 1 -- start start 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa85d000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8580cecf0 0x7fa8581fc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa85d000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa85815bd50 0x7fa85806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa85d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa85815c130 0x7fa85806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa85d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa858169540 con 0x7fa85815c130 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa85d000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581693c0 con 0x7fa85815bd50 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa85d000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa8581696c0 con 0x7fa8580cecf0 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa856800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa85815c130 0x7fa85806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa856800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa85815c130 0x7fa85806dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57482/0 (socket says 172.21.15.44:57482) 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa856800640 1 -- 172.21.15.44:0/4293200127 learned_addr learned my addr 172.21.15.44:0/4293200127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa84fe00640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa85815bd50 0x7fa85806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa855e00640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8580cecf0 0x7fa8581fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:30.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa856800640 1 -- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8580cecf0 msgr2=0x7fa8581fc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa856800640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8580cecf0 0x7fa8581fc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa856800640 1 -- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa85815bd50 msgr2=0x7fa85806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa856800640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa85815bd50 0x7fa85806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa856800640 1 -- 172.21.15.44:0/4293200127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8580b4cd0 con 0x7fa85815c130 2024-09-13T08:40:30.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa856800640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa85815c130 0x7fa85806dd30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa850019870 tx=0x7fa8500092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:30.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa84e000640 1 -- 172.21.15.44:0/4293200127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500bb070 con 0x7fa85815c130 2024-09-13T08:40:30.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa84e000640 1 -- 172.21.15.44:0/4293200127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa850004d40 con 0x7fa85815c130 2024-09-13T08:40:30.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa84e000640 1 -- 172.21.15.44:0/4293200127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500b2040 con 0x7fa85815c130 2024-09-13T08:40:30.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa85d000640 1 -- 172.21.15.44:0/4293200127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8580b4fb0 con 0x7fa85815c130 2024-09-13T08:40:30.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.261+0000 7fa85d000640 1 -- 172.21.15.44:0/4293200127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8580b6c70 con 0x7fa85815c130 2024-09-13T08:40:30.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.265+0000 7fa85d000640 1 -- 172.21.15.44:0/4293200127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8580b6e70 con 0x7fa85815c130 2024-09-13T08:40:30.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.265+0000 7fa84e000640 1 -- 172.21.15.44:0/4293200127 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fa8500be070 con 0x7fa85815c130 2024-09-13T08:40:30.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.265+0000 7fa84e000640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa834073220 0x7fa8340756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:30.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.265+0000 7fa84e000640 1 -- 172.21.15.44:0/4293200127 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fa85012ddc0 con 0x7fa85815c130 2024-09-13T08:40:30.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.265+0000 7fa855e00640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa834073220 0x7fa8340756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:30.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.265+0000 7fa855e00640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa834073220 0x7fa8340756e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa848007fd0 tx=0x7fa848009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:30.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.269+0000 7fa84e000640 1 -- 172.21.15.44:0/4293200127 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa8580b6e70 con 0x7fa85815c130 2024-09-13T08:40:30.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.445+0000 7fa85d000640 1 -- 172.21.15.44:0/4293200127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa8580ce5e0 con 0x7fa85815c130 2024-09-13T08:40:30.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.445+0000 7fa84e000640 1 -- 172.21.15.44:0/4293200127 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) ==== 59+0+1998 (secure 0 0 0) 0x7fa850016970 con 0x7fa85815c130 2024-09-13T08:40:30.449 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-13T08:40:30.449 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-09-13T08:40:30.449 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4094.620852 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3964.014083 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3978.263337 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4002.298162 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4667.000623 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4664.046790 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6403.016405 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4696.908546 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 2/2 2024-09-13T08:40:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-09-13T08:40:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-09-13T08:40:30.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 -- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa834073220 msgr2=0x7fa8340756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa834073220 0x7fa8340756e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa848007fd0 tx=0x7fa848009040 comp rx=0 tx=0).stop 2024-09-13T08:40:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 -- 172.21.15.44:0/4293200127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa85815c130 msgr2=0x7fa85806dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa85815c130 0x7fa85806dd30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa850019870 tx=0x7fa8500092c0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 -- 172.21.15.44:0/4293200127 shutdown_connections 2024-09-13T08:40:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa834073220 0x7fa8340756e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa85815c130 0x7fa85806dd30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa85815bd50 0x7fa85806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 --2- 172.21.15.44:0/4293200127 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8580cecf0 0x7fa8581fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 -- 172.21.15.44:0/4293200127 >> 172.21.15.44:0/4293200127 conn(0x7fa8580d1ea0 msgr2=0x7fa85815b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 -- 172.21.15.44:0/4293200127 shutdown_connections 2024-09-13T08:40:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.449+0000 7fa85d000640 1 -- 172.21.15.44:0/4293200127 wait complete. 2024-09-13T08:40:30.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:40:30.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.617+0000 7f5567400640 1 -- 172.21.15.44:0/2685953215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55600afb10 msgr2=0x7f55600affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.617+0000 7f5567400640 1 --2- 172.21.15.44:0/2685953215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55600afb10 0x7f55600affc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5558009c80 tx=0x7f555801cda0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.617+0000 7f5567400640 1 -- 172.21.15.44:0/2685953215 shutdown_connections 2024-09-13T08:40:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.617+0000 7f5567400640 1 --2- 172.21.15.44:0/2685953215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55600afb10 0x7f55600affc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.617+0000 7f5567400640 1 --2- 172.21.15.44:0/2685953215 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55601605d0 0x7f55600d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.617+0000 7f5567400640 1 -- 172.21.15.44:0/2685953215 >> 172.21.15.44:0/2685953215 conn(0x7f55600d1dd0 msgr2=0x7f55600d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.617+0000 7f5567400640 1 -- 172.21.15.44:0/2685953215 shutdown_connections 2024-09-13T08:40:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.617+0000 7f5567400640 1 -- 172.21.15.44:0/2685953215 wait complete. 2024-09-13T08:40:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5567400640 1 Processor -- start 2024-09-13T08:40:30.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5567400640 1 -- start start 2024-09-13T08:40:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5567400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55600afb10 0x7f55600cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5567400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55600b1650 0x7f55600c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5567400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55601605d0 0x7f55601fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5567400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5560169540 con 0x7f55600afb10 2024-09-13T08:40:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5567400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55600b1430 con 0x7f55600b1650 2024-09-13T08:40:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5567400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f55601696c0 con 0x7f55601605d0 2024-09-13T08:40:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5564a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55600afb10 0x7f55600cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5564a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55600afb10 0x7f55600cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57492/0 (socket says 172.21.15.44:57492) 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5564a00640 1 -- 172.21.15.44:0/2333843406 learned_addr learned my addr 172.21.15.44:0/2333843406 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f555fe00640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55600b1650 0x7f55600c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5565400640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55601605d0 0x7f55601fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5564a00640 1 -- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55601605d0 msgr2=0x7f55601fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5564a00640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55601605d0 0x7f55601fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5564a00640 1 -- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55600b1650 msgr2=0x7f55600c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5564a00640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55600b1650 0x7f55600c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5564a00640 1 -- 172.21.15.44:0/2333843406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556006ded0 con 0x7f55600afb10 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5565400640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55601605d0 0x7f55601fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5564a00640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55600afb10 0x7f55600cd120 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f555400e7d0 tx=0x7f555400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f555cc00640 1 -- 172.21.15.44:0/2333843406 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555400ccc0 con 0x7f55600afb10 2024-09-13T08:40:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f555cc00640 1 -- 172.21.15.44:0/2333843406 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5554010430 con 0x7f55600afb10 2024-09-13T08:40:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f555cc00640 1 -- 172.21.15.44:0/2333843406 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554010860 con 0x7f55600afb10 2024-09-13T08:40:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5567400640 1 -- 172.21.15.44:0/2333843406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556006e1b0 con 0x7f55600afb10 2024-09-13T08:40:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.621+0000 7f5567400640 1 -- 172.21.15.44:0/2333843406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556006e770 con 0x7f55600afb10 2024-09-13T08:40:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.625+0000 7f5567400640 1 -- 172.21.15.44:0/2333843406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55600affc0 con 0x7f55600afb10 2024-09-13T08:40:30.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.625+0000 7f555cc00640 1 -- 172.21.15.44:0/2333843406 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f5554010a00 con 0x7f55600afb10 2024-09-13T08:40:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.625+0000 7f555cc00640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5540073220 0x7f55400756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.625+0000 7f555cc00640 1 -- 172.21.15.44:0/2333843406 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f5554094af0 con 0x7f55600afb10 2024-09-13T08:40:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.625+0000 7f555fe00640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5540073220 0x7f55400756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.625+0000 7f555fe00640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5540073220 0x7f55400756e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f556015fff0 tx=0x7f554c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:30.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.629+0000 7f555cc00640 1 -- 172.21.15.44:0/2333843406 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f555405fbb0 con 0x7f55600afb10 2024-09-13T08:40:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.805+0000 7f5567400640 1 -- 172.21.15.44:0/2333843406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5560071d00 con 0x7f55600afb10 2024-09-13T08:40:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f555cc00640 1 -- 172.21.15.44:0/2333843406 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7f555405f4b0 con 0x7f55600afb10 2024-09-13T08:40:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 -- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5540073220 msgr2=0x7f55400756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5540073220 0x7f55400756e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f556015fff0 tx=0x7f554c009040 comp rx=0 tx=0).stop 2024-09-13T08:40:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 -- 172.21.15.44:0/2333843406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55600afb10 msgr2=0x7f55600cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55600afb10 0x7f55600cd120 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f555400e7d0 tx=0x7f555400ec90 comp rx=0 tx=0).stop 2024-09-13T08:40:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 -- 172.21.15.44:0/2333843406 shutdown_connections 2024-09-13T08:40:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5540073220 0x7f55400756e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55601605d0 0x7f55601fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55600b1650 0x7f55600c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 --2- 172.21.15.44:0/2333843406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55600afb10 0x7f55600cd120 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:30.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 -- 172.21.15.44:0/2333843406 >> 172.21.15.44:0/2333843406 conn(0x7f55600d1dd0 msgr2=0x7f55600b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:30.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 -- 172.21.15.44:0/2333843406 shutdown_connections 2024-09-13T08:40:30.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:30.809+0000 7f5567400640 1 -- 172.21.15.44:0/2333843406 wait complete. 2024-09-13T08:40:30.832 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:40:30.832 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:40:30.912 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:40:30.994 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:40:30.994 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:40:30.994 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:31.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:40:31.003 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:40:31.003 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:31.013 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:40:31.013 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:40:31.014 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:31.054 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:40:31.054 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:40:31.055 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:31.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:40:31.064 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T08:40:31.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:40:31.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.241+0000 7fa970000640 1 -- 172.21.15.44:0/899160614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa96815c130 msgr2=0x7fa9680cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:31.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.241+0000 7fa970000640 1 --2- 172.21.15.44:0/899160614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa96815c130 0x7fa9680cd050 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa9680d0780 tx=0x7fa96001ca80 comp rx=0 tx=0).stop 2024-09-13T08:40:31.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.241+0000 7fa970000640 1 -- 172.21.15.44:0/899160614 shutdown_connections 2024-09-13T08:40:31.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.241+0000 7fa970000640 1 --2- 172.21.15.44:0/899160614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa96815c130 0x7fa9680cd050 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:31.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.241+0000 7fa970000640 1 -- 172.21.15.44:0/899160614 >> 172.21.15.44:0/899160614 conn(0x7fa9680d1ea0 msgr2=0x7fa9680d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:31.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.241+0000 7fa970000640 1 -- 172.21.15.44:0/899160614 shutdown_connections 2024-09-13T08:40:31.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.241+0000 7fa970000640 1 -- 172.21.15.44:0/899160614 wait complete. 2024-09-13T08:40:31.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.241+0000 7fa970000640 1 Processor -- start 2024-09-13T08:40:31.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.241+0000 7fa970000640 1 -- start start 2024-09-13T08:40:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa970000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa9680cecf0 0x7fa9681566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa970000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa96815bd50 0x7fa968151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa970000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa96815c130 0x7fa968152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa970000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa96816a580 con 0x7fa96815c130 2024-09-13T08:40:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa970000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa96816a400 con 0x7fa96815bd50 2024-09-13T08:40:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa970000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa96816a700 con 0x7fa9680cecf0 2024-09-13T08:40:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa96815c130 0x7fa968152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa96815c130 0x7fa968152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57504/0 (socket says 172.21.15.44:57504) 2024-09-13T08:40:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96e000640 1 -- 172.21.15.44:0/2673779018 learned_addr learned my addr 172.21.15.44:0/2673779018 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96cc00640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa96815bd50 0x7fa968151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96d600640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa9680cecf0 0x7fa9681566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96e000640 1 -- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa9680cecf0 msgr2=0x7fa9681566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96e000640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa9680cecf0 0x7fa9681566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96e000640 1 -- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa96815bd50 msgr2=0x7fa968151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96e000640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa96815bd50 0x7fa968151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96e000640 1 -- 172.21.15.44:0/2673779018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa968155030 con 0x7fa96815c130 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa96e000640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa96815c130 0x7fa968152d10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa960009b10 tx=0x7fa960009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa965600640 1 -- 172.21.15.44:0/2673779018 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600bc070 con 0x7fa96815c130 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa965600640 1 -- 172.21.15.44:0/2673779018 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9600042f0 con 0x7fa96815c130 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa970000640 1 -- 172.21.15.44:0/2673779018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa968155310 con 0x7fa96815c130 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa965600640 1 -- 172.21.15.44:0/2673779018 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600b2040 con 0x7fa96815c130 2024-09-13T08:40:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa970000640 1 -- 172.21.15.44:0/2673779018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9681558d0 con 0x7fa96815c130 2024-09-13T08:40:31.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.245+0000 7fa970000640 1 -- 172.21.15.44:0/2673779018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9680ce5e0 con 0x7fa96815c130 2024-09-13T08:40:31.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.249+0000 7fa965600640 1 -- 172.21.15.44:0/2673779018 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fa9600bf070 con 0x7fa96815c130 2024-09-13T08:40:31.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.249+0000 7fa965600640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa948073220 0x7fa9480756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:31.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.249+0000 7fa965600640 1 -- 172.21.15.44:0/2673779018 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fa96012de10 con 0x7fa96815c130 2024-09-13T08:40:31.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.249+0000 7fa96d600640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa948073220 0x7fa9480756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:31.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.249+0000 7fa96d600640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa948073220 0x7fa9480756e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0046c0 tx=0x7fa95c007cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:31.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.253+0000 7fa965600640 1 -- 172.21.15.44:0/2673779018 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa960131030 con 0x7fa96815c130 2024-09-13T08:40:31.265 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:40:31.268 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:40:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 -- 172.21.15.44:0/2673779018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa968072090 con 0x7fa96815c130 2024-09-13T08:40:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa965600640 1 -- 172.21.15.44:0/2673779018 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3911 (secure 0 0 0) 0x7fa960016790 con 0x7fa96815c130 2024-09-13T08:40:31.467 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:31.467 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":20,"btime":"2024-09-13T08:40:25:067994+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4828,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/238336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":238336703},{"type":"v1","addr":"172.21.15.129:6836","nonce":238336703}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2440287716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2440287716},{"type":"v1","addr":"172.21.15.44:6834","nonce":2440287716}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3117701056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":3117701056},{"type":"v1","addr":"172.21.15.129:6837","nonce":3117701056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/688090256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":688090256},{"type":"v1","addr":"172.21.15.44:6835","nonce":688090256}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-13T08:40:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 20 2024-09-13T08:40:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 -- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa948073220 msgr2=0x7fa9480756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa948073220 0x7fa9480756e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0046c0 tx=0x7fa95c007cd0 comp rx=0 tx=0).stop 2024-09-13T08:40:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 -- 172.21.15.44:0/2673779018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa96815c130 msgr2=0x7fa968152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa96815c130 0x7fa968152d10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa960009b10 tx=0x7fa960009260 comp rx=0 tx=0).stop 2024-09-13T08:40:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 -- 172.21.15.44:0/2673779018 shutdown_connections 2024-09-13T08:40:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa948073220 0x7fa9480756e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa96815c130 0x7fa968152d10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa96815bd50 0x7fa968151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 --2- 172.21.15.44:0/2673779018 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa9680cecf0 0x7fa9681566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 -- 172.21.15.44:0/2673779018 >> 172.21.15.44:0/2673779018 conn(0x7fa9680d1ea0 msgr2=0x7fa9680c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 -- 172.21.15.44:0/2673779018 shutdown_connections 2024-09-13T08:40:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:31.465+0000 7fa970000640 1 -- 172.21.15.44:0/2673779018 wait complete. 2024-09-13T08:40:31.489 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T08:40:31.489 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T08:40:31.489 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T08:40:31.489 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:40:31.490 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T08:40:31.490 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T08:40:31.490 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T08:40:31.490 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:40:31.490 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T08:40:31.490 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T08:40:31.490 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T08:40:31.491 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:40:31.491 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T08:40:31.491 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T08:40:31.491 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T08:40:31.491 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:40:31.501 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T08:40:31.502 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T08:40:31.502 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T08:40:31.502 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T08:40:31.507 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T08:40:31.507 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T08:40:31.548 INFO:tasks.ceph.mds.a:Started 2024-09-13T08:40:31.548 DEBUG:teuthology.parallel:result is None 2024-09-13T08:40:31.549 INFO:tasks.ceph.mds.c:Started 2024-09-13T08:40:31.550 DEBUG:teuthology.parallel:result is None 2024-09-13T08:40:31.553 INFO:tasks.ceph.mds.d:Started 2024-09-13T08:40:31.554 DEBUG:teuthology.parallel:result is None 2024-09-13T08:40:31.634 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:40:31.635 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:40:31.646 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:40:33.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-13T08:40:37.508 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T08:40:37.508 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T08:40:37.512 INFO:tasks.ceph.mds.b:Started 2024-09-13T08:40:37.512 DEBUG:teuthology.parallel:result is None 2024-09-13T08:40:37.512 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T08:40:37.602 INFO:tasks.ceph.mds.b.smithi129.stdout:starting mds.b at 2024-09-13T08:40:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.669+0000 7fbf88e00640 1 -- 172.21.15.44:0/4073992699 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf840b49e0 msgr2=0x7fbf8406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.669+0000 7fbf88e00640 1 --2- 172.21.15.44:0/4073992699 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf840b49e0 0x7fbf8406f7b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00ad00 tx=0x7fbf7c01dda0 comp rx=0 tx=0).stop 2024-09-13T08:40:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.669+0000 7fbf88e00640 1 -- 172.21.15.44:0/4073992699 shutdown_connections 2024-09-13T08:40:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.669+0000 7fbf88e00640 1 --2- 172.21.15.44:0/4073992699 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf840b49e0 0x7fbf8406f7b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.669+0000 7fbf88e00640 1 -- 172.21.15.44:0/4073992699 >> 172.21.15.44:0/4073992699 conn(0x7fbf840b18f0 msgr2=0x7fbf840b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.669+0000 7fbf88e00640 1 -- 172.21.15.44:0/4073992699 shutdown_connections 2024-09-13T08:40:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.669+0000 7fbf88e00640 1 -- 172.21.15.44:0/4073992699 wait complete. 2024-09-13T08:40:37.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 Processor -- start 2024-09-13T08:40:37.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 -- start start 2024-09-13T08:40:37.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf840b49e0 0x7fbf840b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:37.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf840cd100 0x7fbf841522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:37.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf8415dae0 0x7fbf8409d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:37.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf84168680 con 0x7fbf840b49e0 2024-09-13T08:40:37.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf84168500 con 0x7fbf8415dae0 2024-09-13T08:40:37.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fbf84168800 con 0x7fbf840cd100 2024-09-13T08:40:37.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf81e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf840b49e0 0x7fbf840b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:37.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf81e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf840b49e0 0x7fbf840b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52116/0 (socket says 172.21.15.44:52116) 2024-09-13T08:40:37.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf81e00640 1 -- 172.21.15.44:0/3990932731 learned_addr learned my addr 172.21.15.44:0/3990932731 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:37.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf81e00640 1 -- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf840cd100 msgr2=0x7fbf841522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:37.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf7be00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf840cd100 0x7fbf841522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:37.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf82800640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf8415dae0 0x7fbf8409d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:37.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf81e00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf840cd100 0x7fbf841522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:37.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf81e00640 1 -- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf8415dae0 msgr2=0x7fbf8409d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:37.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf81e00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf8415dae0 0x7fbf8409d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:37.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf81e00640 1 -- 172.21.15.44:0/3990932731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8409bce0 con 0x7fbf840b49e0 2024-09-13T08:40:37.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf7be00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf840cd100 0x7fbf841522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:37.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf82800640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf8415dae0 0x7fbf8409d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:37.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf81e00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf840b49e0 0x7fbf840b1350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbf74001230 tx=0x7fbf7400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:37.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf7a000640 1 -- 172.21.15.44:0/3990932731 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74016020 con 0x7fbf840b49e0 2024-09-13T08:40:37.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf7a000640 1 -- 172.21.15.44:0/3990932731 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf7400a930 con 0x7fbf840b49e0 2024-09-13T08:40:37.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf7a000640 1 -- 172.21.15.44:0/3990932731 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400ad80 con 0x7fbf840b49e0 2024-09-13T08:40:37.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 -- 172.21.15.44:0/3990932731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8409bfc0 con 0x7fbf840b49e0 2024-09-13T08:40:37.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 -- 172.21.15.44:0/3990932731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8409c580 con 0x7fbf840b49e0 2024-09-13T08:40:37.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.673+0000 7fbf88e00640 1 -- 172.21.15.44:0/3990932731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf840ce810 con 0x7fbf840b49e0 2024-09-13T08:40:37.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.677+0000 7fbf7a000640 1 -- 172.21.15.44:0/3990932731 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fbf74019020 con 0x7fbf840b49e0 2024-09-13T08:40:37.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.677+0000 7fbf7a000640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf60073220 0x7fbf600756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:37.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.677+0000 7fbf7a000640 1 -- 172.21.15.44:0/3990932731 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fbf74094210 con 0x7fbf840b49e0 2024-09-13T08:40:37.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.677+0000 7fbf7be00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf60073220 0x7fbf600756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:37.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.677+0000 7fbf7be00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf60073220 0x7fbf600756e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbf84070d20 tx=0x7fbf6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:37.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.681+0000 7fbf7a000640 1 -- 172.21.15.44:0/3990932731 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fbf74020020 con 0x7fbf840b49e0 2024-09-13T08:40:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:37.869+0000 7fbf88e00640 1 -- 172.21.15.44:0/3990932731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbf8415cab0 con 0x7fbf840b49e0 2024-09-13T08:40:39.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.128+0000 7fbf7a000640 1 -- 172.21.15.44:0/3990932731 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) ==== 97+0+0 (secure 0 0 0) 0x7fbf74092e20 con 0x7fbf840b49e0 2024-09-13T08:40:39.132 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-09-13T08:40:39.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 -- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf60073220 msgr2=0x7fbf600756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf60073220 0x7fbf600756e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbf84070d20 tx=0x7fbf6c009040 comp rx=0 tx=0).stop 2024-09-13T08:40:39.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 -- 172.21.15.44:0/3990932731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf840b49e0 msgr2=0x7fbf840b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf840b49e0 0x7fbf840b1350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbf74001230 tx=0x7fbf7400cb70 comp rx=0 tx=0).stop 2024-09-13T08:40:39.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 -- 172.21.15.44:0/3990932731 shutdown_connections 2024-09-13T08:40:39.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf60073220 0x7fbf600756e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf8415dae0 0x7fbf8409d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf840cd100 0x7fbf841522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 --2- 172.21.15.44:0/3990932731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf840b49e0 0x7fbf840b1350 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 -- 172.21.15.44:0/3990932731 >> 172.21.15.44:0/3990932731 conn(0x7fbf840b18f0 msgr2=0x7fbf84072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:39.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 -- 172.21.15.44:0/3990932731 shutdown_connections 2024-09-13T08:40:39.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.132+0000 7fbf88e00640 1 -- 172.21.15.44:0/3990932731 wait complete. 2024-09-13T08:40:39.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T08:40:39.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 -- 172.21.15.44:0/147460765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb38c0ced80 msgr2=0x7fb38c0ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 --2- 172.21.15.44:0/147460765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb38c0ced80 0x7fb38c0ae9b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb378009840 tx=0x7fb37801c820 comp rx=0 tx=0).stop 2024-09-13T08:40:39.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 -- 172.21.15.44:0/147460765 shutdown_connections 2024-09-13T08:40:39.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 --2- 172.21.15.44:0/147460765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb38c0ced80 0x7fb38c0ae9b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 -- 172.21.15.44:0/147460765 >> 172.21.15.44:0/147460765 conn(0x7fb38c0d1dc0 msgr2=0x7fb38c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:39.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 -- 172.21.15.44:0/147460765 shutdown_connections 2024-09-13T08:40:39.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 -- 172.21.15.44:0/147460765 wait complete. 2024-09-13T08:40:39.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 Processor -- start 2024-09-13T08:40:39.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 -- start start 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb38c0cdb30 0x7fb38c14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb38c0ced80 0x7fb38c1fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb38c15bea0 0x7fb38c155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c16a8a0 con 0x7fb38c0cdb30 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c16a720 con 0x7fb38c15bea0 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb394400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb38c16aa20 con 0x7fb38c0ced80 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb391a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb38c0cdb30 0x7fb38c14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb391a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb38c0cdb30 0x7fb38c14ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52140/0 (socket says 172.21.15.44:52140) 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb391a00640 1 -- 172.21.15.44:0/628993363 learned_addr learned my addr 172.21.15.44:0/628993363 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb391000640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb38c0ced80 0x7fb38c1fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb392400640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb38c15bea0 0x7fb38c155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:39.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb391a00640 1 -- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb38c0ced80 msgr2=0x7fb38c1fbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb391a00640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb38c0ced80 0x7fb38c1fbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb391a00640 1 -- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb38c15bea0 msgr2=0x7fb38c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb391a00640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb38c15bea0 0x7fb38c155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.312+0000 7fb391a00640 1 -- 172.21.15.44:0/628993363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38c0b2f10 con 0x7fb38c0cdb30 2024-09-13T08:40:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.316+0000 7fb391a00640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb38c0cdb30 0x7fb38c14ffb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb38000c400 tx=0x7fb38000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.316+0000 7fb389600640 1 -- 172.21.15.44:0/628993363 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380011790 con 0x7fb38c0cdb30 2024-09-13T08:40:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.316+0000 7fb389600640 1 -- 172.21.15.44:0/628993363 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb380017070 con 0x7fb38c0cdb30 2024-09-13T08:40:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.316+0000 7fb389600640 1 -- 172.21.15.44:0/628993363 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380011ce0 con 0x7fb38c0cdb30 2024-09-13T08:40:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.316+0000 7fb394400640 1 -- 172.21.15.44:0/628993363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38c1fc520 con 0x7fb38c0cdb30 2024-09-13T08:40:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.316+0000 7fb394400640 1 -- 172.21.15.44:0/628993363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38c0b3460 con 0x7fb38c0cdb30 2024-09-13T08:40:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.316+0000 7fb394400640 1 -- 172.21.15.44:0/628993363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38c0ce670 con 0x7fb38c0cdb30 2024-09-13T08:40:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.316+0000 7fb389600640 1 -- 172.21.15.44:0/628993363 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fb38001c460 con 0x7fb38c0cdb30 2024-09-13T08:40:39.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.316+0000 7fb389600640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb36c073220 0x7fb36c0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:39.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.316+0000 7fb389600640 1 -- 172.21.15.44:0/628993363 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7fb380093e20 con 0x7fb38c0cdb30 2024-09-13T08:40:39.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.320+0000 7fb391000640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb36c073220 0x7fb36c0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:39.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.320+0000 7fb391000640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb36c073220 0x7fb36c0756e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb3780af000 tx=0x7fb37801b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:39.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.320+0000 7fb389600640 1 -- 172.21.15.44:0/628993363 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb38c1fc520 con 0x7fb38c0cdb30 2024-09-13T08:40:39.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.556+0000 7fb394400640 1 -- 172.21.15.44:0/628993363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb38c15b3e0 con 0x7fb38c0cdb30 2024-09-13T08:40:39.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.556+0000 7fb389600640 1 -- 172.21.15.44:0/628993363 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7fb38005e810 con 0x7fb38c0cdb30 2024-09-13T08:40:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:39.561 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:40:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-09-13T08:40:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-09-13T08:40:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw==", 2024-09-13T08:40:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:40:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:40:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ==", 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA==", 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-09-13T08:40:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw==", 2024-09-13T08:40:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:40:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:40:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:40:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-09-13T08:40:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg==", 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg==", 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:40:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw==", 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig==", 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ==", 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:40:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g==", 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg==", 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg==", 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA==", 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w==", 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-09-13T08:40:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA==", 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w==", 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:40:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:40:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:40:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:40:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-09-13T08:40:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmDD2LJxAA3s0jEb2EfPsLcu0w145G3Q==", 2024-09-13T08:40:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ==", 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg==", 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T08:40:39.575 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:40:39.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 -- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb36c073220 msgr2=0x7fb36c0756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb36c073220 0x7fb36c0756e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb3780af000 tx=0x7fb37801b060 comp rx=0 tx=0).stop 2024-09-13T08:40:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 -- 172.21.15.44:0/628993363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb38c0cdb30 msgr2=0x7fb38c14ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb38c0cdb30 0x7fb38c14ffb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb38000c400 tx=0x7fb38000c8c0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 -- 172.21.15.44:0/628993363 shutdown_connections 2024-09-13T08:40:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb36c073220 0x7fb36c0756e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb38c15bea0 0x7fb38c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb38c0ced80 0x7fb38c1fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 --2- 172.21.15.44:0/628993363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb38c0cdb30 0x7fb38c14ffb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 -- 172.21.15.44:0/628993363 >> 172.21.15.44:0/628993363 conn(0x7fb38c0d1dc0 msgr2=0x7fb38c06eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 -- 172.21.15.44:0/628993363 shutdown_connections 2024-09-13T08:40:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.560+0000 7fb394400640 1 -- 172.21.15.44:0/628993363 wait complete. 2024-09-13T08:40:39.582 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T08:40:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.728+0000 7fe928000640 1 -- 172.21.15.44:0/1641344026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe908004010 msgr2=0x7fe908004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.728+0000 7fe928000640 1 --2- 172.21.15.44:0/1641344026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe908004010 0x7fe908004810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00c2e0 tx=0x7fe90c01f6e0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.728+0000 7fe926000640 1 -- 172.21.15.44:0/1641344026 reap_dead start 2024-09-13T08:40:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.728+0000 7fe928000640 1 -- 172.21.15.44:0/1641344026 shutdown_connections 2024-09-13T08:40:39.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.728+0000 7fe928000640 1 -- 172.21.15.44:0/1641344026 >> 172.21.15.44:0/1641344026 conn(0x7fe9200d1dc0 msgr2=0x7fe9200d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:39.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.728+0000 7fe928000640 1 -- 172.21.15.44:0/1641344026 shutdown_connections 2024-09-13T08:40:39.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.728+0000 7fe928000640 1 -- 172.21.15.44:0/1641344026 wait complete. 2024-09-13T08:40:39.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.728+0000 7fe928000640 1 Processor -- start 2024-09-13T08:40:39.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.728+0000 7fe928000640 1 -- start start 2024-09-13T08:40:39.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe928000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe908004010 0x7fe9201f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:39.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe928000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9200cbdf0 0x7fe9201593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:39.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe928000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9201f5aa0 0x7fe9201f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:39.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe928000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe92015c440 con 0x7fe9200cbdf0 2024-09-13T08:40:39.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe928000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe9200d2af0 con 0x7fe9201f5aa0 2024-09-13T08:40:39.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe928000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe9200af0c0 con 0x7fe908004010 2024-09-13T08:40:39.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe924c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9200cbdf0 0x7fe9201593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:39.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe924c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9200cbdf0 0x7fe9201593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52158/0 (socket says 172.21.15.44:52158) 2024-09-13T08:40:39.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe924c00640 1 -- 172.21.15.44:0/1967624008 learned_addr learned my addr 172.21.15.44:0/1967624008 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:39.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe925600640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe908004010 0x7fe9201f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:39.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe926000640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9201f5aa0 0x7fe9201f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:39.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe924c00640 1 -- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe908004010 msgr2=0x7fe9201f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe924c00640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe908004010 0x7fe9201f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe924c00640 1 -- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9201f5aa0 msgr2=0x7fe9201f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe924c00640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9201f5aa0 0x7fe9201f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe924c00640 1 -- 172.21.15.44:0/1967624008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9200b3980 con 0x7fe9200cbdf0 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe926000640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9201f5aa0 0x7fe9201f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe925600640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe908004010 0x7fe9201f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe924c00640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9200cbdf0 0x7fe9201593c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe90c01dde0 tx=0x7fe90c006d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe91d600640 1 -- 172.21.15.44:0/1967624008 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0b3070 con 0x7fe9200cbdf0 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe91d600640 1 -- 172.21.15.44:0/1967624008 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe90c003600 con 0x7fe9200cbdf0 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe928000640 1 -- 172.21.15.44:0/1967624008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9201f8230 con 0x7fe9200cbdf0 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe91d600640 1 -- 172.21.15.44:0/1967624008 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c003a50 con 0x7fe9200cbdf0 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe928000640 1 -- 172.21.15.44:0/1967624008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9201f87f0 con 0x7fe9200cbdf0 2024-09-13T08:40:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe928000640 1 -- 172.21.15.44:0/1967624008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9200ae9b0 con 0x7fe9200cbdf0 2024-09-13T08:40:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.732+0000 7fe91d600640 1 -- 172.21.15.44:0/1967624008 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fe90c0c1060 con 0x7fe9200cbdf0 2024-09-13T08:40:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.736+0000 7fe91d600640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe900073250 0x7fe900075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.736+0000 7fe91d600640 1 -- 172.21.15.44:0/1967624008 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7fe90c131580 con 0x7fe9200cbdf0 2024-09-13T08:40:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.736+0000 7fe925600640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe900073250 0x7fe900075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.736+0000 7fe925600640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe900073250 0x7fe900075710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe92015acb0 tx=0x7fe914006c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:39.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.740+0000 7fe91d600640 1 -- 172.21.15.44:0/1967624008 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe90c019490 con 0x7fe9200cbdf0 2024-09-13T08:40:39.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.916+0000 7fe928000640 1 -- 172.21.15.44:0/1967624008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe92015b7f0 con 0x7fe9200cbdf0 2024-09-13T08:40:39.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.916+0000 7fe91d600640 1 -- 172.21.15.44:0/1967624008 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) ==== 84+0+65 (secure 0 0 0) 0x7fe90c0fc5c0 con 0x7fe9200cbdf0 2024-09-13T08:40:39.921 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:39.921 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:40:39.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:39.921 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T08:40:39.921 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T08:40:39.921 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:39.921 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:40:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 -- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe900073250 msgr2=0x7fe900075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe900073250 0x7fe900075710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe92015acb0 tx=0x7fe914006c10 comp rx=0 tx=0).stop 2024-09-13T08:40:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 -- 172.21.15.44:0/1967624008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9200cbdf0 msgr2=0x7fe9201593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9200cbdf0 0x7fe9201593c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe90c01dde0 tx=0x7fe90c006d20 comp rx=0 tx=0).stop 2024-09-13T08:40:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 -- 172.21.15.44:0/1967624008 shutdown_connections 2024-09-13T08:40:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe900073250 0x7fe900075710 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9201f5aa0 0x7fe9201f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9200cbdf0 0x7fe9201593c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 --2- 172.21.15.44:0/1967624008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe908004010 0x7fe9201f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 -- 172.21.15.44:0/1967624008 >> 172.21.15.44:0/1967624008 conn(0x7fe9200d1dc0 msgr2=0x7fe9200afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:39.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 -- 172.21.15.44:0/1967624008 shutdown_connections 2024-09-13T08:40:39.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:39.920+0000 7fe928000640 1 -- 172.21.15.44:0/1967624008 wait complete. 2024-09-13T08:40:39.942 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T08:40:39.942 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T08:40:40.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 -- 172.21.15.44:0/789064726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd840004810 msgr2=0x7fd840004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:40.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 --2- 172.21.15.44:0/789064726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd840004810 0x7fd840004c90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd844009b60 tx=0x7fd84401d2b0 comp rx=0 tx=0).stop 2024-09-13T08:40:40.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd85a800640 1 -- 172.21.15.44:0/789064726 reap_dead start 2024-09-13T08:40:40.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 -- 172.21.15.44:0/789064726 shutdown_connections 2024-09-13T08:40:40.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 -- 172.21.15.44:0/789064726 >> 172.21.15.44:0/789064726 conn(0x7fd85c06dd30 msgr2=0x7fd85c06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:40.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 -- 172.21.15.44:0/789064726 shutdown_connections 2024-09-13T08:40:40.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 -- 172.21.15.44:0/789064726 wait complete. 2024-09-13T08:40:40.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 Processor -- start 2024-09-13T08:40:40.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 -- start start 2024-09-13T08:40:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd840004810 0x7fd85c1f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c0b2410 0x7fd85c143750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd85c149b00 0x7fd85c149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c0b02c0 con 0x7fd840004810 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c0b0140 con 0x7fd85c0b2410 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd861200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd85c166950 con 0x7fd85c149b00 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd859e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd840004810 0x7fd85c1f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd859e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd840004810 0x7fd85c1f6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52182/0 (socket says 172.21.15.44:52182) 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd859e00640 1 -- 172.21.15.44:0/3745080489 learned_addr learned my addr 172.21.15.44:0/3745080489 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd853e00640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c0b2410 0x7fd85c143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd85a800640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd85c149b00 0x7fd85c149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd859e00640 1 -- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd85c149b00 msgr2=0x7fd85c149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd859e00640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd85c149b00 0x7fd85c149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd859e00640 1 -- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c0b2410 msgr2=0x7fd85c143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd859e00640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c0b2410 0x7fd85c143750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd859e00640 1 -- 172.21.15.44:0/3745080489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c0b2870 con 0x7fd840004810 2024-09-13T08:40:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd853e00640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c0b2410 0x7fd85c143750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:40:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.092+0000 7fd859e00640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd840004810 0x7fd85c1f6320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00e5f0 tx=0x7fd84c00eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.096+0000 7fd852000640 1 -- 172.21.15.44:0/3745080489 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c015070 con 0x7fd840004810 2024-09-13T08:40:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.096+0000 7fd852000640 1 -- 172.21.15.44:0/3745080489 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd84c00ca20 con 0x7fd840004810 2024-09-13T08:40:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.096+0000 7fd861200640 1 -- 172.21.15.44:0/3745080489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c0b0a80 con 0x7fd840004810 2024-09-13T08:40:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.096+0000 7fd852000640 1 -- 172.21.15.44:0/3745080489 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00ce70 con 0x7fd840004810 2024-09-13T08:40:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.096+0000 7fd861200640 1 -- 172.21.15.44:0/3745080489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c0b1040 con 0x7fd840004810 2024-09-13T08:40:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.096+0000 7fd861200640 1 -- 172.21.15.44:0/3745080489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd85c06cc20 con 0x7fd840004810 2024-09-13T08:40:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.096+0000 7fd852000640 1 -- 172.21.15.44:0/3745080489 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fd84c018070 con 0x7fd840004810 2024-09-13T08:40:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.096+0000 7fd852000640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd838073250 0x7fd838075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.096+0000 7fd852000640 1 -- 172.21.15.44:0/3745080489 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7fd84c094610 con 0x7fd840004810 2024-09-13T08:40:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.096+0000 7fd853e00640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd838073250 0x7fd838075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.100+0000 7fd853e00640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd838073250 0x7fd838075710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8440ae000 tx=0x7fd84401a050 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.100+0000 7fd852000640 1 -- 172.21.15.44:0/3745080489 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd84c05ef00 con 0x7fd840004810 2024-09-13T08:40:40.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-13T08:40:40.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:40.276+0000 7fd861200640 1 -- 172.21.15.44:0/3745080489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd85c155050 con 0x7fd840004810 2024-09-13T08:40:41.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.144+0000 7fd852000640 1 -- 172.21.15.44:0/3745080489 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) ==== 138+0+0 (secure 0 0 0) 0x7fd84c063930 con 0x7fd840004810 2024-09-13T08:40:41.149 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T08:40:41.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 -- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd838073250 msgr2=0x7fd838075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd838073250 0x7fd838075710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8440ae000 tx=0x7fd84401a050 comp rx=0 tx=0).stop 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 -- 172.21.15.44:0/3745080489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd840004810 msgr2=0x7fd85c1f6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd840004810 0x7fd85c1f6320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00e5f0 tx=0x7fd84c00eab0 comp rx=0 tx=0).stop 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 -- 172.21.15.44:0/3745080489 shutdown_connections 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd838073250 0x7fd838075710 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd85c149b00 0x7fd85c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c0b2410 0x7fd85c143750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 --2- 172.21.15.44:0/3745080489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd840004810 0x7fd85c1f6320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 -- 172.21.15.44:0/3745080489 >> 172.21.15.44:0/3745080489 conn(0x7fd85c06dd30 msgr2=0x7fd85c154670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 -- 172.21.15.44:0/3745080489 shutdown_connections 2024-09-13T08:40:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.148+0000 7fd861200640 1 -- 172.21.15.44:0/3745080489 wait complete. 2024-09-13T08:40:41.180 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T08:40:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.332+0000 7f5532600640 1 -- 172.21.15.44:0/3425011301 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f552c070960 msgr2=0x7f552c0cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.332+0000 7f5532600640 1 --2- 172.21.15.44:0/3425011301 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f552c070960 0x7f552c0cc740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f551c00a950 tx=0x7f551c01dae0 comp rx=0 tx=0).stop 2024-09-13T08:40:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.332+0000 7f5532600640 1 -- 172.21.15.44:0/3425011301 shutdown_connections 2024-09-13T08:40:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.332+0000 7f5532600640 1 --2- 172.21.15.44:0/3425011301 >> v2:172.21.15.129:3300/0 conn(0x7f552c06eeb0 0x7f552c164fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.332+0000 7f5532600640 1 --2- 172.21.15.44:0/3425011301 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f552c070960 0x7f552c0cc740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.332+0000 7f5532600640 1 -- 172.21.15.44:0/3425011301 >> 172.21.15.44:0/3425011301 conn(0x7f552c0b18f0 msgr2=0x7f552c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.332+0000 7f5532600640 1 -- 172.21.15.44:0/3425011301 shutdown_connections 2024-09-13T08:40:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.332+0000 7f5532600640 1 -- 172.21.15.44:0/3425011301 wait complete. 2024-09-13T08:40:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5532600640 1 Processor -- start 2024-09-13T08:40:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5532600640 1 -- start start 2024-09-13T08:40:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5532600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f552c06eeb0 0x7f552c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5532600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f552c070960 0x7f552c1f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5532600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f552c0ccc80 0x7f552c1f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5532600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1676a0 con 0x7f552c070960 2024-09-13T08:40:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5532600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f552c167520 con 0x7f552c0ccc80 2024-09-13T08:40:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5532600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f552c167820 con 0x7f552c06eeb0 2024-09-13T08:40:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5523e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f552c070960 0x7f552c1f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f552c06eeb0 0x7f552c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f552c0ccc80 0x7f552c1f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f552c06eeb0 0x7f552c153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:52250/0 (socket says 172.21.15.44:52250) 2024-09-13T08:40:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552b400640 1 -- 172.21.15.44:0/696853454 learned_addr learned my addr 172.21.15.44:0/696853454 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552b400640 1 -- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f552c0ccc80 msgr2=0x7f552c1f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552b400640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f552c0ccc80 0x7f552c1f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552b400640 1 -- 172.21.15.44:0/696853454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f552c070960 msgr2=0x7f552c1f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552b400640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f552c070960 0x7f552c1f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552b400640 1 -- 172.21.15.44:0/696853454 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c143740 con 0x7f552c06eeb0 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552be00640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f552c0ccc80 0x7f552c1f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5523e00640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f552c070960 0x7f552c1f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f552b400640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f552c06eeb0 0x7f552c153ab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f551c00a8a0 tx=0x7f551c004620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5523400640 1 -- 172.21.15.44:0/696853454 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c0b3040 con 0x7f552c06eeb0 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5523400640 1 -- 172.21.15.44:0/696853454 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f551c005110 con 0x7f552c06eeb0 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5532600640 1 -- 172.21.15.44:0/696853454 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c1439c0 con 0x7f552c06eeb0 2024-09-13T08:40:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5523400640 1 -- 172.21.15.44:0/696853454 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c005540 con 0x7f552c06eeb0 2024-09-13T08:40:41.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.336+0000 7f5532600640 1 -- 172.21.15.44:0/696853454 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c09bf00 con 0x7f552c06eeb0 2024-09-13T08:40:41.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.340+0000 7f5532600640 1 -- 172.21.15.44:0/696853454 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552c069410 con 0x7f552c06eeb0 2024-09-13T08:40:41.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.340+0000 7f5523400640 1 -- 172.21.15.44:0/696853454 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f551c0046d0 con 0x7f552c06eeb0 2024-09-13T08:40:41.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.340+0000 7f5523400640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5508073220 0x7f55080756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:41.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.340+0000 7f5523400640 1 -- 172.21.15.44:0/696853454 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 5227+0+0 (secure 0 0 0) 0x7f551c12ea60 con 0x7f552c06eeb0 2024-09-13T08:40:41.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.340+0000 7f5523e00640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5508073220 0x7f55080756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:41.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.340+0000 7f5523e00640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5508073220 0x7f55080756e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5514007f30 tx=0x7f5514007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:41.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.344+0000 7f5523400640 1 -- 172.21.15.44:0/696853454 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f551c0be060 con 0x7f552c06eeb0 2024-09-13T08:40:41.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:41.520+0000 7f5532600640 1 -- 172.21.15.44:0/696853454 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f552c0cc740 con 0x7f552c06eeb0 2024-09-13T08:40:42.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.156+0000 7f5523400640 1 -- 172.21.15.44:0/696853454 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) ==== 170+0+0 (secure 0 0 0) 0x7f551c0fee20 con 0x7f552c06eeb0 2024-09-13T08:40:42.158 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T08:40:42.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.164+0000 7f5532600640 1 -- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5508073220 msgr2=0x7f55080756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:42.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.164+0000 7f5532600640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5508073220 0x7f55080756e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5514007f30 tx=0x7f5514007ec0 comp rx=0 tx=0).stop 2024-09-13T08:40:42.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.164+0000 7f5532600640 1 -- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f552c06eeb0 msgr2=0x7f552c153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:42.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.164+0000 7f5532600640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f552c06eeb0 0x7f552c153ab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f551c00a8a0 tx=0x7f551c004620 comp rx=0 tx=0).stop 2024-09-13T08:40:42.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.168+0000 7f5532600640 1 -- 172.21.15.44:0/696853454 shutdown_connections 2024-09-13T08:40:42.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.168+0000 7f5532600640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5508073220 0x7f55080756e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:42.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.168+0000 7f5532600640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f552c0ccc80 0x7f552c1f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:42.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.168+0000 7f5532600640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f552c070960 0x7f552c1f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:42.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.168+0000 7f5532600640 1 --2- 172.21.15.44:0/696853454 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f552c06eeb0 0x7f552c153ab0 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f551c00a8a0 tx=0x7f551c004620 comp rx=0 tx=0).stop 2024-09-13T08:40:42.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.168+0000 7f5532600640 1 -- 172.21.15.44:0/696853454 >> 172.21.15.44:0/696853454 conn(0x7f552c0b18f0 msgr2=0x7f552c15b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:42.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.172+0000 7f5532600640 1 -- 172.21.15.44:0/696853454 shutdown_connections 2024-09-13T08:40:42.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.172+0000 7f5532600640 1 -- 172.21.15.44:0/696853454 wait complete. 2024-09-13T08:40:42.207 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T08:40:42.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.356+0000 7f653e000640 1 -- 172.21.15.44:0/117801271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538073a60 msgr2=0x7f653815bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:42.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.356+0000 7f653e000640 1 --2- 172.21.15.44:0/117801271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538073a60 0x7f653815bd70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6528009870 tx=0x7f652801c820 comp rx=0 tx=0).stop 2024-09-13T08:40:42.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 -- 172.21.15.44:0/117801271 shutdown_connections 2024-09-13T08:40:42.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 --2- 172.21.15.44:0/117801271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538073a60 0x7f653815bd70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:42.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 -- 172.21.15.44:0/117801271 >> 172.21.15.44:0/117801271 conn(0x7f65380d0c70 msgr2=0x7f65380d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:42.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 -- 172.21.15.44:0/117801271 shutdown_connections 2024-09-13T08:40:42.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 -- 172.21.15.44:0/117801271 wait complete. 2024-09-13T08:40:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 Processor -- start 2024-09-13T08:40:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 -- start start 2024-09-13T08:40:42.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538073a60 0x7f6538154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:42.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6538073e40 0x7f65380b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:42.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6538151f40 0x7f653814cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65381697f0 con 0x7f6538073a60 2024-09-13T08:40:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6538169670 con 0x7f6538073e40 2024-09-13T08:40:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f653e000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6538169970 con 0x7f6538151f40 2024-09-13T08:40:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538073a60 0x7f6538154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538073a60 0x7f6538154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37280/0 (socket says 172.21.15.44:37280) 2024-09-13T08:40:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537400640 1 -- 172.21.15.44:0/3330559906 learned_addr learned my addr 172.21.15.44:0/3330559906 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:42.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f652fe00640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6538073e40 0x7f65380b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:42.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537e00640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6538151f40 0x7f653814cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537400640 1 -- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6538151f40 msgr2=0x7f653814cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537400640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6538151f40 0x7f653814cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537400640 1 -- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6538073e40 msgr2=0x7f65380b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537400640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6538073e40 0x7f65380b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537400640 1 -- 172.21.15.44:0/3330559906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65380b2a40 con 0x7f6538073a60 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f652fe00640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6538073e40 0x7f65380b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537e00640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6538151f40 0x7f653814cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f6537400640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538073a60 0x7f6538154e10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6528009790 tx=0x7f6528004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.360+0000 7f652f400640 1 -- 172.21.15.44:0/3330559906 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65280bb070 con 0x7f6538073a60 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.364+0000 7f652f400640 1 -- 172.21.15.44:0/3330559906 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6528005030 con 0x7f6538073a60 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.364+0000 7f653e000640 1 -- 172.21.15.44:0/3330559906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65380b2d20 con 0x7f6538073a60 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.364+0000 7f652f400640 1 -- 172.21.15.44:0/3330559906 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65280b2040 con 0x7f6538073a60 2024-09-13T08:40:42.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.364+0000 7f653e000640 1 -- 172.21.15.44:0/3330559906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6538152920 con 0x7f6538073a60 2024-09-13T08:40:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.364+0000 7f653e000640 1 -- 172.21.15.44:0/3330559906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6538152b20 con 0x7f6538073a60 2024-09-13T08:40:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.364+0000 7f652f400640 1 -- 172.21.15.44:0/3330559906 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f65280be070 con 0x7f6538073a60 2024-09-13T08:40:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.364+0000 7f652f400640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6514073220 0x7f65140756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.364+0000 7f652f400640 1 -- 172.21.15.44:0/3330559906 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5609+0+0 (secure 0 0 0) 0x7f652812e510 con 0x7f6538073a60 2024-09-13T08:40:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.364+0000 7f652fe00640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6514073220 0x7f65140756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:42.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.368+0000 7f652fe00640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6514073220 0x7f65140756e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f653814d860 tx=0x7f6520009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:42.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.368+0000 7f652f400640 1 -- 172.21.15.44:0/3330559906 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6538152b20 con 0x7f6538073a60 2024-09-13T08:40:42.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:42.584+0000 7f653e000640 1 -- 172.21.15.44:0/3330559906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f653815bd70 con 0x7f6538073a60 2024-09-13T08:40:43.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.176+0000 7f652f400640 1 -- 172.21.15.44:0/3330559906 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v24) ==== 172+0+0 (secure 0 0 0) 0x7f6528016410 con 0x7f6538073a60 2024-09-13T08:40:43.180 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 6 and data pool 7 2024-09-13T08:40:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.184+0000 7f653e000640 1 -- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6514073220 msgr2=0x7f65140756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.184+0000 7f653e000640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6514073220 0x7f65140756e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f653814d860 tx=0x7f6520009040 comp rx=0 tx=0).stop 2024-09-13T08:40:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.184+0000 7f653e000640 1 -- 172.21.15.44:0/3330559906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538073a60 msgr2=0x7f6538154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.184+0000 7f653e000640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538073a60 0x7f6538154e10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6528009790 tx=0x7f6528004040 comp rx=0 tx=0).stop 2024-09-13T08:40:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.188+0000 7f653e000640 1 -- 172.21.15.44:0/3330559906 shutdown_connections 2024-09-13T08:40:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.188+0000 7f653e000640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6514073220 0x7f65140756e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.188+0000 7f653e000640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6538151f40 0x7f653814cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.188+0000 7f653e000640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6538073e40 0x7f65380b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.188+0000 7f653e000640 1 --2- 172.21.15.44:0/3330559906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538073a60 0x7f6538154e10 secure :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6528009790 tx=0x7f6528004040 comp rx=0 tx=0).stop 2024-09-13T08:40:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.188+0000 7f653e000640 1 -- 172.21.15.44:0/3330559906 >> 172.21.15.44:0/3330559906 conn(0x7f65380d0c70 msgr2=0x7f653815a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.188+0000 7f653e000640 1 -- 172.21.15.44:0/3330559906 shutdown_connections 2024-09-13T08:40:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.188+0000 7f653e000640 1 -- 172.21.15.44:0/3330559906 wait complete. 2024-09-13T08:40:43.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:40:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.368+0000 7f2ef8a00640 1 -- 172.21.15.44:0/808965532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8004810 msgr2=0x7f2ed8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.368+0000 7f2ef8a00640 1 --2- 172.21.15.44:0/808965532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8004810 0x7f2ed8004c90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2edc006f60 tx=0x7f2edc01cea0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.368+0000 7f2ef2800640 1 -- 172.21.15.44:0/808965532 reap_dead start 2024-09-13T08:40:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.368+0000 7f2ef8a00640 1 -- 172.21.15.44:0/808965532 shutdown_connections 2024-09-13T08:40:43.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.368+0000 7f2ef8a00640 1 -- 172.21.15.44:0/808965532 >> 172.21.15.44:0/808965532 conn(0x7f2ef40b1f10 msgr2=0x7f2ef40b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:43.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.368+0000 7f2ef8a00640 1 -- 172.21.15.44:0/808965532 shutdown_connections 2024-09-13T08:40:43.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.368+0000 7f2ef8a00640 1 -- 172.21.15.44:0/808965532 wait complete. 2024-09-13T08:40:43.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.368+0000 7f2ef8a00640 1 Processor -- start 2024-09-13T08:40:43.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.368+0000 7f2ef8a00640 1 -- start start 2024-09-13T08:40:43.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef8a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ed8004810 0x7f2ef4154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:43.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ef40999f0 0x7f2ef4074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:43.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef8a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ef415a810 0x7f2ef415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:43.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef8a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4068dc0 con 0x7f2ef40999f0 2024-09-13T08:40:43.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef8a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4068c40 con 0x7f2ed8004810 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef8a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef4068f40 con 0x7f2ef415a810 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ef40999f0 0x7f2ef4074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ef40999f0 0x7f2ef4074920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37306/0 (socket says 172.21.15.44:37306) 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eebe00640 1 -- 172.21.15.44:0/4166817975 learned_addr learned my addr 172.21.15.44:0/4166817975 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef1e00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ed8004810 0x7f2ef4154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef2800640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ef415a810 0x7f2ef415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eebe00640 1 -- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ef415a810 msgr2=0x7f2ef415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eebe00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ef415a810 0x7f2ef415aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eebe00640 1 -- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ed8004810 msgr2=0x7f2ef4154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eebe00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ed8004810 0x7f2ef4154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eebe00640 1 -- 172.21.15.44:0/4166817975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef41f5b00 con 0x7f2ef40999f0 2024-09-13T08:40:43.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eebe00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ef40999f0 0x7f2ef4074920 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2edc01b5a0 tx=0x7f2edc001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:43.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eea000640 1 -- 172.21.15.44:0/4166817975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0060c0 con 0x7f2ef40999f0 2024-09-13T08:40:43.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eea000640 1 -- 172.21.15.44:0/4166817975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2edc003600 con 0x7f2ef40999f0 2024-09-13T08:40:43.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef8a00640 1 -- 172.21.15.44:0/4166817975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef4074e60 con 0x7f2ef40999f0 2024-09-13T08:40:43.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eea000640 1 -- 172.21.15.44:0/4166817975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc003a10 con 0x7f2ef40999f0 2024-09-13T08:40:43.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef8a00640 1 -- 172.21.15.44:0/4166817975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef41f4dd0 con 0x7f2ef40999f0 2024-09-13T08:40:43.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2ef8a00640 1 -- 172.21.15.44:0/4166817975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef4154c40 con 0x7f2ef40999f0 2024-09-13T08:40:43.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.372+0000 7f2eea000640 1 -- 172.21.15.44:0/4166817975 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f2edc0bf060 con 0x7f2ef40999f0 2024-09-13T08:40:43.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.376+0000 7f2eea000640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ed0073250 0x7f2ed0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:43.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.376+0000 7f2eea000640 1 -- 172.21.15.44:0/4166817975 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f2edc12e890 con 0x7f2ef40999f0 2024-09-13T08:40:43.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.376+0000 7f2ef1e00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ed0073250 0x7f2ed0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:43.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.376+0000 7f2ef1e00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ed0073250 0x7f2ed0075710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4000c10 tx=0x7f2ee4006b20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:43.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.380+0000 7f2eea000640 1 -- 172.21.15.44:0/4166817975 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2edc016c50 con 0x7f2ef40999f0 2024-09-13T08:40:43.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.556+0000 7f2ef8a00640 1 -- 172.21.15.44:0/4166817975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ef40cea80 con 0x7f2ef40999f0 2024-09-13T08:40:43.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.556+0000 7f2eea000640 1 -- 172.21.15.44:0/4166817975 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16694 (secure 0 0 0) 0x7f2edc006260 con 0x7f2ef40999f0 2024-09-13T08:40:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:43.560 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":46,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:40:42.589683+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:40:40.291993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T08:40:41.533879+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:40:43.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 -- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ed0073250 msgr2=0x7f2ed0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ed0073250 0x7f2ed0075710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4000c10 tx=0x7f2ee4006b20 comp rx=0 tx=0).stop 2024-09-13T08:40:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 -- 172.21.15.44:0/4166817975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ef40999f0 msgr2=0x7f2ef4074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ef40999f0 0x7f2ef4074920 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2edc01b5a0 tx=0x7f2edc001c30 comp rx=0 tx=0).stop 2024-09-13T08:40:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 -- 172.21.15.44:0/4166817975 shutdown_connections 2024-09-13T08:40:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ed0073250 0x7f2ed0075710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ef415a810 0x7f2ef415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ef40999f0 0x7f2ef4074920 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 --2- 172.21.15.44:0/4166817975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ed8004810 0x7f2ef4154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 -- 172.21.15.44:0/4166817975 >> 172.21.15.44:0/4166817975 conn(0x7f2ef40b1f10 msgr2=0x7f2ef40cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 -- 172.21.15.44:0/4166817975 shutdown_connections 2024-09-13T08:40:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.560+0000 7f2ef8a00640 1 -- 172.21.15.44:0/4166817975 wait complete. 2024-09-13T08:40:43.582 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:40:43.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.724+0000 7fa9fee00640 1 -- 172.21.15.44:0/3564624217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80b2760 msgr2=0x7fa9f80d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.724+0000 7fa9fee00640 1 --2- 172.21.15.44:0/3564624217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80b2760 0x7fa9f80d0260 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009840 tx=0x7fa9e001c790 comp rx=0 tx=0).stop 2024-09-13T08:40:43.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.724+0000 7fa9fee00640 1 -- 172.21.15.44:0/3564624217 shutdown_connections 2024-09-13T08:40:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.724+0000 7fa9fee00640 1 --2- 172.21.15.44:0/3564624217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80b2760 0x7fa9f80d0260 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.724+0000 7fa9fee00640 1 -- 172.21.15.44:0/3564624217 >> 172.21.15.44:0/3564624217 conn(0x7fa9f80d1e40 msgr2=0x7fa9f80d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.724+0000 7fa9fee00640 1 -- 172.21.15.44:0/3564624217 shutdown_connections 2024-09-13T08:40:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.724+0000 7fa9fee00640 1 -- 172.21.15.44:0/3564624217 wait complete. 2024-09-13T08:40:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 Processor -- start 2024-09-13T08:40:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 -- start start 2024-09-13T08:40:43.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa9f80b2760 0x7fa9f80b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:43.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80b2d00 0x7fa9f80b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:43.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9f80cfe80 0x7fa9f8158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:43.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f816a2f0 con 0x7fa9f80b2d00 2024-09-13T08:40:43.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f816a170 con 0x7fa9f80cfe80 2024-09-13T08:40:43.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f816a470 con 0x7fa9f80b2760 2024-09-13T08:40:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80b2d00 0x7fa9f80b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80b2d00 0x7fa9f80b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37332/0 (socket says 172.21.15.44:37332) 2024-09-13T08:40:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9efe00640 1 -- 172.21.15.44:0/4226423386 learned_addr learned my addr 172.21.15.44:0/4226423386 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fce00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9f80cfe80 0x7fa9f8158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9f7e00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa9f80b2760 0x7fa9f80b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9efe00640 1 -- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa9f80b2760 msgr2=0x7fa9f80b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9efe00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa9f80b2760 0x7fa9f80b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9efe00640 1 -- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9f80cfe80 msgr2=0x7fa9f8158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9efe00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9f80cfe80 0x7fa9f8158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9efe00640 1 -- 172.21.15.44:0/4226423386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f80c6eb0 con 0x7fa9f80b2d00 2024-09-13T08:40:43.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9efe00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80b2d00 0x7fa9f80b1830 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa9e001cc70 tx=0x7fa9e00a65c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:43.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9f4c00640 1 -- 172.21.15.44:0/4226423386 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00be070 con 0x7fa9f80b2d00 2024-09-13T08:40:43.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9f4c00640 1 -- 172.21.15.44:0/4226423386 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9e00a6630 con 0x7fa9f80b2d00 2024-09-13T08:40:43.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9f4c00640 1 -- 172.21.15.44:0/4226423386 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0005260 con 0x7fa9f80b2d00 2024-09-13T08:40:43.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 -- 172.21.15.44:0/4226423386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f81fb770 con 0x7fa9f80b2d00 2024-09-13T08:40:43.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 -- 172.21.15.44:0/4226423386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f81fbd30 con 0x7fa9f80b2d00 2024-09-13T08:40:43.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.728+0000 7fa9fee00640 1 -- 172.21.15.44:0/4226423386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f80d0260 con 0x7fa9f80b2d00 2024-09-13T08:40:43.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.732+0000 7fa9f4c00640 1 -- 172.21.15.44:0/4226423386 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fa9e0005400 con 0x7fa9f80b2d00 2024-09-13T08:40:43.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.732+0000 7fa9f4c00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9d4073220 0x7fa9d40756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:43.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.732+0000 7fa9f4c00640 1 -- 172.21.15.44:0/4226423386 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fa9e012e330 con 0x7fa9f80b2d00 2024-09-13T08:40:43.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.732+0000 7fa9f7e00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9d4073220 0x7fa9d40756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:43.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.732+0000 7fa9f7e00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9d4073220 0x7fa9d40756e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8000f70 tx=0x7fa9e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:43.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.736+0000 7fa9f4c00640 1 -- 172.21.15.44:0/4226423386 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa9e00ac030 con 0x7fa9f80b2d00 2024-09-13T08:40:43.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.912+0000 7fa9fee00640 1 -- 172.21.15.44:0/4226423386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9f815b0e0 con 0x7fa9f80b2d00 2024-09-13T08:40:43.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.912+0000 7fa9f4c00640 1 -- 172.21.15.44:0/4226423386 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16694 (secure 0 0 0) 0x7fa9e0016410 con 0x7fa9f80b2d00 2024-09-13T08:40:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:43.917 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":46,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:40:42.589683+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:40:40.291993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T08:40:41.533879+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:40:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 -- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9d4073220 msgr2=0x7fa9d40756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9d4073220 0x7fa9d40756e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8000f70 tx=0x7fa9e8009040 comp rx=0 tx=0).stop 2024-09-13T08:40:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 -- 172.21.15.44:0/4226423386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80b2d00 msgr2=0x7fa9f80b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80b2d00 0x7fa9f80b1830 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa9e001cc70 tx=0x7fa9e00a65c0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 -- 172.21.15.44:0/4226423386 shutdown_connections 2024-09-13T08:40:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9d4073220 0x7fa9d40756e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9f80cfe80 0x7fa9f8158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80b2d00 0x7fa9f80b1830 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 --2- 172.21.15.44:0/4226423386 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa9f80b2760 0x7fa9f80b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 -- 172.21.15.44:0/4226423386 >> 172.21.15.44:0/4226423386 conn(0x7fa9f80d1e40 msgr2=0x7fa9f80cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 -- 172.21.15.44:0/4226423386 shutdown_connections 2024-09-13T08:40:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:43.916+0000 7fa9fee00640 1 -- 172.21.15.44:0/4226423386 wait complete. 2024-09-13T08:40:43.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T08:40:44.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.088+0000 7f1f7b600640 1 -- 172.21.15.44:0/3133659432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f740b1650 msgr2=0x7f1f740b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:44.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.088+0000 7f1f7b600640 1 --2- 172.21.15.44:0/3133659432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f740b1650 0x7f1f740b1a30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009900 tx=0x7f1f6001c8e0 comp rx=0 tx=0).stop 2024-09-13T08:40:44.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.088+0000 7f1f7b600640 1 -- 172.21.15.44:0/3133659432 shutdown_connections 2024-09-13T08:40:44.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.088+0000 7f1f7b600640 1 --2- 172.21.15.44:0/3133659432 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f740afb10 0x7f1f740affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:44.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.088+0000 7f1f7b600640 1 --2- 172.21.15.44:0/3133659432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f740b1650 0x7f1f740b1a30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:44.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.088+0000 7f1f7b600640 1 -- 172.21.15.44:0/3133659432 >> 172.21.15.44:0/3133659432 conn(0x7f1f740d1dd0 msgr2=0x7f1f740d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:44.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.088+0000 7f1f7b600640 1 -- 172.21.15.44:0/3133659432 shutdown_connections 2024-09-13T08:40:44.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.088+0000 7f1f7b600640 1 -- 172.21.15.44:0/3133659432 wait complete. 2024-09-13T08:40:44.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.088+0000 7f1f7b600640 1 Processor -- start 2024-09-13T08:40:44.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.088+0000 7f1f7b600640 1 -- start start 2024-09-13T08:40:44.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f7b600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f740afb10 0x7f1f7415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:44.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f7b600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f740b1650 0x7f1f74166780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f7b600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f741605d0 0x7f1f7414d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f7b600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f740b0ec0 con 0x7f1f740b1650 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f7b600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f740b1430 con 0x7f1f741605d0 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f7b600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1f74169540 con 0x7f1f740afb10 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f73e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f740b1650 0x7f1f74166780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f73e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f740b1650 0x7f1f74166780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37340/0 (socket says 172.21.15.44:37340) 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f73e00640 1 -- 172.21.15.44:0/1452084779 learned_addr learned my addr 172.21.15.44:0/1452084779 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f79600640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f741605d0 0x7f1f7414d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f78c00640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f740afb10 0x7f1f7415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f73e00640 1 -- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f740afb10 msgr2=0x7f1f7415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f73e00640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f740afb10 0x7f1f7415a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f73e00640 1 -- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f741605d0 msgr2=0x7f1f7414d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:44.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f73e00640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f741605d0 0x7f1f7414d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:44.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f73e00640 1 -- 172.21.15.44:0/1452084779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f740c83f0 con 0x7f1f740b1650 2024-09-13T08:40:44.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f73e00640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f740b1650 0x7f1f74166780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009850 tx=0x7f1f600a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:44.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f70c00640 1 -- 172.21.15.44:0/1452084779 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600b2040 con 0x7f1f740b1650 2024-09-13T08:40:44.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f70c00640 1 -- 172.21.15.44:0/1452084779 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f60004470 con 0x7f1f740b1650 2024-09-13T08:40:44.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f70c00640 1 -- 172.21.15.44:0/1452084779 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60005290 con 0x7f1f740b1650 2024-09-13T08:40:44.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f7b600640 1 -- 172.21.15.44:0/1452084779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f740c8690 con 0x7f1f740b1650 2024-09-13T08:40:44.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f7b600640 1 -- 172.21.15.44:0/1452084779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f740b6040 con 0x7f1f740b1650 2024-09-13T08:40:44.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.092+0000 7f1f7b600640 1 -- 172.21.15.44:0/1452084779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f740affc0 con 0x7f1f740b1650 2024-09-13T08:40:44.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.096+0000 7f1f70c00640 1 -- 172.21.15.44:0/1452084779 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f1f60005430 con 0x7f1f740b1650 2024-09-13T08:40:44.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.096+0000 7f1f70c00640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f54073220 0x7f1f540756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:44.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.096+0000 7f1f70c00640 1 -- 172.21.15.44:0/1452084779 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1f6012d220 con 0x7f1f740b1650 2024-09-13T08:40:44.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.096+0000 7f1f78c00640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f54073220 0x7f1f540756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:44.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.096+0000 7f1f78c00640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f54073220 0x7f1f540756e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f740d2340 tx=0x7f1f64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:44.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.100+0000 7f1f70c00640 1 -- 172.21.15.44:0/1452084779 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1f600164d0 con 0x7f1f740b1650 2024-09-13T08:40:44.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:44.312+0000 7f1f7b600640 1 -- 172.21.15.44:0/1452084779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1f740d1080 con 0x7f1f740b1650 2024-09-13T08:40:45.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.188+0000 7f1f70c00640 1 -- 172.21.15.44:0/1452084779 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f1f600f7850 con 0x7f1f740b1650 2024-09-13T08:40:45.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 -- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f54073220 msgr2=0x7f1f540756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:45.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f54073220 0x7f1f540756e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f740d2340 tx=0x7f1f64009040 comp rx=0 tx=0).stop 2024-09-13T08:40:45.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 -- 172.21.15.44:0/1452084779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f740b1650 msgr2=0x7f1f74166780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:45.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f740b1650 0x7f1f74166780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009850 tx=0x7f1f600a55a0 comp rx=0 tx=0).stop 2024-09-13T08:40:45.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 -- 172.21.15.44:0/1452084779 shutdown_connections 2024-09-13T08:40:45.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f54073220 0x7f1f540756e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:45.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f741605d0 0x7f1f7414d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:45.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f740b1650 0x7f1f74166780 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:45.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 --2- 172.21.15.44:0/1452084779 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f740afb10 0x7f1f7415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:45.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 -- 172.21.15.44:0/1452084779 >> 172.21.15.44:0/1452084779 conn(0x7f1f740d1dd0 msgr2=0x7f1f7415c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:45.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 -- 172.21.15.44:0/1452084779 shutdown_connections 2024-09-13T08:40:45.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.192+0000 7f1f7b600640 1 -- 172.21.15.44:0/1452084779 wait complete. 2024-09-13T08:40:45.219 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T08:40:45.219 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T08:40:45.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 -- 172.21.15.44:0/1508133053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4004810 msgr2=0x7f2dc4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:45.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 --2- 172.21.15.44:0/1508133053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4004810 0x7f2dc4004c90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8006f30 tx=0x7f2dc801ce40 comp rx=0 tx=0).stop 2024-09-13T08:40:45.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 -- 172.21.15.44:0/1508133053 shutdown_connections 2024-09-13T08:40:45.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 --2- 172.21.15.44:0/1508133053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4004810 0x7f2dc4004c90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:45.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 -- 172.21.15.44:0/1508133053 >> 172.21.15.44:0/1508133053 conn(0x7f2ddc0d0ab0 msgr2=0x7f2ddc0d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:45.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 -- 172.21.15.44:0/1508133053 shutdown_connections 2024-09-13T08:40:45.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 -- 172.21.15.44:0/1508133053 wait complete. 2024-09-13T08:40:45.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 Processor -- start 2024-09-13T08:40:45.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 -- start start 2024-09-13T08:40:45.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4004810 0x7f2ddc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ddc1f8a60 0x7f2ddc0b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ddc0b4fa0 0x7f2ddc158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc0cd2a0 con 0x7f2dc4004810 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc0cd120 con 0x7f2ddc0b4fa0 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de3000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2ddc073630 con 0x7f2ddc1f8a60 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4004810 0x7f2ddc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4004810 0x7f2ddc15d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37370/0 (socket says 172.21.15.44:37370) 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.372+0000 7f2de1c00640 1 -- 172.21.15.44:0/3488793825 learned_addr learned my addr 172.21.15.44:0/3488793825 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de1200640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ddc1f8a60 0x7f2ddc0b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de2600640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ddc0b4fa0 0x7f2ddc158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de1c00640 1 -- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ddc1f8a60 msgr2=0x7f2ddc0b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de1c00640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ddc1f8a60 0x7f2ddc0b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de1c00640 1 -- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ddc0b4fa0 msgr2=0x7f2ddc158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de1c00640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ddc0b4fa0 0x7f2ddc158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de1c00640 1 -- 172.21.15.44:0/3488793825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ddc1f7fc0 con 0x7f2dc4004810 2024-09-13T08:40:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de2600640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ddc0b4fa0 0x7f2ddc158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de1c00640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4004810 0x7f2ddc15d040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0005e90 tx=0x7f2dd0011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2dd9600640 1 -- 172.21.15.44:0/3488793825 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0005320 con 0x7f2dc4004810 2024-09-13T08:40:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2dd9600640 1 -- 172.21.15.44:0/3488793825 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dd0018020 con 0x7f2dc4004810 2024-09-13T08:40:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2dd9600640 1 -- 172.21.15.44:0/3488793825 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0003860 con 0x7f2dc4004810 2024-09-13T08:40:45.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de3000640 1 -- 172.21.15.44:0/3488793825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ddc0b54b0 con 0x7f2dc4004810 2024-09-13T08:40:45.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de3000640 1 -- 172.21.15.44:0/3488793825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ddc0b5790 con 0x7f2dc4004810 2024-09-13T08:40:45.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2de3000640 1 -- 172.21.15.44:0/3488793825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ddc0ce890 con 0x7f2dc4004810 2024-09-13T08:40:45.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.376+0000 7f2dd9600640 1 -- 172.21.15.44:0/3488793825 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f2dd0003a00 con 0x7f2dc4004810 2024-09-13T08:40:45.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.380+0000 7f2dd9600640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dbc073250 0x7f2dbc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:45.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.380+0000 7f2dd9600640 1 -- 172.21.15.44:0/3488793825 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f2dd0097bc0 con 0x7f2dc4004810 2024-09-13T08:40:45.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.380+0000 7f2de1200640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dbc073250 0x7f2dbc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:45.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.380+0000 7f2de1200640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dbc073250 0x7f2dbc075710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80ac000 tx=0x7f2dc8005040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:45.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.384+0000 7f2dd9600640 1 -- 172.21.15.44:0/3488793825 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2dd0022020 con 0x7f2dc4004810 2024-09-13T08:40:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:45.592+0000 7f2de3000640 1 -- 172.21.15.44:0/3488793825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2ddc0afdd0 con 0x7f2dc4004810 2024-09-13T08:40:46.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.192+0000 7f2dd9600640 1 -- 172.21.15.44:0/3488793825 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f2dd0067e20 con 0x7f2dc4004810 2024-09-13T08:40:46.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 -- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dbc073250 msgr2=0x7f2dbc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dbc073250 0x7f2dbc075710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80ac000 tx=0x7f2dc8005040 comp rx=0 tx=0).stop 2024-09-13T08:40:46.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 -- 172.21.15.44:0/3488793825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4004810 msgr2=0x7f2ddc15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4004810 0x7f2ddc15d040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0005e90 tx=0x7f2dd0011740 comp rx=0 tx=0).stop 2024-09-13T08:40:46.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 -- 172.21.15.44:0/3488793825 shutdown_connections 2024-09-13T08:40:46.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2dbc073250 0x7f2dbc075710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ddc0b4fa0 0x7f2ddc158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ddc1f8a60 0x7f2ddc0b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 --2- 172.21.15.44:0/3488793825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4004810 0x7f2ddc15d040 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 -- 172.21.15.44:0/3488793825 >> 172.21.15.44:0/3488793825 conn(0x7f2ddc0d0ab0 msgr2=0x7f2ddc0b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:46.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 -- 172.21.15.44:0/3488793825 shutdown_connections 2024-09-13T08:40:46.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.196+0000 7f2de3000640 1 -- 172.21.15.44:0/3488793825 wait complete. 2024-09-13T08:40:46.222 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T08:40:46.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:40:46.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 -- 172.21.15.44:0/3718536148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb8004d50 msgr2=0x7fceb80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 --2- 172.21.15.44:0/3718536148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb8004d50 0x7fceb80051d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009ee0 tx=0x7fcecc01d5f0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced3200640 1 -- 172.21.15.44:0/3718536148 reap_dead start 2024-09-13T08:40:46.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 -- 172.21.15.44:0/3718536148 shutdown_connections 2024-09-13T08:40:46.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 -- 172.21.15.44:0/3718536148 >> 172.21.15.44:0/3718536148 conn(0x7fced40d1dc0 msgr2=0x7fced40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:46.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 -- 172.21.15.44:0/3718536148 shutdown_connections 2024-09-13T08:40:46.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 -- 172.21.15.44:0/3718536148 wait complete. 2024-09-13T08:40:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 Processor -- start 2024-09-13T08:40:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 -- start start 2024-09-13T08:40:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb8004d50 0x7fced4159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fced41f5c40 0x7fced4155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fced41f2df0 0x7fced41f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:46.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fced415c450 con 0x7fceb8004d50 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fced40d2af0 con 0x7fced41f2df0 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced9400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fced40af0c0 con 0x7fced41f5c40 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb8004d50 0x7fced4159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb8004d50 0x7fced4159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37402/0 (socket says 172.21.15.44:37402) 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced2800640 1 -- 172.21.15.44:0/2488021268 learned_addr learned my addr 172.21.15.44:0/2488021268 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fcec9e00640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fced41f5c40 0x7fced4155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced3200640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fced41f2df0 0x7fced41f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced2800640 1 -- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fced41f5c40 msgr2=0x7fced4155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced2800640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fced41f5c40 0x7fced4155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced2800640 1 -- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fced41f2df0 msgr2=0x7fced41f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced2800640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fced41f2df0 0x7fced41f3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced2800640 1 -- 172.21.15.44:0/2488021268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced41f6420 con 0x7fceb8004d50 2024-09-13T08:40:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.372+0000 7fced2800640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb8004d50 0x7fced4159370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcec4006680 tx=0x7fcec40039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.376+0000 7fcecb400640 1 -- 172.21.15.44:0/2488021268 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4015070 con 0x7fceb8004d50 2024-09-13T08:40:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.376+0000 7fcecb400640 1 -- 172.21.15.44:0/2488021268 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcec4011900 con 0x7fceb8004d50 2024-09-13T08:40:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.376+0000 7fced9400640 1 -- 172.21.15.44:0/2488021268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced41f6700 con 0x7fceb8004d50 2024-09-13T08:40:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.376+0000 7fcecb400640 1 -- 172.21.15.44:0/2488021268 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4011d50 con 0x7fceb8004d50 2024-09-13T08:40:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.376+0000 7fced9400640 1 -- 172.21.15.44:0/2488021268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced4155950 con 0x7fceb8004d50 2024-09-13T08:40:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.376+0000 7fced9400640 1 -- 172.21.15.44:0/2488021268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced40cfcf0 con 0x7fceb8004d50 2024-09-13T08:40:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.376+0000 7fcecb400640 1 -- 172.21.15.44:0/2488021268 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fcec4018070 con 0x7fceb8004d50 2024-09-13T08:40:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.376+0000 7fcecb400640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fceb0073220 0x7fceb00756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.376+0000 7fcecb400640 1 -- 172.21.15.44:0/2488021268 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fcec40952c0 con 0x7fceb8004d50 2024-09-13T08:40:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.376+0000 7fcec9e00640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fceb0073220 0x7fceb00756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.380+0000 7fcec9e00640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fceb0073220 0x7fceb00756e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcebc000c10 tx=0x7fcebc006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.380+0000 7fcecb400640 1 -- 172.21.15.44:0/2488021268 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcec405ff40 con 0x7fceb8004d50 2024-09-13T08:40:46.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.592+0000 7fced9400640 1 -- 172.21.15.44:0/2488021268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fced41f2690 con 0x7fceb8004d50 2024-09-13T08:40:46.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.592+0000 7fcecb400640 1 -- 172.21.15.44:0/2488021268 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fcec405f840 con 0x7fceb8004d50 2024-09-13T08:40:46.597 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:46.597 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"btime":"2024-09-13T08:40:46:193076+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3367280951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3367280951},{"type":"v1","addr":"172.21.15.129:6835","nonce":3367280951}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1057940180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1057940180},{"type":"v1","addr":"172.21.15.129:6837","nonce":1057940180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5319,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1413618927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413618927},{"type":"v1","addr":"172.21.15.44:6834","nonce":1413618927}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:40:42.589651+0000","modified":"2024-09-13T08:40:45.596647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":4,"addr":"172.21.15.44:6835/3141962538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3141962538},{"type":"v1","addr":"172.21.15.44:6835","nonce":3141962538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5322,"qdb_cluster":[5322]},"id":3}]} 2024-09-13T08:40:46.598 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-09-13T08:40:46.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 -- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fceb0073220 msgr2=0x7fceb00756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fceb0073220 0x7fceb00756e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcebc000c10 tx=0x7fcebc006b80 comp rx=0 tx=0).stop 2024-09-13T08:40:46.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 -- 172.21.15.44:0/2488021268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb8004d50 msgr2=0x7fced4159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb8004d50 0x7fced4159370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcec4006680 tx=0x7fcec40039d0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 -- 172.21.15.44:0/2488021268 shutdown_connections 2024-09-13T08:40:46.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fceb0073220 0x7fceb00756e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fced41f2df0 0x7fced41f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fced41f5c40 0x7fced4155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 --2- 172.21.15.44:0/2488021268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb8004d50 0x7fced4159370 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 -- 172.21.15.44:0/2488021268 >> 172.21.15.44:0/2488021268 conn(0x7fced40d1dc0 msgr2=0x7fced415a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:46.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 -- 172.21.15.44:0/2488021268 shutdown_connections 2024-09-13T08:40:46.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.596+0000 7fced9400640 1 -- 172.21.15.44:0/2488021268 wait complete. 2024-09-13T08:40:46.619 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:40:46.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.760+0000 7f5cffc00640 1 -- 172.21.15.44:0/3692851980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cf815bea0 msgr2=0x7f5cf80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.760+0000 7f5cffc00640 1 --2- 172.21.15.44:0/3692851980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cf815bea0 0x7f5cf80ce670 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8073d10 tx=0x7f5cec01c830 comp rx=0 tx=0).stop 2024-09-13T08:40:46.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.760+0000 7f5cffc00640 1 -- 172.21.15.44:0/3692851980 shutdown_connections 2024-09-13T08:40:46.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.760+0000 7f5cffc00640 1 --2- 172.21.15.44:0/3692851980 >> v2:172.21.15.129:3300/0 conn(0x7f5cf80cdb30 0x7f5cf80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.760+0000 7f5cffc00640 1 --2- 172.21.15.44:0/3692851980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cf815bea0 0x7f5cf80ce670 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.760+0000 7f5cffc00640 1 -- 172.21.15.44:0/3692851980 >> 172.21.15.44:0/3692851980 conn(0x7f5cf80d1dc0 msgr2=0x7f5cf80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:46.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.760+0000 7f5cffc00640 1 -- 172.21.15.44:0/3692851980 shutdown_connections 2024-09-13T08:40:46.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.760+0000 7f5cffc00640 1 -- 172.21.15.44:0/3692851980 wait complete. 2024-09-13T08:40:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cffc00640 1 Processor -- start 2024-09-13T08:40:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cffc00640 1 -- start start 2024-09-13T08:40:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cffc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5cf80cdb30 0x7f5cf8146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cffc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cf80d28e0 0x7f5cf80b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cffc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cf815bea0 0x7f5cf80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cffc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8168fc0 con 0x7f5cf815bea0 2024-09-13T08:40:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cffc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8168e40 con 0x7f5cf80d28e0 2024-09-13T08:40:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cffc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf8169140 con 0x7f5cf80cdb30 2024-09-13T08:40:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cfdc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cf815bea0 0x7f5cf80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cfd200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5cf80cdb30 0x7f5cf8146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cfdc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cf815bea0 0x7f5cf80c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37420/0 (socket says 172.21.15.44:37420) 2024-09-13T08:40:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cfdc00640 1 -- 172.21.15.44:0/2498494875 learned_addr learned my addr 172.21.15.44:0/2498494875 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:46.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cf7e00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cf80d28e0 0x7f5cf80b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:46.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cfdc00640 1 -- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5cf80cdb30 msgr2=0x7f5cf8146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cfdc00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5cf80cdb30 0x7f5cf8146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cfdc00640 1 -- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cf80d28e0 msgr2=0x7f5cf80b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cfdc00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cf80d28e0 0x7f5cf80b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cfdc00640 1 -- 172.21.15.44:0/2498494875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf80b7c90 con 0x7f5cf815bea0 2024-09-13T08:40:46.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cfdc00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cf815bea0 0x7f5cf80c7760 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000c7c0 tx=0x7f5cf000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:46.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cf4c00640 1 -- 172.21.15.44:0/2498494875 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0003b80 con 0x7f5cf815bea0 2024-09-13T08:40:46.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cf4c00640 1 -- 172.21.15.44:0/2498494875 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cf0003d20 con 0x7f5cf815bea0 2024-09-13T08:40:46.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cffc00640 1 -- 172.21.15.44:0/2498494875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf80b1840 con 0x7f5cf815bea0 2024-09-13T08:40:46.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cf4c00640 1 -- 172.21.15.44:0/2498494875 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000aab0 con 0x7f5cf815bea0 2024-09-13T08:40:46.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.764+0000 7f5cffc00640 1 -- 172.21.15.44:0/2498494875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf814a4f0 con 0x7f5cf815bea0 2024-09-13T08:40:46.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.768+0000 7f5cffc00640 1 -- 172.21.15.44:0/2498494875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf80cc9d0 con 0x7f5cf815bea0 2024-09-13T08:40:46.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.768+0000 7f5cf4c00640 1 -- 172.21.15.44:0/2498494875 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f5cf000ac50 con 0x7f5cf815bea0 2024-09-13T08:40:46.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.768+0000 7f5cf4c00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5cd8073220 0x7f5cd80756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:46.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.768+0000 7f5cf4c00640 1 -- 172.21.15.44:0/2498494875 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f5cf0094230 con 0x7f5cf815bea0 2024-09-13T08:40:46.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.768+0000 7f5cfd200640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5cd8073220 0x7f5cd80756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:46.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.768+0000 7f5cfd200640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5cd8073220 0x7f5cd80756e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0af000 tx=0x7f5cec0a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:46.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.772+0000 7f5cf4c00640 1 -- 172.21.15.44:0/2498494875 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5cf005e860 con 0x7f5cf815bea0 2024-09-13T08:40:46.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.952+0000 7f5cffc00640 1 -- 172.21.15.44:0/2498494875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cf815b420 con 0x7f5cf815bea0 2024-09-13T08:40:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.952+0000 7f5cf4c00640 1 -- 172.21.15.44:0/2498494875 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16681 (secure 0 0 0) 0x7f5cf0063290 con 0x7f5cf815bea0 2024-09-13T08:40:46.957 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:46.957 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":47,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:40:44.178078+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:40:40.291993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T08:40:41.533879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:40:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 -- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5cd8073220 msgr2=0x7f5cd80756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5cd8073220 0x7f5cd80756e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0af000 tx=0x7f5cec0a7040 comp rx=0 tx=0).stop 2024-09-13T08:40:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 -- 172.21.15.44:0/2498494875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cf815bea0 msgr2=0x7f5cf80c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cf815bea0 0x7f5cf80c7760 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000c7c0 tx=0x7f5cf000cc80 comp rx=0 tx=0).stop 2024-09-13T08:40:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 -- 172.21.15.44:0/2498494875 shutdown_connections 2024-09-13T08:40:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5cd8073220 0x7f5cd80756e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cf815bea0 0x7f5cf80c7760 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cf80d28e0 0x7f5cf80b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 --2- 172.21.15.44:0/2498494875 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5cf80cdb30 0x7f5cf8146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 -- 172.21.15.44:0/2498494875 >> 172.21.15.44:0/2498494875 conn(0x7f5cf80d1dc0 msgr2=0x7f5cf80b7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 -- 172.21.15.44:0/2498494875 shutdown_connections 2024-09-13T08:40:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:46.956+0000 7f5cffc00640 1 -- 172.21.15.44:0/2498494875 wait complete. 2024-09-13T08:40:46.979 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:40:46.979 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:40:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.120+0000 7f485e800640 1 -- 172.21.15.44:0/3370354342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f483c0039f0 msgr2=0x7f483c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.120+0000 7f485e800640 1 --2- 172.21.15.44:0/3370354342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f483c0039f0 0x7f483c003dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4848009b60 tx=0x7f484801d270 comp rx=0 tx=0).stop 2024-09-13T08:40:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.120+0000 7f4857e00640 1 -- 172.21.15.44:0/3370354342 reap_dead start 2024-09-13T08:40:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.120+0000 7f485e800640 1 -- 172.21.15.44:0/3370354342 shutdown_connections 2024-09-13T08:40:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.120+0000 7f485e800640 1 -- 172.21.15.44:0/3370354342 >> 172.21.15.44:0/3370354342 conn(0x7f48580d1dc0 msgr2=0x7f48580d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.120+0000 7f485e800640 1 -- 172.21.15.44:0/3370354342 shutdown_connections 2024-09-13T08:40:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.120+0000 7f485e800640 1 -- 172.21.15.44:0/3370354342 wait complete. 2024-09-13T08:40:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f485e800640 1 Processor -- start 2024-09-13T08:40:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f485e800640 1 -- start start 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f485e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f483c0039f0 0x7f485814f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f485e800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4858165cf0 0x7f48580b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f485e800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f485815d2b0 0x7f48580b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f485e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f485816a280 con 0x7f483c0039f0 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f485e800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f485816a100 con 0x7f4858165cf0 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f485e800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f485816a400 con 0x7f485815d2b0 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f4857400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f483c0039f0 0x7f485814f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f4857400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f483c0039f0 0x7f485814f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37436/0 (socket says 172.21.15.44:37436) 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f4857400640 1 -- 172.21.15.44:0/497192385 learned_addr learned my addr 172.21.15.44:0/497192385 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f484fe00640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4858165cf0 0x7f48580b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f4857e00640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f485815d2b0 0x7f48580b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f4857400640 1 -- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f485815d2b0 msgr2=0x7f48580b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f4857400640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f485815d2b0 0x7f48580b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f4857400640 1 -- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4858165cf0 msgr2=0x7f48580b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f4857400640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4858165cf0 0x7f48580b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f4857400640 1 -- 172.21.15.44:0/497192385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48580b4fd0 con 0x7f483c0039f0 2024-09-13T08:40:47.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f484fe00640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4858165cf0 0x7f48580b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:47.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f4857400640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f483c0039f0 0x7f485814f0f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f484801d0a0 tx=0x7f4848005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:47.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f484f400640 1 -- 172.21.15.44:0/497192385 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48480b1040 con 0x7f483c0039f0 2024-09-13T08:40:47.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f484f400640 1 -- 172.21.15.44:0/497192385 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48480bb070 con 0x7f483c0039f0 2024-09-13T08:40:47.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f485e800640 1 -- 172.21.15.44:0/497192385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48580b52b0 con 0x7f483c0039f0 2024-09-13T08:40:47.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.124+0000 7f484f400640 1 -- 172.21.15.44:0/497192385 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848009730 con 0x7f483c0039f0 2024-09-13T08:40:47.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.128+0000 7f485e800640 1 -- 172.21.15.44:0/497192385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48580b5870 con 0x7f483c0039f0 2024-09-13T08:40:47.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.128+0000 7f485e800640 1 -- 172.21.15.44:0/497192385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48580b5c50 con 0x7f483c0039f0 2024-09-13T08:40:47.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.128+0000 7f484f400640 1 -- 172.21.15.44:0/497192385 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f48480046d0 con 0x7f483c0039f0 2024-09-13T08:40:47.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.128+0000 7f484f400640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4834073220 0x7f48340756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.128+0000 7f484f400640 1 -- 172.21.15.44:0/497192385 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f484812eff0 con 0x7f483c0039f0 2024-09-13T08:40:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.128+0000 7f484fe00640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4834073220 0x7f48340756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.132+0000 7f484fe00640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4834073220 0x7f48340756e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4840001ec0 tx=0x7f4840001e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:47.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.132+0000 7f484f400640 1 -- 172.21.15.44:0/497192385 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f48580b5c50 con 0x7f483c0039f0 2024-09-13T08:40:47.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.304+0000 7f485e800640 1 -- 172.21.15.44:0/497192385 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f48580afcd0 con 0x7f4834073220 2024-09-13T08:40:47.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.312+0000 7f484f400640 1 -- 172.21.15.44:0/497192385 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251545 (secure 0 0 0) 0x7f48580afcd0 con 0x7f4834073220 2024-09-13T08:40:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:47.319 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:40:47.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 -- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4834073220 msgr2=0x7f48340756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4834073220 0x7f48340756e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4840001ec0 tx=0x7f4840001e30 comp rx=0 tx=0).stop 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 -- 172.21.15.44:0/497192385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f483c0039f0 msgr2=0x7f485814f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f483c0039f0 0x7f485814f0f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f484801d0a0 tx=0x7f4848005590 comp rx=0 tx=0).stop 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 -- 172.21.15.44:0/497192385 shutdown_connections 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4834073220 0x7f48340756e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f485815d2b0 0x7f48580b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4858165cf0 0x7f48580b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 --2- 172.21.15.44:0/497192385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f483c0039f0 0x7f485814f0f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 -- 172.21.15.44:0/497192385 >> 172.21.15.44:0/497192385 conn(0x7f48580d1dc0 msgr2=0x7f4858169390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 -- 172.21.15.44:0/497192385 shutdown_connections 2024-09-13T08:40:47.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.316+0000 7f485e800640 1 -- 172.21.15.44:0/497192385 wait complete. 2024-09-13T08:40:47.340 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-13T08:40:46.557861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263220,"kb_used_data":5360,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711500,"statfs":{"total":773094113280,"available":772824576000,"internally_reserved":0,"allocated":5488640,"data_stored":2890737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.403999"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193015+0000","last_change":"2024-09-13T08:40:43.188161+0000","last_active":"2024-09-13T08:40:43.193015+0000","last_peered":"2024-09-13T08:40:43.193015+0000","last_clean":"2024-09-13T08:40:43.193015+0000","last_became_active":"2024-09-13T08:40:43.187905+0000","last_became_peered":"2024-09-13T08:40:43.187905+0000","last_unstale":"2024-09-13T08:40:43.193015+0000","last_undegraded":"2024-09-13T08:40:43.193015+0000","last_fullsized":"2024-09-13T08:40:43.193015+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199380+0000","last_change":"2024-09-13T08:40:43.193493+0000","last_active":"2024-09-13T08:40:43.199380+0000","last_peered":"2024-09-13T08:40:43.199380+0000","last_clean":"2024-09-13T08:40:43.199380+0000","last_became_active":"2024-09-13T08:40:43.193324+0000","last_became_peered":"2024-09-13T08:40:43.193324+0000","last_unstale":"2024-09-13T08:40:43.199380+0000","last_undegraded":"2024-09-13T08:40:43.199380+0000","last_fullsized":"2024-09-13T08:40:43.199380+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207011+0000","last_change":"2024-09-13T08:40:43.192784+0000","last_active":"2024-09-13T08:40:43.207011+0000","last_peered":"2024-09-13T08:40:43.207011+0000","last_clean":"2024-09-13T08:40:43.207011+0000","last_became_active":"2024-09-13T08:40:43.192234+0000","last_became_peered":"2024-09-13T08:40:43.192234+0000","last_unstale":"2024-09-13T08:40:43.207011+0000","last_undegraded":"2024-09-13T08:40:43.207011+0000","last_fullsized":"2024-09-13T08:40:43.207011+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:56:45.062628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193097+0000","last_change":"2024-09-13T08:40:43.189058+0000","last_active":"2024-09-13T08:40:43.193097+0000","last_peered":"2024-09-13T08:40:43.193097+0000","last_clean":"2024-09-13T08:40:43.193097+0000","last_became_active":"2024-09-13T08:40:43.188732+0000","last_became_peered":"2024-09-13T08:40:43.188732+0000","last_unstale":"2024-09-13T08:40:43.193097+0000","last_undegraded":"2024-09-13T08:40:43.193097+0000","last_fullsized":"2024-09-13T08:40:43.193097+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207327+0000","last_change":"2024-09-13T08:40:43.204809+0000","last_active":"2024-09-13T08:40:43.207327+0000","last_peered":"2024-09-13T08:40:43.207327+0000","last_clean":"2024-09-13T08:40:43.207327+0000","last_became_active":"2024-09-13T08:40:43.204599+0000","last_became_peered":"2024-09-13T08:40:43.204599+0000","last_unstale":"2024-09-13T08:40:43.207327+0000","last_undegraded":"2024-09-13T08:40:43.207327+0000","last_fullsized":"2024-09-13T08:40:43.207327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:20.538522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157508+0000","last_change":"2024-09-13T08:40:42.157508+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157508+0000","last_undegraded":"2024-09-13T08:40:42.157508+0000","last_fullsized":"2024-09-13T08:40:42.157508+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189864+0000","last_change":"2024-09-13T08:40:43.184765+0000","last_active":"2024-09-13T08:40:43.189864+0000","last_peered":"2024-09-13T08:40:43.189864+0000","last_clean":"2024-09-13T08:40:43.189864+0000","last_became_active":"2024-09-13T08:40:43.184173+0000","last_became_peered":"2024-09-13T08:40:43.184173+0000","last_unstale":"2024-09-13T08:40:43.189864+0000","last_undegraded":"2024-09-13T08:40:43.189864+0000","last_fullsized":"2024-09-13T08:40:43.189864+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:59.176747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190485+0000","last_change":"2024-09-13T08:40:43.180402+0000","last_active":"2024-09-13T08:40:43.190485+0000","last_peered":"2024-09-13T08:40:43.190485+0000","last_clean":"2024-09-13T08:40:43.190485+0000","last_became_active":"2024-09-13T08:40:43.179682+0000","last_became_peered":"2024-09-13T08:40:43.179682+0000","last_unstale":"2024-09-13T08:40:43.190485+0000","last_undegraded":"2024-09-13T08:40:43.190485+0000","last_fullsized":"2024-09-13T08:40:43.190485+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157861+0000","last_change":"2024-09-13T08:40:42.157861+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157861+0000","last_undegraded":"2024-09-13T08:40:42.157861+0000","last_fullsized":"2024-09-13T08:40:42.157861+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189100+0000","last_change":"2024-09-13T08:40:43.183571+0000","last_active":"2024-09-13T08:40:43.189100+0000","last_peered":"2024-09-13T08:40:43.189100+0000","last_clean":"2024-09-13T08:40:43.189100+0000","last_became_active":"2024-09-13T08:40:43.183297+0000","last_became_peered":"2024-09-13T08:40:43.183297+0000","last_unstale":"2024-09-13T08:40:43.189100+0000","last_undegraded":"2024-09-13T08:40:43.189100+0000","last_fullsized":"2024-09-13T08:40:43.189100+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:14.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206352+0000","last_change":"2024-09-13T08:40:43.200664+0000","last_active":"2024-09-13T08:40:43.206352+0000","last_peered":"2024-09-13T08:40:43.206352+0000","last_clean":"2024-09-13T08:40:43.206352+0000","last_became_active":"2024-09-13T08:40:43.200401+0000","last_became_peered":"2024-09-13T08:40:43.200401+0000","last_unstale":"2024-09-13T08:40:43.206352+0000","last_undegraded":"2024-09-13T08:40:43.206352+0000","last_fullsized":"2024-09-13T08:40:43.206352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:14.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199560+0000","last_change":"2024-09-13T08:40:43.188349+0000","last_active":"2024-09-13T08:40:43.199560+0000","last_peered":"2024-09-13T08:40:43.199560+0000","last_clean":"2024-09-13T08:40:43.199560+0000","last_became_active":"2024-09-13T08:40:43.188067+0000","last_became_peered":"2024-09-13T08:40:43.188067+0000","last_unstale":"2024-09-13T08:40:43.199560+0000","last_undegraded":"2024-09-13T08:40:43.199560+0000","last_fullsized":"2024-09-13T08:40:43.199560+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.158438+0000","last_change":"2024-09-13T08:40:42.158438+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.158438+0000","last_undegraded":"2024-09-13T08:40:42.158438+0000","last_fullsized":"2024-09-13T08:40:42.158438+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193350+0000","last_change":"2024-09-13T08:40:43.187316+0000","last_active":"2024-09-13T08:40:43.193350+0000","last_peered":"2024-09-13T08:40:43.193350+0000","last_clean":"2024-09-13T08:40:43.193350+0000","last_became_active":"2024-09-13T08:40:43.187050+0000","last_became_peered":"2024-09-13T08:40:43.187050+0000","last_unstale":"2024-09-13T08:40:43.193350+0000","last_undegraded":"2024-09-13T08:40:43.193350+0000","last_fullsized":"2024-09-13T08:40:43.193350+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189919+0000","last_change":"2024-09-13T08:40:43.178849+0000","last_active":"2024-09-13T08:40:43.189919+0000","last_peered":"2024-09-13T08:40:43.189919+0000","last_clean":"2024-09-13T08:40:43.189919+0000","last_became_active":"2024-09-13T08:40:43.178581+0000","last_became_peered":"2024-09-13T08:40:43.178581+0000","last_unstale":"2024-09-13T08:40:43.189919+0000","last_undegraded":"2024-09-13T08:40:43.189919+0000","last_fullsized":"2024-09-13T08:40:43.189919+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:08.944968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.182653+0000","last_change":"2024-09-13T08:40:43.178443+0000","last_active":"2024-09-13T08:40:43.182653+0000","last_peered":"2024-09-13T08:40:43.182653+0000","last_clean":"2024-09-13T08:40:43.182653+0000","last_became_active":"2024-09-13T08:40:43.178156+0000","last_became_peered":"2024-09-13T08:40:43.178156+0000","last_unstale":"2024-09-13T08:40:43.182653+0000","last_undegraded":"2024-09-13T08:40:43.182653+0000","last_fullsized":"2024-09-13T08:40:43.182653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.186648+0000","last_change":"2024-09-13T08:40:43.181966+0000","last_active":"2024-09-13T08:40:43.186648+0000","last_peered":"2024-09-13T08:40:43.186648+0000","last_clean":"2024-09-13T08:40:43.186648+0000","last_became_active":"2024-09-13T08:40:43.181651+0000","last_became_peered":"2024-09-13T08:40:43.181651+0000","last_unstale":"2024-09-13T08:40:43.186648+0000","last_undegraded":"2024-09-13T08:40:43.186648+0000","last_fullsized":"2024-09-13T08:40:43.186648+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:33.479797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207483+0000","last_change":"2024-09-13T08:40:43.203044+0000","last_active":"2024-09-13T08:40:43.207483+0000","last_peered":"2024-09-13T08:40:43.207483+0000","last_clean":"2024-09-13T08:40:43.207483+0000","last_became_active":"2024-09-13T08:40:43.196038+0000","last_became_peered":"2024-09-13T08:40:43.196038+0000","last_unstale":"2024-09-13T08:40:43.207483+0000","last_undegraded":"2024-09-13T08:40:43.207483+0000","last_fullsized":"2024-09-13T08:40:43.207483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:49:13.250832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207072+0000","last_change":"2024-09-13T08:40:43.200673+0000","last_active":"2024-09-13T08:40:43.207072+0000","last_peered":"2024-09-13T08:40:43.207072+0000","last_clean":"2024-09-13T08:40:43.207072+0000","last_became_active":"2024-09-13T08:40:43.200103+0000","last_became_peered":"2024-09-13T08:40:43.200103+0000","last_unstale":"2024-09-13T08:40:43.207072+0000","last_undegraded":"2024-09-13T08:40:43.207072+0000","last_fullsized":"2024-09-13T08:40:43.207072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:21.298332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.203522+0000","last_change":"2024-09-13T08:40:43.191159+0000","last_active":"2024-09-13T08:40:43.203522+0000","last_peered":"2024-09-13T08:40:43.203522+0000","last_clean":"2024-09-13T08:40:43.203522+0000","last_became_active":"2024-09-13T08:40:43.190876+0000","last_became_peered":"2024-09-13T08:40:43.190876+0000","last_unstale":"2024-09-13T08:40:43.203522+0000","last_undegraded":"2024-09-13T08:40:43.203522+0000","last_fullsized":"2024-09-13T08:40:43.203522+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:49.726267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190142+0000","last_change":"2024-09-13T08:40:43.176763+0000","last_active":"2024-09-13T08:40:43.190142+0000","last_peered":"2024-09-13T08:40:43.190142+0000","last_clean":"2024-09-13T08:40:43.190142+0000","last_became_active":"2024-09-13T08:40:43.176429+0000","last_became_peered":"2024-09-13T08:40:43.176429+0000","last_unstale":"2024-09-13T08:40:43.190142+0000","last_undegraded":"2024-09-13T08:40:43.190142+0000","last_fullsized":"2024-09-13T08:40:43.190142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:14.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.211767+0000","last_change":"2024-09-13T08:40:43.207935+0000","last_active":"2024-09-13T08:40:43.211767+0000","last_peered":"2024-09-13T08:40:43.211767+0000","last_clean":"2024-09-13T08:40:43.211767+0000","last_became_active":"2024-09-13T08:40:43.207661+0000","last_became_peered":"2024-09-13T08:40:43.207661+0000","last_unstale":"2024-09-13T08:40:43.211767+0000","last_undegraded":"2024-09-13T08:40:43.211767+0000","last_fullsized":"2024-09-13T08:40:43.211767+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:43.583764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189838+0000","last_change":"2024-09-13T08:40:43.184477+0000","last_active":"2024-09-13T08:40:43.189838+0000","last_peered":"2024-09-13T08:40:43.189838+0000","last_clean":"2024-09-13T08:40:43.189838+0000","last_became_active":"2024-09-13T08:40:43.183611+0000","last_became_peered":"2024-09-13T08:40:43.183611+0000","last_unstale":"2024-09-13T08:40:43.189838+0000","last_undegraded":"2024-09-13T08:40:43.189838+0000","last_fullsized":"2024-09-13T08:40:43.189838+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:33:32.215525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.175547+0000","last_change":"2024-09-13T08:40:43.173503+0000","last_active":"2024-09-13T08:40:43.175547+0000","last_peered":"2024-09-13T08:40:43.175547+0000","last_clean":"2024-09-13T08:40:43.175547+0000","last_became_active":"2024-09-13T08:40:43.173209+0000","last_became_peered":"2024-09-13T08:40:43.173209+0000","last_unstale":"2024-09-13T08:40:43.175547+0000","last_undegraded":"2024-09-13T08:40:43.175547+0000","last_fullsized":"2024-09-13T08:40:43.175547+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:00.931558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193040+0000","last_change":"2024-09-13T08:40:43.188762+0000","last_active":"2024-09-13T08:40:43.193040+0000","last_peered":"2024-09-13T08:40:43.193040+0000","last_clean":"2024-09-13T08:40:43.193040+0000","last_became_active":"2024-09-13T08:40:43.188481+0000","last_became_peered":"2024-09-13T08:40:43.188481+0000","last_unstale":"2024-09-13T08:40:43.193040+0000","last_undegraded":"2024-09-13T08:40:43.193040+0000","last_fullsized":"2024-09-13T08:40:43.193040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206308+0000","last_change":"2024-09-13T08:40:43.201185+0000","last_active":"2024-09-13T08:40:43.206308+0000","last_peered":"2024-09-13T08:40:43.206308+0000","last_clean":"2024-09-13T08:40:43.206308+0000","last_became_active":"2024-09-13T08:40:43.200898+0000","last_became_peered":"2024-09-13T08:40:43.200898+0000","last_unstale":"2024-09-13T08:40:43.206308+0000","last_undegraded":"2024-09-13T08:40:43.206308+0000","last_fullsized":"2024-09-13T08:40:43.206308+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:32.631857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212364+0000","last_change":"2024-09-13T08:40:43.208083+0000","last_active":"2024-09-13T08:40:43.212364+0000","last_peered":"2024-09-13T08:40:43.212364+0000","last_clean":"2024-09-13T08:40:43.212364+0000","last_became_active":"2024-09-13T08:40:43.207841+0000","last_became_peered":"2024-09-13T08:40:43.207841+0000","last_unstale":"2024-09-13T08:40:43.212364+0000","last_undegraded":"2024-09-13T08:40:43.212364+0000","last_fullsized":"2024-09-13T08:40:43.212364+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:29:52.299097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207593+0000","last_change":"2024-09-13T08:40:43.198438+0000","last_active":"2024-09-13T08:40:43.207593+0000","last_peered":"2024-09-13T08:40:43.207593+0000","last_clean":"2024-09-13T08:40:43.207593+0000","last_became_active":"2024-09-13T08:40:43.197966+0000","last_became_peered":"2024-09-13T08:40:43.197966+0000","last_unstale":"2024-09-13T08:40:43.207593+0000","last_undegraded":"2024-09-13T08:40:43.207593+0000","last_fullsized":"2024-09-13T08:40:43.207593+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:54:02.077806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197404+0000","last_change":"2024-09-13T08:40:43.191493+0000","last_active":"2024-09-13T08:40:43.197404+0000","last_peered":"2024-09-13T08:40:43.197404+0000","last_clean":"2024-09-13T08:40:43.197404+0000","last_became_active":"2024-09-13T08:40:43.191059+0000","last_became_peered":"2024-09-13T08:40:43.191059+0000","last_unstale":"2024-09-13T08:40:43.197404+0000","last_undegraded":"2024-09-13T08:40:43.197404+0000","last_fullsized":"2024-09-13T08:40:43.197404+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.186580+0000","last_change":"2024-09-13T08:40:43.182477+0000","last_active":"2024-09-13T08:40:43.186580+0000","last_peered":"2024-09-13T08:40:43.186580+0000","last_clean":"2024-09-13T08:40:43.186580+0000","last_became_active":"2024-09-13T08:40:43.182226+0000","last_became_peered":"2024-09-13T08:40:43.182226+0000","last_unstale":"2024-09-13T08:40:43.186580+0000","last_undegraded":"2024-09-13T08:40:43.186580+0000","last_fullsized":"2024-09-13T08:40:43.186580+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157876+0000","last_change":"2024-09-13T08:40:42.157876+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157876+0000","last_undegraded":"2024-09-13T08:40:42.157876+0000","last_fullsized":"2024-09-13T08:40:42.157876+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193060+0000","last_change":"2024-09-13T08:40:43.189479+0000","last_active":"2024-09-13T08:40:43.193060+0000","last_peered":"2024-09-13T08:40:43.193060+0000","last_clean":"2024-09-13T08:40:43.193060+0000","last_became_active":"2024-09-13T08:40:43.189207+0000","last_became_peered":"2024-09-13T08:40:43.189207+0000","last_unstale":"2024-09-13T08:40:43.193060+0000","last_undegraded":"2024-09-13T08:40:43.193060+0000","last_fullsized":"2024-09-13T08:40:43.193060+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:37.772751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197183+0000","last_change":"2024-09-13T08:40:43.193080+0000","last_active":"2024-09-13T08:40:43.197183+0000","last_peered":"2024-09-13T08:40:43.197183+0000","last_clean":"2024-09-13T08:40:43.197183+0000","last_became_active":"2024-09-13T08:40:43.192785+0000","last_became_peered":"2024-09-13T08:40:43.192785+0000","last_unstale":"2024-09-13T08:40:43.197183+0000","last_undegraded":"2024-09-13T08:40:43.197183+0000","last_fullsized":"2024-09-13T08:40:43.197183+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:58.112836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.178584+0000","last_change":"2024-09-13T08:40:42.175868+0000","last_active":"2024-09-13T08:40:42.178584+0000","last_peered":"2024-09-13T08:40:42.178584+0000","last_clean":"2024-09-13T08:40:42.178584+0000","last_became_active":"2024-09-13T08:40:42.175587+0000","last_became_peered":"2024-09-13T08:40:42.175587+0000","last_unstale":"2024-09-13T08:40:42.178584+0000","last_undegraded":"2024-09-13T08:40:42.178584+0000","last_fullsized":"2024-09-13T08:40:42.178584+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:33.183213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198234+0000","last_change":"2024-09-13T08:40:43.194685+0000","last_active":"2024-09-13T08:40:43.198234+0000","last_peered":"2024-09-13T08:40:43.198234+0000","last_clean":"2024-09-13T08:40:43.198234+0000","last_became_active":"2024-09-13T08:40:43.194145+0000","last_became_peered":"2024-09-13T08:40:43.194145+0000","last_unstale":"2024-09-13T08:40:43.198234+0000","last_undegraded":"2024-09-13T08:40:43.198234+0000","last_fullsized":"2024-09-13T08:40:43.198234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:37.772751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206000+0000","last_change":"2024-09-13T08:40:42.182278+0000","last_active":"2024-09-13T08:40:43.206000+0000","last_peered":"2024-09-13T08:40:43.206000+0000","last_clean":"2024-09-13T08:40:43.206000+0000","last_became_active":"2024-09-13T08:40:42.181992+0000","last_became_peered":"2024-09-13T08:40:42.181992+0000","last_unstale":"2024-09-13T08:40:43.206000+0000","last_undegraded":"2024-09-13T08:40:43.206000+0000","last_fullsized":"2024-09-13T08:40:43.206000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:51.100086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199534+0000","last_change":"2024-09-13T08:40:43.187867+0000","last_active":"2024-09-13T08:40:43.199534+0000","last_peered":"2024-09-13T08:40:43.199534+0000","last_clean":"2024-09-13T08:40:43.199534+0000","last_became_active":"2024-09-13T08:40:43.187621+0000","last_became_peered":"2024-09-13T08:40:43.187621+0000","last_unstale":"2024-09-13T08:40:43.199534+0000","last_undegraded":"2024-09-13T08:40:43.199534+0000","last_fullsized":"2024-09-13T08:40:43.199534+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.179942+0000","last_change":"2024-09-13T08:40:42.179208+0000","last_active":"2024-09-13T08:40:42.179942+0000","last_peered":"2024-09-13T08:40:42.179942+0000","last_clean":"2024-09-13T08:40:42.179942+0000","last_became_active":"2024-09-13T08:40:42.178617+0000","last_became_peered":"2024-09-13T08:40:42.178617+0000","last_unstale":"2024-09-13T08:40:42.179942+0000","last_undegraded":"2024-09-13T08:40:42.179942+0000","last_fullsized":"2024-09-13T08:40:42.179942+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:01:20.323316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198747+0000","last_change":"2024-09-13T08:40:43.192425+0000","last_active":"2024-09-13T08:40:43.198747+0000","last_peered":"2024-09-13T08:40:43.198747+0000","last_clean":"2024-09-13T08:40:43.198747+0000","last_became_active":"2024-09-13T08:40:43.191801+0000","last_became_peered":"2024-09-13T08:40:43.191801+0000","last_unstale":"2024-09-13T08:40:43.198747+0000","last_undegraded":"2024-09-13T08:40:43.198747+0000","last_fullsized":"2024-09-13T08:40:43.198747+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:11.173292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205568+0000","last_change":"2024-09-13T08:40:42.180695+0000","last_active":"2024-09-13T08:40:43.205568+0000","last_peered":"2024-09-13T08:40:43.205568+0000","last_clean":"2024-09-13T08:40:43.205568+0000","last_became_active":"2024-09-13T08:40:42.180392+0000","last_became_peered":"2024-09-13T08:40:42.180392+0000","last_unstale":"2024-09-13T08:40:43.205568+0000","last_undegraded":"2024-09-13T08:40:43.205568+0000","last_fullsized":"2024-09-13T08:40:43.205568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:40.439749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212438+0000","last_change":"2024-09-13T08:40:43.209100+0000","last_active":"2024-09-13T08:40:43.212438+0000","last_peered":"2024-09-13T08:40:43.212438+0000","last_clean":"2024-09-13T08:40:43.212438+0000","last_became_active":"2024-09-13T08:40:43.208834+0000","last_became_peered":"2024-09-13T08:40:43.208834+0000","last_unstale":"2024-09-13T08:40:43.212438+0000","last_undegraded":"2024-09-13T08:40:43.212438+0000","last_fullsized":"2024-09-13T08:40:43.212438+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:29:51.212164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.168839+0000","last_change":"2024-09-13T08:40:42.166893+0000","last_active":"2024-09-13T08:40:42.168839+0000","last_peered":"2024-09-13T08:40:42.168839+0000","last_clean":"2024-09-13T08:40:42.168839+0000","last_became_active":"2024-09-13T08:40:42.166507+0000","last_became_peered":"2024-09-13T08:40:42.166507+0000","last_unstale":"2024-09-13T08:40:42.168839+0000","last_undegraded":"2024-09-13T08:40:42.168839+0000","last_fullsized":"2024-09-13T08:40:42.168839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:53:24.395981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205301+0000","last_change":"2024-09-13T08:40:43.193379+0000","last_active":"2024-09-13T08:40:43.205301+0000","last_peered":"2024-09-13T08:40:43.205301+0000","last_clean":"2024-09-13T08:40:43.205301+0000","last_became_active":"2024-09-13T08:40:43.193015+0000","last_became_peered":"2024-09-13T08:40:43.193015+0000","last_unstale":"2024-09-13T08:40:43.205301+0000","last_undegraded":"2024-09-13T08:40:43.205301+0000","last_fullsized":"2024-09-13T08:40:43.205301+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:19.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.201537+0000","last_change":"2024-09-13T08:40:42.184150+0000","last_active":"2024-09-13T08:40:43.201537+0000","last_peered":"2024-09-13T08:40:43.201537+0000","last_clean":"2024-09-13T08:40:43.201537+0000","last_became_active":"2024-09-13T08:40:42.183763+0000","last_became_peered":"2024-09-13T08:40:42.183763+0000","last_unstale":"2024-09-13T08:40:43.201537+0000","last_undegraded":"2024-09-13T08:40:43.201537+0000","last_fullsized":"2024-09-13T08:40:43.201537+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:59.988155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190512+0000","last_change":"2024-09-13T08:40:43.187575+0000","last_active":"2024-09-13T08:40:43.190512+0000","last_peered":"2024-09-13T08:40:43.190512+0000","last_clean":"2024-09-13T08:40:43.190512+0000","last_became_active":"2024-09-13T08:40:43.186758+0000","last_became_peered":"2024-09-13T08:40:43.186758+0000","last_unstale":"2024-09-13T08:40:43.190512+0000","last_undegraded":"2024-09-13T08:40:43.190512+0000","last_fullsized":"2024-09-13T08:40:43.190512+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:55.795423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172776+0000","last_change":"2024-09-13T08:40:42.175361+0000","last_active":"2024-09-13T08:40:43.172776+0000","last_peered":"2024-09-13T08:40:43.172776+0000","last_clean":"2024-09-13T08:40:43.172776+0000","last_became_active":"2024-09-13T08:40:42.174980+0000","last_became_peered":"2024-09-13T08:40:42.174980+0000","last_unstale":"2024-09-13T08:40:43.172776+0000","last_undegraded":"2024-09-13T08:40:43.172776+0000","last_fullsized":"2024-09-13T08:40:43.172776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:59.925397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206690+0000","last_change":"2024-09-13T08:40:43.200353+0000","last_active":"2024-09-13T08:40:43.206690+0000","last_peered":"2024-09-13T08:40:43.206690+0000","last_clean":"2024-09-13T08:40:43.206690+0000","last_became_active":"2024-09-13T08:40:43.198409+0000","last_became_peered":"2024-09-13T08:40:43.198409+0000","last_unstale":"2024-09-13T08:40:43.206690+0000","last_undegraded":"2024-09-13T08:40:43.206690+0000","last_fullsized":"2024-09-13T08:40:43.206690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:39.828271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.178415+0000","last_change":"2024-09-13T08:40:42.178338+0000","last_active":"2024-09-13T08:40:43.178415+0000","last_peered":"2024-09-13T08:40:43.178415+0000","last_clean":"2024-09-13T08:40:43.178415+0000","last_became_active":"2024-09-13T08:40:42.178025+0000","last_became_peered":"2024-09-13T08:40:42.178025+0000","last_unstale":"2024-09-13T08:40:43.178415+0000","last_undegraded":"2024-09-13T08:40:43.178415+0000","last_fullsized":"2024-09-13T08:40:43.178415+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:31.891865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.210052+0000","last_change":"2024-09-13T08:40:43.204789+0000","last_active":"2024-09-13T08:40:43.210052+0000","last_peered":"2024-09-13T08:40:43.210052+0000","last_clean":"2024-09-13T08:40:43.210052+0000","last_became_active":"2024-09-13T08:40:43.204114+0000","last_became_peered":"2024-09-13T08:40:43.204114+0000","last_unstale":"2024-09-13T08:40:43.210052+0000","last_undegraded":"2024-09-13T08:40:43.210052+0000","last_fullsized":"2024-09-13T08:40:43.210052+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:31.140294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.174034+0000","last_change":"2024-09-13T08:40:42.178810+0000","last_active":"2024-09-13T08:40:43.174034+0000","last_peered":"2024-09-13T08:40:43.174034+0000","last_clean":"2024-09-13T08:40:43.174034+0000","last_became_active":"2024-09-13T08:40:42.173486+0000","last_became_peered":"2024-09-13T08:40:42.173486+0000","last_unstale":"2024-09-13T08:40:43.174034+0000","last_undegraded":"2024-09-13T08:40:43.174034+0000","last_fullsized":"2024-09-13T08:40:43.174034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:12.229793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207136+0000","last_change":"2024-09-13T08:40:43.194106+0000","last_active":"2024-09-13T08:40:43.207136+0000","last_peered":"2024-09-13T08:40:43.207136+0000","last_clean":"2024-09-13T08:40:43.207136+0000","last_became_active":"2024-09-13T08:40:43.193804+0000","last_became_peered":"2024-09-13T08:40:43.193804+0000","last_unstale":"2024-09-13T08:40:43.207136+0000","last_undegraded":"2024-09-13T08:40:43.207136+0000","last_fullsized":"2024-09-13T08:40:43.207136+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.417971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.183104+0000","last_change":"2024-09-13T08:40:42.183102+0000","last_active":"2024-09-13T08:40:43.183104+0000","last_peered":"2024-09-13T08:40:43.183104+0000","last_clean":"2024-09-13T08:40:43.183104+0000","last_became_active":"2024-09-13T08:40:42.182748+0000","last_became_peered":"2024-09-13T08:40:42.182748+0000","last_unstale":"2024-09-13T08:40:43.183104+0000","last_undegraded":"2024-09-13T08:40:43.183104+0000","last_fullsized":"2024-09-13T08:40:43.183104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:10.167131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.195996+0000","last_change":"2024-09-13T08:40:43.191524+0000","last_active":"2024-09-13T08:40:43.195996+0000","last_peered":"2024-09-13T08:40:43.195996+0000","last_clean":"2024-09-13T08:40:43.195996+0000","last_became_active":"2024-09-13T08:40:43.191240+0000","last_became_peered":"2024-09-13T08:40:43.191240+0000","last_unstale":"2024-09-13T08:40:43.195996+0000","last_undegraded":"2024-09-13T08:40:43.195996+0000","last_fullsized":"2024-09-13T08:40:43.195996+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:33.508450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.209427+0000","last_change":"2024-09-13T08:40:42.177115+0000","last_active":"2024-09-13T08:40:43.209427+0000","last_peered":"2024-09-13T08:40:43.209427+0000","last_clean":"2024-09-13T08:40:43.209427+0000","last_became_active":"2024-09-13T08:40:42.176751+0000","last_became_peered":"2024-09-13T08:40:42.176751+0000","last_unstale":"2024-09-13T08:40:43.209427+0000","last_undegraded":"2024-09-13T08:40:43.209427+0000","last_fullsized":"2024-09-13T08:40:43.209427+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:23.536012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206408+0000","last_change":"2024-09-13T08:40:43.197281+0000","last_active":"2024-09-13T08:40:43.206408+0000","last_peered":"2024-09-13T08:40:43.206408+0000","last_clean":"2024-09-13T08:40:43.206408+0000","last_became_active":"2024-09-13T08:40:43.196989+0000","last_became_peered":"2024-09-13T08:40:43.196989+0000","last_unstale":"2024-09-13T08:40:43.206408+0000","last_undegraded":"2024-09-13T08:40:43.206408+0000","last_fullsized":"2024-09-13T08:40:43.206408+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:03.667062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.214312+0000","last_change":"2024-09-13T08:40:42.180947+0000","last_active":"2024-09-13T08:40:43.214312+0000","last_peered":"2024-09-13T08:40:43.214312+0000","last_clean":"2024-09-13T08:40:43.214312+0000","last_became_active":"2024-09-13T08:40:42.180647+0000","last_became_peered":"2024-09-13T08:40:42.180647+0000","last_unstale":"2024-09-13T08:40:43.214312+0000","last_undegraded":"2024-09-13T08:40:43.214312+0000","last_fullsized":"2024-09-13T08:40:43.214312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:31.625696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204310+0000","last_change":"2024-09-13T08:40:43.177272+0000","last_active":"2024-09-13T08:40:43.204310+0000","last_peered":"2024-09-13T08:40:43.204310+0000","last_clean":"2024-09-13T08:40:43.204310+0000","last_became_active":"2024-09-13T08:40:43.176928+0000","last_became_peered":"2024-09-13T08:40:43.176928+0000","last_unstale":"2024-09-13T08:40:43.204310+0000","last_undegraded":"2024-09-13T08:40:43.204310+0000","last_fullsized":"2024-09-13T08:40:43.204310+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:19.727346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204104+0000","last_change":"2024-09-13T08:40:42.176291+0000","last_active":"2024-09-13T08:40:43.204104+0000","last_peered":"2024-09-13T08:40:43.204104+0000","last_clean":"2024-09-13T08:40:43.204104+0000","last_became_active":"2024-09-13T08:40:42.175697+0000","last_became_peered":"2024-09-13T08:40:42.175697+0000","last_unstale":"2024-09-13T08:40:43.204104+0000","last_undegraded":"2024-09-13T08:40:43.204104+0000","last_fullsized":"2024-09-13T08:40:43.204104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:18.721185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198468+0000","last_change":"2024-09-13T08:40:43.194019+0000","last_active":"2024-09-13T08:40:43.198468+0000","last_peered":"2024-09-13T08:40:43.198468+0000","last_clean":"2024-09-13T08:40:43.198468+0000","last_became_active":"2024-09-13T08:40:43.192433+0000","last_became_peered":"2024-09-13T08:40:43.192433+0000","last_unstale":"2024-09-13T08:40:43.198468+0000","last_undegraded":"2024-09-13T08:40:43.198468+0000","last_fullsized":"2024-09-13T08:40:43.198468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:49:13.250832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172856+0000","last_change":"2024-09-13T08:40:42.176544+0000","last_active":"2024-09-13T08:40:43.172856+0000","last_peered":"2024-09-13T08:40:43.172856+0000","last_clean":"2024-09-13T08:40:43.172856+0000","last_became_active":"2024-09-13T08:40:42.175781+0000","last_became_peered":"2024-09-13T08:40:42.175781+0000","last_unstale":"2024-09-13T08:40:43.172856+0000","last_undegraded":"2024-09-13T08:40:43.172856+0000","last_fullsized":"2024-09-13T08:40:43.172856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:53.422970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198512+0000","last_change":"2024-09-13T08:40:43.195108+0000","last_active":"2024-09-13T08:40:43.198512+0000","last_peered":"2024-09-13T08:40:43.198512+0000","last_clean":"2024-09-13T08:40:43.198512+0000","last_became_active":"2024-09-13T08:40:43.193623+0000","last_became_peered":"2024-09-13T08:40:43.193623+0000","last_unstale":"2024-09-13T08:40:43.198512+0000","last_undegraded":"2024-09-13T08:40:43.198512+0000","last_fullsized":"2024-09-13T08:40:43.198512+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:31.140294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.185074+0000","last_change":"2024-09-13T08:40:42.184583+0000","last_active":"2024-09-13T08:40:42.185074+0000","last_peered":"2024-09-13T08:40:42.185074+0000","last_clean":"2024-09-13T08:40:42.185074+0000","last_became_active":"2024-09-13T08:40:42.184269+0000","last_became_peered":"2024-09-13T08:40:42.184269+0000","last_unstale":"2024-09-13T08:40:42.185074+0000","last_undegraded":"2024-09-13T08:40:42.185074+0000","last_fullsized":"2024-09-13T08:40:42.185074+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:55.743786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.187184+0000","last_change":"2024-09-13T08:40:43.181834+0000","last_active":"2024-09-13T08:40:43.187184+0000","last_peered":"2024-09-13T08:40:43.187184+0000","last_clean":"2024-09-13T08:40:43.187184+0000","last_became_active":"2024-09-13T08:40:43.181558+0000","last_became_peered":"2024-09-13T08:40:43.181558+0000","last_unstale":"2024-09-13T08:40:43.187184+0000","last_undegraded":"2024-09-13T08:40:43.187184+0000","last_fullsized":"2024-09-13T08:40:43.187184+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:08.944968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213300+0000","last_change":"2024-09-13T08:40:42.173542+0000","last_active":"2024-09-13T08:40:43.213300+0000","last_peered":"2024-09-13T08:40:43.213300+0000","last_clean":"2024-09-13T08:40:43.213300+0000","last_became_active":"2024-09-13T08:40:42.173206+0000","last_became_peered":"2024-09-13T08:40:42.173206+0000","last_unstale":"2024-09-13T08:40:43.213300+0000","last_undegraded":"2024-09-13T08:40:43.213300+0000","last_fullsized":"2024-09-13T08:40:43.213300+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:36.766590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212306+0000","last_change":"2024-09-13T08:40:43.210476+0000","last_active":"2024-09-13T08:40:43.212306+0000","last_peered":"2024-09-13T08:40:43.212306+0000","last_clean":"2024-09-13T08:40:43.212306+0000","last_became_active":"2024-09-13T08:40:43.210279+0000","last_became_peered":"2024-09-13T08:40:43.210279+0000","last_unstale":"2024-09-13T08:40:43.212306+0000","last_undegraded":"2024-09-13T08:40:43.212306+0000","last_fullsized":"2024-09-13T08:40:43.212306+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:03.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.209719+0000","last_change":"2024-09-13T08:40:42.176695+0000","last_active":"2024-09-13T08:40:43.209719+0000","last_peered":"2024-09-13T08:40:43.209719+0000","last_clean":"2024-09-13T08:40:43.209719+0000","last_became_active":"2024-09-13T08:40:42.176387+0000","last_became_peered":"2024-09-13T08:40:42.176387+0000","last_unstale":"2024-09-13T08:40:43.209719+0000","last_undegraded":"2024-09-13T08:40:43.209719+0000","last_fullsized":"2024-09-13T08:40:43.209719+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:23.536012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197389+0000","last_change":"2024-09-13T08:40:43.193233+0000","last_active":"2024-09-13T08:40:43.197389+0000","last_peered":"2024-09-13T08:40:43.197389+0000","last_clean":"2024-09-13T08:40:43.197389+0000","last_became_active":"2024-09-13T08:40:43.192944+0000","last_became_peered":"2024-09-13T08:40:43.192944+0000","last_unstale":"2024-09-13T08:40:43.197389+0000","last_undegraded":"2024-09-13T08:40:43.197389+0000","last_fullsized":"2024-09-13T08:40:43.197389+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.417971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213559+0000","last_change":"2024-09-13T08:40:42.180372+0000","last_active":"2024-09-13T08:40:43.213559+0000","last_peered":"2024-09-13T08:40:43.213559+0000","last_clean":"2024-09-13T08:40:43.213559+0000","last_became_active":"2024-09-13T08:40:42.180079+0000","last_became_peered":"2024-09-13T08:40:42.180079+0000","last_unstale":"2024-09-13T08:40:43.213559+0000","last_undegraded":"2024-09-13T08:40:43.213559+0000","last_fullsized":"2024-09-13T08:40:43.213559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:21.515280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.188208+0000","last_change":"2024-09-13T08:40:43.177434+0000","last_active":"2024-09-13T08:40:43.188208+0000","last_peered":"2024-09-13T08:40:43.188208+0000","last_clean":"2024-09-13T08:40:43.188208+0000","last_became_active":"2024-09-13T08:40:43.177163+0000","last_became_peered":"2024-09-13T08:40:43.177163+0000","last_unstale":"2024-09-13T08:40:43.188208+0000","last_undegraded":"2024-09-13T08:40:43.188208+0000","last_fullsized":"2024-09-13T08:40:43.188208+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:14.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213015+0000","last_change":"2024-09-13T08:40:42.185496+0000","last_active":"2024-09-13T08:40:43.213015+0000","last_peered":"2024-09-13T08:40:43.213015+0000","last_clean":"2024-09-13T08:40:43.213015+0000","last_became_active":"2024-09-13T08:40:42.184959+0000","last_became_peered":"2024-09-13T08:40:42.184959+0000","last_unstale":"2024-09-13T08:40:43.213015+0000","last_undegraded":"2024-09-13T08:40:43.213015+0000","last_fullsized":"2024-09-13T08:40:43.213015+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:18:11.442697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207041+0000","last_change":"2024-09-13T08:40:43.193014+0000","last_active":"2024-09-13T08:40:43.207041+0000","last_peered":"2024-09-13T08:40:43.207041+0000","last_clean":"2024-09-13T08:40:43.207041+0000","last_became_active":"2024-09-13T08:40:43.192539+0000","last_became_peered":"2024-09-13T08:40:43.192539+0000","last_unstale":"2024-09-13T08:40:43.207041+0000","last_undegraded":"2024-09-13T08:40:43.207041+0000","last_fullsized":"2024-09-13T08:40:43.207041+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:58.112836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.188854+0000","last_change":"2024-09-13T08:40:42.188197+0000","last_active":"2024-09-13T08:40:42.188854+0000","last_peered":"2024-09-13T08:40:42.188854+0000","last_clean":"2024-09-13T08:40:42.188854+0000","last_became_active":"2024-09-13T08:40:42.187908+0000","last_became_peered":"2024-09-13T08:40:42.187908+0000","last_unstale":"2024-09-13T08:40:42.188854+0000","last_undegraded":"2024-09-13T08:40:42.188854+0000","last_fullsized":"2024-09-13T08:40:42.188854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:30:57.769839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205398+0000","last_change":"2024-09-13T08:40:43.194030+0000","last_active":"2024-09-13T08:40:43.205398+0000","last_peered":"2024-09-13T08:40:43.205398+0000","last_clean":"2024-09-13T08:40:43.205398+0000","last_became_active":"2024-09-13T08:40:43.193719+0000","last_became_peered":"2024-09-13T08:40:43.193719+0000","last_unstale":"2024-09-13T08:40:43.205398+0000","last_undegraded":"2024-09-13T08:40:43.205398+0000","last_fullsized":"2024-09-13T08:40:43.205398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:54:40.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.210039+0000","last_change":"2024-09-13T08:40:42.170852+0000","last_active":"2024-09-13T08:40:43.210039+0000","last_peered":"2024-09-13T08:40:43.210039+0000","last_clean":"2024-09-13T08:40:43.210039+0000","last_became_active":"2024-09-13T08:40:42.170588+0000","last_became_peered":"2024-09-13T08:40:42.170588+0000","last_unstale":"2024-09-13T08:40:43.210039+0000","last_undegraded":"2024-09-13T08:40:43.210039+0000","last_fullsized":"2024-09-13T08:40:43.210039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:32.502289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157478+0000","last_change":"2024-09-13T08:40:42.157478+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157478+0000","last_undegraded":"2024-09-13T08:40:42.157478+0000","last_fullsized":"2024-09-13T08:40:42.157478+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.175195+0000","last_change":"2024-09-13T08:40:42.175444+0000","last_active":"2024-09-13T08:40:43.175195+0000","last_peered":"2024-09-13T08:40:43.175195+0000","last_clean":"2024-09-13T08:40:43.175195+0000","last_became_active":"2024-09-13T08:40:42.174590+0000","last_became_peered":"2024-09-13T08:40:42.174590+0000","last_unstale":"2024-09-13T08:40:43.175195+0000","last_undegraded":"2024-09-13T08:40:43.175195+0000","last_fullsized":"2024-09-13T08:40:43.175195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:32.473636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190104+0000","last_change":"2024-09-13T08:40:43.177933+0000","last_active":"2024-09-13T08:40:43.190104+0000","last_peered":"2024-09-13T08:40:43.190104+0000","last_clean":"2024-09-13T08:40:43.190104+0000","last_became_active":"2024-09-13T08:40:43.177479+0000","last_became_peered":"2024-09-13T08:40:43.177479+0000","last_unstale":"2024-09-13T08:40:43.190104+0000","last_undegraded":"2024-09-13T08:40:43.190104+0000","last_fullsized":"2024-09-13T08:40:43.190104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:54.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.178449+0000","last_change":"2024-09-13T08:40:42.183091+0000","last_active":"2024-09-13T08:40:43.178449+0000","last_peered":"2024-09-13T08:40:43.178449+0000","last_clean":"2024-09-13T08:40:43.178449+0000","last_became_active":"2024-09-13T08:40:42.182685+0000","last_became_peered":"2024-09-13T08:40:42.182685+0000","last_unstale":"2024-09-13T08:40:43.178449+0000","last_undegraded":"2024-09-13T08:40:43.178449+0000","last_fullsized":"2024-09-13T08:40:43.178449+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:03.155901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.192568+0000","last_change":"2024-09-13T08:40:43.181826+0000","last_active":"2024-09-13T08:40:43.192568+0000","last_peered":"2024-09-13T08:40:43.192568+0000","last_clean":"2024-09-13T08:40:43.192568+0000","last_became_active":"2024-09-13T08:40:43.181516+0000","last_became_peered":"2024-09-13T08:40:43.181516+0000","last_unstale":"2024-09-13T08:40:43.192568+0000","last_undegraded":"2024-09-13T08:40:43.192568+0000","last_fullsized":"2024-09-13T08:40:43.192568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.173833+0000","last_change":"2024-09-13T08:40:42.177701+0000","last_active":"2024-09-13T08:40:43.173833+0000","last_peered":"2024-09-13T08:40:43.173833+0000","last_clean":"2024-09-13T08:40:43.173833+0000","last_became_active":"2024-09-13T08:40:42.177418+0000","last_became_peered":"2024-09-13T08:40:42.177418+0000","last_unstale":"2024-09-13T08:40:43.173833+0000","last_undegraded":"2024-09-13T08:40:43.173833+0000","last_fullsized":"2024-09-13T08:40:43.173833+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:23.873609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206487+0000","last_change":"2024-09-13T08:40:43.198409+0000","last_active":"2024-09-13T08:40:43.206487+0000","last_peered":"2024-09-13T08:40:43.206487+0000","last_clean":"2024-09-13T08:40:43.206487+0000","last_became_active":"2024-09-13T08:40:43.198235+0000","last_became_peered":"2024-09-13T08:40:43.198235+0000","last_unstale":"2024-09-13T08:40:43.206487+0000","last_undegraded":"2024-09-13T08:40:43.206487+0000","last_fullsized":"2024-09-13T08:40:43.206487+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:00:32.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.169231+0000","last_change":"2024-09-13T08:40:42.177210+0000","last_active":"2024-09-13T08:40:43.169231+0000","last_peered":"2024-09-13T08:40:43.169231+0000","last_clean":"2024-09-13T08:40:43.169231+0000","last_became_active":"2024-09-13T08:40:42.176919+0000","last_became_peered":"2024-09-13T08:40:42.176919+0000","last_unstale":"2024-09-13T08:40:43.169231+0000","last_undegraded":"2024-09-13T08:40:43.169231+0000","last_fullsized":"2024-09-13T08:40:43.169231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:05.647017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.203491+0000","last_change":"2024-09-13T08:40:43.189382+0000","last_active":"2024-09-13T08:40:43.203491+0000","last_peered":"2024-09-13T08:40:43.203491+0000","last_clean":"2024-09-13T08:40:43.203491+0000","last_became_active":"2024-09-13T08:40:43.188958+0000","last_became_peered":"2024-09-13T08:40:43.188958+0000","last_unstale":"2024-09-13T08:40:43.203491+0000","last_undegraded":"2024-09-13T08:40:43.203491+0000","last_fullsized":"2024-09-13T08:40:43.203491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:12.306506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.173394+0000","last_change":"2024-09-13T08:40:42.186311+0000","last_active":"2024-09-13T08:40:43.173394+0000","last_peered":"2024-09-13T08:40:43.173394+0000","last_clean":"2024-09-13T08:40:43.173394+0000","last_became_active":"2024-09-13T08:40:42.185993+0000","last_became_peered":"2024-09-13T08:40:42.185993+0000","last_unstale":"2024-09-13T08:40:43.173394+0000","last_undegraded":"2024-09-13T08:40:43.173394+0000","last_fullsized":"2024-09-13T08:40:43.173394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:07.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189794+0000","last_change":"2024-09-13T08:40:43.184101+0000","last_active":"2024-09-13T08:40:43.189794+0000","last_peered":"2024-09-13T08:40:43.189794+0000","last_clean":"2024-09-13T08:40:43.189794+0000","last_became_active":"2024-09-13T08:40:43.183722+0000","last_became_peered":"2024-09-13T08:40:43.183722+0000","last_unstale":"2024-09-13T08:40:43.189794+0000","last_undegraded":"2024-09-13T08:40:43.189794+0000","last_fullsized":"2024-09-13T08:40:43.189794+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:46:44.174973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.181735+0000","last_change":"2024-09-13T08:40:42.189558+0000","last_active":"2024-09-13T08:40:43.181735+0000","last_peered":"2024-09-13T08:40:43.181735+0000","last_clean":"2024-09-13T08:40:43.181735+0000","last_became_active":"2024-09-13T08:40:42.189312+0000","last_became_peered":"2024-09-13T08:40:42.189312+0000","last_unstale":"2024-09-13T08:40:43.181735+0000","last_undegraded":"2024-09-13T08:40:43.181735+0000","last_fullsized":"2024-09-13T08:40:43.181735+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:40.270289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204125+0000","last_change":"2024-09-13T08:40:43.192030+0000","last_active":"2024-09-13T08:40:43.204125+0000","last_peered":"2024-09-13T08:40:43.204125+0000","last_clean":"2024-09-13T08:40:43.204125+0000","last_became_active":"2024-09-13T08:40:43.191644+0000","last_became_peered":"2024-09-13T08:40:43.191644+0000","last_unstale":"2024-09-13T08:40:43.204125+0000","last_undegraded":"2024-09-13T08:40:43.204125+0000","last_fullsized":"2024-09-13T08:40:43.204125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:38.459714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172812+0000","last_change":"2024-09-13T08:40:42.179810+0000","last_active":"2024-09-13T08:40:43.172812+0000","last_peered":"2024-09-13T08:40:43.172812+0000","last_clean":"2024-09-13T08:40:43.172812+0000","last_became_active":"2024-09-13T08:40:42.179512+0000","last_became_peered":"2024-09-13T08:40:42.179512+0000","last_unstale":"2024-09-13T08:40:43.172812+0000","last_undegraded":"2024-09-13T08:40:43.172812+0000","last_fullsized":"2024-09-13T08:40:43.172812+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:13.357647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.188531+0000","last_change":"2024-09-13T08:40:43.176719+0000","last_active":"2024-09-13T08:40:43.188531+0000","last_peered":"2024-09-13T08:40:43.188531+0000","last_clean":"2024-09-13T08:40:43.188531+0000","last_became_active":"2024-09-13T08:40:43.176416+0000","last_became_peered":"2024-09-13T08:40:43.176416+0000","last_unstale":"2024-09-13T08:40:43.188531+0000","last_undegraded":"2024-09-13T08:40:43.188531+0000","last_fullsized":"2024-09-13T08:40:43.188531+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:00.931558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.174570+0000","last_change":"2024-09-13T08:40:42.176816+0000","last_active":"2024-09-13T08:40:43.174570+0000","last_peered":"2024-09-13T08:40:43.174570+0000","last_clean":"2024-09-13T08:40:43.174570+0000","last_became_active":"2024-09-13T08:40:42.176236+0000","last_became_peered":"2024-09-13T08:40:42.176236+0000","last_unstale":"2024-09-13T08:40:43.174570+0000","last_undegraded":"2024-09-13T08:40:43.174570+0000","last_fullsized":"2024-09-13T08:40:43.174570+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:14.430820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204331+0000","last_change":"2024-09-13T08:40:43.176967+0000","last_active":"2024-09-13T08:40:43.204331+0000","last_peered":"2024-09-13T08:40:43.204331+0000","last_clean":"2024-09-13T08:40:43.204331+0000","last_became_active":"2024-09-13T08:40:43.176658+0000","last_became_peered":"2024-09-13T08:40:43.176658+0000","last_unstale":"2024-09-13T08:40:43.204331+0000","last_undegraded":"2024-09-13T08:40:43.204331+0000","last_fullsized":"2024-09-13T08:40:43.204331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:41.276450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.177057+0000","last_change":"2024-09-13T08:40:42.174657+0000","last_active":"2024-09-13T08:40:42.177057+0000","last_peered":"2024-09-13T08:40:42.177057+0000","last_clean":"2024-09-13T08:40:42.177057+0000","last_became_active":"2024-09-13T08:40:42.174374+0000","last_became_peered":"2024-09-13T08:40:42.174374+0000","last_unstale":"2024-09-13T08:40:42.177057+0000","last_undegraded":"2024-09-13T08:40:42.177057+0000","last_fullsized":"2024-09-13T08:40:42.177057+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:53:20.682799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.187158+0000","last_change":"2024-09-13T08:40:43.180762+0000","last_active":"2024-09-13T08:40:43.187158+0000","last_peered":"2024-09-13T08:40:43.187158+0000","last_clean":"2024-09-13T08:40:43.187158+0000","last_became_active":"2024-09-13T08:40:43.180576+0000","last_became_peered":"2024-09-13T08:40:43.180576+0000","last_unstale":"2024-09-13T08:40:43.187158+0000","last_undegraded":"2024-09-13T08:40:43.187158+0000","last_fullsized":"2024-09-13T08:40:43.187158+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:54.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.182098+0000","last_change":"2024-09-13T08:40:42.177987+0000","last_active":"2024-09-13T08:40:43.182098+0000","last_peered":"2024-09-13T08:40:43.182098+0000","last_clean":"2024-09-13T08:40:43.182098+0000","last_became_active":"2024-09-13T08:40:42.177653+0000","last_became_peered":"2024-09-13T08:40:42.177653+0000","last_unstale":"2024-09-13T08:40:43.182098+0000","last_undegraded":"2024-09-13T08:40:43.182098+0000","last_fullsized":"2024-09-13T08:40:43.182098+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:44.296127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190447+0000","last_change":"2024-09-13T08:40:43.180067+0000","last_active":"2024-09-13T08:40:43.190447+0000","last_peered":"2024-09-13T08:40:43.190447+0000","last_clean":"2024-09-13T08:40:43.190447+0000","last_became_active":"2024-09-13T08:40:43.179734+0000","last_became_peered":"2024-09-13T08:40:43.179734+0000","last_unstale":"2024-09-13T08:40:43.190447+0000","last_undegraded":"2024-09-13T08:40:43.190447+0000","last_fullsized":"2024-09-13T08:40:43.190447+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:33.479797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.201678+0000","last_change":"2024-09-13T08:40:42.180201+0000","last_active":"2024-09-13T08:40:43.201678+0000","last_peered":"2024-09-13T08:40:43.201678+0000","last_clean":"2024-09-13T08:40:43.201678+0000","last_became_active":"2024-09-13T08:40:42.179874+0000","last_became_peered":"2024-09-13T08:40:42.179874+0000","last_unstale":"2024-09-13T08:40:43.201678+0000","last_undegraded":"2024-09-13T08:40:43.201678+0000","last_fullsized":"2024-09-13T08:40:43.201678+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:25.549286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.171871+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:40:43.171871+0000","last_peered":"2024-09-13T08:40:43.171871+0000","last_clean":"2024-09-13T08:40:43.171871+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:40:43.171871+0000","last_undegraded":"2024-09-13T08:40:43.171871+0000","last_fullsized":"2024-09-13T08:40:43.171871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":214223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":804579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":552960,"data_stored":215893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":214861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":214279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509484,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":804627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":214861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35860,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":520192,"data_stored":207414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:40:47.345 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:40:47.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.524+0000 7fc332600640 1 -- 172.21.15.44:0/3955095044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c15e2d0 msgr2=0x7fc32c0d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.524+0000 7fc332600640 1 --2- 172.21.15.44:0/3955095044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c15e2d0 0x7fc32c0d0a70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0d2090 tx=0x7fc31c01c7d0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.524+0000 7fc332600640 1 -- 172.21.15.44:0/3955095044 shutdown_connections 2024-09-13T08:40:47.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.524+0000 7fc332600640 1 --2- 172.21.15.44:0/3955095044 >> v2:172.21.15.129:3300/0 conn(0x7fc32c0b7a30 0x7fc32c0b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.524+0000 7fc332600640 1 --2- 172.21.15.44:0/3955095044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c15e2d0 0x7fc32c0d0a70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.524+0000 7fc332600640 1 -- 172.21.15.44:0/3955095044 >> 172.21.15.44:0/3955095044 conn(0x7fc32c0d4060 msgr2=0x7fc32c0d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:47.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.524+0000 7fc332600640 1 -- 172.21.15.44:0/3955095044 shutdown_connections 2024-09-13T08:40:47.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.524+0000 7fc332600640 1 -- 172.21.15.44:0/3955095044 wait complete. 2024-09-13T08:40:47.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.524+0000 7fc332600640 1 Processor -- start 2024-09-13T08:40:47.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.524+0000 7fc332600640 1 -- start start 2024-09-13T08:40:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc332600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b7a30 0x7fc32c151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc332600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0d2100 0x7fc32c06d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc332600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c15e2d0 0x7fc32c06db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc332600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c16b1c0 con 0x7fc32c0b7a30 2024-09-13T08:40:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc332600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c16b040 con 0x7fc32c0d2100 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc332600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c16b340 con 0x7fc32c15e2d0 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b7a30 0x7fc32c151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b7a30 0x7fc32c151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37458/0 (socket says 172.21.15.44:37458) 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32b400640 1 -- 172.21.15.44:0/3649689880 learned_addr learned my addr 172.21.15.44:0/3649689880 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32be00640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c15e2d0 0x7fc32c06db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32b400640 1 -- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c15e2d0 msgr2=0x7fc32c06db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc323e00640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0d2100 0x7fc32c06d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32b400640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c15e2d0 0x7fc32c06db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32b400640 1 -- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0d2100 msgr2=0x7fc32c06d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32b400640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0d2100 0x7fc32c06d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32b400640 1 -- 172.21.15.44:0/3649689880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c15f950 con 0x7fc32c0b7a30 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32be00640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c15e2d0 0x7fc32c06db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc323e00640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0d2100 0x7fc32c06d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc32b400640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b7a30 0x7fc32c151280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc31c000c10 tx=0x7fc31c004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc323400640 1 -- 172.21.15.44:0/3649689880 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0bb070 con 0x7fc32c0b7a30 2024-09-13T08:40:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc323400640 1 -- 172.21.15.44:0/3649689880 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc31c005120 con 0x7fc32c0b7a30 2024-09-13T08:40:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc323400640 1 -- 172.21.15.44:0/3649689880 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0b2040 con 0x7fc32c0b7a30 2024-09-13T08:40:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc332600640 1 -- 172.21.15.44:0/3649689880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c15fc30 con 0x7fc32c0b7a30 2024-09-13T08:40:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc332600640 1 -- 172.21.15.44:0/3649689880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c1601f0 con 0x7fc32c0b7a30 2024-09-13T08:40:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc332600640 1 -- 172.21.15.44:0/3649689880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c1608b0 con 0x7fc32c0b7a30 2024-09-13T08:40:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.528+0000 7fc323400640 1 -- 172.21.15.44:0/3649689880 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fc31c0054d0 con 0x7fc32c0b7a30 2024-09-13T08:40:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.532+0000 7fc323400640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073220 0x7fc3080756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.532+0000 7fc323400640 1 -- 172.21.15.44:0/3649689880 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fc31c12df30 con 0x7fc32c0b7a30 2024-09-13T08:40:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.532+0000 7fc323e00640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073220 0x7fc3080756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.532+0000 7fc323e00640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073220 0x7fc3080756e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc32c069700 tx=0x7fc314009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.536+0000 7fc323400640 1 -- 172.21.15.44:0/3649689880 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc32c1608b0 con 0x7fc32c0b7a30 2024-09-13T08:40:47.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.708+0000 7fc332600640 1 -- 172.21.15.44:0/3649689880 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc32c155290 con 0x7fc308073220 2024-09-13T08:40:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.712+0000 7fc323400640 1 -- 172.21.15.44:0/3649689880 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251545 (secure 0 0 0) 0x7fc32c155290 con 0x7fc308073220 2024-09-13T08:40:47.717 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:40:47.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 -- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073220 msgr2=0x7fc3080756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073220 0x7fc3080756e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc32c069700 tx=0x7fc314009040 comp rx=0 tx=0).stop 2024-09-13T08:40:47.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 -- 172.21.15.44:0/3649689880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b7a30 msgr2=0x7fc32c151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b7a30 0x7fc32c151280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc31c000c10 tx=0x7fc31c004220 comp rx=0 tx=0).stop 2024-09-13T08:40:47.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 -- 172.21.15.44:0/3649689880 shutdown_connections 2024-09-13T08:40:47.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073220 0x7fc3080756e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c15e2d0 0x7fc32c06db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0d2100 0x7fc32c06d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 --2- 172.21.15.44:0/3649689880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b7a30 0x7fc32c151280 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 -- 172.21.15.44:0/3649689880 >> 172.21.15.44:0/3649689880 conn(0x7fc32c0d4060 msgr2=0x7fc32c15aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:47.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 -- 172.21.15.44:0/3649689880 shutdown_connections 2024-09-13T08:40:47.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.716+0000 7fc332600640 1 -- 172.21.15.44:0/3649689880 wait complete. 2024-09-13T08:40:47.737 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-13T08:40:46.557861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263220,"kb_used_data":5360,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711500,"statfs":{"total":773094113280,"available":772824576000,"internally_reserved":0,"allocated":5488640,"data_stored":2890737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.403999"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193015+0000","last_change":"2024-09-13T08:40:43.188161+0000","last_active":"2024-09-13T08:40:43.193015+0000","last_peered":"2024-09-13T08:40:43.193015+0000","last_clean":"2024-09-13T08:40:43.193015+0000","last_became_active":"2024-09-13T08:40:43.187905+0000","last_became_peered":"2024-09-13T08:40:43.187905+0000","last_unstale":"2024-09-13T08:40:43.193015+0000","last_undegraded":"2024-09-13T08:40:43.193015+0000","last_fullsized":"2024-09-13T08:40:43.193015+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199380+0000","last_change":"2024-09-13T08:40:43.193493+0000","last_active":"2024-09-13T08:40:43.199380+0000","last_peered":"2024-09-13T08:40:43.199380+0000","last_clean":"2024-09-13T08:40:43.199380+0000","last_became_active":"2024-09-13T08:40:43.193324+0000","last_became_peered":"2024-09-13T08:40:43.193324+0000","last_unstale":"2024-09-13T08:40:43.199380+0000","last_undegraded":"2024-09-13T08:40:43.199380+0000","last_fullsized":"2024-09-13T08:40:43.199380+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207011+0000","last_change":"2024-09-13T08:40:43.192784+0000","last_active":"2024-09-13T08:40:43.207011+0000","last_peered":"2024-09-13T08:40:43.207011+0000","last_clean":"2024-09-13T08:40:43.207011+0000","last_became_active":"2024-09-13T08:40:43.192234+0000","last_became_peered":"2024-09-13T08:40:43.192234+0000","last_unstale":"2024-09-13T08:40:43.207011+0000","last_undegraded":"2024-09-13T08:40:43.207011+0000","last_fullsized":"2024-09-13T08:40:43.207011+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:56:45.062628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193097+0000","last_change":"2024-09-13T08:40:43.189058+0000","last_active":"2024-09-13T08:40:43.193097+0000","last_peered":"2024-09-13T08:40:43.193097+0000","last_clean":"2024-09-13T08:40:43.193097+0000","last_became_active":"2024-09-13T08:40:43.188732+0000","last_became_peered":"2024-09-13T08:40:43.188732+0000","last_unstale":"2024-09-13T08:40:43.193097+0000","last_undegraded":"2024-09-13T08:40:43.193097+0000","last_fullsized":"2024-09-13T08:40:43.193097+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207327+0000","last_change":"2024-09-13T08:40:43.204809+0000","last_active":"2024-09-13T08:40:43.207327+0000","last_peered":"2024-09-13T08:40:43.207327+0000","last_clean":"2024-09-13T08:40:43.207327+0000","last_became_active":"2024-09-13T08:40:43.204599+0000","last_became_peered":"2024-09-13T08:40:43.204599+0000","last_unstale":"2024-09-13T08:40:43.207327+0000","last_undegraded":"2024-09-13T08:40:43.207327+0000","last_fullsized":"2024-09-13T08:40:43.207327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:20.538522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157508+0000","last_change":"2024-09-13T08:40:42.157508+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157508+0000","last_undegraded":"2024-09-13T08:40:42.157508+0000","last_fullsized":"2024-09-13T08:40:42.157508+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189864+0000","last_change":"2024-09-13T08:40:43.184765+0000","last_active":"2024-09-13T08:40:43.189864+0000","last_peered":"2024-09-13T08:40:43.189864+0000","last_clean":"2024-09-13T08:40:43.189864+0000","last_became_active":"2024-09-13T08:40:43.184173+0000","last_became_peered":"2024-09-13T08:40:43.184173+0000","last_unstale":"2024-09-13T08:40:43.189864+0000","last_undegraded":"2024-09-13T08:40:43.189864+0000","last_fullsized":"2024-09-13T08:40:43.189864+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:59.176747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190485+0000","last_change":"2024-09-13T08:40:43.180402+0000","last_active":"2024-09-13T08:40:43.190485+0000","last_peered":"2024-09-13T08:40:43.190485+0000","last_clean":"2024-09-13T08:40:43.190485+0000","last_became_active":"2024-09-13T08:40:43.179682+0000","last_became_peered":"2024-09-13T08:40:43.179682+0000","last_unstale":"2024-09-13T08:40:43.190485+0000","last_undegraded":"2024-09-13T08:40:43.190485+0000","last_fullsized":"2024-09-13T08:40:43.190485+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157861+0000","last_change":"2024-09-13T08:40:42.157861+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157861+0000","last_undegraded":"2024-09-13T08:40:42.157861+0000","last_fullsized":"2024-09-13T08:40:42.157861+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189100+0000","last_change":"2024-09-13T08:40:43.183571+0000","last_active":"2024-09-13T08:40:43.189100+0000","last_peered":"2024-09-13T08:40:43.189100+0000","last_clean":"2024-09-13T08:40:43.189100+0000","last_became_active":"2024-09-13T08:40:43.183297+0000","last_became_peered":"2024-09-13T08:40:43.183297+0000","last_unstale":"2024-09-13T08:40:43.189100+0000","last_undegraded":"2024-09-13T08:40:43.189100+0000","last_fullsized":"2024-09-13T08:40:43.189100+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:14.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206352+0000","last_change":"2024-09-13T08:40:43.200664+0000","last_active":"2024-09-13T08:40:43.206352+0000","last_peered":"2024-09-13T08:40:43.206352+0000","last_clean":"2024-09-13T08:40:43.206352+0000","last_became_active":"2024-09-13T08:40:43.200401+0000","last_became_peered":"2024-09-13T08:40:43.200401+0000","last_unstale":"2024-09-13T08:40:43.206352+0000","last_undegraded":"2024-09-13T08:40:43.206352+0000","last_fullsized":"2024-09-13T08:40:43.206352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:14.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199560+0000","last_change":"2024-09-13T08:40:43.188349+0000","last_active":"2024-09-13T08:40:43.199560+0000","last_peered":"2024-09-13T08:40:43.199560+0000","last_clean":"2024-09-13T08:40:43.199560+0000","last_became_active":"2024-09-13T08:40:43.188067+0000","last_became_peered":"2024-09-13T08:40:43.188067+0000","last_unstale":"2024-09-13T08:40:43.199560+0000","last_undegraded":"2024-09-13T08:40:43.199560+0000","last_fullsized":"2024-09-13T08:40:43.199560+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.158438+0000","last_change":"2024-09-13T08:40:42.158438+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.158438+0000","last_undegraded":"2024-09-13T08:40:42.158438+0000","last_fullsized":"2024-09-13T08:40:42.158438+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193350+0000","last_change":"2024-09-13T08:40:43.187316+0000","last_active":"2024-09-13T08:40:43.193350+0000","last_peered":"2024-09-13T08:40:43.193350+0000","last_clean":"2024-09-13T08:40:43.193350+0000","last_became_active":"2024-09-13T08:40:43.187050+0000","last_became_peered":"2024-09-13T08:40:43.187050+0000","last_unstale":"2024-09-13T08:40:43.193350+0000","last_undegraded":"2024-09-13T08:40:43.193350+0000","last_fullsized":"2024-09-13T08:40:43.193350+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189919+0000","last_change":"2024-09-13T08:40:43.178849+0000","last_active":"2024-09-13T08:40:43.189919+0000","last_peered":"2024-09-13T08:40:43.189919+0000","last_clean":"2024-09-13T08:40:43.189919+0000","last_became_active":"2024-09-13T08:40:43.178581+0000","last_became_peered":"2024-09-13T08:40:43.178581+0000","last_unstale":"2024-09-13T08:40:43.189919+0000","last_undegraded":"2024-09-13T08:40:43.189919+0000","last_fullsized":"2024-09-13T08:40:43.189919+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:08.944968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.182653+0000","last_change":"2024-09-13T08:40:43.178443+0000","last_active":"2024-09-13T08:40:43.182653+0000","last_peered":"2024-09-13T08:40:43.182653+0000","last_clean":"2024-09-13T08:40:43.182653+0000","last_became_active":"2024-09-13T08:40:43.178156+0000","last_became_peered":"2024-09-13T08:40:43.178156+0000","last_unstale":"2024-09-13T08:40:43.182653+0000","last_undegraded":"2024-09-13T08:40:43.182653+0000","last_fullsized":"2024-09-13T08:40:43.182653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.186648+0000","last_change":"2024-09-13T08:40:43.181966+0000","last_active":"2024-09-13T08:40:43.186648+0000","last_peered":"2024-09-13T08:40:43.186648+0000","last_clean":"2024-09-13T08:40:43.186648+0000","last_became_active":"2024-09-13T08:40:43.181651+0000","last_became_peered":"2024-09-13T08:40:43.181651+0000","last_unstale":"2024-09-13T08:40:43.186648+0000","last_undegraded":"2024-09-13T08:40:43.186648+0000","last_fullsized":"2024-09-13T08:40:43.186648+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:33.479797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207483+0000","last_change":"2024-09-13T08:40:43.203044+0000","last_active":"2024-09-13T08:40:43.207483+0000","last_peered":"2024-09-13T08:40:43.207483+0000","last_clean":"2024-09-13T08:40:43.207483+0000","last_became_active":"2024-09-13T08:40:43.196038+0000","last_became_peered":"2024-09-13T08:40:43.196038+0000","last_unstale":"2024-09-13T08:40:43.207483+0000","last_undegraded":"2024-09-13T08:40:43.207483+0000","last_fullsized":"2024-09-13T08:40:43.207483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:49:13.250832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207072+0000","last_change":"2024-09-13T08:40:43.200673+0000","last_active":"2024-09-13T08:40:43.207072+0000","last_peered":"2024-09-13T08:40:43.207072+0000","last_clean":"2024-09-13T08:40:43.207072+0000","last_became_active":"2024-09-13T08:40:43.200103+0000","last_became_peered":"2024-09-13T08:40:43.200103+0000","last_unstale":"2024-09-13T08:40:43.207072+0000","last_undegraded":"2024-09-13T08:40:43.207072+0000","last_fullsized":"2024-09-13T08:40:43.207072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:21.298332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.203522+0000","last_change":"2024-09-13T08:40:43.191159+0000","last_active":"2024-09-13T08:40:43.203522+0000","last_peered":"2024-09-13T08:40:43.203522+0000","last_clean":"2024-09-13T08:40:43.203522+0000","last_became_active":"2024-09-13T08:40:43.190876+0000","last_became_peered":"2024-09-13T08:40:43.190876+0000","last_unstale":"2024-09-13T08:40:43.203522+0000","last_undegraded":"2024-09-13T08:40:43.203522+0000","last_fullsized":"2024-09-13T08:40:43.203522+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:49.726267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190142+0000","last_change":"2024-09-13T08:40:43.176763+0000","last_active":"2024-09-13T08:40:43.190142+0000","last_peered":"2024-09-13T08:40:43.190142+0000","last_clean":"2024-09-13T08:40:43.190142+0000","last_became_active":"2024-09-13T08:40:43.176429+0000","last_became_peered":"2024-09-13T08:40:43.176429+0000","last_unstale":"2024-09-13T08:40:43.190142+0000","last_undegraded":"2024-09-13T08:40:43.190142+0000","last_fullsized":"2024-09-13T08:40:43.190142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:14.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.211767+0000","last_change":"2024-09-13T08:40:43.207935+0000","last_active":"2024-09-13T08:40:43.211767+0000","last_peered":"2024-09-13T08:40:43.211767+0000","last_clean":"2024-09-13T08:40:43.211767+0000","last_became_active":"2024-09-13T08:40:43.207661+0000","last_became_peered":"2024-09-13T08:40:43.207661+0000","last_unstale":"2024-09-13T08:40:43.211767+0000","last_undegraded":"2024-09-13T08:40:43.211767+0000","last_fullsized":"2024-09-13T08:40:43.211767+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:43.583764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189838+0000","last_change":"2024-09-13T08:40:43.184477+0000","last_active":"2024-09-13T08:40:43.189838+0000","last_peered":"2024-09-13T08:40:43.189838+0000","last_clean":"2024-09-13T08:40:43.189838+0000","last_became_active":"2024-09-13T08:40:43.183611+0000","last_became_peered":"2024-09-13T08:40:43.183611+0000","last_unstale":"2024-09-13T08:40:43.189838+0000","last_undegraded":"2024-09-13T08:40:43.189838+0000","last_fullsized":"2024-09-13T08:40:43.189838+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:33:32.215525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.175547+0000","last_change":"2024-09-13T08:40:43.173503+0000","last_active":"2024-09-13T08:40:43.175547+0000","last_peered":"2024-09-13T08:40:43.175547+0000","last_clean":"2024-09-13T08:40:43.175547+0000","last_became_active":"2024-09-13T08:40:43.173209+0000","last_became_peered":"2024-09-13T08:40:43.173209+0000","last_unstale":"2024-09-13T08:40:43.175547+0000","last_undegraded":"2024-09-13T08:40:43.175547+0000","last_fullsized":"2024-09-13T08:40:43.175547+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:00.931558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193040+0000","last_change":"2024-09-13T08:40:43.188762+0000","last_active":"2024-09-13T08:40:43.193040+0000","last_peered":"2024-09-13T08:40:43.193040+0000","last_clean":"2024-09-13T08:40:43.193040+0000","last_became_active":"2024-09-13T08:40:43.188481+0000","last_became_peered":"2024-09-13T08:40:43.188481+0000","last_unstale":"2024-09-13T08:40:43.193040+0000","last_undegraded":"2024-09-13T08:40:43.193040+0000","last_fullsized":"2024-09-13T08:40:43.193040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206308+0000","last_change":"2024-09-13T08:40:43.201185+0000","last_active":"2024-09-13T08:40:43.206308+0000","last_peered":"2024-09-13T08:40:43.206308+0000","last_clean":"2024-09-13T08:40:43.206308+0000","last_became_active":"2024-09-13T08:40:43.200898+0000","last_became_peered":"2024-09-13T08:40:43.200898+0000","last_unstale":"2024-09-13T08:40:43.206308+0000","last_undegraded":"2024-09-13T08:40:43.206308+0000","last_fullsized":"2024-09-13T08:40:43.206308+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:32.631857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212364+0000","last_change":"2024-09-13T08:40:43.208083+0000","last_active":"2024-09-13T08:40:43.212364+0000","last_peered":"2024-09-13T08:40:43.212364+0000","last_clean":"2024-09-13T08:40:43.212364+0000","last_became_active":"2024-09-13T08:40:43.207841+0000","last_became_peered":"2024-09-13T08:40:43.207841+0000","last_unstale":"2024-09-13T08:40:43.212364+0000","last_undegraded":"2024-09-13T08:40:43.212364+0000","last_fullsized":"2024-09-13T08:40:43.212364+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:29:52.299097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207593+0000","last_change":"2024-09-13T08:40:43.198438+0000","last_active":"2024-09-13T08:40:43.207593+0000","last_peered":"2024-09-13T08:40:43.207593+0000","last_clean":"2024-09-13T08:40:43.207593+0000","last_became_active":"2024-09-13T08:40:43.197966+0000","last_became_peered":"2024-09-13T08:40:43.197966+0000","last_unstale":"2024-09-13T08:40:43.207593+0000","last_undegraded":"2024-09-13T08:40:43.207593+0000","last_fullsized":"2024-09-13T08:40:43.207593+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:54:02.077806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197404+0000","last_change":"2024-09-13T08:40:43.191493+0000","last_active":"2024-09-13T08:40:43.197404+0000","last_peered":"2024-09-13T08:40:43.197404+0000","last_clean":"2024-09-13T08:40:43.197404+0000","last_became_active":"2024-09-13T08:40:43.191059+0000","last_became_peered":"2024-09-13T08:40:43.191059+0000","last_unstale":"2024-09-13T08:40:43.197404+0000","last_undegraded":"2024-09-13T08:40:43.197404+0000","last_fullsized":"2024-09-13T08:40:43.197404+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.186580+0000","last_change":"2024-09-13T08:40:43.182477+0000","last_active":"2024-09-13T08:40:43.186580+0000","last_peered":"2024-09-13T08:40:43.186580+0000","last_clean":"2024-09-13T08:40:43.186580+0000","last_became_active":"2024-09-13T08:40:43.182226+0000","last_became_peered":"2024-09-13T08:40:43.182226+0000","last_unstale":"2024-09-13T08:40:43.186580+0000","last_undegraded":"2024-09-13T08:40:43.186580+0000","last_fullsized":"2024-09-13T08:40:43.186580+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157876+0000","last_change":"2024-09-13T08:40:42.157876+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157876+0000","last_undegraded":"2024-09-13T08:40:42.157876+0000","last_fullsized":"2024-09-13T08:40:42.157876+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193060+0000","last_change":"2024-09-13T08:40:43.189479+0000","last_active":"2024-09-13T08:40:43.193060+0000","last_peered":"2024-09-13T08:40:43.193060+0000","last_clean":"2024-09-13T08:40:43.193060+0000","last_became_active":"2024-09-13T08:40:43.189207+0000","last_became_peered":"2024-09-13T08:40:43.189207+0000","last_unstale":"2024-09-13T08:40:43.193060+0000","last_undegraded":"2024-09-13T08:40:43.193060+0000","last_fullsized":"2024-09-13T08:40:43.193060+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:37.772751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197183+0000","last_change":"2024-09-13T08:40:43.193080+0000","last_active":"2024-09-13T08:40:43.197183+0000","last_peered":"2024-09-13T08:40:43.197183+0000","last_clean":"2024-09-13T08:40:43.197183+0000","last_became_active":"2024-09-13T08:40:43.192785+0000","last_became_peered":"2024-09-13T08:40:43.192785+0000","last_unstale":"2024-09-13T08:40:43.197183+0000","last_undegraded":"2024-09-13T08:40:43.197183+0000","last_fullsized":"2024-09-13T08:40:43.197183+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:58.112836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.178584+0000","last_change":"2024-09-13T08:40:42.175868+0000","last_active":"2024-09-13T08:40:42.178584+0000","last_peered":"2024-09-13T08:40:42.178584+0000","last_clean":"2024-09-13T08:40:42.178584+0000","last_became_active":"2024-09-13T08:40:42.175587+0000","last_became_peered":"2024-09-13T08:40:42.175587+0000","last_unstale":"2024-09-13T08:40:42.178584+0000","last_undegraded":"2024-09-13T08:40:42.178584+0000","last_fullsized":"2024-09-13T08:40:42.178584+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:33.183213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198234+0000","last_change":"2024-09-13T08:40:43.194685+0000","last_active":"2024-09-13T08:40:43.198234+0000","last_peered":"2024-09-13T08:40:43.198234+0000","last_clean":"2024-09-13T08:40:43.198234+0000","last_became_active":"2024-09-13T08:40:43.194145+0000","last_became_peered":"2024-09-13T08:40:43.194145+0000","last_unstale":"2024-09-13T08:40:43.198234+0000","last_undegraded":"2024-09-13T08:40:43.198234+0000","last_fullsized":"2024-09-13T08:40:43.198234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:37.772751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206000+0000","last_change":"2024-09-13T08:40:42.182278+0000","last_active":"2024-09-13T08:40:43.206000+0000","last_peered":"2024-09-13T08:40:43.206000+0000","last_clean":"2024-09-13T08:40:43.206000+0000","last_became_active":"2024-09-13T08:40:42.181992+0000","last_became_peered":"2024-09-13T08:40:42.181992+0000","last_unstale":"2024-09-13T08:40:43.206000+0000","last_undegraded":"2024-09-13T08:40:43.206000+0000","last_fullsized":"2024-09-13T08:40:43.206000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:51.100086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199534+0000","last_change":"2024-09-13T08:40:43.187867+0000","last_active":"2024-09-13T08:40:43.199534+0000","last_peered":"2024-09-13T08:40:43.199534+0000","last_clean":"2024-09-13T08:40:43.199534+0000","last_became_active":"2024-09-13T08:40:43.187621+0000","last_became_peered":"2024-09-13T08:40:43.187621+0000","last_unstale":"2024-09-13T08:40:43.199534+0000","last_undegraded":"2024-09-13T08:40:43.199534+0000","last_fullsized":"2024-09-13T08:40:43.199534+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.179942+0000","last_change":"2024-09-13T08:40:42.179208+0000","last_active":"2024-09-13T08:40:42.179942+0000","last_peered":"2024-09-13T08:40:42.179942+0000","last_clean":"2024-09-13T08:40:42.179942+0000","last_became_active":"2024-09-13T08:40:42.178617+0000","last_became_peered":"2024-09-13T08:40:42.178617+0000","last_unstale":"2024-09-13T08:40:42.179942+0000","last_undegraded":"2024-09-13T08:40:42.179942+0000","last_fullsized":"2024-09-13T08:40:42.179942+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:01:20.323316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198747+0000","last_change":"2024-09-13T08:40:43.192425+0000","last_active":"2024-09-13T08:40:43.198747+0000","last_peered":"2024-09-13T08:40:43.198747+0000","last_clean":"2024-09-13T08:40:43.198747+0000","last_became_active":"2024-09-13T08:40:43.191801+0000","last_became_peered":"2024-09-13T08:40:43.191801+0000","last_unstale":"2024-09-13T08:40:43.198747+0000","last_undegraded":"2024-09-13T08:40:43.198747+0000","last_fullsized":"2024-09-13T08:40:43.198747+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:11.173292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205568+0000","last_change":"2024-09-13T08:40:42.180695+0000","last_active":"2024-09-13T08:40:43.205568+0000","last_peered":"2024-09-13T08:40:43.205568+0000","last_clean":"2024-09-13T08:40:43.205568+0000","last_became_active":"2024-09-13T08:40:42.180392+0000","last_became_peered":"2024-09-13T08:40:42.180392+0000","last_unstale":"2024-09-13T08:40:43.205568+0000","last_undegraded":"2024-09-13T08:40:43.205568+0000","last_fullsized":"2024-09-13T08:40:43.205568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:40.439749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212438+0000","last_change":"2024-09-13T08:40:43.209100+0000","last_active":"2024-09-13T08:40:43.212438+0000","last_peered":"2024-09-13T08:40:43.212438+0000","last_clean":"2024-09-13T08:40:43.212438+0000","last_became_active":"2024-09-13T08:40:43.208834+0000","last_became_peered":"2024-09-13T08:40:43.208834+0000","last_unstale":"2024-09-13T08:40:43.212438+0000","last_undegraded":"2024-09-13T08:40:43.212438+0000","last_fullsized":"2024-09-13T08:40:43.212438+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:29:51.212164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.168839+0000","last_change":"2024-09-13T08:40:42.166893+0000","last_active":"2024-09-13T08:40:42.168839+0000","last_peered":"2024-09-13T08:40:42.168839+0000","last_clean":"2024-09-13T08:40:42.168839+0000","last_became_active":"2024-09-13T08:40:42.166507+0000","last_became_peered":"2024-09-13T08:40:42.166507+0000","last_unstale":"2024-09-13T08:40:42.168839+0000","last_undegraded":"2024-09-13T08:40:42.168839+0000","last_fullsized":"2024-09-13T08:40:42.168839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:53:24.395981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205301+0000","last_change":"2024-09-13T08:40:43.193379+0000","last_active":"2024-09-13T08:40:43.205301+0000","last_peered":"2024-09-13T08:40:43.205301+0000","last_clean":"2024-09-13T08:40:43.205301+0000","last_became_active":"2024-09-13T08:40:43.193015+0000","last_became_peered":"2024-09-13T08:40:43.193015+0000","last_unstale":"2024-09-13T08:40:43.205301+0000","last_undegraded":"2024-09-13T08:40:43.205301+0000","last_fullsized":"2024-09-13T08:40:43.205301+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:19.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.201537+0000","last_change":"2024-09-13T08:40:42.184150+0000","last_active":"2024-09-13T08:40:43.201537+0000","last_peered":"2024-09-13T08:40:43.201537+0000","last_clean":"2024-09-13T08:40:43.201537+0000","last_became_active":"2024-09-13T08:40:42.183763+0000","last_became_peered":"2024-09-13T08:40:42.183763+0000","last_unstale":"2024-09-13T08:40:43.201537+0000","last_undegraded":"2024-09-13T08:40:43.201537+0000","last_fullsized":"2024-09-13T08:40:43.201537+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:59.988155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190512+0000","last_change":"2024-09-13T08:40:43.187575+0000","last_active":"2024-09-13T08:40:43.190512+0000","last_peered":"2024-09-13T08:40:43.190512+0000","last_clean":"2024-09-13T08:40:43.190512+0000","last_became_active":"2024-09-13T08:40:43.186758+0000","last_became_peered":"2024-09-13T08:40:43.186758+0000","last_unstale":"2024-09-13T08:40:43.190512+0000","last_undegraded":"2024-09-13T08:40:43.190512+0000","last_fullsized":"2024-09-13T08:40:43.190512+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:55.795423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172776+0000","last_change":"2024-09-13T08:40:42.175361+0000","last_active":"2024-09-13T08:40:43.172776+0000","last_peered":"2024-09-13T08:40:43.172776+0000","last_clean":"2024-09-13T08:40:43.172776+0000","last_became_active":"2024-09-13T08:40:42.174980+0000","last_became_peered":"2024-09-13T08:40:42.174980+0000","last_unstale":"2024-09-13T08:40:43.172776+0000","last_undegraded":"2024-09-13T08:40:43.172776+0000","last_fullsized":"2024-09-13T08:40:43.172776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:59.925397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206690+0000","last_change":"2024-09-13T08:40:43.200353+0000","last_active":"2024-09-13T08:40:43.206690+0000","last_peered":"2024-09-13T08:40:43.206690+0000","last_clean":"2024-09-13T08:40:43.206690+0000","last_became_active":"2024-09-13T08:40:43.198409+0000","last_became_peered":"2024-09-13T08:40:43.198409+0000","last_unstale":"2024-09-13T08:40:43.206690+0000","last_undegraded":"2024-09-13T08:40:43.206690+0000","last_fullsized":"2024-09-13T08:40:43.206690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:39.828271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.178415+0000","last_change":"2024-09-13T08:40:42.178338+0000","last_active":"2024-09-13T08:40:43.178415+0000","last_peered":"2024-09-13T08:40:43.178415+0000","last_clean":"2024-09-13T08:40:43.178415+0000","last_became_active":"2024-09-13T08:40:42.178025+0000","last_became_peered":"2024-09-13T08:40:42.178025+0000","last_unstale":"2024-09-13T08:40:43.178415+0000","last_undegraded":"2024-09-13T08:40:43.178415+0000","last_fullsized":"2024-09-13T08:40:43.178415+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:31.891865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.210052+0000","last_change":"2024-09-13T08:40:43.204789+0000","last_active":"2024-09-13T08:40:43.210052+0000","last_peered":"2024-09-13T08:40:43.210052+0000","last_clean":"2024-09-13T08:40:43.210052+0000","last_became_active":"2024-09-13T08:40:43.204114+0000","last_became_peered":"2024-09-13T08:40:43.204114+0000","last_unstale":"2024-09-13T08:40:43.210052+0000","last_undegraded":"2024-09-13T08:40:43.210052+0000","last_fullsized":"2024-09-13T08:40:43.210052+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:31.140294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.174034+0000","last_change":"2024-09-13T08:40:42.178810+0000","last_active":"2024-09-13T08:40:43.174034+0000","last_peered":"2024-09-13T08:40:43.174034+0000","last_clean":"2024-09-13T08:40:43.174034+0000","last_became_active":"2024-09-13T08:40:42.173486+0000","last_became_peered":"2024-09-13T08:40:42.173486+0000","last_unstale":"2024-09-13T08:40:43.174034+0000","last_undegraded":"2024-09-13T08:40:43.174034+0000","last_fullsized":"2024-09-13T08:40:43.174034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:12.229793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207136+0000","last_change":"2024-09-13T08:40:43.194106+0000","last_active":"2024-09-13T08:40:43.207136+0000","last_peered":"2024-09-13T08:40:43.207136+0000","last_clean":"2024-09-13T08:40:43.207136+0000","last_became_active":"2024-09-13T08:40:43.193804+0000","last_became_peered":"2024-09-13T08:40:43.193804+0000","last_unstale":"2024-09-13T08:40:43.207136+0000","last_undegraded":"2024-09-13T08:40:43.207136+0000","last_fullsized":"2024-09-13T08:40:43.207136+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.417971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.183104+0000","last_change":"2024-09-13T08:40:42.183102+0000","last_active":"2024-09-13T08:40:43.183104+0000","last_peered":"2024-09-13T08:40:43.183104+0000","last_clean":"2024-09-13T08:40:43.183104+0000","last_became_active":"2024-09-13T08:40:42.182748+0000","last_became_peered":"2024-09-13T08:40:42.182748+0000","last_unstale":"2024-09-13T08:40:43.183104+0000","last_undegraded":"2024-09-13T08:40:43.183104+0000","last_fullsized":"2024-09-13T08:40:43.183104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:10.167131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.195996+0000","last_change":"2024-09-13T08:40:43.191524+0000","last_active":"2024-09-13T08:40:43.195996+0000","last_peered":"2024-09-13T08:40:43.195996+0000","last_clean":"2024-09-13T08:40:43.195996+0000","last_became_active":"2024-09-13T08:40:43.191240+0000","last_became_peered":"2024-09-13T08:40:43.191240+0000","last_unstale":"2024-09-13T08:40:43.195996+0000","last_undegraded":"2024-09-13T08:40:43.195996+0000","last_fullsized":"2024-09-13T08:40:43.195996+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:33.508450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.209427+0000","last_change":"2024-09-13T08:40:42.177115+0000","last_active":"2024-09-13T08:40:43.209427+0000","last_peered":"2024-09-13T08:40:43.209427+0000","last_clean":"2024-09-13T08:40:43.209427+0000","last_became_active":"2024-09-13T08:40:42.176751+0000","last_became_peered":"2024-09-13T08:40:42.176751+0000","last_unstale":"2024-09-13T08:40:43.209427+0000","last_undegraded":"2024-09-13T08:40:43.209427+0000","last_fullsized":"2024-09-13T08:40:43.209427+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:23.536012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206408+0000","last_change":"2024-09-13T08:40:43.197281+0000","last_active":"2024-09-13T08:40:43.206408+0000","last_peered":"2024-09-13T08:40:43.206408+0000","last_clean":"2024-09-13T08:40:43.206408+0000","last_became_active":"2024-09-13T08:40:43.196989+0000","last_became_peered":"2024-09-13T08:40:43.196989+0000","last_unstale":"2024-09-13T08:40:43.206408+0000","last_undegraded":"2024-09-13T08:40:43.206408+0000","last_fullsized":"2024-09-13T08:40:43.206408+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:03.667062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.214312+0000","last_change":"2024-09-13T08:40:42.180947+0000","last_active":"2024-09-13T08:40:43.214312+0000","last_peered":"2024-09-13T08:40:43.214312+0000","last_clean":"2024-09-13T08:40:43.214312+0000","last_became_active":"2024-09-13T08:40:42.180647+0000","last_became_peered":"2024-09-13T08:40:42.180647+0000","last_unstale":"2024-09-13T08:40:43.214312+0000","last_undegraded":"2024-09-13T08:40:43.214312+0000","last_fullsized":"2024-09-13T08:40:43.214312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:31.625696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204310+0000","last_change":"2024-09-13T08:40:43.177272+0000","last_active":"2024-09-13T08:40:43.204310+0000","last_peered":"2024-09-13T08:40:43.204310+0000","last_clean":"2024-09-13T08:40:43.204310+0000","last_became_active":"2024-09-13T08:40:43.176928+0000","last_became_peered":"2024-09-13T08:40:43.176928+0000","last_unstale":"2024-09-13T08:40:43.204310+0000","last_undegraded":"2024-09-13T08:40:43.204310+0000","last_fullsized":"2024-09-13T08:40:43.204310+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:19.727346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204104+0000","last_change":"2024-09-13T08:40:42.176291+0000","last_active":"2024-09-13T08:40:43.204104+0000","last_peered":"2024-09-13T08:40:43.204104+0000","last_clean":"2024-09-13T08:40:43.204104+0000","last_became_active":"2024-09-13T08:40:42.175697+0000","last_became_peered":"2024-09-13T08:40:42.175697+0000","last_unstale":"2024-09-13T08:40:43.204104+0000","last_undegraded":"2024-09-13T08:40:43.204104+0000","last_fullsized":"2024-09-13T08:40:43.204104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:18.721185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198468+0000","last_change":"2024-09-13T08:40:43.194019+0000","last_active":"2024-09-13T08:40:43.198468+0000","last_peered":"2024-09-13T08:40:43.198468+0000","last_clean":"2024-09-13T08:40:43.198468+0000","last_became_active":"2024-09-13T08:40:43.192433+0000","last_became_peered":"2024-09-13T08:40:43.192433+0000","last_unstale":"2024-09-13T08:40:43.198468+0000","last_undegraded":"2024-09-13T08:40:43.198468+0000","last_fullsized":"2024-09-13T08:40:43.198468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:49:13.250832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172856+0000","last_change":"2024-09-13T08:40:42.176544+0000","last_active":"2024-09-13T08:40:43.172856+0000","last_peered":"2024-09-13T08:40:43.172856+0000","last_clean":"2024-09-13T08:40:43.172856+0000","last_became_active":"2024-09-13T08:40:42.175781+0000","last_became_peered":"2024-09-13T08:40:42.175781+0000","last_unstale":"2024-09-13T08:40:43.172856+0000","last_undegraded":"2024-09-13T08:40:43.172856+0000","last_fullsized":"2024-09-13T08:40:43.172856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:53.422970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198512+0000","last_change":"2024-09-13T08:40:43.195108+0000","last_active":"2024-09-13T08:40:43.198512+0000","last_peered":"2024-09-13T08:40:43.198512+0000","last_clean":"2024-09-13T08:40:43.198512+0000","last_became_active":"2024-09-13T08:40:43.193623+0000","last_became_peered":"2024-09-13T08:40:43.193623+0000","last_unstale":"2024-09-13T08:40:43.198512+0000","last_undegraded":"2024-09-13T08:40:43.198512+0000","last_fullsized":"2024-09-13T08:40:43.198512+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:31.140294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.185074+0000","last_change":"2024-09-13T08:40:42.184583+0000","last_active":"2024-09-13T08:40:42.185074+0000","last_peered":"2024-09-13T08:40:42.185074+0000","last_clean":"2024-09-13T08:40:42.185074+0000","last_became_active":"2024-09-13T08:40:42.184269+0000","last_became_peered":"2024-09-13T08:40:42.184269+0000","last_unstale":"2024-09-13T08:40:42.185074+0000","last_undegraded":"2024-09-13T08:40:42.185074+0000","last_fullsized":"2024-09-13T08:40:42.185074+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:55.743786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.187184+0000","last_change":"2024-09-13T08:40:43.181834+0000","last_active":"2024-09-13T08:40:43.187184+0000","last_peered":"2024-09-13T08:40:43.187184+0000","last_clean":"2024-09-13T08:40:43.187184+0000","last_became_active":"2024-09-13T08:40:43.181558+0000","last_became_peered":"2024-09-13T08:40:43.181558+0000","last_unstale":"2024-09-13T08:40:43.187184+0000","last_undegraded":"2024-09-13T08:40:43.187184+0000","last_fullsized":"2024-09-13T08:40:43.187184+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:08.944968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213300+0000","last_change":"2024-09-13T08:40:42.173542+0000","last_active":"2024-09-13T08:40:43.213300+0000","last_peered":"2024-09-13T08:40:43.213300+0000","last_clean":"2024-09-13T08:40:43.213300+0000","last_became_active":"2024-09-13T08:40:42.173206+0000","last_became_peered":"2024-09-13T08:40:42.173206+0000","last_unstale":"2024-09-13T08:40:43.213300+0000","last_undegraded":"2024-09-13T08:40:43.213300+0000","last_fullsized":"2024-09-13T08:40:43.213300+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:36.766590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212306+0000","last_change":"2024-09-13T08:40:43.210476+0000","last_active":"2024-09-13T08:40:43.212306+0000","last_peered":"2024-09-13T08:40:43.212306+0000","last_clean":"2024-09-13T08:40:43.212306+0000","last_became_active":"2024-09-13T08:40:43.210279+0000","last_became_peered":"2024-09-13T08:40:43.210279+0000","last_unstale":"2024-09-13T08:40:43.212306+0000","last_undegraded":"2024-09-13T08:40:43.212306+0000","last_fullsized":"2024-09-13T08:40:43.212306+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:03.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.209719+0000","last_change":"2024-09-13T08:40:42.176695+0000","last_active":"2024-09-13T08:40:43.209719+0000","last_peered":"2024-09-13T08:40:43.209719+0000","last_clean":"2024-09-13T08:40:43.209719+0000","last_became_active":"2024-09-13T08:40:42.176387+0000","last_became_peered":"2024-09-13T08:40:42.176387+0000","last_unstale":"2024-09-13T08:40:43.209719+0000","last_undegraded":"2024-09-13T08:40:43.209719+0000","last_fullsized":"2024-09-13T08:40:43.209719+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:23.536012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197389+0000","last_change":"2024-09-13T08:40:43.193233+0000","last_active":"2024-09-13T08:40:43.197389+0000","last_peered":"2024-09-13T08:40:43.197389+0000","last_clean":"2024-09-13T08:40:43.197389+0000","last_became_active":"2024-09-13T08:40:43.192944+0000","last_became_peered":"2024-09-13T08:40:43.192944+0000","last_unstale":"2024-09-13T08:40:43.197389+0000","last_undegraded":"2024-09-13T08:40:43.197389+0000","last_fullsized":"2024-09-13T08:40:43.197389+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.417971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213559+0000","last_change":"2024-09-13T08:40:42.180372+0000","last_active":"2024-09-13T08:40:43.213559+0000","last_peered":"2024-09-13T08:40:43.213559+0000","last_clean":"2024-09-13T08:40:43.213559+0000","last_became_active":"2024-09-13T08:40:42.180079+0000","last_became_peered":"2024-09-13T08:40:42.180079+0000","last_unstale":"2024-09-13T08:40:43.213559+0000","last_undegraded":"2024-09-13T08:40:43.213559+0000","last_fullsized":"2024-09-13T08:40:43.213559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:21.515280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.188208+0000","last_change":"2024-09-13T08:40:43.177434+0000","last_active":"2024-09-13T08:40:43.188208+0000","last_peered":"2024-09-13T08:40:43.188208+0000","last_clean":"2024-09-13T08:40:43.188208+0000","last_became_active":"2024-09-13T08:40:43.177163+0000","last_became_peered":"2024-09-13T08:40:43.177163+0000","last_unstale":"2024-09-13T08:40:43.188208+0000","last_undegraded":"2024-09-13T08:40:43.188208+0000","last_fullsized":"2024-09-13T08:40:43.188208+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:14.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213015+0000","last_change":"2024-09-13T08:40:42.185496+0000","last_active":"2024-09-13T08:40:43.213015+0000","last_peered":"2024-09-13T08:40:43.213015+0000","last_clean":"2024-09-13T08:40:43.213015+0000","last_became_active":"2024-09-13T08:40:42.184959+0000","last_became_peered":"2024-09-13T08:40:42.184959+0000","last_unstale":"2024-09-13T08:40:43.213015+0000","last_undegraded":"2024-09-13T08:40:43.213015+0000","last_fullsized":"2024-09-13T08:40:43.213015+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:18:11.442697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207041+0000","last_change":"2024-09-13T08:40:43.193014+0000","last_active":"2024-09-13T08:40:43.207041+0000","last_peered":"2024-09-13T08:40:43.207041+0000","last_clean":"2024-09-13T08:40:43.207041+0000","last_became_active":"2024-09-13T08:40:43.192539+0000","last_became_peered":"2024-09-13T08:40:43.192539+0000","last_unstale":"2024-09-13T08:40:43.207041+0000","last_undegraded":"2024-09-13T08:40:43.207041+0000","last_fullsized":"2024-09-13T08:40:43.207041+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:58.112836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.188854+0000","last_change":"2024-09-13T08:40:42.188197+0000","last_active":"2024-09-13T08:40:42.188854+0000","last_peered":"2024-09-13T08:40:42.188854+0000","last_clean":"2024-09-13T08:40:42.188854+0000","last_became_active":"2024-09-13T08:40:42.187908+0000","last_became_peered":"2024-09-13T08:40:42.187908+0000","last_unstale":"2024-09-13T08:40:42.188854+0000","last_undegraded":"2024-09-13T08:40:42.188854+0000","last_fullsized":"2024-09-13T08:40:42.188854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:30:57.769839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205398+0000","last_change":"2024-09-13T08:40:43.194030+0000","last_active":"2024-09-13T08:40:43.205398+0000","last_peered":"2024-09-13T08:40:43.205398+0000","last_clean":"2024-09-13T08:40:43.205398+0000","last_became_active":"2024-09-13T08:40:43.193719+0000","last_became_peered":"2024-09-13T08:40:43.193719+0000","last_unstale":"2024-09-13T08:40:43.205398+0000","last_undegraded":"2024-09-13T08:40:43.205398+0000","last_fullsized":"2024-09-13T08:40:43.205398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:54:40.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.210039+0000","last_change":"2024-09-13T08:40:42.170852+0000","last_active":"2024-09-13T08:40:43.210039+0000","last_peered":"2024-09-13T08:40:43.210039+0000","last_clean":"2024-09-13T08:40:43.210039+0000","last_became_active":"2024-09-13T08:40:42.170588+0000","last_became_peered":"2024-09-13T08:40:42.170588+0000","last_unstale":"2024-09-13T08:40:43.210039+0000","last_undegraded":"2024-09-13T08:40:43.210039+0000","last_fullsized":"2024-09-13T08:40:43.210039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:32.502289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157478+0000","last_change":"2024-09-13T08:40:42.157478+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157478+0000","last_undegraded":"2024-09-13T08:40:42.157478+0000","last_fullsized":"2024-09-13T08:40:42.157478+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.175195+0000","last_change":"2024-09-13T08:40:42.175444+0000","last_active":"2024-09-13T08:40:43.175195+0000","last_peered":"2024-09-13T08:40:43.175195+0000","last_clean":"2024-09-13T08:40:43.175195+0000","last_became_active":"2024-09-13T08:40:42.174590+0000","last_became_peered":"2024-09-13T08:40:42.174590+0000","last_unstale":"2024-09-13T08:40:43.175195+0000","last_undegraded":"2024-09-13T08:40:43.175195+0000","last_fullsized":"2024-09-13T08:40:43.175195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:32.473636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190104+0000","last_change":"2024-09-13T08:40:43.177933+0000","last_active":"2024-09-13T08:40:43.190104+0000","last_peered":"2024-09-13T08:40:43.190104+0000","last_clean":"2024-09-13T08:40:43.190104+0000","last_became_active":"2024-09-13T08:40:43.177479+0000","last_became_peered":"2024-09-13T08:40:43.177479+0000","last_unstale":"2024-09-13T08:40:43.190104+0000","last_undegraded":"2024-09-13T08:40:43.190104+0000","last_fullsized":"2024-09-13T08:40:43.190104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:54.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.178449+0000","last_change":"2024-09-13T08:40:42.183091+0000","last_active":"2024-09-13T08:40:43.178449+0000","last_peered":"2024-09-13T08:40:43.178449+0000","last_clean":"2024-09-13T08:40:43.178449+0000","last_became_active":"2024-09-13T08:40:42.182685+0000","last_became_peered":"2024-09-13T08:40:42.182685+0000","last_unstale":"2024-09-13T08:40:43.178449+0000","last_undegraded":"2024-09-13T08:40:43.178449+0000","last_fullsized":"2024-09-13T08:40:43.178449+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:03.155901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.192568+0000","last_change":"2024-09-13T08:40:43.181826+0000","last_active":"2024-09-13T08:40:43.192568+0000","last_peered":"2024-09-13T08:40:43.192568+0000","last_clean":"2024-09-13T08:40:43.192568+0000","last_became_active":"2024-09-13T08:40:43.181516+0000","last_became_peered":"2024-09-13T08:40:43.181516+0000","last_unstale":"2024-09-13T08:40:43.192568+0000","last_undegraded":"2024-09-13T08:40:43.192568+0000","last_fullsized":"2024-09-13T08:40:43.192568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.173833+0000","last_change":"2024-09-13T08:40:42.177701+0000","last_active":"2024-09-13T08:40:43.173833+0000","last_peered":"2024-09-13T08:40:43.173833+0000","last_clean":"2024-09-13T08:40:43.173833+0000","last_became_active":"2024-09-13T08:40:42.177418+0000","last_became_peered":"2024-09-13T08:40:42.177418+0000","last_unstale":"2024-09-13T08:40:43.173833+0000","last_undegraded":"2024-09-13T08:40:43.173833+0000","last_fullsized":"2024-09-13T08:40:43.173833+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:23.873609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206487+0000","last_change":"2024-09-13T08:40:43.198409+0000","last_active":"2024-09-13T08:40:43.206487+0000","last_peered":"2024-09-13T08:40:43.206487+0000","last_clean":"2024-09-13T08:40:43.206487+0000","last_became_active":"2024-09-13T08:40:43.198235+0000","last_became_peered":"2024-09-13T08:40:43.198235+0000","last_unstale":"2024-09-13T08:40:43.206487+0000","last_undegraded":"2024-09-13T08:40:43.206487+0000","last_fullsized":"2024-09-13T08:40:43.206487+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:00:32.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.169231+0000","last_change":"2024-09-13T08:40:42.177210+0000","last_active":"2024-09-13T08:40:43.169231+0000","last_peered":"2024-09-13T08:40:43.169231+0000","last_clean":"2024-09-13T08:40:43.169231+0000","last_became_active":"2024-09-13T08:40:42.176919+0000","last_became_peered":"2024-09-13T08:40:42.176919+0000","last_unstale":"2024-09-13T08:40:43.169231+0000","last_undegraded":"2024-09-13T08:40:43.169231+0000","last_fullsized":"2024-09-13T08:40:43.169231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:05.647017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.203491+0000","last_change":"2024-09-13T08:40:43.189382+0000","last_active":"2024-09-13T08:40:43.203491+0000","last_peered":"2024-09-13T08:40:43.203491+0000","last_clean":"2024-09-13T08:40:43.203491+0000","last_became_active":"2024-09-13T08:40:43.188958+0000","last_became_peered":"2024-09-13T08:40:43.188958+0000","last_unstale":"2024-09-13T08:40:43.203491+0000","last_undegraded":"2024-09-13T08:40:43.203491+0000","last_fullsized":"2024-09-13T08:40:43.203491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:12.306506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.173394+0000","last_change":"2024-09-13T08:40:42.186311+0000","last_active":"2024-09-13T08:40:43.173394+0000","last_peered":"2024-09-13T08:40:43.173394+0000","last_clean":"2024-09-13T08:40:43.173394+0000","last_became_active":"2024-09-13T08:40:42.185993+0000","last_became_peered":"2024-09-13T08:40:42.185993+0000","last_unstale":"2024-09-13T08:40:43.173394+0000","last_undegraded":"2024-09-13T08:40:43.173394+0000","last_fullsized":"2024-09-13T08:40:43.173394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:07.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189794+0000","last_change":"2024-09-13T08:40:43.184101+0000","last_active":"2024-09-13T08:40:43.189794+0000","last_peered":"2024-09-13T08:40:43.189794+0000","last_clean":"2024-09-13T08:40:43.189794+0000","last_became_active":"2024-09-13T08:40:43.183722+0000","last_became_peered":"2024-09-13T08:40:43.183722+0000","last_unstale":"2024-09-13T08:40:43.189794+0000","last_undegraded":"2024-09-13T08:40:43.189794+0000","last_fullsized":"2024-09-13T08:40:43.189794+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:46:44.174973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.181735+0000","last_change":"2024-09-13T08:40:42.189558+0000","last_active":"2024-09-13T08:40:43.181735+0000","last_peered":"2024-09-13T08:40:43.181735+0000","last_clean":"2024-09-13T08:40:43.181735+0000","last_became_active":"2024-09-13T08:40:42.189312+0000","last_became_peered":"2024-09-13T08:40:42.189312+0000","last_unstale":"2024-09-13T08:40:43.181735+0000","last_undegraded":"2024-09-13T08:40:43.181735+0000","last_fullsized":"2024-09-13T08:40:43.181735+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:40.270289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204125+0000","last_change":"2024-09-13T08:40:43.192030+0000","last_active":"2024-09-13T08:40:43.204125+0000","last_peered":"2024-09-13T08:40:43.204125+0000","last_clean":"2024-09-13T08:40:43.204125+0000","last_became_active":"2024-09-13T08:40:43.191644+0000","last_became_peered":"2024-09-13T08:40:43.191644+0000","last_unstale":"2024-09-13T08:40:43.204125+0000","last_undegraded":"2024-09-13T08:40:43.204125+0000","last_fullsized":"2024-09-13T08:40:43.204125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:38.459714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172812+0000","last_change":"2024-09-13T08:40:42.179810+0000","last_active":"2024-09-13T08:40:43.172812+0000","last_peered":"2024-09-13T08:40:43.172812+0000","last_clean":"2024-09-13T08:40:43.172812+0000","last_became_active":"2024-09-13T08:40:42.179512+0000","last_became_peered":"2024-09-13T08:40:42.179512+0000","last_unstale":"2024-09-13T08:40:43.172812+0000","last_undegraded":"2024-09-13T08:40:43.172812+0000","last_fullsized":"2024-09-13T08:40:43.172812+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:13.357647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.188531+0000","last_change":"2024-09-13T08:40:43.176719+0000","last_active":"2024-09-13T08:40:43.188531+0000","last_peered":"2024-09-13T08:40:43.188531+0000","last_clean":"2024-09-13T08:40:43.188531+0000","last_became_active":"2024-09-13T08:40:43.176416+0000","last_became_peered":"2024-09-13T08:40:43.176416+0000","last_unstale":"2024-09-13T08:40:43.188531+0000","last_undegraded":"2024-09-13T08:40:43.188531+0000","last_fullsized":"2024-09-13T08:40:43.188531+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:00.931558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.174570+0000","last_change":"2024-09-13T08:40:42.176816+0000","last_active":"2024-09-13T08:40:43.174570+0000","last_peered":"2024-09-13T08:40:43.174570+0000","last_clean":"2024-09-13T08:40:43.174570+0000","last_became_active":"2024-09-13T08:40:42.176236+0000","last_became_peered":"2024-09-13T08:40:42.176236+0000","last_unstale":"2024-09-13T08:40:43.174570+0000","last_undegraded":"2024-09-13T08:40:43.174570+0000","last_fullsized":"2024-09-13T08:40:43.174570+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:14.430820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204331+0000","last_change":"2024-09-13T08:40:43.176967+0000","last_active":"2024-09-13T08:40:43.204331+0000","last_peered":"2024-09-13T08:40:43.204331+0000","last_clean":"2024-09-13T08:40:43.204331+0000","last_became_active":"2024-09-13T08:40:43.176658+0000","last_became_peered":"2024-09-13T08:40:43.176658+0000","last_unstale":"2024-09-13T08:40:43.204331+0000","last_undegraded":"2024-09-13T08:40:43.204331+0000","last_fullsized":"2024-09-13T08:40:43.204331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:41.276450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.177057+0000","last_change":"2024-09-13T08:40:42.174657+0000","last_active":"2024-09-13T08:40:42.177057+0000","last_peered":"2024-09-13T08:40:42.177057+0000","last_clean":"2024-09-13T08:40:42.177057+0000","last_became_active":"2024-09-13T08:40:42.174374+0000","last_became_peered":"2024-09-13T08:40:42.174374+0000","last_unstale":"2024-09-13T08:40:42.177057+0000","last_undegraded":"2024-09-13T08:40:42.177057+0000","last_fullsized":"2024-09-13T08:40:42.177057+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:53:20.682799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.187158+0000","last_change":"2024-09-13T08:40:43.180762+0000","last_active":"2024-09-13T08:40:43.187158+0000","last_peered":"2024-09-13T08:40:43.187158+0000","last_clean":"2024-09-13T08:40:43.187158+0000","last_became_active":"2024-09-13T08:40:43.180576+0000","last_became_peered":"2024-09-13T08:40:43.180576+0000","last_unstale":"2024-09-13T08:40:43.187158+0000","last_undegraded":"2024-09-13T08:40:43.187158+0000","last_fullsized":"2024-09-13T08:40:43.187158+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:54.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.182098+0000","last_change":"2024-09-13T08:40:42.177987+0000","last_active":"2024-09-13T08:40:43.182098+0000","last_peered":"2024-09-13T08:40:43.182098+0000","last_clean":"2024-09-13T08:40:43.182098+0000","last_became_active":"2024-09-13T08:40:42.177653+0000","last_became_peered":"2024-09-13T08:40:42.177653+0000","last_unstale":"2024-09-13T08:40:43.182098+0000","last_undegraded":"2024-09-13T08:40:43.182098+0000","last_fullsized":"2024-09-13T08:40:43.182098+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:44.296127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190447+0000","last_change":"2024-09-13T08:40:43.180067+0000","last_active":"2024-09-13T08:40:43.190447+0000","last_peered":"2024-09-13T08:40:43.190447+0000","last_clean":"2024-09-13T08:40:43.190447+0000","last_became_active":"2024-09-13T08:40:43.179734+0000","last_became_peered":"2024-09-13T08:40:43.179734+0000","last_unstale":"2024-09-13T08:40:43.190447+0000","last_undegraded":"2024-09-13T08:40:43.190447+0000","last_fullsized":"2024-09-13T08:40:43.190447+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:33.479797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.201678+0000","last_change":"2024-09-13T08:40:42.180201+0000","last_active":"2024-09-13T08:40:43.201678+0000","last_peered":"2024-09-13T08:40:43.201678+0000","last_clean":"2024-09-13T08:40:43.201678+0000","last_became_active":"2024-09-13T08:40:42.179874+0000","last_became_peered":"2024-09-13T08:40:42.179874+0000","last_unstale":"2024-09-13T08:40:43.201678+0000","last_undegraded":"2024-09-13T08:40:43.201678+0000","last_fullsized":"2024-09-13T08:40:43.201678+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:25.549286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.171871+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:40:43.171871+0000","last_peered":"2024-09-13T08:40:43.171871+0000","last_clean":"2024-09-13T08:40:43.171871+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:40:43.171871+0000","last_undegraded":"2024-09-13T08:40:43.171871+0000","last_fullsized":"2024-09-13T08:40:43.171871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":214223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":804579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":552960,"data_stored":215893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":214861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":214279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509484,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":804627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":214861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35860,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":520192,"data_stored":207414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:40:47.742 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:40:47.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.928+0000 7fbe57200640 1 -- 172.21.15.44:0/2236814113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe580b11a0 msgr2=0x7fbe58165670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.928+0000 7fbe57200640 1 --2- 172.21.15.44:0/2236814113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe580b11a0 0x7fbe58165670 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbe58069e80 tx=0x7fbe5001cb80 comp rx=0 tx=0).stop 2024-09-13T08:40:47.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.928+0000 7fbe57200640 1 -- 172.21.15.44:0/2236814113 shutdown_connections 2024-09-13T08:40:47.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.928+0000 7fbe57200640 1 --2- 172.21.15.44:0/2236814113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe580b11a0 0x7fbe58165670 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.928+0000 7fbe57200640 1 --2- 172.21.15.44:0/2236814113 >> v2:172.21.15.129:3300/0 conn(0x7fbe580b0ce0 0x7fbe580b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.928+0000 7fbe57200640 1 -- 172.21.15.44:0/2236814113 >> 172.21.15.44:0/2236814113 conn(0x7fbe5809aa10 msgr2=0x7fbe5809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:47.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.928+0000 7fbe57200640 1 -- 172.21.15.44:0/2236814113 shutdown_connections 2024-09-13T08:40:47.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.928+0000 7fbe57200640 1 -- 172.21.15.44:0/2236814113 wait complete. 2024-09-13T08:40:47.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.928+0000 7fbe57200640 1 Processor -- start 2024-09-13T08:40:47.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.928+0000 7fbe57200640 1 -- start start 2024-09-13T08:40:47.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe57200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe580b0ce0 0x7fbe5814f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe57200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe580b11a0 0x7fbe581488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe57200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe580b4d80 0x7fbe58147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe57200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe58168a70 con 0x7fbe580b0ce0 2024-09-13T08:40:47.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe57200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe581688f0 con 0x7fbe580b11a0 2024-09-13T08:40:47.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe57200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fbe58168bf0 con 0x7fbe580b4d80 2024-09-13T08:40:47.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe55e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe580b0ce0 0x7fbe5814f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe55e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe580b0ce0 0x7fbe5814f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37466/0 (socket says 172.21.15.44:37466) 2024-09-13T08:40:47.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe55e00640 1 -- 172.21.15.44:0/546588723 learned_addr learned my addr 172.21.15.44:0/546588723 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:47.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe55400640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe580b11a0 0x7fbe581488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe56800640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe580b4d80 0x7fbe58147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe55e00640 1 -- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe580b4d80 msgr2=0x7fbe58147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe55e00640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe580b4d80 0x7fbe58147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe55e00640 1 -- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe580b11a0 msgr2=0x7fbe581488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:47.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe55e00640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe580b11a0 0x7fbe581488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:47.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe55e00640 1 -- 172.21.15.44:0/546588723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5814dd50 con 0x7fbe580b0ce0 2024-09-13T08:40:47.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe55e00640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe580b0ce0 0x7fbe5814f380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800c400 tx=0x7fbe4800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe4e000640 1 -- 172.21.15.44:0/546588723 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48011790 con 0x7fbe580b0ce0 2024-09-13T08:40:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe4e000640 1 -- 172.21.15.44:0/546588723 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe48017070 con 0x7fbe580b0ce0 2024-09-13T08:40:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe57200640 1 -- 172.21.15.44:0/546588723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5814e030 con 0x7fbe580b0ce0 2024-09-13T08:40:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe4e000640 1 -- 172.21.15.44:0/546588723 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48011cc0 con 0x7fbe580b0ce0 2024-09-13T08:40:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe57200640 1 -- 172.21.15.44:0/546588723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5814e5f0 con 0x7fbe580b0ce0 2024-09-13T08:40:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.932+0000 7fbe57200640 1 -- 172.21.15.44:0/546588723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe581f8dc0 con 0x7fbe580b0ce0 2024-09-13T08:40:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.936+0000 7fbe4e000640 1 -- 172.21.15.44:0/546588723 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fbe4801c460 con 0x7fbe580b0ce0 2024-09-13T08:40:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.936+0000 7fbe4e000640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe38073250 0x7fbe38075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.936+0000 7fbe4e000640 1 -- 172.21.15.44:0/546588723 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fbe48094020 con 0x7fbe580b0ce0 2024-09-13T08:40:47.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.936+0000 7fbe55400640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe38073250 0x7fbe38075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:47.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.936+0000 7fbe55400640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe38073250 0x7fbe38075710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbe5815ad00 tx=0x7fbe4400b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:47.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:47.940+0000 7fbe4e000640 1 -- 172.21.15.44:0/546588723 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fbe4805e750 con 0x7fbe580b0ce0 2024-09-13T08:40:48.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.112+0000 7fbe57200640 1 -- 172.21.15.44:0/546588723 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe580ce0c0 con 0x7fbe38073250 2024-09-13T08:40:48.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.116+0000 7fbe4e000640 1 -- 172.21.15.44:0/546588723 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251545 (secure 0 0 0) 0x7fbe580ce0c0 con 0x7fbe38073250 2024-09-13T08:40:48.122 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:48.123 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:40:48.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 -- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe38073250 msgr2=0x7fbe38075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe38073250 0x7fbe38075710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbe5815ad00 tx=0x7fbe4400b390 comp rx=0 tx=0).stop 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 -- 172.21.15.44:0/546588723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe580b0ce0 msgr2=0x7fbe5814f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe580b0ce0 0x7fbe5814f380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800c400 tx=0x7fbe4800c8c0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 -- 172.21.15.44:0/546588723 shutdown_connections 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe38073250 0x7fbe38075710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe580b4d80 0x7fbe58147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe580b11a0 0x7fbe581488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 --2- 172.21.15.44:0/546588723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe580b0ce0 0x7fbe5814f380 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 -- 172.21.15.44:0/546588723 >> 172.21.15.44:0/546588723 conn(0x7fbe5809aa10 msgr2=0x7fbe58071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 -- 172.21.15.44:0/546588723 shutdown_connections 2024-09-13T08:40:48.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.120+0000 7fbe57200640 1 -- 172.21.15.44:0/546588723 wait complete. 2024-09-13T08:40:48.127 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-09-13T08:40:48.124+0000 7fd402400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:40:48.143 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-13T08:40:46.557861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263220,"kb_used_data":5360,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711500,"statfs":{"total":773094113280,"available":772824576000,"internally_reserved":0,"allocated":5488640,"data_stored":2890737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.403999"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193015+0000","last_change":"2024-09-13T08:40:43.188161+0000","last_active":"2024-09-13T08:40:43.193015+0000","last_peered":"2024-09-13T08:40:43.193015+0000","last_clean":"2024-09-13T08:40:43.193015+0000","last_became_active":"2024-09-13T08:40:43.187905+0000","last_became_peered":"2024-09-13T08:40:43.187905+0000","last_unstale":"2024-09-13T08:40:43.193015+0000","last_undegraded":"2024-09-13T08:40:43.193015+0000","last_fullsized":"2024-09-13T08:40:43.193015+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199380+0000","last_change":"2024-09-13T08:40:43.193493+0000","last_active":"2024-09-13T08:40:43.199380+0000","last_peered":"2024-09-13T08:40:43.199380+0000","last_clean":"2024-09-13T08:40:43.199380+0000","last_became_active":"2024-09-13T08:40:43.193324+0000","last_became_peered":"2024-09-13T08:40:43.193324+0000","last_unstale":"2024-09-13T08:40:43.199380+0000","last_undegraded":"2024-09-13T08:40:43.199380+0000","last_fullsized":"2024-09-13T08:40:43.199380+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207011+0000","last_change":"2024-09-13T08:40:43.192784+0000","last_active":"2024-09-13T08:40:43.207011+0000","last_peered":"2024-09-13T08:40:43.207011+0000","last_clean":"2024-09-13T08:40:43.207011+0000","last_became_active":"2024-09-13T08:40:43.192234+0000","last_became_peered":"2024-09-13T08:40:43.192234+0000","last_unstale":"2024-09-13T08:40:43.207011+0000","last_undegraded":"2024-09-13T08:40:43.207011+0000","last_fullsized":"2024-09-13T08:40:43.207011+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:56:45.062628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193097+0000","last_change":"2024-09-13T08:40:43.189058+0000","last_active":"2024-09-13T08:40:43.193097+0000","last_peered":"2024-09-13T08:40:43.193097+0000","last_clean":"2024-09-13T08:40:43.193097+0000","last_became_active":"2024-09-13T08:40:43.188732+0000","last_became_peered":"2024-09-13T08:40:43.188732+0000","last_unstale":"2024-09-13T08:40:43.193097+0000","last_undegraded":"2024-09-13T08:40:43.193097+0000","last_fullsized":"2024-09-13T08:40:43.193097+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207327+0000","last_change":"2024-09-13T08:40:43.204809+0000","last_active":"2024-09-13T08:40:43.207327+0000","last_peered":"2024-09-13T08:40:43.207327+0000","last_clean":"2024-09-13T08:40:43.207327+0000","last_became_active":"2024-09-13T08:40:43.204599+0000","last_became_peered":"2024-09-13T08:40:43.204599+0000","last_unstale":"2024-09-13T08:40:43.207327+0000","last_undegraded":"2024-09-13T08:40:43.207327+0000","last_fullsized":"2024-09-13T08:40:43.207327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:20.538522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157508+0000","last_change":"2024-09-13T08:40:42.157508+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157508+0000","last_undegraded":"2024-09-13T08:40:42.157508+0000","last_fullsized":"2024-09-13T08:40:42.157508+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189864+0000","last_change":"2024-09-13T08:40:43.184765+0000","last_active":"2024-09-13T08:40:43.189864+0000","last_peered":"2024-09-13T08:40:43.189864+0000","last_clean":"2024-09-13T08:40:43.189864+0000","last_became_active":"2024-09-13T08:40:43.184173+0000","last_became_peered":"2024-09-13T08:40:43.184173+0000","last_unstale":"2024-09-13T08:40:43.189864+0000","last_undegraded":"2024-09-13T08:40:43.189864+0000","last_fullsized":"2024-09-13T08:40:43.189864+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:59.176747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190485+0000","last_change":"2024-09-13T08:40:43.180402+0000","last_active":"2024-09-13T08:40:43.190485+0000","last_peered":"2024-09-13T08:40:43.190485+0000","last_clean":"2024-09-13T08:40:43.190485+0000","last_became_active":"2024-09-13T08:40:43.179682+0000","last_became_peered":"2024-09-13T08:40:43.179682+0000","last_unstale":"2024-09-13T08:40:43.190485+0000","last_undegraded":"2024-09-13T08:40:43.190485+0000","last_fullsized":"2024-09-13T08:40:43.190485+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157861+0000","last_change":"2024-09-13T08:40:42.157861+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157861+0000","last_undegraded":"2024-09-13T08:40:42.157861+0000","last_fullsized":"2024-09-13T08:40:42.157861+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189100+0000","last_change":"2024-09-13T08:40:43.183571+0000","last_active":"2024-09-13T08:40:43.189100+0000","last_peered":"2024-09-13T08:40:43.189100+0000","last_clean":"2024-09-13T08:40:43.189100+0000","last_became_active":"2024-09-13T08:40:43.183297+0000","last_became_peered":"2024-09-13T08:40:43.183297+0000","last_unstale":"2024-09-13T08:40:43.189100+0000","last_undegraded":"2024-09-13T08:40:43.189100+0000","last_fullsized":"2024-09-13T08:40:43.189100+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:14.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206352+0000","last_change":"2024-09-13T08:40:43.200664+0000","last_active":"2024-09-13T08:40:43.206352+0000","last_peered":"2024-09-13T08:40:43.206352+0000","last_clean":"2024-09-13T08:40:43.206352+0000","last_became_active":"2024-09-13T08:40:43.200401+0000","last_became_peered":"2024-09-13T08:40:43.200401+0000","last_unstale":"2024-09-13T08:40:43.206352+0000","last_undegraded":"2024-09-13T08:40:43.206352+0000","last_fullsized":"2024-09-13T08:40:43.206352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:14.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199560+0000","last_change":"2024-09-13T08:40:43.188349+0000","last_active":"2024-09-13T08:40:43.199560+0000","last_peered":"2024-09-13T08:40:43.199560+0000","last_clean":"2024-09-13T08:40:43.199560+0000","last_became_active":"2024-09-13T08:40:43.188067+0000","last_became_peered":"2024-09-13T08:40:43.188067+0000","last_unstale":"2024-09-13T08:40:43.199560+0000","last_undegraded":"2024-09-13T08:40:43.199560+0000","last_fullsized":"2024-09-13T08:40:43.199560+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.158438+0000","last_change":"2024-09-13T08:40:42.158438+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.158438+0000","last_undegraded":"2024-09-13T08:40:42.158438+0000","last_fullsized":"2024-09-13T08:40:42.158438+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193350+0000","last_change":"2024-09-13T08:40:43.187316+0000","last_active":"2024-09-13T08:40:43.193350+0000","last_peered":"2024-09-13T08:40:43.193350+0000","last_clean":"2024-09-13T08:40:43.193350+0000","last_became_active":"2024-09-13T08:40:43.187050+0000","last_became_peered":"2024-09-13T08:40:43.187050+0000","last_unstale":"2024-09-13T08:40:43.193350+0000","last_undegraded":"2024-09-13T08:40:43.193350+0000","last_fullsized":"2024-09-13T08:40:43.193350+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189919+0000","last_change":"2024-09-13T08:40:43.178849+0000","last_active":"2024-09-13T08:40:43.189919+0000","last_peered":"2024-09-13T08:40:43.189919+0000","last_clean":"2024-09-13T08:40:43.189919+0000","last_became_active":"2024-09-13T08:40:43.178581+0000","last_became_peered":"2024-09-13T08:40:43.178581+0000","last_unstale":"2024-09-13T08:40:43.189919+0000","last_undegraded":"2024-09-13T08:40:43.189919+0000","last_fullsized":"2024-09-13T08:40:43.189919+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:08.944968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.182653+0000","last_change":"2024-09-13T08:40:43.178443+0000","last_active":"2024-09-13T08:40:43.182653+0000","last_peered":"2024-09-13T08:40:43.182653+0000","last_clean":"2024-09-13T08:40:43.182653+0000","last_became_active":"2024-09-13T08:40:43.178156+0000","last_became_peered":"2024-09-13T08:40:43.178156+0000","last_unstale":"2024-09-13T08:40:43.182653+0000","last_undegraded":"2024-09-13T08:40:43.182653+0000","last_fullsized":"2024-09-13T08:40:43.182653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.186648+0000","last_change":"2024-09-13T08:40:43.181966+0000","last_active":"2024-09-13T08:40:43.186648+0000","last_peered":"2024-09-13T08:40:43.186648+0000","last_clean":"2024-09-13T08:40:43.186648+0000","last_became_active":"2024-09-13T08:40:43.181651+0000","last_became_peered":"2024-09-13T08:40:43.181651+0000","last_unstale":"2024-09-13T08:40:43.186648+0000","last_undegraded":"2024-09-13T08:40:43.186648+0000","last_fullsized":"2024-09-13T08:40:43.186648+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:33.479797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207483+0000","last_change":"2024-09-13T08:40:43.203044+0000","last_active":"2024-09-13T08:40:43.207483+0000","last_peered":"2024-09-13T08:40:43.207483+0000","last_clean":"2024-09-13T08:40:43.207483+0000","last_became_active":"2024-09-13T08:40:43.196038+0000","last_became_peered":"2024-09-13T08:40:43.196038+0000","last_unstale":"2024-09-13T08:40:43.207483+0000","last_undegraded":"2024-09-13T08:40:43.207483+0000","last_fullsized":"2024-09-13T08:40:43.207483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:49:13.250832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207072+0000","last_change":"2024-09-13T08:40:43.200673+0000","last_active":"2024-09-13T08:40:43.207072+0000","last_peered":"2024-09-13T08:40:43.207072+0000","last_clean":"2024-09-13T08:40:43.207072+0000","last_became_active":"2024-09-13T08:40:43.200103+0000","last_became_peered":"2024-09-13T08:40:43.200103+0000","last_unstale":"2024-09-13T08:40:43.207072+0000","last_undegraded":"2024-09-13T08:40:43.207072+0000","last_fullsized":"2024-09-13T08:40:43.207072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:21.298332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.203522+0000","last_change":"2024-09-13T08:40:43.191159+0000","last_active":"2024-09-13T08:40:43.203522+0000","last_peered":"2024-09-13T08:40:43.203522+0000","last_clean":"2024-09-13T08:40:43.203522+0000","last_became_active":"2024-09-13T08:40:43.190876+0000","last_became_peered":"2024-09-13T08:40:43.190876+0000","last_unstale":"2024-09-13T08:40:43.203522+0000","last_undegraded":"2024-09-13T08:40:43.203522+0000","last_fullsized":"2024-09-13T08:40:43.203522+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:49.726267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190142+0000","last_change":"2024-09-13T08:40:43.176763+0000","last_active":"2024-09-13T08:40:43.190142+0000","last_peered":"2024-09-13T08:40:43.190142+0000","last_clean":"2024-09-13T08:40:43.190142+0000","last_became_active":"2024-09-13T08:40:43.176429+0000","last_became_peered":"2024-09-13T08:40:43.176429+0000","last_unstale":"2024-09-13T08:40:43.190142+0000","last_undegraded":"2024-09-13T08:40:43.190142+0000","last_fullsized":"2024-09-13T08:40:43.190142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:14.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.211767+0000","last_change":"2024-09-13T08:40:43.207935+0000","last_active":"2024-09-13T08:40:43.211767+0000","last_peered":"2024-09-13T08:40:43.211767+0000","last_clean":"2024-09-13T08:40:43.211767+0000","last_became_active":"2024-09-13T08:40:43.207661+0000","last_became_peered":"2024-09-13T08:40:43.207661+0000","last_unstale":"2024-09-13T08:40:43.211767+0000","last_undegraded":"2024-09-13T08:40:43.211767+0000","last_fullsized":"2024-09-13T08:40:43.211767+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:43.583764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189838+0000","last_change":"2024-09-13T08:40:43.184477+0000","last_active":"2024-09-13T08:40:43.189838+0000","last_peered":"2024-09-13T08:40:43.189838+0000","last_clean":"2024-09-13T08:40:43.189838+0000","last_became_active":"2024-09-13T08:40:43.183611+0000","last_became_peered":"2024-09-13T08:40:43.183611+0000","last_unstale":"2024-09-13T08:40:43.189838+0000","last_undegraded":"2024-09-13T08:40:43.189838+0000","last_fullsized":"2024-09-13T08:40:43.189838+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:33:32.215525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.175547+0000","last_change":"2024-09-13T08:40:43.173503+0000","last_active":"2024-09-13T08:40:43.175547+0000","last_peered":"2024-09-13T08:40:43.175547+0000","last_clean":"2024-09-13T08:40:43.175547+0000","last_became_active":"2024-09-13T08:40:43.173209+0000","last_became_peered":"2024-09-13T08:40:43.173209+0000","last_unstale":"2024-09-13T08:40:43.175547+0000","last_undegraded":"2024-09-13T08:40:43.175547+0000","last_fullsized":"2024-09-13T08:40:43.175547+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:00.931558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193040+0000","last_change":"2024-09-13T08:40:43.188762+0000","last_active":"2024-09-13T08:40:43.193040+0000","last_peered":"2024-09-13T08:40:43.193040+0000","last_clean":"2024-09-13T08:40:43.193040+0000","last_became_active":"2024-09-13T08:40:43.188481+0000","last_became_peered":"2024-09-13T08:40:43.188481+0000","last_unstale":"2024-09-13T08:40:43.193040+0000","last_undegraded":"2024-09-13T08:40:43.193040+0000","last_fullsized":"2024-09-13T08:40:43.193040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206308+0000","last_change":"2024-09-13T08:40:43.201185+0000","last_active":"2024-09-13T08:40:43.206308+0000","last_peered":"2024-09-13T08:40:43.206308+0000","last_clean":"2024-09-13T08:40:43.206308+0000","last_became_active":"2024-09-13T08:40:43.200898+0000","last_became_peered":"2024-09-13T08:40:43.200898+0000","last_unstale":"2024-09-13T08:40:43.206308+0000","last_undegraded":"2024-09-13T08:40:43.206308+0000","last_fullsized":"2024-09-13T08:40:43.206308+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:32.631857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212364+0000","last_change":"2024-09-13T08:40:43.208083+0000","last_active":"2024-09-13T08:40:43.212364+0000","last_peered":"2024-09-13T08:40:43.212364+0000","last_clean":"2024-09-13T08:40:43.212364+0000","last_became_active":"2024-09-13T08:40:43.207841+0000","last_became_peered":"2024-09-13T08:40:43.207841+0000","last_unstale":"2024-09-13T08:40:43.212364+0000","last_undegraded":"2024-09-13T08:40:43.212364+0000","last_fullsized":"2024-09-13T08:40:43.212364+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:29:52.299097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207593+0000","last_change":"2024-09-13T08:40:43.198438+0000","last_active":"2024-09-13T08:40:43.207593+0000","last_peered":"2024-09-13T08:40:43.207593+0000","last_clean":"2024-09-13T08:40:43.207593+0000","last_became_active":"2024-09-13T08:40:43.197966+0000","last_became_peered":"2024-09-13T08:40:43.197966+0000","last_unstale":"2024-09-13T08:40:43.207593+0000","last_undegraded":"2024-09-13T08:40:43.207593+0000","last_fullsized":"2024-09-13T08:40:43.207593+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:54:02.077806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197404+0000","last_change":"2024-09-13T08:40:43.191493+0000","last_active":"2024-09-13T08:40:43.197404+0000","last_peered":"2024-09-13T08:40:43.197404+0000","last_clean":"2024-09-13T08:40:43.197404+0000","last_became_active":"2024-09-13T08:40:43.191059+0000","last_became_peered":"2024-09-13T08:40:43.191059+0000","last_unstale":"2024-09-13T08:40:43.197404+0000","last_undegraded":"2024-09-13T08:40:43.197404+0000","last_fullsized":"2024-09-13T08:40:43.197404+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.186580+0000","last_change":"2024-09-13T08:40:43.182477+0000","last_active":"2024-09-13T08:40:43.186580+0000","last_peered":"2024-09-13T08:40:43.186580+0000","last_clean":"2024-09-13T08:40:43.186580+0000","last_became_active":"2024-09-13T08:40:43.182226+0000","last_became_peered":"2024-09-13T08:40:43.182226+0000","last_unstale":"2024-09-13T08:40:43.186580+0000","last_undegraded":"2024-09-13T08:40:43.186580+0000","last_fullsized":"2024-09-13T08:40:43.186580+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157876+0000","last_change":"2024-09-13T08:40:42.157876+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157876+0000","last_undegraded":"2024-09-13T08:40:42.157876+0000","last_fullsized":"2024-09-13T08:40:42.157876+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193060+0000","last_change":"2024-09-13T08:40:43.189479+0000","last_active":"2024-09-13T08:40:43.193060+0000","last_peered":"2024-09-13T08:40:43.193060+0000","last_clean":"2024-09-13T08:40:43.193060+0000","last_became_active":"2024-09-13T08:40:43.189207+0000","last_became_peered":"2024-09-13T08:40:43.189207+0000","last_unstale":"2024-09-13T08:40:43.193060+0000","last_undegraded":"2024-09-13T08:40:43.193060+0000","last_fullsized":"2024-09-13T08:40:43.193060+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:37.772751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197183+0000","last_change":"2024-09-13T08:40:43.193080+0000","last_active":"2024-09-13T08:40:43.197183+0000","last_peered":"2024-09-13T08:40:43.197183+0000","last_clean":"2024-09-13T08:40:43.197183+0000","last_became_active":"2024-09-13T08:40:43.192785+0000","last_became_peered":"2024-09-13T08:40:43.192785+0000","last_unstale":"2024-09-13T08:40:43.197183+0000","last_undegraded":"2024-09-13T08:40:43.197183+0000","last_fullsized":"2024-09-13T08:40:43.197183+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:58.112836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.178584+0000","last_change":"2024-09-13T08:40:42.175868+0000","last_active":"2024-09-13T08:40:42.178584+0000","last_peered":"2024-09-13T08:40:42.178584+0000","last_clean":"2024-09-13T08:40:42.178584+0000","last_became_active":"2024-09-13T08:40:42.175587+0000","last_became_peered":"2024-09-13T08:40:42.175587+0000","last_unstale":"2024-09-13T08:40:42.178584+0000","last_undegraded":"2024-09-13T08:40:42.178584+0000","last_fullsized":"2024-09-13T08:40:42.178584+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:33.183213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198234+0000","last_change":"2024-09-13T08:40:43.194685+0000","last_active":"2024-09-13T08:40:43.198234+0000","last_peered":"2024-09-13T08:40:43.198234+0000","last_clean":"2024-09-13T08:40:43.198234+0000","last_became_active":"2024-09-13T08:40:43.194145+0000","last_became_peered":"2024-09-13T08:40:43.194145+0000","last_unstale":"2024-09-13T08:40:43.198234+0000","last_undegraded":"2024-09-13T08:40:43.198234+0000","last_fullsized":"2024-09-13T08:40:43.198234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:37.772751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206000+0000","last_change":"2024-09-13T08:40:42.182278+0000","last_active":"2024-09-13T08:40:43.206000+0000","last_peered":"2024-09-13T08:40:43.206000+0000","last_clean":"2024-09-13T08:40:43.206000+0000","last_became_active":"2024-09-13T08:40:42.181992+0000","last_became_peered":"2024-09-13T08:40:42.181992+0000","last_unstale":"2024-09-13T08:40:43.206000+0000","last_undegraded":"2024-09-13T08:40:43.206000+0000","last_fullsized":"2024-09-13T08:40:43.206000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:51.100086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199534+0000","last_change":"2024-09-13T08:40:43.187867+0000","last_active":"2024-09-13T08:40:43.199534+0000","last_peered":"2024-09-13T08:40:43.199534+0000","last_clean":"2024-09-13T08:40:43.199534+0000","last_became_active":"2024-09-13T08:40:43.187621+0000","last_became_peered":"2024-09-13T08:40:43.187621+0000","last_unstale":"2024-09-13T08:40:43.199534+0000","last_undegraded":"2024-09-13T08:40:43.199534+0000","last_fullsized":"2024-09-13T08:40:43.199534+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.179942+0000","last_change":"2024-09-13T08:40:42.179208+0000","last_active":"2024-09-13T08:40:42.179942+0000","last_peered":"2024-09-13T08:40:42.179942+0000","last_clean":"2024-09-13T08:40:42.179942+0000","last_became_active":"2024-09-13T08:40:42.178617+0000","last_became_peered":"2024-09-13T08:40:42.178617+0000","last_unstale":"2024-09-13T08:40:42.179942+0000","last_undegraded":"2024-09-13T08:40:42.179942+0000","last_fullsized":"2024-09-13T08:40:42.179942+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:01:20.323316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198747+0000","last_change":"2024-09-13T08:40:43.192425+0000","last_active":"2024-09-13T08:40:43.198747+0000","last_peered":"2024-09-13T08:40:43.198747+0000","last_clean":"2024-09-13T08:40:43.198747+0000","last_became_active":"2024-09-13T08:40:43.191801+0000","last_became_peered":"2024-09-13T08:40:43.191801+0000","last_unstale":"2024-09-13T08:40:43.198747+0000","last_undegraded":"2024-09-13T08:40:43.198747+0000","last_fullsized":"2024-09-13T08:40:43.198747+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:11.173292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205568+0000","last_change":"2024-09-13T08:40:42.180695+0000","last_active":"2024-09-13T08:40:43.205568+0000","last_peered":"2024-09-13T08:40:43.205568+0000","last_clean":"2024-09-13T08:40:43.205568+0000","last_became_active":"2024-09-13T08:40:42.180392+0000","last_became_peered":"2024-09-13T08:40:42.180392+0000","last_unstale":"2024-09-13T08:40:43.205568+0000","last_undegraded":"2024-09-13T08:40:43.205568+0000","last_fullsized":"2024-09-13T08:40:43.205568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:40.439749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212438+0000","last_change":"2024-09-13T08:40:43.209100+0000","last_active":"2024-09-13T08:40:43.212438+0000","last_peered":"2024-09-13T08:40:43.212438+0000","last_clean":"2024-09-13T08:40:43.212438+0000","last_became_active":"2024-09-13T08:40:43.208834+0000","last_became_peered":"2024-09-13T08:40:43.208834+0000","last_unstale":"2024-09-13T08:40:43.212438+0000","last_undegraded":"2024-09-13T08:40:43.212438+0000","last_fullsized":"2024-09-13T08:40:43.212438+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:29:51.212164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.168839+0000","last_change":"2024-09-13T08:40:42.166893+0000","last_active":"2024-09-13T08:40:42.168839+0000","last_peered":"2024-09-13T08:40:42.168839+0000","last_clean":"2024-09-13T08:40:42.168839+0000","last_became_active":"2024-09-13T08:40:42.166507+0000","last_became_peered":"2024-09-13T08:40:42.166507+0000","last_unstale":"2024-09-13T08:40:42.168839+0000","last_undegraded":"2024-09-13T08:40:42.168839+0000","last_fullsized":"2024-09-13T08:40:42.168839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:53:24.395981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205301+0000","last_change":"2024-09-13T08:40:43.193379+0000","last_active":"2024-09-13T08:40:43.205301+0000","last_peered":"2024-09-13T08:40:43.205301+0000","last_clean":"2024-09-13T08:40:43.205301+0000","last_became_active":"2024-09-13T08:40:43.193015+0000","last_became_peered":"2024-09-13T08:40:43.193015+0000","last_unstale":"2024-09-13T08:40:43.205301+0000","last_undegraded":"2024-09-13T08:40:43.205301+0000","last_fullsized":"2024-09-13T08:40:43.205301+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:19.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.201537+0000","last_change":"2024-09-13T08:40:42.184150+0000","last_active":"2024-09-13T08:40:43.201537+0000","last_peered":"2024-09-13T08:40:43.201537+0000","last_clean":"2024-09-13T08:40:43.201537+0000","last_became_active":"2024-09-13T08:40:42.183763+0000","last_became_peered":"2024-09-13T08:40:42.183763+0000","last_unstale":"2024-09-13T08:40:43.201537+0000","last_undegraded":"2024-09-13T08:40:43.201537+0000","last_fullsized":"2024-09-13T08:40:43.201537+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:59.988155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190512+0000","last_change":"2024-09-13T08:40:43.187575+0000","last_active":"2024-09-13T08:40:43.190512+0000","last_peered":"2024-09-13T08:40:43.190512+0000","last_clean":"2024-09-13T08:40:43.190512+0000","last_became_active":"2024-09-13T08:40:43.186758+0000","last_became_peered":"2024-09-13T08:40:43.186758+0000","last_unstale":"2024-09-13T08:40:43.190512+0000","last_undegraded":"2024-09-13T08:40:43.190512+0000","last_fullsized":"2024-09-13T08:40:43.190512+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:55.795423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172776+0000","last_change":"2024-09-13T08:40:42.175361+0000","last_active":"2024-09-13T08:40:43.172776+0000","last_peered":"2024-09-13T08:40:43.172776+0000","last_clean":"2024-09-13T08:40:43.172776+0000","last_became_active":"2024-09-13T08:40:42.174980+0000","last_became_peered":"2024-09-13T08:40:42.174980+0000","last_unstale":"2024-09-13T08:40:43.172776+0000","last_undegraded":"2024-09-13T08:40:43.172776+0000","last_fullsized":"2024-09-13T08:40:43.172776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:59.925397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206690+0000","last_change":"2024-09-13T08:40:43.200353+0000","last_active":"2024-09-13T08:40:43.206690+0000","last_peered":"2024-09-13T08:40:43.206690+0000","last_clean":"2024-09-13T08:40:43.206690+0000","last_became_active":"2024-09-13T08:40:43.198409+0000","last_became_peered":"2024-09-13T08:40:43.198409+0000","last_unstale":"2024-09-13T08:40:43.206690+0000","last_undegraded":"2024-09-13T08:40:43.206690+0000","last_fullsized":"2024-09-13T08:40:43.206690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:39.828271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.178415+0000","last_change":"2024-09-13T08:40:42.178338+0000","last_active":"2024-09-13T08:40:43.178415+0000","last_peered":"2024-09-13T08:40:43.178415+0000","last_clean":"2024-09-13T08:40:43.178415+0000","last_became_active":"2024-09-13T08:40:42.178025+0000","last_became_peered":"2024-09-13T08:40:42.178025+0000","last_unstale":"2024-09-13T08:40:43.178415+0000","last_undegraded":"2024-09-13T08:40:43.178415+0000","last_fullsized":"2024-09-13T08:40:43.178415+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:31.891865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.210052+0000","last_change":"2024-09-13T08:40:43.204789+0000","last_active":"2024-09-13T08:40:43.210052+0000","last_peered":"2024-09-13T08:40:43.210052+0000","last_clean":"2024-09-13T08:40:43.210052+0000","last_became_active":"2024-09-13T08:40:43.204114+0000","last_became_peered":"2024-09-13T08:40:43.204114+0000","last_unstale":"2024-09-13T08:40:43.210052+0000","last_undegraded":"2024-09-13T08:40:43.210052+0000","last_fullsized":"2024-09-13T08:40:43.210052+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:31.140294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.174034+0000","last_change":"2024-09-13T08:40:42.178810+0000","last_active":"2024-09-13T08:40:43.174034+0000","last_peered":"2024-09-13T08:40:43.174034+0000","last_clean":"2024-09-13T08:40:43.174034+0000","last_became_active":"2024-09-13T08:40:42.173486+0000","last_became_peered":"2024-09-13T08:40:42.173486+0000","last_unstale":"2024-09-13T08:40:43.174034+0000","last_undegraded":"2024-09-13T08:40:43.174034+0000","last_fullsized":"2024-09-13T08:40:43.174034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:12.229793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207136+0000","last_change":"2024-09-13T08:40:43.194106+0000","last_active":"2024-09-13T08:40:43.207136+0000","last_peered":"2024-09-13T08:40:43.207136+0000","last_clean":"2024-09-13T08:40:43.207136+0000","last_became_active":"2024-09-13T08:40:43.193804+0000","last_became_peered":"2024-09-13T08:40:43.193804+0000","last_unstale":"2024-09-13T08:40:43.207136+0000","last_undegraded":"2024-09-13T08:40:43.207136+0000","last_fullsized":"2024-09-13T08:40:43.207136+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.417971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.183104+0000","last_change":"2024-09-13T08:40:42.183102+0000","last_active":"2024-09-13T08:40:43.183104+0000","last_peered":"2024-09-13T08:40:43.183104+0000","last_clean":"2024-09-13T08:40:43.183104+0000","last_became_active":"2024-09-13T08:40:42.182748+0000","last_became_peered":"2024-09-13T08:40:42.182748+0000","last_unstale":"2024-09-13T08:40:43.183104+0000","last_undegraded":"2024-09-13T08:40:43.183104+0000","last_fullsized":"2024-09-13T08:40:43.183104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:10.167131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.195996+0000","last_change":"2024-09-13T08:40:43.191524+0000","last_active":"2024-09-13T08:40:43.195996+0000","last_peered":"2024-09-13T08:40:43.195996+0000","last_clean":"2024-09-13T08:40:43.195996+0000","last_became_active":"2024-09-13T08:40:43.191240+0000","last_became_peered":"2024-09-13T08:40:43.191240+0000","last_unstale":"2024-09-13T08:40:43.195996+0000","last_undegraded":"2024-09-13T08:40:43.195996+0000","last_fullsized":"2024-09-13T08:40:43.195996+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:33.508450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.209427+0000","last_change":"2024-09-13T08:40:42.177115+0000","last_active":"2024-09-13T08:40:43.209427+0000","last_peered":"2024-09-13T08:40:43.209427+0000","last_clean":"2024-09-13T08:40:43.209427+0000","last_became_active":"2024-09-13T08:40:42.176751+0000","last_became_peered":"2024-09-13T08:40:42.176751+0000","last_unstale":"2024-09-13T08:40:43.209427+0000","last_undegraded":"2024-09-13T08:40:43.209427+0000","last_fullsized":"2024-09-13T08:40:43.209427+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:23.536012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206408+0000","last_change":"2024-09-13T08:40:43.197281+0000","last_active":"2024-09-13T08:40:43.206408+0000","last_peered":"2024-09-13T08:40:43.206408+0000","last_clean":"2024-09-13T08:40:43.206408+0000","last_became_active":"2024-09-13T08:40:43.196989+0000","last_became_peered":"2024-09-13T08:40:43.196989+0000","last_unstale":"2024-09-13T08:40:43.206408+0000","last_undegraded":"2024-09-13T08:40:43.206408+0000","last_fullsized":"2024-09-13T08:40:43.206408+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:03.667062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.214312+0000","last_change":"2024-09-13T08:40:42.180947+0000","last_active":"2024-09-13T08:40:43.214312+0000","last_peered":"2024-09-13T08:40:43.214312+0000","last_clean":"2024-09-13T08:40:43.214312+0000","last_became_active":"2024-09-13T08:40:42.180647+0000","last_became_peered":"2024-09-13T08:40:42.180647+0000","last_unstale":"2024-09-13T08:40:43.214312+0000","last_undegraded":"2024-09-13T08:40:43.214312+0000","last_fullsized":"2024-09-13T08:40:43.214312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:31.625696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204310+0000","last_change":"2024-09-13T08:40:43.177272+0000","last_active":"2024-09-13T08:40:43.204310+0000","last_peered":"2024-09-13T08:40:43.204310+0000","last_clean":"2024-09-13T08:40:43.204310+0000","last_became_active":"2024-09-13T08:40:43.176928+0000","last_became_peered":"2024-09-13T08:40:43.176928+0000","last_unstale":"2024-09-13T08:40:43.204310+0000","last_undegraded":"2024-09-13T08:40:43.204310+0000","last_fullsized":"2024-09-13T08:40:43.204310+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:19.727346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204104+0000","last_change":"2024-09-13T08:40:42.176291+0000","last_active":"2024-09-13T08:40:43.204104+0000","last_peered":"2024-09-13T08:40:43.204104+0000","last_clean":"2024-09-13T08:40:43.204104+0000","last_became_active":"2024-09-13T08:40:42.175697+0000","last_became_peered":"2024-09-13T08:40:42.175697+0000","last_unstale":"2024-09-13T08:40:43.204104+0000","last_undegraded":"2024-09-13T08:40:43.204104+0000","last_fullsized":"2024-09-13T08:40:43.204104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:18.721185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198468+0000","last_change":"2024-09-13T08:40:43.194019+0000","last_active":"2024-09-13T08:40:43.198468+0000","last_peered":"2024-09-13T08:40:43.198468+0000","last_clean":"2024-09-13T08:40:43.198468+0000","last_became_active":"2024-09-13T08:40:43.192433+0000","last_became_peered":"2024-09-13T08:40:43.192433+0000","last_unstale":"2024-09-13T08:40:43.198468+0000","last_undegraded":"2024-09-13T08:40:43.198468+0000","last_fullsized":"2024-09-13T08:40:43.198468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:49:13.250832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172856+0000","last_change":"2024-09-13T08:40:42.176544+0000","last_active":"2024-09-13T08:40:43.172856+0000","last_peered":"2024-09-13T08:40:43.172856+0000","last_clean":"2024-09-13T08:40:43.172856+0000","last_became_active":"2024-09-13T08:40:42.175781+0000","last_became_peered":"2024-09-13T08:40:42.175781+0000","last_unstale":"2024-09-13T08:40:43.172856+0000","last_undegraded":"2024-09-13T08:40:43.172856+0000","last_fullsized":"2024-09-13T08:40:43.172856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:53.422970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198512+0000","last_change":"2024-09-13T08:40:43.195108+0000","last_active":"2024-09-13T08:40:43.198512+0000","last_peered":"2024-09-13T08:40:43.198512+0000","last_clean":"2024-09-13T08:40:43.198512+0000","last_became_active":"2024-09-13T08:40:43.193623+0000","last_became_peered":"2024-09-13T08:40:43.193623+0000","last_unstale":"2024-09-13T08:40:43.198512+0000","last_undegraded":"2024-09-13T08:40:43.198512+0000","last_fullsized":"2024-09-13T08:40:43.198512+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:31.140294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.185074+0000","last_change":"2024-09-13T08:40:42.184583+0000","last_active":"2024-09-13T08:40:42.185074+0000","last_peered":"2024-09-13T08:40:42.185074+0000","last_clean":"2024-09-13T08:40:42.185074+0000","last_became_active":"2024-09-13T08:40:42.184269+0000","last_became_peered":"2024-09-13T08:40:42.184269+0000","last_unstale":"2024-09-13T08:40:42.185074+0000","last_undegraded":"2024-09-13T08:40:42.185074+0000","last_fullsized":"2024-09-13T08:40:42.185074+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:55.743786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.187184+0000","last_change":"2024-09-13T08:40:43.181834+0000","last_active":"2024-09-13T08:40:43.187184+0000","last_peered":"2024-09-13T08:40:43.187184+0000","last_clean":"2024-09-13T08:40:43.187184+0000","last_became_active":"2024-09-13T08:40:43.181558+0000","last_became_peered":"2024-09-13T08:40:43.181558+0000","last_unstale":"2024-09-13T08:40:43.187184+0000","last_undegraded":"2024-09-13T08:40:43.187184+0000","last_fullsized":"2024-09-13T08:40:43.187184+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:08.944968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213300+0000","last_change":"2024-09-13T08:40:42.173542+0000","last_active":"2024-09-13T08:40:43.213300+0000","last_peered":"2024-09-13T08:40:43.213300+0000","last_clean":"2024-09-13T08:40:43.213300+0000","last_became_active":"2024-09-13T08:40:42.173206+0000","last_became_peered":"2024-09-13T08:40:42.173206+0000","last_unstale":"2024-09-13T08:40:43.213300+0000","last_undegraded":"2024-09-13T08:40:43.213300+0000","last_fullsized":"2024-09-13T08:40:43.213300+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:36.766590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212306+0000","last_change":"2024-09-13T08:40:43.210476+0000","last_active":"2024-09-13T08:40:43.212306+0000","last_peered":"2024-09-13T08:40:43.212306+0000","last_clean":"2024-09-13T08:40:43.212306+0000","last_became_active":"2024-09-13T08:40:43.210279+0000","last_became_peered":"2024-09-13T08:40:43.210279+0000","last_unstale":"2024-09-13T08:40:43.212306+0000","last_undegraded":"2024-09-13T08:40:43.212306+0000","last_fullsized":"2024-09-13T08:40:43.212306+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:03.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.209719+0000","last_change":"2024-09-13T08:40:42.176695+0000","last_active":"2024-09-13T08:40:43.209719+0000","last_peered":"2024-09-13T08:40:43.209719+0000","last_clean":"2024-09-13T08:40:43.209719+0000","last_became_active":"2024-09-13T08:40:42.176387+0000","last_became_peered":"2024-09-13T08:40:42.176387+0000","last_unstale":"2024-09-13T08:40:43.209719+0000","last_undegraded":"2024-09-13T08:40:43.209719+0000","last_fullsized":"2024-09-13T08:40:43.209719+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:23.536012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197389+0000","last_change":"2024-09-13T08:40:43.193233+0000","last_active":"2024-09-13T08:40:43.197389+0000","last_peered":"2024-09-13T08:40:43.197389+0000","last_clean":"2024-09-13T08:40:43.197389+0000","last_became_active":"2024-09-13T08:40:43.192944+0000","last_became_peered":"2024-09-13T08:40:43.192944+0000","last_unstale":"2024-09-13T08:40:43.197389+0000","last_undegraded":"2024-09-13T08:40:43.197389+0000","last_fullsized":"2024-09-13T08:40:43.197389+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.417971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213559+0000","last_change":"2024-09-13T08:40:42.180372+0000","last_active":"2024-09-13T08:40:43.213559+0000","last_peered":"2024-09-13T08:40:43.213559+0000","last_clean":"2024-09-13T08:40:43.213559+0000","last_became_active":"2024-09-13T08:40:42.180079+0000","last_became_peered":"2024-09-13T08:40:42.180079+0000","last_unstale":"2024-09-13T08:40:43.213559+0000","last_undegraded":"2024-09-13T08:40:43.213559+0000","last_fullsized":"2024-09-13T08:40:43.213559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:21.515280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.188208+0000","last_change":"2024-09-13T08:40:43.177434+0000","last_active":"2024-09-13T08:40:43.188208+0000","last_peered":"2024-09-13T08:40:43.188208+0000","last_clean":"2024-09-13T08:40:43.188208+0000","last_became_active":"2024-09-13T08:40:43.177163+0000","last_became_peered":"2024-09-13T08:40:43.177163+0000","last_unstale":"2024-09-13T08:40:43.188208+0000","last_undegraded":"2024-09-13T08:40:43.188208+0000","last_fullsized":"2024-09-13T08:40:43.188208+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:14.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213015+0000","last_change":"2024-09-13T08:40:42.185496+0000","last_active":"2024-09-13T08:40:43.213015+0000","last_peered":"2024-09-13T08:40:43.213015+0000","last_clean":"2024-09-13T08:40:43.213015+0000","last_became_active":"2024-09-13T08:40:42.184959+0000","last_became_peered":"2024-09-13T08:40:42.184959+0000","last_unstale":"2024-09-13T08:40:43.213015+0000","last_undegraded":"2024-09-13T08:40:43.213015+0000","last_fullsized":"2024-09-13T08:40:43.213015+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:18:11.442697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207041+0000","last_change":"2024-09-13T08:40:43.193014+0000","last_active":"2024-09-13T08:40:43.207041+0000","last_peered":"2024-09-13T08:40:43.207041+0000","last_clean":"2024-09-13T08:40:43.207041+0000","last_became_active":"2024-09-13T08:40:43.192539+0000","last_became_peered":"2024-09-13T08:40:43.192539+0000","last_unstale":"2024-09-13T08:40:43.207041+0000","last_undegraded":"2024-09-13T08:40:43.207041+0000","last_fullsized":"2024-09-13T08:40:43.207041+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:58.112836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.188854+0000","last_change":"2024-09-13T08:40:42.188197+0000","last_active":"2024-09-13T08:40:42.188854+0000","last_peered":"2024-09-13T08:40:42.188854+0000","last_clean":"2024-09-13T08:40:42.188854+0000","last_became_active":"2024-09-13T08:40:42.187908+0000","last_became_peered":"2024-09-13T08:40:42.187908+0000","last_unstale":"2024-09-13T08:40:42.188854+0000","last_undegraded":"2024-09-13T08:40:42.188854+0000","last_fullsized":"2024-09-13T08:40:42.188854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:30:57.769839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205398+0000","last_change":"2024-09-13T08:40:43.194030+0000","last_active":"2024-09-13T08:40:43.205398+0000","last_peered":"2024-09-13T08:40:43.205398+0000","last_clean":"2024-09-13T08:40:43.205398+0000","last_became_active":"2024-09-13T08:40:43.193719+0000","last_became_peered":"2024-09-13T08:40:43.193719+0000","last_unstale":"2024-09-13T08:40:43.205398+0000","last_undegraded":"2024-09-13T08:40:43.205398+0000","last_fullsized":"2024-09-13T08:40:43.205398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:54:40.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.210039+0000","last_change":"2024-09-13T08:40:42.170852+0000","last_active":"2024-09-13T08:40:43.210039+0000","last_peered":"2024-09-13T08:40:43.210039+0000","last_clean":"2024-09-13T08:40:43.210039+0000","last_became_active":"2024-09-13T08:40:42.170588+0000","last_became_peered":"2024-09-13T08:40:42.170588+0000","last_unstale":"2024-09-13T08:40:43.210039+0000","last_undegraded":"2024-09-13T08:40:43.210039+0000","last_fullsized":"2024-09-13T08:40:43.210039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:32.502289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-13T08:40:42.157478+0000","last_change":"2024-09-13T08:40:42.157478+0000","last_active":"2024-09-13T08:40:42.149568+0000","last_peered":"2024-09-13T08:40:42.149568+0000","last_clean":"2024-09-13T08:40:42.149568+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:40:42.157478+0000","last_undegraded":"2024-09-13T08:40:42.157478+0000","last_fullsized":"2024-09-13T08:40:42.157478+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.175195+0000","last_change":"2024-09-13T08:40:42.175444+0000","last_active":"2024-09-13T08:40:43.175195+0000","last_peered":"2024-09-13T08:40:43.175195+0000","last_clean":"2024-09-13T08:40:43.175195+0000","last_became_active":"2024-09-13T08:40:42.174590+0000","last_became_peered":"2024-09-13T08:40:42.174590+0000","last_unstale":"2024-09-13T08:40:43.175195+0000","last_undegraded":"2024-09-13T08:40:43.175195+0000","last_fullsized":"2024-09-13T08:40:43.175195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:32.473636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190104+0000","last_change":"2024-09-13T08:40:43.177933+0000","last_active":"2024-09-13T08:40:43.190104+0000","last_peered":"2024-09-13T08:40:43.190104+0000","last_clean":"2024-09-13T08:40:43.190104+0000","last_became_active":"2024-09-13T08:40:43.177479+0000","last_became_peered":"2024-09-13T08:40:43.177479+0000","last_unstale":"2024-09-13T08:40:43.190104+0000","last_undegraded":"2024-09-13T08:40:43.190104+0000","last_fullsized":"2024-09-13T08:40:43.190104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:54.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.178449+0000","last_change":"2024-09-13T08:40:42.183091+0000","last_active":"2024-09-13T08:40:43.178449+0000","last_peered":"2024-09-13T08:40:43.178449+0000","last_clean":"2024-09-13T08:40:43.178449+0000","last_became_active":"2024-09-13T08:40:42.182685+0000","last_became_peered":"2024-09-13T08:40:42.182685+0000","last_unstale":"2024-09-13T08:40:43.178449+0000","last_undegraded":"2024-09-13T08:40:43.178449+0000","last_fullsized":"2024-09-13T08:40:43.178449+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:03.155901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.192568+0000","last_change":"2024-09-13T08:40:43.181826+0000","last_active":"2024-09-13T08:40:43.192568+0000","last_peered":"2024-09-13T08:40:43.192568+0000","last_clean":"2024-09-13T08:40:43.192568+0000","last_became_active":"2024-09-13T08:40:43.181516+0000","last_became_peered":"2024-09-13T08:40:43.181516+0000","last_unstale":"2024-09-13T08:40:43.192568+0000","last_undegraded":"2024-09-13T08:40:43.192568+0000","last_fullsized":"2024-09-13T08:40:43.192568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.173833+0000","last_change":"2024-09-13T08:40:42.177701+0000","last_active":"2024-09-13T08:40:43.173833+0000","last_peered":"2024-09-13T08:40:43.173833+0000","last_clean":"2024-09-13T08:40:43.173833+0000","last_became_active":"2024-09-13T08:40:42.177418+0000","last_became_peered":"2024-09-13T08:40:42.177418+0000","last_unstale":"2024-09-13T08:40:43.173833+0000","last_undegraded":"2024-09-13T08:40:43.173833+0000","last_fullsized":"2024-09-13T08:40:43.173833+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:23.873609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206487+0000","last_change":"2024-09-13T08:40:43.198409+0000","last_active":"2024-09-13T08:40:43.206487+0000","last_peered":"2024-09-13T08:40:43.206487+0000","last_clean":"2024-09-13T08:40:43.206487+0000","last_became_active":"2024-09-13T08:40:43.198235+0000","last_became_peered":"2024-09-13T08:40:43.198235+0000","last_unstale":"2024-09-13T08:40:43.206487+0000","last_undegraded":"2024-09-13T08:40:43.206487+0000","last_fullsized":"2024-09-13T08:40:43.206487+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:00:32.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.169231+0000","last_change":"2024-09-13T08:40:42.177210+0000","last_active":"2024-09-13T08:40:43.169231+0000","last_peered":"2024-09-13T08:40:43.169231+0000","last_clean":"2024-09-13T08:40:43.169231+0000","last_became_active":"2024-09-13T08:40:42.176919+0000","last_became_peered":"2024-09-13T08:40:42.176919+0000","last_unstale":"2024-09-13T08:40:43.169231+0000","last_undegraded":"2024-09-13T08:40:43.169231+0000","last_fullsized":"2024-09-13T08:40:43.169231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:05.647017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.203491+0000","last_change":"2024-09-13T08:40:43.189382+0000","last_active":"2024-09-13T08:40:43.203491+0000","last_peered":"2024-09-13T08:40:43.203491+0000","last_clean":"2024-09-13T08:40:43.203491+0000","last_became_active":"2024-09-13T08:40:43.188958+0000","last_became_peered":"2024-09-13T08:40:43.188958+0000","last_unstale":"2024-09-13T08:40:43.203491+0000","last_undegraded":"2024-09-13T08:40:43.203491+0000","last_fullsized":"2024-09-13T08:40:43.203491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:12.306506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.173394+0000","last_change":"2024-09-13T08:40:42.186311+0000","last_active":"2024-09-13T08:40:43.173394+0000","last_peered":"2024-09-13T08:40:43.173394+0000","last_clean":"2024-09-13T08:40:43.173394+0000","last_became_active":"2024-09-13T08:40:42.185993+0000","last_became_peered":"2024-09-13T08:40:42.185993+0000","last_unstale":"2024-09-13T08:40:43.173394+0000","last_undegraded":"2024-09-13T08:40:43.173394+0000","last_fullsized":"2024-09-13T08:40:43.173394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:07.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189794+0000","last_change":"2024-09-13T08:40:43.184101+0000","last_active":"2024-09-13T08:40:43.189794+0000","last_peered":"2024-09-13T08:40:43.189794+0000","last_clean":"2024-09-13T08:40:43.189794+0000","last_became_active":"2024-09-13T08:40:43.183722+0000","last_became_peered":"2024-09-13T08:40:43.183722+0000","last_unstale":"2024-09-13T08:40:43.189794+0000","last_undegraded":"2024-09-13T08:40:43.189794+0000","last_fullsized":"2024-09-13T08:40:43.189794+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:46:44.174973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.181735+0000","last_change":"2024-09-13T08:40:42.189558+0000","last_active":"2024-09-13T08:40:43.181735+0000","last_peered":"2024-09-13T08:40:43.181735+0000","last_clean":"2024-09-13T08:40:43.181735+0000","last_became_active":"2024-09-13T08:40:42.189312+0000","last_became_peered":"2024-09-13T08:40:42.189312+0000","last_unstale":"2024-09-13T08:40:43.181735+0000","last_undegraded":"2024-09-13T08:40:43.181735+0000","last_fullsized":"2024-09-13T08:40:43.181735+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:40.270289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204125+0000","last_change":"2024-09-13T08:40:43.192030+0000","last_active":"2024-09-13T08:40:43.204125+0000","last_peered":"2024-09-13T08:40:43.204125+0000","last_clean":"2024-09-13T08:40:43.204125+0000","last_became_active":"2024-09-13T08:40:43.191644+0000","last_became_peered":"2024-09-13T08:40:43.191644+0000","last_unstale":"2024-09-13T08:40:43.204125+0000","last_undegraded":"2024-09-13T08:40:43.204125+0000","last_fullsized":"2024-09-13T08:40:43.204125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:38.459714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172812+0000","last_change":"2024-09-13T08:40:42.179810+0000","last_active":"2024-09-13T08:40:43.172812+0000","last_peered":"2024-09-13T08:40:43.172812+0000","last_clean":"2024-09-13T08:40:43.172812+0000","last_became_active":"2024-09-13T08:40:42.179512+0000","last_became_peered":"2024-09-13T08:40:42.179512+0000","last_unstale":"2024-09-13T08:40:43.172812+0000","last_undegraded":"2024-09-13T08:40:43.172812+0000","last_fullsized":"2024-09-13T08:40:43.172812+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:13.357647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.188531+0000","last_change":"2024-09-13T08:40:43.176719+0000","last_active":"2024-09-13T08:40:43.188531+0000","last_peered":"2024-09-13T08:40:43.188531+0000","last_clean":"2024-09-13T08:40:43.188531+0000","last_became_active":"2024-09-13T08:40:43.176416+0000","last_became_peered":"2024-09-13T08:40:43.176416+0000","last_unstale":"2024-09-13T08:40:43.188531+0000","last_undegraded":"2024-09-13T08:40:43.188531+0000","last_fullsized":"2024-09-13T08:40:43.188531+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:00.931558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.174570+0000","last_change":"2024-09-13T08:40:42.176816+0000","last_active":"2024-09-13T08:40:43.174570+0000","last_peered":"2024-09-13T08:40:43.174570+0000","last_clean":"2024-09-13T08:40:43.174570+0000","last_became_active":"2024-09-13T08:40:42.176236+0000","last_became_peered":"2024-09-13T08:40:42.176236+0000","last_unstale":"2024-09-13T08:40:43.174570+0000","last_undegraded":"2024-09-13T08:40:43.174570+0000","last_fullsized":"2024-09-13T08:40:43.174570+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:14.430820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204331+0000","last_change":"2024-09-13T08:40:43.176967+0000","last_active":"2024-09-13T08:40:43.204331+0000","last_peered":"2024-09-13T08:40:43.204331+0000","last_clean":"2024-09-13T08:40:43.204331+0000","last_became_active":"2024-09-13T08:40:43.176658+0000","last_became_peered":"2024-09-13T08:40:43.176658+0000","last_unstale":"2024-09-13T08:40:43.204331+0000","last_undegraded":"2024-09-13T08:40:43.204331+0000","last_fullsized":"2024-09-13T08:40:43.204331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:41.276450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T08:40:42.177057+0000","last_change":"2024-09-13T08:40:42.174657+0000","last_active":"2024-09-13T08:40:42.177057+0000","last_peered":"2024-09-13T08:40:42.177057+0000","last_clean":"2024-09-13T08:40:42.177057+0000","last_became_active":"2024-09-13T08:40:42.174374+0000","last_became_peered":"2024-09-13T08:40:42.174374+0000","last_unstale":"2024-09-13T08:40:42.177057+0000","last_undegraded":"2024-09-13T08:40:42.177057+0000","last_fullsized":"2024-09-13T08:40:42.177057+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:53:20.682799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.187158+0000","last_change":"2024-09-13T08:40:43.180762+0000","last_active":"2024-09-13T08:40:43.187158+0000","last_peered":"2024-09-13T08:40:43.187158+0000","last_clean":"2024-09-13T08:40:43.187158+0000","last_became_active":"2024-09-13T08:40:43.180576+0000","last_became_peered":"2024-09-13T08:40:43.180576+0000","last_unstale":"2024-09-13T08:40:43.187158+0000","last_undegraded":"2024-09-13T08:40:43.187158+0000","last_fullsized":"2024-09-13T08:40:43.187158+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:54.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.182098+0000","last_change":"2024-09-13T08:40:42.177987+0000","last_active":"2024-09-13T08:40:43.182098+0000","last_peered":"2024-09-13T08:40:43.182098+0000","last_clean":"2024-09-13T08:40:43.182098+0000","last_became_active":"2024-09-13T08:40:42.177653+0000","last_became_peered":"2024-09-13T08:40:42.177653+0000","last_unstale":"2024-09-13T08:40:43.182098+0000","last_undegraded":"2024-09-13T08:40:43.182098+0000","last_fullsized":"2024-09-13T08:40:43.182098+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:44.296127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190447+0000","last_change":"2024-09-13T08:40:43.180067+0000","last_active":"2024-09-13T08:40:43.190447+0000","last_peered":"2024-09-13T08:40:43.190447+0000","last_clean":"2024-09-13T08:40:43.190447+0000","last_became_active":"2024-09-13T08:40:43.179734+0000","last_became_peered":"2024-09-13T08:40:43.179734+0000","last_unstale":"2024-09-13T08:40:43.190447+0000","last_undegraded":"2024-09-13T08:40:43.190447+0000","last_fullsized":"2024-09-13T08:40:43.190447+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:33.479797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.201678+0000","last_change":"2024-09-13T08:40:42.180201+0000","last_active":"2024-09-13T08:40:43.201678+0000","last_peered":"2024-09-13T08:40:43.201678+0000","last_clean":"2024-09-13T08:40:43.201678+0000","last_became_active":"2024-09-13T08:40:42.179874+0000","last_became_peered":"2024-09-13T08:40:42.179874+0000","last_unstale":"2024-09-13T08:40:43.201678+0000","last_undegraded":"2024-09-13T08:40:43.201678+0000","last_fullsized":"2024-09-13T08:40:43.201678+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:25.549286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.171871+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:40:43.171871+0000","last_peered":"2024-09-13T08:40:43.171871+0000","last_clean":"2024-09-13T08:40:43.171871+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:40:43.171871+0000","last_undegraded":"2024-09-13T08:40:43.171871+0000","last_fullsized":"2024-09-13T08:40:43.171871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":214223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":804579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":552960,"data_stored":215893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":214861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":214279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509484,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":804627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":214861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35860,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":520192,"data_stored":207414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:40:48.149 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3a is not active+clean 2024-09-13T08:40:48.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:40:42.157508+0000', 'last_change': '2024-09-13T08:40:42.157508+0000', 'last_active': '2024-09-13T08:40:42.149568+0000', 'last_peered': '2024-09-13T08:40:42.149568+0000', 'last_clean': '2024-09-13T08:40:42.149568+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:40:42.157508+0000', 'last_undegraded': '2024-09-13T08:40:42.157508+0000', 'last_fullsized': '2024-09-13T08:40:42.157508+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'last_clean_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-13T08:40:48.149 INFO:tasks.ceph.ceph_manager.ceph:PG 7.37 is not active+clean 2024-09-13T08:40:48.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:40:42.157861+0000', 'last_change': '2024-09-13T08:40:42.157861+0000', 'last_active': '2024-09-13T08:40:42.149568+0000', 'last_peered': '2024-09-13T08:40:42.149568+0000', 'last_clean': '2024-09-13T08:40:42.149568+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:40:42.157861+0000', 'last_undegraded': '2024-09-13T08:40:42.157861+0000', 'last_fullsized': '2024-09-13T08:40:42.157861+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'last_clean_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-13T08:40:48.149 INFO:tasks.ceph.ceph_manager.ceph:PG 7.33 is not active+clean 2024-09-13T08:40:48.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:40:42.158438+0000', 'last_change': '2024-09-13T08:40:42.158438+0000', 'last_active': '2024-09-13T08:40:42.149568+0000', 'last_peered': '2024-09-13T08:40:42.149568+0000', 'last_clean': '2024-09-13T08:40:42.149568+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:40:42.158438+0000', 'last_undegraded': '2024-09-13T08:40:42.158438+0000', 'last_fullsized': '2024-09-13T08:40:42.158438+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'last_clean_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-13T08:40:48.149 INFO:tasks.ceph.ceph_manager.ceph:PG 7.21 is not active+clean 2024-09-13T08:40:48.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:40:42.157876+0000', 'last_change': '2024-09-13T08:40:42.157876+0000', 'last_active': '2024-09-13T08:40:42.149568+0000', 'last_peered': '2024-09-13T08:40:42.149568+0000', 'last_clean': '2024-09-13T08:40:42.149568+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:40:42.157876+0000', 'last_undegraded': '2024-09-13T08:40:42.157876+0000', 'last_fullsized': '2024-09-13T08:40:42.157876+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'last_clean_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-13T08:40:48.150 INFO:tasks.ceph.ceph_manager.ceph:PG 7.b is not active+clean 2024-09-13T08:40:48.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:40:42.157478+0000', 'last_change': '2024-09-13T08:40:42.157478+0000', 'last_active': '2024-09-13T08:40:42.149568+0000', 'last_peered': '2024-09-13T08:40:42.149568+0000', 'last_clean': '2024-09-13T08:40:42.149568+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:40:42.157478+0000', 'last_undegraded': '2024-09-13T08:40:42.157478+0000', 'last_fullsized': '2024-09-13T08:40:42.157478+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'last_clean_scrub_stamp': '2024-09-13T08:40:42.149568+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-13T08:40:48.150 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T08:40:48.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.332+0000 7f56be000640 1 -- 172.21.15.44:0/3143140959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b80cfe80 msgr2=0x7f56b80d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.332+0000 7f56be000640 1 --2- 172.21.15.44:0/3143140959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b80cfe80 0x7f56b80d0260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009840 tx=0x7f56a801c790 comp rx=0 tx=0).stop 2024-09-13T08:40:48.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.332+0000 7f56be000640 1 -- 172.21.15.44:0/3143140959 shutdown_connections 2024-09-13T08:40:48.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.332+0000 7f56be000640 1 --2- 172.21.15.44:0/3143140959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b80cfe80 0x7f56b80d0260 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.332+0000 7f56be000640 1 -- 172.21.15.44:0/3143140959 >> 172.21.15.44:0/3143140959 conn(0x7f56b80d1e40 msgr2=0x7f56b80d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:48.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.332+0000 7f56be000640 1 -- 172.21.15.44:0/3143140959 shutdown_connections 2024-09-13T08:40:48.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.332+0000 7f56be000640 1 -- 172.21.15.44:0/3143140959 wait complete. 2024-09-13T08:40:48.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.332+0000 7f56be000640 1 Processor -- start 2024-09-13T08:40:48.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.332+0000 7f56be000640 1 -- start start 2024-09-13T08:40:48.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56be000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b80b2760 0x7f56b81504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:48.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56be000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b80b2d00 0x7f56b8151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:48.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56be000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b80cfe80 0x7f56b80b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:48.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56be000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f56b816a2f0 con 0x7f56b80cfe80 2024-09-13T08:40:48.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56be000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56b816a170 con 0x7f56b80b2760 2024-09-13T08:40:48.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56be000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f56b816a470 con 0x7f56b80b2d00 2024-09-13T08:40:48.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b80cfe80 0x7f56b80b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:48.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b80cfe80 0x7f56b80b6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37488/0 (socket says 172.21.15.44:37488) 2024-09-13T08:40:48.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56b7e00640 1 -- 172.21.15.44:0/2162901366 learned_addr learned my addr 172.21.15.44:0/2162901366 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:48.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56b7400640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b80b2760 0x7f56b81504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:48.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56afe00640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b80b2d00 0x7f56b8151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:48.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56b7e00640 1 -- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b80b2d00 msgr2=0x7f56b8151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56b7e00640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b80b2d00 0x7f56b8151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56b7e00640 1 -- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b80b2760 msgr2=0x7f56b81504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56b7e00640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b80b2760 0x7f56b81504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56b7e00640 1 -- 172.21.15.44:0/2162901366 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b81519b0 con 0x7f56b80cfe80 2024-09-13T08:40:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56b7e00640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b80cfe80 0x7f56b80b6fb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f56b000caf0 tx=0x7f56b000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56af400640 1 -- 172.21.15.44:0/2162901366 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0016020 con 0x7f56b80cfe80 2024-09-13T08:40:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56af400640 1 -- 172.21.15.44:0/2162901366 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56b000aac0 con 0x7f56b80cfe80 2024-09-13T08:40:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56af400640 1 -- 172.21.15.44:0/2162901366 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b000ce30 con 0x7f56b80cfe80 2024-09-13T08:40:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56be000640 1 -- 172.21.15.44:0/2162901366 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b8151c90 con 0x7f56b80cfe80 2024-09-13T08:40:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56be000640 1 -- 172.21.15.44:0/2162901366 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b8156820 con 0x7f56b80cfe80 2024-09-13T08:40:48.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.336+0000 7f56be000640 1 -- 172.21.15.44:0/2162901366 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b8156a20 con 0x7f56b80cfe80 2024-09-13T08:40:48.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.340+0000 7f56af400640 1 -- 172.21.15.44:0/2162901366 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f56b0019020 con 0x7f56b80cfe80 2024-09-13T08:40:48.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.340+0000 7f56af400640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5694073220 0x7f56940756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:48.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.340+0000 7f56af400640 1 -- 172.21.15.44:0/2162901366 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f56b0094150 con 0x7f56b80cfe80 2024-09-13T08:40:48.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.340+0000 7f56b7400640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5694073220 0x7f56940756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:48.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.340+0000 7f56b7400640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5694073220 0x7f56940756e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f56a80af000 tx=0x7f56a80b2040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:48.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.344+0000 7f56af400640 1 -- 172.21.15.44:0/2162901366 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f56b8156a20 con 0x7f56b80cfe80 2024-09-13T08:40:48.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.580+0000 7f56be000640 1 -- 172.21.15.44:0/2162901366 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f56b80d0260 con 0x7f56b80cfe80 2024-09-13T08:40:48.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.580+0000 7f56af400640 1 -- 172.21.15.44:0/2162901366 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1141 (secure 0 0 0) 0x7f56b0020030 con 0x7f56b80cfe80 2024-09-13T08:40:48.585 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:{"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":219,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":8,"num_up_osds":8,"osd_up_since":1726216643,"num_in_osds":8,"osd_in_since":1726216632,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592196,"bytes_used":269537280,"bytes_avail":772824576000,"bytes_total":773094113280,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":2463,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":27,"btime":"2024-09-13T08:40:46:193076+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5322}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-13T08:39:26.537326+0000","services":{}},"progress_events":{}} 2024-09-13T08:40:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 -- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5694073220 msgr2=0x7f56940756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5694073220 0x7f56940756e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f56a80af000 tx=0x7f56a80b2040 comp rx=0 tx=0).stop 2024-09-13T08:40:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 -- 172.21.15.44:0/2162901366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b80cfe80 msgr2=0x7f56b80b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b80cfe80 0x7f56b80b6fb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f56b000caf0 tx=0x7f56b000cbe0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 -- 172.21.15.44:0/2162901366 shutdown_connections 2024-09-13T08:40:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5694073220 0x7f56940756e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b80cfe80 0x7f56b80b6fb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b80b2d00 0x7f56b8151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 --2- 172.21.15.44:0/2162901366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b80b2760 0x7f56b81504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 -- 172.21.15.44:0/2162901366 >> 172.21.15.44:0/2162901366 conn(0x7f56b80d1e40 msgr2=0x7f56b8158f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 -- 172.21.15.44:0/2162901366 shutdown_connections 2024-09-13T08:40:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.584+0000 7f56be000640 1 -- 172.21.15.44:0/2162901366 wait complete. 2024-09-13T08:40:48.606 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-13T08:40:48.606 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:40:48.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 -- 172.21.15.44:0/185809051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25cc0cdb30 msgr2=0x7f25cc0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 --2- 172.21.15.44:0/185809051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25cc0cdb30 0x7f25cc0cebb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0745d0 tx=0x7f25b401c770 comp rx=0 tx=0).stop 2024-09-13T08:40:48.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 -- 172.21.15.44:0/185809051 shutdown_connections 2024-09-13T08:40:48.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 --2- 172.21.15.44:0/185809051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25cc0cdb30 0x7f25cc0cebb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 --2- 172.21.15.44:0/185809051 >> v2:172.21.15.129:3300/0 conn(0x7f25cc15bea0 0x7f25cc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 -- 172.21.15.44:0/185809051 >> 172.21.15.44:0/185809051 conn(0x7f25cc0d1dc0 msgr2=0x7f25cc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:48.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 -- 172.21.15.44:0/185809051 shutdown_connections 2024-09-13T08:40:48.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 -- 172.21.15.44:0/185809051 wait complete. 2024-09-13T08:40:48.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 Processor -- start 2024-09-13T08:40:48.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 -- start start 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25cc0cdb30 0x7f25cc1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25cc0d28e0 0x7f25cc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25cc15bea0 0x7f25cc0b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc168fc0 con 0x7f25cc0d28e0 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc168e40 con 0x7f25cc0cdb30 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25d2200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f25cc169140 con 0x7f25cc15bea0 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25c2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25cc0d28e0 0x7f25cc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25c2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25cc0d28e0 0x7f25cc0b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37520/0 (socket says 172.21.15.44:37520) 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25c2a00640 1 -- 172.21.15.44:0/945823384 learned_addr learned my addr 172.21.15.44:0/945823384 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25cbe00640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25cc15bea0 0x7f25cc0b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25cb400640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25cc0cdb30 0x7f25cc1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25c2a00640 1 -- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25cc15bea0 msgr2=0x7f25cc0b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25c2a00640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25cc15bea0 0x7f25cc0b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25c2a00640 1 -- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25cc0cdb30 msgr2=0x7f25cc1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25c2a00640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25cc0cdb30 0x7f25cc1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25c2a00640 1 -- 172.21.15.44:0/945823384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25cc0ccbe0 con 0x7f25cc0d28e0 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25cbe00640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25cc15bea0 0x7f25cc0b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25cb400640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25cc0cdb30 0x7f25cc1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.752+0000 7f25c2a00640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25cc0d28e0 0x7f25cc0b12d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f25b4000c10 tx=0x7f25b40a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.756+0000 7f25c3e00640 1 -- 172.21.15.44:0/945823384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b40bc070 con 0x7f25cc0d28e0 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.756+0000 7f25c3e00640 1 -- 172.21.15.44:0/945823384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25b4004430 con 0x7f25cc0d28e0 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.756+0000 7f25c3e00640 1 -- 172.21.15.44:0/945823384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b40b2040 con 0x7f25cc0d28e0 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.756+0000 7f25d2200640 1 -- 172.21.15.44:0/945823384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25cc06dd70 con 0x7f25cc0d28e0 2024-09-13T08:40:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.756+0000 7f25d2200640 1 -- 172.21.15.44:0/945823384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25cc06e330 con 0x7f25cc0d28e0 2024-09-13T08:40:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.756+0000 7f25d2200640 1 -- 172.21.15.44:0/945823384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25cc0cc9d0 con 0x7f25cc0d28e0 2024-09-13T08:40:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.756+0000 7f25c3e00640 1 -- 172.21.15.44:0/945823384 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f25b4003860 con 0x7f25cc0d28e0 2024-09-13T08:40:48.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.756+0000 7f25c3e00640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25a8073220 0x7f25a80756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:48.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.760+0000 7f25c3e00640 1 -- 172.21.15.44:0/945823384 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f25b412dfa0 con 0x7f25cc0d28e0 2024-09-13T08:40:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.760+0000 7f25cb400640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25a8073220 0x7f25a80756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.760+0000 7f25cb400640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25a8073220 0x7f25a80756e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005fd0 tx=0x7f25bc005dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:48.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.764+0000 7f25c3e00640 1 -- 172.21.15.44:0/945823384 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f25b40163f0 con 0x7f25cc0d28e0 2024-09-13T08:40:48.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.940+0000 7f25d2200640 1 -- 172.21.15.44:0/945823384 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25cc0762f0 con 0x7f25a8073220 2024-09-13T08:40:48.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.944+0000 7f25c3e00640 1 -- 172.21.15.44:0/945823384 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251960 (secure 0 0 0) 0x7f25cc0762f0 con 0x7f25a8073220 2024-09-13T08:40:48.949 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:48.950 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:40:48.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 -- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25a8073220 msgr2=0x7f25a80756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25a8073220 0x7f25a80756e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005fd0 tx=0x7f25bc005dc0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 -- 172.21.15.44:0/945823384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25cc0d28e0 msgr2=0x7f25cc0b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25cc0d28e0 0x7f25cc0b12d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f25b4000c10 tx=0x7f25b40a5650 comp rx=0 tx=0).stop 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 -- 172.21.15.44:0/945823384 shutdown_connections 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25a8073220 0x7f25a80756e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25cc15bea0 0x7f25cc0b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25cc0d28e0 0x7f25cc0b12d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 --2- 172.21.15.44:0/945823384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25cc0cdb30 0x7f25cc1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 -- 172.21.15.44:0/945823384 >> 172.21.15.44:0/945823384 conn(0x7f25cc0d1dc0 msgr2=0x7f25cc0b7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 -- 172.21.15.44:0/945823384 shutdown_connections 2024-09-13T08:40:48.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:48.948+0000 7f25d2200640 1 -- 172.21.15.44:0/945823384 wait complete. 2024-09-13T08:40:48.971 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-13T08:40:48.558313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263248,"kb_used_data":5392,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711472,"statfs":{"total":773094113280,"available":772824547328,"internally_reserved":0,"allocated":5521408,"data_stored":2904428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.403553"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193015+0000","last_change":"2024-09-13T08:40:43.188161+0000","last_active":"2024-09-13T08:40:43.193015+0000","last_peered":"2024-09-13T08:40:43.193015+0000","last_clean":"2024-09-13T08:40:43.193015+0000","last_became_active":"2024-09-13T08:40:43.187905+0000","last_became_peered":"2024-09-13T08:40:43.187905+0000","last_unstale":"2024-09-13T08:40:43.193015+0000","last_undegraded":"2024-09-13T08:40:43.193015+0000","last_fullsized":"2024-09-13T08:40:43.193015+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199380+0000","last_change":"2024-09-13T08:40:43.193493+0000","last_active":"2024-09-13T08:40:43.199380+0000","last_peered":"2024-09-13T08:40:43.199380+0000","last_clean":"2024-09-13T08:40:43.199380+0000","last_became_active":"2024-09-13T08:40:43.193324+0000","last_became_peered":"2024-09-13T08:40:43.193324+0000","last_unstale":"2024-09-13T08:40:43.199380+0000","last_undegraded":"2024-09-13T08:40:43.199380+0000","last_fullsized":"2024-09-13T08:40:43.199380+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207011+0000","last_change":"2024-09-13T08:40:43.192784+0000","last_active":"2024-09-13T08:40:43.207011+0000","last_peered":"2024-09-13T08:40:43.207011+0000","last_clean":"2024-09-13T08:40:43.207011+0000","last_became_active":"2024-09-13T08:40:43.192234+0000","last_became_peered":"2024-09-13T08:40:43.192234+0000","last_unstale":"2024-09-13T08:40:43.207011+0000","last_undegraded":"2024-09-13T08:40:43.207011+0000","last_fullsized":"2024-09-13T08:40:43.207011+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:56:45.062628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193097+0000","last_change":"2024-09-13T08:40:43.189058+0000","last_active":"2024-09-13T08:40:43.193097+0000","last_peered":"2024-09-13T08:40:43.193097+0000","last_clean":"2024-09-13T08:40:43.193097+0000","last_became_active":"2024-09-13T08:40:43.188732+0000","last_became_peered":"2024-09-13T08:40:43.188732+0000","last_unstale":"2024-09-13T08:40:43.193097+0000","last_undegraded":"2024-09-13T08:40:43.193097+0000","last_fullsized":"2024-09-13T08:40:43.193097+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207327+0000","last_change":"2024-09-13T08:40:43.204809+0000","last_active":"2024-09-13T08:40:43.207327+0000","last_peered":"2024-09-13T08:40:43.207327+0000","last_clean":"2024-09-13T08:40:43.207327+0000","last_became_active":"2024-09-13T08:40:43.204599+0000","last_became_peered":"2024-09-13T08:40:43.204599+0000","last_unstale":"2024-09-13T08:40:43.207327+0000","last_undegraded":"2024-09-13T08:40:43.207327+0000","last_fullsized":"2024-09-13T08:40:43.207327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:20.538522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190256+0000","last_change":"2024-09-13T08:40:43.195703+0000","last_active":"2024-09-13T08:40:44.190256+0000","last_peered":"2024-09-13T08:40:44.190256+0000","last_clean":"2024-09-13T08:40:44.190256+0000","last_became_active":"2024-09-13T08:40:43.195440+0000","last_became_peered":"2024-09-13T08:40:43.195440+0000","last_unstale":"2024-09-13T08:40:44.190256+0000","last_undegraded":"2024-09-13T08:40:44.190256+0000","last_fullsized":"2024-09-13T08:40:44.190256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:01:36.322094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189864+0000","last_change":"2024-09-13T08:40:43.184765+0000","last_active":"2024-09-13T08:40:43.189864+0000","last_peered":"2024-09-13T08:40:43.189864+0000","last_clean":"2024-09-13T08:40:43.189864+0000","last_became_active":"2024-09-13T08:40:43.184173+0000","last_became_peered":"2024-09-13T08:40:43.184173+0000","last_unstale":"2024-09-13T08:40:43.189864+0000","last_undegraded":"2024-09-13T08:40:43.189864+0000","last_fullsized":"2024-09-13T08:40:43.189864+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:59.176747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190485+0000","last_change":"2024-09-13T08:40:43.180402+0000","last_active":"2024-09-13T08:40:43.190485+0000","last_peered":"2024-09-13T08:40:43.190485+0000","last_clean":"2024-09-13T08:40:43.190485+0000","last_became_active":"2024-09-13T08:40:43.179682+0000","last_became_peered":"2024-09-13T08:40:43.179682+0000","last_unstale":"2024-09-13T08:40:43.190485+0000","last_undegraded":"2024-09-13T08:40:43.190485+0000","last_fullsized":"2024-09-13T08:40:43.190485+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190878+0000","last_change":"2024-09-13T08:40:43.191570+0000","last_active":"2024-09-13T08:40:44.190878+0000","last_peered":"2024-09-13T08:40:44.190878+0000","last_clean":"2024-09-13T08:40:44.190878+0000","last_became_active":"2024-09-13T08:40:43.191245+0000","last_became_peered":"2024-09-13T08:40:43.191245+0000","last_unstale":"2024-09-13T08:40:44.190878+0000","last_undegraded":"2024-09-13T08:40:44.190878+0000","last_fullsized":"2024-09-13T08:40:44.190878+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:37.131273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189100+0000","last_change":"2024-09-13T08:40:43.183571+0000","last_active":"2024-09-13T08:40:43.189100+0000","last_peered":"2024-09-13T08:40:43.189100+0000","last_clean":"2024-09-13T08:40:43.189100+0000","last_became_active":"2024-09-13T08:40:43.183297+0000","last_became_peered":"2024-09-13T08:40:43.183297+0000","last_unstale":"2024-09-13T08:40:43.189100+0000","last_undegraded":"2024-09-13T08:40:43.189100+0000","last_fullsized":"2024-09-13T08:40:43.189100+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:14.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206352+0000","last_change":"2024-09-13T08:40:43.200664+0000","last_active":"2024-09-13T08:40:43.206352+0000","last_peered":"2024-09-13T08:40:43.206352+0000","last_clean":"2024-09-13T08:40:43.206352+0000","last_became_active":"2024-09-13T08:40:43.200401+0000","last_became_peered":"2024-09-13T08:40:43.200401+0000","last_unstale":"2024-09-13T08:40:43.206352+0000","last_undegraded":"2024-09-13T08:40:43.206352+0000","last_fullsized":"2024-09-13T08:40:43.206352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:14.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199560+0000","last_change":"2024-09-13T08:40:43.188349+0000","last_active":"2024-09-13T08:40:43.199560+0000","last_peered":"2024-09-13T08:40:43.199560+0000","last_clean":"2024-09-13T08:40:43.199560+0000","last_became_active":"2024-09-13T08:40:43.188067+0000","last_became_peered":"2024-09-13T08:40:43.188067+0000","last_unstale":"2024-09-13T08:40:43.199560+0000","last_undegraded":"2024-09-13T08:40:43.199560+0000","last_fullsized":"2024-09-13T08:40:43.199560+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190345+0000","last_change":"2024-09-13T08:40:43.197739+0000","last_active":"2024-09-13T08:40:44.190345+0000","last_peered":"2024-09-13T08:40:44.190345+0000","last_clean":"2024-09-13T08:40:44.190345+0000","last_became_active":"2024-09-13T08:40:43.195345+0000","last_became_peered":"2024-09-13T08:40:43.195345+0000","last_unstale":"2024-09-13T08:40:44.190345+0000","last_undegraded":"2024-09-13T08:40:44.190345+0000","last_fullsized":"2024-09-13T08:40:44.190345+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:44:01.824760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193350+0000","last_change":"2024-09-13T08:40:43.187316+0000","last_active":"2024-09-13T08:40:43.193350+0000","last_peered":"2024-09-13T08:40:43.193350+0000","last_clean":"2024-09-13T08:40:43.193350+0000","last_became_active":"2024-09-13T08:40:43.187050+0000","last_became_peered":"2024-09-13T08:40:43.187050+0000","last_unstale":"2024-09-13T08:40:43.193350+0000","last_undegraded":"2024-09-13T08:40:43.193350+0000","last_fullsized":"2024-09-13T08:40:43.193350+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189919+0000","last_change":"2024-09-13T08:40:43.178849+0000","last_active":"2024-09-13T08:40:43.189919+0000","last_peered":"2024-09-13T08:40:43.189919+0000","last_clean":"2024-09-13T08:40:43.189919+0000","last_became_active":"2024-09-13T08:40:43.178581+0000","last_became_peered":"2024-09-13T08:40:43.178581+0000","last_unstale":"2024-09-13T08:40:43.189919+0000","last_undegraded":"2024-09-13T08:40:43.189919+0000","last_fullsized":"2024-09-13T08:40:43.189919+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:08.944968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.182653+0000","last_change":"2024-09-13T08:40:43.178443+0000","last_active":"2024-09-13T08:40:43.182653+0000","last_peered":"2024-09-13T08:40:43.182653+0000","last_clean":"2024-09-13T08:40:43.182653+0000","last_became_active":"2024-09-13T08:40:43.178156+0000","last_became_peered":"2024-09-13T08:40:43.178156+0000","last_unstale":"2024-09-13T08:40:43.182653+0000","last_undegraded":"2024-09-13T08:40:43.182653+0000","last_fullsized":"2024-09-13T08:40:43.182653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.186648+0000","last_change":"2024-09-13T08:40:43.181966+0000","last_active":"2024-09-13T08:40:43.186648+0000","last_peered":"2024-09-13T08:40:43.186648+0000","last_clean":"2024-09-13T08:40:43.186648+0000","last_became_active":"2024-09-13T08:40:43.181651+0000","last_became_peered":"2024-09-13T08:40:43.181651+0000","last_unstale":"2024-09-13T08:40:43.186648+0000","last_undegraded":"2024-09-13T08:40:43.186648+0000","last_fullsized":"2024-09-13T08:40:43.186648+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:33.479797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207483+0000","last_change":"2024-09-13T08:40:43.203044+0000","last_active":"2024-09-13T08:40:43.207483+0000","last_peered":"2024-09-13T08:40:43.207483+0000","last_clean":"2024-09-13T08:40:43.207483+0000","last_became_active":"2024-09-13T08:40:43.196038+0000","last_became_peered":"2024-09-13T08:40:43.196038+0000","last_unstale":"2024-09-13T08:40:43.207483+0000","last_undegraded":"2024-09-13T08:40:43.207483+0000","last_fullsized":"2024-09-13T08:40:43.207483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:49:13.250832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207072+0000","last_change":"2024-09-13T08:40:43.200673+0000","last_active":"2024-09-13T08:40:43.207072+0000","last_peered":"2024-09-13T08:40:43.207072+0000","last_clean":"2024-09-13T08:40:43.207072+0000","last_became_active":"2024-09-13T08:40:43.200103+0000","last_became_peered":"2024-09-13T08:40:43.200103+0000","last_unstale":"2024-09-13T08:40:43.207072+0000","last_undegraded":"2024-09-13T08:40:43.207072+0000","last_fullsized":"2024-09-13T08:40:43.207072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:21.298332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.203522+0000","last_change":"2024-09-13T08:40:43.191159+0000","last_active":"2024-09-13T08:40:43.203522+0000","last_peered":"2024-09-13T08:40:43.203522+0000","last_clean":"2024-09-13T08:40:43.203522+0000","last_became_active":"2024-09-13T08:40:43.190876+0000","last_became_peered":"2024-09-13T08:40:43.190876+0000","last_unstale":"2024-09-13T08:40:43.203522+0000","last_undegraded":"2024-09-13T08:40:43.203522+0000","last_fullsized":"2024-09-13T08:40:43.203522+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:49.726267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190142+0000","last_change":"2024-09-13T08:40:43.176763+0000","last_active":"2024-09-13T08:40:43.190142+0000","last_peered":"2024-09-13T08:40:43.190142+0000","last_clean":"2024-09-13T08:40:43.190142+0000","last_became_active":"2024-09-13T08:40:43.176429+0000","last_became_peered":"2024-09-13T08:40:43.176429+0000","last_unstale":"2024-09-13T08:40:43.190142+0000","last_undegraded":"2024-09-13T08:40:43.190142+0000","last_fullsized":"2024-09-13T08:40:43.190142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:14.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.211767+0000","last_change":"2024-09-13T08:40:43.207935+0000","last_active":"2024-09-13T08:40:43.211767+0000","last_peered":"2024-09-13T08:40:43.211767+0000","last_clean":"2024-09-13T08:40:43.211767+0000","last_became_active":"2024-09-13T08:40:43.207661+0000","last_became_peered":"2024-09-13T08:40:43.207661+0000","last_unstale":"2024-09-13T08:40:43.211767+0000","last_undegraded":"2024-09-13T08:40:43.211767+0000","last_fullsized":"2024-09-13T08:40:43.211767+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:43.583764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189838+0000","last_change":"2024-09-13T08:40:43.184477+0000","last_active":"2024-09-13T08:40:43.189838+0000","last_peered":"2024-09-13T08:40:43.189838+0000","last_clean":"2024-09-13T08:40:43.189838+0000","last_became_active":"2024-09-13T08:40:43.183611+0000","last_became_peered":"2024-09-13T08:40:43.183611+0000","last_unstale":"2024-09-13T08:40:43.189838+0000","last_undegraded":"2024-09-13T08:40:43.189838+0000","last_fullsized":"2024-09-13T08:40:43.189838+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:33:32.215525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.175547+0000","last_change":"2024-09-13T08:40:43.173503+0000","last_active":"2024-09-13T08:40:43.175547+0000","last_peered":"2024-09-13T08:40:43.175547+0000","last_clean":"2024-09-13T08:40:43.175547+0000","last_became_active":"2024-09-13T08:40:43.173209+0000","last_became_peered":"2024-09-13T08:40:43.173209+0000","last_unstale":"2024-09-13T08:40:43.175547+0000","last_undegraded":"2024-09-13T08:40:43.175547+0000","last_fullsized":"2024-09-13T08:40:43.175547+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:00.931558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193040+0000","last_change":"2024-09-13T08:40:43.188762+0000","last_active":"2024-09-13T08:40:43.193040+0000","last_peered":"2024-09-13T08:40:43.193040+0000","last_clean":"2024-09-13T08:40:43.193040+0000","last_became_active":"2024-09-13T08:40:43.188481+0000","last_became_peered":"2024-09-13T08:40:43.188481+0000","last_unstale":"2024-09-13T08:40:43.193040+0000","last_undegraded":"2024-09-13T08:40:43.193040+0000","last_fullsized":"2024-09-13T08:40:43.193040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:32.898026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206308+0000","last_change":"2024-09-13T08:40:43.201185+0000","last_active":"2024-09-13T08:40:43.206308+0000","last_peered":"2024-09-13T08:40:43.206308+0000","last_clean":"2024-09-13T08:40:43.206308+0000","last_became_active":"2024-09-13T08:40:43.200898+0000","last_became_peered":"2024-09-13T08:40:43.200898+0000","last_unstale":"2024-09-13T08:40:43.206308+0000","last_undegraded":"2024-09-13T08:40:43.206308+0000","last_fullsized":"2024-09-13T08:40:43.206308+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:32.631857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212364+0000","last_change":"2024-09-13T08:40:43.208083+0000","last_active":"2024-09-13T08:40:43.212364+0000","last_peered":"2024-09-13T08:40:43.212364+0000","last_clean":"2024-09-13T08:40:43.212364+0000","last_became_active":"2024-09-13T08:40:43.207841+0000","last_became_peered":"2024-09-13T08:40:43.207841+0000","last_unstale":"2024-09-13T08:40:43.212364+0000","last_undegraded":"2024-09-13T08:40:43.212364+0000","last_fullsized":"2024-09-13T08:40:43.212364+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:29:52.299097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207593+0000","last_change":"2024-09-13T08:40:43.198438+0000","last_active":"2024-09-13T08:40:43.207593+0000","last_peered":"2024-09-13T08:40:43.207593+0000","last_clean":"2024-09-13T08:40:43.207593+0000","last_became_active":"2024-09-13T08:40:43.197966+0000","last_became_peered":"2024-09-13T08:40:43.197966+0000","last_unstale":"2024-09-13T08:40:43.207593+0000","last_undegraded":"2024-09-13T08:40:43.207593+0000","last_fullsized":"2024-09-13T08:40:43.207593+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:54:02.077806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197404+0000","last_change":"2024-09-13T08:40:43.191493+0000","last_active":"2024-09-13T08:40:43.197404+0000","last_peered":"2024-09-13T08:40:43.197404+0000","last_clean":"2024-09-13T08:40:43.197404+0000","last_became_active":"2024-09-13T08:40:43.191059+0000","last_became_peered":"2024-09-13T08:40:43.191059+0000","last_unstale":"2024-09-13T08:40:43.197404+0000","last_undegraded":"2024-09-13T08:40:43.197404+0000","last_fullsized":"2024-09-13T08:40:43.197404+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:04.162062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.186580+0000","last_change":"2024-09-13T08:40:43.182477+0000","last_active":"2024-09-13T08:40:43.186580+0000","last_peered":"2024-09-13T08:40:43.186580+0000","last_clean":"2024-09-13T08:40:43.186580+0000","last_became_active":"2024-09-13T08:40:43.182226+0000","last_became_peered":"2024-09-13T08:40:43.182226+0000","last_unstale":"2024-09-13T08:40:43.186580+0000","last_undegraded":"2024-09-13T08:40:43.186580+0000","last_fullsized":"2024-09-13T08:40:43.186580+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:35.436344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189122+0000","last_change":"2024-09-13T08:40:43.197098+0000","last_active":"2024-09-13T08:40:44.189122+0000","last_peered":"2024-09-13T08:40:44.189122+0000","last_clean":"2024-09-13T08:40:44.189122+0000","last_became_active":"2024-09-13T08:40:43.196818+0000","last_became_peered":"2024-09-13T08:40:43.196818+0000","last_unstale":"2024-09-13T08:40:44.189122+0000","last_undegraded":"2024-09-13T08:40:44.189122+0000","last_fullsized":"2024-09-13T08:40:44.189122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:41:32.293180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.193060+0000","last_change":"2024-09-13T08:40:43.189479+0000","last_active":"2024-09-13T08:40:43.193060+0000","last_peered":"2024-09-13T08:40:43.193060+0000","last_clean":"2024-09-13T08:40:43.193060+0000","last_became_active":"2024-09-13T08:40:43.189207+0000","last_became_peered":"2024-09-13T08:40:43.189207+0000","last_unstale":"2024-09-13T08:40:43.193060+0000","last_undegraded":"2024-09-13T08:40:43.193060+0000","last_fullsized":"2024-09-13T08:40:43.193060+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:37.772751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197183+0000","last_change":"2024-09-13T08:40:43.193080+0000","last_active":"2024-09-13T08:40:43.197183+0000","last_peered":"2024-09-13T08:40:43.197183+0000","last_clean":"2024-09-13T08:40:43.197183+0000","last_became_active":"2024-09-13T08:40:43.192785+0000","last_became_peered":"2024-09-13T08:40:43.192785+0000","last_unstale":"2024-09-13T08:40:43.197183+0000","last_undegraded":"2024-09-13T08:40:43.197183+0000","last_fullsized":"2024-09-13T08:40:43.197183+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:58.112836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190840+0000","last_change":"2024-09-13T08:40:42.175868+0000","last_active":"2024-09-13T08:40:44.190840+0000","last_peered":"2024-09-13T08:40:44.190840+0000","last_clean":"2024-09-13T08:40:44.190840+0000","last_became_active":"2024-09-13T08:40:42.175587+0000","last_became_peered":"2024-09-13T08:40:42.175587+0000","last_unstale":"2024-09-13T08:40:44.190840+0000","last_undegraded":"2024-09-13T08:40:44.190840+0000","last_fullsized":"2024-09-13T08:40:44.190840+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:28.527981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198234+0000","last_change":"2024-09-13T08:40:43.194685+0000","last_active":"2024-09-13T08:40:43.198234+0000","last_peered":"2024-09-13T08:40:43.198234+0000","last_clean":"2024-09-13T08:40:43.198234+0000","last_became_active":"2024-09-13T08:40:43.194145+0000","last_became_peered":"2024-09-13T08:40:43.194145+0000","last_unstale":"2024-09-13T08:40:43.198234+0000","last_undegraded":"2024-09-13T08:40:43.198234+0000","last_fullsized":"2024-09-13T08:40:43.198234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:37.772751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206000+0000","last_change":"2024-09-13T08:40:42.182278+0000","last_active":"2024-09-13T08:40:43.206000+0000","last_peered":"2024-09-13T08:40:43.206000+0000","last_clean":"2024-09-13T08:40:43.206000+0000","last_became_active":"2024-09-13T08:40:42.181992+0000","last_became_peered":"2024-09-13T08:40:42.181992+0000","last_unstale":"2024-09-13T08:40:43.206000+0000","last_undegraded":"2024-09-13T08:40:43.206000+0000","last_fullsized":"2024-09-13T08:40:43.206000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:51.100086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.199534+0000","last_change":"2024-09-13T08:40:43.187867+0000","last_active":"2024-09-13T08:40:43.199534+0000","last_peered":"2024-09-13T08:40:43.199534+0000","last_clean":"2024-09-13T08:40:43.199534+0000","last_became_active":"2024-09-13T08:40:43.187621+0000","last_became_peered":"2024-09-13T08:40:43.187621+0000","last_unstale":"2024-09-13T08:40:43.199534+0000","last_undegraded":"2024-09-13T08:40:43.199534+0000","last_fullsized":"2024-09-13T08:40:43.199534+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190724+0000","last_change":"2024-09-13T08:40:42.179208+0000","last_active":"2024-09-13T08:40:44.190724+0000","last_peered":"2024-09-13T08:40:44.190724+0000","last_clean":"2024-09-13T08:40:44.190724+0000","last_became_active":"2024-09-13T08:40:42.178617+0000","last_became_peered":"2024-09-13T08:40:42.178617+0000","last_unstale":"2024-09-13T08:40:44.190724+0000","last_undegraded":"2024-09-13T08:40:44.190724+0000","last_fullsized":"2024-09-13T08:40:44.190724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:35:45.639180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198747+0000","last_change":"2024-09-13T08:40:43.192425+0000","last_active":"2024-09-13T08:40:43.198747+0000","last_peered":"2024-09-13T08:40:43.198747+0000","last_clean":"2024-09-13T08:40:43.198747+0000","last_became_active":"2024-09-13T08:40:43.191801+0000","last_became_peered":"2024-09-13T08:40:43.191801+0000","last_unstale":"2024-09-13T08:40:43.198747+0000","last_undegraded":"2024-09-13T08:40:43.198747+0000","last_fullsized":"2024-09-13T08:40:43.198747+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:11.173292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205568+0000","last_change":"2024-09-13T08:40:42.180695+0000","last_active":"2024-09-13T08:40:43.205568+0000","last_peered":"2024-09-13T08:40:43.205568+0000","last_clean":"2024-09-13T08:40:43.205568+0000","last_became_active":"2024-09-13T08:40:42.180392+0000","last_became_peered":"2024-09-13T08:40:42.180392+0000","last_unstale":"2024-09-13T08:40:43.205568+0000","last_undegraded":"2024-09-13T08:40:43.205568+0000","last_fullsized":"2024-09-13T08:40:43.205568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:40.439749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212438+0000","last_change":"2024-09-13T08:40:43.209100+0000","last_active":"2024-09-13T08:40:43.212438+0000","last_peered":"2024-09-13T08:40:43.212438+0000","last_clean":"2024-09-13T08:40:43.212438+0000","last_became_active":"2024-09-13T08:40:43.208834+0000","last_became_peered":"2024-09-13T08:40:43.208834+0000","last_unstale":"2024-09-13T08:40:43.212438+0000","last_undegraded":"2024-09-13T08:40:43.212438+0000","last_fullsized":"2024-09-13T08:40:43.212438+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:29:51.212164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190745+0000","last_change":"2024-09-13T08:40:42.166893+0000","last_active":"2024-09-13T08:40:44.190745+0000","last_peered":"2024-09-13T08:40:44.190745+0000","last_clean":"2024-09-13T08:40:44.190745+0000","last_became_active":"2024-09-13T08:40:42.166507+0000","last_became_peered":"2024-09-13T08:40:42.166507+0000","last_unstale":"2024-09-13T08:40:44.190745+0000","last_undegraded":"2024-09-13T08:40:44.190745+0000","last_fullsized":"2024-09-13T08:40:44.190745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:20.818876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205301+0000","last_change":"2024-09-13T08:40:43.193379+0000","last_active":"2024-09-13T08:40:43.205301+0000","last_peered":"2024-09-13T08:40:43.205301+0000","last_clean":"2024-09-13T08:40:43.205301+0000","last_became_active":"2024-09-13T08:40:43.193015+0000","last_became_peered":"2024-09-13T08:40:43.193015+0000","last_unstale":"2024-09-13T08:40:43.205301+0000","last_undegraded":"2024-09-13T08:40:43.205301+0000","last_fullsized":"2024-09-13T08:40:43.205301+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:19.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.201537+0000","last_change":"2024-09-13T08:40:42.184150+0000","last_active":"2024-09-13T08:40:43.201537+0000","last_peered":"2024-09-13T08:40:43.201537+0000","last_clean":"2024-09-13T08:40:43.201537+0000","last_became_active":"2024-09-13T08:40:42.183763+0000","last_became_peered":"2024-09-13T08:40:42.183763+0000","last_unstale":"2024-09-13T08:40:43.201537+0000","last_undegraded":"2024-09-13T08:40:43.201537+0000","last_fullsized":"2024-09-13T08:40:43.201537+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:59.988155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190512+0000","last_change":"2024-09-13T08:40:43.187575+0000","last_active":"2024-09-13T08:40:43.190512+0000","last_peered":"2024-09-13T08:40:43.190512+0000","last_clean":"2024-09-13T08:40:43.190512+0000","last_became_active":"2024-09-13T08:40:43.186758+0000","last_became_peered":"2024-09-13T08:40:43.186758+0000","last_unstale":"2024-09-13T08:40:43.190512+0000","last_undegraded":"2024-09-13T08:40:43.190512+0000","last_fullsized":"2024-09-13T08:40:43.190512+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:55.795423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172776+0000","last_change":"2024-09-13T08:40:42.175361+0000","last_active":"2024-09-13T08:40:43.172776+0000","last_peered":"2024-09-13T08:40:43.172776+0000","last_clean":"2024-09-13T08:40:43.172776+0000","last_became_active":"2024-09-13T08:40:42.174980+0000","last_became_peered":"2024-09-13T08:40:42.174980+0000","last_unstale":"2024-09-13T08:40:43.172776+0000","last_undegraded":"2024-09-13T08:40:43.172776+0000","last_fullsized":"2024-09-13T08:40:43.172776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:59.925397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206690+0000","last_change":"2024-09-13T08:40:43.200353+0000","last_active":"2024-09-13T08:40:43.206690+0000","last_peered":"2024-09-13T08:40:43.206690+0000","last_clean":"2024-09-13T08:40:43.206690+0000","last_became_active":"2024-09-13T08:40:43.198409+0000","last_became_peered":"2024-09-13T08:40:43.198409+0000","last_unstale":"2024-09-13T08:40:43.206690+0000","last_undegraded":"2024-09-13T08:40:43.206690+0000","last_fullsized":"2024-09-13T08:40:43.206690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:39.828271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.178415+0000","last_change":"2024-09-13T08:40:42.178338+0000","last_active":"2024-09-13T08:40:43.178415+0000","last_peered":"2024-09-13T08:40:43.178415+0000","last_clean":"2024-09-13T08:40:43.178415+0000","last_became_active":"2024-09-13T08:40:42.178025+0000","last_became_peered":"2024-09-13T08:40:42.178025+0000","last_unstale":"2024-09-13T08:40:43.178415+0000","last_undegraded":"2024-09-13T08:40:43.178415+0000","last_fullsized":"2024-09-13T08:40:43.178415+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:31.891865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.210052+0000","last_change":"2024-09-13T08:40:43.204789+0000","last_active":"2024-09-13T08:40:43.210052+0000","last_peered":"2024-09-13T08:40:43.210052+0000","last_clean":"2024-09-13T08:40:43.210052+0000","last_became_active":"2024-09-13T08:40:43.204114+0000","last_became_peered":"2024-09-13T08:40:43.204114+0000","last_unstale":"2024-09-13T08:40:43.210052+0000","last_undegraded":"2024-09-13T08:40:43.210052+0000","last_fullsized":"2024-09-13T08:40:43.210052+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:31.140294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.174034+0000","last_change":"2024-09-13T08:40:42.178810+0000","last_active":"2024-09-13T08:40:43.174034+0000","last_peered":"2024-09-13T08:40:43.174034+0000","last_clean":"2024-09-13T08:40:43.174034+0000","last_became_active":"2024-09-13T08:40:42.173486+0000","last_became_peered":"2024-09-13T08:40:42.173486+0000","last_unstale":"2024-09-13T08:40:43.174034+0000","last_undegraded":"2024-09-13T08:40:43.174034+0000","last_fullsized":"2024-09-13T08:40:43.174034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:12.229793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207136+0000","last_change":"2024-09-13T08:40:43.194106+0000","last_active":"2024-09-13T08:40:43.207136+0000","last_peered":"2024-09-13T08:40:43.207136+0000","last_clean":"2024-09-13T08:40:43.207136+0000","last_became_active":"2024-09-13T08:40:43.193804+0000","last_became_peered":"2024-09-13T08:40:43.193804+0000","last_unstale":"2024-09-13T08:40:43.207136+0000","last_undegraded":"2024-09-13T08:40:43.207136+0000","last_fullsized":"2024-09-13T08:40:43.207136+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.417971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.183104+0000","last_change":"2024-09-13T08:40:42.183102+0000","last_active":"2024-09-13T08:40:43.183104+0000","last_peered":"2024-09-13T08:40:43.183104+0000","last_clean":"2024-09-13T08:40:43.183104+0000","last_became_active":"2024-09-13T08:40:42.182748+0000","last_became_peered":"2024-09-13T08:40:42.182748+0000","last_unstale":"2024-09-13T08:40:43.183104+0000","last_undegraded":"2024-09-13T08:40:43.183104+0000","last_fullsized":"2024-09-13T08:40:43.183104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:10.167131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.195996+0000","last_change":"2024-09-13T08:40:43.191524+0000","last_active":"2024-09-13T08:40:43.195996+0000","last_peered":"2024-09-13T08:40:43.195996+0000","last_clean":"2024-09-13T08:40:43.195996+0000","last_became_active":"2024-09-13T08:40:43.191240+0000","last_became_peered":"2024-09-13T08:40:43.191240+0000","last_unstale":"2024-09-13T08:40:43.195996+0000","last_undegraded":"2024-09-13T08:40:43.195996+0000","last_fullsized":"2024-09-13T08:40:43.195996+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:33.508450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.209427+0000","last_change":"2024-09-13T08:40:42.177115+0000","last_active":"2024-09-13T08:40:43.209427+0000","last_peered":"2024-09-13T08:40:43.209427+0000","last_clean":"2024-09-13T08:40:43.209427+0000","last_became_active":"2024-09-13T08:40:42.176751+0000","last_became_peered":"2024-09-13T08:40:42.176751+0000","last_unstale":"2024-09-13T08:40:43.209427+0000","last_undegraded":"2024-09-13T08:40:43.209427+0000","last_fullsized":"2024-09-13T08:40:43.209427+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:23.536012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206408+0000","last_change":"2024-09-13T08:40:43.197281+0000","last_active":"2024-09-13T08:40:43.206408+0000","last_peered":"2024-09-13T08:40:43.206408+0000","last_clean":"2024-09-13T08:40:43.206408+0000","last_became_active":"2024-09-13T08:40:43.196989+0000","last_became_peered":"2024-09-13T08:40:43.196989+0000","last_unstale":"2024-09-13T08:40:43.206408+0000","last_undegraded":"2024-09-13T08:40:43.206408+0000","last_fullsized":"2024-09-13T08:40:43.206408+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:03.667062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.214312+0000","last_change":"2024-09-13T08:40:42.180947+0000","last_active":"2024-09-13T08:40:43.214312+0000","last_peered":"2024-09-13T08:40:43.214312+0000","last_clean":"2024-09-13T08:40:43.214312+0000","last_became_active":"2024-09-13T08:40:42.180647+0000","last_became_peered":"2024-09-13T08:40:42.180647+0000","last_unstale":"2024-09-13T08:40:43.214312+0000","last_undegraded":"2024-09-13T08:40:43.214312+0000","last_fullsized":"2024-09-13T08:40:43.214312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:31.625696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204310+0000","last_change":"2024-09-13T08:40:43.177272+0000","last_active":"2024-09-13T08:40:43.204310+0000","last_peered":"2024-09-13T08:40:43.204310+0000","last_clean":"2024-09-13T08:40:43.204310+0000","last_became_active":"2024-09-13T08:40:43.176928+0000","last_became_peered":"2024-09-13T08:40:43.176928+0000","last_unstale":"2024-09-13T08:40:43.204310+0000","last_undegraded":"2024-09-13T08:40:43.204310+0000","last_fullsized":"2024-09-13T08:40:43.204310+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:19.727346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204104+0000","last_change":"2024-09-13T08:40:42.176291+0000","last_active":"2024-09-13T08:40:43.204104+0000","last_peered":"2024-09-13T08:40:43.204104+0000","last_clean":"2024-09-13T08:40:43.204104+0000","last_became_active":"2024-09-13T08:40:42.175697+0000","last_became_peered":"2024-09-13T08:40:42.175697+0000","last_unstale":"2024-09-13T08:40:43.204104+0000","last_undegraded":"2024-09-13T08:40:43.204104+0000","last_fullsized":"2024-09-13T08:40:43.204104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:18.721185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198468+0000","last_change":"2024-09-13T08:40:43.194019+0000","last_active":"2024-09-13T08:40:43.198468+0000","last_peered":"2024-09-13T08:40:43.198468+0000","last_clean":"2024-09-13T08:40:43.198468+0000","last_became_active":"2024-09-13T08:40:43.192433+0000","last_became_peered":"2024-09-13T08:40:43.192433+0000","last_unstale":"2024-09-13T08:40:43.198468+0000","last_undegraded":"2024-09-13T08:40:43.198468+0000","last_fullsized":"2024-09-13T08:40:43.198468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:49:13.250832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172856+0000","last_change":"2024-09-13T08:40:42.176544+0000","last_active":"2024-09-13T08:40:43.172856+0000","last_peered":"2024-09-13T08:40:43.172856+0000","last_clean":"2024-09-13T08:40:43.172856+0000","last_became_active":"2024-09-13T08:40:42.175781+0000","last_became_peered":"2024-09-13T08:40:42.175781+0000","last_unstale":"2024-09-13T08:40:43.172856+0000","last_undegraded":"2024-09-13T08:40:43.172856+0000","last_fullsized":"2024-09-13T08:40:43.172856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:53.422970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.198512+0000","last_change":"2024-09-13T08:40:43.195108+0000","last_active":"2024-09-13T08:40:43.198512+0000","last_peered":"2024-09-13T08:40:43.198512+0000","last_clean":"2024-09-13T08:40:43.198512+0000","last_became_active":"2024-09-13T08:40:43.193623+0000","last_became_peered":"2024-09-13T08:40:43.193623+0000","last_unstale":"2024-09-13T08:40:43.198512+0000","last_undegraded":"2024-09-13T08:40:43.198512+0000","last_fullsized":"2024-09-13T08:40:43.198512+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:31.140294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189309+0000","last_change":"2024-09-13T08:40:42.184583+0000","last_active":"2024-09-13T08:40:44.189309+0000","last_peered":"2024-09-13T08:40:44.189309+0000","last_clean":"2024-09-13T08:40:44.189309+0000","last_became_active":"2024-09-13T08:40:42.184269+0000","last_became_peered":"2024-09-13T08:40:42.184269+0000","last_unstale":"2024-09-13T08:40:44.189309+0000","last_undegraded":"2024-09-13T08:40:44.189309+0000","last_fullsized":"2024-09-13T08:40:44.189309+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:37.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.187184+0000","last_change":"2024-09-13T08:40:43.181834+0000","last_active":"2024-09-13T08:40:43.187184+0000","last_peered":"2024-09-13T08:40:43.187184+0000","last_clean":"2024-09-13T08:40:43.187184+0000","last_became_active":"2024-09-13T08:40:43.181558+0000","last_became_peered":"2024-09-13T08:40:43.181558+0000","last_unstale":"2024-09-13T08:40:43.187184+0000","last_undegraded":"2024-09-13T08:40:43.187184+0000","last_fullsized":"2024-09-13T08:40:43.187184+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:08.944968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213300+0000","last_change":"2024-09-13T08:40:42.173542+0000","last_active":"2024-09-13T08:40:43.213300+0000","last_peered":"2024-09-13T08:40:43.213300+0000","last_clean":"2024-09-13T08:40:43.213300+0000","last_became_active":"2024-09-13T08:40:42.173206+0000","last_became_peered":"2024-09-13T08:40:42.173206+0000","last_unstale":"2024-09-13T08:40:43.213300+0000","last_undegraded":"2024-09-13T08:40:43.213300+0000","last_fullsized":"2024-09-13T08:40:43.213300+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:36.766590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.212306+0000","last_change":"2024-09-13T08:40:43.210476+0000","last_active":"2024-09-13T08:40:43.212306+0000","last_peered":"2024-09-13T08:40:43.212306+0000","last_clean":"2024-09-13T08:40:43.212306+0000","last_became_active":"2024-09-13T08:40:43.210279+0000","last_became_peered":"2024-09-13T08:40:43.210279+0000","last_unstale":"2024-09-13T08:40:43.212306+0000","last_undegraded":"2024-09-13T08:40:43.212306+0000","last_fullsized":"2024-09-13T08:40:43.212306+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:03.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.209719+0000","last_change":"2024-09-13T08:40:42.176695+0000","last_active":"2024-09-13T08:40:43.209719+0000","last_peered":"2024-09-13T08:40:43.209719+0000","last_clean":"2024-09-13T08:40:43.209719+0000","last_became_active":"2024-09-13T08:40:42.176387+0000","last_became_peered":"2024-09-13T08:40:42.176387+0000","last_unstale":"2024-09-13T08:40:43.209719+0000","last_undegraded":"2024-09-13T08:40:43.209719+0000","last_fullsized":"2024-09-13T08:40:43.209719+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:23.536012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.197389+0000","last_change":"2024-09-13T08:40:43.193233+0000","last_active":"2024-09-13T08:40:43.197389+0000","last_peered":"2024-09-13T08:40:43.197389+0000","last_clean":"2024-09-13T08:40:43.197389+0000","last_became_active":"2024-09-13T08:40:43.192944+0000","last_became_peered":"2024-09-13T08:40:43.192944+0000","last_unstale":"2024-09-13T08:40:43.197389+0000","last_undegraded":"2024-09-13T08:40:43.197389+0000","last_fullsized":"2024-09-13T08:40:43.197389+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.417971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213559+0000","last_change":"2024-09-13T08:40:42.180372+0000","last_active":"2024-09-13T08:40:43.213559+0000","last_peered":"2024-09-13T08:40:43.213559+0000","last_clean":"2024-09-13T08:40:43.213559+0000","last_became_active":"2024-09-13T08:40:42.180079+0000","last_became_peered":"2024-09-13T08:40:42.180079+0000","last_unstale":"2024-09-13T08:40:43.213559+0000","last_undegraded":"2024-09-13T08:40:43.213559+0000","last_fullsized":"2024-09-13T08:40:43.213559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:21.515280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.188208+0000","last_change":"2024-09-13T08:40:43.177434+0000","last_active":"2024-09-13T08:40:43.188208+0000","last_peered":"2024-09-13T08:40:43.188208+0000","last_clean":"2024-09-13T08:40:43.188208+0000","last_became_active":"2024-09-13T08:40:43.177163+0000","last_became_peered":"2024-09-13T08:40:43.177163+0000","last_unstale":"2024-09-13T08:40:43.188208+0000","last_undegraded":"2024-09-13T08:40:43.188208+0000","last_fullsized":"2024-09-13T08:40:43.188208+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:14.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.213015+0000","last_change":"2024-09-13T08:40:42.185496+0000","last_active":"2024-09-13T08:40:43.213015+0000","last_peered":"2024-09-13T08:40:43.213015+0000","last_clean":"2024-09-13T08:40:43.213015+0000","last_became_active":"2024-09-13T08:40:42.184959+0000","last_became_peered":"2024-09-13T08:40:42.184959+0000","last_unstale":"2024-09-13T08:40:43.213015+0000","last_undegraded":"2024-09-13T08:40:43.213015+0000","last_fullsized":"2024-09-13T08:40:43.213015+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:18:11.442697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.207041+0000","last_change":"2024-09-13T08:40:43.193014+0000","last_active":"2024-09-13T08:40:43.207041+0000","last_peered":"2024-09-13T08:40:43.207041+0000","last_clean":"2024-09-13T08:40:43.207041+0000","last_became_active":"2024-09-13T08:40:43.192539+0000","last_became_peered":"2024-09-13T08:40:43.192539+0000","last_unstale":"2024-09-13T08:40:43.207041+0000","last_undegraded":"2024-09-13T08:40:43.207041+0000","last_fullsized":"2024-09-13T08:40:43.207041+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:58.112836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190476+0000","last_change":"2024-09-13T08:40:42.188197+0000","last_active":"2024-09-13T08:40:44.190476+0000","last_peered":"2024-09-13T08:40:44.190476+0000","last_clean":"2024-09-13T08:40:44.190476+0000","last_became_active":"2024-09-13T08:40:42.187908+0000","last_became_peered":"2024-09-13T08:40:42.187908+0000","last_unstale":"2024-09-13T08:40:44.190476+0000","last_undegraded":"2024-09-13T08:40:44.190476+0000","last_fullsized":"2024-09-13T08:40:44.190476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:36:31.947351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.205398+0000","last_change":"2024-09-13T08:40:43.194030+0000","last_active":"2024-09-13T08:40:43.205398+0000","last_peered":"2024-09-13T08:40:43.205398+0000","last_clean":"2024-09-13T08:40:43.205398+0000","last_became_active":"2024-09-13T08:40:43.193719+0000","last_became_peered":"2024-09-13T08:40:43.193719+0000","last_unstale":"2024-09-13T08:40:43.205398+0000","last_undegraded":"2024-09-13T08:40:43.205398+0000","last_fullsized":"2024-09-13T08:40:43.205398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:54:40.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.210039+0000","last_change":"2024-09-13T08:40:42.170852+0000","last_active":"2024-09-13T08:40:43.210039+0000","last_peered":"2024-09-13T08:40:43.210039+0000","last_clean":"2024-09-13T08:40:43.210039+0000","last_became_active":"2024-09-13T08:40:42.170588+0000","last_became_peered":"2024-09-13T08:40:42.170588+0000","last_unstale":"2024-09-13T08:40:43.210039+0000","last_undegraded":"2024-09-13T08:40:43.210039+0000","last_fullsized":"2024-09-13T08:40:43.210039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:32.502289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190568+0000","last_change":"2024-09-13T08:40:43.203021+0000","last_active":"2024-09-13T08:40:44.190568+0000","last_peered":"2024-09-13T08:40:44.190568+0000","last_clean":"2024-09-13T08:40:44.190568+0000","last_became_active":"2024-09-13T08:40:43.199689+0000","last_became_peered":"2024-09-13T08:40:43.199689+0000","last_unstale":"2024-09-13T08:40:44.190568+0000","last_undegraded":"2024-09-13T08:40:44.190568+0000","last_fullsized":"2024-09-13T08:40:44.190568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:52.407146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.175195+0000","last_change":"2024-09-13T08:40:42.175444+0000","last_active":"2024-09-13T08:40:43.175195+0000","last_peered":"2024-09-13T08:40:43.175195+0000","last_clean":"2024-09-13T08:40:43.175195+0000","last_became_active":"2024-09-13T08:40:42.174590+0000","last_became_peered":"2024-09-13T08:40:42.174590+0000","last_unstale":"2024-09-13T08:40:43.175195+0000","last_undegraded":"2024-09-13T08:40:43.175195+0000","last_fullsized":"2024-09-13T08:40:43.175195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:32.473636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190104+0000","last_change":"2024-09-13T08:40:43.177933+0000","last_active":"2024-09-13T08:40:43.190104+0000","last_peered":"2024-09-13T08:40:43.190104+0000","last_clean":"2024-09-13T08:40:43.190104+0000","last_became_active":"2024-09-13T08:40:43.177479+0000","last_became_peered":"2024-09-13T08:40:43.177479+0000","last_unstale":"2024-09-13T08:40:43.190104+0000","last_undegraded":"2024-09-13T08:40:43.190104+0000","last_fullsized":"2024-09-13T08:40:43.190104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:54.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.178449+0000","last_change":"2024-09-13T08:40:42.183091+0000","last_active":"2024-09-13T08:40:43.178449+0000","last_peered":"2024-09-13T08:40:43.178449+0000","last_clean":"2024-09-13T08:40:43.178449+0000","last_became_active":"2024-09-13T08:40:42.182685+0000","last_became_peered":"2024-09-13T08:40:42.182685+0000","last_unstale":"2024-09-13T08:40:43.178449+0000","last_undegraded":"2024-09-13T08:40:43.178449+0000","last_fullsized":"2024-09-13T08:40:43.178449+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:03.155901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.192568+0000","last_change":"2024-09-13T08:40:43.181826+0000","last_active":"2024-09-13T08:40:43.192568+0000","last_peered":"2024-09-13T08:40:43.192568+0000","last_clean":"2024-09-13T08:40:43.192568+0000","last_became_active":"2024-09-13T08:40:43.181516+0000","last_became_peered":"2024-09-13T08:40:43.181516+0000","last_unstale":"2024-09-13T08:40:43.192568+0000","last_undegraded":"2024-09-13T08:40:43.192568+0000","last_fullsized":"2024-09-13T08:40:43.192568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:46.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.173833+0000","last_change":"2024-09-13T08:40:42.177701+0000","last_active":"2024-09-13T08:40:43.173833+0000","last_peered":"2024-09-13T08:40:43.173833+0000","last_clean":"2024-09-13T08:40:43.173833+0000","last_became_active":"2024-09-13T08:40:42.177418+0000","last_became_peered":"2024-09-13T08:40:42.177418+0000","last_unstale":"2024-09-13T08:40:43.173833+0000","last_undegraded":"2024-09-13T08:40:43.173833+0000","last_fullsized":"2024-09-13T08:40:43.173833+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:23.873609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.206487+0000","last_change":"2024-09-13T08:40:43.198409+0000","last_active":"2024-09-13T08:40:43.206487+0000","last_peered":"2024-09-13T08:40:43.206487+0000","last_clean":"2024-09-13T08:40:43.206487+0000","last_became_active":"2024-09-13T08:40:43.198235+0000","last_became_peered":"2024-09-13T08:40:43.198235+0000","last_unstale":"2024-09-13T08:40:43.206487+0000","last_undegraded":"2024-09-13T08:40:43.206487+0000","last_fullsized":"2024-09-13T08:40:43.206487+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:00:32.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.169231+0000","last_change":"2024-09-13T08:40:42.177210+0000","last_active":"2024-09-13T08:40:43.169231+0000","last_peered":"2024-09-13T08:40:43.169231+0000","last_clean":"2024-09-13T08:40:43.169231+0000","last_became_active":"2024-09-13T08:40:42.176919+0000","last_became_peered":"2024-09-13T08:40:42.176919+0000","last_unstale":"2024-09-13T08:40:43.169231+0000","last_undegraded":"2024-09-13T08:40:43.169231+0000","last_fullsized":"2024-09-13T08:40:43.169231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:05.647017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.203491+0000","last_change":"2024-09-13T08:40:43.189382+0000","last_active":"2024-09-13T08:40:43.203491+0000","last_peered":"2024-09-13T08:40:43.203491+0000","last_clean":"2024-09-13T08:40:43.203491+0000","last_became_active":"2024-09-13T08:40:43.188958+0000","last_became_peered":"2024-09-13T08:40:43.188958+0000","last_unstale":"2024-09-13T08:40:43.203491+0000","last_undegraded":"2024-09-13T08:40:43.203491+0000","last_fullsized":"2024-09-13T08:40:43.203491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:12.306506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.173394+0000","last_change":"2024-09-13T08:40:42.186311+0000","last_active":"2024-09-13T08:40:43.173394+0000","last_peered":"2024-09-13T08:40:43.173394+0000","last_clean":"2024-09-13T08:40:43.173394+0000","last_became_active":"2024-09-13T08:40:42.185993+0000","last_became_peered":"2024-09-13T08:40:42.185993+0000","last_unstale":"2024-09-13T08:40:43.173394+0000","last_undegraded":"2024-09-13T08:40:43.173394+0000","last_fullsized":"2024-09-13T08:40:43.173394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:07.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.189794+0000","last_change":"2024-09-13T08:40:43.184101+0000","last_active":"2024-09-13T08:40:43.189794+0000","last_peered":"2024-09-13T08:40:43.189794+0000","last_clean":"2024-09-13T08:40:43.189794+0000","last_became_active":"2024-09-13T08:40:43.183722+0000","last_became_peered":"2024-09-13T08:40:43.183722+0000","last_unstale":"2024-09-13T08:40:43.189794+0000","last_undegraded":"2024-09-13T08:40:43.189794+0000","last_fullsized":"2024-09-13T08:40:43.189794+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:46:44.174973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.181735+0000","last_change":"2024-09-13T08:40:42.189558+0000","last_active":"2024-09-13T08:40:43.181735+0000","last_peered":"2024-09-13T08:40:43.181735+0000","last_clean":"2024-09-13T08:40:43.181735+0000","last_became_active":"2024-09-13T08:40:42.189312+0000","last_became_peered":"2024-09-13T08:40:42.189312+0000","last_unstale":"2024-09-13T08:40:43.181735+0000","last_undegraded":"2024-09-13T08:40:43.181735+0000","last_fullsized":"2024-09-13T08:40:43.181735+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:40.270289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204125+0000","last_change":"2024-09-13T08:40:43.192030+0000","last_active":"2024-09-13T08:40:43.204125+0000","last_peered":"2024-09-13T08:40:43.204125+0000","last_clean":"2024-09-13T08:40:43.204125+0000","last_became_active":"2024-09-13T08:40:43.191644+0000","last_became_peered":"2024-09-13T08:40:43.191644+0000","last_unstale":"2024-09-13T08:40:43.204125+0000","last_undegraded":"2024-09-13T08:40:43.204125+0000","last_fullsized":"2024-09-13T08:40:43.204125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:38.459714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.172812+0000","last_change":"2024-09-13T08:40:42.179810+0000","last_active":"2024-09-13T08:40:43.172812+0000","last_peered":"2024-09-13T08:40:43.172812+0000","last_clean":"2024-09-13T08:40:43.172812+0000","last_became_active":"2024-09-13T08:40:42.179512+0000","last_became_peered":"2024-09-13T08:40:42.179512+0000","last_unstale":"2024-09-13T08:40:43.172812+0000","last_undegraded":"2024-09-13T08:40:43.172812+0000","last_fullsized":"2024-09-13T08:40:43.172812+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:13.357647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.188531+0000","last_change":"2024-09-13T08:40:43.176719+0000","last_active":"2024-09-13T08:40:43.188531+0000","last_peered":"2024-09-13T08:40:43.188531+0000","last_clean":"2024-09-13T08:40:43.188531+0000","last_became_active":"2024-09-13T08:40:43.176416+0000","last_became_peered":"2024-09-13T08:40:43.176416+0000","last_unstale":"2024-09-13T08:40:43.188531+0000","last_undegraded":"2024-09-13T08:40:43.188531+0000","last_fullsized":"2024-09-13T08:40:43.188531+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:00.931558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.174570+0000","last_change":"2024-09-13T08:40:42.176816+0000","last_active":"2024-09-13T08:40:43.174570+0000","last_peered":"2024-09-13T08:40:43.174570+0000","last_clean":"2024-09-13T08:40:43.174570+0000","last_became_active":"2024-09-13T08:40:42.176236+0000","last_became_peered":"2024-09-13T08:40:42.176236+0000","last_unstale":"2024-09-13T08:40:43.174570+0000","last_undegraded":"2024-09-13T08:40:43.174570+0000","last_fullsized":"2024-09-13T08:40:43.174570+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:14.430820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.204331+0000","last_change":"2024-09-13T08:40:43.176967+0000","last_active":"2024-09-13T08:40:43.204331+0000","last_peered":"2024-09-13T08:40:43.204331+0000","last_clean":"2024-09-13T08:40:43.204331+0000","last_became_active":"2024-09-13T08:40:43.176658+0000","last_became_peered":"2024-09-13T08:40:43.176658+0000","last_unstale":"2024-09-13T08:40:43.204331+0000","last_undegraded":"2024-09-13T08:40:43.204331+0000","last_fullsized":"2024-09-13T08:40:43.204331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:41.276450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189410+0000","last_change":"2024-09-13T08:40:42.174657+0000","last_active":"2024-09-13T08:40:44.189410+0000","last_peered":"2024-09-13T08:40:44.189410+0000","last_clean":"2024-09-13T08:40:44.189410+0000","last_became_active":"2024-09-13T08:40:42.174374+0000","last_became_peered":"2024-09-13T08:40:42.174374+0000","last_unstale":"2024-09-13T08:40:44.189410+0000","last_undegraded":"2024-09-13T08:40:44.189410+0000","last_fullsized":"2024-09-13T08:40:44.189410+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:30.284584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.187158+0000","last_change":"2024-09-13T08:40:43.180762+0000","last_active":"2024-09-13T08:40:43.187158+0000","last_peered":"2024-09-13T08:40:43.187158+0000","last_clean":"2024-09-13T08:40:43.187158+0000","last_became_active":"2024-09-13T08:40:43.180576+0000","last_became_peered":"2024-09-13T08:40:43.180576+0000","last_unstale":"2024-09-13T08:40:43.187158+0000","last_undegraded":"2024-09-13T08:40:43.187158+0000","last_fullsized":"2024-09-13T08:40:43.187158+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:54.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.182098+0000","last_change":"2024-09-13T08:40:42.177987+0000","last_active":"2024-09-13T08:40:43.182098+0000","last_peered":"2024-09-13T08:40:43.182098+0000","last_clean":"2024-09-13T08:40:43.182098+0000","last_became_active":"2024-09-13T08:40:42.177653+0000","last_became_peered":"2024-09-13T08:40:42.177653+0000","last_unstale":"2024-09-13T08:40:43.182098+0000","last_undegraded":"2024-09-13T08:40:43.182098+0000","last_fullsized":"2024-09-13T08:40:43.182098+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:44.296127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.190447+0000","last_change":"2024-09-13T08:40:43.180067+0000","last_active":"2024-09-13T08:40:43.190447+0000","last_peered":"2024-09-13T08:40:43.190447+0000","last_clean":"2024-09-13T08:40:43.190447+0000","last_became_active":"2024-09-13T08:40:43.179734+0000","last_became_peered":"2024-09-13T08:40:43.179734+0000","last_unstale":"2024-09-13T08:40:43.190447+0000","last_undegraded":"2024-09-13T08:40:43.190447+0000","last_fullsized":"2024-09-13T08:40:43.190447+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:33.479797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.201678+0000","last_change":"2024-09-13T08:40:42.180201+0000","last_active":"2024-09-13T08:40:43.201678+0000","last_peered":"2024-09-13T08:40:43.201678+0000","last_clean":"2024-09-13T08:40:43.201678+0000","last_became_active":"2024-09-13T08:40:42.179874+0000","last_became_peered":"2024-09-13T08:40:42.179874+0000","last_unstale":"2024-09-13T08:40:43.201678+0000","last_undegraded":"2024-09-13T08:40:43.201678+0000","last_fullsized":"2024-09-13T08:40:43.201678+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:25.549286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T08:40:43.171871+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:40:43.171871+0000","last_peered":"2024-09-13T08:40:43.171871+0000","last_clean":"2024-09-13T08:40:43.171871+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:40:43.171871+0000","last_undegraded":"2024-09-13T08:40:43.171871+0000","last_fullsized":"2024-09-13T08:40:43.171871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":214223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":804579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":552960,"data_stored":215893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":214861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":214279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509484,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":804627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":214861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35888,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335952,"statfs":{"total":96636764160,"available":96600014848,"internally_reserved":0,"allocated":552960,"data_stored":221105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:40:52.034 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:40:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.188+0000 7fb6a8400640 1 -- 172.21.15.44:0/2542881748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb688004810 msgr2=0x7fb688004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.188+0000 7fb6a8400640 1 --2- 172.21.15.44:0/2542881748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb688004810 0x7fb688004c90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb68c001830 tx=0x7fb68c01c8a0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.188+0000 7fb6a6400640 1 -- 172.21.15.44:0/2542881748 reap_dead start 2024-09-13T08:40:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.188+0000 7fb6a8400640 1 -- 172.21.15.44:0/2542881748 shutdown_connections 2024-09-13T08:40:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.188+0000 7fb6a8400640 1 -- 172.21.15.44:0/2542881748 >> 172.21.15.44:0/2542881748 conn(0x7fb6a00b18f0 msgr2=0x7fb6a00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:52.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a8400640 1 -- 172.21.15.44:0/2542881748 shutdown_connections 2024-09-13T08:40:52.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a8400640 1 -- 172.21.15.44:0/2542881748 wait complete. 2024-09-13T08:40:52.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a8400640 1 Processor -- start 2024-09-13T08:40:52.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a8400640 1 -- start start 2024-09-13T08:40:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a8400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb688004810 0x7fb6a00ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a8400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a01fbbb0 0x7fb6a01574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a8400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6a00b0040 0x7fb6a00b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a8400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a00cce00 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a8400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a00ccc80 con 0x7fb6a00b0040 2024-09-13T08:40:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a8400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a00ccf80 con 0x7fb688004810 2024-09-13T08:40:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a01fbbb0 0x7fb6a01574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:52.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a01fbbb0 0x7fb6a01574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45770/0 (socket says 172.21.15.44:45770) 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a5000640 1 -- 172.21.15.44:0/2554970285 learned_addr learned my addr 172.21.15.44:0/2554970285 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a5a00640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb688004810 0x7fb6a00ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a6400640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6a00b0040 0x7fb6a00b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a5000640 1 -- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb688004810 msgr2=0x7fb6a00ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a5000640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb688004810 0x7fb6a00ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a5000640 1 -- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6a00b0040 msgr2=0x7fb6a00b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a5000640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6a00b0040 0x7fb6a00b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a5000640 1 -- 172.21.15.44:0/2554970285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a006f5e0 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.192+0000 7fb6a6400640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6a00b0040 0x7fb6a00b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.196+0000 7fb6a5000640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a01fbbb0 0x7fb6a01574a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb68c005040 tx=0x7fb68c0a5c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.196+0000 7fb69d600640 1 -- 172.21.15.44:0/2554970285 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0068d0 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.196+0000 7fb69d600640 1 -- 172.21.15.44:0/2554970285 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb68c0038f0 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.196+0000 7fb69d600640 1 -- 172.21.15.44:0/2554970285 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c001480 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.196+0000 7fb6a8400640 1 -- 172.21.15.44:0/2554970285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a01fc010 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.196+0000 7fb6a8400640 1 -- 172.21.15.44:0/2554970285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a01f7fe0 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.196+0000 7fb6a8400640 1 -- 172.21.15.44:0/2554970285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a0069410 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.196+0000 7fb69d600640 1 -- 172.21.15.44:0/2554970285 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fb68c003d00 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.200+0000 7fb69d600640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb680073190 0x7fb680075650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:52.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.200+0000 7fb69d600640 1 -- 172.21.15.44:0/2554970285 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fb68c12e6d0 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.200+0000 7fb6a5a00640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb680073190 0x7fb680075650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:52.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.200+0000 7fb6a5a00640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb680073190 0x7fb680075650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb69400abf0 tx=0x7fb69400a2f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:52.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.204+0000 7fb69d600640 1 -- 172.21.15.44:0/2554970285 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb68c016650 con 0x7fb6a01fbbb0 2024-09-13T08:40:52.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.376+0000 7fb6a8400640 1 -- 172.21.15.44:0/2554970285 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6a00b3e10 con 0x7fb680073190 2024-09-13T08:40:52.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.384+0000 7fb69d600640 1 -- 172.21.15.44:0/2554970285 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7fb6a00b3e10 con 0x7fb680073190 2024-09-13T08:40:52.388 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:52.390 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:40:52.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 -- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb680073190 msgr2=0x7fb680075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:52.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb680073190 0x7fb680075650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb69400abf0 tx=0x7fb69400a2f0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 -- 172.21.15.44:0/2554970285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a01fbbb0 msgr2=0x7fb6a01574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:52.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a01fbbb0 0x7fb6a01574a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb68c005040 tx=0x7fb68c0a5c90 comp rx=0 tx=0).stop 2024-09-13T08:40:52.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 -- 172.21.15.44:0/2554970285 shutdown_connections 2024-09-13T08:40:52.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb680073190 0x7fb680075650 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6a00b0040 0x7fb6a00b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a01fbbb0 0x7fb6a01574a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 --2- 172.21.15.44:0/2554970285 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb688004810 0x7fb6a00ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 -- 172.21.15.44:0/2554970285 >> 172.21.15.44:0/2554970285 conn(0x7fb6a00b18f0 msgr2=0x7fb6a01597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:52.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 -- 172.21.15.44:0/2554970285 shutdown_connections 2024-09-13T08:40:52.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.388+0000 7fb6a8400640 1 -- 172.21.15.44:0/2554970285 wait complete. 2024-09-13T08:40:52.410 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-13T08:40:50.558830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263308,"kb_used_data":5480,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711412,"statfs":{"total":773094113280,"available":772824485888,"internally_reserved":0,"allocated":5611520,"data_stored":2949468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002003"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383557+0000","last_change":"2024-09-13T08:40:43.188161+0000","last_active":"2024-09-13T08:40:44.383557+0000","last_peered":"2024-09-13T08:40:44.383557+0000","last_clean":"2024-09-13T08:40:44.383557+0000","last_became_active":"2024-09-13T08:40:43.187905+0000","last_became_peered":"2024-09-13T08:40:43.187905+0000","last_unstale":"2024-09-13T08:40:44.383557+0000","last_undegraded":"2024-09-13T08:40:44.383557+0000","last_fullsized":"2024-09-13T08:40:44.383557+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:02.005810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383975+0000","last_change":"2024-09-13T08:40:43.193493+0000","last_active":"2024-09-13T08:40:44.383975+0000","last_peered":"2024-09-13T08:40:44.383975+0000","last_clean":"2024-09-13T08:40:44.383975+0000","last_became_active":"2024-09-13T08:40:43.193324+0000","last_became_peered":"2024-09-13T08:40:43.193324+0000","last_unstale":"2024-09-13T08:40:44.383975+0000","last_undegraded":"2024-09-13T08:40:44.383975+0000","last_fullsized":"2024-09-13T08:40:44.383975+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:55.795423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.188691+0000","last_change":"2024-09-13T08:40:43.192784+0000","last_active":"2024-09-13T08:40:44.188691+0000","last_peered":"2024-09-13T08:40:44.188691+0000","last_clean":"2024-09-13T08:40:44.188691+0000","last_became_active":"2024-09-13T08:40:43.192234+0000","last_became_peered":"2024-09-13T08:40:43.192234+0000","last_unstale":"2024-09-13T08:40:44.188691+0000","last_undegraded":"2024-09-13T08:40:44.188691+0000","last_fullsized":"2024-09-13T08:40:44.188691+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:03.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190079+0000","last_change":"2024-09-13T08:40:43.189058+0000","last_active":"2024-09-13T08:40:44.190079+0000","last_peered":"2024-09-13T08:40:44.190079+0000","last_clean":"2024-09-13T08:40:44.190079+0000","last_became_active":"2024-09-13T08:40:43.188732+0000","last_became_peered":"2024-09-13T08:40:43.188732+0000","last_unstale":"2024-09-13T08:40:44.190079+0000","last_undegraded":"2024-09-13T08:40:44.190079+0000","last_fullsized":"2024-09-13T08:40:44.190079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:26.555447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383214+0000","last_change":"2024-09-13T08:40:43.204809+0000","last_active":"2024-09-13T08:40:44.383214+0000","last_peered":"2024-09-13T08:40:44.383214+0000","last_clean":"2024-09-13T08:40:44.383214+0000","last_became_active":"2024-09-13T08:40:43.204599+0000","last_became_peered":"2024-09-13T08:40:43.204599+0000","last_unstale":"2024-09-13T08:40:44.383214+0000","last_undegraded":"2024-09-13T08:40:44.383214+0000","last_fullsized":"2024-09-13T08:40:44.383214+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:45:39.219148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190256+0000","last_change":"2024-09-13T08:40:43.195703+0000","last_active":"2024-09-13T08:40:44.190256+0000","last_peered":"2024-09-13T08:40:44.190256+0000","last_clean":"2024-09-13T08:40:44.190256+0000","last_became_active":"2024-09-13T08:40:43.195440+0000","last_became_peered":"2024-09-13T08:40:43.195440+0000","last_unstale":"2024-09-13T08:40:44.190256+0000","last_undegraded":"2024-09-13T08:40:44.190256+0000","last_fullsized":"2024-09-13T08:40:44.190256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:01:36.322094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192474+0000","last_change":"2024-09-13T08:40:43.184765+0000","last_active":"2024-09-13T08:40:44.192474+0000","last_peered":"2024-09-13T08:40:44.192474+0000","last_clean":"2024-09-13T08:40:44.192474+0000","last_became_active":"2024-09-13T08:40:43.184173+0000","last_became_peered":"2024-09-13T08:40:43.184173+0000","last_unstale":"2024-09-13T08:40:44.192474+0000","last_undegraded":"2024-09-13T08:40:44.192474+0000","last_fullsized":"2024-09-13T08:40:44.192474+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:55:21.007952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189260+0000","last_change":"2024-09-13T08:40:43.180402+0000","last_active":"2024-09-13T08:40:44.189260+0000","last_peered":"2024-09-13T08:40:44.189260+0000","last_clean":"2024-09-13T08:40:44.189260+0000","last_became_active":"2024-09-13T08:40:43.179682+0000","last_became_peered":"2024-09-13T08:40:43.179682+0000","last_unstale":"2024-09-13T08:40:44.189260+0000","last_undegraded":"2024-09-13T08:40:44.189260+0000","last_fullsized":"2024-09-13T08:40:44.189260+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:15.024462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190878+0000","last_change":"2024-09-13T08:40:43.191570+0000","last_active":"2024-09-13T08:40:44.190878+0000","last_peered":"2024-09-13T08:40:44.190878+0000","last_clean":"2024-09-13T08:40:44.190878+0000","last_became_active":"2024-09-13T08:40:43.191245+0000","last_became_peered":"2024-09-13T08:40:43.191245+0000","last_unstale":"2024-09-13T08:40:44.190878+0000","last_undegraded":"2024-09-13T08:40:44.190878+0000","last_fullsized":"2024-09-13T08:40:44.190878+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:37.131273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383154+0000","last_change":"2024-09-13T08:40:43.183571+0000","last_active":"2024-09-13T08:40:44.383154+0000","last_peered":"2024-09-13T08:40:44.383154+0000","last_clean":"2024-09-13T08:40:44.383154+0000","last_became_active":"2024-09-13T08:40:43.183297+0000","last_became_peered":"2024-09-13T08:40:43.183297+0000","last_unstale":"2024-09-13T08:40:44.383154+0000","last_undegraded":"2024-09-13T08:40:44.383154+0000","last_fullsized":"2024-09-13T08:40:44.383154+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:57:53.889256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189930+0000","last_change":"2024-09-13T08:40:43.200664+0000","last_active":"2024-09-13T08:40:44.189930+0000","last_peered":"2024-09-13T08:40:44.189930+0000","last_clean":"2024-09-13T08:40:44.189930+0000","last_became_active":"2024-09-13T08:40:43.200401+0000","last_became_peered":"2024-09-13T08:40:43.200401+0000","last_unstale":"2024-09-13T08:40:44.189930+0000","last_undegraded":"2024-09-13T08:40:44.189930+0000","last_fullsized":"2024-09-13T08:40:44.189930+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:15:42.918471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.384197+0000","last_change":"2024-09-13T08:40:43.188349+0000","last_active":"2024-09-13T08:40:44.384197+0000","last_peered":"2024-09-13T08:40:44.384197+0000","last_clean":"2024-09-13T08:40:44.384197+0000","last_became_active":"2024-09-13T08:40:43.188067+0000","last_became_peered":"2024-09-13T08:40:43.188067+0000","last_unstale":"2024-09-13T08:40:44.384197+0000","last_undegraded":"2024-09-13T08:40:44.384197+0000","last_fullsized":"2024-09-13T08:40:44.384197+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:38.459714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190345+0000","last_change":"2024-09-13T08:40:43.197739+0000","last_active":"2024-09-13T08:40:44.190345+0000","last_peered":"2024-09-13T08:40:44.190345+0000","last_clean":"2024-09-13T08:40:44.190345+0000","last_became_active":"2024-09-13T08:40:43.195345+0000","last_became_peered":"2024-09-13T08:40:43.195345+0000","last_unstale":"2024-09-13T08:40:44.190345+0000","last_undegraded":"2024-09-13T08:40:44.190345+0000","last_fullsized":"2024-09-13T08:40:44.190345+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:44:01.824760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190121+0000","last_change":"2024-09-13T08:40:43.187316+0000","last_active":"2024-09-13T08:40:44.190121+0000","last_peered":"2024-09-13T08:40:44.190121+0000","last_clean":"2024-09-13T08:40:44.190121+0000","last_became_active":"2024-09-13T08:40:43.187050+0000","last_became_peered":"2024-09-13T08:40:43.187050+0000","last_unstale":"2024-09-13T08:40:44.190121+0000","last_undegraded":"2024-09-13T08:40:44.190121+0000","last_fullsized":"2024-09-13T08:40:44.190121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:03.724107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189662+0000","last_change":"2024-09-13T08:40:43.178849+0000","last_active":"2024-09-13T08:40:44.189662+0000","last_peered":"2024-09-13T08:40:44.189662+0000","last_clean":"2024-09-13T08:40:44.189662+0000","last_became_active":"2024-09-13T08:40:43.178581+0000","last_became_peered":"2024-09-13T08:40:43.178581+0000","last_unstale":"2024-09-13T08:40:44.189662+0000","last_undegraded":"2024-09-13T08:40:44.189662+0000","last_fullsized":"2024-09-13T08:40:44.189662+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:54:40.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382798+0000","last_change":"2024-09-13T08:40:43.178443+0000","last_active":"2024-09-13T08:40:44.382798+0000","last_peered":"2024-09-13T08:40:44.382798+0000","last_clean":"2024-09-13T08:40:44.382798+0000","last_became_active":"2024-09-13T08:40:43.178156+0000","last_became_peered":"2024-09-13T08:40:43.178156+0000","last_unstale":"2024-09-13T08:40:44.382798+0000","last_undegraded":"2024-09-13T08:40:44.382798+0000","last_fullsized":"2024-09-13T08:40:44.382798+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:49:17.385287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.384021+0000","last_change":"2024-09-13T08:40:43.181966+0000","last_active":"2024-09-13T08:40:44.384021+0000","last_peered":"2024-09-13T08:40:44.384021+0000","last_clean":"2024-09-13T08:40:44.384021+0000","last_became_active":"2024-09-13T08:40:43.181651+0000","last_became_peered":"2024-09-13T08:40:43.181651+0000","last_unstale":"2024-09-13T08:40:44.384021+0000","last_undegraded":"2024-09-13T08:40:44.384021+0000","last_fullsized":"2024-09-13T08:40:44.384021+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:49.726267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.188796+0000","last_change":"2024-09-13T08:40:43.203044+0000","last_active":"2024-09-13T08:40:44.188796+0000","last_peered":"2024-09-13T08:40:44.188796+0000","last_clean":"2024-09-13T08:40:44.188796+0000","last_became_active":"2024-09-13T08:40:43.196038+0000","last_became_peered":"2024-09-13T08:40:43.196038+0000","last_unstale":"2024-09-13T08:40:44.188796+0000","last_undegraded":"2024-09-13T08:40:44.188796+0000","last_fullsized":"2024-09-13T08:40:44.188796+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:35.566813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383483+0000","last_change":"2024-09-13T08:40:43.200673+0000","last_active":"2024-09-13T08:40:44.383483+0000","last_peered":"2024-09-13T08:40:44.383483+0000","last_clean":"2024-09-13T08:40:44.383483+0000","last_became_active":"2024-09-13T08:40:43.200103+0000","last_became_peered":"2024-09-13T08:40:43.200103+0000","last_unstale":"2024-09-13T08:40:44.383483+0000","last_undegraded":"2024-09-13T08:40:44.383483+0000","last_fullsized":"2024-09-13T08:40:44.383483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:14.577393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383240+0000","last_change":"2024-09-13T08:40:43.191159+0000","last_active":"2024-09-13T08:40:44.383240+0000","last_peered":"2024-09-13T08:40:44.383240+0000","last_clean":"2024-09-13T08:40:44.383240+0000","last_became_active":"2024-09-13T08:40:43.190876+0000","last_became_peered":"2024-09-13T08:40:43.190876+0000","last_unstale":"2024-09-13T08:40:44.383240+0000","last_undegraded":"2024-09-13T08:40:44.383240+0000","last_fullsized":"2024-09-13T08:40:44.383240+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:17.541873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190249+0000","last_change":"2024-09-13T08:40:43.176763+0000","last_active":"2024-09-13T08:40:44.190249+0000","last_peered":"2024-09-13T08:40:44.190249+0000","last_clean":"2024-09-13T08:40:44.190249+0000","last_became_active":"2024-09-13T08:40:43.176429+0000","last_became_peered":"2024-09-13T08:40:43.176429+0000","last_unstale":"2024-09-13T08:40:44.190249+0000","last_undegraded":"2024-09-13T08:40:44.190249+0000","last_fullsized":"2024-09-13T08:40:44.190249+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:18:13.165036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382839+0000","last_change":"2024-09-13T08:40:43.207935+0000","last_active":"2024-09-13T08:40:44.382839+0000","last_peered":"2024-09-13T08:40:44.382839+0000","last_clean":"2024-09-13T08:40:44.382839+0000","last_became_active":"2024-09-13T08:40:43.207661+0000","last_became_peered":"2024-09-13T08:40:43.207661+0000","last_unstale":"2024-09-13T08:40:44.382839+0000","last_undegraded":"2024-09-13T08:40:44.382839+0000","last_fullsized":"2024-09-13T08:40:44.382839+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:44:29.543300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192705+0000","last_change":"2024-09-13T08:40:43.184477+0000","last_active":"2024-09-13T08:40:44.192705+0000","last_peered":"2024-09-13T08:40:44.192705+0000","last_clean":"2024-09-13T08:40:44.192705+0000","last_became_active":"2024-09-13T08:40:43.183611+0000","last_became_peered":"2024-09-13T08:40:43.183611+0000","last_unstale":"2024-09-13T08:40:44.192705+0000","last_undegraded":"2024-09-13T08:40:44.192705+0000","last_fullsized":"2024-09-13T08:40:44.192705+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:07:21.199475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382868+0000","last_change":"2024-09-13T08:40:43.173503+0000","last_active":"2024-09-13T08:40:44.382868+0000","last_peered":"2024-09-13T08:40:44.382868+0000","last_clean":"2024-09-13T08:40:44.382868+0000","last_became_active":"2024-09-13T08:40:43.173209+0000","last_became_peered":"2024-09-13T08:40:43.173209+0000","last_unstale":"2024-09-13T08:40:44.382868+0000","last_undegraded":"2024-09-13T08:40:44.382868+0000","last_fullsized":"2024-09-13T08:40:44.382868+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:37.772751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383556+0000","last_change":"2024-09-13T08:40:43.188762+0000","last_active":"2024-09-13T08:40:44.383556+0000","last_peered":"2024-09-13T08:40:44.383556+0000","last_clean":"2024-09-13T08:40:44.383556+0000","last_became_active":"2024-09-13T08:40:43.188481+0000","last_became_peered":"2024-09-13T08:40:43.188481+0000","last_unstale":"2024-09-13T08:40:44.383556+0000","last_undegraded":"2024-09-13T08:40:44.383556+0000","last_fullsized":"2024-09-13T08:40:44.383556+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:49:44.520726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189805+0000","last_change":"2024-09-13T08:40:43.201185+0000","last_active":"2024-09-13T08:40:44.189805+0000","last_peered":"2024-09-13T08:40:44.189805+0000","last_clean":"2024-09-13T08:40:44.189805+0000","last_became_active":"2024-09-13T08:40:43.200898+0000","last_became_peered":"2024-09-13T08:40:43.200898+0000","last_unstale":"2024-09-13T08:40:44.189805+0000","last_undegraded":"2024-09-13T08:40:44.189805+0000","last_fullsized":"2024-09-13T08:40:44.189805+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:38.536150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192812+0000","last_change":"2024-09-13T08:40:43.208083+0000","last_active":"2024-09-13T08:40:44.192812+0000","last_peered":"2024-09-13T08:40:44.192812+0000","last_clean":"2024-09-13T08:40:44.192812+0000","last_became_active":"2024-09-13T08:40:43.207841+0000","last_became_peered":"2024-09-13T08:40:43.207841+0000","last_unstale":"2024-09-13T08:40:44.192812+0000","last_undegraded":"2024-09-13T08:40:44.192812+0000","last_fullsized":"2024-09-13T08:40:44.192812+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:34.262274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192601+0000","last_change":"2024-09-13T08:40:43.198438+0000","last_active":"2024-09-13T08:40:44.192601+0000","last_peered":"2024-09-13T08:40:44.192601+0000","last_clean":"2024-09-13T08:40:44.192601+0000","last_became_active":"2024-09-13T08:40:43.197966+0000","last_became_peered":"2024-09-13T08:40:43.197966+0000","last_unstale":"2024-09-13T08:40:44.192601+0000","last_undegraded":"2024-09-13T08:40:44.192601+0000","last_fullsized":"2024-09-13T08:40:44.192601+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:01:43.537917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383290+0000","last_change":"2024-09-13T08:40:43.191493+0000","last_active":"2024-09-13T08:40:44.383290+0000","last_peered":"2024-09-13T08:40:44.383290+0000","last_clean":"2024-09-13T08:40:44.383290+0000","last_became_active":"2024-09-13T08:40:43.191059+0000","last_became_peered":"2024-09-13T08:40:43.191059+0000","last_unstale":"2024-09-13T08:40:44.383290+0000","last_undegraded":"2024-09-13T08:40:44.383290+0000","last_fullsized":"2024-09-13T08:40:44.383290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:15.024462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383043+0000","last_change":"2024-09-13T08:40:43.182477+0000","last_active":"2024-09-13T08:40:44.383043+0000","last_peered":"2024-09-13T08:40:44.383043+0000","last_clean":"2024-09-13T08:40:44.383043+0000","last_became_active":"2024-09-13T08:40:43.182226+0000","last_became_peered":"2024-09-13T08:40:43.182226+0000","last_unstale":"2024-09-13T08:40:44.383043+0000","last_undegraded":"2024-09-13T08:40:44.383043+0000","last_fullsized":"2024-09-13T08:40:44.383043+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:14.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189122+0000","last_change":"2024-09-13T08:40:43.197098+0000","last_active":"2024-09-13T08:40:44.189122+0000","last_peered":"2024-09-13T08:40:44.189122+0000","last_clean":"2024-09-13T08:40:44.189122+0000","last_became_active":"2024-09-13T08:40:43.196818+0000","last_became_peered":"2024-09-13T08:40:43.196818+0000","last_unstale":"2024-09-13T08:40:44.189122+0000","last_undegraded":"2024-09-13T08:40:44.189122+0000","last_fullsized":"2024-09-13T08:40:44.189122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:41:32.293180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189294+0000","last_change":"2024-09-13T08:40:43.189479+0000","last_active":"2024-09-13T08:40:44.189294+0000","last_peered":"2024-09-13T08:40:44.189294+0000","last_clean":"2024-09-13T08:40:44.189294+0000","last_became_active":"2024-09-13T08:40:43.189207+0000","last_became_peered":"2024-09-13T08:40:43.189207+0000","last_unstale":"2024-09-13T08:40:44.189294+0000","last_undegraded":"2024-09-13T08:40:44.189294+0000","last_fullsized":"2024-09-13T08:40:44.189294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:49:44.520726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383724+0000","last_change":"2024-09-13T08:40:43.193080+0000","last_active":"2024-09-13T08:40:44.383724+0000","last_peered":"2024-09-13T08:40:44.383724+0000","last_clean":"2024-09-13T08:40:44.383724+0000","last_became_active":"2024-09-13T08:40:43.192785+0000","last_became_peered":"2024-09-13T08:40:43.192785+0000","last_unstale":"2024-09-13T08:40:44.383724+0000","last_undegraded":"2024-09-13T08:40:44.383724+0000","last_fullsized":"2024-09-13T08:40:44.383724+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:54.242903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190840+0000","last_change":"2024-09-13T08:40:42.175868+0000","last_active":"2024-09-13T08:40:44.190840+0000","last_peered":"2024-09-13T08:40:44.190840+0000","last_clean":"2024-09-13T08:40:44.190840+0000","last_became_active":"2024-09-13T08:40:42.175587+0000","last_became_peered":"2024-09-13T08:40:42.175587+0000","last_unstale":"2024-09-13T08:40:44.190840+0000","last_undegraded":"2024-09-13T08:40:44.190840+0000","last_fullsized":"2024-09-13T08:40:44.190840+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:28.527981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383709+0000","last_change":"2024-09-13T08:40:43.194685+0000","last_active":"2024-09-13T08:40:44.383709+0000","last_peered":"2024-09-13T08:40:44.383709+0000","last_clean":"2024-09-13T08:40:44.383709+0000","last_became_active":"2024-09-13T08:40:43.194145+0000","last_became_peered":"2024-09-13T08:40:43.194145+0000","last_unstale":"2024-09-13T08:40:44.383709+0000","last_undegraded":"2024-09-13T08:40:44.383709+0000","last_fullsized":"2024-09-13T08:40:44.383709+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:04:05.255094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192753+0000","last_change":"2024-09-13T08:40:42.182278+0000","last_active":"2024-09-13T08:40:44.192753+0000","last_peered":"2024-09-13T08:40:44.192753+0000","last_clean":"2024-09-13T08:40:44.192753+0000","last_became_active":"2024-09-13T08:40:42.181992+0000","last_became_peered":"2024-09-13T08:40:42.181992+0000","last_unstale":"2024-09-13T08:40:44.192753+0000","last_undegraded":"2024-09-13T08:40:44.192753+0000","last_fullsized":"2024-09-13T08:40:44.192753+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:51.100086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383869+0000","last_change":"2024-09-13T08:40:43.187867+0000","last_active":"2024-09-13T08:40:44.383869+0000","last_peered":"2024-09-13T08:40:44.383869+0000","last_clean":"2024-09-13T08:40:44.383869+0000","last_became_active":"2024-09-13T08:40:43.187621+0000","last_became_peered":"2024-09-13T08:40:43.187621+0000","last_unstale":"2024-09-13T08:40:44.383869+0000","last_undegraded":"2024-09-13T08:40:44.383869+0000","last_fullsized":"2024-09-13T08:40:44.383869+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:32.631857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190724+0000","last_change":"2024-09-13T08:40:42.179208+0000","last_active":"2024-09-13T08:40:44.190724+0000","last_peered":"2024-09-13T08:40:44.190724+0000","last_clean":"2024-09-13T08:40:44.190724+0000","last_became_active":"2024-09-13T08:40:42.178617+0000","last_became_peered":"2024-09-13T08:40:42.178617+0000","last_unstale":"2024-09-13T08:40:44.190724+0000","last_undegraded":"2024-09-13T08:40:44.190724+0000","last_fullsized":"2024-09-13T08:40:44.190724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:35:45.639180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190078+0000","last_change":"2024-09-13T08:40:43.192425+0000","last_active":"2024-09-13T08:40:44.190078+0000","last_peered":"2024-09-13T08:40:44.190078+0000","last_clean":"2024-09-13T08:40:44.190078+0000","last_became_active":"2024-09-13T08:40:43.191801+0000","last_became_peered":"2024-09-13T08:40:43.191801+0000","last_unstale":"2024-09-13T08:40:44.190078+0000","last_undegraded":"2024-09-13T08:40:44.190078+0000","last_fullsized":"2024-09-13T08:40:44.190078+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:15.901750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192763+0000","last_change":"2024-09-13T08:40:42.180695+0000","last_active":"2024-09-13T08:40:44.192763+0000","last_peered":"2024-09-13T08:40:44.192763+0000","last_clean":"2024-09-13T08:40:44.192763+0000","last_became_active":"2024-09-13T08:40:42.180392+0000","last_became_peered":"2024-09-13T08:40:42.180392+0000","last_unstale":"2024-09-13T08:40:44.192763+0000","last_undegraded":"2024-09-13T08:40:44.192763+0000","last_fullsized":"2024-09-13T08:40:44.192763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:40.439749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192543+0000","last_change":"2024-09-13T08:40:43.209100+0000","last_active":"2024-09-13T08:40:44.192543+0000","last_peered":"2024-09-13T08:40:44.192543+0000","last_clean":"2024-09-13T08:40:44.192543+0000","last_became_active":"2024-09-13T08:40:43.208834+0000","last_became_peered":"2024-09-13T08:40:43.208834+0000","last_unstale":"2024-09-13T08:40:44.192543+0000","last_undegraded":"2024-09-13T08:40:44.192543+0000","last_fullsized":"2024-09-13T08:40:44.192543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:55.259581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190745+0000","last_change":"2024-09-13T08:40:42.166893+0000","last_active":"2024-09-13T08:40:44.190745+0000","last_peered":"2024-09-13T08:40:44.190745+0000","last_clean":"2024-09-13T08:40:44.190745+0000","last_became_active":"2024-09-13T08:40:42.166507+0000","last_became_peered":"2024-09-13T08:40:42.166507+0000","last_unstale":"2024-09-13T08:40:44.190745+0000","last_undegraded":"2024-09-13T08:40:44.190745+0000","last_fullsized":"2024-09-13T08:40:44.190745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:20.818876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383443+0000","last_change":"2024-09-13T08:40:43.193379+0000","last_active":"2024-09-13T08:40:44.383443+0000","last_peered":"2024-09-13T08:40:44.383443+0000","last_clean":"2024-09-13T08:40:44.383443+0000","last_became_active":"2024-09-13T08:40:43.193015+0000","last_became_peered":"2024-09-13T08:40:43.193015+0000","last_unstale":"2024-09-13T08:40:44.383443+0000","last_undegraded":"2024-09-13T08:40:44.383443+0000","last_fullsized":"2024-09-13T08:40:44.383443+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:40.764211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192825+0000","last_change":"2024-09-13T08:40:42.184150+0000","last_active":"2024-09-13T08:40:44.192825+0000","last_peered":"2024-09-13T08:40:44.192825+0000","last_clean":"2024-09-13T08:40:44.192825+0000","last_became_active":"2024-09-13T08:40:42.183763+0000","last_became_peered":"2024-09-13T08:40:42.183763+0000","last_unstale":"2024-09-13T08:40:44.192825+0000","last_undegraded":"2024-09-13T08:40:44.192825+0000","last_fullsized":"2024-09-13T08:40:44.192825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:59.988155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383695+0000","last_change":"2024-09-13T08:40:43.187575+0000","last_active":"2024-09-13T08:40:44.383695+0000","last_peered":"2024-09-13T08:40:44.383695+0000","last_clean":"2024-09-13T08:40:44.383695+0000","last_became_active":"2024-09-13T08:40:43.186758+0000","last_became_peered":"2024-09-13T08:40:43.186758+0000","last_unstale":"2024-09-13T08:40:44.383695+0000","last_undegraded":"2024-09-13T08:40:44.383695+0000","last_fullsized":"2024-09-13T08:40:44.383695+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:38:52.313781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383045+0000","last_change":"2024-09-13T08:40:42.175361+0000","last_active":"2024-09-13T08:40:44.383045+0000","last_peered":"2024-09-13T08:40:44.383045+0000","last_clean":"2024-09-13T08:40:44.383045+0000","last_became_active":"2024-09-13T08:40:42.174980+0000","last_became_peered":"2024-09-13T08:40:42.174980+0000","last_unstale":"2024-09-13T08:40:44.383045+0000","last_undegraded":"2024-09-13T08:40:44.383045+0000","last_fullsized":"2024-09-13T08:40:44.383045+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:59.925397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192486+0000","last_change":"2024-09-13T08:40:43.200353+0000","last_active":"2024-09-13T08:40:44.192486+0000","last_peered":"2024-09-13T08:40:44.192486+0000","last_clean":"2024-09-13T08:40:44.192486+0000","last_became_active":"2024-09-13T08:40:43.198409+0000","last_became_peered":"2024-09-13T08:40:43.198409+0000","last_unstale":"2024-09-13T08:40:44.192486+0000","last_undegraded":"2024-09-13T08:40:44.192486+0000","last_fullsized":"2024-09-13T08:40:44.192486+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:10.635103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382632+0000","last_change":"2024-09-13T08:40:42.178338+0000","last_active":"2024-09-13T08:40:44.382632+0000","last_peered":"2024-09-13T08:40:44.382632+0000","last_clean":"2024-09-13T08:40:44.382632+0000","last_became_active":"2024-09-13T08:40:42.178025+0000","last_became_peered":"2024-09-13T08:40:42.178025+0000","last_unstale":"2024-09-13T08:40:44.382632+0000","last_undegraded":"2024-09-13T08:40:44.382632+0000","last_fullsized":"2024-09-13T08:40:44.382632+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:31.891865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189048+0000","last_change":"2024-09-13T08:40:43.204789+0000","last_active":"2024-09-13T08:40:44.189048+0000","last_peered":"2024-09-13T08:40:44.189048+0000","last_clean":"2024-09-13T08:40:44.189048+0000","last_became_active":"2024-09-13T08:40:43.204114+0000","last_became_peered":"2024-09-13T08:40:43.204114+0000","last_unstale":"2024-09-13T08:40:44.189048+0000","last_undegraded":"2024-09-13T08:40:44.189048+0000","last_fullsized":"2024-09-13T08:40:44.189048+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:20.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.188894+0000","last_change":"2024-09-13T08:40:42.178810+0000","last_active":"2024-09-13T08:40:44.188894+0000","last_peered":"2024-09-13T08:40:44.188894+0000","last_clean":"2024-09-13T08:40:44.188894+0000","last_became_active":"2024-09-13T08:40:42.173486+0000","last_became_peered":"2024-09-13T08:40:42.173486+0000","last_unstale":"2024-09-13T08:40:44.188894+0000","last_undegraded":"2024-09-13T08:40:44.188894+0000","last_fullsized":"2024-09-13T08:40:44.188894+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:12.229793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.188889+0000","last_change":"2024-09-13T08:40:43.194106+0000","last_active":"2024-09-13T08:40:44.188889+0000","last_peered":"2024-09-13T08:40:44.188889+0000","last_clean":"2024-09-13T08:40:44.188889+0000","last_became_active":"2024-09-13T08:40:43.193804+0000","last_became_peered":"2024-09-13T08:40:43.193804+0000","last_unstale":"2024-09-13T08:40:44.188889+0000","last_undegraded":"2024-09-13T08:40:44.188889+0000","last_fullsized":"2024-09-13T08:40:44.188889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:54.242903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383168+0000","last_change":"2024-09-13T08:40:42.183102+0000","last_active":"2024-09-13T08:40:44.383168+0000","last_peered":"2024-09-13T08:40:44.383168+0000","last_clean":"2024-09-13T08:40:44.383168+0000","last_became_active":"2024-09-13T08:40:42.182748+0000","last_became_peered":"2024-09-13T08:40:42.182748+0000","last_unstale":"2024-09-13T08:40:44.383168+0000","last_undegraded":"2024-09-13T08:40:44.383168+0000","last_fullsized":"2024-09-13T08:40:44.383168+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:10.167131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189809+0000","last_change":"2024-09-13T08:40:43.191524+0000","last_active":"2024-09-13T08:40:44.189809+0000","last_peered":"2024-09-13T08:40:44.189809+0000","last_clean":"2024-09-13T08:40:44.189809+0000","last_became_active":"2024-09-13T08:40:43.191240+0000","last_became_peered":"2024-09-13T08:40:43.191240+0000","last_unstale":"2024-09-13T08:40:44.189809+0000","last_undegraded":"2024-09-13T08:40:44.189809+0000","last_fullsized":"2024-09-13T08:40:44.189809+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:04:05.255094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383782+0000","last_change":"2024-09-13T08:40:42.177115+0000","last_active":"2024-09-13T08:40:44.383782+0000","last_peered":"2024-09-13T08:40:44.383782+0000","last_clean":"2024-09-13T08:40:44.383782+0000","last_became_active":"2024-09-13T08:40:42.176751+0000","last_became_peered":"2024-09-13T08:40:42.176751+0000","last_unstale":"2024-09-13T08:40:44.383782+0000","last_undegraded":"2024-09-13T08:40:44.383782+0000","last_fullsized":"2024-09-13T08:40:44.383782+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:23.536012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192901+0000","last_change":"2024-09-13T08:40:43.197281+0000","last_active":"2024-09-13T08:40:44.192901+0000","last_peered":"2024-09-13T08:40:44.192901+0000","last_clean":"2024-09-13T08:40:44.192901+0000","last_became_active":"2024-09-13T08:40:43.196989+0000","last_became_peered":"2024-09-13T08:40:43.196989+0000","last_unstale":"2024-09-13T08:40:44.192901+0000","last_undegraded":"2024-09-13T08:40:44.192901+0000","last_fullsized":"2024-09-13T08:40:44.192901+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:19:36.287659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383465+0000","last_change":"2024-09-13T08:40:42.180947+0000","last_active":"2024-09-13T08:40:44.383465+0000","last_peered":"2024-09-13T08:40:44.383465+0000","last_clean":"2024-09-13T08:40:44.383465+0000","last_became_active":"2024-09-13T08:40:42.180647+0000","last_became_peered":"2024-09-13T08:40:42.180647+0000","last_unstale":"2024-09-13T08:40:44.383465+0000","last_undegraded":"2024-09-13T08:40:44.383465+0000","last_fullsized":"2024-09-13T08:40:44.383465+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:31.625696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.188719+0000","last_change":"2024-09-13T08:40:43.177272+0000","last_active":"2024-09-13T08:40:44.188719+0000","last_peered":"2024-09-13T08:40:44.188719+0000","last_clean":"2024-09-13T08:40:44.188719+0000","last_became_active":"2024-09-13T08:40:43.176928+0000","last_became_peered":"2024-09-13T08:40:43.176928+0000","last_unstale":"2024-09-13T08:40:44.188719+0000","last_undegraded":"2024-09-13T08:40:44.188719+0000","last_fullsized":"2024-09-13T08:40:44.188719+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:56.468346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383540+0000","last_change":"2024-09-13T08:40:42.176291+0000","last_active":"2024-09-13T08:40:44.383540+0000","last_peered":"2024-09-13T08:40:44.383540+0000","last_clean":"2024-09-13T08:40:44.383540+0000","last_became_active":"2024-09-13T08:40:42.175697+0000","last_became_peered":"2024-09-13T08:40:42.175697+0000","last_unstale":"2024-09-13T08:40:44.383540+0000","last_undegraded":"2024-09-13T08:40:44.383540+0000","last_fullsized":"2024-09-13T08:40:44.383540+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:18.721185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383373+0000","last_change":"2024-09-13T08:40:43.194019+0000","last_active":"2024-09-13T08:40:44.383373+0000","last_peered":"2024-09-13T08:40:44.383373+0000","last_clean":"2024-09-13T08:40:44.383373+0000","last_became_active":"2024-09-13T08:40:43.192433+0000","last_became_peered":"2024-09-13T08:40:43.192433+0000","last_unstale":"2024-09-13T08:40:44.383373+0000","last_undegraded":"2024-09-13T08:40:44.383373+0000","last_fullsized":"2024-09-13T08:40:44.383373+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:35.566813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383255+0000","last_change":"2024-09-13T08:40:42.176544+0000","last_active":"2024-09-13T08:40:44.383255+0000","last_peered":"2024-09-13T08:40:44.383255+0000","last_clean":"2024-09-13T08:40:44.383255+0000","last_became_active":"2024-09-13T08:40:42.175781+0000","last_became_peered":"2024-09-13T08:40:42.175781+0000","last_unstale":"2024-09-13T08:40:44.383255+0000","last_undegraded":"2024-09-13T08:40:44.383255+0000","last_fullsized":"2024-09-13T08:40:44.383255+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:53.422970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382821+0000","last_change":"2024-09-13T08:40:43.195108+0000","last_active":"2024-09-13T08:40:44.382821+0000","last_peered":"2024-09-13T08:40:44.382821+0000","last_clean":"2024-09-13T08:40:44.382821+0000","last_became_active":"2024-09-13T08:40:43.193623+0000","last_became_peered":"2024-09-13T08:40:43.193623+0000","last_unstale":"2024-09-13T08:40:44.382821+0000","last_undegraded":"2024-09-13T08:40:44.382821+0000","last_fullsized":"2024-09-13T08:40:44.382821+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:56.468346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189309+0000","last_change":"2024-09-13T08:40:42.184583+0000","last_active":"2024-09-13T08:40:44.189309+0000","last_peered":"2024-09-13T08:40:44.189309+0000","last_clean":"2024-09-13T08:40:44.189309+0000","last_became_active":"2024-09-13T08:40:42.184269+0000","last_became_peered":"2024-09-13T08:40:42.184269+0000","last_unstale":"2024-09-13T08:40:44.189309+0000","last_undegraded":"2024-09-13T08:40:44.189309+0000","last_fullsized":"2024-09-13T08:40:44.189309+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:37.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382966+0000","last_change":"2024-09-13T08:40:43.181834+0000","last_active":"2024-09-13T08:40:44.382966+0000","last_peered":"2024-09-13T08:40:44.382966+0000","last_clean":"2024-09-13T08:40:44.382966+0000","last_became_active":"2024-09-13T08:40:43.181558+0000","last_became_peered":"2024-09-13T08:40:43.181558+0000","last_unstale":"2024-09-13T08:40:44.382966+0000","last_undegraded":"2024-09-13T08:40:44.382966+0000","last_fullsized":"2024-09-13T08:40:44.382966+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:33.508450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":23,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382760+0000","last_change":"2024-09-13T08:40:42.173542+0000","last_active":"2024-09-13T08:40:44.382760+0000","last_peered":"2024-09-13T08:40:44.382760+0000","last_clean":"2024-09-13T08:40:44.382760+0000","last_became_active":"2024-09-13T08:40:42.173206+0000","last_became_peered":"2024-09-13T08:40:42.173206+0000","last_unstale":"2024-09-13T08:40:44.382760+0000","last_undegraded":"2024-09-13T08:40:44.382760+0000","last_fullsized":"2024-09-13T08:40:44.382760+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:36.766590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383742+0000","last_change":"2024-09-13T08:40:43.210476+0000","last_active":"2024-09-13T08:40:44.383742+0000","last_peered":"2024-09-13T08:40:44.383742+0000","last_clean":"2024-09-13T08:40:44.383742+0000","last_became_active":"2024-09-13T08:40:43.210279+0000","last_became_peered":"2024-09-13T08:40:43.210279+0000","last_unstale":"2024-09-13T08:40:44.383742+0000","last_undegraded":"2024-09-13T08:40:44.383742+0000","last_fullsized":"2024-09-13T08:40:44.383742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:20.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190015+0000","last_change":"2024-09-13T08:40:42.176695+0000","last_active":"2024-09-13T08:40:44.190015+0000","last_peered":"2024-09-13T08:40:44.190015+0000","last_clean":"2024-09-13T08:40:44.190015+0000","last_became_active":"2024-09-13T08:40:42.176387+0000","last_became_peered":"2024-09-13T08:40:42.176387+0000","last_unstale":"2024-09-13T08:40:44.190015+0000","last_undegraded":"2024-09-13T08:40:44.190015+0000","last_fullsized":"2024-09-13T08:40:44.190015+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:23.536012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383976+0000","last_change":"2024-09-13T08:40:43.193233+0000","last_active":"2024-09-13T08:40:44.383976+0000","last_peered":"2024-09-13T08:40:44.383976+0000","last_clean":"2024-09-13T08:40:44.383976+0000","last_became_active":"2024-09-13T08:40:43.192944+0000","last_became_peered":"2024-09-13T08:40:43.192944+0000","last_unstale":"2024-09-13T08:40:44.383976+0000","last_undegraded":"2024-09-13T08:40:44.383976+0000","last_fullsized":"2024-09-13T08:40:44.383976+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:26:15.236926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.188982+0000","last_change":"2024-09-13T08:40:42.180372+0000","last_active":"2024-09-13T08:40:44.188982+0000","last_peered":"2024-09-13T08:40:44.188982+0000","last_clean":"2024-09-13T08:40:44.188982+0000","last_became_active":"2024-09-13T08:40:42.180079+0000","last_became_peered":"2024-09-13T08:40:42.180079+0000","last_unstale":"2024-09-13T08:40:44.188982+0000","last_undegraded":"2024-09-13T08:40:44.188982+0000","last_fullsized":"2024-09-13T08:40:44.188982+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:21.515280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.384039+0000","last_change":"2024-09-13T08:40:43.177434+0000","last_active":"2024-09-13T08:40:44.384039+0000","last_peered":"2024-09-13T08:40:44.384039+0000","last_clean":"2024-09-13T08:40:44.384039+0000","last_became_active":"2024-09-13T08:40:43.177163+0000","last_became_peered":"2024-09-13T08:40:43.177163+0000","last_unstale":"2024-09-13T08:40:44.384039+0000","last_undegraded":"2024-09-13T08:40:44.384039+0000","last_fullsized":"2024-09-13T08:40:44.384039+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:12.306506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.188941+0000","last_change":"2024-09-13T08:40:42.185496+0000","last_active":"2024-09-13T08:40:44.188941+0000","last_peered":"2024-09-13T08:40:44.188941+0000","last_clean":"2024-09-13T08:40:44.188941+0000","last_became_active":"2024-09-13T08:40:42.184959+0000","last_became_peered":"2024-09-13T08:40:42.184959+0000","last_unstale":"2024-09-13T08:40:44.188941+0000","last_undegraded":"2024-09-13T08:40:44.188941+0000","last_fullsized":"2024-09-13T08:40:44.188941+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:18:11.442697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.188764+0000","last_change":"2024-09-13T08:40:43.193014+0000","last_active":"2024-09-13T08:40:44.188764+0000","last_peered":"2024-09-13T08:40:44.188764+0000","last_clean":"2024-09-13T08:40:44.188764+0000","last_became_active":"2024-09-13T08:40:43.192539+0000","last_became_peered":"2024-09-13T08:40:43.192539+0000","last_unstale":"2024-09-13T08:40:44.188764+0000","last_undegraded":"2024-09-13T08:40:44.188764+0000","last_fullsized":"2024-09-13T08:40:44.188764+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:44:29.543300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190476+0000","last_change":"2024-09-13T08:40:42.188197+0000","last_active":"2024-09-13T08:40:44.190476+0000","last_peered":"2024-09-13T08:40:44.190476+0000","last_clean":"2024-09-13T08:40:44.190476+0000","last_became_active":"2024-09-13T08:40:42.187908+0000","last_became_peered":"2024-09-13T08:40:42.187908+0000","last_unstale":"2024-09-13T08:40:44.190476+0000","last_undegraded":"2024-09-13T08:40:44.190476+0000","last_fullsized":"2024-09-13T08:40:44.190476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:36:31.947351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382873+0000","last_change":"2024-09-13T08:40:43.194030+0000","last_active":"2024-09-13T08:40:44.382873+0000","last_peered":"2024-09-13T08:40:44.382873+0000","last_clean":"2024-09-13T08:40:44.382873+0000","last_became_active":"2024-09-13T08:40:43.193719+0000","last_became_peered":"2024-09-13T08:40:43.193719+0000","last_unstale":"2024-09-13T08:40:44.382873+0000","last_undegraded":"2024-09-13T08:40:44.382873+0000","last_fullsized":"2024-09-13T08:40:44.382873+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:38.536150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383615+0000","last_change":"2024-09-13T08:40:42.170852+0000","last_active":"2024-09-13T08:40:44.383615+0000","last_peered":"2024-09-13T08:40:44.383615+0000","last_clean":"2024-09-13T08:40:44.383615+0000","last_became_active":"2024-09-13T08:40:42.170588+0000","last_became_peered":"2024-09-13T08:40:42.170588+0000","last_unstale":"2024-09-13T08:40:44.383615+0000","last_undegraded":"2024-09-13T08:40:44.383615+0000","last_fullsized":"2024-09-13T08:40:44.383615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:32.502289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190568+0000","last_change":"2024-09-13T08:40:43.203021+0000","last_active":"2024-09-13T08:40:44.190568+0000","last_peered":"2024-09-13T08:40:44.190568+0000","last_clean":"2024-09-13T08:40:44.190568+0000","last_became_active":"2024-09-13T08:40:43.199689+0000","last_became_peered":"2024-09-13T08:40:43.199689+0000","last_unstale":"2024-09-13T08:40:44.190568+0000","last_undegraded":"2024-09-13T08:40:44.190568+0000","last_fullsized":"2024-09-13T08:40:44.190568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:52.407146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383312+0000","last_change":"2024-09-13T08:40:42.175444+0000","last_active":"2024-09-13T08:40:44.383312+0000","last_peered":"2024-09-13T08:40:44.383312+0000","last_clean":"2024-09-13T08:40:44.383312+0000","last_became_active":"2024-09-13T08:40:42.174590+0000","last_became_peered":"2024-09-13T08:40:42.174590+0000","last_unstale":"2024-09-13T08:40:44.383312+0000","last_undegraded":"2024-09-13T08:40:44.383312+0000","last_fullsized":"2024-09-13T08:40:44.383312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:32.473636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189484+0000","last_change":"2024-09-13T08:40:43.177933+0000","last_active":"2024-09-13T08:40:44.189484+0000","last_peered":"2024-09-13T08:40:44.189484+0000","last_clean":"2024-09-13T08:40:44.189484+0000","last_became_active":"2024-09-13T08:40:43.177479+0000","last_became_peered":"2024-09-13T08:40:43.177479+0000","last_unstale":"2024-09-13T08:40:44.189484+0000","last_undegraded":"2024-09-13T08:40:44.189484+0000","last_fullsized":"2024-09-13T08:40:44.189484+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:22:32.123093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382618+0000","last_change":"2024-09-13T08:40:42.183091+0000","last_active":"2024-09-13T08:40:44.382618+0000","last_peered":"2024-09-13T08:40:44.382618+0000","last_clean":"2024-09-13T08:40:44.382618+0000","last_became_active":"2024-09-13T08:40:42.182685+0000","last_became_peered":"2024-09-13T08:40:42.182685+0000","last_unstale":"2024-09-13T08:40:44.382618+0000","last_undegraded":"2024-09-13T08:40:44.382618+0000","last_fullsized":"2024-09-13T08:40:44.382618+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:03.155901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189456+0000","last_change":"2024-09-13T08:40:43.181826+0000","last_active":"2024-09-13T08:40:44.189456+0000","last_peered":"2024-09-13T08:40:44.189456+0000","last_clean":"2024-09-13T08:40:44.189456+0000","last_became_active":"2024-09-13T08:40:43.181516+0000","last_became_peered":"2024-09-13T08:40:43.181516+0000","last_unstale":"2024-09-13T08:40:44.189456+0000","last_undegraded":"2024-09-13T08:40:44.189456+0000","last_fullsized":"2024-09-13T08:40:44.189456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:14.577393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192388+0000","last_change":"2024-09-13T08:40:42.177701+0000","last_active":"2024-09-13T08:40:44.192388+0000","last_peered":"2024-09-13T08:40:44.192388+0000","last_clean":"2024-09-13T08:40:44.192388+0000","last_became_active":"2024-09-13T08:40:42.177418+0000","last_became_peered":"2024-09-13T08:40:42.177418+0000","last_unstale":"2024-09-13T08:40:44.192388+0000","last_undegraded":"2024-09-13T08:40:44.192388+0000","last_fullsized":"2024-09-13T08:40:44.192388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:23.873609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192884+0000","last_change":"2024-09-13T08:40:43.198409+0000","last_active":"2024-09-13T08:40:44.192884+0000","last_peered":"2024-09-13T08:40:44.192884+0000","last_clean":"2024-09-13T08:40:44.192884+0000","last_became_active":"2024-09-13T08:40:43.198235+0000","last_became_peered":"2024-09-13T08:40:43.198235+0000","last_unstale":"2024-09-13T08:40:44.192884+0000","last_undegraded":"2024-09-13T08:40:44.192884+0000","last_fullsized":"2024-09-13T08:40:44.192884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:05.822435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190074+0000","last_change":"2024-09-13T08:40:42.177210+0000","last_active":"2024-09-13T08:40:44.190074+0000","last_peered":"2024-09-13T08:40:44.190074+0000","last_clean":"2024-09-13T08:40:44.190074+0000","last_became_active":"2024-09-13T08:40:42.176919+0000","last_became_peered":"2024-09-13T08:40:42.176919+0000","last_unstale":"2024-09-13T08:40:44.190074+0000","last_undegraded":"2024-09-13T08:40:44.190074+0000","last_fullsized":"2024-09-13T08:40:44.190074+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:05.647017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383477+0000","last_change":"2024-09-13T08:40:43.189382+0000","last_active":"2024-09-13T08:40:44.383477+0000","last_peered":"2024-09-13T08:40:44.383477+0000","last_clean":"2024-09-13T08:40:44.383477+0000","last_became_active":"2024-09-13T08:40:43.188958+0000","last_became_peered":"2024-09-13T08:40:43.188958+0000","last_unstale":"2024-09-13T08:40:44.383477+0000","last_undegraded":"2024-09-13T08:40:44.383477+0000","last_fullsized":"2024-09-13T08:40:44.383477+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:57.531742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383462+0000","last_change":"2024-09-13T08:40:42.186311+0000","last_active":"2024-09-13T08:40:44.383462+0000","last_peered":"2024-09-13T08:40:44.383462+0000","last_clean":"2024-09-13T08:40:44.383462+0000","last_became_active":"2024-09-13T08:40:42.185993+0000","last_became_peered":"2024-09-13T08:40:42.185993+0000","last_unstale":"2024-09-13T08:40:44.383462+0000","last_undegraded":"2024-09-13T08:40:44.383462+0000","last_fullsized":"2024-09-13T08:40:44.383462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:07.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.193026+0000","last_change":"2024-09-13T08:40:43.184101+0000","last_active":"2024-09-13T08:40:44.193026+0000","last_peered":"2024-09-13T08:40:44.193026+0000","last_clean":"2024-09-13T08:40:44.193026+0000","last_became_active":"2024-09-13T08:40:43.183722+0000","last_became_peered":"2024-09-13T08:40:43.183722+0000","last_unstale":"2024-09-13T08:40:44.193026+0000","last_undegraded":"2024-09-13T08:40:44.193026+0000","last_fullsized":"2024-09-13T08:40:44.193026+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:01.418591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383365+0000","last_change":"2024-09-13T08:40:42.189558+0000","last_active":"2024-09-13T08:40:44.383365+0000","last_peered":"2024-09-13T08:40:44.383365+0000","last_clean":"2024-09-13T08:40:44.383365+0000","last_became_active":"2024-09-13T08:40:42.189312+0000","last_became_peered":"2024-09-13T08:40:42.189312+0000","last_unstale":"2024-09-13T08:40:44.383365+0000","last_undegraded":"2024-09-13T08:40:44.383365+0000","last_fullsized":"2024-09-13T08:40:44.383365+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:40.270289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382998+0000","last_change":"2024-09-13T08:40:43.192030+0000","last_active":"2024-09-13T08:40:44.382998+0000","last_peered":"2024-09-13T08:40:44.382998+0000","last_clean":"2024-09-13T08:40:44.382998+0000","last_became_active":"2024-09-13T08:40:43.191644+0000","last_became_peered":"2024-09-13T08:40:43.191644+0000","last_unstale":"2024-09-13T08:40:44.382998+0000","last_undegraded":"2024-09-13T08:40:44.382998+0000","last_fullsized":"2024-09-13T08:40:44.382998+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:53:12.975649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383368+0000","last_change":"2024-09-13T08:40:42.179810+0000","last_active":"2024-09-13T08:40:44.383368+0000","last_peered":"2024-09-13T08:40:44.383368+0000","last_clean":"2024-09-13T08:40:44.383368+0000","last_became_active":"2024-09-13T08:40:42.179512+0000","last_became_peered":"2024-09-13T08:40:42.179512+0000","last_unstale":"2024-09-13T08:40:44.383368+0000","last_undegraded":"2024-09-13T08:40:44.383368+0000","last_fullsized":"2024-09-13T08:40:44.383368+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:13.357647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189663+0000","last_change":"2024-09-13T08:40:43.176719+0000","last_active":"2024-09-13T08:40:44.189663+0000","last_peered":"2024-09-13T08:40:44.189663+0000","last_clean":"2024-09-13T08:40:44.189663+0000","last_became_active":"2024-09-13T08:40:43.176416+0000","last_became_peered":"2024-09-13T08:40:43.176416+0000","last_unstale":"2024-09-13T08:40:44.189663+0000","last_undegraded":"2024-09-13T08:40:44.189663+0000","last_fullsized":"2024-09-13T08:40:44.189663+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:02.005810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192932+0000","last_change":"2024-09-13T08:40:42.176816+0000","last_active":"2024-09-13T08:40:44.192932+0000","last_peered":"2024-09-13T08:40:44.192932+0000","last_clean":"2024-09-13T08:40:44.192932+0000","last_became_active":"2024-09-13T08:40:42.176236+0000","last_became_peered":"2024-09-13T08:40:42.176236+0000","last_unstale":"2024-09-13T08:40:44.192932+0000","last_undegraded":"2024-09-13T08:40:44.192932+0000","last_fullsized":"2024-09-13T08:40:44.192932+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:14.430820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.188423+0000","last_change":"2024-09-13T08:40:43.176967+0000","last_active":"2024-09-13T08:40:44.188423+0000","last_peered":"2024-09-13T08:40:44.188423+0000","last_clean":"2024-09-13T08:40:44.188423+0000","last_became_active":"2024-09-13T08:40:43.176658+0000","last_became_peered":"2024-09-13T08:40:43.176658+0000","last_unstale":"2024-09-13T08:40:44.188423+0000","last_undegraded":"2024-09-13T08:40:44.188423+0000","last_fullsized":"2024-09-13T08:40:44.188423+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:43.583764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.189410+0000","last_change":"2024-09-13T08:40:42.174657+0000","last_active":"2024-09-13T08:40:44.189410+0000","last_peered":"2024-09-13T08:40:44.189410+0000","last_clean":"2024-09-13T08:40:44.189410+0000","last_became_active":"2024-09-13T08:40:42.174374+0000","last_became_peered":"2024-09-13T08:40:42.174374+0000","last_unstale":"2024-09-13T08:40:44.189410+0000","last_undegraded":"2024-09-13T08:40:44.189410+0000","last_fullsized":"2024-09-13T08:40:44.189410+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:30.284584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.382988+0000","last_change":"2024-09-13T08:40:43.180762+0000","last_active":"2024-09-13T08:40:44.382988+0000","last_peered":"2024-09-13T08:40:44.382988+0000","last_clean":"2024-09-13T08:40:44.382988+0000","last_became_active":"2024-09-13T08:40:43.180576+0000","last_became_peered":"2024-09-13T08:40:43.180576+0000","last_unstale":"2024-09-13T08:40:44.382988+0000","last_undegraded":"2024-09-13T08:40:44.382988+0000","last_fullsized":"2024-09-13T08:40:44.382988+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:11.173292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.383027+0000","last_change":"2024-09-13T08:40:42.177987+0000","last_active":"2024-09-13T08:40:44.383027+0000","last_peered":"2024-09-13T08:40:44.383027+0000","last_clean":"2024-09-13T08:40:44.383027+0000","last_became_active":"2024-09-13T08:40:42.177653+0000","last_became_peered":"2024-09-13T08:40:42.177653+0000","last_unstale":"2024-09-13T08:40:44.383027+0000","last_undegraded":"2024-09-13T08:40:44.383027+0000","last_fullsized":"2024-09-13T08:40:44.383027+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:44.296127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.190538+0000","last_change":"2024-09-13T08:40:43.180067+0000","last_active":"2024-09-13T08:40:44.190538+0000","last_peered":"2024-09-13T08:40:44.190538+0000","last_clean":"2024-09-13T08:40:44.190538+0000","last_became_active":"2024-09-13T08:40:43.179734+0000","last_became_peered":"2024-09-13T08:40:43.179734+0000","last_unstale":"2024-09-13T08:40:44.190538+0000","last_undegraded":"2024-09-13T08:40:44.190538+0000","last_fullsized":"2024-09-13T08:40:44.190538+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:42.149568+0000","last_clean_scrub_stamp":"2024-09-13T08:40:42.149568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:24.879770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.192992+0000","last_change":"2024-09-13T08:40:42.180201+0000","last_active":"2024-09-13T08:40:44.192992+0000","last_peered":"2024-09-13T08:40:44.192992+0000","last_clean":"2024-09-13T08:40:44.192992+0000","last_became_active":"2024-09-13T08:40:42.179874+0000","last_became_peered":"2024-09-13T08:40:42.179874+0000","last_unstale":"2024-09-13T08:40:44.192992+0000","last_undegraded":"2024-09-13T08:40:44.192992+0000","last_fullsized":"2024-09-13T08:40:44.192992+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:40:41.143406+0000","last_clean_scrub_stamp":"2024-09-13T08:40:41.143406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:25.549286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":157,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T08:40:44.188435+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:40:44.188435+0000","last_peered":"2024-09-13T08:40:44.188435+0000","last_clean":"2024-09-13T08:40:44.188435+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:40:44.188435+0000","last_undegraded":"2024-09-13T08:40:44.188435+0000","last_fullsized":"2024-09-13T08:40:44.188435+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":548864,"data_stored":220523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1142784,"data_stored":810879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509484,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":565248,"data_stored":222193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":557056,"data_stored":222101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":548864,"data_stored":220579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":810927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35888,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335952,"statfs":{"total":96636764160,"available":96600014848,"internally_reserved":0,"allocated":552960,"data_stored":221105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:40:52.414 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:40:52.414 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T08:40:52.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 -- 172.21.15.44:0/2346522422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f459015bd50 msgr2=0x7f45900ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:52.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 --2- 172.21.15.44:0/2346522422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f459015bd50 0x7f45900ce5e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f459006d870 tx=0x7f458001c6e0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 -- 172.21.15.44:0/2346522422 shutdown_connections 2024-09-13T08:40:52.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 --2- 172.21.15.44:0/2346522422 >> v2:172.21.15.129:3300/0 conn(0x7f459015c130 0x7f45900ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 --2- 172.21.15.44:0/2346522422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f459015bd50 0x7f45900ce5e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 -- 172.21.15.44:0/2346522422 >> 172.21.15.44:0/2346522422 conn(0x7f45900d1ea0 msgr2=0x7f45900d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:52.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 -- 172.21.15.44:0/2346522422 shutdown_connections 2024-09-13T08:40:52.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 -- 172.21.15.44:0/2346522422 wait complete. 2024-09-13T08:40:52.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 Processor -- start 2024-09-13T08:40:52.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 -- start start 2024-09-13T08:40:52.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45900cf4e0 0x7f459014f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f459015bd50 0x7f459006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f459015c130 0x7f45900c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4590168e50 con 0x7f459015bd50 2024-09-13T08:40:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4590168cd0 con 0x7f45900cf4e0 2024-09-13T08:40:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.604+0000 7f4596400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4590168fd0 con 0x7f459015c130 2024-09-13T08:40:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f459015bd50 0x7f459006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f459015bd50 0x7f459006dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45792/0 (socket says 172.21.15.44:45792) 2024-09-13T08:40:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587e00640 1 -- 172.21.15.44:0/3106042533 learned_addr learned my addr 172.21.15.44:0/3106042533 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f458f400640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45900cf4e0 0x7f459014f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f458fe00640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f459015c130 0x7f45900c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:52.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587e00640 1 -- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f459015c130 msgr2=0x7f45900c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587e00640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f459015c130 0x7f45900c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587e00640 1 -- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45900cf4e0 msgr2=0x7f459014f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587e00640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45900cf4e0 0x7f459014f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587e00640 1 -- 172.21.15.44:0/3106042533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45900c9a90 con 0x7f459015bd50 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587e00640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f459015bd50 0x7f459006dd30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f457800e7a0 tx=0x7f457800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587400640 1 -- 172.21.15.44:0/3106042533 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457800ccc0 con 0x7f459015bd50 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587400640 1 -- 172.21.15.44:0/3106042533 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f457800ce60 con 0x7f459015bd50 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4596400640 1 -- 172.21.15.44:0/3106042533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459006f940 con 0x7f459015bd50 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587400640 1 -- 172.21.15.44:0/3106042533 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578005260 con 0x7f459015bd50 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4596400640 1 -- 172.21.15.44:0/3106042533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459014f840 con 0x7f459015bd50 2024-09-13T08:40:52.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4596400640 1 -- 172.21.15.44:0/3106042533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45900c5dd0 con 0x7f459015bd50 2024-09-13T08:40:52.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.608+0000 7f4587400640 1 -- 172.21.15.44:0/3106042533 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f4578005400 con 0x7f459015bd50 2024-09-13T08:40:52.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.612+0000 7f4587400640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f456c073220 0x7f456c0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:52.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.612+0000 7f4587400640 1 -- 172.21.15.44:0/3106042533 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f4578094ef0 con 0x7f459015bd50 2024-09-13T08:40:52.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.612+0000 7f458f400640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f456c073220 0x7f456c0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:52.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.612+0000 7f458f400640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f456c073220 0x7f456c0756e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f45800af000 tx=0x7f458001af20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:52.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.616+0000 7f4587400640 1 -- 172.21.15.44:0/3106042533 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f457805fae0 con 0x7f459015bd50 2024-09-13T08:40:52.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.852+0000 7f4596400640 1 -- 172.21.15.44:0/3106042533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f45900721a0 con 0x7f459015bd50 2024-09-13T08:40:52.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.852+0000 7f4587400640 1 -- 172.21.15.44:0/3106042533 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f4578068020 con 0x7f459015bd50 2024-09-13T08:40:52.857 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T08:40:52.857 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:40:52.857 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-09-13T08:40:52.857 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-09-13T08:40:52.857 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-13T08:40:52.857 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-09-13T08:40:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 -- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f456c073220 msgr2=0x7f456c0756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f456c073220 0x7f456c0756e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f45800af000 tx=0x7f458001af20 comp rx=0 tx=0).stop 2024-09-13T08:40:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 -- 172.21.15.44:0/3106042533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f459015bd50 msgr2=0x7f459006dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f459015bd50 0x7f459006dd30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f457800e7a0 tx=0x7f457800ec60 comp rx=0 tx=0).stop 2024-09-13T08:40:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 -- 172.21.15.44:0/3106042533 shutdown_connections 2024-09-13T08:40:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f456c073220 0x7f456c0756e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f459015c130 0x7f45900c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f459015bd50 0x7f459006dd30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 --2- 172.21.15.44:0/3106042533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45900cf4e0 0x7f459014f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:52.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 -- 172.21.15.44:0/3106042533 >> 172.21.15.44:0/3106042533 conn(0x7f45900d1ea0 msgr2=0x7f45900d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:52.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 -- 172.21.15.44:0/3106042533 shutdown_connections 2024-09-13T08:40:52.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:52.856+0000 7f4596400640 1 -- 172.21.15.44:0/3106042533 wait complete. 2024-09-13T08:40:52.879 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:40:53.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.024+0000 7f00c6200640 1 -- 172.21.15.44:0/1901831326 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00c015bea0 msgr2=0x7f00c00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.024+0000 7f00c6200640 1 --2- 172.21.15.44:0/1901831326 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00c015bea0 0x7f00c00ce670 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009840 tx=0x7f00b001c820 comp rx=0 tx=0).stop 2024-09-13T08:40:53.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.024+0000 7f00c6200640 1 -- 172.21.15.44:0/1901831326 shutdown_connections 2024-09-13T08:40:53.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.024+0000 7f00c6200640 1 --2- 172.21.15.44:0/1901831326 >> v2:172.21.15.129:3300/0 conn(0x7f00c00d28e0 0x7f00c00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.024+0000 7f00c6200640 1 --2- 172.21.15.44:0/1901831326 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00c015bea0 0x7f00c00ce670 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.024+0000 7f00c6200640 1 -- 172.21.15.44:0/1901831326 >> 172.21.15.44:0/1901831326 conn(0x7f00c00d1dc0 msgr2=0x7f00c00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:53.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.024+0000 7f00c6200640 1 -- 172.21.15.44:0/1901831326 shutdown_connections 2024-09-13T08:40:53.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.024+0000 7f00c6200640 1 -- 172.21.15.44:0/1901831326 wait complete. 2024-09-13T08:40:53.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.024+0000 7f00c6200640 1 Processor -- start 2024-09-13T08:40:53.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.024+0000 7f00c6200640 1 -- start start 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00c6200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00c00cdb30 0x7f00c00cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00c6200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00c00d28e0 0x7f00c009bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00c6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00c015bea0 0x7f00c00ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00c6200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0168f10 con 0x7f00c015bea0 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00c6200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0168d90 con 0x7f00c00d28e0 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00c6200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f00c0169090 con 0x7f00c00cdb30 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00c015bea0 0x7f00c00ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00c015bea0 0x7f00c00ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45796/0 (socket says 172.21.15.44:45796) 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00bfe00640 1 -- 172.21.15.44:0/2763347718 learned_addr learned my addr 172.21.15.44:0/2763347718 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00b6a00640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00c00d28e0 0x7f00c009bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00bf400640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00c00cdb30 0x7f00c00cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00bfe00640 1 -- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00c00cdb30 msgr2=0x7f00c00cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00bfe00640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00c00cdb30 0x7f00c00cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00bfe00640 1 -- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00c00d28e0 msgr2=0x7f00c009bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00bfe00640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00c00d28e0 0x7f00c009bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00bfe00640 1 -- 172.21.15.44:0/2763347718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c00b4130 con 0x7f00c015bea0 2024-09-13T08:40:53.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00bfe00640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00c015bea0 0x7f00c00ca320 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009d30 tx=0x7f00b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:53.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00b7e00640 1 -- 172.21.15.44:0/2763347718 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8003b80 con 0x7f00c015bea0 2024-09-13T08:40:53.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00c6200640 1 -- 172.21.15.44:0/2763347718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c00b43f0 con 0x7f00c015bea0 2024-09-13T08:40:53.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00b7e00640 1 -- 172.21.15.44:0/2763347718 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00b8003d20 con 0x7f00c015bea0 2024-09-13T08:40:53.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00b7e00640 1 -- 172.21.15.44:0/2763347718 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8004280 con 0x7f00c015bea0 2024-09-13T08:40:53.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00c6200640 1 -- 172.21.15.44:0/2763347718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c00cb170 con 0x7f00c015bea0 2024-09-13T08:40:53.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.028+0000 7f00c6200640 1 -- 172.21.15.44:0/2763347718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c00cc9d0 con 0x7f00c015bea0 2024-09-13T08:40:53.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.032+0000 7f00b7e00640 1 -- 172.21.15.44:0/2763347718 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f00b8004420 con 0x7f00c015bea0 2024-09-13T08:40:53.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.032+0000 7f00b7e00640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f009c073220 0x7f009c0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.032+0000 7f00b7e00640 1 -- 172.21.15.44:0/2763347718 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f00b8095120 con 0x7f00c015bea0 2024-09-13T08:40:53.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.032+0000 7f00bf400640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f009c073220 0x7f009c0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.032+0000 7f00bf400640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f009c073220 0x7f009c0756e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f00b00af000 tx=0x7f00b001b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:53.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.036+0000 7f00b7e00640 1 -- 172.21.15.44:0/2763347718 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f00b805f710 con 0x7f00c015bea0 2024-09-13T08:40:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.248+0000 7f00c6200640 1 -- 172.21.15.44:0/2763347718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00c015a300 con 0x7f00c015bea0 2024-09-13T08:40:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00b7e00640 1 -- 172.21.15.44:0/2763347718 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7f00b8064140 con 0x7f00c015bea0 2024-09-13T08:40:53.254 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:40:53.254 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"btime":"2024-09-13T08:40:46:193076+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3367280951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3367280951},{"type":"v1","addr":"172.21.15.129:6835","nonce":3367280951}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1057940180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1057940180},{"type":"v1","addr":"172.21.15.129:6837","nonce":1057940180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5319,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1413618927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413618927},{"type":"v1","addr":"172.21.15.44:6834","nonce":1413618927}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:40:42.589651+0000","modified":"2024-09-13T08:40:45.596647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":4,"addr":"172.21.15.44:6835/3141962538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3141962538},{"type":"v1","addr":"172.21.15.44:6835","nonce":3141962538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5322,"qdb_cluster":[5322]},"id":3}]} 2024-09-13T08:40:53.254 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-09-13T08:40:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 -- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f009c073220 msgr2=0x7f009c0756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f009c073220 0x7f009c0756e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f00b00af000 tx=0x7f00b001b060 comp rx=0 tx=0).stop 2024-09-13T08:40:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 -- 172.21.15.44:0/2763347718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00c015bea0 msgr2=0x7f00c00ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00c015bea0 0x7f00c00ca320 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009d30 tx=0x7f00b800c6a0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 -- 172.21.15.44:0/2763347718 shutdown_connections 2024-09-13T08:40:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f009c073220 0x7f009c0756e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00c015bea0 0x7f00c00ca320 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00c00d28e0 0x7f00c009bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 --2- 172.21.15.44:0/2763347718 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00c00cdb30 0x7f00c00cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 -- 172.21.15.44:0/2763347718 >> 172.21.15.44:0/2763347718 conn(0x7f00c00d1dc0 msgr2=0x7f00c00af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:53.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 -- 172.21.15.44:0/2763347718 shutdown_connections 2024-09-13T08:40:53.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.252+0000 7f00c6200640 1 -- 172.21.15.44:0/2763347718 wait complete. 2024-09-13T08:40:53.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T08:40:42.589651+0000', 'modified': '2024-09-13T08:40:45.596647+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5322}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5322': {'gid': 5322, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.44:6835/3141962538', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 3141962538}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 3141962538}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5322, 'qdb_cluster': [5322]} 2024-09-13T08:40:53.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T08:40:53.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T08:40:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 -- 172.21.15.44:0/2386467782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f898415bd50 msgr2=0x7f89840ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 --2- 172.21.15.44:0/2386467782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f898415bd50 0x7f89840ce5e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f89840d0a60 tx=0x7f897401c780 comp rx=0 tx=0).stop 2024-09-13T08:40:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 -- 172.21.15.44:0/2386467782 shutdown_connections 2024-09-13T08:40:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 --2- 172.21.15.44:0/2386467782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f89840cf4e0 0x7f89840c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 --2- 172.21.15.44:0/2386467782 >> v2:172.21.15.129:3300/0 conn(0x7f898415c130 0x7f89840ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 --2- 172.21.15.44:0/2386467782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f898415bd50 0x7f89840ce5e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 -- 172.21.15.44:0/2386467782 >> 172.21.15.44:0/2386467782 conn(0x7f89840d1ea0 msgr2=0x7f89840d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 -- 172.21.15.44:0/2386467782 shutdown_connections 2024-09-13T08:40:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 -- 172.21.15.44:0/2386467782 wait complete. 2024-09-13T08:40:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 Processor -- start 2024-09-13T08:40:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 -- start start 2024-09-13T08:40:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89840cf4e0 0x7f89840cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898415bd50 0x7f89840b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f898415c130 0x7f89841593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f89840ccac0 con 0x7f898415c130 2024-09-13T08:40:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f89840cc940 con 0x7f89840cf4e0 2024-09-13T08:40:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.832+0000 7f8989e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f89840ccc40 con 0x7f898415bd50 2024-09-13T08:40:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8983400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f898415c130 0x7f89841593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8983400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f898415c130 0x7f89841593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45818/0 (socket says 172.21.15.44:45818) 2024-09-13T08:40:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8983400640 1 -- 172.21.15.44:0/4277994627 learned_addr learned my addr 172.21.15.44:0/4277994627 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f897a000640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898415bd50 0x7f89840b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8982a00640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89840cf4e0 0x7f89840cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8983400640 1 -- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898415bd50 msgr2=0x7f89840b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8983400640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898415bd50 0x7f89840b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8983400640 1 -- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89840cf4e0 msgr2=0x7f89840cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8983400640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89840cf4e0 0x7f89840cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8983400640 1 -- 172.21.15.44:0/4277994627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89840cf110 con 0x7f898415c130 2024-09-13T08:40:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8983400640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f898415c130 0x7f89841593d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f897c00eab0 tx=0x7f897c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f897be00640 1 -- 172.21.15.44:0/4277994627 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c004b30 con 0x7f898415c130 2024-09-13T08:40:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f897be00640 1 -- 172.21.15.44:0/4277994627 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f897c004dc0 con 0x7f898415c130 2024-09-13T08:40:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f897be00640 1 -- 172.21.15.44:0/4277994627 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c005280 con 0x7f898415c130 2024-09-13T08:40:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8989e00640 1 -- 172.21.15.44:0/4277994627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89840aed50 con 0x7f898415c130 2024-09-13T08:40:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f8989e00640 1 -- 172.21.15.44:0/4277994627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89840af310 con 0x7f898415c130 2024-09-13T08:40:53.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.836+0000 7f897be00640 1 -- 172.21.15.44:0/4277994627 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f897c005420 con 0x7f898415c130 2024-09-13T08:40:53.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897be00640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8960073220 0x7f89600756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897be00640 1 -- 172.21.15.44:0/4277994627 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f897c095130 con 0x7f898415c130 2024-09-13T08:40:53.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f898d2b6000 1 Processor -- start 2024-09-13T08:40:53.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f8982a00640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8960073220 0x7f89600756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f898d2b6000 1 -- start start 2024-09-13T08:40:53.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f8982a00640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8960073220 0x7f89600756e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f89740af000 tx=0x7f897401afc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:53.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26759d0 0x556ed2675db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed2629e10 0x556ed262a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f898d2b6000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556ed254a3f0 con 0x556ed26762f0 2024-09-13T08:40:53.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f898d2b6000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ed2487b20 con 0x556ed26759d0 2024-09-13T08:40:53.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f898d2b6000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x556ed2476f50 con 0x556ed2629e10 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26298d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45820/0 (socket says 172.21.15.44:45820) 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897a000640 1 -- 172.21.15.44:0/592443617 learned_addr learned my addr 172.21.15.44:0/592443617 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f8982a00640 1 --2- 172.21.15.44:0/592443617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26759d0 0x556ed2675db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f8983400640 1 --2- 172.21.15.44:0/592443617 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed2629e10 0x556ed262a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897a000640 1 -- 172.21.15.44:0/592443617 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed2629e10 msgr2=0x556ed262a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897a000640 1 --2- 172.21.15.44:0/592443617 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed2629e10 0x556ed262a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897a000640 1 -- 172.21.15.44:0/592443617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26759d0 msgr2=0x556ed2675db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897a000640 1 --2- 172.21.15.44:0/592443617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26759d0 0x556ed2675db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897a000640 1 -- 172.21.15.44:0/592443617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed243e590 con 0x556ed26762f0 2024-09-13T08:40:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f8982a00640 1 --2- 172.21.15.44:0/592443617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26759d0 0x556ed2675db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f897a000640 1 --2- 172.21.15.44:0/592443617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26298d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f896c009b90 tx=0x7f896c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2be5351efa97e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f8973e00640 1 -- 172.21.15.44:0/592443617 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c004050 con 0x556ed26762f0 2024-09-13T08:40:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f8973e00640 1 -- 172.21.15.44:0/592443617 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f896c0042e0 con 0x556ed26762f0 2024-09-13T08:40:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f8973e00640 1 -- 172.21.15.44:0/592443617 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c005280 con 0x556ed26762f0 2024-09-13T08:40:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f898d2b6000 1 -- 172.21.15.44:0/592443617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 msgr2=0x556ed26298d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.840+0000 7f898d2b6000 1 --2- 172.21.15.44:0/592443617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26298d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f896c009b90 tx=0x7f896c011810 comp rx=0 tx=0).stop 2024-09-13T08:40:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 -- 172.21.15.44:0/592443617 shutdown_connections 2024-09-13T08:40:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 --2- 172.21.15.44:0/592443617 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed2629e10 0x556ed262a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 --2- 172.21.15.44:0/592443617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26298d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 --2- 172.21.15.44:0/592443617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26759d0 0x556ed2675db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 -- 172.21.15.44:0/592443617 >> 172.21.15.44:0/592443617 conn(0x556ed2513130 msgr2=0x556ed2658e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 -- 172.21.15.44:0/592443617 shutdown_connections 2024-09-13T08:40:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 -- 172.21.15.44:0/592443617 wait complete. 2024-09-13T08:40:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 -- start start 2024-09-13T08:40:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 Processor -- start 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed2656e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26762f0 0x556ed26573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556ed24115d0 con 0x556ed2629e10 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ed247ad60 con 0x556ed26762f0 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x556ed241d500 con 0x556ed26759d0 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 10 client.0 _ll_register_callbacks cb 0x556ed2421810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8982a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8982a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45834/0 (socket says 172.21.15.44:45834) 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8982a00640 1 -- 172.21.15.44:0/2631932703 learned_addr learned my addr 172.21.15.44:0/2631932703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f897a000640 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed2656e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8983400640 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26762f0 0x556ed26573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8982a00640 1 -- 172.21.15.44:0/2631932703 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 msgr2=0x556ed2656e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8982a00640 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed2656e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8982a00640 1 -- 172.21.15.44:0/2631932703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26762f0 msgr2=0x556ed26573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8982a00640 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26762f0 0x556ed26573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8982a00640 1 -- 172.21.15.44:0/2631932703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed2415170 con 0x556ed2629e10 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8982a00640 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26566a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f89740096c0 tx=0x7f8974009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c3362c1128f146 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8970c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8970c00640 1 -- 172.21.15.44:0/2631932703 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89740c9060 con 0x556ed2629e10 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8970c00640 1 -- 172.21.15.44:0/2631932703 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f8974005440 con 0x556ed2629e10 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 -- 172.21.15.44:0/2631932703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ed240f0f0 con 0x556ed2629e10 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8970c00640 1 -- 172.21.15.44:0/2631932703 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89740b2040 con 0x556ed2629e10 2024-09-13T08:40:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 10 client.5487 fetch_fsmap learned FSMap version 27 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 1 -- 172.21.15.44:0/2631932703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x556ed24fbef0 con 0x556ed2629e10 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8970c00640 1 -- 172.21.15.44:0/2631932703 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556ed240f0f0 con 0x556ed2629e10 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f8970c00640 1 -- 172.21.15.44:0/2631932703 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f89740c3070 con 0x556ed2629e10 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.844+0000 7f898d2b6000 10 client.5487 fetch_fsmap finished waiting for FSMap version 27 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 10 client.5487 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 20 client.5487 populate_metadata read hostname 'smithi044' 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.44:6833/3141962538,v1:172.21.15.44:6835/3141962538] conn(0x556ed26580f0 0x556ed2687c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 4 client.5487 mds_command: new command op to 5322 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 -- 172.21.15.44:0/2631932703 --> [v2:172.21.15.44:6833/3141962538,v1:172.21.15.44:6835/3141962538] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ed2489a70 con 0x556ed26580f0 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f897a000640 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.44:6833/3141962538,v1:172.21.15.44:6835/3141962538] conn(0x556ed26580f0 0x556ed2687c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f897a000640 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.44:6833/3141962538,v1:172.21.15.44:6835/3141962538] conn(0x556ed26580f0 0x556ed2687c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e94b4315f31967cc server_cookie=edf4c1f5880477af in_seq=0 out_seq=0 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f8970c00640 10 client.5487 ms_handle_connect on v2:172.21.15.44:6833/3141962538 2024-09-13T08:40:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f8970c00640 1 -- 172.21.15.44:0/2631932703 <== mds.0 v2:172.21.15.44:6833/3141962538 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ed2489a70 con 0x556ed26580f0 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f8970c00640 10 client.5487 handle_command_reply: tid=0 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 client.5487 shutdown 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 -- 172.21.15.44:0/2631932703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 msgr2=0x556ed26566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26566a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f89740096c0 tx=0x7f8974009260 comp rx=0 tx=0).stop 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 -- 172.21.15.44:0/2631932703 shutdown_connections 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.44:6833/3141962538,v1:172.21.15.44:6835/3141962538] conn(0x556ed26580f0 0x556ed2687c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26762f0 0x556ed26573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed2656e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f8970c00640 0 client.5487 ms_handle_reset on v2:172.21.15.44:6833/3141962538 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 --2- 172.21.15.44:0/2631932703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26566a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 -- 172.21.15.44:0/2631932703 >> 172.21.15.44:0/2631932703 conn(0x556ed2513130 msgr2=0x556ed2628fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:53.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 -- 172.21.15.44:0/2631932703 shutdown_connections 2024-09-13T08:40:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 1 -- 172.21.15.44:0/2631932703 wait complete. 2024-09-13T08:40:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.848+0000 7f898d2b6000 20 client.5487 trim_cache size 0 max 16384 2024-09-13T08:40:53.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 Processor -- start 2024-09-13T08:40:53.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 -- start start 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed26e8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26762f0 0x556ed26e8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8984203c90 con 0x556ed2629e10 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8984202d50 con 0x556ed26762f0 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f898406e790 con 0x556ed26759d0 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8982a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8982a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26e7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45842/0 (socket says 172.21.15.44:45842) 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8982a00640 1 -- 172.21.15.44:0/1052066141 learned_addr learned my addr 172.21.15.44:0/1052066141 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f897a000640 1 --2- 172.21.15.44:0/1052066141 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed26e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8983400640 1 --2- 172.21.15.44:0/1052066141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26762f0 0x556ed26e8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8982a00640 1 -- 172.21.15.44:0/1052066141 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 msgr2=0x556ed26e8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:40:53.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8982a00640 1 --2- 172.21.15.44:0/1052066141 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed26e8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8982a00640 1 -- 172.21.15.44:0/1052066141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26762f0 msgr2=0x556ed26e8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:40:53.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8982a00640 1 --2- 172.21.15.44:0/1052066141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26762f0 0x556ed26e8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8982a00640 1 -- 172.21.15.44:0/1052066141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed24d2d90 con 0x556ed2629e10 2024-09-13T08:40:53.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8982a00640 1 --2- 172.21.15.44:0/1052066141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26e7dd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f897401c7c0 tx=0x7f89740047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50eb7c0a648c9423 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:53.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8973400640 1 -- 172.21.15.44:0/1052066141 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89740049a0 con 0x556ed2629e10 2024-09-13T08:40:53.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8973400640 1 -- 172.21.15.44:0/1052066141 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8974004c30 con 0x556ed2629e10 2024-09-13T08:40:53.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f8973400640 1 -- 172.21.15.44:0/1052066141 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89740a5620 con 0x556ed2629e10 2024-09-13T08:40:53.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 -- 172.21.15.44:0/1052066141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 msgr2=0x556ed26e7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 --2- 172.21.15.44:0/1052066141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26e7dd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f897401c7c0 tx=0x7f89740047e0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 -- 172.21.15.44:0/1052066141 shutdown_connections 2024-09-13T08:40:53.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 --2- 172.21.15.44:0/1052066141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed26762f0 0x556ed26e8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 --2- 172.21.15.44:0/1052066141 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed26e8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 --2- 172.21.15.44:0/1052066141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed2629e10 0x556ed26e7dd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 -- 172.21.15.44:0/1052066141 >> 172.21.15.44:0/1052066141 conn(0x556ed2513130 msgr2=0x556ed2631aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:53.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 -- 172.21.15.44:0/1052066141 shutdown_connections 2024-09-13T08:40:53.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.868+0000 7f898d2b6000 1 -- 172.21.15.44:0/1052066141 wait complete. 2024-09-13T08:40:53.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 -- start start 2024-09-13T08:40:53.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 Processor -- start 2024-09-13T08:40:53.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed2629e10 0x556ed26eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed26ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26ec470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556ed24fba60 con 0x556ed26762f0 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ed24868f0 con 0x556ed2629e10 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x556ed2536730 con 0x556ed26759d0 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 10 client.0 _ll_register_callbacks cb 0x7f896c005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8983400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8983400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26ec470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45850/0 (socket says 172.21.15.44:45850) 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8983400640 1 -- 172.21.15.44:0/1996257022 learned_addr learned my addr 172.21.15.44:0/1996257022 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8982a00640 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed2629e10 0x556ed26eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f897a000640 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed26ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8983400640 1 -- 172.21.15.44:0/1996257022 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 msgr2=0x556ed26ebf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:40:53.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8983400640 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed26ebf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8983400640 1 -- 172.21.15.44:0/1996257022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed2629e10 msgr2=0x556ed26eb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8983400640 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed2629e10 0x556ed26eb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8983400640 1 -- 172.21.15.44:0/1996257022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed243da90 con 0x556ed26762f0 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8983400640 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26ec470 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f897c000f90 tx=0x7f897c068e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93309f4f6d961e01 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8970c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8970c00640 1 -- 172.21.15.44:0/1996257022 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c01d040 con 0x556ed26762f0 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8970c00640 1 -- 172.21.15.44:0/1996257022 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f897c005920 con 0x556ed26762f0 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 -- 172.21.15.44:0/1996257022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8974004c30 con 0x556ed26762f0 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8970c00640 1 -- 172.21.15.44:0/1996257022 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f897c05ef80 con 0x556ed26762f0 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 10 client.5493 fetch_fsmap learned FSMap version 27 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 -- 172.21.15.44:0/1996257022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x556ed241c050 con 0x556ed26762f0 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8970c00640 1 -- 172.21.15.44:0/1996257022 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8974004c30 con 0x556ed26762f0 2024-09-13T08:40:53.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8970c00640 1 -- 172.21.15.44:0/1996257022 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f897c066440 con 0x556ed26762f0 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 10 client.5493 fetch_fsmap finished waiting for FSMap version 27 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 10 client.5493 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 20 client.5493 populate_metadata read hostname 'smithi044' 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.44:6833/3141962538,v1:172.21.15.44:6835/3141962538] conn(0x556ed268b260 0x556ed26f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 4 client.5493 mds_command: new command op to 5322 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f898d2b6000 1 -- 172.21.15.44:0/1996257022 --> [v2:172.21.15.44:6833/3141962538,v1:172.21.15.44:6835/3141962538] -- command(tid 0: {"prefix": "status"}) -- 0x556ed2485520 con 0x556ed268b260 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.872+0000 7f8982a00640 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.44:6833/3141962538,v1:172.21.15.44:6835/3141962538] conn(0x556ed268b260 0x556ed26f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8982a00640 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.44:6833/3141962538,v1:172.21.15.44:6835/3141962538] conn(0x556ed268b260 0x556ed26f4c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d79e6739629fb0c8 server_cookie=e507187b5597c19d in_seq=0 out_seq=0 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8970c00640 10 client.5493 ms_handle_connect on v2:172.21.15.44:6833/3141962538 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8970c00640 1 -- 172.21.15.44:0/1996257022 <== mds.0 v2:172.21.15.44:6833/3141962538 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x556ed2485520 con 0x556ed268b260 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8970c00640 10 client.5493 handle_command_reply: tid=0 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 client.5493 shutdown 2024-09-13T08:40:53.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 -- 172.21.15.44:0/1996257022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 msgr2=0x556ed26ec470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26ec470 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f897c000f90 tx=0x7f897c068e90 comp rx=0 tx=0).stop 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 -- 172.21.15.44:0/1996257022 shutdown_connections 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.44:6833/3141962538,v1:172.21.15.44:6835/3141962538] conn(0x556ed268b260 0x556ed26f4c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ed26762f0 0x556ed26ec470 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ed26759d0 0x556ed26ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8970c00640 0 client.5493 ms_handle_reset on v2:172.21.15.44:6833/3141962538 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 --2- 172.21.15.44:0/1996257022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ed2629e10 0x556ed26eb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 -- 172.21.15.44:0/1996257022 >> 172.21.15.44:0/1996257022 conn(0x556ed2513130 msgr2=0x556ed26745b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 -- 172.21.15.44:0/1996257022 shutdown_connections 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 1 -- 172.21.15.44:0/1996257022 wait complete. 2024-09-13T08:40:53.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f898d2b6000 20 client.5493 trim_cache size 0 max 16384 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 5322, 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 10.699413592000001, 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 47, 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 47, 2024-09-13T08:40:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 22.240145429999998 2024-09-13T08:40:53.884 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:40:53.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 -- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8960073220 msgr2=0x7f89600756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8960073220 0x7f89600756e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f89740af000 tx=0x7f897401afc0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 -- 172.21.15.44:0/4277994627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f898415c130 msgr2=0x7f89841593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f898415c130 0x7f89841593d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f897c00eab0 tx=0x7f897c00ef70 comp rx=0 tx=0).stop 2024-09-13T08:40:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 -- 172.21.15.44:0/4277994627 shutdown_connections 2024-09-13T08:40:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8960073220 0x7f89600756e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f898415c130 0x7f89841593d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898415bd50 0x7f89840b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 --2- 172.21.15.44:0/4277994627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89840cf4e0 0x7f89840cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 -- 172.21.15.44:0/4277994627 >> 172.21.15.44:0/4277994627 conn(0x7f89840d1ea0 msgr2=0x7f89840c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 -- 172.21.15.44:0/4277994627 shutdown_connections 2024-09-13T08:40:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:53.876+0000 7f8989e00640 1 -- 172.21.15.44:0/4277994627 wait complete. 2024-09-13T08:40:53.898 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:40:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.040+0000 7f56dc000640 1 -- 172.21.15.44:0/2422115381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56d415bd50 msgr2=0x7f56d40ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.040+0000 7f56dc000640 1 --2- 172.21.15.44:0/2422115381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56d415bd50 0x7f56d40ce5e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009900 tx=0x7f56c801c4a0 comp rx=0 tx=0).stop 2024-09-13T08:40:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 -- 172.21.15.44:0/2422115381 shutdown_connections 2024-09-13T08:40:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 --2- 172.21.15.44:0/2422115381 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56d40cf4e0 0x7f56d40cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 --2- 172.21.15.44:0/2422115381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56d415bd50 0x7f56d40ce5e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 -- 172.21.15.44:0/2422115381 >> 172.21.15.44:0/2422115381 conn(0x7f56d40d1ea0 msgr2=0x7f56d40d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 -- 172.21.15.44:0/2422115381 shutdown_connections 2024-09-13T08:40:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 -- 172.21.15.44:0/2422115381 wait complete. 2024-09-13T08:40:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 Processor -- start 2024-09-13T08:40:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 -- start start 2024-09-13T08:40:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56d40cf4e0 0x7f56d40b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56d415bd50 0x7f56d4152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d415c130 0x7f56d40c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4169b80 con 0x7f56d415bd50 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4169a00 con 0x7f56d415c130 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f56d4169d00 con 0x7f56d40cf4e0 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56d415bd50 0x7f56d4152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56d415bd50 0x7f56d4152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45868/0 (socket says 172.21.15.44:45868) 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d8c00640 1 -- 172.21.15.44:0/2216826506 learned_addr learned my addr 172.21.15.44:0/2216826506 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56da000640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d415c130 0x7f56d40c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d9600640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56d40cf4e0 0x7f56d40b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d8c00640 1 -- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56d40cf4e0 msgr2=0x7f56d40b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d8c00640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56d40cf4e0 0x7f56d40b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d8c00640 1 -- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d415c130 msgr2=0x7f56d40c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d8c00640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d415c130 0x7f56d40c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d8c00640 1 -- 172.21.15.44:0/2216826506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d4154830 con 0x7f56d415bd50 2024-09-13T08:40:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d8c00640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56d415bd50 0x7f56d4152700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f56c000c3c0 tx=0x7f56c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d1600640 1 -- 172.21.15.44:0/2216826506 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0017070 con 0x7f56d415bd50 2024-09-13T08:40:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d1600640 1 -- 172.21.15.44:0/2216826506 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56c00053c0 con 0x7f56d415bd50 2024-09-13T08:40:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56d1600640 1 -- 172.21.15.44:0/2216826506 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0003860 con 0x7f56d415bd50 2024-09-13T08:40:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.044+0000 7f56dc000640 1 -- 172.21.15.44:0/2216826506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d4154b10 con 0x7f56d415bd50 2024-09-13T08:40:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.048+0000 7f56dc000640 1 -- 172.21.15.44:0/2216826506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d41550d0 con 0x7f56d415bd50 2024-09-13T08:40:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.048+0000 7f56dc000640 1 -- 172.21.15.44:0/2216826506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d41555b0 con 0x7f56d415bd50 2024-09-13T08:40:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.048+0000 7f56d1600640 1 -- 172.21.15.44:0/2216826506 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f56c001a070 con 0x7f56d415bd50 2024-09-13T08:40:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.048+0000 7f56d1600640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56b4073220 0x7f56b40756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.048+0000 7f56d1600640 1 -- 172.21.15.44:0/2216826506 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f56c0093790 con 0x7f56d415bd50 2024-09-13T08:40:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.048+0000 7f56d9600640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56b4073220 0x7f56b40756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.052+0000 7f56d9600640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56b4073220 0x7f56b40756e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f56c80ae000 tx=0x7f56c801ace0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:54.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.052+0000 7f56d1600640 1 -- 172.21.15.44:0/2216826506 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f56d41555b0 con 0x7f56d415bd50 2024-09-13T08:40:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 -- 172.21.15.44:0/2216826506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f56d40cdca0 con 0x7f56d415bd50 2024-09-13T08:40:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56d1600640 1 -- 172.21.15.44:0/2216826506 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f56c005e4a0 con 0x7f56d415bd50 2024-09-13T08:40:54.271 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:40:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 -- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56b4073220 msgr2=0x7f56b40756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56b4073220 0x7f56b40756e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f56c80ae000 tx=0x7f56c801ace0 comp rx=0 tx=0).stop 2024-09-13T08:40:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 -- 172.21.15.44:0/2216826506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56d415bd50 msgr2=0x7f56d4152700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56d415bd50 0x7f56d4152700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f56c000c3c0 tx=0x7f56c000c880 comp rx=0 tx=0).stop 2024-09-13T08:40:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 -- 172.21.15.44:0/2216826506 shutdown_connections 2024-09-13T08:40:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f56b4073220 0x7f56b40756e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d415c130 0x7f56d40c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56d415bd50 0x7f56d4152700 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 --2- 172.21.15.44:0/2216826506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56d40cf4e0 0x7f56d40b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 -- 172.21.15.44:0/2216826506 >> 172.21.15.44:0/2216826506 conn(0x7f56d40d1ea0 msgr2=0x7f56d415a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 -- 172.21.15.44:0/2216826506 shutdown_connections 2024-09-13T08:40:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54.268+0000 7f56dc000640 1 -- 172.21.15.44:0/2216826506 wait complete. 2024-09-13T08:40:54.293 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T08:40:54.293 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:40:54.293 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:40:54.293 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T08:40:54.294 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:40:54.294 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T08:40:54.294 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T08:40:54.294 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:40:54.294 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:40:54.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:54.294 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:40:54.304 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:40:54.304 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:40:54.304 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:40:54.304 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:40:54.304 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:40:54.304 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:40:54.304 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:40:54.304 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42228sec preferred_lft 42228sec 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:40:54.305 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:40:54.306 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T08:40:54.307 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:40:54.307 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:40:54.307 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:40:54.307 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T08:40:54.307 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:40:54.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:54.307 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:40:54.363 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:40:54.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:54.366 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:40:54.419 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:40:54.419 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:40:54.419 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:40:54.419 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:40:54.419 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:40:54.419 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:40:54.419 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:40:54.419 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:40:54.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:40:54.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:40:54.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:40:54.420 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:40:54.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:40:54.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:40:54.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:54.557 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:40:54.566 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:40:54.566 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T08:40:54.566 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:40:54.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:54.566 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T08:40:54.633 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:40:54.633 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T08:40:54.685 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:40:54.686 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:40:54.686 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:40:54.741 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:40:54.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:54.741 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:40:54.830 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:40:54.830 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:40:54.830 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:40:54.845 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:40:54.846 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:40:54.846 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:40:54.846 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:40:54.846 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:40:54.855 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:40:54.856 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:40:54.856 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:40:54.856 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:40:54.856 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:40:54.856 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:40:54.856 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:40:54.856 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:40:54.860 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:40:54.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:54.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:40:54.880 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:40:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:40:54.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:40:54.984 INFO:teuthology.orchestra.run.smithi044.stdout:client5502 2024-09-13T08:40:54.984 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:40:54.985 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:54.994 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-13T08:40:55.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.188+0000 7f0012e00640 1 -- 172.21.15.44:0/1935709436 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f000c0b0b60 msgr2=0x7f000c0ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:55.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.188+0000 7f0012e00640 1 --2- 172.21.15.44:0/1935709436 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f000c0b0b60 0x7f000c0ccbd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0004009c80 tx=0x7f000401cda0 comp rx=0 tx=0).stop 2024-09-13T08:40:55.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.188+0000 7f0012e00640 1 -- 172.21.15.44:0/1935709436 shutdown_connections 2024-09-13T08:40:55.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.188+0000 7f0012e00640 1 --2- 172.21.15.44:0/1935709436 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f000c0b0b60 0x7f000c0ccbd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:55.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.188+0000 7f0012e00640 1 --2- 172.21.15.44:0/1935709436 >> v2:172.21.15.129:3300/0 conn(0x7f000c15c1f0 0x7f000c0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:55.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.188+0000 7f0012e00640 1 -- 172.21.15.44:0/1935709436 >> 172.21.15.44:0/1935709436 conn(0x7f000c0d1da0 msgr2=0x7f000c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:55.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.188+0000 7f0012e00640 1 -- 172.21.15.44:0/1935709436 shutdown_connections 2024-09-13T08:40:55.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.188+0000 7f0012e00640 1 -- 172.21.15.44:0/1935709436 wait complete. 2024-09-13T08:40:55.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 Processor -- start 2024-09-13T08:40:55.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 -- start start 2024-09-13T08:40:55.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f000c0b0b60 0x7f000c1495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000c15be10 0x7f000c0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000c15c1f0 0x7f000c0af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f000c169040 con 0x7f000c15be10 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f000c168ec0 con 0x7f000c15c1f0 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f000c1691c0 con 0x7f000c0b0b60 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000c15be10 0x7f000c0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000c15be10 0x7f000c0af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45876/0 (socket says 172.21.15.44:45876) 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000b400640 1 -- 172.21.15.44:0/2840510569 learned_addr learned my addr 172.21.15.44:0/2840510569 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0010e00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000c15c1f0 0x7f000c0af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000be00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f000c0b0b60 0x7f000c1495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000b400640 1 -- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f000c0b0b60 msgr2=0x7f000c1495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000b400640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f000c0b0b60 0x7f000c1495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000b400640 1 -- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000c15c1f0 msgr2=0x7f000c0af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000b400640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000c15c1f0 0x7f000c0af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000b400640 1 -- 172.21.15.44:0/2840510569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c1519f0 con 0x7f000c15be10 2024-09-13T08:40:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000be00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f000c0b0b60 0x7f000c1495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:55.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f000b400640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000c15be10 0x7f000c0af4b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efff800e970 tx=0x7efff800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0003e00640 1 -- 172.21.15.44:0/2840510569 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff800cd80 con 0x7f000c15be10 2024-09-13T08:40:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0003e00640 1 -- 172.21.15.44:0/2840510569 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efff8004020 con 0x7f000c15be10 2024-09-13T08:40:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 -- 172.21.15.44:0/2840510569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c151cd0 con 0x7f000c15be10 2024-09-13T08:40:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0003e00640 1 -- 172.21.15.44:0/2840510569 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8004470 con 0x7f000c15be10 2024-09-13T08:40:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 -- 172.21.15.44:0/2840510569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c06dd30 con 0x7f000c15be10 2024-09-13T08:40:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.192+0000 7f0012e00640 1 -- 172.21.15.44:0/2840510569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c0ccbd0 con 0x7f000c15be10 2024-09-13T08:40:55.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.196+0000 7f0003e00640 1 -- 172.21.15.44:0/2840510569 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7efff8003600 con 0x7f000c15be10 2024-09-13T08:40:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.196+0000 7f0003e00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7effec073220 0x7effec0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.196+0000 7f0003e00640 1 -- 172.21.15.44:0/2840510569 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7efff8094e90 con 0x7f000c15be10 2024-09-13T08:40:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.196+0000 7f000be00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7effec073220 0x7effec0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.196+0000 7f000be00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7effec073220 0x7effec0756e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efffc000f70 tx=0x7efffc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:55.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.200+0000 7f0003e00640 1 -- 172.21.15.44:0/2840510569 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7efff805fa80 con 0x7f000c15be10 2024-09-13T08:40:55.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:55.416+0000 7f0012e00640 1 -- 172.21.15.44:0/2840510569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f000c0b3f10 con 0x7f000c15be10 2024-09-13T08:40:56.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.232+0000 7f0003e00640 1 -- 172.21.15.44:0/2840510569 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v28) ==== 136+0+0 (secure 0 0 0) 0x7efff805f380 con 0x7f000c15be10 2024-09-13T08:40:56.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 -- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7effec073220 msgr2=0x7effec0756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7effec073220 0x7effec0756e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efffc000f70 tx=0x7efffc009040 comp rx=0 tx=0).stop 2024-09-13T08:40:56.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 -- 172.21.15.44:0/2840510569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000c15be10 msgr2=0x7f000c0af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000c15be10 0x7f000c0af4b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efff800e970 tx=0x7efff800ee30 comp rx=0 tx=0).stop 2024-09-13T08:40:56.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 -- 172.21.15.44:0/2840510569 shutdown_connections 2024-09-13T08:40:56.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7effec073220 0x7effec0756e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000c15c1f0 0x7f000c0af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000c15be10 0x7f000c0af4b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 --2- 172.21.15.44:0/2840510569 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f000c0b0b60 0x7f000c1495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 -- 172.21.15.44:0/2840510569 >> 172.21.15.44:0/2840510569 conn(0x7f000c0d1da0 msgr2=0x7f000c0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:56.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 -- 172.21.15.44:0/2840510569 shutdown_connections 2024-09-13T08:40:56.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.236+0000 7f0012e00640 1 -- 172.21.15.44:0/2840510569 wait complete. 2024-09-13T08:40:56.263 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:40:56.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 -- 172.21.15.44:0/897867145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea941617b0 msgr2=0x7fea940d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 --2- 172.21.15.44:0/897867145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea941617b0 0x7fea940d0c50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fea80009900 tx=0x7fea8001c870 comp rx=0 tx=0).stop 2024-09-13T08:40:56.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 -- 172.21.15.44:0/897867145 shutdown_connections 2024-09-13T08:40:56.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 --2- 172.21.15.44:0/897867145 >> v2:172.21.15.129:3300/0 conn(0x7fea940d29e0 0x7fea940b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 --2- 172.21.15.44:0/897867145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea941617b0 0x7fea940d0c50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 -- 172.21.15.44:0/897867145 >> 172.21.15.44:0/897867145 conn(0x7fea940d3e00 msgr2=0x7fea940d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:56.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 -- 172.21.15.44:0/897867145 shutdown_connections 2024-09-13T08:40:56.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 -- 172.21.15.44:0/897867145 wait complete. 2024-09-13T08:40:56.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 Processor -- start 2024-09-13T08:40:56.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 -- start start 2024-09-13T08:40:56.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea940d29e0 0x7fea94152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:56.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea941613d0 0x7fea940c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:56.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea941617b0 0x7fea94156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:56.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fea9416acc0 con 0x7fea941613d0 2024-09-13T08:40:56.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fea940b81e0 con 0x7fea940d29e0 2024-09-13T08:40:56.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.412+0000 7fea9a200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fea9416ae40 con 0x7fea941617b0 2024-09-13T08:40:56.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea93e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea941613d0 0x7fea940c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:56.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea93e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea941613d0 0x7fea940c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45900/0 (socket says 172.21.15.44:45900) 2024-09-13T08:40:56.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea93e00640 1 -- 172.21.15.44:0/3457226267 learned_addr learned my addr 172.21.15.44:0/3457226267 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:56.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea98e00640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea940d29e0 0x7fea94152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:56.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea99800640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea941617b0 0x7fea94156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:56.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea93e00640 1 -- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea941617b0 msgr2=0x7fea94156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea93e00640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea941617b0 0x7fea94156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea93e00640 1 -- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea940d29e0 msgr2=0x7fea94152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea93e00640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea940d29e0 0x7fea94152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea93e00640 1 -- 172.21.15.44:0/3457226267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9406e4c0 con 0x7fea941613d0 2024-09-13T08:40:56.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea98e00640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea940d29e0 0x7fea94152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:56.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea93e00640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea941613d0 0x7fea940c7790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fea800b0040 tx=0x7fea80005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:56.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea90c00640 1 -- 172.21.15.44:0/3457226267 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800b2040 con 0x7fea941613d0 2024-09-13T08:40:56.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea90c00640 1 -- 172.21.15.44:0/3457226267 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea80005230 con 0x7fea941613d0 2024-09-13T08:40:56.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea9a200640 1 -- 172.21.15.44:0/3457226267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea94153090 con 0x7fea941613d0 2024-09-13T08:40:56.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea90c00640 1 -- 172.21.15.44:0/3457226267 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800046d0 con 0x7fea941613d0 2024-09-13T08:40:56.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea9a200640 1 -- 172.21.15.44:0/3457226267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea940c8370 con 0x7fea941613d0 2024-09-13T08:40:56.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea9a200640 1 -- 172.21.15.44:0/3457226267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea940b6f80 con 0x7fea941613d0 2024-09-13T08:40:56.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.416+0000 7fea90c00640 1 -- 172.21.15.44:0/3457226267 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7fea80004870 con 0x7fea941613d0 2024-09-13T08:40:56.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.420+0000 7fea90c00640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fea74073220 0x7fea740756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:56.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.420+0000 7fea90c00640 1 -- 172.21.15.44:0/3457226267 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fea8012eae0 con 0x7fea941613d0 2024-09-13T08:40:56.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.420+0000 7fea98e00640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fea74073220 0x7fea740756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:56.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.420+0000 7fea98e00640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fea74073220 0x7fea740756e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea88001750 tx=0x7fea88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:56.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.424+0000 7fea90c00640 1 -- 172.21.15.44:0/3457226267 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fea800c4020 con 0x7fea941613d0 2024-09-13T08:40:56.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.620+0000 7fea9a200640 1 -- 172.21.15.44:0/3457226267 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fea9415e3c0 con 0x7fea74073220 2024-09-13T08:40:56.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea90c00640 1 -- 172.21.15.44:0/3457226267 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fea9415e3c0 con 0x7fea74073220 2024-09-13T08:40:56.626 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:40:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:40:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:40:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:40:56.626 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:40:56.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 -- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fea74073220 msgr2=0x7fea740756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fea74073220 0x7fea740756e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea88001750 tx=0x7fea88009040 comp rx=0 tx=0).stop 2024-09-13T08:40:56.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 -- 172.21.15.44:0/3457226267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea941613d0 msgr2=0x7fea940c7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea941613d0 0x7fea940c7790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fea800b0040 tx=0x7fea80005de0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 -- 172.21.15.44:0/3457226267 shutdown_connections 2024-09-13T08:40:56.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fea74073220 0x7fea740756e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fea941617b0 0x7fea94156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea941613d0 0x7fea940c7790 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 --2- 172.21.15.44:0/3457226267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea940d29e0 0x7fea94152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 -- 172.21.15.44:0/3457226267 >> 172.21.15.44:0/3457226267 conn(0x7fea940d3e00 msgr2=0x7fea9406dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:56.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 -- 172.21.15.44:0/3457226267 shutdown_connections 2024-09-13T08:40:56.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.624+0000 7fea9a200640 1 -- 172.21.15.44:0/3457226267 wait complete. 2024-09-13T08:40:56.649 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:40:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.796+0000 7f069c200640 1 -- 172.21.15.44:0/2475968467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f069415dae0 msgr2=0x7f06940acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.796+0000 7f069c200640 1 --2- 172.21.15.44:0/2475968467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f069415dae0 0x7f06940acd40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0688009840 tx=0x7f068801c790 comp rx=0 tx=0).stop 2024-09-13T08:40:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.796+0000 7f069c200640 1 -- 172.21.15.44:0/2475968467 shutdown_connections 2024-09-13T08:40:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.796+0000 7f069c200640 1 --2- 172.21.15.44:0/2475968467 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06940cd100 0x7f06940b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.796+0000 7f069c200640 1 --2- 172.21.15.44:0/2475968467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f069415dae0 0x7f06940acd40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.796+0000 7f069c200640 1 -- 172.21.15.44:0/2475968467 >> 172.21.15.44:0/2475968467 conn(0x7f06940b18f0 msgr2=0x7f06940b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.796+0000 7f069c200640 1 -- 172.21.15.44:0/2475968467 shutdown_connections 2024-09-13T08:40:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.796+0000 7f069c200640 1 -- 172.21.15.44:0/2475968467 wait complete. 2024-09-13T08:40:56.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.796+0000 7f069c200640 1 Processor -- start 2024-09-13T08:40:56.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.796+0000 7f069c200640 1 -- start start 2024-09-13T08:40:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069c200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06940b4810 0x7f069409d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069c200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06940cd100 0x7f06941435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069c200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f069415dae0 0x7f06940b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069c200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06941674b0 con 0x7f06940cd100 2024-09-13T08:40:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069c200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0694167330 con 0x7f069415dae0 2024-09-13T08:40:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069c200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0694167630 con 0x7f06940b4810 2024-09-13T08:40:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0698e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06940cd100 0x7f06941435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0698e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06940cd100 0x7f06941435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45910/0 (socket says 172.21.15.44:45910) 2024-09-13T08:40:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0698e00640 1 -- 172.21.15.44:0/1971814465 learned_addr learned my addr 172.21.15.44:0/1971814465 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0699800640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06940b4810 0x7f069409d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069a200640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f069415dae0 0x7f06940b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0698e00640 1 -- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06940b4810 msgr2=0x7f069409d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0698e00640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06940b4810 0x7f069409d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0698e00640 1 -- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f069415dae0 msgr2=0x7f06940b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0698e00640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f069415dae0 0x7f06940b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0698e00640 1 -- 172.21.15.44:0/1971814465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069409af40 con 0x7f06940cd100 2024-09-13T08:40:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069a200640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f069415dae0 0x7f06940b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:40:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0698e00640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06940cd100 0x7f06941435b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f068000ead0 tx=0x7f068000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:56.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0691600640 1 -- 172.21.15.44:0/1971814465 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068001b070 con 0x7f06940cd100 2024-09-13T08:40:56.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0691600640 1 -- 172.21.15.44:0/1971814465 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0680016410 con 0x7f06940cd100 2024-09-13T08:40:56.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f0691600640 1 -- 172.21.15.44:0/1971814465 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680016860 con 0x7f06940cd100 2024-09-13T08:40:56.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069c200640 1 -- 172.21.15.44:0/1971814465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069409b220 con 0x7f06940cd100 2024-09-13T08:40:56.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069c200640 1 -- 172.21.15.44:0/1971814465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069409b7e0 con 0x7f06940cd100 2024-09-13T08:40:56.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.800+0000 7f069c200640 1 -- 172.21.15.44:0/1971814465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0694069410 con 0x7f06940cd100 2024-09-13T08:40:56.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.804+0000 7f0691600640 1 -- 172.21.15.44:0/1971814465 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f068001e070 con 0x7f06940cd100 2024-09-13T08:40:56.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.804+0000 7f0691600640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0674073220 0x7f06740756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:56.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.804+0000 7f0691600640 1 -- 172.21.15.44:0/1971814465 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f0680094690 con 0x7f06940cd100 2024-09-13T08:40:56.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.804+0000 7f0699800640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0674073220 0x7f06740756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:56.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.804+0000 7f0699800640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0674073220 0x7f06740756e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f06880ae000 tx=0x7f068801afd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:56.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.808+0000 7f0691600640 1 -- 172.21.15.44:0/1971814465 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f068005ed30 con 0x7f06940cd100 2024-09-13T08:40:56.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.984+0000 7f069c200640 1 -- 172.21.15.44:0/1971814465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f06940d0090 con 0x7f06940cd100 2024-09-13T08:40:56.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f0691600640 1 -- 172.21.15.44:0/1971814465 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7f0680063760 con 0x7f06940cd100 2024-09-13T08:40:56.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 -- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0674073220 msgr2=0x7f06740756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0674073220 0x7f06740756e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f06880ae000 tx=0x7f068801afd0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 -- 172.21.15.44:0/1971814465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06940cd100 msgr2=0x7f06941435b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:56.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06940cd100 0x7f06941435b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f068000ead0 tx=0x7f068000ef90 comp rx=0 tx=0).stop 2024-09-13T08:40:56.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 -- 172.21.15.44:0/1971814465 shutdown_connections 2024-09-13T08:40:56.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0674073220 0x7f06740756e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f069415dae0 0x7f06940b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06940cd100 0x7f06941435b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 --2- 172.21.15.44:0/1971814465 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06940b4810 0x7f069409d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:56.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 -- 172.21.15.44:0/1971814465 >> 172.21.15.44:0/1971814465 conn(0x7f06940b18f0 msgr2=0x7f06940aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:56.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 -- 172.21.15.44:0/1971814465 shutdown_connections 2024-09-13T08:40:56.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:56.988+0000 7f069c200640 1 -- 172.21.15.44:0/1971814465 wait complete. 2024-09-13T08:40:57.013 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-13T08:40:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.160+0000 7f86eee00640 1 -- 172.21.15.44:0/180012127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0004d50 msgr2=0x7f86d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.160+0000 7f86eee00640 1 --2- 172.21.15.44:0/180012127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0004d50 0x7f86d00051d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f86e80d26d0 tx=0x7f86e001d280 comp rx=0 tx=0).stop 2024-09-13T08:40:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.160+0000 7f86ee400640 1 -- 172.21.15.44:0/180012127 reap_dead start 2024-09-13T08:40:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.160+0000 7f86eee00640 1 -- 172.21.15.44:0/180012127 shutdown_connections 2024-09-13T08:40:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.160+0000 7f86eee00640 1 -- 172.21.15.44:0/180012127 >> 172.21.15.44:0/180012127 conn(0x7f86e809aa10 msgr2=0x7f86e809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.160+0000 7f86eee00640 1 -- 172.21.15.44:0/180012127 shutdown_connections 2024-09-13T08:40:57.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.160+0000 7f86eee00640 1 -- 172.21.15.44:0/180012127 wait complete. 2024-09-13T08:40:57.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.160+0000 7f86eee00640 1 Processor -- start 2024-09-13T08:40:57.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.160+0000 7f86eee00640 1 -- start start 2024-09-13T08:40:57.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86eee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86d0004d50 0x7f86e80c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86eee00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86e81f9800 0x7f86e81f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86eee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e80c7880 0x7f86e80ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86eee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86e80ccd30 con 0x7f86e80c7880 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86eee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86e80d0880 con 0x7f86d0004d50 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86eee00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f86e80cceb0 con 0x7f86e81f9800 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86ee400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e80c7880 0x7f86e80ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86eda00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86d0004d50 0x7f86e80c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86ee400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e80c7880 0x7f86e80ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45926/0 (socket says 172.21.15.44:45926) 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86ee400640 1 -- 172.21.15.44:0/3801139935 learned_addr learned my addr 172.21.15.44:0/3801139935 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86ed000640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86e81f9800 0x7f86e81f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86ee400640 1 -- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86e81f9800 msgr2=0x7f86e81f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86ee400640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86e81f9800 0x7f86e81f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86ee400640 1 -- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86d0004d50 msgr2=0x7f86e80c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86ee400640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86d0004d50 0x7f86e80c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86ee400640 1 -- 172.21.15.44:0/3801139935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e80c8120 con 0x7f86e80c7880 2024-09-13T08:40:57.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86ee400640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e80c7880 0x7f86e80ae830 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f86e0098e50 tx=0x7f86e0006610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:57.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86e5600640 1 -- 172.21.15.44:0/3801139935 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e000c280 con 0x7f86e80c7880 2024-09-13T08:40:57.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86e5600640 1 -- 172.21.15.44:0/3801139935 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86e0006980 con 0x7f86e80c7880 2024-09-13T08:40:57.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86eee00640 1 -- 172.21.15.44:0/3801139935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e80c5560 con 0x7f86e80c7880 2024-09-13T08:40:57.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86e5600640 1 -- 172.21.15.44:0/3801139935 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0002090 con 0x7f86e80c7880 2024-09-13T08:40:57.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86eee00640 1 -- 172.21.15.44:0/3801139935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e80c5b20 con 0x7f86e80c7880 2024-09-13T08:40:57.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.164+0000 7f86eee00640 1 -- 172.21.15.44:0/3801139935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e80c5cf0 con 0x7f86e80c7880 2024-09-13T08:40:57.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.168+0000 7f86e5600640 1 -- 172.21.15.44:0/3801139935 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f86e00ac020 con 0x7f86e80c7880 2024-09-13T08:40:57.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.168+0000 7f86e5600640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86c8073220 0x7f86c80756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.168+0000 7f86e5600640 1 -- 172.21.15.44:0/3801139935 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f86e012e2e0 con 0x7f86e80c7880 2024-09-13T08:40:57.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.168+0000 7f86eda00640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86c8073220 0x7f86c80756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.168+0000 7f86eda00640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86c8073220 0x7f86c80756e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86e80afc80 tx=0x7f86dc0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:57.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.172+0000 7f86e5600640 1 -- 172.21.15.44:0/3801139935 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f86e00bf060 con 0x7f86e80c7880 2024-09-13T08:40:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.348+0000 7f86eee00640 1 -- 172.21.15.44:0/3801139935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f86e80cdc00 con 0x7f86e80c7880 2024-09-13T08:40:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.348+0000 7f86e5600640 1 -- 172.21.15.44:0/3801139935 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v31) ==== 116+0+2 (secure 0 0 0) 0x7f86e0017030 con 0x7f86e80c7880 2024-09-13T08:40:57.354 INFO:teuthology.orchestra.run.smithi044.stdout:4 2024-09-13T08:40:57.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 -- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86c8073220 msgr2=0x7f86c80756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86c8073220 0x7f86c80756e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86e80afc80 tx=0x7f86dc0091e0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 -- 172.21.15.44:0/3801139935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e80c7880 msgr2=0x7f86e80ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e80c7880 0x7f86e80ae830 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f86e0098e50 tx=0x7f86e0006610 comp rx=0 tx=0).stop 2024-09-13T08:40:57.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 -- 172.21.15.44:0/3801139935 shutdown_connections 2024-09-13T08:40:57.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86c8073220 0x7f86c80756e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e80c7880 0x7f86e80ae830 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86e81f9800 0x7f86e81f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 --2- 172.21.15.44:0/3801139935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86d0004d50 0x7f86e80c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 -- 172.21.15.44:0/3801139935 >> 172.21.15.44:0/3801139935 conn(0x7f86e809aa10 msgr2=0x7f86e80c85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:57.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 -- 172.21.15.44:0/3801139935 shutdown_connections 2024-09-13T08:40:57.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.352+0000 7f86eee00640 1 -- 172.21.15.44:0/3801139935 wait complete. 2024-09-13T08:40:57.376 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:40:57.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 -- 172.21.15.44:0/4068137360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28901605d0 msgr2=0x7f28900d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 --2- 172.21.15.44:0/4068137360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28901605d0 0x7f28900d0690 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2884009840 tx=0x7f288401c820 comp rx=0 tx=0).stop 2024-09-13T08:40:57.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 -- 172.21.15.44:0/4068137360 shutdown_connections 2024-09-13T08:40:57.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 --2- 172.21.15.44:0/4068137360 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28900afb10 0x7f28900affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 --2- 172.21.15.44:0/4068137360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28901605d0 0x7f28900d0690 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 -- 172.21.15.44:0/4068137360 >> 172.21.15.44:0/4068137360 conn(0x7f28900d1dd0 msgr2=0x7f28900d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:57.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 -- 172.21.15.44:0/4068137360 shutdown_connections 2024-09-13T08:40:57.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 -- 172.21.15.44:0/4068137360 wait complete. 2024-09-13T08:40:57.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 Processor -- start 2024-09-13T08:40:57.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 -- start start 2024-09-13T08:40:57.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28900afb10 0x7f2890158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28900b1650 0x7f289014b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28901605d0 0x7f28900b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28900b13a0 con 0x7f28900afb10 2024-09-13T08:40:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f28900b1220 con 0x7f28901605d0 2024-09-13T08:40:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2898400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2890169540 con 0x7f28900b1650 2024-09-13T08:40:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2895a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28900afb10 0x7f2890158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2895a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28900afb10 0x7f2890158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45954/0 (socket says 172.21.15.44:45954) 2024-09-13T08:40:57.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2895a00640 1 -- 172.21.15.44:0/268953514 learned_addr learned my addr 172.21.15.44:0/268953514 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:57.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.524+0000 7f2895000640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28900b1650 0x7f289014b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f2896400640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28901605d0 0x7f28900b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f2895a00640 1 -- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28900b1650 msgr2=0x7f289014b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f2895a00640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28900b1650 0x7f289014b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f2895a00640 1 -- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28901605d0 msgr2=0x7f28900b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f2895a00640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28901605d0 0x7f28900b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f2895a00640 1 -- 172.21.15.44:0/268953514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28901fb3d0 con 0x7f28900afb10 2024-09-13T08:40:57.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f2895a00640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28900afb10 0x7f2890158600 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2884009790 tx=0x7f28840a6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:57.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f288d600640 1 -- 172.21.15.44:0/268953514 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884004100 con 0x7f28900afb10 2024-09-13T08:40:57.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f288d600640 1 -- 172.21.15.44:0/268953514 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2884004390 con 0x7f28900afb10 2024-09-13T08:40:57.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f2898400640 1 -- 172.21.15.44:0/268953514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28901fb6b0 con 0x7f28900afb10 2024-09-13T08:40:57.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f288d600640 1 -- 172.21.15.44:0/268953514 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28840052d0 con 0x7f28900afb10 2024-09-13T08:40:57.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f2898400640 1 -- 172.21.15.44:0/268953514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28901fbc70 con 0x7f28900afb10 2024-09-13T08:40:57.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f2898400640 1 -- 172.21.15.44:0/268953514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28900affc0 con 0x7f28900afb10 2024-09-13T08:40:57.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f288d600640 1 -- 172.21.15.44:0/268953514 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f28840be020 con 0x7f28900afb10 2024-09-13T08:40:57.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.528+0000 7f288d600640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2870073220 0x7f28700756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.532+0000 7f288d600640 1 -- 172.21.15.44:0/268953514 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f288412eca0 con 0x7f28900afb10 2024-09-13T08:40:57.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.532+0000 7f2895000640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2870073220 0x7f28700756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.532+0000 7f2895000640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2870073220 0x7f28700756e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f28901fb000 tx=0x7f287c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:57.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.532+0000 7f288d600640 1 -- 172.21.15.44:0/268953514 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f28840c5020 con 0x7f28900afb10 2024-09-13T08:40:57.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.712+0000 7f2898400640 1 -- 172.21.15.44:0/268953514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f28900d1080 con 0x7f28900afb10 2024-09-13T08:40:57.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f288d600640 1 -- 172.21.15.44:0/268953514 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v32)=0 v32) ==== 131+0+0 (secure 0 0 0) 0x7f2884016410 con 0x7f28900afb10 2024-09-13T08:40:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 -- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2870073220 msgr2=0x7f28700756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2870073220 0x7f28700756e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f28901fb000 tx=0x7f287c009040 comp rx=0 tx=0).stop 2024-09-13T08:40:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 -- 172.21.15.44:0/268953514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28900afb10 msgr2=0x7f2890158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28900afb10 0x7f2890158600 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2884009790 tx=0x7f28840a6600 comp rx=0 tx=0).stop 2024-09-13T08:40:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 -- 172.21.15.44:0/268953514 shutdown_connections 2024-09-13T08:40:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2870073220 0x7f28700756e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28901605d0 0x7f28900b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28900b1650 0x7f289014b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 --2- 172.21.15.44:0/268953514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28900afb10 0x7f2890158600 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 -- 172.21.15.44:0/268953514 >> 172.21.15.44:0/268953514 conn(0x7f28900d1dd0 msgr2=0x7f2890070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 -- 172.21.15.44:0/268953514 shutdown_connections 2024-09-13T08:40:57.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.716+0000 7f2898400640 1 -- 172.21.15.44:0/268953514 wait complete. 2024-09-13T08:40:57.734 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:57.728+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:57.734 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:57.728+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:57.734 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:57.728+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:57.734 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:57.728+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:57.734 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:57.728+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:57.740 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-13T08:40:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.888+0000 7f0bd8600640 1 -- 172.21.15.44:0/1757466693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd015b000 msgr2=0x7f0bd015e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.888+0000 7f0bd8600640 1 --2- 172.21.15.44:0/1757466693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd015b000 0x7f0bd015e580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0076610 tx=0x7f0bc801dc70 comp rx=0 tx=0).stop 2024-09-13T08:40:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.888+0000 7f0bd8600640 1 -- 172.21.15.44:0/1757466693 shutdown_connections 2024-09-13T08:40:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.888+0000 7f0bd8600640 1 --2- 172.21.15.44:0/1757466693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd015b000 0x7f0bd015e580 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.888+0000 7f0bd8600640 1 --2- 172.21.15.44:0/1757466693 >> v2:172.21.15.129:3300/0 conn(0x7f0bd0070960 0x7f0bd006fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.888+0000 7f0bd8600640 1 -- 172.21.15.44:0/1757466693 >> 172.21.15.44:0/1757466693 conn(0x7f0bd00cfbc0 msgr2=0x7f0bd00cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.888+0000 7f0bd8600640 1 -- 172.21.15.44:0/1757466693 shutdown_connections 2024-09-13T08:40:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.888+0000 7f0bd8600640 1 -- 172.21.15.44:0/1757466693 wait complete. 2024-09-13T08:40:57.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.888+0000 7f0bd8600640 1 Processor -- start 2024-09-13T08:40:57.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.888+0000 7f0bd8600640 1 -- start start 2024-09-13T08:40:57.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd8600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bd0070960 0x7f0bd01f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd8600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd00cf2b0 0x7f0bd0155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd8600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd015b000 0x7f0bd00b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd8600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0167390 con 0x7f0bd00cf2b0 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd8600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0167210 con 0x7f0bd015b000 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd8600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd0167510 con 0x7f0bd0070960 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd00cf2b0 0x7f0bd0155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd00cf2b0 0x7f0bd0155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45972/0 (socket says 172.21.15.44:45972) 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd5200640 1 -- 172.21.15.44:0/2743781050 learned_addr learned my addr 172.21.15.44:0/2743781050 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd6600640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd015b000 0x7f0bd00b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd5c00640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bd0070960 0x7f0bd01f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd5200640 1 -- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bd0070960 msgr2=0x7f0bd01f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd5200640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bd0070960 0x7f0bd01f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd5200640 1 -- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd015b000 msgr2=0x7f0bd00b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:57.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd5200640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd015b000 0x7f0bd00b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:57.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd5200640 1 -- 172.21.15.44:0/2743781050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd0155d40 con 0x7f0bd00cf2b0 2024-09-13T08:40:57.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd5200640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd00cf2b0 0x7f0bd0155800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00c530 tx=0x7f0bbc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:57.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bcd600640 1 -- 172.21.15.44:0/2743781050 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc017070 con 0x7f0bd00cf2b0 2024-09-13T08:40:57.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bcd600640 1 -- 172.21.15.44:0/2743781050 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bbc0053c0 con 0x7f0bd00cf2b0 2024-09-13T08:40:57.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bcd600640 1 -- 172.21.15.44:0/2743781050 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc012890 con 0x7f0bd00cf2b0 2024-09-13T08:40:57.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd8600640 1 -- 172.21.15.44:0/2743781050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd0153d20 con 0x7f0bd00cf2b0 2024-09-13T08:40:57.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd8600640 1 -- 172.21.15.44:0/2743781050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd01f8af0 con 0x7f0bd00cf2b0 2024-09-13T08:40:57.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.892+0000 7f0bd8600640 1 -- 172.21.15.44:0/2743781050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd0069410 con 0x7f0bd00cf2b0 2024-09-13T08:40:57.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.896+0000 7f0bcd600640 1 -- 172.21.15.44:0/2743781050 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 11) ==== 97386+0+0 (secure 0 0 0) 0x7f0bbc01a070 con 0x7f0bd00cf2b0 2024-09-13T08:40:57.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.896+0000 7f0bcd600640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0bb0073220 0x7f0bb00756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:57.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.896+0000 7f0bcd600640 1 -- 172.21.15.44:0/2743781050 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f0bbc093650 con 0x7f0bd00cf2b0 2024-09-13T08:40:57.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.896+0000 7f0bd5c00640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0bb0073220 0x7f0bb00756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:57.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.896+0000 7f0bd5c00640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0bb0073220 0x7f0bb00756e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400bc00 tx=0x7f0bc400b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:57.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:57.900+0000 7f0bcd600640 1 -- 172.21.15.44:0/2743781050 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0bd015a580 con 0x7f0bd00cf2b0 2024-09-13T08:40:58.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:58.100+0000 7f0bd8600640 1 -- 172.21.15.44:0/2743781050 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f0bd00d07b0 con 0x7f0bb0073220 2024-09-13T08:40:59.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.260+0000 7f0bcd600640 1 -- 172.21.15.44:0/2743781050 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 12) ==== 97437+0+0 (secure 0 0 0) 0x7f0bbc059ff0 con 0x7f0bd00cf2b0 2024-09-13T08:40:59.508 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:59.504+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:59.509 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:59.504+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:59.509 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:59.504+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:59.509 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:59.504+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:59.509 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:40:59.504+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:40:59.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.552+0000 7f0bcd600640 1 -- 172.21.15.44:0/2743781050 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0bd00d07b0 con 0x7f0bb0073220 2024-09-13T08:40:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 -- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0bb0073220 msgr2=0x7f0bb00756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0bb0073220 0x7f0bb00756e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400bc00 tx=0x7f0bc400b390 comp rx=0 tx=0).stop 2024-09-13T08:40:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 -- 172.21.15.44:0/2743781050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd00cf2b0 msgr2=0x7f0bd0155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd00cf2b0 0x7f0bd0155800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00c530 tx=0x7f0bbc00c9f0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 -- 172.21.15.44:0/2743781050 shutdown_connections 2024-09-13T08:40:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0bb0073220 0x7f0bb00756e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd015b000 0x7f0bd00b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd00cf2b0 0x7f0bd0155800 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 --2- 172.21.15.44:0/2743781050 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bd0070960 0x7f0bd01f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 -- 172.21.15.44:0/2743781050 >> 172.21.15.44:0/2743781050 conn(0x7f0bd00cfbc0 msgr2=0x7f0bd01569f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:59.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 -- 172.21.15.44:0/2743781050 shutdown_connections 2024-09-13T08:40:59.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.556+0000 7f0bd8600640 1 -- 172.21.15.44:0/2743781050 wait complete. 2024-09-13T08:40:59.580 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:40:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 -- 172.21.15.44:0/3558081783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9c15bea0 msgr2=0x7fca9c0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 --2- 172.21.15.44:0/3558081783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9c15bea0 0x7fca9c0b3130 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fca9c069820 tx=0x7fca9401cd80 comp rx=0 tx=0).stop 2024-09-13T08:40:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 -- 172.21.15.44:0/3558081783 shutdown_connections 2024-09-13T08:40:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 --2- 172.21.15.44:0/3558081783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9c15bea0 0x7fca9c0b3130 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 --2- 172.21.15.44:0/3558081783 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca9c0b4bd0 0x7fca9c0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 --2- 172.21.15.44:0/3558081783 >> v2:172.21.15.129:3300/0 conn(0x7fca9c15b580 0x7fca9c15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 -- 172.21.15.44:0/3558081783 >> 172.21.15.44:0/3558081783 conn(0x7fca9c0b18f0 msgr2=0x7fca9c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 -- 172.21.15.44:0/3558081783 shutdown_connections 2024-09-13T08:40:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 -- 172.21.15.44:0/3558081783 wait complete. 2024-09-13T08:40:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 Processor -- start 2024-09-13T08:40:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 -- start start 2024-09-13T08:40:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca9c0b4bd0 0x7fca9c1ed920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9c15b580 0x7fca9c1524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca9c15bea0 0x7fca9c152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c0cc8e0 con 0x7fca9c15b580 2024-09-13T08:40:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c0cc760 con 0x7fca9c0b4bd0 2024-09-13T08:40:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fcaa2400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fca9c0cca60 con 0x7fca9c15bea0 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca93e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9c15b580 0x7fca9c1524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca93e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9c15b580 0x7fca9c1524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45978/0 (socket says 172.21.15.44:45978) 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca93e00640 1 -- 172.21.15.44:0/3807869340 learned_addr learned my addr 172.21.15.44:0/3807869340 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca9b400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca9c0b4bd0 0x7fca9c1ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca9be00640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca9c15bea0 0x7fca9c152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca93e00640 1 -- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca9c15bea0 msgr2=0x7fca9c152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca93e00640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca9c15bea0 0x7fca9c152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca93e00640 1 -- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca9c0b4bd0 msgr2=0x7fca9c1ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca93e00640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca9c0b4bd0 0x7fca9c1ed920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca93e00640 1 -- 172.21.15.44:0/3807869340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c153210 con 0x7fca9c15b580 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca9b400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca9c0b4bd0 0x7fca9c1ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.732+0000 7fca9be00640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca9c15bea0 0x7fca9c152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:40:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fca93e00640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9c15b580 0x7fca9c1524c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fca8400e9c0 tx=0x7fca8400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fca93400640 1 -- 172.21.15.44:0/3807869340 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8400cd80 con 0x7fca9c15b580 2024-09-13T08:40:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fca93400640 1 -- 172.21.15.44:0/3807869340 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca84004030 con 0x7fca9c15b580 2024-09-13T08:40:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fcaa2400640 1 -- 172.21.15.44:0/3807869340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c1534f0 con 0x7fca9c15b580 2024-09-13T08:40:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fca93400640 1 -- 172.21.15.44:0/3807869340 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84004480 con 0x7fca9c15b580 2024-09-13T08:40:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fcaa2400640 1 -- 172.21.15.44:0/3807869340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c153ab0 con 0x7fca9c15b580 2024-09-13T08:40:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fcaa2400640 1 -- 172.21.15.44:0/3807869340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9c15b960 con 0x7fca9c15b580 2024-09-13T08:40:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fca93400640 1 -- 172.21.15.44:0/3807869340 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 12) ==== 97437+0+0 (secure 0 0 0) 0x7fca84003600 con 0x7fca9c15b580 2024-09-13T08:40:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fca93400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca780732b0 0x7fca78075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:40:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fca93400640 1 -- 172.21.15.44:0/3807869340 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fca84095210 con 0x7fca9c15b580 2024-09-13T08:40:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.736+0000 7fca9b400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca780732b0 0x7fca78075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:40:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.740+0000 7fca9b400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca780732b0 0x7fca78075770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009ae0 tx=0x7fca8c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:40:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.740+0000 7fca93400640 1 -- 172.21.15.44:0/3807869340 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fca8405fd30 con 0x7fca9c15b580 2024-09-13T08:40:59.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.948+0000 7fcaa2400640 1 -- 172.21.15.44:0/3807869340 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fca9c15d840 con 0x7fca780732b0 2024-09-13T08:40:59.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.952+0000 7fca93400640 1 -- 172.21.15.44:0/3807869340 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fca9c15d840 con 0x7fca780732b0 2024-09-13T08:40:59.956 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285 2024-09-13T08:40:59.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 -- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca780732b0 msgr2=0x7fca78075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:59.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca780732b0 0x7fca78075770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009ae0 tx=0x7fca8c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 -- 172.21.15.44:0/3807869340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9c15b580 msgr2=0x7fca9c1524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:40:59.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9c15b580 0x7fca9c1524c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fca8400e9c0 tx=0x7fca8400ee80 comp rx=0 tx=0).stop 2024-09-13T08:40:59.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 -- 172.21.15.44:0/3807869340 shutdown_connections 2024-09-13T08:40:59.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca780732b0 0x7fca78075770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca9c15bea0 0x7fca9c152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9c15b580 0x7fca9c1524c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 --2- 172.21.15.44:0/3807869340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca9c0b4bd0 0x7fca9c1ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:40:59.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 -- 172.21.15.44:0/3807869340 >> 172.21.15.44:0/3807869340 conn(0x7fca9c0b18f0 msgr2=0x7fca9c15a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:40:59.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 -- 172.21.15.44:0/3807869340 shutdown_connections 2024-09-13T08:40:59.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:40:59.956+0000 7fcaa2400640 1 -- 172.21.15.44:0/3807869340 wait complete. 2024-09-13T08:40:59.977 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285 2024-09-13T08:40:59.977 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:40:59.977 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:40:59.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:40:59.986 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:41:01.362 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:41:01.365 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:41:09.761 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:41:09.761 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:41:09.762 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.71187 s, 111 MB/s 2024-09-13T08:41:09.763 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:41:09.763 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:41:09.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:41:09.772 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:41:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:41:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:41:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.06872 s, 118 MB/s 2024-09-13T08:41:18.897 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:41:18.898 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:41:18.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:41:18.907 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:41:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:41:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:41:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.2796 s, 104 MB/s 2024-09-13T08:41:29.243 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:41:29.243 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:41:29.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:41:29.252 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:41:31.419 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:41:31.422 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:41:38.346 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:41:38.346 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:41:38.346 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.04372 s, 119 MB/s 2024-09-13T08:41:38.347 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:41:38.347 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:41:38.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:41:38.356 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:41:47.746 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:41:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:41:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.34009 s, 115 MB/s 2024-09-13T08:41:47.748 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:41:47.748 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:41:47.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:41:47.757 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:41:57.224 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:41:57.225 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:41:57.225 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.4104 s, 114 MB/s 2024-09-13T08:41:57.226 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:41:57.226 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:41:57.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:41:57.235 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:42:01.475 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:42:01.479 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:42:06.634 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:42:06.634 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:42:06.634 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.34791 s, 115 MB/s 2024-09-13T08:42:06.635 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:42:06.635 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:42:06.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:42:06.644 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:42:16.789 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:42:16.790 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:42:16.790 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.0944 s, 106 MB/s 2024-09-13T08:42:16.791 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:42:16.791 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:42:16.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:42:16.800 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:42:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:42:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:42:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.69053 s, 111 MB/s 2024-09-13T08:42:26.542 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:42:26.543 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:42:26.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:42:26.551 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:42:31.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:42:31.545 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:42:35.802 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:42:35.803 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:42:35.803 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.19676 s, 117 MB/s 2024-09-13T08:42:35.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-13T08:42:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.951+0000 7f232ca00640 1 -- 172.21.15.44:0/2489500058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23280cf2b0 msgr2=0x7f232806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.951+0000 7f232ca00640 1 --2- 172.21.15.44:0/2489500058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23280cf2b0 0x7f232806fcc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f231800aa10 tx=0x7f231801dc30 comp rx=0 tx=0).stop 2024-09-13T08:42:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.951+0000 7f232ca00640 1 -- 172.21.15.44:0/2489500058 shutdown_connections 2024-09-13T08:42:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.951+0000 7f232ca00640 1 --2- 172.21.15.44:0/2489500058 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2328072f80 0x7f232815e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.951+0000 7f232ca00640 1 --2- 172.21.15.44:0/2489500058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23280cf2b0 0x7f232806fcc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.951+0000 7f232ca00640 1 -- 172.21.15.44:0/2489500058 >> 172.21.15.44:0/2489500058 conn(0x7f23280cfbc0 msgr2=0x7f23280cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.951+0000 7f232ca00640 1 -- 172.21.15.44:0/2489500058 shutdown_connections 2024-09-13T08:42:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.951+0000 7f232ca00640 1 -- 172.21.15.44:0/2489500058 wait complete. 2024-09-13T08:42:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 Processor -- start 2024-09-13T08:42:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 -- start start 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2328072f80 0x7f2328147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23280cf2b0 0x7f23280b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f232815b000 0x7f2328158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2328167120 con 0x7f23280cf2b0 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23280ce6e0 con 0x7f232815b000 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f23281672a0 con 0x7f2328072f80 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23280cf2b0 0x7f23280b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f2325e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2328072f80 0x7f2328147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23280cf2b0 0x7f23280b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39038/0 (socket says 172.21.15.44:39038) 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231fe00640 1 -- 172.21.15.44:0/518599312 learned_addr learned my addr 172.21.15.44:0/518599312 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:35.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f2326800640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f232815b000 0x7f2328158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231fe00640 1 -- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2328072f80 msgr2=0x7f2328147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231fe00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2328072f80 0x7f2328147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231fe00640 1 -- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f232815b000 msgr2=0x7f2328158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231fe00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f232815b000 0x7f2328158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231fe00640 1 -- 172.21.15.44:0/518599312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23280b2d70 con 0x7f23280cf2b0 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f2326800640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f232815b000 0x7f2328158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231fe00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23280cf2b0 0x7f23280b3b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f231000c3f0 tx=0x7f231000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231e000640 1 -- 172.21.15.44:0/518599312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310017070 con 0x7f23280cf2b0 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231e000640 1 -- 172.21.15.44:0/518599312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23100053c0 con 0x7f23280cf2b0 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f231e000640 1 -- 172.21.15.44:0/518599312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310003860 con 0x7f23280cf2b0 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 -- 172.21.15.44:0/518599312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23280b3050 con 0x7f23280cf2b0 2024-09-13T08:42:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 -- 172.21.15.44:0/518599312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23280b3610 con 0x7f23280cf2b0 2024-09-13T08:42:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.955+0000 7f232ca00640 1 -- 172.21.15.44:0/518599312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2328069410 con 0x7f23280cf2b0 2024-09-13T08:42:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.959+0000 7f231e000640 1 -- 172.21.15.44:0/518599312 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 15) ==== 97386+0+0 (secure 0 0 0) 0x7f231001a070 con 0x7f23280cf2b0 2024-09-13T08:42:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.959+0000 7f231e000640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23040731c0 0x7f2304075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.959+0000 7f231e000640 1 -- 172.21.15.44:0/518599312 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f23100937a0 con 0x7f23280cf2b0 2024-09-13T08:42:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.959+0000 7f2325e00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23040731c0 0x7f2304075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.959+0000 7f2325e00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23040731c0 0x7f2304075680 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23180b0000 tx=0x7f231801c470 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:35.963+0000 7f231e000640 1 -- 172.21.15.44:0/518599312 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f232815a580 con 0x7f23280cf2b0 2024-09-13T08:42:36.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.167+0000 7f232ca00640 1 -- 172.21.15.44:0/518599312 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f232815e580 con 0x7f23040731c0 2024-09-13T08:42:36.176 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:36.175+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:36.176 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:36.175+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:36.176 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:36.175+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:36.176 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:36.175+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:36.176 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:36.175+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.191+0000 7f231e000640 1 -- 172.21.15.44:0/518599312 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f232815e580 con 0x7f23040731c0 2024-09-13T08:42:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 -- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23040731c0 msgr2=0x7f2304075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23040731c0 0x7f2304075680 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23180b0000 tx=0x7f231801c470 comp rx=0 tx=0).stop 2024-09-13T08:42:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 -- 172.21.15.44:0/518599312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23280cf2b0 msgr2=0x7f23280b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23280cf2b0 0x7f23280b3b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f231000c3f0 tx=0x7f231000c8b0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 -- 172.21.15.44:0/518599312 shutdown_connections 2024-09-13T08:42:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f23040731c0 0x7f2304075680 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f232815b000 0x7f2328158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23280cf2b0 0x7f23280b3b30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 --2- 172.21.15.44:0/518599312 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2328072f80 0x7f2328147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 -- 172.21.15.44:0/518599312 >> 172.21.15.44:0/518599312 conn(0x7f23280cfbc0 msgr2=0x7f232806e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 -- 172.21.15.44:0/518599312 shutdown_connections 2024-09-13T08:42:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.195+0000 7f232ca00640 1 -- 172.21.15.44:0/518599312 wait complete. 2024-09-13T08:42:36.218 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:42:36.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.363+0000 7fc95c200640 1 -- 172.21.15.44:0/543913358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc95415f390 msgr2=0x7fc95415cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:36.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.363+0000 7fc95c200640 1 --2- 172.21.15.44:0/543913358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc95415f390 0x7fc95415cdb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc948009900 tx=0x7fc94801c850 comp rx=0 tx=0).stop 2024-09-13T08:42:36.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.363+0000 7fc95c200640 1 -- 172.21.15.44:0/543913358 shutdown_connections 2024-09-13T08:42:36.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.363+0000 7fc95c200640 1 --2- 172.21.15.44:0/543913358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc95415f390 0x7fc95415cdb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.363+0000 7fc95c200640 1 -- 172.21.15.44:0/543913358 >> 172.21.15.44:0/543913358 conn(0x7fc95406ecb0 msgr2=0x7fc95406f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:36.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.363+0000 7fc95c200640 1 -- 172.21.15.44:0/543913358 shutdown_connections 2024-09-13T08:42:36.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.363+0000 7fc95c200640 1 -- 172.21.15.44:0/543913358 wait complete. 2024-09-13T08:42:36.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 Processor -- start 2024-09-13T08:42:36.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 -- start start 2024-09-13T08:42:36.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc9540b3d80 0x7fc9540ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc95415d4f0 0x7fc954149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc95415f390 0x7fc9541468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc95416a360 con 0x7fc95415d4f0 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc95416a1e0 con 0x7fc95415f390 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc95416a4e0 con 0x7fc9540b3d80 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc958e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc95415d4f0 0x7fc954149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc958e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc95415d4f0 0x7fc954149460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39046/0 (socket says 172.21.15.44:39046) 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc958e00640 1 -- 172.21.15.44:0/1158133515 learned_addr learned my addr 172.21.15.44:0/1158133515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc959800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc9540b3d80 0x7fc9540ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95a200640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc95415f390 0x7fc9541468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:36.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc958e00640 1 -- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc9540b3d80 msgr2=0x7fc9540ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:36.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc958e00640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc9540b3d80 0x7fc9540ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc958e00640 1 -- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc95415f390 msgr2=0x7fc9541468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:36.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc958e00640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc95415f390 0x7fc9541468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc958e00640 1 -- 172.21.15.44:0/1158133515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95414b310 con 0x7fc95415d4f0 2024-09-13T08:42:36.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc958e00640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc95415d4f0 0x7fc954149460 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc94000c530 tx=0x7fc94000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:36.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc951600640 1 -- 172.21.15.44:0/1158133515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940017070 con 0x7fc95415d4f0 2024-09-13T08:42:36.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc951600640 1 -- 172.21.15.44:0/1158133515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9400053c0 con 0x7fc95415d4f0 2024-09-13T08:42:36.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc951600640 1 -- 172.21.15.44:0/1158133515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940012890 con 0x7fc95415d4f0 2024-09-13T08:42:36.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 -- 172.21.15.44:0/1158133515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95414b5f0 con 0x7fc95415d4f0 2024-09-13T08:42:36.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 -- 172.21.15.44:0/1158133515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc95414bbb0 con 0x7fc95415d4f0 2024-09-13T08:42:36.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.367+0000 7fc95c200640 1 -- 172.21.15.44:0/1158133515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc95414bdb0 con 0x7fc95415d4f0 2024-09-13T08:42:36.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.371+0000 7fc951600640 1 -- 172.21.15.44:0/1158133515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 15) ==== 97386+0+0 (secure 0 0 0) 0x7fc94001a070 con 0x7fc95415d4f0 2024-09-13T08:42:36.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.371+0000 7fc951600640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc934073190 0x7fc934075650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:36.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.371+0000 7fc951600640 1 -- 172.21.15.44:0/1158133515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fc940093760 con 0x7fc95415d4f0 2024-09-13T08:42:36.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.371+0000 7fc959800640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc934073190 0x7fc934075650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:36.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.371+0000 7fc959800640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc934073190 0x7fc934075650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc9480af000 tx=0x7fc94801b090 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:36.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.375+0000 7fc951600640 1 -- 172.21.15.44:0/1158133515 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc95414bdb0 con 0x7fc95415d4f0 2024-09-13T08:42:36.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.579+0000 7fc95c200640 1 -- 172.21.15.44:0/1158133515 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fc95415cd30 con 0x7fc934073190 2024-09-13T08:42:36.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.583+0000 7fc951600640 1 -- 172.21.15.44:0/1158133515 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fc95415cd30 con 0x7fc934073190 2024-09-13T08:42:36.588 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285 2024-09-13T08:42:36.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 -- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc934073190 msgr2=0x7fc934075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:36.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc934073190 0x7fc934075650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc9480af000 tx=0x7fc94801b090 comp rx=0 tx=0).stop 2024-09-13T08:42:36.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 -- 172.21.15.44:0/1158133515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc95415d4f0 msgr2=0x7fc954149460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:36.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc95415d4f0 0x7fc954149460 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc94000c530 tx=0x7fc94000c9f0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 -- 172.21.15.44:0/1158133515 shutdown_connections 2024-09-13T08:42:36.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc934073190 0x7fc934075650 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc95415f390 0x7fc9541468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc95415d4f0 0x7fc954149460 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 --2- 172.21.15.44:0/1158133515 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc9540b3d80 0x7fc9540ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:36.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 -- 172.21.15.44:0/1158133515 >> 172.21.15.44:0/1158133515 conn(0x7fc95406ecb0 msgr2=0x7fc95406f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:36.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 -- 172.21.15.44:0/1158133515 shutdown_connections 2024-09-13T08:42:36.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:36.587+0000 7fc95c200640 1 -- 172.21.15.44:0/1158133515 wait complete. 2024-09-13T08:42:36.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:42:36.609 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285 2024-09-13T08:42:36.609 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:42:36.618 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285 2024-09-13T08:42:36.619 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="10488365056" 2024-09-13T08:42:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:42:36.620 INFO:tasks.cephfs.test_volumes:r_size = 10488365056 exp_size = 10737418240 2024-09-13T08:42:38.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:42:38.621 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285 2024-09-13T08:42:38.621 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:42:38.632 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/ddeddbff-56f5-457d-9d8c-4d302c773285 2024-09-13T08:42:38.632 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="10737418240" 2024-09-13T08:42:38.632 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:42:38.633 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-13T08:42:38.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone56670 2024-09-13T08:42:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.839+0000 7f9d8fe00640 1 -- 172.21.15.44:0/2658425423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d74005220 msgr2=0x7f9d7400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.839+0000 7f9d8fe00640 1 --2- 172.21.15.44:0/2658425423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d74005220 0x7f9d7400b9d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9d88006fd0 tx=0x7f9d8801caf0 comp rx=0 tx=0).stop 2024-09-13T08:42:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.839+0000 7f9d8fe00640 1 -- 172.21.15.44:0/2658425423 shutdown_connections 2024-09-13T08:42:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.839+0000 7f9d8fe00640 1 --2- 172.21.15.44:0/2658425423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d74005220 0x7f9d7400b9d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.839+0000 7f9d8fe00640 1 -- 172.21.15.44:0/2658425423 >> 172.21.15.44:0/2658425423 conn(0x7f9d900d3e70 msgr2=0x7f9d900d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.839+0000 7f9d8fe00640 1 -- 172.21.15.44:0/2658425423 shutdown_connections 2024-09-13T08:42:38.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.839+0000 7f9d8fe00640 1 -- 172.21.15.44:0/2658425423 wait complete. 2024-09-13T08:42:38.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.839+0000 7f9d8fe00640 1 Processor -- start 2024-09-13T08:42:38.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.839+0000 7f9d8fe00640 1 -- start start 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d74005220 0x7f9d900b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d900b8230 0x7f9d9015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d90157680 0x7f9d9015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d900d1e80 con 0x7f9d74005220 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d900d4bf0 con 0x7f9d90157680 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9d900d2000 con 0x7f9d900b8230 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d74005220 0x7f9d900b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d74005220 0x7f9d900b7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39072/0 (socket says 172.21.15.44:39072) 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8ea00640 1 -- 172.21.15.44:0/3696526468 learned_addr learned my addr 172.21.15.44:0/3696526468 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d86000640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d900b8230 0x7f9d9015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8ea00640 1 -- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d900b8230 msgr2=0x7f9d9015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8f400640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d90157680 0x7f9d9015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8ea00640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d900b8230 0x7f9d9015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8ea00640 1 -- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d90157680 msgr2=0x7f9d9015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8ea00640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d90157680 0x7f9d9015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8ea00640 1 -- 172.21.15.44:0/3696526468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d90159330 con 0x7f9d74005220 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8f400640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d90157680 0x7f9d9015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8ea00640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d74005220 0x7f9d900b7cf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9d80011ae0 tx=0x7f9d80009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d87400640 1 -- 172.21.15.44:0/3696526468 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8001e070 con 0x7f9d74005220 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d87400640 1 -- 172.21.15.44:0/3696526468 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d80004460 con 0x7f9d74005220 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d87400640 1 -- 172.21.15.44:0/3696526468 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000fc40 con 0x7f9d74005220 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8fe00640 1 -- 172.21.15.44:0/3696526468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d90159610 con 0x7f9d74005220 2024-09-13T08:42:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8fe00640 1 -- 172.21.15.44:0/3696526468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d90159bd0 con 0x7f9d74005220 2024-09-13T08:42:38.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d8fe00640 1 -- 172.21.15.44:0/3696526468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9015d170 con 0x7f9d74005220 2024-09-13T08:42:38.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.843+0000 7f9d87400640 1 -- 172.21.15.44:0/3696526468 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 97437+0+0 (secure 0 0 0) 0x7f9d80021070 con 0x7f9d74005220 2024-09-13T08:42:38.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.847+0000 7f9d87400640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d6c0732e0 0x7f9d6c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:38.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.847+0000 7f9d87400640 1 -- 172.21.15.44:0/3696526468 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f9d800950b0 con 0x7f9d74005220 2024-09-13T08:42:38.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.847+0000 7f9d86000640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d6c0732e0 0x7f9d6c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:38.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.847+0000 7f9d86000640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d6c0732e0 0x7f9d6c0757a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9d90157070 tx=0x7f9d78006730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:38.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:38.851+0000 7f9d87400640 1 -- 172.21.15.44:0/3696526468 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9d8005f4f0 con 0x7f9d74005220 2024-09-13T08:42:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.059+0000 7f9d8fe00640 1 -- 172.21.15.44:0/3696526468 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone56670", "target": ["mon-mgr", ""]}) -- 0x7f9d901615d0 con 0x7f9d6c0732e0 2024-09-13T08:42:39.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.135+0000 7f9d87400640 1 -- 172.21.15.44:0/3696526468 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d901615d0 con 0x7f9d6c0732e0 2024-09-13T08:42:39.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.135+0000 7f9d8fe00640 1 -- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d6c0732e0 msgr2=0x7f9d6c0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.135+0000 7f9d8fe00640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d6c0732e0 0x7f9d6c0757a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9d90157070 tx=0x7f9d78006730 comp rx=0 tx=0).stop 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.135+0000 7f9d8fe00640 1 -- 172.21.15.44:0/3696526468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d74005220 msgr2=0x7f9d900b7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.135+0000 7f9d8fe00640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d74005220 0x7f9d900b7cf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9d80011ae0 tx=0x7f9d80009e50 comp rx=0 tx=0).stop 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.139+0000 7f9d8fe00640 1 -- 172.21.15.44:0/3696526468 shutdown_connections 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.139+0000 7f9d8fe00640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d6c0732e0 0x7f9d6c0757a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.139+0000 7f9d8fe00640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d90157680 0x7f9d9015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.139+0000 7f9d8fe00640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d900b8230 0x7f9d9015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.139+0000 7f9d8fe00640 1 --2- 172.21.15.44:0/3696526468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d74005220 0x7f9d900b7cf0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.139+0000 7f9d8fe00640 1 -- 172.21.15.44:0/3696526468 >> 172.21.15.44:0/3696526468 conn(0x7f9d900d3e70 msgr2=0x7f9d9015f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.139+0000 7f9d8fe00640 1 -- 172.21.15.44:0/3696526468 shutdown_connections 2024-09-13T08:42:39.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.139+0000 7f9d8fe00640 1 -- 172.21.15.44:0/3696526468 wait complete. 2024-09-13T08:42:39.143 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:39.143+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:39.143 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:39.143+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:39.143 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:39.143+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:39.143 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:39.143+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:39.143 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:39.143+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:39.159 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:39.159+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:39.159 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:39.159+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:39.159 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:39.159+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:39.159 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:39.159+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:39.159 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:39.159+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:39.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone23241 2024-09-13T08:42:39.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.311+0000 7f9609000640 1 -- 172.21.15.44:0/3327146608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960415af10 msgr2=0x7f9604073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:39.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.311+0000 7f9609000640 1 --2- 172.21.15.44:0/3327146608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960415af10 0x7f9604073790 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009c80 tx=0x7f95f401f5a0 comp rx=0 tx=0).stop 2024-09-13T08:42:39.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.311+0000 7f9609000640 1 -- 172.21.15.44:0/3327146608 shutdown_connections 2024-09-13T08:42:39.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.311+0000 7f9609000640 1 --2- 172.21.15.44:0/3327146608 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f960406f940 0x7f960406fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:39.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.311+0000 7f9609000640 1 --2- 172.21.15.44:0/3327146608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960415af10 0x7f9604073790 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:39.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.311+0000 7f9609000640 1 -- 172.21.15.44:0/3327146608 >> 172.21.15.44:0/3327146608 conn(0x7f96040d0b10 msgr2=0x7f96040d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:39.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.311+0000 7f9609000640 1 -- 172.21.15.44:0/3327146608 shutdown_connections 2024-09-13T08:42:39.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.311+0000 7f9609000640 1 -- 172.21.15.44:0/3327146608 wait complete. 2024-09-13T08:42:39.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.311+0000 7f9609000640 1 Processor -- start 2024-09-13T08:42:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.311+0000 7f9609000640 1 -- start start 2024-09-13T08:42:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9609000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960406f940 0x7f96040b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9609000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96040cfe40 0x7f96041546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9609000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96041e6f40 0x7f96041e73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9609000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96040706a0 con 0x7f960406f940 2024-09-13T08:42:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9609000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9604070520 con 0x7f96040cfe40 2024-09-13T08:42:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9609000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9604070820 con 0x7f96041e6f40 2024-09-13T08:42:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960406f940 0x7f96040b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960406f940 0x7f96040b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39088/0 (socket says 172.21.15.44:39088) 2024-09-13T08:42:39.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603400640 1 -- 172.21.15.44:0/1370667435 learned_addr learned my addr 172.21.15.44:0/1370667435 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f95faa00640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96040cfe40 0x7f96041546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603e00640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96041e6f40 0x7f96041e73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603400640 1 -- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96041e6f40 msgr2=0x7f96041e73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603400640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96041e6f40 0x7f96041e73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603400640 1 -- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96040cfe40 msgr2=0x7f96041546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603400640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96040cfe40 0x7f96041546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603400640 1 -- 172.21.15.44:0/1370667435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96041f4970 con 0x7f960406f940 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603e00640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96041e6f40 0x7f96041e73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9603400640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960406f940 0x7f96040b5ff0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000f90 tx=0x7f95f40af410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f95fbe00640 1 -- 172.21.15.44:0/1370667435 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4004e60 con 0x7f960406f940 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f95fbe00640 1 -- 172.21.15.44:0/1370667435 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95f40be070 con 0x7f960406f940 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f95fbe00640 1 -- 172.21.15.44:0/1370667435 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f40afaa0 con 0x7f960406f940 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9609000640 1 -- 172.21.15.44:0/1370667435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96041f4c50 con 0x7f960406f940 2024-09-13T08:42:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9609000640 1 -- 172.21.15.44:0/1370667435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96041f5210 con 0x7f960406f940 2024-09-13T08:42:39.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.315+0000 7f9609000640 1 -- 172.21.15.44:0/1370667435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960406fdd0 con 0x7f960406f940 2024-09-13T08:42:39.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.319+0000 7f95fbe00640 1 -- 172.21.15.44:0/1370667435 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 97437+0+0 (secure 0 0 0) 0x7f95f40afc40 con 0x7f960406f940 2024-09-13T08:42:39.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.319+0000 7f95fbe00640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f95e00732e0 0x7f95e00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:39.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.319+0000 7f95fbe00640 1 -- 172.21.15.44:0/1370667435 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f95f4132f10 con 0x7f960406f940 2024-09-13T08:42:39.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.319+0000 7f95faa00640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f95e00732e0 0x7f95e00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:39.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.319+0000 7f95faa00640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f95e00732e0 0x7f95e00757a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f96040c40b0 tx=0x7f95ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:39.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.323+0000 7f95fbe00640 1 -- 172.21.15.44:0/1370667435 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f95f40ca070 con 0x7f960406f940 2024-09-13T08:42:39.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:39.531+0000 7f9609000640 1 -- 172.21.15.44:0/1370667435 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone23241", "target": ["mon-mgr", ""]}) -- 0x7f96041e7910 con 0x7f95e00732e0 2024-09-13T08:42:40.145 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:40.143+0000 7f8ab8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:40.145 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:40.143+0000 7f8ab8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:40.145 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:40.143+0000 7f8ab8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:40.145 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:40.143+0000 7f8ab8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:40.145 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:40.143+0000 7f8ab8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.231+0000 7f95fbe00640 1 -- 172.21.15.44:0/1370667435 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f95ec007c50 con 0x7f95e00732e0 2024-09-13T08:42:40.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 -- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f95e00732e0 msgr2=0x7f95e00757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:40.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f95e00732e0 0x7f95e00757a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f96040c40b0 tx=0x7f95ec009040 comp rx=0 tx=0).stop 2024-09-13T08:42:40.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 -- 172.21.15.44:0/1370667435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960406f940 msgr2=0x7f96040b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:40.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960406f940 0x7f96040b5ff0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000f90 tx=0x7f95f40af410 comp rx=0 tx=0).stop 2024-09-13T08:42:40.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 -- 172.21.15.44:0/1370667435 shutdown_connections 2024-09-13T08:42:40.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f95e00732e0 0x7f95e00757a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:40.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f96041e6f40 0x7f96041e73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:40.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96040cfe40 0x7f96041546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:40.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 --2- 172.21.15.44:0/1370667435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960406f940 0x7f96040b5ff0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:40.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 -- 172.21.15.44:0/1370667435 >> 172.21.15.44:0/1370667435 conn(0x7f96040d0b10 msgr2=0x7f96041e7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:40.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 -- 172.21.15.44:0/1370667435 shutdown_connections 2024-09-13T08:42:40.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.235+0000 7f9609000640 1 -- 172.21.15.44:0/1370667435 wait complete. 2024-09-13T08:42:40.277 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone53352 2024-09-13T08:42:40.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.507+0000 7f77c6800640 1 -- 172.21.15.44:0/3745048206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a40052f0 msgr2=0x7f77a400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:40.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.507+0000 7f77c6800640 1 --2- 172.21.15.44:0/3745048206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a40052f0 0x7f77a400b9d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f77b0005fd0 tx=0x7f77b001cd80 comp rx=0 tx=0).stop 2024-09-13T08:42:40.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.507+0000 7f77c6800640 1 -- 172.21.15.44:0/3745048206 shutdown_connections 2024-09-13T08:42:40.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.507+0000 7f77c6800640 1 --2- 172.21.15.44:0/3745048206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a40052f0 0x7f77a400b9d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:40.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.507+0000 7f77c6800640 1 -- 172.21.15.44:0/3745048206 >> 172.21.15.44:0/3745048206 conn(0x7f77c00b18f0 msgr2=0x7f77c00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:40.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.507+0000 7f77c6800640 1 -- 172.21.15.44:0/3745048206 shutdown_connections 2024-09-13T08:42:40.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.507+0000 7f77c6800640 1 -- 172.21.15.44:0/3745048206 wait complete. 2024-09-13T08:42:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77c6800640 1 Processor -- start 2024-09-13T08:42:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77c6800640 1 -- start start 2024-09-13T08:42:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77c6800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a40052f0 0x7f77c01f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77c6800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77c0153de0 0x7f77c0150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77c6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77c00cb850 0x7f77c00cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77c6800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b4660 con 0x7f77c00cb850 2024-09-13T08:42:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77c6800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b44e0 con 0x7f77a40052f0 2024-09-13T08:42:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77c6800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00b47e0 con 0x7f77c0153de0 2024-09-13T08:42:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77c00cb850 0x7f77c00cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77c00cb850 0x7f77c00cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39104/0 (socket says 172.21.15.44:39104) 2024-09-13T08:42:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77bfe00640 1 -- 172.21.15.44:0/2236226087 learned_addr learned my addr 172.21.15.44:0/2236226087 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:40.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77bfe00640 1 -- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77c0153de0 msgr2=0x7f77c0150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:40.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77bfe00640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77c0153de0 0x7f77c0150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77bfe00640 1 -- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a40052f0 msgr2=0x7f77c01f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77bfe00640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a40052f0 0x7f77c01f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77bfe00640 1 -- 172.21.15.44:0/2236226087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c01f80d0 con 0x7f77c00cb850 2024-09-13T08:42:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77bfe00640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77c00cb850 0x7f77c00cbd00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f77b000ba60 tx=0x7f77b00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77b7400640 1 -- 172.21.15.44:0/2236226087 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b00be070 con 0x7f77c00cb850 2024-09-13T08:42:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77b7400640 1 -- 172.21.15.44:0/2236226087 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77b0005d20 con 0x7f77c00cb850 2024-09-13T08:42:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77c6800640 1 -- 172.21.15.44:0/2236226087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c01f83b0 con 0x7f77c00cb850 2024-09-13T08:42:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77b7400640 1 -- 172.21.15.44:0/2236226087 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b00ab040 con 0x7f77c00cb850 2024-09-13T08:42:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.511+0000 7f77c6800640 1 -- 172.21.15.44:0/2236226087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c01f8970 con 0x7f77c00cb850 2024-09-13T08:42:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.515+0000 7f77c6800640 1 -- 172.21.15.44:0/2236226087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c00ce9d0 con 0x7f77c00cb850 2024-09-13T08:42:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.515+0000 7f77b7400640 1 -- 172.21.15.44:0/2236226087 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 97437+0+0 (secure 0 0 0) 0x7f77b0004930 con 0x7f77c00cb850 2024-09-13T08:42:40.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.519+0000 7f77b7400640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77a00732b0 0x7f77a0075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:40.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.519+0000 7f77b7400640 1 -- 172.21.15.44:0/2236226087 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f77b012e480 con 0x7f77c00cb850 2024-09-13T08:42:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.519+0000 7f77bf400640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77a00732b0 0x7f77a0075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:40.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.519+0000 7f77bf400640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77a00732b0 0x7f77a0075770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009f60 tx=0x7f77b8004ac0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.523+0000 7f77b7400640 1 -- 172.21.15.44:0/2236226087 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f77b0016a90 con 0x7f77c00cb850 2024-09-13T08:42:40.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.827+0000 7f77b7400640 1 -- 172.21.15.44:0/2236226087 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 17) ==== 97590+0+0 (secure 0 0 0) 0x7f77b00f4b90 con 0x7f77c00cb850 2024-09-13T08:42:40.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:40.831+0000 7f77c6800640 1 -- 172.21.15.44:0/2236226087 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone53352", "target": ["mon-mgr", ""]}) -- 0x7f77c015cb10 con 0x7f77a00732b0 2024-09-13T08:42:41.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.327+0000 7f77b7400640 1 -- 172.21.15.44:0/2236226087 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f77c015cb10 con 0x7f77a00732b0 2024-09-13T08:42:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.335+0000 7f77c6800640 1 -- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77a00732b0 msgr2=0x7f77a0075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.335+0000 7f77c6800640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77a00732b0 0x7f77a0075770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009f60 tx=0x7f77b8004ac0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.335+0000 7f77c6800640 1 -- 172.21.15.44:0/2236226087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77c00cb850 msgr2=0x7f77c00cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.335+0000 7f77c6800640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77c00cb850 0x7f77c00cbd00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f77b000ba60 tx=0x7f77b00056a0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.335+0000 7f77c6800640 1 -- 172.21.15.44:0/2236226087 shutdown_connections 2024-09-13T08:42:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.335+0000 7f77c6800640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f77a00732b0 0x7f77a0075770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.335+0000 7f77c6800640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77c00cb850 0x7f77c00cbd00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.335+0000 7f77c6800640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77c0153de0 0x7f77c0150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.335+0000 7f77c6800640 1 --2- 172.21.15.44:0/2236226087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a40052f0 0x7f77c01f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.335+0000 7f77c6800640 1 -- 172.21.15.44:0/2236226087 >> 172.21.15.44:0/2236226087 conn(0x7f77c00b18f0 msgr2=0x7f77c00ad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:41.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.339+0000 7f77c6800640 1 -- 172.21.15.44:0/2236226087 shutdown_connections 2024-09-13T08:42:41.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.339+0000 7f77c6800640 1 -- 172.21.15.44:0/2236226087 wait complete. 2024-09-13T08:42:41.367 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:41.363+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:41.368 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:41.363+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:41.368 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:41.363+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:41.368 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:41.363+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:41.368 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:41.363+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:41.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone57693 2024-09-13T08:42:41.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.599+0000 7f0bd3400640 1 -- 172.21.15.44:0/3329817820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bcc0ccc60 msgr2=0x7f0bcc15ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:41.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.599+0000 7f0bd3400640 1 --2- 172.21.15.44:0/3329817820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bcc0ccc60 0x7f0bcc15ea20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009900 tx=0x7f0bbc01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.599+0000 7f0bd3400640 1 -- 172.21.15.44:0/3329817820 shutdown_connections 2024-09-13T08:42:41.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.599+0000 7f0bd3400640 1 --2- 172.21.15.44:0/3329817820 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bcc071740 0x7f0bcc06f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.599+0000 7f0bd3400640 1 --2- 172.21.15.44:0/3329817820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bcc0ccc60 0x7f0bcc15ea20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.599+0000 7f0bd3400640 1 -- 172.21.15.44:0/3329817820 >> 172.21.15.44:0/3329817820 conn(0x7f0bcc0b2060 msgr2=0x7f0bcc0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:41.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.599+0000 7f0bd3400640 1 -- 172.21.15.44:0/3329817820 shutdown_connections 2024-09-13T08:42:41.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.599+0000 7f0bd3400640 1 -- 172.21.15.44:0/3329817820 wait complete. 2024-09-13T08:42:41.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.599+0000 7f0bd3400640 1 Processor -- start 2024-09-13T08:42:41.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd3400640 1 -- start start 2024-09-13T08:42:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bcc071740 0x7f0bcc0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd3400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bcc0ccc60 0x7f0bcc1500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc15a5f0 0x7f0bcc1672e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd3400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc168780 con 0x7f0bcc071740 2024-09-13T08:42:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd3400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc168600 con 0x7f0bcc15a5f0 2024-09-13T08:42:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd3400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc168900 con 0x7f0bcc0ccc60 2024-09-13T08:42:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bcc071740 0x7f0bcc0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bcc071740 0x7f0bcc0b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39122/0 (socket says 172.21.15.44:39122) 2024-09-13T08:42:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd0a00640 1 -- 172.21.15.44:0/377548967 learned_addr learned my addr 172.21.15.44:0/377548967 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:41.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bc3e00640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bcc0ccc60 0x7f0bcc1500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:41.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd0a00640 1 -- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bcc0ccc60 msgr2=0x7f0bcc1500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:41.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd0a00640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bcc0ccc60 0x7f0bcc1500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd0a00640 1 -- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc15a5f0 msgr2=0x7f0bcc1672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:41.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd0a00640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc15a5f0 0x7f0bcc1672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:41.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd0a00640 1 -- 172.21.15.44:0/377548967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc159690 con 0x7f0bcc071740 2024-09-13T08:42:41.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bd0a00640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bcc071740 0x7f0bcc0b3a20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009850 tx=0x7f0bbc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:41.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.603+0000 7f0bc9600640 1 -- 172.21.15.44:0/377548967 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0b2040 con 0x7f0bcc071740 2024-09-13T08:42:41.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.607+0000 7f0bd3400640 1 -- 172.21.15.44:0/377548967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc1650a0 con 0x7f0bcc071740 2024-09-13T08:42:41.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.607+0000 7f0bd3400640 1 -- 172.21.15.44:0/377548967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc159930 con 0x7f0bcc071740 2024-09-13T08:42:41.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.607+0000 7f0bc9600640 1 -- 172.21.15.44:0/377548967 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bbc004430 con 0x7f0bcc071740 2024-09-13T08:42:41.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.607+0000 7f0bc9600640 1 -- 172.21.15.44:0/377548967 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc005250 con 0x7f0bcc071740 2024-09-13T08:42:41.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.607+0000 7f0bd3400640 1 -- 172.21.15.44:0/377548967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba40052a0 con 0x7f0bcc071740 2024-09-13T08:42:41.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.611+0000 7f0bc9600640 1 -- 172.21.15.44:0/377548967 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 97590+0+0 (secure 0 0 0) 0x7f0bbc0053f0 con 0x7f0bcc071740 2024-09-13T08:42:41.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.615+0000 7f0bc9600640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ba8073460 0x7f0ba8075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:41.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.615+0000 7f0bc9600640 1 -- 172.21.15.44:0/377548967 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f0bbc12ea40 con 0x7f0bcc071740 2024-09-13T08:42:41.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.619+0000 7f0bc9600640 1 -- 172.21.15.44:0/377548967 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0bbc0164d0 con 0x7f0bcc071740 2024-09-13T08:42:41.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.623+0000 7f0bc3e00640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ba8073460 0x7f0ba8075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:41.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:41.623+0000 7f0bc3e00640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ba8073460 0x7f0ba8075920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0b5f70 tx=0x7f0bb4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:42.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:42.003+0000 7f0bd3400640 1 -- 172.21.15.44:0/377548967 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone57693", "target": ["mon-mgr", ""]}) -- 0x7f0ba4000c90 con 0x7f0ba8073460 2024-09-13T08:42:42.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:42.899+0000 7f0bc9600640 1 -- 172.21.15.44:0/377548967 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 18) ==== 97641+0+0 (secure 0 0 0) 0x7f0bbc0f4e40 con 0x7f0bcc071740 2024-09-13T08:42:43.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.675+0000 7f0bc9600640 1 -- 172.21.15.44:0/377548967 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ba4000c90 con 0x7f0ba8073460 2024-09-13T08:42:43.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.679+0000 7f0bd3400640 1 -- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ba8073460 msgr2=0x7f0ba8075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:43.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.679+0000 7f0bd3400640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ba8073460 0x7f0ba8075920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0b5f70 tx=0x7f0bb4008040 comp rx=0 tx=0).stop 2024-09-13T08:42:43.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.679+0000 7f0bd3400640 1 -- 172.21.15.44:0/377548967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bcc071740 msgr2=0x7f0bcc0b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:43.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.679+0000 7f0bd3400640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bcc071740 0x7f0bcc0b3a20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009850 tx=0x7f0bbc004040 comp rx=0 tx=0).stop 2024-09-13T08:42:43.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.683+0000 7f0bd3400640 1 -- 172.21.15.44:0/377548967 shutdown_connections 2024-09-13T08:42:43.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.683+0000 7f0bd3400640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ba8073460 0x7f0ba8075920 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:43.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.683+0000 7f0bd3400640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc15a5f0 0x7f0bcc1672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:43.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.683+0000 7f0bd3400640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0bcc0ccc60 0x7f0bcc1500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:43.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.683+0000 7f0bd3400640 1 --2- 172.21.15.44:0/377548967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bcc071740 0x7f0bcc0b3a20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:43.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.683+0000 7f0bd3400640 1 -- 172.21.15.44:0/377548967 >> 172.21.15.44:0/377548967 conn(0x7f0bcc0b2060 msgr2=0x7f0bcc0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:43.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.687+0000 7f0bd3400640 1 -- 172.21.15.44:0/377548967 shutdown_connections 2024-09-13T08:42:43.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.687+0000 7f0bd3400640 1 -- 172.21.15.44:0/377548967 wait complete. 2024-09-13T08:42:43.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:42:43.799 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:43.791+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:43.800 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:43.791+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:43.800 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:43.791+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:43.800 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:43.791+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:43.800 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:43.791+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.951+0000 7f4e26a00640 1 -- 172.21.15.44:0/49272513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e20073f10 msgr2=0x7f4e200c6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.951+0000 7f4e26a00640 1 --2- 172.21.15.44:0/49272513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e20073f10 0x7f4e200c6530 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4e2007af00 tx=0x7f4e1801c8c0 comp rx=0 tx=0).stop 2024-09-13T08:42:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.951+0000 7f4e26a00640 1 -- 172.21.15.44:0/49272513 shutdown_connections 2024-09-13T08:42:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.951+0000 7f4e26a00640 1 --2- 172.21.15.44:0/49272513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e20073f10 0x7f4e200c6530 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.951+0000 7f4e26a00640 1 --2- 172.21.15.44:0/49272513 >> v2:172.21.15.129:3300/0 conn(0x7f4e2015bb80 0x7f4e2006f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.951+0000 7f4e26a00640 1 -- 172.21.15.44:0/49272513 >> 172.21.15.44:0/49272513 conn(0x7f4e200d0b30 msgr2=0x7f4e200d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:43.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.951+0000 7f4e26a00640 1 -- 172.21.15.44:0/49272513 shutdown_connections 2024-09-13T08:42:43.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.951+0000 7f4e26a00640 1 -- 172.21.15.44:0/49272513 wait complete. 2024-09-13T08:42:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.955+0000 7f4e26a00640 1 Processor -- start 2024-09-13T08:42:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.955+0000 7f4e26a00640 1 -- start start 2024-09-13T08:42:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.955+0000 7f4e26a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e20073f10 0x7f4e20155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.955+0000 7f4e26a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e200c60c0 0x7f4e200b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.955+0000 7f4e26a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e2015bb80 0x7f4e200b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.955+0000 7f4e26a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2006cfb0 con 0x7f4e200c60c0 2024-09-13T08:42:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.955+0000 7f4e26a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2006ce30 con 0x7f4e20073f10 2024-09-13T08:42:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.955+0000 7f4e26a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2006d130 con 0x7f4e2015bb80 2024-09-13T08:42:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e24c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e200c60c0 0x7f4e200b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e24c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e200c60c0 0x7f4e200b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52394/0 (socket says 172.21.15.44:52394) 2024-09-13T08:42:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e24c00640 1 -- 172.21.15.44:0/2802720942 learned_addr learned my addr 172.21.15.44:0/2802720942 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e24c00640 1 -- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e2015bb80 msgr2=0x7f4e200b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e24c00640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e2015bb80 0x7f4e200b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e24c00640 1 -- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e20073f10 msgr2=0x7f4e20155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e24c00640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e20073f10 0x7f4e20155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e24c00640 1 -- 172.21.15.44:0/2802720942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e200c5a70 con 0x7f4e200c60c0 2024-09-13T08:42:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e24c00640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e200c60c0 0x7f4e200b5be0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00c450 tx=0x7f4e0c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e1d600640 1 -- 172.21.15.44:0/2802720942 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c012830 con 0x7f4e200c60c0 2024-09-13T08:42:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e26a00640 1 -- 172.21.15.44:0/2802720942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e20148b40 con 0x7f4e200c60c0 2024-09-13T08:42:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e1d600640 1 -- 172.21.15.44:0/2802720942 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e0c017070 con 0x7f4e200c60c0 2024-09-13T08:42:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e1d600640 1 -- 172.21.15.44:0/2802720942 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c012c10 con 0x7f4e200c60c0 2024-09-13T08:42:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e26a00640 1 -- 172.21.15.44:0/2802720942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e200b6660 con 0x7f4e200c60c0 2024-09-13T08:42:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.959+0000 7f4e26a00640 1 -- 172.21.15.44:0/2802720942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dfc0052a0 con 0x7f4e200c60c0 2024-09-13T08:42:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.963+0000 7f4e1d600640 1 -- 172.21.15.44:0/2802720942 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 97641+0+0 (secure 0 0 0) 0x7f4e0c011380 con 0x7f4e200c60c0 2024-09-13T08:42:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.963+0000 7f4e1d600640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e000734f0 0x7f4e000759b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.963+0000 7f4e1d600640 1 -- 172.21.15.44:0/2802720942 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f4e0c094720 con 0x7f4e200c60c0 2024-09-13T08:42:43.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.971+0000 7f4e25600640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e000734f0 0x7f4e000759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:43.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.971+0000 7f4e25600640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e000734f0 0x7f4e000759b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e14005c50 tx=0x7f4e14005be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:43.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:43.971+0000 7f4e1d600640 1 -- 172.21.15.44:0/2802720942 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4e0c022020 con 0x7f4e200c60c0 2024-09-13T08:42:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.323+0000 7f4e26a00640 1 -- 172.21.15.44:0/2802720942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4dfc005860 con 0x7f4e200c60c0 2024-09-13T08:42:44.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.323+0000 7f4e1d600640 1 -- 172.21.15.44:0/2802720942 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2160 (secure 0 0 0) 0x7f4e0c05ee60 con 0x7f4e200c60c0 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:42:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 335, 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 49, 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:42:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 2585, 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 10738587225, 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 21794078720, 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 751300034560, 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 85, 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 89441471, 2024-09-13T08:42:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 0, 2024-09-13T08:42:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 22 2024-09-13T08:42:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:42:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 28, 2024-09-13T08:42:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:40:56:233184+0000", 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 3, 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 3, 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 5322 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:42:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:42:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:42:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:42:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:42:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:42:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:42:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:42:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:42:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 11, 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:41:48.574493+0000", 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": { 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-ongoing-clones": { 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "1 ongoing clones - average progress is 0.859% (0s)\n [............................] ", 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.0085899997502565384, 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:42:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:42:44.336 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:42:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 -- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e000734f0 msgr2=0x7f4e000759b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e000734f0 0x7f4e000759b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e14005c50 tx=0x7f4e14005be0 comp rx=0 tx=0).stop 2024-09-13T08:42:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 -- 172.21.15.44:0/2802720942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e200c60c0 msgr2=0x7f4e200b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e200c60c0 0x7f4e200b5be0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00c450 tx=0x7f4e0c00c910 comp rx=0 tx=0).stop 2024-09-13T08:42:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 -- 172.21.15.44:0/2802720942 shutdown_connections 2024-09-13T08:42:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e000734f0 0x7f4e000759b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e2015bb80 0x7f4e200b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e200c60c0 0x7f4e200b5be0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 --2- 172.21.15.44:0/2802720942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e20073f10 0x7f4e20155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 -- 172.21.15.44:0/2802720942 >> 172.21.15.44:0/2802720942 conn(0x7f4e200d0b30 msgr2=0x7f4e200b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 -- 172.21.15.44:0/2802720942 shutdown_connections 2024-09-13T08:42:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.327+0000 7f4e26a00640 1 -- 172.21.15.44:0/2802720942 wait complete. 2024-09-13T08:42:44.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone56670 2024-09-13T08:42:44.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.611+0000 7f491fe00640 1 -- 172.21.15.44:0/1130200382 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49000053d0 msgr2=0x7f490000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:44.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.611+0000 7f491fe00640 1 --2- 172.21.15.44:0/1130200382 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49000053d0 0x7f490000b9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49180cd230 tx=0x7f490c01cc20 comp rx=0 tx=0).stop 2024-09-13T08:42:44.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.611+0000 7f491de00640 1 -- 172.21.15.44:0/1130200382 reap_dead start 2024-09-13T08:42:44.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.611+0000 7f491fe00640 1 -- 172.21.15.44:0/1130200382 shutdown_connections 2024-09-13T08:42:44.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.611+0000 7f491fe00640 1 -- 172.21.15.44:0/1130200382 >> 172.21.15.44:0/1130200382 conn(0x7f49180b18f0 msgr2=0x7f49180b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:44.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.615+0000 7f491fe00640 1 -- 172.21.15.44:0/1130200382 shutdown_connections 2024-09-13T08:42:44.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.615+0000 7f491fe00640 1 -- 172.21.15.44:0/1130200382 wait complete. 2024-09-13T08:42:44.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.615+0000 7f491fe00640 1 Processor -- start 2024-09-13T08:42:44.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.615+0000 7f491fe00640 1 -- start start 2024-09-13T08:42:44.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.615+0000 7f491fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49000053d0 0x7f4918159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:44.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.615+0000 7f491fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491815a080 0x7f49181f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:44.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.615+0000 7f491fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49181f96f0 0x7f49180acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:44.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.615+0000 7f491fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49180b48d0 con 0x7f491815a080 2024-09-13T08:42:44.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49180b4750 con 0x7f49000053d0 2024-09-13T08:42:44.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f49180b4a50 con 0x7f49181f96f0 2024-09-13T08:42:44.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491815a080 0x7f49181f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:44.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491de00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49181f96f0 0x7f49180acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:44.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491815a080 0x7f49181f91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52440/0 (socket says 172.21.15.44:52440) 2024-09-13T08:42:44.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491ca00640 1 -- 172.21.15.44:0/2212883782 learned_addr learned my addr 172.21.15.44:0/2212883782 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:44.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491ca00640 1 -- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49181f96f0 msgr2=0x7f49180acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:44.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491ca00640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49181f96f0 0x7f49180acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:44.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491ca00640 1 -- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49000053d0 msgr2=0x7f4918159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:44.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491ca00640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49000053d0 0x7f4918159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:44.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491ca00640 1 -- 172.21.15.44:0/2212883782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4918159890 con 0x7f491815a080 2024-09-13T08:42:44.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491ca00640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491815a080 0x7f49181f91b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f49100013e0 tx=0x7f4910006810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:44.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f4915600640 1 -- 172.21.15.44:0/2212883782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910015070 con 0x7f491815a080 2024-09-13T08:42:44.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491fe00640 1 -- 172.21.15.44:0/2212883782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4918072870 con 0x7f491815a080 2024-09-13T08:42:44.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f4915600640 1 -- 172.21.15.44:0/2212883782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f491000fd60 con 0x7f491815a080 2024-09-13T08:42:44.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f4915600640 1 -- 172.21.15.44:0/2212883782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910011690 con 0x7f491815a080 2024-09-13T08:42:44.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.619+0000 7f491fe00640 1 -- 172.21.15.44:0/2212883782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4918158590 con 0x7f491815a080 2024-09-13T08:42:44.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.623+0000 7f491fe00640 1 -- 172.21.15.44:0/2212883782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4918069410 con 0x7f491815a080 2024-09-13T08:42:44.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.623+0000 7f4915600640 1 -- 172.21.15.44:0/2212883782 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 97641+0+0 (secure 0 0 0) 0x7f4910011830 con 0x7f491815a080 2024-09-13T08:42:44.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.623+0000 7f4915600640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48f8077930 0x7f48f8079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:44.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.623+0000 7f4915600640 1 -- 172.21.15.44:0/2212883782 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f4910095c10 con 0x7f491815a080 2024-09-13T08:42:44.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.631+0000 7f491d400640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48f8077930 0x7f48f8079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.631+0000 7f4915600640 1 -- 172.21.15.44:0/2212883782 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4910060410 con 0x7f491815a080 2024-09-13T08:42:44.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.639+0000 7f491d400640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48f8077930 0x7f48f8079df0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4918074b90 tx=0x7f49040094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:44.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.951+0000 7f4915600640 1 -- 172.21.15.44:0/2212883782 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 19) ==== 97692+0+0 (secure 0 0 0) 0x7f491005bf80 con 0x7f491815a080 2024-09-13T08:42:44.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:44.987+0000 7f491fe00640 1 -- 172.21.15.44:0/2212883782 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone56670", "target": ["mon-mgr", ""]}) -- 0x7f49180cd3d0 con 0x7f48f8077930 2024-09-13T08:42:45.162 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:45.159+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:45.162 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:45.159+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:45.162 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:45.159+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:45.162 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:45.159+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:45.162 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:45.159+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:47.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.051+0000 7f4915600640 1 -- 172.21.15.44:0/2212883782 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7f491005ef70 con 0x7f491815a080 2024-09-13T08:42:47.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f4915600640 1 -- 172.21.15.44:0/2212883782 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f49180cd3d0 con 0x7f48f8077930 2024-09-13T08:42:47.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 -- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48f8077930 msgr2=0x7f48f8079df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48f8077930 0x7f48f8079df0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4918074b90 tx=0x7f49040094f0 comp rx=0 tx=0).stop 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 -- 172.21.15.44:0/2212883782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491815a080 msgr2=0x7f49181f91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491815a080 0x7f49181f91b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f49100013e0 tx=0x7f4910006810 comp rx=0 tx=0).stop 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 -- 172.21.15.44:0/2212883782 shutdown_connections 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48f8077930 0x7f48f8079df0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49181f96f0 0x7f49180acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491815a080 0x7f49181f91b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 --2- 172.21.15.44:0/2212883782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49000053d0 0x7f4918159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 -- 172.21.15.44:0/2212883782 >> 172.21.15.44:0/2212883782 conn(0x7f49180b18f0 msgr2=0x7f4918165a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 -- 172.21.15.44:0/2212883782 shutdown_connections 2024-09-13T08:42:47.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.147+0000 7f491fe00640 1 -- 172.21.15.44:0/2212883782 wait complete. 2024-09-13T08:42:47.179 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone23241 2024-09-13T08:42:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.435+0000 7f7d54a00640 1 -- 172.21.15.44:0/2092596406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d34004e70 msgr2=0x7f7d3400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.435+0000 7f7d54a00640 1 --2- 172.21.15.44:0/2092596406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d34004e70 0x7f7d3400b9d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c640 tx=0x7f7d4801fa80 comp rx=0 tx=0).stop 2024-09-13T08:42:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.435+0000 7f7d4f200640 1 -- 172.21.15.44:0/2092596406 reap_dead start 2024-09-13T08:42:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.435+0000 7f7d54a00640 1 -- 172.21.15.44:0/2092596406 shutdown_connections 2024-09-13T08:42:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.435+0000 7f7d54a00640 1 -- 172.21.15.44:0/2092596406 >> 172.21.15.44:0/2092596406 conn(0x7f7d500d1bc0 msgr2=0x7f7d500d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.435+0000 7f7d54a00640 1 -- 172.21.15.44:0/2092596406 shutdown_connections 2024-09-13T08:42:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.435+0000 7f7d54a00640 1 -- 172.21.15.44:0/2092596406 wait complete. 2024-09-13T08:42:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.439+0000 7f7d54a00640 1 Processor -- start 2024-09-13T08:42:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.439+0000 7f7d54a00640 1 -- start start 2024-09-13T08:42:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d54a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d34004e70 0x7f7d500b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d54a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d501f9860 0x7f7d500b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d54a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d500b70b0 0x7f7d501fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d54a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d500c6f60 con 0x7f7d500b70b0 2024-09-13T08:42:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d54a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d500c6de0 con 0x7f7d34004e70 2024-09-13T08:42:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d54a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7d500c70e0 con 0x7f7d501f9860 2024-09-13T08:42:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d4f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d500b70b0 0x7f7d501fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d4f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d500b70b0 0x7f7d501fc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52464/0 (socket says 172.21.15.44:52464) 2024-09-13T08:42:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d4f200640 1 -- 172.21.15.44:0/2246625730 learned_addr learned my addr 172.21.15.44:0/2246625730 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d4f200640 1 -- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d501f9860 msgr2=0x7f7d500b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:47.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d4f200640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d501f9860 0x7f7d500b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:47.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d4f200640 1 -- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d34004e70 msgr2=0x7f7d500b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:47.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d4f200640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d34004e70 0x7f7d500b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:47.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.443+0000 7f7d4f200640 1 -- 172.21.15.44:0/2246625730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d501fce70 con 0x7f7d500b70b0 2024-09-13T08:42:47.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.447+0000 7f7d4f200640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d500b70b0 0x7f7d501fc100 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c730 tx=0x7f7d4801e310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:47.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.447+0000 7f7d46a00640 1 -- 172.21.15.44:0/2246625730 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480011d0 con 0x7f7d500b70b0 2024-09-13T08:42:47.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.447+0000 7f7d46a00640 1 -- 172.21.15.44:0/2246625730 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d48003980 con 0x7f7d500b70b0 2024-09-13T08:42:47.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.447+0000 7f7d54a00640 1 -- 172.21.15.44:0/2246625730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d501f8aa0 con 0x7f7d500b70b0 2024-09-13T08:42:47.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.447+0000 7f7d46a00640 1 -- 172.21.15.44:0/2246625730 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48003d50 con 0x7f7d500b70b0 2024-09-13T08:42:47.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.447+0000 7f7d54a00640 1 -- 172.21.15.44:0/2246625730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d501f9060 con 0x7f7d500b70b0 2024-09-13T08:42:47.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.447+0000 7f7d54a00640 1 -- 172.21.15.44:0/2246625730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d500d2ac0 con 0x7f7d500b70b0 2024-09-13T08:42:47.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.455+0000 7f7d46a00640 1 -- 172.21.15.44:0/2246625730 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7f7d480a9650 con 0x7f7d500b70b0 2024-09-13T08:42:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.455+0000 7f7d46a00640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d2c073610 0x7f7d2c075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.455+0000 7f7d46a00640 1 -- 172.21.15.44:0/2246625730 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f7d48132330 con 0x7f7d500b70b0 2024-09-13T08:42:47.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.455+0000 7f7d46a00640 1 -- 172.21.15.44:0/2246625730 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7d48019830 con 0x7f7d500b70b0 2024-09-13T08:42:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.463+0000 7f7d4e800640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d2c073610 0x7f7d2c075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.547+0000 7f7d4e800640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d2c073610 0x7f7d2c075ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000a820 tx=0x7f7d40005af0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:47.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:47.779+0000 7f7d54a00640 1 -- 172.21.15.44:0/2246625730 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone23241", "target": ["mon-mgr", ""]}) -- 0x7f7d5006a170 con 0x7f7d2c073610 2024-09-13T08:42:48.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.767+0000 7f7d46a00640 1 -- 172.21.15.44:0/2246625730 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7d5006a170 con 0x7f7d2c073610 2024-09-13T08:42:48.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.771+0000 7f7d54a00640 1 -- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d2c073610 msgr2=0x7f7d2c075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:48.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.771+0000 7f7d54a00640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d2c073610 0x7f7d2c075ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000a820 tx=0x7f7d40005af0 comp rx=0 tx=0).stop 2024-09-13T08:42:48.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.771+0000 7f7d54a00640 1 -- 172.21.15.44:0/2246625730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d500b70b0 msgr2=0x7f7d501fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:48.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.771+0000 7f7d54a00640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d500b70b0 0x7f7d501fc100 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c730 tx=0x7f7d4801e310 comp rx=0 tx=0).stop 2024-09-13T08:42:48.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.771+0000 7f7d54a00640 1 -- 172.21.15.44:0/2246625730 shutdown_connections 2024-09-13T08:42:48.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.771+0000 7f7d54a00640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d2c073610 0x7f7d2c075ad0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:48.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.771+0000 7f7d54a00640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d500b70b0 0x7f7d501fc100 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:48.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.771+0000 7f7d54a00640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d501f9860 0x7f7d500b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:48.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.771+0000 7f7d54a00640 1 --2- 172.21.15.44:0/2246625730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d34004e70 0x7f7d500b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:48.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.771+0000 7f7d54a00640 1 -- 172.21.15.44:0/2246625730 >> 172.21.15.44:0/2246625730 conn(0x7f7d500d1bc0 msgr2=0x7f7d500c49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:48.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.775+0000 7f7d54a00640 1 -- 172.21.15.44:0/2246625730 shutdown_connections 2024-09-13T08:42:48.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:48.775+0000 7f7d54a00640 1 -- 172.21.15.44:0/2246625730 wait complete. 2024-09-13T08:42:48.809 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone53352 2024-09-13T08:42:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.007+0000 7efc9ca00640 1 -- 172.21.15.44:0/2683056412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc98074540 msgr2=0x7efc9806f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:49.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.007+0000 7efc9ca00640 1 --2- 172.21.15.44:0/2683056412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc98074540 0x7efc9806f300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efc90009c80 tx=0x7efc9001f5a0 comp rx=0 tx=0).stop 2024-09-13T08:42:49.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.007+0000 7efc9ca00640 1 -- 172.21.15.44:0/2683056412 shutdown_connections 2024-09-13T08:42:49.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.007+0000 7efc9ca00640 1 --2- 172.21.15.44:0/2683056412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc98074540 0x7efc9806f300 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:49.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.007+0000 7efc9ca00640 1 --2- 172.21.15.44:0/2683056412 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc9815a740 0x7efc9815ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:49.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.007+0000 7efc9ca00640 1 -- 172.21.15.44:0/2683056412 >> 172.21.15.44:0/2683056412 conn(0x7efc9806cc20 msgr2=0x7efc9806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:49.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.007+0000 7efc9ca00640 1 -- 172.21.15.44:0/2683056412 shutdown_connections 2024-09-13T08:42:49.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.007+0000 7efc9ca00640 1 -- 172.21.15.44:0/2683056412 wait complete. 2024-09-13T08:42:49.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.011+0000 7efc9ca00640 1 Processor -- start 2024-09-13T08:42:49.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.011+0000 7efc9ca00640 1 -- start start 2024-09-13T08:42:49.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.011+0000 7efc9ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc98074540 0x7efc9814ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.011+0000 7efc9ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc9815a740 0x7efc9815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.011+0000 7efc9ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc9815b060 0x7efc981458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.011+0000 7efc9ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efc980d0470 con 0x7efc9815b060 2024-09-13T08:42:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.011+0000 7efc9ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc980d02f0 con 0x7efc98074540 2024-09-13T08:42:49.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.011+0000 7efc9ca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7efc9814f5c0 con 0x7efc9815a740 2024-09-13T08:42:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.011+0000 7efc97e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc9815b060 0x7efc981458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc97e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc9815b060 0x7efc981458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52484/0 (socket says 172.21.15.44:52484) 2024-09-13T08:42:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc97e00640 1 -- 172.21.15.44:0/692842611 learned_addr learned my addr 172.21.15.44:0/692842611 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc97e00640 1 -- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc9815a740 msgr2=0x7efc9815c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc97e00640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc9815a740 0x7efc9815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc97e00640 1 -- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc98074540 msgr2=0x7efc9814ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc97e00640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc98074540 0x7efc9814ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc97e00640 1 -- 172.21.15.44:0/692842611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9814f810 con 0x7efc9815b060 2024-09-13T08:42:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc97e00640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc9815b060 0x7efc981458f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efc90009bd0 tx=0x7efc900af410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc8f400640 1 -- 172.21.15.44:0/692842611 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900afaa0 con 0x7efc9815b060 2024-09-13T08:42:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc8f400640 1 -- 172.21.15.44:0/692842611 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc900af600 con 0x7efc9815b060 2024-09-13T08:42:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc9ca00640 1 -- 172.21.15.44:0/692842611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9814faf0 con 0x7efc9815b060 2024-09-13T08:42:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc8f400640 1 -- 172.21.15.44:0/692842611 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90004000 con 0x7efc9815b060 2024-09-13T08:42:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc9ca00640 1 -- 172.21.15.44:0/692842611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9815bcf0 con 0x7efc9815b060 2024-09-13T08:42:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.015+0000 7efc9ca00640 1 -- 172.21.15.44:0/692842611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc9814fd30 con 0x7efc9815b060 2024-09-13T08:42:49.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.019+0000 7efc8f400640 1 -- 172.21.15.44:0/692842611 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7efc900041a0 con 0x7efc9815b060 2024-09-13T08:42:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.019+0000 7efc8f400640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc74073640 0x7efc74075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.019+0000 7efc8f400640 1 -- 172.21.15.44:0/692842611 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7efc90133e40 con 0x7efc9815b060 2024-09-13T08:42:49.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.027+0000 7efc8f400640 1 -- 172.21.15.44:0/692842611 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7efc9814fd30 con 0x7efc9815b060 2024-09-13T08:42:49.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.043+0000 7efc97400640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc74073640 0x7efc74075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:49.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.047+0000 7efc97400640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc74073640 0x7efc74075b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc880058d0 tx=0x7efc88005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:49.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.407+0000 7efc9ca00640 1 -- 172.21.15.44:0/692842611 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone53352", "target": ["mon-mgr", ""]}) -- 0x7efc9814f260 con 0x7efc74073640 2024-09-13T08:42:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.967+0000 7efc8f400640 1 -- 172.21.15.44:0/692842611 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc8800db50 con 0x7efc74073640 2024-09-13T08:42:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 -- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc74073640 msgr2=0x7efc74075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc74073640 0x7efc74075b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc880058d0 tx=0x7efc88005860 comp rx=0 tx=0).stop 2024-09-13T08:42:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 -- 172.21.15.44:0/692842611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc9815b060 msgr2=0x7efc981458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc9815b060 0x7efc981458f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efc90009bd0 tx=0x7efc900af410 comp rx=0 tx=0).stop 2024-09-13T08:42:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 -- 172.21.15.44:0/692842611 shutdown_connections 2024-09-13T08:42:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc74073640 0x7efc74075b00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc9815b060 0x7efc981458f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc9815a740 0x7efc9815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 --2- 172.21.15.44:0/692842611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc98074540 0x7efc9814ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 -- 172.21.15.44:0/692842611 >> 172.21.15.44:0/692842611 conn(0x7efc9806cc20 msgr2=0x7efc9806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.971+0000 7efc9ca00640 1 -- 172.21.15.44:0/692842611 shutdown_connections 2024-09-13T08:42:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:49.975+0000 7efc9ca00640 1 -- 172.21.15.44:0/692842611 wait complete. 2024-09-13T08:42:50.021 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone57693 2024-09-13T08:42:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 -- 172.21.15.44:0/3248845151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be41613c0 msgr2=0x7f2be4161bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 --2- 172.21.15.44:0/3248845151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be41613c0 0x7f2be4161bf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2be40cf8a0 tx=0x7f2bd801cc30 comp rx=0 tx=0).stop 2024-09-13T08:42:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 -- 172.21.15.44:0/3248845151 shutdown_connections 2024-09-13T08:42:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 --2- 172.21.15.44:0/3248845151 >> v2:172.21.15.129:3300/0 conn(0x7f2be40b3190 0x7f2be4162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 --2- 172.21.15.44:0/3248845151 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be41617a0 0x7f2be4162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 --2- 172.21.15.44:0/3248845151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be41613c0 0x7f2be4161bf0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 -- 172.21.15.44:0/3248845151 >> 172.21.15.44:0/3248845151 conn(0x7f2be40d1dd0 msgr2=0x7f2be40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 -- 172.21.15.44:0/3248845151 shutdown_connections 2024-09-13T08:42:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 -- 172.21.15.44:0/3248845151 wait complete. 2024-09-13T08:42:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 Processor -- start 2024-09-13T08:42:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 -- start start 2024-09-13T08:42:50.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be40b3190 0x7f2be41500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:50.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be41613c0 0x7f2be41faf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:50.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be41617a0 0x7f2be41fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:50.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2be40b17e0 con 0x7f2be41617a0 2024-09-13T08:42:50.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2be40b1660 con 0x7f2be41613c0 2024-09-13T08:42:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2beba00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2be40b1960 con 0x7f2be40b3190 2024-09-13T08:42:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2be9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be41617a0 0x7f2be41fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2be9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be41617a0 0x7f2be41fb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52500/0 (socket says 172.21.15.44:52500) 2024-09-13T08:42:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.251+0000 7f2be9a00640 1 -- 172.21.15.44:0/3852347567 learned_addr learned my addr 172.21.15.44:0/3852347567 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2be9a00640 1 -- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be40b3190 msgr2=0x7f2be41500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2be9a00640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be40b3190 0x7f2be41500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2be9a00640 1 -- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be41613c0 msgr2=0x7f2be41faf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2be9a00640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be41613c0 0x7f2be41faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2be9a00640 1 -- 172.21.15.44:0/3852347567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be4150b10 con 0x7f2be41617a0 2024-09-13T08:42:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2be9a00640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be41617a0 0x7f2be41fb4b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00d750 tx=0x7f2bdc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2be0c00640 1 -- 172.21.15.44:0/3852347567 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc017070 con 0x7f2be41617a0 2024-09-13T08:42:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2be0c00640 1 -- 172.21.15.44:0/3852347567 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bdc009d20 con 0x7f2be41617a0 2024-09-13T08:42:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2be0c00640 1 -- 172.21.15.44:0/3852347567 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00de60 con 0x7f2be41617a0 2024-09-13T08:42:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2beba00640 1 -- 172.21.15.44:0/3852347567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be41fbd80 con 0x7f2be41617a0 2024-09-13T08:42:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2beba00640 1 -- 172.21.15.44:0/3852347567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be40c9180 con 0x7f2be41617a0 2024-09-13T08:42:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.255+0000 7f2beba00640 1 -- 172.21.15.44:0/3852347567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be40c9380 con 0x7f2be41617a0 2024-09-13T08:42:50.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.259+0000 7f2be0c00640 1 -- 172.21.15.44:0/3852347567 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7f2bdc005280 con 0x7f2be41617a0 2024-09-13T08:42:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.259+0000 7f2be0c00640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2bc4073610 0x7f2bc4075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.259+0000 7f2be0c00640 1 -- 172.21.15.44:0/3852347567 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f2bdc095280 con 0x7f2be41617a0 2024-09-13T08:42:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.259+0000 7f2be9000640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2bc4073610 0x7f2bc4075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.267+0000 7f2be0c00640 1 -- 172.21.15.44:0/3852347567 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2be40c9380 con 0x7f2be41617a0 2024-09-13T08:42:50.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.271+0000 7f2be9000640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2bc4073610 0x7f2bc4075ad0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80af000 tx=0x7f2bd80199d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.595+0000 7f2beba00640 1 -- 172.21.15.44:0/3852347567 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone57693", "target": ["mon-mgr", ""]}) -- 0x7f2be40b7c60 con 0x7f2bc4073610 2024-09-13T08:42:50.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2be0c00640 1 -- 172.21.15.44:0/3852347567 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2be40b7c60 con 0x7f2bc4073610 2024-09-13T08:42:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 -- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2bc4073610 msgr2=0x7f2bc4075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2bc4073610 0x7f2bc4075ad0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80af000 tx=0x7f2bd80199d0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 -- 172.21.15.44:0/3852347567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be41617a0 msgr2=0x7f2be41fb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be41617a0 0x7f2be41fb4b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00d750 tx=0x7f2bdc00dc10 comp rx=0 tx=0).stop 2024-09-13T08:42:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 -- 172.21.15.44:0/3852347567 shutdown_connections 2024-09-13T08:42:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2bc4073610 0x7f2bc4075ad0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be41617a0 0x7f2be41fb4b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be41613c0 0x7f2be41faf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 --2- 172.21.15.44:0/3852347567 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2be40b3190 0x7f2be41500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 -- 172.21.15.44:0/3852347567 >> 172.21.15.44:0/3852347567 conn(0x7f2be40d1dd0 msgr2=0x7f2be40af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:50.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 -- 172.21.15.44:0/3852347567 shutdown_connections 2024-09-13T08:42:50.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.755+0000 7f2beba00640 1 -- 172.21.15.44:0/3852347567 wait complete. 2024-09-13T08:42:50.781 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:42:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.923+0000 7ff5ba400640 1 -- 172.21.15.44:0/1719915977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5b415bea0 msgr2=0x7ff5b40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.923+0000 7ff5ba400640 1 --2- 172.21.15.44:0/1719915977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5b415bea0 0x7ff5b40ce670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009840 tx=0x7ff5a401c790 comp rx=0 tx=0).stop 2024-09-13T08:42:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.923+0000 7ff5ba400640 1 -- 172.21.15.44:0/1719915977 shutdown_connections 2024-09-13T08:42:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.923+0000 7ff5ba400640 1 --2- 172.21.15.44:0/1719915977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5b415bea0 0x7ff5b40ce670 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.923+0000 7ff5ba400640 1 -- 172.21.15.44:0/1719915977 >> 172.21.15.44:0/1719915977 conn(0x7ff5b40d1dc0 msgr2=0x7ff5b40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.923+0000 7ff5ba400640 1 -- 172.21.15.44:0/1719915977 shutdown_connections 2024-09-13T08:42:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.923+0000 7ff5ba400640 1 -- 172.21.15.44:0/1719915977 wait complete. 2024-09-13T08:42:50.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 Processor -- start 2024-09-13T08:42:50.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 -- start start 2024-09-13T08:42:50.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5b40cdb30 0x7ff5b4150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b40d28e0 0x7ff5b40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5b415bea0 0x7ff5b40c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b416a930 con 0x7ff5b415bea0 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b416a7b0 con 0x7ff5b40d28e0 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b416aab0 con 0x7ff5b40cdb30 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5b415bea0 0x7ff5b40c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5b415bea0 0x7ff5b40c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52518/0 (socket says 172.21.15.44:52518) 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5b3e00640 1 -- 172.21.15.44:0/320023300 learned_addr learned my addr 172.21.15.44:0/320023300 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5abe00640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b40d28e0 0x7ff5b40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5b3400640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5b40cdb30 0x7ff5b4150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5b3e00640 1 -- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5b40cdb30 msgr2=0x7ff5b4150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5b3e00640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5b40cdb30 0x7ff5b4150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5b3e00640 1 -- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b40d28e0 msgr2=0x7ff5b40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:50.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5b3e00640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b40d28e0 0x7ff5b40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:50.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5b3e00640 1 -- 172.21.15.44:0/320023300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b406de60 con 0x7ff5b415bea0 2024-09-13T08:42:50.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5b3e00640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5b415bea0 0x7ff5b40c7ca0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00c7c0 tx=0x7ff5ac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:50.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ab400640 1 -- 172.21.15.44:0/320023300 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac003b80 con 0x7ff5b415bea0 2024-09-13T08:42:50.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ab400640 1 -- 172.21.15.44:0/320023300 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5ac003d20 con 0x7ff5b415bea0 2024-09-13T08:42:50.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ab400640 1 -- 172.21.15.44:0/320023300 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac00aab0 con 0x7ff5b415bea0 2024-09-13T08:42:50.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 -- 172.21.15.44:0/320023300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b406e140 con 0x7ff5b415bea0 2024-09-13T08:42:50.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 -- 172.21.15.44:0/320023300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b4072330 con 0x7ff5b415bea0 2024-09-13T08:42:50.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.927+0000 7ff5ba400640 1 -- 172.21.15.44:0/320023300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b40ce670 con 0x7ff5b415bea0 2024-09-13T08:42:50.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.931+0000 7ff5ab400640 1 -- 172.21.15.44:0/320023300 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7ff5ac00ac50 con 0x7ff5b415bea0 2024-09-13T08:42:50.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.931+0000 7ff5ab400640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff590073640 0x7ff590075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:50.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.931+0000 7ff5ab400640 1 -- 172.21.15.44:0/320023300 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7ff5ac094de0 con 0x7ff5b415bea0 2024-09-13T08:42:50.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.931+0000 7ff5b3400640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff590073640 0x7ff590075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:50.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.931+0000 7ff5b3400640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff590073640 0x7ff590075b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40af000 tx=0x7ff5a401afd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:50.935+0000 7ff5ab400640 1 -- 172.21.15.44:0/320023300 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff5ac09b080 con 0x7ff5b415bea0 2024-09-13T08:42:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.135+0000 7ff5ba400640 1 -- 172.21.15.44:0/320023300 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff5b40727a0 con 0x7ff590073640 2024-09-13T08:42:51.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.135+0000 7ff5ab400640 1 -- 172.21.15.44:0/320023300 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff5b40727a0 con 0x7ff590073640 2024-09-13T08:42:51.139 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:42:51.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:42:51.139 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:42:51.139 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:42:51.140 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:42:51.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 -- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff590073640 msgr2=0x7ff590075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:51.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff590073640 0x7ff590075b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40af000 tx=0x7ff5a401afd0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 -- 172.21.15.44:0/320023300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5b415bea0 msgr2=0x7ff5b40c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:51.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5b415bea0 0x7ff5b40c7ca0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00c7c0 tx=0x7ff5ac00cc80 comp rx=0 tx=0).stop 2024-09-13T08:42:51.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 -- 172.21.15.44:0/320023300 shutdown_connections 2024-09-13T08:42:51.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff590073640 0x7ff590075b00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5b415bea0 0x7ff5b40c7ca0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b40d28e0 0x7ff5b40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 --2- 172.21.15.44:0/320023300 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5b40cdb30 0x7ff5b4150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 -- 172.21.15.44:0/320023300 >> 172.21.15.44:0/320023300 conn(0x7ff5b40d1dc0 msgr2=0x7ff5b40b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:51.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 -- 172.21.15.44:0/320023300 shutdown_connections 2024-09-13T08:42:51.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.139+0000 7ff5ba400640 1 -- 172.21.15.44:0/320023300 wait complete. 2024-09-13T08:42:51.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-13T08:42:51.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 -- 172.21.15.44:0/1127096700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f424c0ccea0 msgr2=0x7f424c0cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:51.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 --2- 172.21.15.44:0/1127096700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f424c0ccea0 0x7f424c0cd280 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f424c0751c0 tx=0x7f423c020320 comp rx=0 tx=0).stop 2024-09-13T08:42:51.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 -- 172.21.15.44:0/1127096700 shutdown_connections 2024-09-13T08:42:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 --2- 172.21.15.44:0/1127096700 >> v2:172.21.15.129:3300/0 conn(0x7f424c06fb90 0x7f424c0caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 --2- 172.21.15.44:0/1127096700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f424c0ccea0 0x7f424c0cd280 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 -- 172.21.15.44:0/1127096700 >> 172.21.15.44:0/1127096700 conn(0x7f424c0b2060 msgr2=0x7f424c0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 -- 172.21.15.44:0/1127096700 shutdown_connections 2024-09-13T08:42:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 -- 172.21.15.44:0/1127096700 wait complete. 2024-09-13T08:42:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 Processor -- start 2024-09-13T08:42:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 -- start start 2024-09-13T08:42:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f424c06fb90 0x7f424c15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f424c0ccea0 0x7f424c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f424c15aa30 0x7f424c0c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f424c068e30 con 0x7f424c06fb90 2024-09-13T08:42:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f424c068cb0 con 0x7f424c15aa30 2024-09-13T08:42:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f4252400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f424c068fb0 con 0x7f424c0ccea0 2024-09-13T08:42:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f424b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f424c06fb90 0x7f424c15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f424b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f424c06fb90 0x7f424c15ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52534/0 (socket says 172.21.15.44:52534) 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.311+0000 7f424b400640 1 -- 172.21.15.44:0/3062315574 learned_addr learned my addr 172.21.15.44:0/3062315574 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f424be00640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f424c15aa30 0x7f424c0c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f4243e00640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f424c0ccea0 0x7f424c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f424b400640 1 -- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f424c0ccea0 msgr2=0x7f424c0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f424b400640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f424c0ccea0 0x7f424c0aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f424b400640 1 -- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f424c15aa30 msgr2=0x7f424c0c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f424b400640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f424c15aa30 0x7f424c0c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f424b400640 1 -- 172.21.15.44:0/3062315574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424c0af2d0 con 0x7f424c06fb90 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f424b400640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f424c06fb90 0x7f424c15ae30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f423c000c10 tx=0x7f423c0b0480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f4243400640 1 -- 172.21.15.44:0/3062315574 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c0b5050 con 0x7f424c06fb90 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f4243400640 1 -- 172.21.15.44:0/3062315574 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f423c0c1070 con 0x7f424c06fb90 2024-09-13T08:42:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f4243400640 1 -- 172.21.15.44:0/3062315574 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c003840 con 0x7f424c06fb90 2024-09-13T08:42:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f4252400640 1 -- 172.21.15.44:0/3062315574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424c0c61e0 con 0x7f424c06fb90 2024-09-13T08:42:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f4252400640 1 -- 172.21.15.44:0/3062315574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424c0c67a0 con 0x7f424c06fb90 2024-09-13T08:42:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f4252400640 1 -- 172.21.15.44:0/3062315574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424c0caf80 con 0x7f424c06fb90 2024-09-13T08:42:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.315+0000 7f4243400640 1 -- 172.21.15.44:0/3062315574 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7f423c0039e0 con 0x7f424c06fb90 2024-09-13T08:42:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.319+0000 7f4243400640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4228073610 0x7f4228075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.319+0000 7f4243400640 1 -- 172.21.15.44:0/3062315574 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f423c133890 con 0x7f424c06fb90 2024-09-13T08:42:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.319+0000 7f4243e00640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4228073610 0x7f4228075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.319+0000 7f4243e00640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4228073610 0x7f4228075ad0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f424c1f82a0 tx=0x7f4234009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.323+0000 7f4243400640 1 -- 172.21.15.44:0/3062315574 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f423c0a9e20 con 0x7f424c06fb90 2024-09-13T08:42:51.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.523+0000 7f4252400640 1 -- 172.21.15.44:0/3062315574 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f424c0cd560 con 0x7f4228073610 2024-09-13T08:42:51.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.531+0000 7f4243400640 1 -- 172.21.15.44:0/3062315574 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+234 (secure 0 0 0) 0x7f424c0cd560 con 0x7f4228073610 2024-09-13T08:42:51.535 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:42:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:42:51.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone23241" 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sv1" 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone53352" 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone57693" 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone56670" 2024-09-13T08:42:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:42:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:42:51.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 -- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4228073610 msgr2=0x7f4228075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4228073610 0x7f4228075ad0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f424c1f82a0 tx=0x7f4234009040 comp rx=0 tx=0).stop 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 -- 172.21.15.44:0/3062315574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f424c06fb90 msgr2=0x7f424c15ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f424c06fb90 0x7f424c15ae30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f423c000c10 tx=0x7f423c0b0480 comp rx=0 tx=0).stop 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 -- 172.21.15.44:0/3062315574 shutdown_connections 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4228073610 0x7f4228075ad0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f424c15aa30 0x7f424c0c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f424c0ccea0 0x7f424c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 --2- 172.21.15.44:0/3062315574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f424c06fb90 0x7f424c15ae30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 -- 172.21.15.44:0/3062315574 >> 172.21.15.44:0/3062315574 conn(0x7f424c0b2060 msgr2=0x7f424c0afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 -- 172.21.15.44:0/3062315574 shutdown_connections 2024-09-13T08:42:51.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.535+0000 7f4252400640 1 -- 172.21.15.44:0/3062315574 wait complete. 2024-09-13T08:42:51.556 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone23241 2024-09-13T08:42:51.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.707+0000 7f3b32c00640 1 -- 172.21.15.44:0/568428942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b2c15a9d0 msgr2=0x7f3b2c0ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:51.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.707+0000 7f3b32c00640 1 --2- 172.21.15.44:0/568428942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b2c15a9d0 0x7f3b2c0ce350 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400a950 tx=0x7f3b1401db70 comp rx=0 tx=0).stop 2024-09-13T08:42:51.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.707+0000 7f3b32c00640 1 -- 172.21.15.44:0/568428942 shutdown_connections 2024-09-13T08:42:51.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.707+0000 7f3b32c00640 1 --2- 172.21.15.44:0/568428942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b2c15a9d0 0x7f3b2c0ce350 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.707+0000 7f3b32c00640 1 -- 172.21.15.44:0/568428942 >> 172.21.15.44:0/568428942 conn(0x7f3b2c0cfbc0 msgr2=0x7f3b2c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:51.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.707+0000 7f3b32c00640 1 -- 172.21.15.44:0/568428942 shutdown_connections 2024-09-13T08:42:51.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.707+0000 7f3b32c00640 1 -- 172.21.15.44:0/568428942 wait complete. 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.707+0000 7f3b32c00640 1 Processor -- start 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.707+0000 7f3b32c00640 1 -- start start 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b32c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3b2c0734f0 0x7f3b2c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b32c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b2c0ce890 0x7f3b2c148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b32c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b2c15a9d0 0x7f3b2c0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b32c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c06fd00 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b32c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c06fb80 con 0x7f3b2c0ce890 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b32c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c0addd0 con 0x7f3b2c0734f0 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b30c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b2c15a9d0 0x7f3b2c0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b30c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b2c15a9d0 0x7f3b2c0cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52562/0 (socket says 172.21.15.44:52562) 2024-09-13T08:42:51.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b30c00640 1 -- 172.21.15.44:0/2368574836 learned_addr learned my addr 172.21.15.44:0/2368574836 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b23e00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b2c0ce890 0x7f3b2c148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b30c00640 1 -- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3b2c0734f0 msgr2=0x7f3b2c0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b2be00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3b2c0734f0 0x7f3b2c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b30c00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3b2c0734f0 0x7f3b2c0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b30c00640 1 -- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b2c0ce890 msgr2=0x7f3b2c148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b30c00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b2c0ce890 0x7f3b2c148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b30c00640 1 -- 172.21.15.44:0/2368574836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c1fbfb0 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b23e00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b2c0ce890 0x7f3b2c148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b2be00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3b2c0734f0 0x7f3b2c0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b30c00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b2c15a9d0 0x7f3b2c0cc5b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400c760 tx=0x7f3b2400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b28c00640 1 -- 172.21.15.44:0/2368574836 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24003db0 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b28c00640 1 -- 172.21.15.44:0/2368574836 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b24005070 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b32c00640 1 -- 172.21.15.44:0/2368574836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c0ae000 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b28c00640 1 -- 172.21.15.44:0/2368574836 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b240054c0 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b32c00640 1 -- 172.21.15.44:0/2368574836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c1fb7b0 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.711+0000 7f3b32c00640 1 -- 172.21.15.44:0/2368574836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2c069410 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.715+0000 7f3b28c00640 1 -- 172.21.15.44:0/2368574836 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7f3b24005660 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.715+0000 7f3b28c00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3b08073610 0x7f3b08075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:51.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.715+0000 7f3b28c00640 1 -- 172.21.15.44:0/2368574836 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f3b24094f90 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.715+0000 7f3b2be00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3b08073610 0x7f3b08075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:51.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.715+0000 7f3b2be00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3b08073610 0x7f3b08075ad0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c000f70 tx=0x7f3b1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:51.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.719+0000 7f3b28c00640 1 -- 172.21.15.44:0/2368574836 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3b2405f1d0 con 0x7f3b2c15a9d0 2024-09-13T08:42:51.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:51.919+0000 7f3b32c00640 1 -- 172.21.15.44:0/2368574836 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone23241", "target": ["mon-mgr", ""]}) -- 0x7f3b2c0ce350 con 0x7f3b08073610 2024-09-13T08:42:56.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.663+0000 7f3b28c00640 1 -- 172.21.15.44:0/2368574836 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b2c0ce350 con 0x7f3b08073610 2024-09-13T08:42:56.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 -- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3b08073610 msgr2=0x7f3b08075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:56.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3b08073610 0x7f3b08075ad0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c000f70 tx=0x7f3b1c009040 comp rx=0 tx=0).stop 2024-09-13T08:42:56.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 -- 172.21.15.44:0/2368574836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b2c15a9d0 msgr2=0x7f3b2c0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:56.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b2c15a9d0 0x7f3b2c0cc5b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400c760 tx=0x7f3b2400cc20 comp rx=0 tx=0).stop 2024-09-13T08:42:56.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 -- 172.21.15.44:0/2368574836 shutdown_connections 2024-09-13T08:42:56.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3b08073610 0x7f3b08075ad0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:56.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b2c15a9d0 0x7f3b2c0cc5b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:56.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b2c0ce890 0x7f3b2c148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:56.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 --2- 172.21.15.44:0/2368574836 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3b2c0734f0 0x7f3b2c0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:56.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 -- 172.21.15.44:0/2368574836 >> 172.21.15.44:0/2368574836 conn(0x7f3b2c0cfbc0 msgr2=0x7f3b2c075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:56.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 -- 172.21.15.44:0/2368574836 shutdown_connections 2024-09-13T08:42:56.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.667+0000 7f3b32c00640 1 -- 172.21.15.44:0/2368574836 wait complete. 2024-09-13T08:42:56.673 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:56.671+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:56.673 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:56.671+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:56.673 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:56.671+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:56.673 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:56.671+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:56.673 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:56.671+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:56.694 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:56.695+0000 7f8ac7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:56.694 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:56.695+0000 7f8ac7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:56.695 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:56.695+0000 7f8ac7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:56.695 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:56.695+0000 7f8ac7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:56.695 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:56.695+0000 7f8ac7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:56.701 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-13T08:42:56.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.855+0000 7f591de00640 1 -- 172.21.15.44:0/2732409507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59180cf440 msgr2=0x7f59180ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:56.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.855+0000 7f591de00640 1 --2- 172.21.15.44:0/2732409507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59180cf440 0x7f59180ce350 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f590000a950 tx=0x7f590001db70 comp rx=0 tx=0).stop 2024-09-13T08:42:56.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.855+0000 7f591de00640 1 -- 172.21.15.44:0/2732409507 shutdown_connections 2024-09-13T08:42:56.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.855+0000 7f591de00640 1 --2- 172.21.15.44:0/2732409507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59180cf440 0x7f59180ce350 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:56.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.855+0000 7f591de00640 1 --2- 172.21.15.44:0/2732409507 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59180734f0 0x7f591815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:56.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.855+0000 7f591de00640 1 -- 172.21.15.44:0/2732409507 >> 172.21.15.44:0/2732409507 conn(0x7f59180cfbc0 msgr2=0x7f59180cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:56.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.855+0000 7f591de00640 1 -- 172.21.15.44:0/2732409507 shutdown_connections 2024-09-13T08:42:56.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.855+0000 7f591de00640 1 -- 172.21.15.44:0/2732409507 wait complete. 2024-09-13T08:42:56.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.855+0000 7f591de00640 1 Processor -- start 2024-09-13T08:42:56.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.855+0000 7f591de00640 1 -- start start 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f591de00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59180734f0 0x7f591814a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f591de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59180ce890 0x7f59180b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f591de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59180cf440 0x7f591809c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f591de00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59181674e0 con 0x7f59180cf440 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f591de00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5918167360 con 0x7f59180ce890 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f591de00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5918167660 con 0x7f59180734f0 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f5917400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59180cf440 0x7f591809c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f5916a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59180734f0 0x7f591814a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f5917400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59180cf440 0x7f591809c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40938/0 (socket says 172.21.15.44:40938) 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f5917400640 1 -- 172.21.15.44:0/798595420 learned_addr learned my addr 172.21.15.44:0/798595420 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:56.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f590fe00640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59180ce890 0x7f59180b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:56.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f5917400640 1 -- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59180734f0 msgr2=0x7f591814a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:56.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f5917400640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59180734f0 0x7f591814a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:56.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f5917400640 1 -- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59180ce890 msgr2=0x7f59180b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:56.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f5917400640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59180ce890 0x7f59180b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:56.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f5917400640 1 -- 172.21.15.44:0/798595420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591809aea0 con 0x7f59180cf440 2024-09-13T08:42:56.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f5917400640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59180cf440 0x7f591809c4f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f591000c730 tx=0x7f591000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:56.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f590f400640 1 -- 172.21.15.44:0/798595420 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910003db0 con 0x7f59180cf440 2024-09-13T08:42:56.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f590f400640 1 -- 172.21.15.44:0/798595420 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f591000ce40 con 0x7f59180cf440 2024-09-13T08:42:56.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f591de00640 1 -- 172.21.15.44:0/798595420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591809b180 con 0x7f59180cf440 2024-09-13T08:42:56.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f590f400640 1 -- 172.21.15.44:0/798595420 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100052d0 con 0x7f59180cf440 2024-09-13T08:42:56.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f591de00640 1 -- 172.21.15.44:0/798595420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591809b740 con 0x7f59180cf440 2024-09-13T08:42:56.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f591de00640 1 -- 172.21.15.44:0/798595420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5918069410 con 0x7f59180cf440 2024-09-13T08:42:56.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.859+0000 7f590f400640 1 -- 172.21.15.44:0/798595420 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7f5910005470 con 0x7f59180cf440 2024-09-13T08:42:56.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.863+0000 7f590f400640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58f4073610 0x7f58f4075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:56.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.863+0000 7f590f400640 1 -- 172.21.15.44:0/798595420 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f5910094e40 con 0x7f59180cf440 2024-09-13T08:42:56.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.863+0000 7f5916a00640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58f4073610 0x7f58f4075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:56.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.863+0000 7f5916a00640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58f4073610 0x7f58f4075ad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f590800bc00 tx=0x7f590800b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:56.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:56.867+0000 7f590f400640 1 -- 172.21.15.44:0/798595420 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f591009b080 con 0x7f59180cf440 2024-09-13T08:42:57.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.075+0000 7f591de00640 1 -- 172.21.15.44:0/798595420 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f591815a9d0 con 0x7f58f4073610 2024-09-13T08:42:57.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.087+0000 7f590f400640 1 -- 172.21.15.44:0/798595420 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f591815a9d0 con 0x7f58f4073610 2024-09-13T08:42:57.093 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:42:57.093 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:42:57.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:42:57.093 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1" 2024-09-13T08:42:57.093 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:42:57.093 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:42:57.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 -- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58f4073610 msgr2=0x7f58f4075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58f4073610 0x7f58f4075ad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f590800bc00 tx=0x7f590800b390 comp rx=0 tx=0).stop 2024-09-13T08:42:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 -- 172.21.15.44:0/798595420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59180cf440 msgr2=0x7f591809c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59180cf440 0x7f591809c4f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f591000c730 tx=0x7f591000cbf0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 -- 172.21.15.44:0/798595420 shutdown_connections 2024-09-13T08:42:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58f4073610 0x7f58f4075ad0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59180cf440 0x7f591809c4f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59180ce890 0x7f59180b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 --2- 172.21.15.44:0/798595420 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59180734f0 0x7f591814a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 -- 172.21.15.44:0/798595420 >> 172.21.15.44:0/798595420 conn(0x7f59180cfbc0 msgr2=0x7f591815dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 -- 172.21.15.44:0/798595420 shutdown_connections 2024-09-13T08:42:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.091+0000 7f591de00640 1 -- 172.21.15.44:0/798595420 wait complete. 2024-09-13T08:42:57.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-13T08:42:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 -- 172.21.15.44:0/1810911870 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdb540d2590 msgr2=0x7fdb540d4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 --2- 172.21.15.44:0/1810911870 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdb540d2590 0x7fdb540d4990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009870 tx=0x7fdb4001c830 comp rx=0 tx=0).stop 2024-09-13T08:42:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 -- 172.21.15.44:0/1810911870 shutdown_connections 2024-09-13T08:42:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 --2- 172.21.15.44:0/1810911870 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdb540d2590 0x7fdb540d4990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 -- 172.21.15.44:0/1810911870 >> 172.21.15.44:0/1810911870 conn(0x7fdb540d3e70 msgr2=0x7fdb540d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 -- 172.21.15.44:0/1810911870 shutdown_connections 2024-09-13T08:42:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 -- 172.21.15.44:0/1810911870 wait complete. 2024-09-13T08:42:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 Processor -- start 2024-09-13T08:42:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 -- start start 2024-09-13T08:42:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb540d2590 0x7fdb5415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdb5415df50 0x7fdb541546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5415ef20 0x7fdb5415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5416bfd0 con 0x7fdb540d2590 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5416be50 con 0x7fdb5415ef20 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.267+0000 7fdb5a000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5416c150 con 0x7fdb5415df50 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb58c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb540d2590 0x7fdb5415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb58c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb540d2590 0x7fdb5415b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40948/0 (socket says 172.21.15.44:40948) 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb58c00640 1 -- 172.21.15.44:0/901268005 learned_addr learned my addr 172.21.15.44:0/901268005 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb53e00640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdb5415df50 0x7fdb541546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb59600640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5415ef20 0x7fdb5415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb58c00640 1 -- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdb5415df50 msgr2=0x7fdb541546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb58c00640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdb5415df50 0x7fdb541546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb58c00640 1 -- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5415ef20 msgr2=0x7fdb5415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb58c00640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5415ef20 0x7fdb5415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb58c00640 1 -- 172.21.15.44:0/901268005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb54154bf0 con 0x7fdb540d2590 2024-09-13T08:42:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb58c00640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb540d2590 0x7fdb5415b530 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c680 tx=0x7fdb4800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb50c00640 1 -- 172.21.15.44:0/901268005 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48016020 con 0x7fdb540d2590 2024-09-13T08:42:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb50c00640 1 -- 172.21.15.44:0/901268005 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb4800a950 con 0x7fdb540d2590 2024-09-13T08:42:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb5a000640 1 -- 172.21.15.44:0/901268005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5415c500 con 0x7fdb540d2590 2024-09-13T08:42:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb50c00640 1 -- 172.21.15.44:0/901268005 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800ada0 con 0x7fdb540d2590 2024-09-13T08:42:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb5a000640 1 -- 172.21.15.44:0/901268005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5415cac0 con 0x7fdb540d2590 2024-09-13T08:42:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb5a000640 1 -- 172.21.15.44:0/901268005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb540d4990 con 0x7fdb540d2590 2024-09-13T08:42:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.271+0000 7fdb50c00640 1 -- 172.21.15.44:0/901268005 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7fdb48019020 con 0x7fdb540d2590 2024-09-13T08:42:57.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.275+0000 7fdb50c00640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdb34073610 0x7fdb34075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:57.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.275+0000 7fdb50c00640 1 -- 172.21.15.44:0/901268005 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fdb48094de0 con 0x7fdb540d2590 2024-09-13T08:42:57.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.275+0000 7fdb53e00640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdb34073610 0x7fdb34075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:57.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.275+0000 7fdb53e00640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdb34073610 0x7fdb34075ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb400af000 tx=0x7fdb4001b070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.279+0000 7fdb50c00640 1 -- 172.21.15.44:0/901268005 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdb48020030 con 0x7fdb540d2590 2024-09-13T08:42:57.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.491+0000 7fdb5a000640 1 -- 172.21.15.44:0/901268005 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb540b6220 con 0x7fdb34073610 2024-09-13T08:42:57.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.523+0000 7fdb50c00640 1 -- 172.21.15.44:0/901268005 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdb540b6220 con 0x7fdb34073610 2024-09-13T08:42:57.528 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:42:57.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 -- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdb34073610 msgr2=0x7fdb34075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdb34073610 0x7fdb34075ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb400af000 tx=0x7fdb4001b070 comp rx=0 tx=0).stop 2024-09-13T08:42:57.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 -- 172.21.15.44:0/901268005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb540d2590 msgr2=0x7fdb5415b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb540d2590 0x7fdb5415b530 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c680 tx=0x7fdb4800cb40 comp rx=0 tx=0).stop 2024-09-13T08:42:57.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 -- 172.21.15.44:0/901268005 shutdown_connections 2024-09-13T08:42:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdb34073610 0x7fdb34075ad0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5415ef20 0x7fdb5415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdb5415df50 0x7fdb541546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 --2- 172.21.15.44:0/901268005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb540d2590 0x7fdb5415b530 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 -- 172.21.15.44:0/901268005 >> 172.21.15.44:0/901268005 conn(0x7fdb540d3e70 msgr2=0x7fdb540d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 -- 172.21.15.44:0/901268005 shutdown_connections 2024-09-13T08:42:57.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.527+0000 7fdb5a000640 1 -- 172.21.15.44:0/901268005 wait complete. 2024-09-13T08:42:57.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-13T08:42:57.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 -- 172.21.15.44:0/4090756876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe35c003a40 msgr2=0x7fe35c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 --2- 172.21.15.44:0/4090756876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe35c003a40 0x7fe35c003e60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe36800c310 tx=0x7fe36801da20 comp rx=0 tx=0).stop 2024-09-13T08:42:57.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe376800640 1 -- 172.21.15.44:0/4090756876 reap_dead start 2024-09-13T08:42:57.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 -- 172.21.15.44:0/4090756876 shutdown_connections 2024-09-13T08:42:57.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 -- 172.21.15.44:0/4090756876 >> 172.21.15.44:0/4090756876 conn(0x7fe3780cce90 msgr2=0x7fe3780cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:57.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 -- 172.21.15.44:0/4090756876 shutdown_connections 2024-09-13T08:42:57.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 -- 172.21.15.44:0/4090756876 wait complete. 2024-09-13T08:42:57.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 Processor -- start 2024-09-13T08:42:57.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 -- start start 2024-09-13T08:42:57.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe35c003a40 0x7fe378150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:57.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3780b5080 0x7fe3780b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:57.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3780b3360 0x7fe3780b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe378166b30 con 0x7fe3780b3360 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe3780cf9d0 con 0x7fe3780b5080 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.699+0000 7fe37ca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe378166cb0 con 0x7fe35c003a40 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe376800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3780b3360 0x7fe3780b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe376800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3780b3360 0x7fe3780b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40962/0 (socket says 172.21.15.44:40962) 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe376800640 1 -- 172.21.15.44:0/4294686069 learned_addr learned my addr 172.21.15.44:0/4294686069 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe375e00640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe35c003a40 0x7fe378150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe36fe00640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3780b5080 0x7fe3780b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe376800640 1 -- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe35c003a40 msgr2=0x7fe378150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe376800640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe35c003a40 0x7fe378150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe376800640 1 -- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3780b5080 msgr2=0x7fe3780b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe376800640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3780b5080 0x7fe3780b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe376800640 1 -- 172.21.15.44:0/4294686069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe378150220 con 0x7fe3780b3360 2024-09-13T08:42:57.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe376800640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3780b3360 0x7fe3780b3810 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe3780c63b0 tx=0x7fe370010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:57.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe36e000640 1 -- 172.21.15.44:0/4294686069 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37000ecb0 con 0x7fe3780b3360 2024-09-13T08:42:57.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe36e000640 1 -- 172.21.15.44:0/4294686069 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe370012450 con 0x7fe3780b3360 2024-09-13T08:42:57.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe37ca00640 1 -- 172.21.15.44:0/4294686069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3780b5d50 con 0x7fe3780b3360 2024-09-13T08:42:57.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe36e000640 1 -- 172.21.15.44:0/4294686069 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3700128a0 con 0x7fe3780b3360 2024-09-13T08:42:57.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe37ca00640 1 -- 172.21.15.44:0/4294686069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3781504c0 con 0x7fe3780b3360 2024-09-13T08:42:57.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe37ca00640 1 -- 172.21.15.44:0/4294686069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe378073a10 con 0x7fe3780b3360 2024-09-13T08:42:57.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.703+0000 7fe36e000640 1 -- 172.21.15.44:0/4294686069 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7fe37001a030 con 0x7fe3780b3360 2024-09-13T08:42:57.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.707+0000 7fe36e000640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe354073610 0x7fe354075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:57.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.707+0000 7fe36e000640 1 -- 172.21.15.44:0/4294686069 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fe3700955b0 con 0x7fe3780b3360 2024-09-13T08:42:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.707+0000 7fe375e00640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe354073610 0x7fe354075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.707+0000 7fe375e00640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe354073610 0x7fe354075ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe3680ac000 tx=0x7fe368019e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:57.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.711+0000 7fe36e000640 1 -- 172.21.15.44:0/4294686069 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe37005f570 con 0x7fe3780b3360 2024-09-13T08:42:57.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.911+0000 7fe37ca00640 1 -- 172.21.15.44:0/4294686069 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe378068e70 con 0x7fe354073610 2024-09-13T08:42:57.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.931+0000 7fe36e000640 1 -- 172.21.15.44:0/4294686069 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe378068e70 con 0x7fe354073610 2024-09-13T08:42:57.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 -- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe354073610 msgr2=0x7fe354075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe354073610 0x7fe354075ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe3680ac000 tx=0x7fe368019e00 comp rx=0 tx=0).stop 2024-09-13T08:42:57.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 -- 172.21.15.44:0/4294686069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3780b3360 msgr2=0x7fe3780b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:57.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3780b3360 0x7fe3780b3810 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe3780c63b0 tx=0x7fe370010da0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 -- 172.21.15.44:0/4294686069 shutdown_connections 2024-09-13T08:42:57.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe354073610 0x7fe354075ad0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3780b3360 0x7fe3780b3810 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3780b5080 0x7fe3780b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 --2- 172.21.15.44:0/4294686069 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe35c003a40 0x7fe378150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:57.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 -- 172.21.15.44:0/4294686069 >> 172.21.15.44:0/4294686069 conn(0x7fe3780cce90 msgr2=0x7fe378152320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:57.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 -- 172.21.15.44:0/4294686069 shutdown_connections 2024-09-13T08:42:57.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:57.935+0000 7fe37ca00640 1 -- 172.21.15.44:0/4294686069 wait complete. 2024-09-13T08:42:57.963 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone53352 2024-09-13T08:42:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.111+0000 7fc0be800640 1 -- 172.21.15.44:0/3231079266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0b80734f0 msgr2=0x7fc0b815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.111+0000 7fc0be800640 1 --2- 172.21.15.44:0/3231079266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0b80734f0 0x7fc0b815a9d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800a950 tx=0x7fc0a801db70 comp rx=0 tx=0).stop 2024-09-13T08:42:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.111+0000 7fc0be800640 1 -- 172.21.15.44:0/3231079266 shutdown_connections 2024-09-13T08:42:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.111+0000 7fc0be800640 1 --2- 172.21.15.44:0/3231079266 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0b80cf440 0x7fc0b80ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.111+0000 7fc0be800640 1 --2- 172.21.15.44:0/3231079266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0b80734f0 0x7fc0b815a9d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.111+0000 7fc0be800640 1 -- 172.21.15.44:0/3231079266 >> 172.21.15.44:0/3231079266 conn(0x7fc0b80cfbc0 msgr2=0x7fc0b80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.111+0000 7fc0be800640 1 -- 172.21.15.44:0/3231079266 shutdown_connections 2024-09-13T08:42:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.111+0000 7fc0be800640 1 -- 172.21.15.44:0/3231079266 wait complete. 2024-09-13T08:42:58.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0be800640 1 Processor -- start 2024-09-13T08:42:58.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0be800640 1 -- start start 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0be800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0b80734f0 0x7fc0b814a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0be800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0b80ce890 0x7fc0b809cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0be800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b80cf440 0x7fc0b80b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0be800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8167360 con 0x7fc0b80cf440 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0be800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b815ed10 con 0x7fc0b80734f0 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0be800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b81674e0 con 0x7fc0b80ce890 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b80cf440 0x7fc0b80b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b80cf440 0x7fc0b80b44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40964/0 (socket says 172.21.15.44:40964) 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7e00640 1 -- 172.21.15.44:0/2176158186 learned_addr learned my addr 172.21.15.44:0/2176158186 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7400640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0b80734f0 0x7fc0b814a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0afe00640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0b80ce890 0x7fc0b809cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7400640 1 -- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0b80ce890 msgr2=0x7fc0b809cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7400640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0b80ce890 0x7fc0b809cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7400640 1 -- 172.21.15.44:0/2176158186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b80cf440 msgr2=0x7fc0b80b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7400640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b80cf440 0x7fc0b80b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7400640 1 -- 172.21.15.44:0/2176158186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b809d630 con 0x7fc0b80734f0 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7e00640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b80cf440 0x7fc0b80b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0b7400640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0b80734f0 0x7fc0b814a220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80ad000 tx=0x7fc0a8005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0af400640 1 -- 172.21.15.44:0/2176158186 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80b3040 con 0x7fc0b80734f0 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.115+0000 7fc0af400640 1 -- 172.21.15.44:0/2176158186 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a8003600 con 0x7fc0b80734f0 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.119+0000 7fc0af400640 1 -- 172.21.15.44:0/2176158186 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8003a10 con 0x7fc0b80734f0 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.119+0000 7fc0be800640 1 -- 172.21.15.44:0/2176158186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b809abb0 con 0x7fc0b80734f0 2024-09-13T08:42:58.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.119+0000 7fc0be800640 1 -- 172.21.15.44:0/2176158186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b809b170 con 0x7fc0b80734f0 2024-09-13T08:42:58.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.119+0000 7fc0be800640 1 -- 172.21.15.44:0/2176158186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b8069410 con 0x7fc0b80734f0 2024-09-13T08:42:58.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.119+0000 7fc0af400640 1 -- 172.21.15.44:0/2176158186 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7fc0a8003bb0 con 0x7fc0b80734f0 2024-09-13T08:42:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.119+0000 7fc0af400640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc094073640 0x7fc094075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.119+0000 7fc0af400640 1 -- 172.21.15.44:0/2176158186 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fc0a812f390 con 0x7fc0b80734f0 2024-09-13T08:42:58.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.119+0000 7fc0afe00640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc094073640 0x7fc094075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.123+0000 7fc0afe00640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc094073640 0x7fc094075b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00096c0 tx=0x7fc0a0009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:58.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.127+0000 7fc0af400640 1 -- 172.21.15.44:0/2176158186 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc0a8017760 con 0x7fc0b80734f0 2024-09-13T08:42:58.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.331+0000 7fc0be800640 1 -- 172.21.15.44:0/2176158186 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone53352", "target": ["mon-mgr", ""]}) -- 0x7fc0b815a9d0 con 0x7fc094073640 2024-09-13T08:42:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0af400640 1 -- 172.21.15.44:0/2176158186 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc0b815a9d0 con 0x7fc094073640 2024-09-13T08:42:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 -- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc094073640 msgr2=0x7fc094075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc094073640 0x7fc094075b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00096c0 tx=0x7fc0a0009260 comp rx=0 tx=0).stop 2024-09-13T08:42:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 -- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0b80734f0 msgr2=0x7fc0b814a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0b80734f0 0x7fc0b814a220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80ad000 tx=0x7fc0a8005780 comp rx=0 tx=0).stop 2024-09-13T08:42:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 -- 172.21.15.44:0/2176158186 shutdown_connections 2024-09-13T08:42:58.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc094073640 0x7fc094075b00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b80cf440 0x7fc0b80b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc0b80ce890 0x7fc0b809cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 --2- 172.21.15.44:0/2176158186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0b80734f0 0x7fc0b814a220 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 -- 172.21.15.44:0/2176158186 >> 172.21.15.44:0/2176158186 conn(0x7fc0b80cfbc0 msgr2=0x7fc0b81545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 -- 172.21.15.44:0/2176158186 shutdown_connections 2024-09-13T08:42:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.355+0000 7fc0be800640 1 -- 172.21.15.44:0/2176158186 wait complete. 2024-09-13T08:42:58.370 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:58.367+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:58.370 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:58.367+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:58.370 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:58.367+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:58.370 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:58.367+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:58.370 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:42:58.367+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:42:58.387 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone57693 2024-09-13T08:42:58.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.535+0000 7f2ee9c00640 1 -- 172.21.15.44:0/2046704779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ee415be10 msgr2=0x7f2ee40ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.535+0000 7f2ee9c00640 1 --2- 172.21.15.44:0/2046704779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ee415be10 0x7f2ee40ce700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40b0fb0 tx=0x7f2ed401f0b0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.535+0000 7f2ee9c00640 1 -- 172.21.15.44:0/2046704779 shutdown_connections 2024-09-13T08:42:58.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.535+0000 7f2ee9c00640 1 --2- 172.21.15.44:0/2046704779 >> v2:172.21.15.129:3300/0 conn(0x7f2ee40b0b60 0x7f2ee40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.535+0000 7f2ee9c00640 1 --2- 172.21.15.44:0/2046704779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ee415be10 0x7f2ee40ce700 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.535+0000 7f2ee9c00640 1 -- 172.21.15.44:0/2046704779 >> 172.21.15.44:0/2046704779 conn(0x7f2ee40d1da0 msgr2=0x7f2ee40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:58.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.535+0000 7f2ee9c00640 1 -- 172.21.15.44:0/2046704779 shutdown_connections 2024-09-13T08:42:58.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.535+0000 7f2ee9c00640 1 -- 172.21.15.44:0/2046704779 wait complete. 2024-09-13T08:42:58.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.535+0000 7f2ee9c00640 1 Processor -- start 2024-09-13T08:42:58.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee9c00640 1 -- start start 2024-09-13T08:42:58.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee9c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ee40b0b60 0x7f2ee40c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ee415be10 0x7f2ee4154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee9c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee415c1f0 0x7f2ee4155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee9c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40cd290 con 0x7f2ee415be10 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee9c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40cd110 con 0x7f2ee415c1f0 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee9c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee40cd410 con 0x7f2ee40b0b60 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ee415be10 0x7f2ee4154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ee415be10 0x7f2ee4154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40988/0 (socket says 172.21.15.44:40988) 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee2000640 1 -- 172.21.15.44:0/3142195145 learned_addr learned my addr 172.21.15.44:0/3142195145 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee2a00640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ee40b0b60 0x7f2ee40c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee3400640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee415c1f0 0x7f2ee4155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee2000640 1 -- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ee40b0b60 msgr2=0x7f2ee40c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee2000640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ee40b0b60 0x7f2ee40c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee2000640 1 -- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee415c1f0 msgr2=0x7f2ee4155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee2000640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee415c1f0 0x7f2ee4155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee2000640 1 -- 172.21.15.44:0/3142195145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee41588c0 con 0x7f2ee415be10 2024-09-13T08:42:58.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee2000640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ee415be10 0x7f2ee4154570 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c3c0 tx=0x7f2ed000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:58.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2edaa00640 1 -- 172.21.15.44:0/3142195145 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0017070 con 0x7f2ee415be10 2024-09-13T08:42:58.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2edaa00640 1 -- 172.21.15.44:0/3142195145 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ed00053c0 con 0x7f2ee415be10 2024-09-13T08:42:58.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2edaa00640 1 -- 172.21.15.44:0/3142195145 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0003860 con 0x7f2ee415be10 2024-09-13T08:42:58.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee9c00640 1 -- 172.21.15.44:0/3142195145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee4158ba0 con 0x7f2ee415be10 2024-09-13T08:42:58.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee9c00640 1 -- 172.21.15.44:0/3142195145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee4159160 con 0x7f2ee415be10 2024-09-13T08:42:58.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.539+0000 7f2ee9c00640 1 -- 172.21.15.44:0/3142195145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee40ccbd0 con 0x7f2ee415be10 2024-09-13T08:42:58.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.543+0000 7f2edaa00640 1 -- 172.21.15.44:0/3142195145 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 97743+0+0 (secure 0 0 0) 0x7f2ed001a070 con 0x7f2ee415be10 2024-09-13T08:42:58.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.543+0000 7f2edaa00640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ec4073610 0x7f2ec4075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.543+0000 7f2edaa00640 1 -- 172.21.15.44:0/3142195145 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f2ed0094340 con 0x7f2ee415be10 2024-09-13T08:42:58.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.543+0000 7f2ee2a00640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ec4073610 0x7f2ec4075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.543+0000 7f2ee2a00640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ec4073610 0x7f2ec4075ad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40b2000 tx=0x7f2ed40aa040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:58.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.547+0000 7f2edaa00640 1 -- 172.21.15.44:0/3142195145 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2ed005def0 con 0x7f2ee415be10 2024-09-13T08:42:58.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.663+0000 7f2edaa00640 1 -- 172.21.15.44:0/3142195145 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f2ed005a160 con 0x7f2ee415be10 2024-09-13T08:42:58.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.755+0000 7f2ee9c00640 1 -- 172.21.15.44:0/3142195145 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone57693", "target": ["mon-mgr", ""]}) -- 0x7f2ee4070350 con 0x7f2ec4073610 2024-09-13T08:42:58.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.775+0000 7f2edaa00640 1 -- 172.21.15.44:0/3142195145 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ee4070350 con 0x7f2ec4073610 2024-09-13T08:42:58.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 -- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ec4073610 msgr2=0x7f2ec4075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ec4073610 0x7f2ec4075ad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40b2000 tx=0x7f2ed40aa040 comp rx=0 tx=0).stop 2024-09-13T08:42:58.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 -- 172.21.15.44:0/3142195145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ee415be10 msgr2=0x7f2ee4154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ee415be10 0x7f2ee4154570 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c3c0 tx=0x7f2ed000c880 comp rx=0 tx=0).stop 2024-09-13T08:42:58.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 -- 172.21.15.44:0/3142195145 shutdown_connections 2024-09-13T08:42:58.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2ec4073610 0x7f2ec4075ad0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee415c1f0 0x7f2ee4155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ee415be10 0x7f2ee4154570 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 --2- 172.21.15.44:0/3142195145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2ee40b0b60 0x7f2ee40c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 -- 172.21.15.44:0/3142195145 >> 172.21.15.44:0/3142195145 conn(0x7f2ee40d1da0 msgr2=0x7f2ee40716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:58.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 -- 172.21.15.44:0/3142195145 shutdown_connections 2024-09-13T08:42:58.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.779+0000 7f2ee9c00640 1 -- 172.21.15.44:0/3142195145 wait complete. 2024-09-13T08:42:58.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone56670 2024-09-13T08:42:58.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.955+0000 7f3bc7600640 1 -- 172.21.15.44:0/881138275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc00734f0 msgr2=0x7f3bc015a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.955+0000 7f3bc7600640 1 --2- 172.21.15.44:0/881138275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc00734f0 0x7f3bc015a9d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400a950 tx=0x7f3bb401dab0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.955+0000 7f3bc7600640 1 -- 172.21.15.44:0/881138275 shutdown_connections 2024-09-13T08:42:58.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.955+0000 7f3bc7600640 1 --2- 172.21.15.44:0/881138275 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bc00cea60 0x7f3bc006eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.955+0000 7f3bc7600640 1 --2- 172.21.15.44:0/881138275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc00734f0 0x7f3bc015a9d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.955+0000 7f3bc7600640 1 -- 172.21.15.44:0/881138275 >> 172.21.15.44:0/881138275 conn(0x7f3bc00cfbc0 msgr2=0x7f3bc00cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:58.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.955+0000 7f3bc7600640 1 -- 172.21.15.44:0/881138275 shutdown_connections 2024-09-13T08:42:58.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.955+0000 7f3bc7600640 1 -- 172.21.15.44:0/881138275 wait complete. 2024-09-13T08:42:58.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc7600640 1 Processor -- start 2024-09-13T08:42:58.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc7600640 1 -- start start 2024-09-13T08:42:58.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc7600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc00734f0 0x7f3bc014a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc7600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc00cea60 0x7f3bc00b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc7600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bc00cf440 0x7f3bc00b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc7600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc006faf0 con 0x7f3bc00734f0 2024-09-13T08:42:58.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc7600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc006f970 con 0x7f3bc00cea60 2024-09-13T08:42:58.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc7600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc006fc70 con 0x7f3bc00cf440 2024-09-13T08:42:58.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc00734f0 0x7f3bc014a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc00734f0 0x7f3bc014a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40996/0 (socket says 172.21.15.44:40996) 2024-09-13T08:42:58.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc4c00640 1 -- 172.21.15.44:0/1339380341 learned_addr learned my addr 172.21.15.44:0/1339380341 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:42:58.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc5600640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bc00cf440 0x7f3bc00b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bbfe00640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc00cea60 0x7f3bc00b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc4c00640 1 -- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bc00cf440 msgr2=0x7f3bc00b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc4c00640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bc00cf440 0x7f3bc00b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc4c00640 1 -- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc00cea60 msgr2=0x7f3bc00b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:58.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc4c00640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc00cea60 0x7f3bc00b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:58.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc4c00640 1 -- 172.21.15.44:0/1339380341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc01534f0 con 0x7f3bc00734f0 2024-09-13T08:42:58.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc4c00640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc00734f0 0x7f3bc014a220 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3bb401c160 tx=0x7f3bb4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:58.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bbcc00640 1 -- 172.21.15.44:0/1339380341 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40b3040 con 0x7f3bc00734f0 2024-09-13T08:42:58.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bbcc00640 1 -- 172.21.15.44:0/1339380341 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bb4005310 con 0x7f3bc00734f0 2024-09-13T08:42:58.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc7600640 1 -- 172.21.15.44:0/1339380341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc015d020 con 0x7f3bc00734f0 2024-09-13T08:42:58.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bbcc00640 1 -- 172.21.15.44:0/1339380341 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40046d0 con 0x7f3bc00734f0 2024-09-13T08:42:58.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.959+0000 7f3bc7600640 1 -- 172.21.15.44:0/1339380341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc015d5e0 con 0x7f3bc00734f0 2024-09-13T08:42:58.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.963+0000 7f3bc7600640 1 -- 172.21.15.44:0/1339380341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc0069410 con 0x7f3bc00734f0 2024-09-13T08:42:58.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.963+0000 7f3bbcc00640 1 -- 172.21.15.44:0/1339380341 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f3bb4004890 con 0x7f3bc00734f0 2024-09-13T08:42:58.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.963+0000 7f3bbcc00640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ba00737c0 0x7f3ba0075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:42:58.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.963+0000 7f3bbcc00640 1 -- 172.21.15.44:0/1339380341 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f3bb40be070 con 0x7f3bc00734f0 2024-09-13T08:42:58.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.963+0000 7f3bbfe00640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ba00737c0 0x7f3ba0075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:42:58.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.967+0000 7f3bbfe00640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ba00737c0 0x7f3ba0075c80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00b0e70 tx=0x7f3bac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:42:58.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:58.967+0000 7f3bbcc00640 1 -- 172.21.15.44:0/1339380341 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3bb4017760 con 0x7f3bc00734f0 2024-09-13T08:42:59.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.171+0000 7f3bc7600640 1 -- 172.21.15.44:0/1339380341 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone56670", "target": ["mon-mgr", ""]}) -- 0x7f3bc015a9d0 con 0x7f3ba00737c0 2024-09-13T08:42:59.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.191+0000 7f3bbcc00640 1 -- 172.21.15.44:0/1339380341 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3bc015a9d0 con 0x7f3ba00737c0 2024-09-13T08:42:59.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.191+0000 7f3bc7600640 1 -- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ba00737c0 msgr2=0x7f3ba0075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:59.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.191+0000 7f3bc7600640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ba00737c0 0x7f3ba0075c80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00b0e70 tx=0x7f3bac009040 comp rx=0 tx=0).stop 2024-09-13T08:42:59.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.191+0000 7f3bc7600640 1 -- 172.21.15.44:0/1339380341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc00734f0 msgr2=0x7f3bc014a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:42:59.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.191+0000 7f3bc7600640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc00734f0 0x7f3bc014a220 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3bb401c160 tx=0x7f3bb4004620 comp rx=0 tx=0).stop 2024-09-13T08:42:59.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.195+0000 7f3bc7600640 1 -- 172.21.15.44:0/1339380341 shutdown_connections 2024-09-13T08:42:59.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.195+0000 7f3bc7600640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ba00737c0 0x7f3ba0075c80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:59.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.195+0000 7f3bc7600640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bc00cf440 0x7f3bc00b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:59.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.195+0000 7f3bc7600640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc00cea60 0x7f3bc00b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:59.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.195+0000 7f3bc7600640 1 --2- 172.21.15.44:0/1339380341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc00734f0 0x7f3bc014a220 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:42:59.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.195+0000 7f3bc7600640 1 -- 172.21.15.44:0/1339380341 >> 172.21.15.44:0/1339380341 conn(0x7f3bc00cfbc0 msgr2=0x7f3bc0151110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:42:59.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.195+0000 7f3bc7600640 1 -- 172.21.15.44:0/1339380341 shutdown_connections 2024-09-13T08:42:59.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59.195+0000 7f3bc7600640 1 -- 172.21.15.44:0/1339380341 wait complete. 2024-09-13T08:42:59.226 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-13T08:42:59.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:42:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:43:00.002 INFO:teuthology.orchestra.run.smithi044.stdout:2 2024-09-13T08:43:00.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:43:00.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:43:00.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.155+0000 7f92a3000640 1 -- 172.21.15.44:0/1443486126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c073e40 msgr2=0x7f929c0cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:00.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.155+0000 7f92a3000640 1 --2- 172.21.15.44:0/1443486126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c073e40 0x7f929c0cb8c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9284009900 tx=0x7f928401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:43:00.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.155+0000 7f92a3000640 1 -- 172.21.15.44:0/1443486126 shutdown_connections 2024-09-13T08:43:00.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.155+0000 7f92a3000640 1 --2- 172.21.15.44:0/1443486126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c073e40 0x7f929c0cb8c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:00.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.155+0000 7f92a3000640 1 --2- 172.21.15.44:0/1443486126 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f929c073a60 0x7f929c15bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:00.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.155+0000 7f92a3000640 1 -- 172.21.15.44:0/1443486126 >> 172.21.15.44:0/1443486126 conn(0x7f929c0d0c70 msgr2=0x7f929c0d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:00.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.155+0000 7f92a3000640 1 -- 172.21.15.44:0/1443486126 shutdown_connections 2024-09-13T08:43:00.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.155+0000 7f92a3000640 1 -- 172.21.15.44:0/1443486126 wait complete. 2024-09-13T08:43:00.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.155+0000 7f92a3000640 1 Processor -- start 2024-09-13T08:43:00.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.155+0000 7f92a3000640 1 -- start start 2024-09-13T08:43:00.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f92a3000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f929c073a60 0x7f929c165d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f92a3000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f929c073e40 0x7f929c14c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f92a3000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c151830 0x7f929c1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f92a3000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f929c0cf4e0 con 0x7f929c151830 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f92a3000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f929c0cf360 con 0x7f929c073a60 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f92a3000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f929c0cf660 con 0x7f929c073e40 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f92a1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c151830 0x7f929c1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f929be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f929c073a60 0x7f929c165d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f929be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f929c073a60 0x7f929c165d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:45948/0 (socket says 172.21.15.44:45948) 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f929be00640 1 -- 172.21.15.44:0/1130177576 learned_addr learned my addr 172.21.15.44:0/1130177576 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f9293e00640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f929c073e40 0x7f929c14c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:00.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f929be00640 1 -- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f929c073e40 msgr2=0x7f929c14c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:00.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f929be00640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f929c073e40 0x7f929c14c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:00.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f929be00640 1 -- 172.21.15.44:0/1130177576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c151830 msgr2=0x7f929c1509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:00.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f929be00640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c151830 0x7f929c1509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:00.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f929be00640 1 -- 172.21.15.44:0/1130177576 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929c1fb1c0 con 0x7f929c073a60 2024-09-13T08:43:00.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f929be00640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f929c073a60 0x7f929c165d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f928c00e940 tx=0x7f928c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:00.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f9298c00640 1 -- 172.21.15.44:0/1130177576 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c00cd80 con 0x7f929c073a60 2024-09-13T08:43:00.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f9298c00640 1 -- 172.21.15.44:0/1130177576 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f928c005020 con 0x7f929c073a60 2024-09-13T08:43:00.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f92a3000640 1 -- 172.21.15.44:0/1130177576 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f929c1fb480 con 0x7f929c073a60 2024-09-13T08:43:00.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f9298c00640 1 -- 172.21.15.44:0/1130177576 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c005450 con 0x7f929c073a60 2024-09-13T08:43:00.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f92a3000640 1 -- 172.21.15.44:0/1130177576 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f929c1fba40 con 0x7f929c073a60 2024-09-13T08:43:00.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.159+0000 7f92a3000640 1 -- 172.21.15.44:0/1130177576 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f929c0cfb50 con 0x7f929c073a60 2024-09-13T08:43:00.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.163+0000 7f9298c00640 1 -- 172.21.15.44:0/1130177576 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f928c0047b0 con 0x7f929c073a60 2024-09-13T08:43:00.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.163+0000 7f9298c00640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92780737c0 0x7f9278075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:00.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.163+0000 7f9298c00640 1 -- 172.21.15.44:0/1130177576 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f928c0932d0 con 0x7f929c073a60 2024-09-13T08:43:00.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.163+0000 7f9293e00640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92780737c0 0x7f9278075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:00.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.163+0000 7f9293e00640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92780737c0 0x7f9278075c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f92840af000 tx=0x7f928401b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:00.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.167+0000 7f9298c00640 1 -- 172.21.15.44:0/1130177576 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f928c05f920 con 0x7f929c073a60 2024-09-13T08:43:00.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.387+0000 7f92a3000640 1 -- 172.21.15.44:0/1130177576 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f929c0747f0 con 0x7f929c073a60 2024-09-13T08:43:00.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.387+0000 7f9298c00640 1 -- 172.21.15.44:0/1130177576 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v32)=0 v32) ==== 131+0+0 (secure 0 0 0) 0x7f928c064350 con 0x7f929c073a60 2024-09-13T08:43:00.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 -- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92780737c0 msgr2=0x7f9278075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:00.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92780737c0 0x7f9278075c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f92840af000 tx=0x7f928401b120 comp rx=0 tx=0).stop 2024-09-13T08:43:00.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 -- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f929c073a60 msgr2=0x7f929c165d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:00.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f929c073a60 0x7f929c165d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f928c00e940 tx=0x7f928c00ee00 comp rx=0 tx=0).stop 2024-09-13T08:43:00.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 -- 172.21.15.44:0/1130177576 shutdown_connections 2024-09-13T08:43:00.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92780737c0 0x7f9278075c80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:00.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c151830 0x7f929c1509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:00.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f929c073e40 0x7f929c14c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:00.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 --2- 172.21.15.44:0/1130177576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f929c073a60 0x7f929c165d40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:00.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 -- 172.21.15.44:0/1130177576 >> 172.21.15.44:0/1130177576 conn(0x7f929c0d0c70 msgr2=0x7f929c0706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:00.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 -- 172.21.15.44:0/1130177576 shutdown_connections 2024-09-13T08:43:00.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.391+0000 7f92a3000640 1 -- 172.21.15.44:0/1130177576 wait complete. 2024-09-13T08:43:00.427 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:43:00.428 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:43:00.544 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:43:00.646 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:43:00.647 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:43:00.647 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:43:00.647 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:00.657 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:43:00.658 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:00.718 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:43:00.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.991+0000 7fe962c00640 1 -- 172.21.15.44:0/2009770525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe940003a10 msgr2=0x7fe940003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:00.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.991+0000 7fe962c00640 1 --2- 172.21.15.44:0/2009770525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe940003a10 0x7fe940003e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe95c06b0f0 tx=0x7fe94c01f680 comp rx=0 tx=0).stop 2024-09-13T08:43:00.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.991+0000 7fe962c00640 1 -- 172.21.15.44:0/2009770525 shutdown_connections 2024-09-13T08:43:00.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.991+0000 7fe962c00640 1 --2- 172.21.15.44:0/2009770525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe940003a10 0x7fe940003e10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:00.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.991+0000 7fe962c00640 1 -- 172.21.15.44:0/2009770525 >> 172.21.15.44:0/2009770525 conn(0x7fe95c0d1dc0 msgr2=0x7fe95c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:00.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.991+0000 7fe962c00640 1 -- 172.21.15.44:0/2009770525 shutdown_connections 2024-09-13T08:43:00.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.991+0000 7fe962c00640 1 -- 172.21.15.44:0/2009770525 wait complete. 2024-09-13T08:43:00.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe962c00640 1 Processor -- start 2024-09-13T08:43:00.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe962c00640 1 -- start start 2024-09-13T08:43:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe962c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe940003a10 0x7fe95c201d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe962c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe95c0b64d0 0x7fe95c15e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe962c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe95c0b6950 0x7fe95c15e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe962c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c15c490 con 0x7fe95c0b6950 2024-09-13T08:43:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe962c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c0d2af0 con 0x7fe940003a10 2024-09-13T08:43:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe962c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe95c0ce670 con 0x7fe95c0b64d0 2024-09-13T08:43:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe95b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe95c0b64d0 0x7fe95c15e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe95be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe940003a10 0x7fe95c201d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe95b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe95c0b64d0 0x7fe95c15e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:37726/0 (socket says 172.21.15.44:37726) 2024-09-13T08:43:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe95b400640 1 -- 172.21.15.44:0/4293711084 learned_addr learned my addr 172.21.15.44:0/4293711084 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.995+0000 7fe960c00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe95c0b6950 0x7fe95c15e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe95be00640 1 -- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe95c0b64d0 msgr2=0x7fe95c15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe95be00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe95c0b64d0 0x7fe95c15e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe95be00640 1 -- 172.21.15.44:0/4293711084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe95c0b6950 msgr2=0x7fe95c15e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe95be00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe95c0b6950 0x7fe95c15e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe95be00640 1 -- 172.21.15.44:0/4293711084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95c2022c0 con 0x7fe940003a10 2024-09-13T08:43:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe95b400640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe95c0b64d0 0x7fe95c15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:43:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe95be00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe940003a10 0x7fe95c201d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe94000c3c0 tx=0x7fe94c001db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe953e00640 1 -- 172.21.15.44:0/4293711084 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0be070 con 0x7fe940003a10 2024-09-13T08:43:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe953e00640 1 -- 172.21.15.44:0/4293711084 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe94c0063e0 con 0x7fe940003a10 2024-09-13T08:43:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe953e00640 1 -- 172.21.15.44:0/4293711084 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c003600 con 0x7fe940003a10 2024-09-13T08:43:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe962c00640 1 -- 172.21.15.44:0/4293711084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95c15c990 con 0x7fe940003a10 2024-09-13T08:43:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe962c00640 1 -- 172.21.15.44:0/4293711084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95c15cf50 con 0x7fe940003a10 2024-09-13T08:43:01.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:00.999+0000 7fe962c00640 1 -- 172.21.15.44:0/4293711084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95c0cc9d0 con 0x7fe940003a10 2024-09-13T08:43:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.003+0000 7fe953e00640 1 -- 172.21.15.44:0/4293711084 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fe94c0c1070 con 0x7fe940003a10 2024-09-13T08:43:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.003+0000 7fe953e00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe93c073810 0x7fe93c075cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:01.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.003+0000 7fe953e00640 1 -- 172.21.15.44:0/4293711084 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fe94c12f890 con 0x7fe940003a10 2024-09-13T08:43:01.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.003+0000 7fe95b400640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe93c073810 0x7fe93c075cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:01.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.007+0000 7fe95b400640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe93c073810 0x7fe93c075cd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe95c15f880 tx=0x7fe94800b310 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:01.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.007+0000 7fe953e00640 1 -- 172.21.15.44:0/4293711084 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe95c2022c0 con 0x7fe940003a10 2024-09-13T08:43:01.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.343+0000 7fe962c00640 1 -- 172.21.15.44:0/4293711084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe95c072340 con 0x7fe940003a10 2024-09-13T08:43:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.343+0000 7fe953e00640 1 -- 172.21.15.44:0/4293711084 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) ==== 80+0+0 (secure 0 0 0) 0x7fe94c019430 con 0x7fe940003a10 2024-09-13T08:43:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:43:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 -- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe93c073810 msgr2=0x7fe93c075cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe93c073810 0x7fe93c075cd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe95c15f880 tx=0x7fe94800b310 comp rx=0 tx=0).stop 2024-09-13T08:43:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 -- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe940003a10 msgr2=0x7fe95c201d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe940003a10 0x7fe95c201d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe94000c3c0 tx=0x7fe94c001db0 comp rx=0 tx=0).stop 2024-09-13T08:43:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 -- 172.21.15.44:0/4293711084 shutdown_connections 2024-09-13T08:43:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe93c073810 0x7fe93c075cd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe95c0b6950 0x7fe95c15e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe95c0b64d0 0x7fe95c15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 --2- 172.21.15.44:0/4293711084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe940003a10 0x7fe95c201d80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 -- 172.21.15.44:0/4293711084 >> 172.21.15.44:0/4293711084 conn(0x7fe95c0d1dc0 msgr2=0x7fe95c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 -- 172.21.15.44:0/4293711084 shutdown_connections 2024-09-13T08:43:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.347+0000 7fe962c00640 1 -- 172.21.15.44:0/4293711084 wait complete. 2024-09-13T08:43:01.503 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T08:43:01.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:01.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T08:43:01.580 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:43:01.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:01.581 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T08:43:01.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:43:01.604 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:43:01.605 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:43:01.605 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:01.616 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:43:01.616 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:43:01.616 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:43:01.616 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:01.652 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:43:01.664 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:43:01.664 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:43:01.664 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:43:01.664 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:01.722 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:43:01.722 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:43:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.975+0000 7f335dc00640 1 -- 172.21.15.44:0/3184968340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f333c004810 msgr2=0x7f333c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.975+0000 7f335dc00640 1 --2- 172.21.15.44:0/3184968340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f333c004810 0x7f333c004c90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f33440096f0 tx=0x7f334401f460 comp rx=0 tx=0).stop 2024-09-13T08:43:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.975+0000 7f335dc00640 1 -- 172.21.15.44:0/3184968340 shutdown_connections 2024-09-13T08:43:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.975+0000 7f335dc00640 1 --2- 172.21.15.44:0/3184968340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f333c004810 0x7f333c004c90 secure :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f33440096f0 tx=0x7f334401f460 comp rx=0 tx=0).stop 2024-09-13T08:43:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.975+0000 7f335dc00640 1 -- 172.21.15.44:0/3184968340 >> 172.21.15.44:0/3184968340 conn(0x7f33580d0c90 msgr2=0x7f33580d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.979+0000 7f335dc00640 1 -- 172.21.15.44:0/3184968340 shutdown_connections 2024-09-13T08:43:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.979+0000 7f335dc00640 1 -- 172.21.15.44:0/3184968340 wait complete. 2024-09-13T08:43:01.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.979+0000 7f335dc00640 1 Processor -- start 2024-09-13T08:43:01.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.979+0000 7f335dc00640 1 -- start start 2024-09-13T08:43:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f335dc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33580c5ca0 0x7f33580b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f335dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33580c6080 0x7f33580ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f335dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33580b5be0 0x7f33580ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f335dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f335806ffb0 con 0x7f33580c6080 2024-09-13T08:43:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f335dc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f335806fe30 con 0x7f33580b5be0 2024-09-13T08:43:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f335dc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3358070130 con 0x7f33580c5ca0 2024-09-13T08:43:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3357400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33580b5be0 0x7f33580ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3357400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33580b5be0 0x7f33580ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:45978/0 (socket says 172.21.15.44:45978) 2024-09-13T08:43:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3357400640 1 -- 172.21.15.44:0/2465752813 learned_addr learned my addr 172.21.15.44:0/2465752813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3356a00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33580c5ca0 0x7f33580b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3356000640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33580c6080 0x7f33580ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3356a00640 1 -- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33580b5be0 msgr2=0x7f33580ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3356a00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33580b5be0 0x7f33580ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3356a00640 1 -- 172.21.15.44:0/2465752813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33580c6080 msgr2=0x7f33580ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3356a00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33580c6080 0x7f33580ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3356a00640 1 -- 172.21.15.44:0/2465752813 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33580c6530 con 0x7f33580c5ca0 2024-09-13T08:43:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f3356a00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33580c5ca0 0x7f33580b56a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3348003ed0 tx=0x7f3348001640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.983+0000 7f334ea00640 1 -- 172.21.15.44:0/2465752813 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348015070 con 0x7f33580c5ca0 2024-09-13T08:43:01.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.987+0000 7f334ea00640 1 -- 172.21.15.44:0/2465752813 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3348011840 con 0x7f33580c5ca0 2024-09-13T08:43:01.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.987+0000 7f335dc00640 1 -- 172.21.15.44:0/2465752813 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33580c6810 con 0x7f33580c5ca0 2024-09-13T08:43:01.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.987+0000 7f334ea00640 1 -- 172.21.15.44:0/2465752813 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348011c70 con 0x7f33580c5ca0 2024-09-13T08:43:01.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.987+0000 7f335dc00640 1 -- 172.21.15.44:0/2465752813 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33580c6dd0 con 0x7f33580c5ca0 2024-09-13T08:43:01.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.987+0000 7f335dc00640 1 -- 172.21.15.44:0/2465752813 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3358070690 con 0x7f33580c5ca0 2024-09-13T08:43:01.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.991+0000 7f334ea00640 1 -- 172.21.15.44:0/2465752813 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f334803d070 con 0x7f33580c5ca0 2024-09-13T08:43:01.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.991+0000 7f334ea00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3338073840 0x7f3338075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:01.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.991+0000 7f334ea00640 1 -- 172.21.15.44:0/2465752813 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 5910+0+0 (secure 0 0 0) 0x7f3348096740 con 0x7f33580c5ca0 2024-09-13T08:43:01.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.995+0000 7f3356000640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3338073840 0x7f3338075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:02.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.995+0000 7f3356000640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3338073840 0x7f3338075d00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f33440b1000 tx=0x7f334401c200 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:02.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:01.999+0000 7f334ea00640 1 -- 172.21.15.44:0/2465752813 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f33480602d0 con 0x7f33580c5ca0 2024-09-13T08:43:02.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.379+0000 7f335dc00640 1 -- 172.21.15.44:0/2465752813 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3358071350 con 0x7f33580c5ca0 2024-09-13T08:43:02.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.379+0000 7f334ea00640 1 -- 172.21.15.44:0/2465752813 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5229 (secure 0 0 0) 0x7f3348064d00 con 0x7f33580c5ca0 2024-09-13T08:43:02.384 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:02.384 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"btime":"2024-09-13T08:40:56:233184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3367280951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3367280951},{"type":"v1","addr":"172.21.15.129:6835","nonce":3367280951}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1057940180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1057940180},{"type":"v1","addr":"172.21.15.129:6837","nonce":1057940180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5319,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1413618927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413618927},{"type":"v1","addr":"172.21.15.44:6834","nonce":1413618927}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:40:42.589651+0000","modified":"2024-09-13T08:40:45.596647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":4,"addr":"172.21.15.44:6835/3141962538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3141962538},{"type":"v1","addr":"172.21.15.44:6835","nonce":3141962538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5322,"qdb_cluster":[5322]},"id":3}]} 2024-09-13T08:43:02.385 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 28 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 -- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3338073840 msgr2=0x7f3338075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3338073840 0x7f3338075d00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f33440b1000 tx=0x7f334401c200 comp rx=0 tx=0).stop 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 -- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33580c5ca0 msgr2=0x7f33580b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33580c5ca0 0x7f33580b56a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3348003ed0 tx=0x7f3348001640 comp rx=0 tx=0).stop 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 -- 172.21.15.44:0/2465752813 shutdown_connections 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3338073840 0x7f3338075d00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33580b5be0 0x7f33580ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33580c6080 0x7f33580ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 --2- 172.21.15.44:0/2465752813 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33580c5ca0 0x7f33580b56a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 -- 172.21.15.44:0/2465752813 >> 172.21.15.44:0/2465752813 conn(0x7f33580d0c90 msgr2=0x7f3358156d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:02.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 -- 172.21.15.44:0/2465752813 shutdown_connections 2024-09-13T08:43:02.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.383+0000 7f335dc00640 1 -- 172.21.15.44:0/2465752813 wait complete. 2024-09-13T08:43:02.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:43:02.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 -- 172.21.15.44:0/2532865216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94f8074330 msgr2=0x7f94f806f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 --2- 172.21.15.44:0/2532865216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94f8074330 0x7f94f806f9a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009bc0 tx=0x7f94f001cb70 comp rx=0 tx=0).stop 2024-09-13T08:43:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 -- 172.21.15.44:0/2532865216 shutdown_connections 2024-09-13T08:43:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 --2- 172.21.15.44:0/2532865216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94f8074330 0x7f94f806f9a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 --2- 172.21.15.44:0/2532865216 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f94f815af60 0x7f94f80cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 -- 172.21.15.44:0/2532865216 >> 172.21.15.44:0/2532865216 conn(0x7f94f80d0c90 msgr2=0x7f94f80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 -- 172.21.15.44:0/2532865216 shutdown_connections 2024-09-13T08:43:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 -- 172.21.15.44:0/2532865216 wait complete. 2024-09-13T08:43:02.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 Processor -- start 2024-09-13T08:43:02.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 -- start start 2024-09-13T08:43:02.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f94f80737c0 0x7f94f80c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:02.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f8074330 0x7f94f80c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94f815af60 0x7f94f80c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f94f8167ab0 con 0x7f94f815af60 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94f8167930 con 0x7f94f8074330 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94ff000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f94f8167c30 con 0x7f94f80737c0 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94fd000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94f815af60 0x7f94f80c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94fd000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94f815af60 0x7f94f80c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38098/0 (socket says 172.21.15.44:38098) 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94fd000640 1 -- 172.21.15.44:0/11290750 learned_addr learned my addr 172.21.15.44:0/11290750 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94f7e00640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f94f80737c0 0x7f94f80c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94efe00640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f8074330 0x7f94f80c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94fd000640 1 -- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f94f80737c0 msgr2=0x7f94f80c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94fd000640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f94f80737c0 0x7f94f80c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94fd000640 1 -- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f8074330 msgr2=0x7f94f80c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:02.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94fd000640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f8074330 0x7f94f80c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.551+0000 7f94fd000640 1 -- 172.21.15.44:0/11290750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f81f8290 con 0x7f94f815af60 2024-09-13T08:43:02.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94fd000640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94f815af60 0x7f94f80c5520 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f94f001a7a0 tx=0x7f94f00a5a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:02.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94f4c00640 1 -- 172.21.15.44:0/11290750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f00b2040 con 0x7f94f815af60 2024-09-13T08:43:02.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94f4c00640 1 -- 172.21.15.44:0/11290750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94f0004d40 con 0x7f94f815af60 2024-09-13T08:43:02.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94ff000640 1 -- 172.21.15.44:0/11290750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f81f8570 con 0x7f94f815af60 2024-09-13T08:43:02.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94f4c00640 1 -- 172.21.15.44:0/11290750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0004300 con 0x7f94f815af60 2024-09-13T08:43:02.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94ff000640 1 -- 172.21.15.44:0/11290750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f81f8b30 con 0x7f94f815af60 2024-09-13T08:43:02.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94ff000640 1 -- 172.21.15.44:0/11290750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f806f9a0 con 0x7f94f815af60 2024-09-13T08:43:02.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94f4c00640 1 -- 172.21.15.44:0/11290750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f94f00be060 con 0x7f94f815af60 2024-09-13T08:43:02.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94f4c00640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94d40737c0 0x7f94d4075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:02.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94f4c00640 1 -- 172.21.15.44:0/11290750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5910+0+0 (secure 0 0 0) 0x7f94f012c2f0 con 0x7f94f815af60 2024-09-13T08:43:02.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.555+0000 7f94f7e00640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94d40737c0 0x7f94d4075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:02.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.559+0000 7f94f7e00640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94d40737c0 0x7f94d4075c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f94f80cac30 tx=0x7f94e800b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:02.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.559+0000 7f94f4c00640 1 -- 172.21.15.44:0/11290750 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f94f0016790 con 0x7f94f815af60 2024-09-13T08:43:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.771+0000 7f94ff000640 1 -- 172.21.15.44:0/11290750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94f80b4b10 con 0x7f94f815af60 2024-09-13T08:43:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.771+0000 7f94f4c00640 1 -- 172.21.15.44:0/11290750 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5229 (secure 0 0 0) 0x7f94f00f8940 con 0x7f94f815af60 2024-09-13T08:43:02.777 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:02.777 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"btime":"2024-09-13T08:40:56:233184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3367280951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3367280951},{"type":"v1","addr":"172.21.15.129:6835","nonce":3367280951}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1057940180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1057940180},{"type":"v1","addr":"172.21.15.129:6837","nonce":1057940180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5319,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1413618927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413618927},{"type":"v1","addr":"172.21.15.44:6834","nonce":1413618927}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:40:42.589651+0000","modified":"2024-09-13T08:40:45.596647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":4,"addr":"172.21.15.44:6835/3141962538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3141962538},{"type":"v1","addr":"172.21.15.44:6835","nonce":3141962538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5322,"qdb_cluster":[5322]},"id":3}]} 2024-09-13T08:43:02.777 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 28 2024-09-13T08:43:02.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 -- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94d40737c0 msgr2=0x7f94d4075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:02.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94d40737c0 0x7f94d4075c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f94f80cac30 tx=0x7f94e800b2e0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 -- 172.21.15.44:0/11290750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94f815af60 msgr2=0x7f94f80c5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:02.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94f815af60 0x7f94f80c5520 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f94f001a7a0 tx=0x7f94f00a5a60 comp rx=0 tx=0).stop 2024-09-13T08:43:02.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 -- 172.21.15.44:0/11290750 shutdown_connections 2024-09-13T08:43:02.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94d40737c0 0x7f94d4075c80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94f815af60 0x7f94f80c5520 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f8074330 0x7f94f80c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 --2- 172.21.15.44:0/11290750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f94f80737c0 0x7f94f80c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 -- 172.21.15.44:0/11290750 >> 172.21.15.44:0/11290750 conn(0x7f94f80d0c90 msgr2=0x7f94f8153390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:02.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 -- 172.21.15.44:0/11290750 shutdown_connections 2024-09-13T08:43:02.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.775+0000 7f94ff000640 1 -- 172.21.15.44:0/11290750 wait complete. 2024-09-13T08:43:02.799 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:43:02.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.939+0000 7fc8c6a00640 1 -- 172.21.15.44:0/2574182277 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c0072c90 msgr2=0x7fc8c0073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:02.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.939+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/2574182277 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c0072c90 0x7fc8c0073120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009bf0 tx=0x7fc8b801f250 comp rx=0 tx=0).stop 2024-09-13T08:43:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 -- 172.21.15.44:0/2574182277 shutdown_connections 2024-09-13T08:43:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/2574182277 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c0072c90 0x7fc8c0073120 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/2574182277 >> v2:172.21.15.129:3300/0 conn(0x7fc8c015e1b0 0x7fc8c00705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 -- 172.21.15.44:0/2574182277 >> 172.21.15.44:0/2574182277 conn(0x7fc8c00b1c20 msgr2=0x7fc8c00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 -- 172.21.15.44:0/2574182277 shutdown_connections 2024-09-13T08:43:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 -- 172.21.15.44:0/2574182277 wait complete. 2024-09-13T08:43:02.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 Processor -- start 2024-09-13T08:43:02.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 -- start start 2024-09-13T08:43:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c0072c90 0x7fc8c014dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c00af150 0x7fc8c0153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8c015e1b0 0x7fc8c01544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c00ae720 con 0x7fc8c00af150 2024-09-13T08:43:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c00ae5a0 con 0x7fc8c015e1b0 2024-09-13T08:43:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c00ae8a0 con 0x7fc8c0072c90 2024-09-13T08:43:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c00af150 0x7fc8c0153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c00af150 0x7fc8c0153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38102/0 (socket says 172.21.15.44:38102) 2024-09-13T08:43:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8b7e00640 1 -- 172.21.15.44:0/1099709235 learned_addr learned my addr 172.21.15.44:0/1099709235 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c4a00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8c015e1b0 0x7fc8c01544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8b7e00640 1 -- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c0072c90 msgr2=0x7fc8c014dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8b7e00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c0072c90 0x7fc8c014dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8b7e00640 1 -- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8c015e1b0 msgr2=0x7fc8c01544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8b7e00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8c015e1b0 0x7fc8c01544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8b7e00640 1 -- 172.21.15.44:0/1099709235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c01566c0 con 0x7fc8c00af150 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c4a00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8c015e1b0 0x7fc8c01544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8b7e00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c00af150 0x7fc8c0153fa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800c700 tx=0x7fc8a800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8bcc00640 1 -- 172.21.15.44:0/1099709235 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800a840 con 0x7fc8c00af150 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8bcc00640 1 -- 172.21.15.44:0/1099709235 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8a8017070 con 0x7fc8c00af150 2024-09-13T08:43:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8bcc00640 1 -- 172.21.15.44:0/1099709235 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800ad90 con 0x7fc8c00af150 2024-09-13T08:43:02.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.943+0000 7fc8c6a00640 1 -- 172.21.15.44:0/1099709235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c009b000 con 0x7fc8c00af150 2024-09-13T08:43:02.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.947+0000 7fc8c6a00640 1 -- 172.21.15.44:0/1099709235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c009b5c0 con 0x7fc8c00af150 2024-09-13T08:43:02.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.947+0000 7fc8c6a00640 1 -- 172.21.15.44:0/1099709235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c009ae20 con 0x7fc8c00af150 2024-09-13T08:43:02.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.947+0000 7fc8bcc00640 1 -- 172.21.15.44:0/1099709235 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fc8a801c460 con 0x7fc8c00af150 2024-09-13T08:43:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.947+0000 7fc8bcc00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc89c0737f0 0x7fc89c075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.947+0000 7fc8bcc00640 1 -- 172.21.15.44:0/1099709235 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5910+0+0 (secure 0 0 0) 0x7fc8a8092630 con 0x7fc8c00af150 2024-09-13T08:43:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.947+0000 7fc8bfe00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc89c0737f0 0x7fc89c075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:02.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.951+0000 7fc8bfe00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc89c0737f0 0x7fc89c075cb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00058d0 tx=0x7fc8b0005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:02.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:02.951+0000 7fc8bcc00640 1 -- 172.21.15.44:0/1099709235 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc8c009ae20 con 0x7fc8c00af150 2024-09-13T08:43:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.127+0000 7fc8c6a00640 1 -- 172.21.15.44:0/1099709235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8c00aed50 con 0x7fc8c00af150 2024-09-13T08:43:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.127+0000 7fc8bcc00640 1 -- 172.21.15.44:0/1099709235 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17021 (secure 0 0 0) 0x7fc8a805f2d0 con 0x7fc8c00af150 2024-09-13T08:43:03.133 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:03.133 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":50,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:43:01.667378+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:40:40.291993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T08:40:41.533879+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.1b","mappings":[{"from":5,"to":7}]},{"pgid":"7.20","mappings":[{"from":5,"to":0}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:43:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 -- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc89c0737f0 msgr2=0x7fc89c075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc89c0737f0 0x7fc89c075cb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00058d0 tx=0x7fc8b0005860 comp rx=0 tx=0).stop 2024-09-13T08:43:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 -- 172.21.15.44:0/1099709235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c00af150 msgr2=0x7fc8c0153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c00af150 0x7fc8c0153fa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800c700 tx=0x7fc8a800cbc0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 -- 172.21.15.44:0/1099709235 shutdown_connections 2024-09-13T08:43:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc89c0737f0 0x7fc89c075cb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8c015e1b0 0x7fc8c01544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c00af150 0x7fc8c0153fa0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/1099709235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c0072c90 0x7fc8c014dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 -- 172.21.15.44:0/1099709235 >> 172.21.15.44:0/1099709235 conn(0x7fc8c00b1c20 msgr2=0x7fc8c00afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:03.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 -- 172.21.15.44:0/1099709235 shutdown_connections 2024-09-13T08:43:03.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.131+0000 7fc8c6a00640 1 -- 172.21.15.44:0/1099709235 wait complete. 2024-09-13T08:43:03.154 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:43:03.154 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:43:03.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 -- 172.21.15.44:0/2668617425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f035815f000 msgr2=0x7f03580d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 --2- 172.21.15.44:0/2668617425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f035815f000 0x7f03580d01b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f034c009840 tx=0x7f034c01c700 comp rx=0 tx=0).stop 2024-09-13T08:43:03.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 -- 172.21.15.44:0/2668617425 shutdown_connections 2024-09-13T08:43:03.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 --2- 172.21.15.44:0/2668617425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f035815f000 0x7f03580d01b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 -- 172.21.15.44:0/2668617425 >> 172.21.15.44:0/2668617425 conn(0x7f035809aa10 msgr2=0x7f035809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:03.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 -- 172.21.15.44:0/2668617425 shutdown_connections 2024-09-13T08:43:03.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 -- 172.21.15.44:0/2668617425 wait complete. 2024-09-13T08:43:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 Processor -- start 2024-09-13T08:43:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 -- start start 2024-09-13T08:43:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03580b2590 0x7f0358158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f03580cefb0 0x7f0358157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f035815f000 0x7f0358157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f035816a110 con 0x7f035815f000 2024-09-13T08:43:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0358169f90 con 0x7f03580b2590 2024-09-13T08:43:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035f000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f035816a290 con 0x7f03580cefb0 2024-09-13T08:43:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f035815f000 0x7f0358157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03580b2590 0x7f0358158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03580b2590 0x7f0358158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:37490/0 (socket says 172.21.15.44:37490) 2024-09-13T08:43:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035dc00640 1 -- 172.21.15.44:0/4124771441 learned_addr learned my addr 172.21.15.44:0/4124771441 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035e600640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f035815f000 0x7f0358157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38128/0 (socket says 172.21.15.44:38128) 2024-09-13T08:43:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035d200640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f03580cefb0 0x7f0358157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035e600640 1 -- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f03580cefb0 msgr2=0x7f0358157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035e600640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f03580cefb0 0x7f0358157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035e600640 1 -- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03580b2590 msgr2=0x7f0358158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035e600640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03580b2590 0x7f0358158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.299+0000 7f035e600640 1 -- 172.21.15.44:0/4124771441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0358156a60 con 0x7f035815f000 2024-09-13T08:43:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.303+0000 7f035e600640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f035815f000 0x7f0358157580 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f035000c700 tx=0x7f035000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.303+0000 7f0355600640 1 -- 172.21.15.44:0/4124771441 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350003b80 con 0x7f035815f000 2024-09-13T08:43:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.303+0000 7f0355600640 1 -- 172.21.15.44:0/4124771441 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0350003d20 con 0x7f035815f000 2024-09-13T08:43:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.303+0000 7f0355600640 1 -- 172.21.15.44:0/4124771441 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035000aab0 con 0x7f035815f000 2024-09-13T08:43:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.303+0000 7f035f000640 1 -- 172.21.15.44:0/4124771441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03580c9030 con 0x7f035815f000 2024-09-13T08:43:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.303+0000 7f035f000640 1 -- 172.21.15.44:0/4124771441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03580c98d0 con 0x7f035815f000 2024-09-13T08:43:03.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.303+0000 7f035f000640 1 -- 172.21.15.44:0/4124771441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0358159af0 con 0x7f035815f000 2024-09-13T08:43:03.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.303+0000 7f0355600640 1 -- 172.21.15.44:0/4124771441 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f035000ac50 con 0x7f035815f000 2024-09-13T08:43:03.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.307+0000 7f0355600640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03380737c0 0x7f0338075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:03.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.307+0000 7f0355600640 1 -- 172.21.15.44:0/4124771441 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5910+0+0 (secure 0 0 0) 0x7f03500922e0 con 0x7f035815f000 2024-09-13T08:43:03.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.307+0000 7f035dc00640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03380737c0 0x7f0338075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:03.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.307+0000 7f035dc00640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03380737c0 0x7f0338075c80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f034c01a330 tx=0x7f034c01af40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.311+0000 7f0355600640 1 -- 172.21.15.44:0/4124771441 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f035005e990 con 0x7f035815f000 2024-09-13T08:43:03.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.519+0000 7f035f000640 1 -- 172.21.15.44:0/4124771441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03580c9200 con 0x7f035815f000 2024-09-13T08:43:03.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.519+0000 7f0355600640 1 -- 172.21.15.44:0/4124771441 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5229 (secure 0 0 0) 0x7f03500633c0 con 0x7f035815f000 2024-09-13T08:43:03.525 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:03.525 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"btime":"2024-09-13T08:40:56:233184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3367280951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3367280951},{"type":"v1","addr":"172.21.15.129:6835","nonce":3367280951}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1057940180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1057940180},{"type":"v1","addr":"172.21.15.129:6837","nonce":1057940180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5319,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1413618927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413618927},{"type":"v1","addr":"172.21.15.44:6834","nonce":1413618927}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:40:42.589651+0000","modified":"2024-09-13T08:40:45.596647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":4,"addr":"172.21.15.44:6835/3141962538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3141962538},{"type":"v1","addr":"172.21.15.44:6835","nonce":3141962538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5322,"qdb_cluster":[5322]},"id":3}]} 2024-09-13T08:43:03.525 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 28 2024-09-13T08:43:03.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 -- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03380737c0 msgr2=0x7f0338075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03380737c0 0x7f0338075c80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f034c01a330 tx=0x7f034c01af40 comp rx=0 tx=0).stop 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 -- 172.21.15.44:0/4124771441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f035815f000 msgr2=0x7f0358157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f035815f000 0x7f0358157580 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f035000c700 tx=0x7f035000cbc0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 -- 172.21.15.44:0/4124771441 shutdown_connections 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f03380737c0 0x7f0338075c80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f035815f000 0x7f0358157580 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f03580cefb0 0x7f0358157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 --2- 172.21.15.44:0/4124771441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03580b2590 0x7f0358158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 -- 172.21.15.44:0/4124771441 >> 172.21.15.44:0/4124771441 conn(0x7f035809aa10 msgr2=0x7f035815e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 -- 172.21.15.44:0/4124771441 shutdown_connections 2024-09-13T08:43:03.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.523+0000 7f035f000640 1 -- 172.21.15.44:0/4124771441 wait complete. 2024-09-13T08:43:03.546 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:43:03.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.687+0000 7f6e01800640 1 -- 172.21.15.44:0/4131168490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc06cc20 msgr2=0x7f6dfc0737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.687+0000 7f6e01800640 1 --2- 172.21.15.44:0/4131168490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc06cc20 0x7f6dfc0737c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009840 tx=0x7f6dec01c7b0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.687+0000 7f6e01800640 1 -- 172.21.15.44:0/4131168490 shutdown_connections 2024-09-13T08:43:03.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.687+0000 7f6e01800640 1 --2- 172.21.15.44:0/4131168490 >> v2:172.21.15.129:3300/0 conn(0x7f6dfc071030 0x7f6dfc071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.687+0000 7f6e01800640 1 --2- 172.21.15.44:0/4131168490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc06cc20 0x7f6dfc0737c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.687+0000 7f6e01800640 1 -- 172.21.15.44:0/4131168490 >> 172.21.15.44:0/4131168490 conn(0x7f6dfc0d0c90 msgr2=0x7f6dfc0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:03.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.687+0000 7f6e01800640 1 -- 172.21.15.44:0/4131168490 shutdown_connections 2024-09-13T08:43:03.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.687+0000 7f6e01800640 1 -- 172.21.15.44:0/4131168490 wait complete. 2024-09-13T08:43:03.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6e01800640 1 Processor -- start 2024-09-13T08:43:03.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6e01800640 1 -- start start 2024-09-13T08:43:03.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6e01800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dfc06cc20 0x7f6dfc153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6e01800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc071030 0x7f6dfc0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6e01800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dfc15c1c0 0x7f6dfc0c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6e01800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc168f30 con 0x7f6dfc071030 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6e01800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc168db0 con 0x7f6dfc15c1c0 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6e01800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6dfc1690b0 con 0x7f6dfc06cc20 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc071030 0x7f6dfc0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc071030 0x7f6dfc0c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38150/0 (socket says 172.21.15.44:38150) 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df3e00640 1 -- 172.21.15.44:0/3948650670 learned_addr learned my addr 172.21.15.44:0/3948650670 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6dfaa00640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dfc06cc20 0x7f6dfc153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6dfb400640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dfc15c1c0 0x7f6dfc0c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df3e00640 1 -- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dfc06cc20 msgr2=0x7f6dfc153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df3e00640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dfc06cc20 0x7f6dfc153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df3e00640 1 -- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dfc15c1c0 msgr2=0x7f6dfc0c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df3e00640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dfc15c1c0 0x7f6dfc0c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df3e00640 1 -- 172.21.15.44:0/3948650670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc0b10a0 con 0x7f6dfc071030 2024-09-13T08:43:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df3e00640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc071030 0x7f6dfc0c39d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c430 tx=0x7f6de400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df2a00640 1 -- 172.21.15.44:0/3948650670 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4011790 con 0x7f6dfc071030 2024-09-13T08:43:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df2a00640 1 -- 172.21.15.44:0/3948650670 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6de4017070 con 0x7f6dfc071030 2024-09-13T08:43:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6df2a00640 1 -- 172.21.15.44:0/3948650670 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4011ce0 con 0x7f6dfc071030 2024-09-13T08:43:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6e01800640 1 -- 172.21.15.44:0/3948650670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc1f8f90 con 0x7f6dfc071030 2024-09-13T08:43:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.691+0000 7f6e01800640 1 -- 172.21.15.44:0/3948650670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc153200 con 0x7f6dfc071030 2024-09-13T08:43:03.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.695+0000 7f6e01800640 1 -- 172.21.15.44:0/3948650670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dfc0737c0 con 0x7f6dfc071030 2024-09-13T08:43:03.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.695+0000 7f6df2a00640 1 -- 172.21.15.44:0/3948650670 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f6de401c460 con 0x7f6dfc071030 2024-09-13T08:43:03.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.695+0000 7f6df2a00640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dd80737c0 0x7f6dd8075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:03.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.695+0000 7f6df2a00640 1 -- 172.21.15.44:0/3948650670 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5910+0+0 (secure 0 0 0) 0x7f6de40922d0 con 0x7f6dfc071030 2024-09-13T08:43:03.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.695+0000 7f6dfaa00640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dd80737c0 0x7f6dd8075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:03.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.699+0000 7f6dfaa00640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dd80737c0 0x7f6dd8075c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0af000 tx=0x7f6dec01aff0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.699+0000 7f6df2a00640 1 -- 172.21.15.44:0/3948650670 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6dfc0b10a0 con 0x7f6dfc071030 2024-09-13T08:43:03.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.915+0000 7f6e01800640 1 -- 172.21.15.44:0/3948650670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dfc0a1cb0 con 0x7f6dfc071030 2024-09-13T08:43:03.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.915+0000 7f6df2a00640 1 -- 172.21.15.44:0/3948650670 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5229 (secure 0 0 0) 0x7f6de405ea40 con 0x7f6dfc071030 2024-09-13T08:43:03.920 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:03.920 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"btime":"2024-09-13T08:40:56:233184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3367280951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3367280951},{"type":"v1","addr":"172.21.15.129:6835","nonce":3367280951}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1057940180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1057940180},{"type":"v1","addr":"172.21.15.129:6837","nonce":1057940180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5319,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1413618927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413618927},{"type":"v1","addr":"172.21.15.44:6834","nonce":1413618927}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:40:42.589651+0000","modified":"2024-09-13T08:40:45.596647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":4,"addr":"172.21.15.44:6835/3141962538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3141962538},{"type":"v1","addr":"172.21.15.44:6835","nonce":3141962538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5322,"qdb_cluster":[5322]},"id":3}]} 2024-09-13T08:43:03.920 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 28 2024-09-13T08:43:03.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 -- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dd80737c0 msgr2=0x7f6dd8075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dd80737c0 0x7f6dd8075c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0af000 tx=0x7f6dec01aff0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 -- 172.21.15.44:0/3948650670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc071030 msgr2=0x7f6dfc0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:03.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc071030 0x7f6dfc0c39d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c430 tx=0x7f6de400c8f0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 -- 172.21.15.44:0/3948650670 shutdown_connections 2024-09-13T08:43:03.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dd80737c0 0x7f6dd8075c80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dfc15c1c0 0x7f6dfc0c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc071030 0x7f6dfc0c39d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 --2- 172.21.15.44:0/3948650670 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dfc06cc20 0x7f6dfc153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:03.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 -- 172.21.15.44:0/3948650670 >> 172.21.15.44:0/3948650670 conn(0x7f6dfc0d0c90 msgr2=0x7f6dfc0b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:03.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 -- 172.21.15.44:0/3948650670 shutdown_connections 2024-09-13T08:43:03.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:03.919+0000 7f6e01800640 1 -- 172.21.15.44:0/3948650670 wait complete. 2024-09-13T08:43:03.941 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:43:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 -- 172.21.15.44:0/3076109396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01840737c0 msgr2=0x7f018415c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 --2- 172.21.15.44:0/3076109396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01840737c0 0x7f018415c1c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f016c009870 tx=0x7f016c01c7f0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 -- 172.21.15.44:0/3076109396 shutdown_connections 2024-09-13T08:43:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 --2- 172.21.15.44:0/3076109396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01840737c0 0x7f018415c1c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 -- 172.21.15.44:0/3076109396 >> 172.21.15.44:0/3076109396 conn(0x7f01840d0c90 msgr2=0x7f01840d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 -- 172.21.15.44:0/3076109396 shutdown_connections 2024-09-13T08:43:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 -- 172.21.15.44:0/3076109396 wait complete. 2024-09-13T08:43:04.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 Processor -- start 2024-09-13T08:43:04.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 -- start start 2024-09-13T08:43:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018406cc20 0x7f01840b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0184071030 0x7f0184155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01840737c0 0x7f01841f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f018416a900 con 0x7f018406cc20 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f018416a780 con 0x7f01840737c0 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0189800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f018416aa80 con 0x7f0184071030 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0182a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018406cc20 0x7f01840b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0182a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018406cc20 0x7f01840b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38172/0 (socket says 172.21.15.44:38172) 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0182a00640 1 -- 172.21.15.44:0/867710811 learned_addr learned my addr 172.21.15.44:0/867710811 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0183400640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01840737c0 0x7f01841f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f017be00640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0184071030 0x7f0184155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0182a00640 1 -- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0184071030 msgr2=0x7f0184155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0182a00640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0184071030 0x7f0184155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0182a00640 1 -- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01840737c0 msgr2=0x7f01841f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0182a00640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01840737c0 0x7f01841f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.095+0000 7f0182a00640 1 -- 172.21.15.44:0/867710811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01840b5150 con 0x7f018406cc20 2024-09-13T08:43:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f0182a00640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018406cc20 0x7f01840b6610 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f017400c4c0 tx=0x7f017400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f017aa00640 1 -- 172.21.15.44:0/867710811 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174016020 con 0x7f018406cc20 2024-09-13T08:43:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f017aa00640 1 -- 172.21.15.44:0/867710811 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0174012940 con 0x7f018406cc20 2024-09-13T08:43:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f017aa00640 1 -- 172.21.15.44:0/867710811 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174012d90 con 0x7f018406cc20 2024-09-13T08:43:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f0189800640 1 -- 172.21.15.44:0/867710811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01841f90e0 con 0x7f018406cc20 2024-09-13T08:43:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f0189800640 1 -- 172.21.15.44:0/867710811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01840b4950 con 0x7f018406cc20 2024-09-13T08:43:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f0189800640 1 -- 172.21.15.44:0/867710811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f018415c1c0 con 0x7f018406cc20 2024-09-13T08:43:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f017aa00640 1 -- 172.21.15.44:0/867710811 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f0174019020 con 0x7f018406cc20 2024-09-13T08:43:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f017aa00640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f01600737f0 0x7f0160075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f017aa00640 1 -- 172.21.15.44:0/867710811 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5910+0+0 (secure 0 0 0) 0x7f0174092680 con 0x7f018406cc20 2024-09-13T08:43:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.099+0000 7f017be00640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f01600737f0 0x7f0160075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.103+0000 7f017be00640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f01600737f0 0x7f0160075cb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f016c0af000 tx=0x7f016c01b030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:04.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.103+0000 7f017aa00640 1 -- 172.21.15.44:0/867710811 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0174020020 con 0x7f018406cc20 2024-09-13T08:43:04.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.283+0000 7f0189800640 1 -- 172.21.15.44:0/867710811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01840b6c10 con 0x7f018406cc20 2024-09-13T08:43:04.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f017aa00640 1 -- 172.21.15.44:0/867710811 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17021 (secure 0 0 0) 0x7f017405f3e0 con 0x7f018406cc20 2024-09-13T08:43:04.290 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:04.290 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":50,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:43:01.667378+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:40:40.291993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T08:40:41.533879+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.1b","mappings":[{"from":5,"to":7}]},{"pgid":"7.20","mappings":[{"from":5,"to":0}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:43:04.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 -- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f01600737f0 msgr2=0x7f0160075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f01600737f0 0x7f0160075cb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f016c0af000 tx=0x7f016c01b030 comp rx=0 tx=0).stop 2024-09-13T08:43:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 -- 172.21.15.44:0/867710811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018406cc20 msgr2=0x7f01840b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018406cc20 0x7f01840b6610 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f017400c4c0 tx=0x7f017400c980 comp rx=0 tx=0).stop 2024-09-13T08:43:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 -- 172.21.15.44:0/867710811 shutdown_connections 2024-09-13T08:43:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f01600737f0 0x7f0160075cb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01840737c0 0x7f01841f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0184071030 0x7f0184155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 --2- 172.21.15.44:0/867710811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018406cc20 0x7f01840b6610 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 -- 172.21.15.44:0/867710811 >> 172.21.15.44:0/867710811 conn(0x7f01840d0c90 msgr2=0x7f01840a0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 -- 172.21.15.44:0/867710811 shutdown_connections 2024-09-13T08:43:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.287+0000 7f0189800640 1 -- 172.21.15.44:0/867710811 wait complete. 2024-09-13T08:43:04.313 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T08:43:04.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 -- 172.21.15.44:0/3864408832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa48815db50 msgr2=0x7fa4880acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 --2- 172.21.15.44:0/3864408832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa48815db50 0x7fa4880acd40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa478009900 tx=0x7fa47801c850 comp rx=0 tx=0).stop 2024-09-13T08:43:04.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 -- 172.21.15.44:0/3864408832 shutdown_connections 2024-09-13T08:43:04.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 --2- 172.21.15.44:0/3864408832 >> v2:172.21.15.129:3300/0 conn(0x7fa4880b4a50 0x7fa4880ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 --2- 172.21.15.44:0/3864408832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa48815db50 0x7fa4880acd40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 -- 172.21.15.44:0/3864408832 >> 172.21.15.44:0/3864408832 conn(0x7fa4880b18f0 msgr2=0x7fa4880b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:04.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 -- 172.21.15.44:0/3864408832 shutdown_connections 2024-09-13T08:43:04.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 -- 172.21.15.44:0/3864408832 wait complete. 2024-09-13T08:43:04.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 Processor -- start 2024-09-13T08:43:04.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 -- start start 2024-09-13T08:43:04.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa4880b4a50 0x7fa4881524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4880cd100 0x7fa4881518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48815db50 0x7fa488165880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa488165110 con 0x7fa48815db50 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa4880cf550 con 0x7fa4880cd100 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48ec00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa488165290 con 0x7fa4880b4a50 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48815db50 0x7fa488165880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48815db50 0x7fa488165880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38186/0 (socket says 172.21.15.44:38186) 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48cc00640 1 -- 172.21.15.44:0/1185087688 learned_addr learned my addr 172.21.15.44:0/1185087688 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa487e00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa4880b4a50 0x7fa4881524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa47fe00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4880cd100 0x7fa4881518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48cc00640 1 -- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa4880b4a50 msgr2=0x7fa4881524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48cc00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa4880b4a50 0x7fa4881524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48cc00640 1 -- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4880cd100 msgr2=0x7fa4881518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48cc00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4880cd100 0x7fa4881518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.459+0000 7fa48cc00640 1 -- 172.21.15.44:0/1185087688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa488152df0 con 0x7fa48815db50 2024-09-13T08:43:04.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.463+0000 7fa48cc00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48815db50 0x7fa488165880 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa480009d30 tx=0x7fa48000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:04.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.463+0000 7fa484c00640 1 -- 172.21.15.44:0/1185087688 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480003b80 con 0x7fa48815db50 2024-09-13T08:43:04.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.463+0000 7fa484c00640 1 -- 172.21.15.44:0/1185087688 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa480003d20 con 0x7fa48815db50 2024-09-13T08:43:04.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.463+0000 7fa484c00640 1 -- 172.21.15.44:0/1185087688 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480004280 con 0x7fa48815db50 2024-09-13T08:43:04.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.463+0000 7fa48ec00640 1 -- 172.21.15.44:0/1185087688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48815c300 con 0x7fa48815db50 2024-09-13T08:43:04.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.463+0000 7fa48ec00640 1 -- 172.21.15.44:0/1185087688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48815c8c0 con 0x7fa48815db50 2024-09-13T08:43:04.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.463+0000 7fa48ec00640 1 -- 172.21.15.44:0/1185087688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4880ce7f0 con 0x7fa48815db50 2024-09-13T08:43:04.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.463+0000 7fa484c00640 1 -- 172.21.15.44:0/1185087688 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fa480004420 con 0x7fa48815db50 2024-09-13T08:43:04.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.463+0000 7fa484c00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa4640737c0 0x7fa464075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.463+0000 7fa484c00640 1 -- 172.21.15.44:0/1185087688 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5910+0+0 (secure 0 0 0) 0x7fa4800933c0 con 0x7fa48815db50 2024-09-13T08:43:04.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.467+0000 7fa487e00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa4640737c0 0x7fa464075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.467+0000 7fa487e00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa4640737c0 0x7fa464075c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa4780af000 tx=0x7fa47801b090 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:04.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.467+0000 7fa484c00640 1 -- 172.21.15.44:0/1185087688 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa480060060 con 0x7fa48815db50 2024-09-13T08:43:04.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.679+0000 7fa48ec00640 1 -- 172.21.15.44:0/1185087688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa4880b1510 con 0x7fa48815db50 2024-09-13T08:43:04.685 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T08:43:04.679+0000 7f03b3c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T08:43:04.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.683+0000 7fa484c00640 1 -- 172.21.15.44:0/1185087688 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7fa48005f960 con 0x7fa48815db50 2024-09-13T08:43:04.687 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T08:43:04.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 -- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa4640737c0 msgr2=0x7fa464075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa4640737c0 0x7fa464075c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa4780af000 tx=0x7fa47801b090 comp rx=0 tx=0).stop 2024-09-13T08:43:04.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 -- 172.21.15.44:0/1185087688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48815db50 msgr2=0x7fa488165880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48815db50 0x7fa488165880 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa480009d30 tx=0x7fa48000c6a0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 -- 172.21.15.44:0/1185087688 shutdown_connections 2024-09-13T08:43:04.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa4640737c0 0x7fa464075c80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48815db50 0x7fa488165880 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4880cd100 0x7fa4881518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 --2- 172.21.15.44:0/1185087688 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa4880b4a50 0x7fa4881524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 -- 172.21.15.44:0/1185087688 >> 172.21.15.44:0/1185087688 conn(0x7fa4880b18f0 msgr2=0x7fa488072d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:04.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 -- 172.21.15.44:0/1185087688 shutdown_connections 2024-09-13T08:43:04.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.687+0000 7fa48ec00640 1 -- 172.21.15.44:0/1185087688 wait complete. 2024-09-13T08:43:04.714 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T08:43:04.760 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:43:04.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.863+0000 7fb581e00640 1 -- 172.21.15.44:0/3251032914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb57c0d04d0 msgr2=0x7fb57c0af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.863+0000 7fb581e00640 1 --2- 172.21.15.44:0/3251032914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb57c0d04d0 0x7fb57c0af200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb574009bc0 tx=0x7fb57401f3b0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.863+0000 7fb581e00640 1 -- 172.21.15.44:0/3251032914 shutdown_connections 2024-09-13T08:43:04.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.863+0000 7fb581e00640 1 --2- 172.21.15.44:0/3251032914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb57c0d04d0 0x7fb57c0af200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.863+0000 7fb581e00640 1 --2- 172.21.15.44:0/3251032914 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb57c0d00f0 0x7fb57c0b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.863+0000 7fb581e00640 1 -- 172.21.15.44:0/3251032914 >> 172.21.15.44:0/3251032914 conn(0x7fb57c0d1e90 msgr2=0x7fb57c0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.863+0000 7fb581e00640 1 -- 172.21.15.44:0/3251032914 shutdown_connections 2024-09-13T08:43:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 -- 172.21.15.44:0/3251032914 wait complete. 2024-09-13T08:43:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 Processor -- start 2024-09-13T08:43:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 -- start start 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb57c0d00f0 0x7fb57c0c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb57c0d04d0 0x7fb57c0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb57c0d29b0 0x7fb57c1f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c0b0ed0 con 0x7fb57c0d04d0 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c0b0d50 con 0x7fb57c0d00f0 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c0af740 con 0x7fb57c0d29b0 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb57c0d04d0 0x7fb57c0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb57c0d04d0 0x7fb57c0c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38192/0 (socket says 172.21.15.44:38192) 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573e00640 1 -- 172.21.15.44:0/1918936687 learned_addr learned my addr 172.21.15.44:0/1918936687 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb57aa00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb57c0d00f0 0x7fb57c0c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb57b400640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb57c0d29b0 0x7fb57c1f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573e00640 1 -- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb57c0d29b0 msgr2=0x7fb57c1f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573e00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb57c0d29b0 0x7fb57c1f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573e00640 1 -- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb57c0d00f0 msgr2=0x7fb57c0c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573e00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb57c0d00f0 0x7fb57c0c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573e00640 1 -- 172.21.15.44:0/1918936687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c0b5a90 con 0x7fb57c0d04d0 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb57b400640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb57c0d29b0 0x7fb57c1f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573e00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb57c0d04d0 0x7fb57c0c81b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c430 tx=0x7fb56400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573400640 1 -- 172.21.15.44:0/1918936687 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564011790 con 0x7fb57c0d04d0 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573400640 1 -- 172.21.15.44:0/1918936687 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb564017070 con 0x7fb57c0d04d0 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb573400640 1 -- 172.21.15.44:0/1918936687 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564011ce0 con 0x7fb57c0d04d0 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 -- 172.21.15.44:0/1918936687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c0b5d70 con 0x7fb57c0d04d0 2024-09-13T08:43:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.867+0000 7fb581e00640 1 -- 172.21.15.44:0/1918936687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c0cd820 con 0x7fb57c0d04d0 2024-09-13T08:43:04.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.871+0000 7fb581e00640 1 -- 172.21.15.44:0/1918936687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb57c0af200 con 0x7fb57c0d04d0 2024-09-13T08:43:04.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.871+0000 7fb573400640 1 -- 172.21.15.44:0/1918936687 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fb56401c460 con 0x7fb57c0d04d0 2024-09-13T08:43:04.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.871+0000 7fb573400640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5580737c0 0x7fb558075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:04.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.871+0000 7fb573400640 1 -- 172.21.15.44:0/1918936687 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5968+0+0 (secure 0 0 0) 0x7fb564092450 con 0x7fb57c0d04d0 2024-09-13T08:43:04.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.871+0000 7fb57aa00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5580737c0 0x7fb558075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:04.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.871+0000 7fb57aa00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5580737c0 0x7fb558075c80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009be0 tx=0x7fb56c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:04.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:04.875+0000 7fb573400640 1 -- 172.21.15.44:0/1918936687 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb56405f1d0 con 0x7fb57c0d04d0 2024-09-13T08:43:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.087+0000 7fb581e00640 1 -- 172.21.15.44:0/1918936687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb57c0b78d0 con 0x7fb57c0d04d0 2024-09-13T08:43:05.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb573400640 1 -- 172.21.15.44:0/1918936687 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7fb56405ead0 con 0x7fb57c0d04d0 2024-09-13T08:43:05.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 -- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5580737c0 msgr2=0x7fb558075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:05.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5580737c0 0x7fb558075c80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009be0 tx=0x7fb56c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:43:05.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 -- 172.21.15.44:0/1918936687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb57c0d04d0 msgr2=0x7fb57c0c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:05.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb57c0d04d0 0x7fb57c0c81b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c430 tx=0x7fb56400c8f0 comp rx=0 tx=0).stop 2024-09-13T08:43:05.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 -- 172.21.15.44:0/1918936687 shutdown_connections 2024-09-13T08:43:05.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5580737c0 0x7fb558075c80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:05.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb57c0d29b0 0x7fb57c1f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:05.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb57c0d04d0 0x7fb57c0c81b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:05.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 --2- 172.21.15.44:0/1918936687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb57c0d00f0 0x7fb57c0c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:05.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 -- 172.21.15.44:0/1918936687 >> 172.21.15.44:0/1918936687 conn(0x7fb57c0d1e90 msgr2=0x7fb57c15acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:05.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 -- 172.21.15.44:0/1918936687 shutdown_connections 2024-09-13T08:43:05.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.695+0000 7fb581e00640 1 -- 172.21.15.44:0/1918936687 wait complete. 2024-09-13T08:43:05.726 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T08:43:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 -- 172.21.15.44:0/3253054715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faee8003cc0 msgr2=0x7faee80040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 --2- 172.21.15.44:0/3253054715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faee8003cc0 0x7faee80040c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faef4009b60 tx=0x7faef401d270 comp rx=0 tx=0).stop 2024-09-13T08:43:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf08e00640 1 -- 172.21.15.44:0/3253054715 reap_dead start 2024-09-13T08:43:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 -- 172.21.15.44:0/3253054715 shutdown_connections 2024-09-13T08:43:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 -- 172.21.15.44:0/3253054715 >> 172.21.15.44:0/3253054715 conn(0x7faf040d1e80 msgr2=0x7faf040d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 -- 172.21.15.44:0/3253054715 shutdown_connections 2024-09-13T08:43:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 -- 172.21.15.44:0/3253054715 wait complete. 2024-09-13T08:43:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 Processor -- start 2024-09-13T08:43:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 -- start start 2024-09-13T08:43:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faee8003cc0 0x7faf041f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf0415a540 0x7faf040ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf04158dc0 0x7faf040b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf04167e50 con 0x7faf04158dc0 2024-09-13T08:43:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf040b05f0 con 0x7faee8003cc0 2024-09-13T08:43:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf0ae00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7faf04167fd0 con 0x7faf0415a540 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf08e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf04158dc0 0x7faf040b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf08e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf04158dc0 0x7faf040b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38206/0 (socket says 172.21.15.44:38206) 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf08e00640 1 -- 172.21.15.44:0/3820010535 learned_addr learned my addr 172.21.15.44:0/3820010535 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf03e00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faee8003cc0 0x7faf041f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faefbe00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf0415a540 0x7faf040ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf08e00640 1 -- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf0415a540 msgr2=0x7faf040ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf08e00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf0415a540 0x7faf040ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf08e00640 1 -- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faee8003cc0 msgr2=0x7faf041f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf08e00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faee8003cc0 0x7faf041f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf08e00640 1 -- 172.21.15.44:0/3820010535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf041594a0 con 0x7faf04158dc0 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faefbe00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf0415a540 0x7faf040ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:43:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.923+0000 7faf08e00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf04158dc0 0x7faf040b5790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7faf04073a70 tx=0x7faefc001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.927+0000 7faf00c00640 1 -- 172.21.15.44:0/3820010535 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc010890 con 0x7faf04158dc0 2024-09-13T08:43:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.927+0000 7faf00c00640 1 -- 172.21.15.44:0/3820010535 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faefc017070 con 0x7faf04158dc0 2024-09-13T08:43:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.927+0000 7faf00c00640 1 -- 172.21.15.44:0/3820010535 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc010e20 con 0x7faf04158dc0 2024-09-13T08:43:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.927+0000 7faf0ae00640 1 -- 172.21.15.44:0/3820010535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf04159780 con 0x7faf04158dc0 2024-09-13T08:43:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.927+0000 7faf0ae00640 1 -- 172.21.15.44:0/3820010535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf04159d40 con 0x7faf04158dc0 2024-09-13T08:43:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.927+0000 7faf0ae00640 1 -- 172.21.15.44:0/3820010535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf040cfdf0 con 0x7faf04158dc0 2024-09-13T08:43:05.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.927+0000 7faf00c00640 1 -- 172.21.15.44:0/3820010535 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7faefc01c460 con 0x7faf04158dc0 2024-09-13T08:43:05.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.927+0000 7faf00c00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee00737c0 0x7faee0075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:05.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.927+0000 7faf00c00640 1 -- 172.21.15.44:0/3820010535 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5968+0+0 (secure 0 0 0) 0x7faefc092fb0 con 0x7faf04158dc0 2024-09-13T08:43:05.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.927+0000 7faf03e00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee00737c0 0x7faee0075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:05.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.931+0000 7faf03e00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee00737c0 0x7faee0075c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faef40ae000 tx=0x7faef401a010 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:05.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:05.931+0000 7faf00c00640 1 -- 172.21.15.44:0/3820010535 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7faefc05fc70 con 0x7faf04158dc0 2024-09-13T08:43:06.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.111+0000 7faf0ae00640 1 -- 172.21.15.44:0/3820010535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faf0415ee10 con 0x7faf04158dc0 2024-09-13T08:43:06.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.703+0000 7faf00c00640 1 -- 172.21.15.44:0/3820010535 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) ==== 181+0+0 (secure 0 0 0) 0x7faefc05f570 con 0x7faf04158dc0 2024-09-13T08:43:06.708 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T08:43:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 -- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee00737c0 msgr2=0x7faee0075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee00737c0 0x7faee0075c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faef40ae000 tx=0x7faef401a010 comp rx=0 tx=0).stop 2024-09-13T08:43:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 -- 172.21.15.44:0/3820010535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf04158dc0 msgr2=0x7faf040b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf04158dc0 0x7faf040b5790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7faf04073a70 tx=0x7faefc001b90 comp rx=0 tx=0).stop 2024-09-13T08:43:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 -- 172.21.15.44:0/3820010535 shutdown_connections 2024-09-13T08:43:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faee00737c0 0x7faee0075c80 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faef40ae000 tx=0x7faef401a010 comp rx=0 tx=0).stop 2024-09-13T08:43:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf04158dc0 0x7faf040b5790 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faf0415a540 0x7faf040ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 --2- 172.21.15.44:0/3820010535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faee8003cc0 0x7faf041f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 -- 172.21.15.44:0/3820010535 >> 172.21.15.44:0/3820010535 conn(0x7faf040d1e80 msgr2=0x7faf040713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 -- 172.21.15.44:0/3820010535 shutdown_connections 2024-09-13T08:43:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.707+0000 7faf0ae00640 1 -- 172.21.15.44:0/3820010535 wait complete. 2024-09-13T08:43:06.737 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T08:43:06.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.887+0000 7f50cda00640 1 -- 172.21.15.44:0/1343103549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ac0051d0 msgr2=0x7f50ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:06.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.887+0000 7f50cda00640 1 --2- 172.21.15.44:0/1343103549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ac0051d0 0x7f50ac00b9d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009ee0 tx=0x7f50c001d630 comp rx=0 tx=0).stop 2024-09-13T08:43:06.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.887+0000 7f50cda00640 1 -- 172.21.15.44:0/1343103549 shutdown_connections 2024-09-13T08:43:06.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.887+0000 7f50cda00640 1 --2- 172.21.15.44:0/1343103549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ac0051d0 0x7f50ac00b9d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:06.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.887+0000 7f50cda00640 1 -- 172.21.15.44:0/1343103549 >> 172.21.15.44:0/1343103549 conn(0x7f50c80d1da0 msgr2=0x7f50c80d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:06.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50cda00640 1 -- 172.21.15.44:0/1343103549 shutdown_connections 2024-09-13T08:43:06.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50cda00640 1 -- 172.21.15.44:0/1343103549 wait complete. 2024-09-13T08:43:06.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50cda00640 1 Processor -- start 2024-09-13T08:43:06.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50cda00640 1 -- start start 2024-09-13T08:43:06.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50cda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ac0051d0 0x7f50c80cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:06.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50cda00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50c8151240 0x7f50c80c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:06.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50cda00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50c80b2760 0x7f50c80b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:06.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50cda00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50c80cea90 con 0x7f50ac0051d0 2024-09-13T08:43:06.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50cda00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50c80ce910 con 0x7f50c8151240 2024-09-13T08:43:06.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50cda00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f50c80cec10 con 0x7f50c80b2760 2024-09-13T08:43:06.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ac0051d0 0x7f50c80cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ac0051d0 0x7f50c80cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38220/0 (socket says 172.21.15.44:38220) 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50c6a00640 1 -- 172.21.15.44:0/274517727 learned_addr learned my addr 172.21.15.44:0/274517727 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50bfe00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50c8151240 0x7f50c80c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50c7400640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50c80b2760 0x7f50c80b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50c6a00640 1 -- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50c80b2760 msgr2=0x7f50c80b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50c6a00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50c80b2760 0x7f50c80b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50c6a00640 1 -- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50c8151240 msgr2=0x7f50c80c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50c6a00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50c8151240 0x7f50c80c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50c6a00640 1 -- 172.21.15.44:0/274517727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c80cb5b0 con 0x7f50ac0051d0 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.891+0000 7f50c6a00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ac0051d0 0x7f50c80cad80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f50b8003eb0 tx=0x7f50b8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.895+0000 7f50bea00640 1 -- 172.21.15.44:0/274517727 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8015070 con 0x7f50ac0051d0 2024-09-13T08:43:06.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.895+0000 7f50bea00640 1 -- 172.21.15.44:0/274517727 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50b8011840 con 0x7f50ac0051d0 2024-09-13T08:43:06.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.895+0000 7f50cda00640 1 -- 172.21.15.44:0/274517727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c80cb890 con 0x7f50ac0051d0 2024-09-13T08:43:06.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.895+0000 7f50bea00640 1 -- 172.21.15.44:0/274517727 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8011c70 con 0x7f50ac0051d0 2024-09-13T08:43:06.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.895+0000 7f50cda00640 1 -- 172.21.15.44:0/274517727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c80cb2c0 con 0x7f50ac0051d0 2024-09-13T08:43:06.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.895+0000 7f50cda00640 1 -- 172.21.15.44:0/274517727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c80b3120 con 0x7f50ac0051d0 2024-09-13T08:43:06.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.895+0000 7f50bea00640 1 -- 172.21.15.44:0/274517727 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f50b8018070 con 0x7f50ac0051d0 2024-09-13T08:43:06.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.895+0000 7f50bea00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50a40737f0 0x7f50a4075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:06.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.895+0000 7f50bea00640 1 -- 172.21.15.44:0/274517727 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5530+0+0 (secure 0 0 0) 0x7f50b80931e0 con 0x7f50ac0051d0 2024-09-13T08:43:06.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.895+0000 7f50bfe00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50a40737f0 0x7f50a4075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:06.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.899+0000 7f50bfe00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50a40737f0 0x7f50a4075cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f50c81573a0 tx=0x7f50b00067b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:06.899+0000 7f50bea00640 1 -- 172.21.15.44:0/274517727 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f50b805f830 con 0x7f50ac0051d0 2024-09-13T08:43:07.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.075+0000 7f50cda00640 1 -- 172.21.15.44:0/274517727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f50c80c61c0 con 0x7f50ac0051d0 2024-09-13T08:43:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.707+0000 7f50bea00640 1 -- 172.21.15.44:0/274517727 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) ==== 169+0+0 (secure 0 0 0) 0x7f50b8064260 con 0x7f50ac0051d0 2024-09-13T08:43:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T08:43:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 -- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50a40737f0 msgr2=0x7f50a4075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50a40737f0 0x7f50a4075cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f50c81573a0 tx=0x7f50b00067b0 comp rx=0 tx=0).stop 2024-09-13T08:43:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 -- 172.21.15.44:0/274517727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ac0051d0 msgr2=0x7f50c80cad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ac0051d0 0x7f50c80cad80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f50b8003eb0 tx=0x7f50b8001640 comp rx=0 tx=0).stop 2024-09-13T08:43:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 -- 172.21.15.44:0/274517727 shutdown_connections 2024-09-13T08:43:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50a40737f0 0x7f50a4075cb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50c80b2760 0x7f50c80b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50c8151240 0x7f50c80c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 --2- 172.21.15.44:0/274517727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ac0051d0 0x7f50c80cad80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 -- 172.21.15.44:0/274517727 >> 172.21.15.44:0/274517727 conn(0x7f50c80d1da0 msgr2=0x7f50c8071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.711+0000 7f50cda00640 1 -- 172.21.15.44:0/274517727 shutdown_connections 2024-09-13T08:43:07.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.715+0000 7f50cda00640 1 -- 172.21.15.44:0/274517727 wait complete. 2024-09-13T08:43:07.746 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-13T08:43:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.907+0000 7f077fa00640 1 -- 172.21.15.44:0/1221597453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07780cf4e0 msgr2=0x7f07780cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:07.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.907+0000 7f077fa00640 1 --2- 172.21.15.44:0/1221597453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07780cf4e0 0x7f07780cd950 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0770009bc0 tx=0x7f077001ca80 comp rx=0 tx=0).stop 2024-09-13T08:43:07.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.907+0000 7f077fa00640 1 -- 172.21.15.44:0/1221597453 shutdown_connections 2024-09-13T08:43:07.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.907+0000 7f077fa00640 1 --2- 172.21.15.44:0/1221597453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07780cf4e0 0x7f07780cd950 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:07.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.907+0000 7f077fa00640 1 --2- 172.21.15.44:0/1221597453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f077815bd50 0x7f07780ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:07.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.907+0000 7f077fa00640 1 -- 172.21.15.44:0/1221597453 >> 172.21.15.44:0/1221597453 conn(0x7f07780d1ea0 msgr2=0x7f07780d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:07.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.907+0000 7f077fa00640 1 -- 172.21.15.44:0/1221597453 shutdown_connections 2024-09-13T08:43:07.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.907+0000 7f077fa00640 1 -- 172.21.15.44:0/1221597453 wait complete. 2024-09-13T08:43:07.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.907+0000 7f077fa00640 1 Processor -- start 2024-09-13T08:43:07.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.907+0000 7f077fa00640 1 -- start start 2024-09-13T08:43:07.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077fa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cf4e0 0x7f0778150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:07.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077fa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f077815bd50 0x7f07781523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:07.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077fa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f077815c130 0x7f0778152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:07.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077fa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0778168f00 con 0x7f077815c130 2024-09-13T08:43:07.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077fa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0778168d80 con 0x7f07780cf4e0 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077fa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0778169080 con 0x7f077815bd50 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f077815c130 0x7f0778152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077d000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cf4e0 0x7f0778150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f077815c130 0x7f0778152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38248/0 (socket says 172.21.15.44:38248) 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077d000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cf4e0 0x7f0778150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:37624/0 (socket says 172.21.15.44:37624) 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077da00640 1 -- 172.21.15.44:0/3244082727 learned_addr learned my addr 172.21.15.44:0/3244082727 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f0777e00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f077815bd50 0x7f07781523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077da00640 1 -- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f077815bd50 msgr2=0x7f07781523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077da00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f077815bd50 0x7f07781523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077da00640 1 -- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cf4e0 msgr2=0x7f0778150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077da00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cf4e0 0x7f0778150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077da00640 1 -- 172.21.15.44:0/3244082727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0778152ed0 con 0x7f077815c130 2024-09-13T08:43:07.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077da00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f077815c130 0x7f0778152900 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f077001a6b0 tx=0x7f0770003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:07.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f0774c00640 1 -- 172.21.15.44:0/3244082727 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07700b2040 con 0x7f077815c130 2024-09-13T08:43:07.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f0774c00640 1 -- 172.21.15.44:0/3244082727 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0770004d40 con 0x7f077815c130 2024-09-13T08:43:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f0774c00640 1 -- 172.21.15.44:0/3244082727 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770004210 con 0x7f077815c130 2024-09-13T08:43:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077fa00640 1 -- 172.21.15.44:0/3244082727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07781531b0 con 0x7f077815c130 2024-09-13T08:43:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077fa00640 1 -- 172.21.15.44:0/3244082727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0778151bc0 con 0x7f077815c130 2024-09-13T08:43:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.911+0000 7f077fa00640 1 -- 172.21.15.44:0/3244082727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07780ce5e0 con 0x7f077815c130 2024-09-13T08:43:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.915+0000 7f0774c00640 1 -- 172.21.15.44:0/3244082727 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f07700be060 con 0x7f077815c130 2024-09-13T08:43:07.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.915+0000 7f0774c00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07580737c0 0x7f0758075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:07.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.915+0000 7f0774c00640 1 -- 172.21.15.44:0/3244082727 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4971+0+0 (secure 0 0 0) 0x7f077012c260 con 0x7f077815c130 2024-09-13T08:43:07.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.915+0000 7f077d000640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07580737c0 0x7f0758075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:07.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.915+0000 7f077d000640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07580737c0 0x7f0758075c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f07780713d0 tx=0x7f076c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:07.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:07.919+0000 7f0774c00640 1 -- 172.21.15.44:0/3244082727 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0770016790 con 0x7f077815c130 2024-09-13T08:43:08.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.095+0000 7f077fa00640 1 -- 172.21.15.44:0/3244082727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f0778151970 con 0x7f077815c130 2024-09-13T08:43:08.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.099+0000 7f0774c00640 1 -- 172.21.15.44:0/3244082727 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f07700f88b0 con 0x7f077815c130 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 -- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07580737c0 msgr2=0x7f0758075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07580737c0 0x7f0758075c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f07780713d0 tx=0x7f076c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 -- 172.21.15.44:0/3244082727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f077815c130 msgr2=0x7f0778152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f077815c130 0x7f0778152900 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f077001a6b0 tx=0x7f0770003f60 comp rx=0 tx=0).stop 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 -- 172.21.15.44:0/3244082727 shutdown_connections 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07580737c0 0x7f0758075c80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f077815c130 0x7f0778152900 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f077815bd50 0x7f07781523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 --2- 172.21.15.44:0/3244082727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07780cf4e0 0x7f0778150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 -- 172.21.15.44:0/3244082727 >> 172.21.15.44:0/3244082727 conn(0x7f07780d1ea0 msgr2=0x7f07780d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 -- 172.21.15.44:0/3244082727 shutdown_connections 2024-09-13T08:43:08.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.103+0000 7f077fa00640 1 -- 172.21.15.44:0/3244082727 wait complete. 2024-09-13T08:43:08.125 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-13T08:43:08.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.267+0000 7fda8fc00640 1 -- 172.21.15.44:0/2106297859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda880b1f40 msgr2=0x7fda880cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:08.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.267+0000 7fda8fc00640 1 --2- 172.21.15.44:0/2106297859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda880b1f40 0x7fda880cfda0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fda880740f0 tx=0x7fda8001cc30 comp rx=0 tx=0).stop 2024-09-13T08:43:08.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.267+0000 7fda8fc00640 1 -- 172.21.15.44:0/2106297859 shutdown_connections 2024-09-13T08:43:08.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.267+0000 7fda8fc00640 1 --2- 172.21.15.44:0/2106297859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda880b1f40 0x7fda880cfda0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.267+0000 7fda8fc00640 1 --2- 172.21.15.44:0/2106297859 >> v2:172.21.15.129:3300/0 conn(0x7fda880b2e80 0x7fda880b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.267+0000 7fda8fc00640 1 --2- 172.21.15.44:0/2106297859 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda8815f490 0x7fda880b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.267+0000 7fda8fc00640 1 -- 172.21.15.44:0/2106297859 >> 172.21.15.44:0/2106297859 conn(0x7fda880d1e30 msgr2=0x7fda880d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:08.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.267+0000 7fda8fc00640 1 -- 172.21.15.44:0/2106297859 shutdown_connections 2024-09-13T08:43:08.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.267+0000 7fda8fc00640 1 -- 172.21.15.44:0/2106297859 wait complete. 2024-09-13T08:43:08.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8fc00640 1 Processor -- start 2024-09-13T08:43:08.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8fc00640 1 -- start start 2024-09-13T08:43:08.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8fc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda880b1f40 0x7fda881579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:08.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8fc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda880b2e80 0x7fda88158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8fc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda8815f490 0x7fda880c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8fc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda880b0010 con 0x7fda880b1f40 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8fc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda880afe90 con 0x7fda880b2e80 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8fc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fda880b0190 con 0x7fda8815f490 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda880b1f40 0x7fda881579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda880b1f40 0x7fda881579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38274/0 (socket says 172.21.15.44:38274) 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8d200640 1 -- 172.21.15.44:0/2531720790 learned_addr learned my addr 172.21.15.44:0/2531720790 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda87e00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda880b2e80 0x7fda88158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8dc00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda8815f490 0x7fda880c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8d200640 1 -- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda8815f490 msgr2=0x7fda880c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8d200640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda8815f490 0x7fda880c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8d200640 1 -- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda880b2e80 msgr2=0x7fda88158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:08.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8d200640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda880b2e80 0x7fda88158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8d200640 1 -- 172.21.15.44:0/2531720790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda880b6fc0 con 0x7fda880b1f40 2024-09-13T08:43:08.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8dc00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda8815f490 0x7fda880c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:08.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8d200640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda880b1f40 0x7fda881579f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00e7d0 tx=0x7fda7c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:08.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda84c00640 1 -- 172.21.15.44:0/2531720790 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c00ccc0 con 0x7fda880b1f40 2024-09-13T08:43:08.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda84c00640 1 -- 172.21.15.44:0/2531720790 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda7c010430 con 0x7fda880b1f40 2024-09-13T08:43:08.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda84c00640 1 -- 172.21.15.44:0/2531720790 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c010860 con 0x7fda880b1f40 2024-09-13T08:43:08.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8fc00640 1 -- 172.21.15.44:0/2531720790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda880b72a0 con 0x7fda880b1f40 2024-09-13T08:43:08.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.271+0000 7fda8fc00640 1 -- 172.21.15.44:0/2531720790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda880b7a70 con 0x7fda880b1f40 2024-09-13T08:43:08.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.275+0000 7fda8fc00640 1 -- 172.21.15.44:0/2531720790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda880cfda0 con 0x7fda880b1f40 2024-09-13T08:43:08.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.275+0000 7fda84c00640 1 -- 172.21.15.44:0/2531720790 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fda7c010a00 con 0x7fda880b1f40 2024-09-13T08:43:08.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.275+0000 7fda84c00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda680737c0 0x7fda68075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:08.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.275+0000 7fda84c00640 1 -- 172.21.15.44:0/2531720790 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4971+0+0 (secure 0 0 0) 0x7fda7c092fc0 con 0x7fda880b1f40 2024-09-13T08:43:08.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.275+0000 7fda87e00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda680737c0 0x7fda68075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:08.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.275+0000 7fda87e00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda680737c0 0x7fda68075c80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda880b18b0 tx=0x7fda740091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.279+0000 7fda84c00640 1 -- 172.21.15.44:0/2531720790 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fda7c05f670 con 0x7fda880b1f40 2024-09-13T08:43:08.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.515+0000 7fda8fc00640 1 -- 172.21.15.44:0/2531720790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7fda88071a40 con 0x7fda880b1f40 2024-09-13T08:43:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda84c00640 1 -- 172.21.15.44:0/2531720790 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v254) ==== 166+0+0 (secure 0 0 0) 0x7fda7c0640a0 con 0x7fda880b1f40 2024-09-13T08:43:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 -- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda680737c0 msgr2=0x7fda68075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda680737c0 0x7fda68075c80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda880b18b0 tx=0x7fda740091e0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 -- 172.21.15.44:0/2531720790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda880b1f40 msgr2=0x7fda881579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda880b1f40 0x7fda881579f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00e7d0 tx=0x7fda7c00ec90 comp rx=0 tx=0).stop 2024-09-13T08:43:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 -- 172.21.15.44:0/2531720790 shutdown_connections 2024-09-13T08:43:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda680737c0 0x7fda68075c80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda8815f490 0x7fda880c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda880b2e80 0x7fda88158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 --2- 172.21.15.44:0/2531720790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda880b1f40 0x7fda881579f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 -- 172.21.15.44:0/2531720790 >> 172.21.15.44:0/2531720790 conn(0x7fda880d1e30 msgr2=0x7fda88071e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:08.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 -- 172.21.15.44:0/2531720790 shutdown_connections 2024-09-13T08:43:08.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.719+0000 7fda8fc00640 1 -- 172.21.15.44:0/2531720790 wait complete. 2024-09-13T08:43:08.746 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-13T08:43:08.746 INFO:tasks.cephfs_test_runner:Starting test: test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-13T08:43:08.746 INFO:tasks.cephfs_test_runner:test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-13T08:43:08.746 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:43:08.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 -- 172.21.15.44:0/3213922732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34815c130 msgr2=0x7ff3480ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:08.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 --2- 172.21.15.44:0/3213922732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34815c130 0x7ff3480ceb20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff330009840 tx=0x7ff33001c700 comp rx=0 tx=0).stop 2024-09-13T08:43:08.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 -- 172.21.15.44:0/3213922732 shutdown_connections 2024-09-13T08:43:08.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 --2- 172.21.15.44:0/3213922732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34815c130 0x7ff3480ceb20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 --2- 172.21.15.44:0/3213922732 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34815bd50 0x7ff3480ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 -- 172.21.15.44:0/3213922732 >> 172.21.15.44:0/3213922732 conn(0x7ff3480d1ea0 msgr2=0x7ff3480d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:08.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 -- 172.21.15.44:0/3213922732 shutdown_connections 2024-09-13T08:43:08.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 -- 172.21.15.44:0/3213922732 wait complete. 2024-09-13T08:43:08.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 Processor -- start 2024-09-13T08:43:08.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 -- start start 2024-09-13T08:43:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3480cf4e0 0x7ff34814d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34815bd50 0x7ff3480cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34815c130 0x7ff3480cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff348168e50 con 0x7ff34815bd50 2024-09-13T08:43:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff348168cd0 con 0x7ff3480cf4e0 2024-09-13T08:43:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff34f200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff348168fd0 con 0x7ff34815c130 2024-09-13T08:43:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.891+0000 7ff33f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34815bd50 0x7ff3480cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff33f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34815bd50 0x7ff3480cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38286/0 (socket says 172.21.15.44:38286) 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff33f400640 1 -- 172.21.15.44:0/1697810594 learned_addr learned my addr 172.21.15.44:0/1697810594 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff34d200640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34815c130 0x7ff3480cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff347e00640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3480cf4e0 0x7ff34814d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff33f400640 1 -- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34815c130 msgr2=0x7ff3480cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff33f400640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34815c130 0x7ff3480cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff33f400640 1 -- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3480cf4e0 msgr2=0x7ff34814d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff33f400640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3480cf4e0 0x7ff34814d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff33f400640 1 -- 172.21.15.44:0/1697810594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34814f4a0 con 0x7ff34815bd50 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff33f400640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34815bd50 0x7ff3480cb350 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff330009790 tx=0x7ff330004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff344c00640 1 -- 172.21.15.44:0/1697810594 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330005270 con 0x7ff34815bd50 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff344c00640 1 -- 172.21.15.44:0/1697810594 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff330005410 con 0x7ff34815bd50 2024-09-13T08:43:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff34f200640 1 -- 172.21.15.44:0/1697810594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3481f9720 con 0x7ff34815bd50 2024-09-13T08:43:08.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff344c00640 1 -- 172.21.15.44:0/1697810594 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3300046d0 con 0x7ff34815bd50 2024-09-13T08:43:08.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff34f200640 1 -- 172.21.15.44:0/1697810594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3481f9a00 con 0x7ff34815bd50 2024-09-13T08:43:08.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff34f200640 1 -- 172.21.15.44:0/1697810594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3480c5dd0 con 0x7ff34815bd50 2024-09-13T08:43:08.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.895+0000 7ff344c00640 1 -- 172.21.15.44:0/1697810594 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7ff330004870 con 0x7ff34815bd50 2024-09-13T08:43:08.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.899+0000 7ff344c00640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3240737c0 0x7ff324075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:08.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.899+0000 7ff344c00640 1 -- 172.21.15.44:0/1697810594 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4971+0+0 (secure 0 0 0) 0x7ff33012d5e0 con 0x7ff34815bd50 2024-09-13T08:43:08.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.899+0000 7ff347e00640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3240737c0 0x7ff324075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:08.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.899+0000 7ff347e00640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3240737c0 0x7ff324075c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff338006fd0 tx=0x7ff338008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:08.903+0000 7ff344c00640 1 -- 172.21.15.44:0/1697810594 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff3300c6020 con 0x7ff34815bd50 2024-09-13T08:43:09.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.087+0000 7ff34f200640 1 -- 172.21.15.44:0/1697810594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3480d2600 con 0x7ff34815bd50 2024-09-13T08:43:09.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.087+0000 7ff344c00640 1 -- 172.21.15.44:0/1697810594 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+13087 (secure 0 0 0) 0x7ff330016410 con 0x7ff34815bd50 2024-09-13T08:43:09.092 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:09.092 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":53,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:43:07.709447+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/3141962538":"2024-09-14T08:43:04.686096+0000","172.21.15.44:6833/3141962538":"2024-09-14T08:43:04.686096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:43:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 -- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3240737c0 msgr2=0x7ff324075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3240737c0 0x7ff324075c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff338006fd0 tx=0x7ff338008040 comp rx=0 tx=0).stop 2024-09-13T08:43:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 -- 172.21.15.44:0/1697810594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34815bd50 msgr2=0x7ff3480cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34815bd50 0x7ff3480cb350 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff330009790 tx=0x7ff330004040 comp rx=0 tx=0).stop 2024-09-13T08:43:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 -- 172.21.15.44:0/1697810594 shutdown_connections 2024-09-13T08:43:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3240737c0 0x7ff324075c80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34815c130 0x7ff3480cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34815bd50 0x7ff3480cb350 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 --2- 172.21.15.44:0/1697810594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3480cf4e0 0x7ff34814d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 -- 172.21.15.44:0/1697810594 >> 172.21.15.44:0/1697810594 conn(0x7ff3480d1ea0 msgr2=0x7ff348152250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 -- 172.21.15.44:0/1697810594 shutdown_connections 2024-09-13T08:43:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.091+0000 7ff34f200640 1 -- 172.21.15.44:0/1697810594 wait complete. 2024-09-13T08:43:09.115 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T08:37:24.563344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T08:43:09.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T08:43:09.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.263+0000 7fd385400640 1 -- 172.21.15.44:0/1851209012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3800cbdb0 msgr2=0x7fd3800b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:09.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.263+0000 7fd385400640 1 --2- 172.21.15.44:0/1851209012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3800cbdb0 0x7fd3800b2740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd368009870 tx=0x7fd36801c820 comp rx=0 tx=0).stop 2024-09-13T08:43:09.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.263+0000 7fd385400640 1 -- 172.21.15.44:0/1851209012 shutdown_connections 2024-09-13T08:43:09.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.263+0000 7fd385400640 1 --2- 172.21.15.44:0/1851209012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3800cbdb0 0x7fd3800b2740 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.263+0000 7fd385400640 1 --2- 172.21.15.44:0/1851209012 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd38015ac70 0x7fd3800cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.263+0000 7fd385400640 1 -- 172.21.15.44:0/1851209012 >> 172.21.15.44:0/1851209012 conn(0x7fd3800cd740 msgr2=0x7fd3800cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.263+0000 7fd385400640 1 -- 172.21.15.44:0/1851209012 shutdown_connections 2024-09-13T08:43:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.263+0000 7fd385400640 1 -- 172.21.15.44:0/1851209012 wait complete. 2024-09-13T08:43:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.263+0000 7fd385400640 1 Processor -- start 2024-09-13T08:43:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.263+0000 7fd385400640 1 -- start start 2024-09-13T08:43:09.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd385400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3800b2c80 0x7fd38015a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:09.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd385400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3800cbdb0 0x7fd3800caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:09.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd385400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38015ac70 0x7fd3800c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:09.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd385400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd380167c00 con 0x7fd3800cbdb0 2024-09-13T08:43:09.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd385400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd380167a80 con 0x7fd38015ac70 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd385400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd380167d80 con 0x7fd3800b2c80 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd377e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3800cbdb0 0x7fd3800caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd377e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3800cbdb0 0x7fd3800caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38302/0 (socket says 172.21.15.44:38302) 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd377e00640 1 -- 172.21.15.44:0/873072178 learned_addr learned my addr 172.21.15.44:0/873072178 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd37e800640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3800b2c80 0x7fd38015a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd37f200640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38015ac70 0x7fd3800c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd377e00640 1 -- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3800b2c80 msgr2=0x7fd38015a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd377e00640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3800b2c80 0x7fd38015a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd377e00640 1 -- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38015ac70 msgr2=0x7fd3800c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd377e00640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38015ac70 0x7fd3800c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd377e00640 1 -- 172.21.15.44:0/873072178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3800c85e0 con 0x7fd3800cbdb0 2024-09-13T08:43:09.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd377e00640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3800cbdb0 0x7fd3800caaf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd36801aed0 tx=0x7fd3680a66e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:09.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd376a00640 1 -- 172.21.15.44:0/873072178 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3680040c0 con 0x7fd3800cbdb0 2024-09-13T08:43:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd376a00640 1 -- 172.21.15.44:0/873072178 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd368004350 con 0x7fd3800cbdb0 2024-09-13T08:43:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd376a00640 1 -- 172.21.15.44:0/873072178 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368005280 con 0x7fd3800cbdb0 2024-09-13T08:43:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd385400640 1 -- 172.21.15.44:0/873072178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3800c88c0 con 0x7fd3800cbdb0 2024-09-13T08:43:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd385400640 1 -- 172.21.15.44:0/873072178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3800ca5d0 con 0x7fd3800cbdb0 2024-09-13T08:43:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.267+0000 7fd385400640 1 -- 172.21.15.44:0/873072178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3800b2740 con 0x7fd3800cbdb0 2024-09-13T08:43:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.271+0000 7fd376a00640 1 -- 172.21.15.44:0/873072178 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fd3680c2020 con 0x7fd3800cbdb0 2024-09-13T08:43:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.271+0000 7fd376a00640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd35c0737f0 0x7fd35c075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.271+0000 7fd376a00640 1 -- 172.21.15.44:0/873072178 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4971+0+0 (secure 0 0 0) 0x7fd36812d390 con 0x7fd3800cbdb0 2024-09-13T08:43:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.271+0000 7fd37e800640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd35c0737f0 0x7fd35c075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:09.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.271+0000 7fd37e800640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd35c0737f0 0x7fd35c075cb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd370009ad0 tx=0x7fd3700091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:09.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.275+0000 7fd376a00640 1 -- 172.21.15.44:0/873072178 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd368016410 con 0x7fd3800cbdb0 2024-09-13T08:43:09.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.451+0000 7fd385400640 1 -- 172.21.15.44:0/873072178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3800c5710 con 0x7fd3800cbdb0 2024-09-13T08:43:09.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.451+0000 7fd376a00640 1 -- 172.21.15.44:0/873072178 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7fd3680fa030 con 0x7fd3800cbdb0 2024-09-13T08:43:09.456 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T08:43:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 -- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd35c0737f0 msgr2=0x7fd35c075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd35c0737f0 0x7fd35c075cb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd370009ad0 tx=0x7fd3700091e0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 -- 172.21.15.44:0/873072178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3800cbdb0 msgr2=0x7fd3800caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3800cbdb0 0x7fd3800caaf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd36801aed0 tx=0x7fd3680a66e0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 -- 172.21.15.44:0/873072178 shutdown_connections 2024-09-13T08:43:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd35c0737f0 0x7fd35c075cb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38015ac70 0x7fd3800c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3800cbdb0 0x7fd3800caaf0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 --2- 172.21.15.44:0/873072178 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3800b2c80 0x7fd38015a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 -- 172.21.15.44:0/873072178 >> 172.21.15.44:0/873072178 conn(0x7fd3800cd740 msgr2=0x7fd380157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 -- 172.21.15.44:0/873072178 shutdown_connections 2024-09-13T08:43:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.455+0000 7fd385400640 1 -- 172.21.15.44:0/873072178 wait complete. 2024-09-13T08:43:09.477 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-13T08:43:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.619+0000 7f4eed800640 1 -- 172.21.15.44:0/2314286618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ecc005220 msgr2=0x7f4ecc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.619+0000 7f4eed800640 1 --2- 172.21.15.44:0/2314286618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ecc005220 0x7f4ecc00b9d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009d40 tx=0x7f4ee001cfc0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.619+0000 7f4eece00640 1 -- 172.21.15.44:0/2314286618 reap_dead start 2024-09-13T08:43:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.619+0000 7f4eed800640 1 -- 172.21.15.44:0/2314286618 shutdown_connections 2024-09-13T08:43:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.619+0000 7f4eed800640 1 -- 172.21.15.44:0/2314286618 >> 172.21.15.44:0/2314286618 conn(0x7f4ee80d3e00 msgr2=0x7f4ee80d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.619+0000 7f4eed800640 1 -- 172.21.15.44:0/2314286618 shutdown_connections 2024-09-13T08:43:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.619+0000 7f4eed800640 1 -- 172.21.15.44:0/2314286618 wait complete. 2024-09-13T08:43:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eed800640 1 Processor -- start 2024-09-13T08:43:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eed800640 1 -- start start 2024-09-13T08:43:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eed800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4ecc005220 0x7f4ee81f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eed800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee80c78e0 0x7f4ee8143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eed800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee815bbc0 0x7f4ee815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eed800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8166ca0 con 0x7f4ee80c78e0 2024-09-13T08:43:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eed800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee80b81a0 con 0x7f4ee815bbc0 2024-09-13T08:43:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eed800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee8166e20 con 0x7f4ecc005220 2024-09-13T08:43:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4edfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee80c78e0 0x7f4ee8143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4edfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee80c78e0 0x7f4ee8143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38316/0 (socket says 172.21.15.44:38316) 2024-09-13T08:43:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4edfe00640 1 -- 172.21.15.44:0/240328882 learned_addr learned my addr 172.21.15.44:0/240328882 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eece00640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee815bbc0 0x7f4ee815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4edfe00640 1 -- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4ecc005220 msgr2=0x7f4ee81f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4edfe00640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4ecc005220 0x7f4ee81f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4edfe00640 1 -- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee815bbc0 msgr2=0x7f4ee815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4edfe00640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee815bbc0 0x7f4ee815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4edfe00640 1 -- 172.21.15.44:0/240328882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee80b5cd0 con 0x7f4ee80c78e0 2024-09-13T08:43:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eece00640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee815bbc0 0x7f4ee815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:43:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4edfe00640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee80c78e0 0x7f4ee8143e80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0003f20 tx=0x7f4ed0001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4ee4c00640 1 -- 172.21.15.44:0/240328882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0011640 con 0x7f4ee80c78e0 2024-09-13T08:43:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4ee4c00640 1 -- 172.21.15.44:0/240328882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ed0016020 con 0x7f4ee80c78e0 2024-09-13T08:43:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4ee4c00640 1 -- 172.21.15.44:0/240328882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0011a80 con 0x7f4ee80c78e0 2024-09-13T08:43:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eed800640 1 -- 172.21.15.44:0/240328882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee81f8f70 con 0x7f4ee80c78e0 2024-09-13T08:43:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.623+0000 7f4eed800640 1 -- 172.21.15.44:0/240328882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee80ca710 con 0x7f4ee80c78e0 2024-09-13T08:43:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.627+0000 7f4eed800640 1 -- 172.21.15.44:0/240328882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee80d0540 con 0x7f4ee80c78e0 2024-09-13T08:43:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.627+0000 7f4ee4c00640 1 -- 172.21.15.44:0/240328882 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f4ed0011c20 con 0x7f4ee80c78e0 2024-09-13T08:43:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.627+0000 7f4ee4c00640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ec40737c0 0x7f4ec4075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.627+0000 7f4ee4c00640 1 -- 172.21.15.44:0/240328882 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4971+0+0 (secure 0 0 0) 0x7f4ed0093400 con 0x7f4ee80c78e0 2024-09-13T08:43:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.627+0000 7f4ee7e00640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ec40737c0 0x7f4ec4075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.627+0000 7f4ee7e00640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ec40737c0 0x7f4ec4075c80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8001800 tx=0x7f4ed8000f40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.631+0000 7f4ee4c00640 1 -- 172.21.15.44:0/240328882 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4ed0020020 con 0x7f4ee80c78e0 2024-09-13T08:43:09.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:09.867+0000 7f4eed800640 1 -- 172.21.15.44:0/240328882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7f4ee815fa40 con 0x7f4ee80c78e0 2024-09-13T08:43:10.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4ee4c00640 1 -- 172.21.15.44:0/240328882 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v256) ==== 170+0+0 (secure 0 0 0) 0x7f4ee81f8f70 con 0x7f4ee80c78e0 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 -- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ec40737c0 msgr2=0x7f4ec4075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ec40737c0 0x7f4ec4075c80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8001800 tx=0x7f4ed8000f40 comp rx=0 tx=0).stop 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 -- 172.21.15.44:0/240328882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee80c78e0 msgr2=0x7f4ee8143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee80c78e0 0x7f4ee8143e80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0003f20 tx=0x7f4ed0001700 comp rx=0 tx=0).stop 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 -- 172.21.15.44:0/240328882 shutdown_connections 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ec40737c0 0x7f4ec4075c80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee815bbc0 0x7f4ee815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee80c78e0 0x7f4ee8143e80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 --2- 172.21.15.44:0/240328882 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4ecc005220 0x7f4ee81f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 -- 172.21.15.44:0/240328882 >> 172.21.15.44:0/240328882 conn(0x7f4ee80d3e00 msgr2=0x7f4ee806ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 -- 172.21.15.44:0/240328882 shutdown_connections 2024-09-13T08:43:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.727+0000 7f4eed800640 1 -- 172.21.15.44:0/240328882 wait complete. 2024-09-13T08:43:10.758 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-13T08:43:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 -- 172.21.15.44:0/1993063023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f380cdb30 msgr2=0x7f4f380af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:10.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 --2- 172.21.15.44:0/1993063023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f380cdb30 0x7f4f380af0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009bf0 tx=0x7f4f3001cb00 comp rx=0 tx=0).stop 2024-09-13T08:43:10.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 -- 172.21.15.44:0/1993063023 shutdown_connections 2024-09-13T08:43:10.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 --2- 172.21.15.44:0/1993063023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f380cdb30 0x7f4f380af0c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:10.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 -- 172.21.15.44:0/1993063023 >> 172.21.15.44:0/1993063023 conn(0x7f4f380d1dc0 msgr2=0x7f4f380d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:10.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 -- 172.21.15.44:0/1993063023 shutdown_connections 2024-09-13T08:43:10.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 -- 172.21.15.44:0/1993063023 wait complete. 2024-09-13T08:43:10.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 Processor -- start 2024-09-13T08:43:10.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 -- start start 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f380cdb30 0x7f4f38163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f380ced80 0x7f4f380b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f3815bea0 0x7f4f3815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3816a8f0 con 0x7f4f380cdb30 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3816a770 con 0x7f4f380ced80 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f3da00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3816aa70 con 0x7f4f3815bea0 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f36a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f380cdb30 0x7f4f38163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f36a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f380cdb30 0x7f4f38163ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38328/0 (socket says 172.21.15.44:38328) 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f36a00640 1 -- 172.21.15.44:0/1474295227 learned_addr learned my addr 172.21.15.44:0/1474295227 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f37400640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f3815bea0 0x7f4f3815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f2fe00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f380ced80 0x7f4f380b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f36a00640 1 -- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f3815bea0 msgr2=0x7f4f3815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f36a00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f3815bea0 0x7f4f3815a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f36a00640 1 -- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f380ced80 msgr2=0x7f4f380b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f36a00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f380ced80 0x7f4f380b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f36a00640 1 -- 172.21.15.44:0/1474295227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f381f8be0 con 0x7f4f380cdb30 2024-09-13T08:43:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f37400640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f3815bea0 0x7f4f3815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f2fe00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f380ced80 0x7f4f380b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.907+0000 7f4f36a00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f380cdb30 0x7f4f38163ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f28001230 tx=0x7f4f2800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.911+0000 7f4f2ea00640 1 -- 172.21.15.44:0/1474295227 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28016020 con 0x7f4f380cdb30 2024-09-13T08:43:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.911+0000 7f4f2ea00640 1 -- 172.21.15.44:0/1474295227 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f2800a930 con 0x7f4f380cdb30 2024-09-13T08:43:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.911+0000 7f4f3da00640 1 -- 172.21.15.44:0/1474295227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f381f8e80 con 0x7f4f380cdb30 2024-09-13T08:43:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.911+0000 7f4f2ea00640 1 -- 172.21.15.44:0/1474295227 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800ad80 con 0x7f4f380cdb30 2024-09-13T08:43:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.911+0000 7f4f3da00640 1 -- 172.21.15.44:0/1474295227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3815da00 con 0x7f4f380cdb30 2024-09-13T08:43:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.911+0000 7f4f3da00640 1 -- 172.21.15.44:0/1474295227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f380ce670 con 0x7f4f380cdb30 2024-09-13T08:43:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.911+0000 7f4f2ea00640 1 -- 172.21.15.44:0/1474295227 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f4f28019020 con 0x7f4f380cdb30 2024-09-13T08:43:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.911+0000 7f4f2ea00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4f140737c0 0x7f4f14075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.911+0000 7f4f2ea00640 1 -- 172.21.15.44:0/1474295227 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4971+0+0 (secure 0 0 0) 0x7f4f28092860 con 0x7f4f380cdb30 2024-09-13T08:43:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.915+0000 7f4f2fe00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4f140737c0 0x7f4f14075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.915+0000 7f4f2fe00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4f140737c0 0x7f4f14075c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4f380b01b0 tx=0x7f4f20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:10.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:10.915+0000 7f4f2ea00640 1 -- 172.21.15.44:0/1474295227 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4f28020020 con 0x7f4f380cdb30 2024-09-13T08:43:11.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 -- 172.21.15.44:0/1474295227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f4f3806fa70 con 0x7f4f380cdb30 2024-09-13T08:43:11.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f2ea00640 1 -- 172.21.15.44:0/1474295227 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+1998 (secure 0 0 0) 0x7f4f2805f500 con 0x7f4f380cdb30 2024-09-13T08:43:11.095 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-13T08:43:11.095 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-09-13T08:43:11.095 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-13T08:43:11.095 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-13T08:43:11.095 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4094.620852 2024-09-13T08:43:11.095 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3964.014083 2024-09-13T08:43:11.095 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3978.263337 2024-09-13T08:43:11.095 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4002.298162 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4667.000623 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4664.046790 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6403.016405 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4696.908546 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 2/2 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-13T08:43:11.096 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-09-13T08:43:11.097 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-09-13T08:43:11.097 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-09-13T08:43:11.097 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-09-13T08:43:11.097 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-09-13T08:43:11.097 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-09-13T08:43:11.097 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-09-13T08:43:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 -- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4f140737c0 msgr2=0x7f4f14075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4f140737c0 0x7f4f14075c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4f380b01b0 tx=0x7f4f20009040 comp rx=0 tx=0).stop 2024-09-13T08:43:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 -- 172.21.15.44:0/1474295227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f380cdb30 msgr2=0x7f4f38163ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f380cdb30 0x7f4f38163ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f28001230 tx=0x7f4f2800cb70 comp rx=0 tx=0).stop 2024-09-13T08:43:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 -- 172.21.15.44:0/1474295227 shutdown_connections 2024-09-13T08:43:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4f140737c0 0x7f4f14075c80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f3815bea0 0x7f4f3815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f380ced80 0x7f4f380b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 --2- 172.21.15.44:0/1474295227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f380cdb30 0x7f4f38163ed0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 -- 172.21.15.44:0/1474295227 >> 172.21.15.44:0/1474295227 conn(0x7f4f380d1dc0 msgr2=0x7f4f380b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 -- 172.21.15.44:0/1474295227 shutdown_connections 2024-09-13T08:43:11.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.091+0000 7f4f3da00640 1 -- 172.21.15.44:0/1474295227 wait complete. 2024-09-13T08:43:11.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:43:11.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.263+0000 7fdeef200640 1 -- 172.21.15.44:0/4144160506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdef00b0d10 msgr2=0x7fdef00d1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:11.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.263+0000 7fdeef200640 1 --2- 172.21.15.44:0/4144160506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdef00b0d10 0x7fdef00d1bb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009840 tx=0x7fdedc01c820 comp rx=0 tx=0).stop 2024-09-13T08:43:11.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.263+0000 7fdeef200640 1 -- 172.21.15.44:0/4144160506 shutdown_connections 2024-09-13T08:43:11.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.263+0000 7fdeef200640 1 --2- 172.21.15.44:0/4144160506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdef00d20f0 0x7fdef00cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.263+0000 7fdeef200640 1 --2- 172.21.15.44:0/4144160506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdef00b0d10 0x7fdef00d1bb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.263+0000 7fdeef200640 1 -- 172.21.15.44:0/4144160506 >> 172.21.15.44:0/4144160506 conn(0x7fdef009aa10 msgr2=0x7fdef009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:11.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.263+0000 7fdeef200640 1 -- 172.21.15.44:0/4144160506 shutdown_connections 2024-09-13T08:43:11.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.263+0000 7fdeef200640 1 -- 172.21.15.44:0/4144160506 wait complete. 2024-09-13T08:43:11.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 Processor -- start 2024-09-13T08:43:11.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 -- start start 2024-09-13T08:43:11.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdef00b0d10 0x7fdef014ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:11.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdef00d20f0 0x7fdef01f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:11.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdef015bd40 0x7fdef01f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:11.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdef0168c50 con 0x7fdef015bd40 2024-09-13T08:43:11.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdef0168ad0 con 0x7fdef00d20f0 2024-09-13T08:43:11.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdef0168dd0 con 0x7fdef00b0d10 2024-09-13T08:43:11.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdef015bd40 0x7fdef01f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:11.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdef015bd40 0x7fdef01f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38336/0 (socket says 172.21.15.44:38336) 2024-09-13T08:43:11.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeee800640 1 -- 172.21.15.44:0/1831484203 learned_addr learned my addr 172.21.15.44:0/1831484203 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:11.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeede00640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdef00b0d10 0x7fdef014ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:11.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeed400640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdef00d20f0 0x7fdef01f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:11.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeee800640 1 -- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdef00b0d10 msgr2=0x7fdef014ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:11.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeee800640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdef00b0d10 0x7fdef014ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeee800640 1 -- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdef00d20f0 msgr2=0x7fdef01f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:11.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeee800640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdef00d20f0 0x7fdef01f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeee800640 1 -- 172.21.15.44:0/1831484203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef01f86b0 con 0x7fdef015bd40 2024-09-13T08:43:11.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeee800640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdef015bd40 0x7fdef01f7a90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdee800eb20 tx=0x7fdee800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:11.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdee6000640 1 -- 172.21.15.44:0/1831484203 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8004b30 con 0x7fdef015bd40 2024-09-13T08:43:11.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdee6000640 1 -- 172.21.15.44:0/1831484203 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdee8004cf0 con 0x7fdef015bd40 2024-09-13T08:43:11.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 -- 172.21.15.44:0/1831484203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef01f83d0 con 0x7fdef015bd40 2024-09-13T08:43:11.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdee6000640 1 -- 172.21.15.44:0/1831484203 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8005280 con 0x7fdef015bd40 2024-09-13T08:43:11.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 -- 172.21.15.44:0/1831484203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef0157220 con 0x7fdef015bd40 2024-09-13T08:43:11.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.267+0000 7fdeef200640 1 -- 172.21.15.44:0/1831484203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef00d1bb0 con 0x7fdef015bd40 2024-09-13T08:43:11.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.271+0000 7fdee6000640 1 -- 172.21.15.44:0/1831484203 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fdee8005420 con 0x7fdef015bd40 2024-09-13T08:43:11.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.271+0000 7fdee6000640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fded00737c0 0x7fded0075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:11.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.271+0000 7fdee6000640 1 -- 172.21.15.44:0/1831484203 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4971+0+0 (secure 0 0 0) 0x7fdee80932c0 con 0x7fdef015bd40 2024-09-13T08:43:11.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.271+0000 7fdeede00640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fded00737c0 0x7fded0075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:11.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.271+0000 7fdeede00640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fded00737c0 0x7fded0075c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdee00046c0 tx=0x7fdee0007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:11.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.275+0000 7fdee6000640 1 -- 172.21.15.44:0/1831484203 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdee805f910 con 0x7fdef015bd40 2024-09-13T08:43:11.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.451+0000 7fdeef200640 1 -- 172.21.15.44:0/1831484203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdef009b0d0 con 0x7fdef015bd40 2024-09-13T08:43:11.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdee6000640 1 -- 172.21.15.44:0/1831484203 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fdee8064340 con 0x7fdef015bd40 2024-09-13T08:43:11.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 -- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fded00737c0 msgr2=0x7fded0075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:11.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fded00737c0 0x7fded0075c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdee00046c0 tx=0x7fdee0007ec0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 -- 172.21.15.44:0/1831484203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdef015bd40 msgr2=0x7fdef01f7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdef015bd40 0x7fdef01f7a90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdee800eb20 tx=0x7fdee800c2d0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 -- 172.21.15.44:0/1831484203 shutdown_connections 2024-09-13T08:43:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fded00737c0 0x7fded0075c80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdef015bd40 0x7fdef01f7a90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdef00d20f0 0x7fdef01f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 --2- 172.21.15.44:0/1831484203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdef00b0d10 0x7fdef014ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 -- 172.21.15.44:0/1831484203 >> 172.21.15.44:0/1831484203 conn(0x7fdef009aa10 msgr2=0x7fdef01511a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 -- 172.21.15.44:0/1831484203 shutdown_connections 2024-09-13T08:43:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.455+0000 7fdeef200640 1 -- 172.21.15.44:0/1831484203 wait complete. 2024-09-13T08:43:11.479 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:43:11.480 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:43:11.567 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:43:11.717 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:43:11.717 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:43:11.717 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:11.726 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:43:11.726 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:43:11.726 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:11.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:43:11.737 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:43:11.737 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:11.781 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:43:11.781 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:43:11.781 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:11.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:43:11.790 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T08:43:11.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:43:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.971+0000 7fe16f400640 1 -- 172.21.15.44:0/918324728 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1680cdb30 msgr2=0x7fe1680aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.971+0000 7fe16f400640 1 --2- 172.21.15.44:0/918324728 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1680cdb30 0x7fe1680aeef0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe160009c80 tx=0x7fe16001cda0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.971+0000 7fe16f400640 1 -- 172.21.15.44:0/918324728 shutdown_connections 2024-09-13T08:43:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.971+0000 7fe16f400640 1 --2- 172.21.15.44:0/918324728 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1680cdb30 0x7fe1680aeef0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.971+0000 7fe16f400640 1 --2- 172.21.15.44:0/918324728 >> v2:172.21.15.129:3300/0 conn(0x7fe1680ced80 0x7fe1680ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.971+0000 7fe16f400640 1 -- 172.21.15.44:0/918324728 >> 172.21.15.44:0/918324728 conn(0x7fe1680d1dc0 msgr2=0x7fe1680d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.971+0000 7fe16f400640 1 -- 172.21.15.44:0/918324728 shutdown_connections 2024-09-13T08:43:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.971+0000 7fe16f400640 1 -- 172.21.15.44:0/918324728 wait complete. 2024-09-13T08:43:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.971+0000 7fe16f400640 1 Processor -- start 2024-09-13T08:43:11.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.971+0000 7fe16f400640 1 -- start start 2024-09-13T08:43:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1680cdb30 0x7fe168156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16f400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1680ced80 0x7fe168156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe16815bea0 0x7fe168158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16f400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe168169340 con 0x7fe16815bea0 2024-09-13T08:43:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16f400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681691c0 con 0x7fe1680cdb30 2024-09-13T08:43:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16f400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe1681694c0 con 0x7fe1680ced80 2024-09-13T08:43:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe16815bea0 0x7fe168158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe16815bea0 0x7fe168158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38352/0 (socket says 172.21.15.44:38352) 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16d400640 1 -- 172.21.15.44:0/2629595729 learned_addr learned my addr 172.21.15.44:0/2629595729 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe167e00640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1680ced80 0x7fe168156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16d400640 1 -- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1680ced80 msgr2=0x7fe168156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16ca00640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1680cdb30 0x7fe168156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16d400640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1680ced80 0x7fe168156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16d400640 1 -- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1680cdb30 msgr2=0x7fe168156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16d400640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1680cdb30 0x7fe168156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16d400640 1 -- 172.21.15.44:0/2629595729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1680c98b0 con 0x7fe16815bea0 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe167e00640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1680ced80 0x7fe168156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16ca00640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1680cdb30 0x7fe168156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16d400640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe16815bea0 0x7fe168158b90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe16001b450 tx=0x7fe1600092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe164c00640 1 -- 172.21.15.44:0/2629595729 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1600bb070 con 0x7fe16815bea0 2024-09-13T08:43:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe164c00640 1 -- 172.21.15.44:0/2629595729 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe160004d40 con 0x7fe16815bea0 2024-09-13T08:43:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe164c00640 1 -- 172.21.15.44:0/2629595729 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1600b2040 con 0x7fe16815bea0 2024-09-13T08:43:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16f400640 1 -- 172.21.15.44:0/2629595729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1680c9b50 con 0x7fe16815bea0 2024-09-13T08:43:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16f400640 1 -- 172.21.15.44:0/2629595729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16806df80 con 0x7fe16815bea0 2024-09-13T08:43:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.975+0000 7fe16f400640 1 -- 172.21.15.44:0/2629595729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1680b7b10 con 0x7fe16815bea0 2024-09-13T08:43:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.979+0000 7fe164c00640 1 -- 172.21.15.44:0/2629595729 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fe1600be070 con 0x7fe16815bea0 2024-09-13T08:43:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.979+0000 7fe164c00640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1480737f0 0x7fe148075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.979+0000 7fe164c00640 1 -- 172.21.15.44:0/2629595729 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4971+0+0 (secure 0 0 0) 0x7fe16012c3f0 con 0x7fe16815bea0 2024-09-13T08:43:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.979+0000 7fe16ca00640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1480737f0 0x7fe148075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:11.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.979+0000 7fe16ca00640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1480737f0 0x7fe148075cb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe1680cc3d0 tx=0x7fe15c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:11.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:11.983+0000 7fe164c00640 1 -- 172.21.15.44:0/2629595729 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe160016990 con 0x7fe16815bea0 2024-09-13T08:43:12.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.191+0000 7fe16f400640 1 -- 172.21.15.44:0/2629595729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe16806dd30 con 0x7fe16815bea0 2024-09-13T08:43:12.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.191+0000 7fe164c00640 1 -- 172.21.15.44:0/2629595729 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+3917 (secure 0 0 0) 0x7fe1600f9010 con 0x7fe16815bea0 2024-09-13T08:43:12.195 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:12.196 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":30,"btime":"2024-09-13T08:43:05:696496+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3367280951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3367280951},{"type":"v1","addr":"172.21.15.129:6835","nonce":3367280951}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1057940180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1057940180},{"type":"v1","addr":"172.21.15.129:6837","nonce":1057940180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5319,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1413618927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413618927},{"type":"v1","addr":"172.21.15.44:6834","nonce":1413618927}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2642135493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2642135493},{"type":"v1","addr":"172.21.15.44:6835","nonce":2642135493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-09-13T08:43:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 30 2024-09-13T08:43:12.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 -- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1480737f0 msgr2=0x7fe148075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:12.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1480737f0 0x7fe148075cb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe1680cc3d0 tx=0x7fe15c009040 comp rx=0 tx=0).stop 2024-09-13T08:43:12.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 -- 172.21.15.44:0/2629595729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe16815bea0 msgr2=0x7fe168158b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:12.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe16815bea0 0x7fe168158b90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe16001b450 tx=0x7fe1600092e0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 -- 172.21.15.44:0/2629595729 shutdown_connections 2024-09-13T08:43:12.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1480737f0 0x7fe148075cb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe16815bea0 0x7fe168158b90 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1680ced80 0x7fe168156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 --2- 172.21.15.44:0/2629595729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1680cdb30 0x7fe168156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 -- 172.21.15.44:0/2629595729 >> 172.21.15.44:0/2629595729 conn(0x7fe1680d1dc0 msgr2=0x7fe1680d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 -- 172.21.15.44:0/2629595729 shutdown_connections 2024-09-13T08:43:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.195+0000 7fe16f400640 1 -- 172.21.15.44:0/2629595729 wait complete. 2024-09-13T08:43:12.218 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T08:43:12.218 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T08:43:12.219 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T08:43:12.219 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:43:12.219 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T08:43:12.219 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T08:43:12.219 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T08:43:12.219 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:43:12.219 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T08:43:12.219 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T08:43:12.220 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T08:43:12.220 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:43:12.220 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T08:43:12.220 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T08:43:12.220 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T08:43:12.220 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:43:12.232 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T08:43:12.232 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T08:43:12.233 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T08:43:12.233 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T08:43:12.235 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T08:43:12.235 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T08:43:12.237 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T08:43:12.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T08:43:12.276 INFO:tasks.ceph.mds.a:Started 2024-09-13T08:43:12.276 DEBUG:teuthology.parallel:result is None 2024-09-13T08:43:12.277 INFO:tasks.ceph.mds.c:Started 2024-09-13T08:43:12.277 DEBUG:teuthology.parallel:result is None 2024-09-13T08:43:12.284 INFO:tasks.ceph.mds.b:Started 2024-09-13T08:43:12.284 DEBUG:teuthology.parallel:result is None 2024-09-13T08:43:12.286 INFO:tasks.ceph.mds.d:Started 2024-09-13T08:43:12.286 DEBUG:teuthology.parallel:result is None 2024-09-13T08:43:12.287 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T08:43:12.363 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:43:12.364 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:43:12.380 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:43:12.380 INFO:tasks.ceph.mds.b.smithi129.stdout:starting mds.b at 2024-09-13T08:43:12.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 -- 172.21.15.44:0/1907905053 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbe4161660 msgr2=0x7fcbe40d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:12.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 --2- 172.21.15.44:0/1907905053 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbe4161660 0x7fcbe40d16a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009870 tx=0x7fcbd801c550 comp rx=0 tx=0).stop 2024-09-13T08:43:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 -- 172.21.15.44:0/1907905053 shutdown_connections 2024-09-13T08:43:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 --2- 172.21.15.44:0/1907905053 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbe4161660 0x7fcbe40d16a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 -- 172.21.15.44:0/1907905053 >> 172.21.15.44:0/1907905053 conn(0x7fcbe40d1e30 msgr2=0x7fcbe40d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 -- 172.21.15.44:0/1907905053 shutdown_connections 2024-09-13T08:43:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 -- 172.21.15.44:0/1907905053 wait complete. 2024-09-13T08:43:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 Processor -- start 2024-09-13T08:43:12.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 -- start start 2024-09-13T08:43:12.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbe406dd30 0x7fcbe4158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:12.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbe40b33d0 0x7fcbe414de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:12.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbe4161660 0x7fcbe40b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:12.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe416c850 con 0x7fcbe4161660 2024-09-13T08:43:12.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe416c6d0 con 0x7fcbe406dd30 2024-09-13T08:43:12.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.439+0000 7fcbebe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe416c9d0 con 0x7fcbe40b33d0 2024-09-13T08:43:12.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbe4161660 0x7fcbe40b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:12.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe8a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbe40b33d0 0x7fcbe414de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:12.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbe406dd30 0x7fcbe4158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:12.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbe406dd30 0x7fcbe4158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:53056/0 (socket says 172.21.15.44:53056) 2024-09-13T08:43:12.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9400640 1 -- 172.21.15.44:0/8280550 learned_addr learned my addr 172.21.15.44:0/8280550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:12.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe8a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbe40b33d0 0x7fcbe414de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:38490/0 (socket says 172.21.15.44:38490) 2024-09-13T08:43:12.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9e00640 1 -- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbe40b33d0 msgr2=0x7fcbe414de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:12.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9e00640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbe40b33d0 0x7fcbe414de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9e00640 1 -- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbe406dd30 msgr2=0x7fcbe4158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:12.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9e00640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbe406dd30 0x7fcbe4158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9e00640 1 -- 172.21.15.44:0/8280550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe40b6c90 con 0x7fcbe4161660 2024-09-13T08:43:12.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9400640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbe406dd30 0x7fcbe4158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:43:12.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe9e00640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbe4161660 0x7fcbe40b6240 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00caa0 tx=0x7fcbdc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:12.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe1600640 1 -- 172.21.15.44:0/8280550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc003d90 con 0x7fcbe4161660 2024-09-13T08:43:12.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe1600640 1 -- 172.21.15.44:0/8280550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbdc005070 con 0x7fcbe4161660 2024-09-13T08:43:12.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe1600640 1 -- 172.21.15.44:0/8280550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0054c0 con 0x7fcbe4161660 2024-09-13T08:43:12.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbebe00640 1 -- 172.21.15.44:0/8280550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe4158080 con 0x7fcbe4161660 2024-09-13T08:43:12.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbebe00640 1 -- 172.21.15.44:0/8280550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe4150c00 con 0x7fcbe4161660 2024-09-13T08:43:12.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbebe00640 1 -- 172.21.15.44:0/8280550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe40d16a0 con 0x7fcbe4161660 2024-09-13T08:43:12.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.443+0000 7fcbe1600640 1 -- 172.21.15.44:0/8280550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fcbdc005660 con 0x7fcbe4161660 2024-09-13T08:43:12.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.447+0000 7fcbe1600640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbc40737c0 0x7fcbc4075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:12.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.447+0000 7fcbe1600640 1 -- 172.21.15.44:0/8280550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4971+0+0 (secure 0 0 0) 0x7fcbdc092620 con 0x7fcbe4161660 2024-09-13T08:43:12.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.447+0000 7fcbe9400640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbc40737c0 0x7fcbc4075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:12.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.447+0000 7fcbe9400640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbc40737c0 0x7fcbc4075c80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcbd801a180 tx=0x7fcbd80b2040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:12.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.451+0000 7fcbe1600640 1 -- 172.21.15.44:0/8280550 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcbdc05ec70 con 0x7fcbe4161660 2024-09-13T08:43:12.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.639+0000 7fcbebe00640 1 -- 172.21.15.44:0/8280550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fcbe415af50 con 0x7fcbe4161660 2024-09-13T08:43:12.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.739+0000 7fcbe1600640 1 -- 172.21.15.44:0/8280550 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) ==== 97+0+0 (secure 0 0 0) 0x7fcbdc0636a0 con 0x7fcbe4161660 2024-09-13T08:43:12.744 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-09-13T08:43:12.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 -- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbc40737c0 msgr2=0x7fcbc4075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:12.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbc40737c0 0x7fcbc4075c80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcbd801a180 tx=0x7fcbd80b2040 comp rx=0 tx=0).stop 2024-09-13T08:43:12.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 -- 172.21.15.44:0/8280550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbe4161660 msgr2=0x7fcbe40b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:12.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbe4161660 0x7fcbe40b6240 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00caa0 tx=0x7fcbdc00cf60 comp rx=0 tx=0).stop 2024-09-13T08:43:12.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 -- 172.21.15.44:0/8280550 shutdown_connections 2024-09-13T08:43:12.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbc40737c0 0x7fcbc4075c80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbe4161660 0x7fcbe40b6240 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbe40b33d0 0x7fcbe414de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 --2- 172.21.15.44:0/8280550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbe406dd30 0x7fcbe4158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 -- 172.21.15.44:0/8280550 >> 172.21.15.44:0/8280550 conn(0x7fcbe40d1e30 msgr2=0x7fcbe40d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:12.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 -- 172.21.15.44:0/8280550 shutdown_connections 2024-09-13T08:43:12.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.743+0000 7fcbebe00640 1 -- 172.21.15.44:0/8280550 wait complete. 2024-09-13T08:43:12.769 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T08:43:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 -- 172.21.15.44:0/3534924972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a20003a40 msgr2=0x7f4a20003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 --2- 172.21.15.44:0/3534924972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a20003a40 0x7f4a20003e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0016f0 tx=0x7f4a2c01c7b0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3da00640 1 -- 172.21.15.44:0/3534924972 reap_dead start 2024-09-13T08:43:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 -- 172.21.15.44:0/3534924972 shutdown_connections 2024-09-13T08:43:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 -- 172.21.15.44:0/3534924972 >> 172.21.15.44:0/3534924972 conn(0x7f4a380b18f0 msgr2=0x7f4a380b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 -- 172.21.15.44:0/3534924972 shutdown_connections 2024-09-13T08:43:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 -- 172.21.15.44:0/3534924972 wait complete. 2024-09-13T08:43:12.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 Processor -- start 2024-09-13T08:43:12.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 -- start start 2024-09-13T08:43:12.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a20003a40 0x7f4a381360c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:12.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a38136600 0x7f4a380738c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:12.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a38073e00 0x7f4a38132c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:12.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38166060 con 0x7f4a20003a40 2024-09-13T08:43:12.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38165ee0 con 0x7f4a38073e00 2024-09-13T08:43:12.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3fa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4a381661e0 con 0x7f4a38136600 2024-09-13T08:43:12.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a20003a40 0x7f4a381360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:12.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a20003a40 0x7f4a381360c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46676/0 (socket says 172.21.15.44:46676) 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3d000640 1 -- 172.21.15.44:0/2344344048 learned_addr learned my addr 172.21.15.44:0/2344344048 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a37e00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a38136600 0x7f4a380738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3da00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a38073e00 0x7f4a38132c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3d000640 1 -- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a38136600 msgr2=0x7f4a380738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3d000640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a38136600 0x7f4a380738c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3d000640 1 -- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a38073e00 msgr2=0x7f4a38132c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3d000640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a38073e00 0x7f4a38132c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3d000640 1 -- 172.21.15.44:0/2344344048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a381fb3c0 con 0x7f4a20003a40 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.915+0000 7f4a3da00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a38073e00 0x7f4a38132c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.919+0000 7f4a3d000640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a20003a40 0x7f4a381360c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c01cc90 tx=0x7f4a2c003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.919+0000 7f4a34c00640 1 -- 172.21.15.44:0/2344344048 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0ba070 con 0x7f4a20003a40 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.919+0000 7f4a34c00640 1 -- 172.21.15.44:0/2344344048 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a2c0af040 con 0x7f4a20003a40 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.919+0000 7f4a34c00640 1 -- 172.21.15.44:0/2344344048 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0ae070 con 0x7f4a20003a40 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.919+0000 7f4a3fa00640 1 -- 172.21.15.44:0/2344344048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a381fb660 con 0x7f4a20003a40 2024-09-13T08:43:12.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.919+0000 7f4a3fa00640 1 -- 172.21.15.44:0/2344344048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a381fbc20 con 0x7f4a20003a40 2024-09-13T08:43:12.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.919+0000 7f4a3fa00640 1 -- 172.21.15.44:0/2344344048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a381fc2e0 con 0x7f4a20003a40 2024-09-13T08:43:12.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.919+0000 7f4a34c00640 1 -- 172.21.15.44:0/2344344048 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f4a2c006350 con 0x7f4a20003a40 2024-09-13T08:43:12.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.919+0000 7f4a34c00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a180737c0 0x7f4a18075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:12.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.919+0000 7f4a34c00640 1 -- 172.21.15.44:0/2344344048 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4885+0+0 (secure 0 0 0) 0x7f4a2c12ba80 con 0x7f4a20003a40 2024-09-13T08:43:12.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.923+0000 7f4a37e00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a180737c0 0x7f4a18075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:12.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.923+0000 7f4a37e00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a180737c0 0x7f4a18075c80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4a24000fc0 tx=0x7f4a240067b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:12.923+0000 7f4a34c00640 1 -- 172.21.15.44:0/2344344048 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4a381fc2e0 con 0x7f4a20003a40 2024-09-13T08:43:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.159+0000 7f4a3fa00640 1 -- 172.21.15.44:0/2344344048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4a3806fca0 con 0x7f4a20003a40 2024-09-13T08:43:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a34c00640 1 -- 172.21.15.44:0/2344344048 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f4a2c016510 con 0x7f4a20003a40 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw==", 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:43:13.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ==", 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA==", 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw==", 2024-09-13T08:43:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg==", 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:43:13.169 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg==", 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.170 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw==", 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig==", 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:43:13.171 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ==", 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-09-13T08:43:13.172 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g==", 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg==", 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:43:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg==", 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.174 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA==", 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w==", 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:43:13.175 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA==", 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w==", 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-09-13T08:43:13.177 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmDD2LJxAA3s0jEb2EfPsLcu0w145G3Q==", 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ==", 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.178 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg==", 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T08:43:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:43:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 -- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a180737c0 msgr2=0x7f4a18075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a180737c0 0x7f4a18075c80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4a24000fc0 tx=0x7f4a240067b0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 -- 172.21.15.44:0/2344344048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a20003a40 msgr2=0x7f4a381360c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a20003a40 0x7f4a381360c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c01cc90 tx=0x7f4a2c003bb0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 -- 172.21.15.44:0/2344344048 shutdown_connections 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4a180737c0 0x7f4a18075c80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a38073e00 0x7f4a38132c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4a38136600 0x7f4a380738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 --2- 172.21.15.44:0/2344344048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a20003a40 0x7f4a381360c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 -- 172.21.15.44:0/2344344048 >> 172.21.15.44:0/2344344048 conn(0x7f4a380b18f0 msgr2=0x7f4a380cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 -- 172.21.15.44:0/2344344048 shutdown_connections 2024-09-13T08:43:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.163+0000 7f4a3fa00640 1 -- 172.21.15.44:0/2344344048 wait complete. 2024-09-13T08:43:13.187 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T08:43:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3304021109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e1815f900 msgr2=0x7f4e1815fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 --2- 172.21.15.44:0/3304021109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e1815f900 0x7f4e1815fd80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4e18069820 tx=0x7f4e1001ccf0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3304021109 shutdown_connections 2024-09-13T08:43:13.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 --2- 172.21.15.44:0/3304021109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e1815f900 0x7f4e1815fd80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 --2- 172.21.15.44:0/3304021109 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e180cdae0 0x7f4e180cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 --2- 172.21.15.44:0/3304021109 >> v2:172.21.15.129:3300/0 conn(0x7f4e1815f300 0x7f4e180ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3304021109 >> 172.21.15.44:0/3304021109 conn(0x7f4e180b18f0 msgr2=0x7f4e180b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:13.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3304021109 shutdown_connections 2024-09-13T08:43:13.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3304021109 wait complete. 2024-09-13T08:43:13.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 Processor -- start 2024-09-13T08:43:13.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 -- start start 2024-09-13T08:43:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e180cdae0 0x7f4e180b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e1815f300 0x7f4e181f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e1815f900 0x7f4e180cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e180733f0 con 0x7f4e1815f300 2024-09-13T08:43:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1806ffe0 con 0x7f4e180cdae0 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e1ca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4e18073570 con 0x7f4e1815f900 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e1815f300 0x7f4e181f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e1815f300 0x7f4e181f7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46700/0 (socket says 172.21.15.44:46700) 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.331+0000 7f4e0fe00640 1 -- 172.21.15.44:0/3350704615 learned_addr learned my addr 172.21.15.44:0/3350704615 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e15e00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e180cdae0 0x7f4e180b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e16800640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e1815f900 0x7f4e180cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e0fe00640 1 -- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e1815f900 msgr2=0x7f4e180cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e0fe00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e1815f900 0x7f4e180cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e0fe00640 1 -- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e180cdae0 msgr2=0x7f4e180b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e0fe00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e180cdae0 0x7f4e180b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e0fe00640 1 -- 172.21.15.44:0/3350704615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e181f84c0 con 0x7f4e1815f300 2024-09-13T08:43:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e0fe00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e1815f300 0x7f4e181f7f80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000d980 tx=0x7f4e0000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e0e000640 1 -- 172.21.15.44:0/3350704615 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00017070 con 0x7f4e1815f300 2024-09-13T08:43:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e0e000640 1 -- 172.21.15.44:0/3350704615 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e00009d20 con 0x7f4e1815f300 2024-09-13T08:43:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e0e000640 1 -- 172.21.15.44:0/3350704615 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00004280 con 0x7f4e1815f300 2024-09-13T08:43:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3350704615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e18159d70 con 0x7f4e1815f300 2024-09-13T08:43:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3350704615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1815a330 con 0x7f4e1815f300 2024-09-13T08:43:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3350704615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e18069410 con 0x7f4e1815f300 2024-09-13T08:43:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.335+0000 7f4e0e000640 1 -- 172.21.15.44:0/3350704615 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f4e00003600 con 0x7f4e1815f300 2024-09-13T08:43:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.339+0000 7f4e0e000640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4df40737c0 0x7f4df4075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.339+0000 7f4e0e000640 1 -- 172.21.15.44:0/3350704615 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4885+0+0 (secure 0 0 0) 0x7f4e000927a0 con 0x7f4e1815f300 2024-09-13T08:43:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.339+0000 7f4e15e00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4df40737c0 0x7f4df4075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.343+0000 7f4e15e00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4df40737c0 0x7f4df4075c80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800bb80 tx=0x7f4e0800b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.343+0000 7f4e0e000640 1 -- 172.21.15.44:0/3350704615 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4e0005edf0 con 0x7f4e1815f300 2024-09-13T08:43:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3350704615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4e18151cf0 con 0x7f4e1815f300 2024-09-13T08:43:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e0e000640 1 -- 172.21.15.44:0/3350704615 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) ==== 84+0+65 (secure 0 0 0) 0x7f4e00063820 con 0x7f4e1815f300 2024-09-13T08:43:13.522 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:13.522 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:43:13.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:13.522 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T08:43:13.523 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T08:43:13.523 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:13.523 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:43:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4df40737c0 msgr2=0x7f4df4075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4df40737c0 0x7f4df4075c80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800bb80 tx=0x7f4e0800b390 comp rx=0 tx=0).stop 2024-09-13T08:43:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3350704615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e1815f300 msgr2=0x7f4e181f7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e1815f300 0x7f4e181f7f80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000d980 tx=0x7f4e0000de40 comp rx=0 tx=0).stop 2024-09-13T08:43:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3350704615 shutdown_connections 2024-09-13T08:43:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4df40737c0 0x7f4df4075c80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e1815f900 0x7f4e180cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e1815f300 0x7f4e181f7f80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 --2- 172.21.15.44:0/3350704615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e180cdae0 0x7f4e180b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3350704615 >> 172.21.15.44:0/3350704615 conn(0x7f4e180b18f0 msgr2=0x7f4e180b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3350704615 shutdown_connections 2024-09-13T08:43:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.519+0000 7f4e1ca00640 1 -- 172.21.15.44:0/3350704615 wait complete. 2024-09-13T08:43:13.543 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T08:43:13.544 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T08:43:13.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.691+0000 7f7aba400640 1 -- 172.21.15.44:0/3155074948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab40b0b60 msgr2=0x7f7ab40ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:13.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.691+0000 7f7aba400640 1 --2- 172.21.15.44:0/3155074948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab40b0b60 0x7f7ab40ccbd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009bc0 tx=0x7f7aac01f3b0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.691+0000 7f7aba400640 1 -- 172.21.15.44:0/3155074948 shutdown_connections 2024-09-13T08:43:13.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.691+0000 7f7aba400640 1 --2- 172.21.15.44:0/3155074948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab40b0b60 0x7f7ab40ccbd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.691+0000 7f7aba400640 1 --2- 172.21.15.44:0/3155074948 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7ab415c1f0 0x7f7ab40cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.691+0000 7f7aba400640 1 -- 172.21.15.44:0/3155074948 >> 172.21.15.44:0/3155074948 conn(0x7f7ab40d1da0 msgr2=0x7f7ab40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:13.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.691+0000 7f7aba400640 1 -- 172.21.15.44:0/3155074948 shutdown_connections 2024-09-13T08:43:13.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.691+0000 7f7aba400640 1 -- 172.21.15.44:0/3155074948 wait complete. 2024-09-13T08:43:13.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.691+0000 7f7aba400640 1 Processor -- start 2024-09-13T08:43:13.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.691+0000 7f7aba400640 1 -- start start 2024-09-13T08:43:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aba400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7ab40b0b60 0x7f7ab41465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aba400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab415be10 0x7f7ab40b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aba400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ab415c1f0 0x7f7ab41f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aba400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab40cd320 con 0x7f7ab415be10 2024-09-13T08:43:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aba400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab40cf590 con 0x7f7ab415c1f0 2024-09-13T08:43:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aba400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab40cd4a0 con 0x7f7ab40b0b60 2024-09-13T08:43:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aabe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab415be10 0x7f7ab40b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aabe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab415be10 0x7f7ab40b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46722/0 (socket says 172.21.15.44:46722) 2024-09-13T08:43:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aabe00640 1 -- 172.21.15.44:0/261578684 learned_addr learned my addr 172.21.15.44:0/261578684 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7ab3400640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7ab40b0b60 0x7f7ab41465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7ab3e00640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ab415c1f0 0x7f7ab41f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aabe00640 1 -- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7ab40b0b60 msgr2=0x7f7ab41465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aabe00640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7ab40b0b60 0x7f7ab41465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aabe00640 1 -- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ab415c1f0 msgr2=0x7f7ab41f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aabe00640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ab415c1f0 0x7f7ab41f7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aabe00640 1 -- 172.21.15.44:0/261578684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab40c8700 con 0x7f7ab415be10 2024-09-13T08:43:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aabe00640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab415be10 0x7f7ab40b73c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00c9f0 tx=0x7f7a9c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aab400640 1 -- 172.21.15.44:0/261578684 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c007bb0 con 0x7f7ab415be10 2024-09-13T08:43:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aab400640 1 -- 172.21.15.44:0/261578684 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a9c007e40 con 0x7f7ab415be10 2024-09-13T08:43:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aab400640 1 -- 172.21.15.44:0/261578684 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00f690 con 0x7f7ab415be10 2024-09-13T08:43:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aba400640 1 -- 172.21.15.44:0/261578684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab4146b30 con 0x7f7ab415be10 2024-09-13T08:43:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aba400640 1 -- 172.21.15.44:0/261578684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab40c7c20 con 0x7f7ab415be10 2024-09-13T08:43:13.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aba400640 1 -- 172.21.15.44:0/261578684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab40ccbd0 con 0x7f7ab415be10 2024-09-13T08:43:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.695+0000 7f7aab400640 1 -- 172.21.15.44:0/261578684 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f7a9c00f830 con 0x7f7ab415be10 2024-09-13T08:43:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.699+0000 7f7aab400640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a900737c0 0x7f7a90075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.699+0000 7f7aab400640 1 -- 172.21.15.44:0/261578684 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4885+0+0 (secure 0 0 0) 0x7f7a9c092d70 con 0x7f7ab415be10 2024-09-13T08:43:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.699+0000 7f7ab3400640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a900737c0 0x7f7a90075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.699+0000 7f7ab3400640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a900737c0 0x7f7a90075c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4143a70 tx=0x7f7aa4007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:13.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.703+0000 7f7aab400640 1 -- 172.21.15.44:0/261578684 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7a9c014030 con 0x7f7ab415be10 2024-09-13T08:43:13.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:13.879+0000 7f7aba400640 1 -- 172.21.15.44:0/261578684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7ab40b3500 con 0x7f7ab415be10 2024-09-13T08:43:14.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.759+0000 7f7aab400640 1 -- 172.21.15.44:0/261578684 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) ==== 138+0+0 (secure 0 0 0) 0x7f7a9c05f3c0 con 0x7f7ab415be10 2024-09-13T08:43:14.762 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T08:43:14.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 -- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a900737c0 msgr2=0x7f7a90075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:14.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a900737c0 0x7f7a90075c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4143a70 tx=0x7f7aa4007ea0 comp rx=0 tx=0).stop 2024-09-13T08:43:14.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 -- 172.21.15.44:0/261578684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab415be10 msgr2=0x7f7ab40b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:14.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab415be10 0x7f7ab40b73c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00c9f0 tx=0x7f7a9c00ceb0 comp rx=0 tx=0).stop 2024-09-13T08:43:14.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 -- 172.21.15.44:0/261578684 shutdown_connections 2024-09-13T08:43:14.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a900737c0 0x7f7a90075c80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:14.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ab415c1f0 0x7f7ab41f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:14.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab415be10 0x7f7ab40b73c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:14.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 --2- 172.21.15.44:0/261578684 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7ab40b0b60 0x7f7ab41465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:14.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 -- 172.21.15.44:0/261578684 >> 172.21.15.44:0/261578684 conn(0x7f7ab40d1da0 msgr2=0x7f7ab40cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:14.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 -- 172.21.15.44:0/261578684 shutdown_connections 2024-09-13T08:43:14.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.763+0000 7f7aba400640 1 -- 172.21.15.44:0/261578684 wait complete. 2024-09-13T08:43:14.796 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T08:43:14.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 -- 172.21.15.44:0/1300949376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc0005220 msgr2=0x7f3bc000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:14.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 --2- 172.21.15.44:0/1300949376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc0005220 0x7f3bc000b9d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0002240 tx=0x7f3bd001d220 comp rx=0 tx=0).stop 2024-09-13T08:43:14.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 -- 172.21.15.44:0/1300949376 shutdown_connections 2024-09-13T08:43:14.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 --2- 172.21.15.44:0/1300949376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc0005220 0x7f3bc000b9d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:14.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 -- 172.21.15.44:0/1300949376 >> 172.21.15.44:0/1300949376 conn(0x7f3bd80d1cf0 msgr2=0x7f3bd80d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:14.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 -- 172.21.15.44:0/1300949376 shutdown_connections 2024-09-13T08:43:14.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 -- 172.21.15.44:0/1300949376 wait complete. 2024-09-13T08:43:14.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 Processor -- start 2024-09-13T08:43:14.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 -- start start 2024-09-13T08:43:14.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc0005220 0x7f3bd80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:14.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bd80cf0c0 0x7f3bd80c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:14.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd80c9b80 0x7f3bd81f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd815fc40 con 0x7f3bd80c9b80 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd80d0480 con 0x7f3bc0005220 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdde00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd815fdc0 con 0x7f3bd80cf0c0 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.951+0000 7f3bdd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd80c9b80 0x7f3bd81f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd80c9b80 0x7f3bd81f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46738/0 (socket says 172.21.15.44:46738) 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdd400640 1 -- 172.21.15.44:0/1313099346 learned_addr learned my addr 172.21.15.44:0/1313099346 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bd7e00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bd80cf0c0 0x7f3bd80c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdca00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc0005220 0x7f3bd80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdd400640 1 -- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bd80cf0c0 msgr2=0x7f3bd80c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdd400640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bd80cf0c0 0x7f3bd80c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdd400640 1 -- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc0005220 msgr2=0x7f3bd80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:14.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdd400640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc0005220 0x7f3bd80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:14.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdd400640 1 -- 172.21.15.44:0/1313099346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd80cdb20 con 0x7f3bd80c9b80 2024-09-13T08:43:14.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdca00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc0005220 0x7f3bd80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:14.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdd400640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd80c9b80 0x7f3bd81f4c40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3bd001dac0 tx=0x7f3bd0004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:14.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bd4c00640 1 -- 172.21.15.44:0/1313099346 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0001a80 con 0x7f3bd80c9b80 2024-09-13T08:43:14.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bd4c00640 1 -- 172.21.15.44:0/1313099346 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bd0003980 con 0x7f3bd80c9b80 2024-09-13T08:43:14.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdde00640 1 -- 172.21.15.44:0/1313099346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd80cddc0 con 0x7f3bd80c9b80 2024-09-13T08:43:14.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bd4c00640 1 -- 172.21.15.44:0/1313099346 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0003d90 con 0x7f3bd80c9b80 2024-09-13T08:43:14.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdde00640 1 -- 172.21.15.44:0/1313099346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd80ce380 con 0x7f3bd80c9b80 2024-09-13T08:43:14.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bdde00640 1 -- 172.21.15.44:0/1313099346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd8160320 con 0x7f3bd80c9b80 2024-09-13T08:43:14.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.955+0000 7f3bd4c00640 1 -- 172.21.15.44:0/1313099346 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f3bd0006070 con 0x7f3bd80c9b80 2024-09-13T08:43:14.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.959+0000 7f3bd4c00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3bb80737f0 0x7f3bb8075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:14.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.959+0000 7f3bd4c00640 1 -- 172.21.15.44:0/1313099346 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5255+0+0 (secure 0 0 0) 0x7f3bd012ce90 con 0x7f3bd80c9b80 2024-09-13T08:43:14.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.959+0000 7f3bdca00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3bb80737f0 0x7f3bb8075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:14.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.959+0000 7f3bdca00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3bb80737f0 0x7f3bb8075cb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc000c10 tx=0x7f3bcc006b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:14.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:14.959+0000 7f3bd4c00640 1 -- 172.21.15.44:0/1313099346 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3bd00c8080 con 0x7f3bd80c9b80 2024-09-13T08:43:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.139+0000 7f3bdde00640 1 -- 172.21.15.44:0/1313099346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3bd80b1b80 con 0x7f3bd80c9b80 2024-09-13T08:43:15.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.763+0000 7f3bd4c00640 1 -- 172.21.15.44:0/1313099346 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) ==== 170+0+0 (secure 0 0 0) 0x7f3bd0016fd0 con 0x7f3bd80c9b80 2024-09-13T08:43:15.765 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T08:43:15.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.763+0000 7f3bdde00640 1 -- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3bb80737f0 msgr2=0x7f3bb8075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:15.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.763+0000 7f3bdde00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3bb80737f0 0x7f3bb8075cb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc000c10 tx=0x7f3bcc006b60 comp rx=0 tx=0).stop 2024-09-13T08:43:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.763+0000 7f3bdde00640 1 -- 172.21.15.44:0/1313099346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd80c9b80 msgr2=0x7f3bd81f4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.763+0000 7f3bdde00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd80c9b80 0x7f3bd81f4c40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3bd001dac0 tx=0x7f3bd0004f00 comp rx=0 tx=0).stop 2024-09-13T08:43:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.767+0000 7f3bdde00640 1 -- 172.21.15.44:0/1313099346 shutdown_connections 2024-09-13T08:43:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.767+0000 7f3bdde00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3bb80737f0 0x7f3bb8075cb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.767+0000 7f3bdde00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd80c9b80 0x7f3bd81f4c40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.767+0000 7f3bdde00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3bd80cf0c0 0x7f3bd80c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.767+0000 7f3bdde00640 1 --2- 172.21.15.44:0/1313099346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc0005220 0x7f3bd80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.767+0000 7f3bdde00640 1 -- 172.21.15.44:0/1313099346 >> 172.21.15.44:0/1313099346 conn(0x7f3bd80d1cf0 msgr2=0x7f3bd8071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.767+0000 7f3bdde00640 1 -- 172.21.15.44:0/1313099346 shutdown_connections 2024-09-13T08:43:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.767+0000 7f3bdde00640 1 -- 172.21.15.44:0/1313099346 wait complete. 2024-09-13T08:43:15.808 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T08:43:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.955+0000 7f4e8f200640 1 -- 172.21.15.44:0/136361624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e900d22c0 msgr2=0x7f4e900cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.955+0000 7f4e8f200640 1 --2- 172.21.15.44:0/136361624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e900d22c0 0x7f4e900cc620 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009c80 tx=0x7f4e8801cda0 comp rx=0 tx=0).stop 2024-09-13T08:43:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.955+0000 7f4e8f200640 1 -- 172.21.15.44:0/136361624 shutdown_connections 2024-09-13T08:43:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.955+0000 7f4e8f200640 1 --2- 172.21.15.44:0/136361624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e900d22c0 0x7f4e900cc620 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.955+0000 7f4e8f200640 1 --2- 172.21.15.44:0/136361624 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e9015bd40 0x7f4e900d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.955+0000 7f4e8f200640 1 -- 172.21.15.44:0/136361624 >> 172.21.15.44:0/136361624 conn(0x7f4e9009aa10 msgr2=0x7f4e9009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:15.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.955+0000 7f4e8f200640 1 -- 172.21.15.44:0/136361624 shutdown_connections 2024-09-13T08:43:15.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.955+0000 7f4e8f200640 1 -- 172.21.15.44:0/136361624 wait complete. 2024-09-13T08:43:15.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.955+0000 7f4e8f200640 1 Processor -- start 2024-09-13T08:43:15.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.955+0000 7f4e8f200640 1 -- start start 2024-09-13T08:43:15.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e900b0d10 0x7f4e9014f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:15.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8f200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e900d22c0 0x7f4e900cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:15.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9015bd40 0x7f4e901f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:15.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90168d00 con 0x7f4e9015bd40 2024-09-13T08:43:15.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90168b80 con 0x7f4e900b0d10 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8f200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4e90168e80 con 0x7f4e900d22c0 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9015bd40 0x7f4e901f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9015bd40 0x7f4e901f71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46754/0 (socket says 172.21.15.44:46754) 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e900b0d10 0x7f4e9014f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8e800640 1 -- 172.21.15.44:0/2848309212 learned_addr learned my addr 172.21.15.44:0/2848309212 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e85400640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e900d22c0 0x7f4e900cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8e800640 1 -- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e900d22c0 msgr2=0x7f4e900cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8e800640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e900d22c0 0x7f4e900cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8e800640 1 -- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e900b0d10 msgr2=0x7f4e9014f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8e800640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e900b0d10 0x7f4e9014f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8e800640 1 -- 172.21.15.44:0/2848309212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e900cbd30 con 0x7f4e9015bd40 2024-09-13T08:43:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8e800640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9015bd40 0x7f4e901f71c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009bd0 tx=0x7f4e880092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e86a00640 1 -- 172.21.15.44:0/2848309212 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e880bb070 con 0x7f4e9015bd40 2024-09-13T08:43:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e86a00640 1 -- 172.21.15.44:0/2848309212 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e88004c50 con 0x7f4e9015bd40 2024-09-13T08:43:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e86a00640 1 -- 172.21.15.44:0/2848309212 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e880b2040 con 0x7f4e9015bd40 2024-09-13T08:43:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8f200640 1 -- 172.21.15.44:0/2848309212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e900c64c0 con 0x7f4e9015bd40 2024-09-13T08:43:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8f200640 1 -- 172.21.15.44:0/2848309212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e901501d0 con 0x7f4e9015bd40 2024-09-13T08:43:15.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.959+0000 7f4e8f200640 1 -- 172.21.15.44:0/2848309212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e900cc620 con 0x7f4e9015bd40 2024-09-13T08:43:15.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.963+0000 7f4e86a00640 1 -- 172.21.15.44:0/2848309212 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f4e880a5bb0 con 0x7f4e9015bd40 2024-09-13T08:43:15.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.963+0000 7f4e86a00640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e6c0737f0 0x7f4e6c075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:15.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.963+0000 7f4e86a00640 1 -- 172.21.15.44:0/2848309212 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5637+0+0 (secure 0 0 0) 0x7f4e8812c760 con 0x7f4e9015bd40 2024-09-13T08:43:15.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.963+0000 7f4e8de00640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e6c0737f0 0x7f4e6c075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:15.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.963+0000 7f4e8de00640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e6c0737f0 0x7f4e6c075cb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4e900b5130 tx=0x7f4e800091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:15.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:15.967+0000 7f4e86a00640 1 -- 172.21.15.44:0/2848309212 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4e880c4020 con 0x7f4e9015bd40 2024-09-13T08:43:16.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.183+0000 7f4e8f200640 1 -- 172.21.15.44:0/2848309212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4e9009fba0 con 0x7f4e9015bd40 2024-09-13T08:43:16.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.779+0000 7f4e86a00640 1 -- 172.21.15.44:0/2848309212 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v33) ==== 172+0+0 (secure 0 0 0) 0x7f4e88016990 con 0x7f4e9015bd40 2024-09-13T08:43:16.784 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 8 and data pool 9 2024-09-13T08:43:16.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 -- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e6c0737f0 msgr2=0x7f4e6c075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:16.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e6c0737f0 0x7f4e6c075cb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4e900b5130 tx=0x7f4e800091e0 comp rx=0 tx=0).stop 2024-09-13T08:43:16.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 -- 172.21.15.44:0/2848309212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9015bd40 msgr2=0x7f4e901f71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:16.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9015bd40 0x7f4e901f71c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009bd0 tx=0x7f4e880092e0 comp rx=0 tx=0).stop 2024-09-13T08:43:16.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 -- 172.21.15.44:0/2848309212 shutdown_connections 2024-09-13T08:43:16.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e6c0737f0 0x7f4e6c075cb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:16.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9015bd40 0x7f4e901f71c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:16.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e900d22c0 0x7f4e900cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:16.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 --2- 172.21.15.44:0/2848309212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e900b0d10 0x7f4e9014f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:16.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 -- 172.21.15.44:0/2848309212 >> 172.21.15.44:0/2848309212 conn(0x7f4e9009aa10 msgr2=0x7f4e900ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:16.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 -- 172.21.15.44:0/2848309212 shutdown_connections 2024-09-13T08:43:16.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.783+0000 7f4e8f200640 1 -- 172.21.15.44:0/2848309212 wait complete. 2024-09-13T08:43:16.840 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:43:16.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 -- 172.21.15.44:0/3500314637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fa815bd50 msgr2=0x7f0fa80ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:16.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 --2- 172.21.15.44:0/3500314637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fa815bd50 0x7f0fa80ce5e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009840 tx=0x7f0f9801c700 comp rx=0 tx=0).stop 2024-09-13T08:43:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 -- 172.21.15.44:0/3500314637 shutdown_connections 2024-09-13T08:43:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 --2- 172.21.15.44:0/3500314637 >> v2:172.21.15.129:3300/0 conn(0x7f0fa80cf4e0 0x7f0fa80cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 --2- 172.21.15.44:0/3500314637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fa815bd50 0x7f0fa80ce5e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 -- 172.21.15.44:0/3500314637 >> 172.21.15.44:0/3500314637 conn(0x7f0fa80d1ea0 msgr2=0x7f0fa80d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 -- 172.21.15.44:0/3500314637 shutdown_connections 2024-09-13T08:43:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 -- 172.21.15.44:0/3500314637 wait complete. 2024-09-13T08:43:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 Processor -- start 2024-09-13T08:43:16.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 -- start start 2024-09-13T08:43:16.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fa80cf4e0 0x7f0fa81595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:16.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa815bd50 0x7f0fa80cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:16.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fa815c130 0x7f0fa80ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:16.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8168e50 con 0x7f0fa815bd50 2024-09-13T08:43:16.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8168cd0 con 0x7f0fa80cf4e0 2024-09-13T08:43:16.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0face00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa8168fd0 con 0x7f0fa815c130 2024-09-13T08:43:16.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0f9fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa815bd50 0x7f0fa80cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:16.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0f9fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa815bd50 0x7f0fa80cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46756/0 (socket says 172.21.15.44:46756) 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.987+0000 7f0f9fe00640 1 -- 172.21.15.44:0/1064939855 learned_addr learned my addr 172.21.15.44:0/1064939855 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0fa6800640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fa815c130 0x7f0fa80ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0fa5e00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fa80cf4e0 0x7f0fa81595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0f9fe00640 1 -- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fa815c130 msgr2=0x7f0fa80ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0f9fe00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fa815c130 0x7f0fa80ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0f9fe00640 1 -- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fa80cf4e0 msgr2=0x7f0fa81595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0f9fe00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fa80cf4e0 0x7f0fa81595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0f9fe00640 1 -- 172.21.15.44:0/1064939855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa80caca0 con 0x7f0fa815bd50 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0fa5e00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fa80cf4e0 0x7f0fa81595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0f9fe00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa815bd50 0x7f0fa80cb460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000c460 tx=0x7f0f9000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0f9e000640 1 -- 172.21.15.44:0/1064939855 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90011790 con 0x7f0fa815bd50 2024-09-13T08:43:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0f9e000640 1 -- 172.21.15.44:0/1064939855 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f90017070 con 0x7f0fa815bd50 2024-09-13T08:43:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0f9e000640 1 -- 172.21.15.44:0/1064939855 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90011ce0 con 0x7f0fa815bd50 2024-09-13T08:43:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0face00640 1 -- 172.21.15.44:0/1064939855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa809c0d0 con 0x7f0fa815bd50 2024-09-13T08:43:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0face00640 1 -- 172.21.15.44:0/1064939855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa80cb9a0 con 0x7f0fa815bd50 2024-09-13T08:43:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0face00640 1 -- 172.21.15.44:0/1064939855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa80ce5e0 con 0x7f0fa815bd50 2024-09-13T08:43:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.991+0000 7f0f9e000640 1 -- 172.21.15.44:0/1064939855 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f0f9001c460 con 0x7f0fa815bd50 2024-09-13T08:43:16.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.995+0000 7f0f9e000640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f840737c0 0x7f0f84075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:16.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.995+0000 7f0f9e000640 1 -- 172.21.15.44:0/1064939855 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f0f90092420 con 0x7f0fa815bd50 2024-09-13T08:43:16.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.995+0000 7f0fa5e00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f840737c0 0x7f0f84075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:16.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.995+0000 7f0fa5e00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f840737c0 0x7f0f84075c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f980af000 tx=0x7f0f9801af40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:17.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:16.999+0000 7f0f9e000640 1 -- 172.21.15.44:0/1064939855 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0f9005f180 con 0x7f0fa815bd50 2024-09-13T08:43:17.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.171+0000 7f0face00640 1 -- 172.21.15.44:0/1064939855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fa80b3040 con 0x7f0fa815bd50 2024-09-13T08:43:17.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0f9e000640 1 -- 172.21.15.44:0/1064939855 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16793 (secure 0 0 0) 0x7f0f9005ea80 con 0x7f0fa815bd50 2024-09-13T08:43:17.178 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:17.179 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":60,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:43:16.187540+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:43:13.893284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-13T08:43:15.154456+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:43:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 -- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f840737c0 msgr2=0x7f0f84075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f840737c0 0x7f0f84075c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f980af000 tx=0x7f0f9801af40 comp rx=0 tx=0).stop 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 -- 172.21.15.44:0/1064939855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa815bd50 msgr2=0x7f0fa80cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa815bd50 0x7f0fa80cb460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000c460 tx=0x7f0f9000c860 comp rx=0 tx=0).stop 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 -- 172.21.15.44:0/1064939855 shutdown_connections 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f840737c0 0x7f0f84075c80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fa815c130 0x7f0fa80ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa815bd50 0x7f0fa80cb460 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 --2- 172.21.15.44:0/1064939855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fa80cf4e0 0x7f0fa81595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 -- 172.21.15.44:0/1064939855 >> 172.21.15.44:0/1064939855 conn(0x7f0fa80d1ea0 msgr2=0x7f0fa80d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 -- 172.21.15.44:0/1064939855 shutdown_connections 2024-09-13T08:43:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.175+0000 7f0face00640 1 -- 172.21.15.44:0/1064939855 wait complete. 2024-09-13T08:43:17.201 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:43:17.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.343+0000 7f651c600640 1 -- 172.21.15.44:0/2679041526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64fc004d50 msgr2=0x7f64fc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:17.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.343+0000 7f651c600640 1 --2- 172.21.15.44:0/2679041526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64fc004d50 0x7f64fc0051d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f650c009eb0 tx=0x7f650c01d5f0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.343+0000 7f651a600640 1 -- 172.21.15.44:0/2679041526 reap_dead start 2024-09-13T08:43:17.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.343+0000 7f651c600640 1 -- 172.21.15.44:0/2679041526 shutdown_connections 2024-09-13T08:43:17.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.343+0000 7f651c600640 1 -- 172.21.15.44:0/2679041526 >> 172.21.15.44:0/2679041526 conn(0x7f65140d1dc0 msgr2=0x7f65140d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:17.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651c600640 1 -- 172.21.15.44:0/2679041526 shutdown_connections 2024-09-13T08:43:17.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651c600640 1 -- 172.21.15.44:0/2679041526 wait complete. 2024-09-13T08:43:17.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651c600640 1 Processor -- start 2024-09-13T08:43:17.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651c600640 1 -- start start 2024-09-13T08:43:17.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651c600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64fc004d50 0x7f651409f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:17.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651c600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f651415b290 0x7f6514157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:17.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651c600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651409d680 0x7f651409db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651c600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f651415c400 con 0x7f651409d680 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651c600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f651415c280 con 0x7f651415b290 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651c600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f65140ce670 con 0x7f64fc004d50 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651409d680 0x7f651409db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651409d680 0x7f651409db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46772/0 (socket says 172.21.15.44:46772) 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651a600640 1 -- 172.21.15.44:0/281478327 learned_addr learned my addr 172.21.15.44:0/281478327 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f6519c00640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64fc004d50 0x7f651409f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f6519200640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f651415b290 0x7f6514157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651a600640 1 -- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64fc004d50 msgr2=0x7f651409f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651a600640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64fc004d50 0x7f651409f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651a600640 1 -- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f651415b290 msgr2=0x7f6514157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:17.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651a600640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f651415b290 0x7f6514157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651a600640 1 -- 172.21.15.44:0/281478327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65140b4420 con 0x7f651409d680 2024-09-13T08:43:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f6519200640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f651415b290 0x7f6514157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f651a600640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651409d680 0x7f651409db00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f650c01d060 tx=0x7f650c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:17.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.347+0000 7f6511600640 1 -- 172.21.15.44:0/281478327 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c0ad050 con 0x7f651409d680 2024-09-13T08:43:17.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.351+0000 7f6511600640 1 -- 172.21.15.44:0/281478327 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f650c0a7a90 con 0x7f651409d680 2024-09-13T08:43:17.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.351+0000 7f6511600640 1 -- 172.21.15.44:0/281478327 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c0a7e40 con 0x7f651409d680 2024-09-13T08:43:17.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.351+0000 7f651c600640 1 -- 172.21.15.44:0/281478327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651409c6e0 con 0x7f651409d680 2024-09-13T08:43:17.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.351+0000 7f651c600640 1 -- 172.21.15.44:0/281478327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65140b3f00 con 0x7f651409d680 2024-09-13T08:43:17.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.351+0000 7f651c600640 1 -- 172.21.15.44:0/281478327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65140cebb0 con 0x7f651409d680 2024-09-13T08:43:17.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.351+0000 7f6511600640 1 -- 172.21.15.44:0/281478327 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f650c0b0070 con 0x7f651409d680 2024-09-13T08:43:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.351+0000 7f6511600640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f64f40737c0 0x7f64f4075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.355+0000 7f6511600640 1 -- 172.21.15.44:0/281478327 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f650c12d350 con 0x7f651409d680 2024-09-13T08:43:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.355+0000 7f6519c00640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f64f40737c0 0x7f64f4075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.355+0000 7f6519c00640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f64f40737c0 0x7f64f4075c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6508005fd0 tx=0x7f650800b260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:17.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.359+0000 7f6511600640 1 -- 172.21.15.44:0/281478327 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f650c017300 con 0x7f651409d680 2024-09-13T08:43:17.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 -- 172.21.15.44:0/281478327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f651406e340 con 0x7f651409d680 2024-09-13T08:43:17.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f6511600640 1 -- 172.21.15.44:0/281478327 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16793 (secure 0 0 0) 0x7f650c0f99a0 con 0x7f651409d680 2024-09-13T08:43:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:17.543 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":60,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:43:16.187540+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:43:13.893284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-13T08:43:15.154456+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:43:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 -- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f64f40737c0 msgr2=0x7f64f4075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f64f40737c0 0x7f64f4075c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6508005fd0 tx=0x7f650800b260 comp rx=0 tx=0).stop 2024-09-13T08:43:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 -- 172.21.15.44:0/281478327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651409d680 msgr2=0x7f651409db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651409d680 0x7f651409db00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f650c01d060 tx=0x7f650c009780 comp rx=0 tx=0).stop 2024-09-13T08:43:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 -- 172.21.15.44:0/281478327 shutdown_connections 2024-09-13T08:43:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f64f40737c0 0x7f64f4075c80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651409d680 0x7f651409db00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f651415b290 0x7f6514157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 --2- 172.21.15.44:0/281478327 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64fc004d50 0x7f651409f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 -- 172.21.15.44:0/281478327 >> 172.21.15.44:0/281478327 conn(0x7f65140d1dc0 msgr2=0x7f65140b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 -- 172.21.15.44:0/281478327 shutdown_connections 2024-09-13T08:43:17.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.539+0000 7f651c600640 1 -- 172.21.15.44:0/281478327 wait complete. 2024-09-13T08:43:17.565 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T08:43:17.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 -- 172.21.15.44:0/3384837863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75fc1605d0 msgr2=0x7f75fc0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:17.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 --2- 172.21.15.44:0/3384837863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75fc1605d0 0x7f75fc0d0690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009840 tx=0x7f75f001c820 comp rx=0 tx=0).stop 2024-09-13T08:43:17.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 -- 172.21.15.44:0/3384837863 shutdown_connections 2024-09-13T08:43:17.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 --2- 172.21.15.44:0/3384837863 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75fc0afb10 0x7f75fc0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 --2- 172.21.15.44:0/3384837863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75fc1605d0 0x7f75fc0d0690 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 -- 172.21.15.44:0/3384837863 >> 172.21.15.44:0/3384837863 conn(0x7f75fc0d1dd0 msgr2=0x7f75fc0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:17.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 -- 172.21.15.44:0/3384837863 shutdown_connections 2024-09-13T08:43:17.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 -- 172.21.15.44:0/3384837863 wait complete. 2024-09-13T08:43:17.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 Processor -- start 2024-09-13T08:43:17.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 -- start start 2024-09-13T08:43:17.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75fc0afb10 0x7f75fc15f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:17.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75fc0b1650 0x7f75fc0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:17.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75fc1605d0 0x7f75fc0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0b13a0 con 0x7f75fc0b1650 2024-09-13T08:43:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0b1220 con 0x7f75fc0afb10 2024-09-13T08:43:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7604200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc169540 con 0x7f75fc1605d0 2024-09-13T08:43:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7600e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75fc0b1650 0x7f75fc0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7601800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75fc0afb10 0x7f75fc15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7600e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75fc0b1650 0x7f75fc0b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46778/0 (socket says 172.21.15.44:46778) 2024-09-13T08:43:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7600e00640 1 -- 172.21.15.44:0/1682009640 learned_addr learned my addr 172.21.15.44:0/1682009640 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7602200640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75fc1605d0 0x7f75fc0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:17.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7600e00640 1 -- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75fc1605d0 msgr2=0x7f75fc0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:17.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7600e00640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75fc1605d0 0x7f75fc0b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7600e00640 1 -- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75fc0afb10 msgr2=0x7f75fc15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7600e00640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75fc0afb10 0x7f75fc15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.715+0000 7f7600e00640 1 -- 172.21.15.44:0/1682009640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc15cdd0 con 0x7f75fc0b1650 2024-09-13T08:43:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.719+0000 7f7600e00640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75fc0b1650 0x7f75fc0b3850 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75e800c500 tx=0x7f75e800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.719+0000 7f75f9600640 1 -- 172.21.15.44:0/1682009640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8017070 con 0x7f75fc0b1650 2024-09-13T08:43:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.719+0000 7f75f9600640 1 -- 172.21.15.44:0/1682009640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75e8003600 con 0x7f75fc0b1650 2024-09-13T08:43:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.719+0000 7f7604200640 1 -- 172.21.15.44:0/1682009640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc15e840 con 0x7f75fc0b1650 2024-09-13T08:43:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.719+0000 7f75f9600640 1 -- 172.21.15.44:0/1682009640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8003a90 con 0x7f75fc0b1650 2024-09-13T08:43:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.719+0000 7f7604200640 1 -- 172.21.15.44:0/1682009640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc15ee00 con 0x7f75fc0b1650 2024-09-13T08:43:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.719+0000 7f7604200640 1 -- 172.21.15.44:0/1682009640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75fc0affc0 con 0x7f75fc0b1650 2024-09-13T08:43:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.719+0000 7f75f9600640 1 -- 172.21.15.44:0/1682009640 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f75e801a070 con 0x7f75fc0b1650 2024-09-13T08:43:17.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.719+0000 7f75f9600640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75dc0737c0 0x7f75dc075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:17.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.719+0000 7f75f9600640 1 -- 172.21.15.44:0/1682009640 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f75e8091b10 con 0x7f75fc0b1650 2024-09-13T08:43:17.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.723+0000 7f7601800640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75dc0737c0 0x7f75dc075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:17.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.723+0000 7f7601800640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75dc0737c0 0x7f75dc075c80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f75f00af000 tx=0x7f75f001b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:17.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.723+0000 7f75f9600640 1 -- 172.21.15.44:0/1682009640 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f75e805e220 con 0x7f75fc0b1650 2024-09-13T08:43:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:17.939+0000 7f7604200640 1 -- 172.21.15.44:0/1682009640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f75fc0d1080 con 0x7f75fc0b1650 2024-09-13T08:43:18.758 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-09-13T08:43:18.755+0000 7f4465e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:43:18.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.815+0000 7f75f9600640 1 -- 172.21.15.44:0/1682009640 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) ==== 118+0+0 (secure 0 0 0) 0x7f75e8062c50 con 0x7f75fc0b1650 2024-09-13T08:43:18.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 -- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75dc0737c0 msgr2=0x7f75dc075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:18.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75dc0737c0 0x7f75dc075c80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f75f00af000 tx=0x7f75f001b060 comp rx=0 tx=0).stop 2024-09-13T08:43:18.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 -- 172.21.15.44:0/1682009640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75fc0b1650 msgr2=0x7f75fc0b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:18.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75fc0b1650 0x7f75fc0b3850 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75e800c500 tx=0x7f75e800c9c0 comp rx=0 tx=0).stop 2024-09-13T08:43:18.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 -- 172.21.15.44:0/1682009640 shutdown_connections 2024-09-13T08:43:18.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75dc0737c0 0x7f75dc075c80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:18.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75fc1605d0 0x7f75fc0b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:18.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75fc0b1650 0x7f75fc0b3850 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:18.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 --2- 172.21.15.44:0/1682009640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75fc0afb10 0x7f75fc15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:18.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 -- 172.21.15.44:0/1682009640 >> 172.21.15.44:0/1682009640 conn(0x7f75fc0d1dd0 msgr2=0x7f75fc0b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:18.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 -- 172.21.15.44:0/1682009640 shutdown_connections 2024-09-13T08:43:18.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:18.819+0000 7f7604200640 1 -- 172.21.15.44:0/1682009640 wait complete. 2024-09-13T08:43:18.848 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T08:43:18.849 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T08:43:19.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 -- 172.21.15.44:0/3509985486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e600b1650 msgr2=0x7f4e600b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:19.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 --2- 172.21.15.44:0/3509985486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e600b1650 0x7f4e600b1a30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009840 tx=0x7f4e4801c820 comp rx=0 tx=0).stop 2024-09-13T08:43:19.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 -- 172.21.15.44:0/3509985486 shutdown_connections 2024-09-13T08:43:19.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 --2- 172.21.15.44:0/3509985486 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e600afb10 0x7f4e600affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:19.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 --2- 172.21.15.44:0/3509985486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e600b1650 0x7f4e600b1a30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:19.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 -- 172.21.15.44:0/3509985486 >> 172.21.15.44:0/3509985486 conn(0x7f4e600d1dd0 msgr2=0x7f4e600d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:19.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 -- 172.21.15.44:0/3509985486 shutdown_connections 2024-09-13T08:43:19.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 -- 172.21.15.44:0/3509985486 wait complete. 2024-09-13T08:43:19.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 Processor -- start 2024-09-13T08:43:19.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 -- start start 2024-09-13T08:43:19.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e600afb10 0x7f4e6006f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:19.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e600b1650 0x7f4e6006e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:19.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e601605d0 0x7f4e600cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:19.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e600b0e60 con 0x7f4e600afb10 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e600b1430 con 0x7f4e601605d0 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e65a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4e60169540 con 0x7f4e600b1650 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e600afb10 0x7f4e6006f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e600afb10 0x7f4e6006f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46794/0 (socket says 172.21.15.44:46794) 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e5ea00640 1 -- 172.21.15.44:0/1010274622 learned_addr learned my addr 172.21.15.44:0/1010274622 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e57e00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e600b1650 0x7f4e6006e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e5f400640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e601605d0 0x7f4e600cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e5ea00640 1 -- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e600b1650 msgr2=0x7f4e6006e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e5ea00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e600b1650 0x7f4e6006e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e5ea00640 1 -- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e601605d0 msgr2=0x7f4e600cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e5ea00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e601605d0 0x7f4e600cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:19.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e5ea00640 1 -- 172.21.15.44:0/1010274622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e60159990 con 0x7f4e600afb10 2024-09-13T08:43:19.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.003+0000 7f4e5ea00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e600afb10 0x7f4e6006f3b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4e50001ec0 tx=0x7f4e5000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:19.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.007+0000 7f4e56a00640 1 -- 172.21.15.44:0/1010274622 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50017070 con 0x7f4e600afb10 2024-09-13T08:43:19.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.007+0000 7f4e56a00640 1 -- 172.21.15.44:0/1010274622 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e50003600 con 0x7f4e600afb10 2024-09-13T08:43:19.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.007+0000 7f4e56a00640 1 -- 172.21.15.44:0/1010274622 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50003a50 con 0x7f4e600afb10 2024-09-13T08:43:19.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.007+0000 7f4e65a00640 1 -- 172.21.15.44:0/1010274622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e60158bd0 con 0x7f4e600afb10 2024-09-13T08:43:19.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.007+0000 7f4e65a00640 1 -- 172.21.15.44:0/1010274622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e60159190 con 0x7f4e600afb10 2024-09-13T08:43:19.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.007+0000 7f4e65a00640 1 -- 172.21.15.44:0/1010274622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e600affc0 con 0x7f4e600afb10 2024-09-13T08:43:19.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.007+0000 7f4e56a00640 1 -- 172.21.15.44:0/1010274622 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f4e5001a070 con 0x7f4e600afb10 2024-09-13T08:43:19.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.007+0000 7f4e56a00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e3c0737c0 0x7f4e3c075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:19.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.007+0000 7f4e56a00640 1 -- 172.21.15.44:0/1010274622 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f4e50091930 con 0x7f4e600afb10 2024-09-13T08:43:19.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.011+0000 7f4e57e00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e3c0737c0 0x7f4e3c075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:19.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.011+0000 7f4e57e00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e3c0737c0 0x7f4e3c075c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4e480af000 tx=0x7f4e4801b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:19.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.015+0000 7f4e56a00640 1 -- 172.21.15.44:0/1010274622 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4e5005df80 con 0x7f4e600afb10 2024-09-13T08:43:19.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.243+0000 7f4e65a00640 1 -- 172.21.15.44:0/1010274622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4e600d2350 con 0x7f4e600afb10 2024-09-13T08:43:19.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.819+0000 7f4e56a00640 1 -- 172.21.15.44:0/1010274622 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7f4e500629b0 con 0x7f4e600afb10 2024-09-13T08:43:19.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 -- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e3c0737c0 msgr2=0x7f4e3c075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:19.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e3c0737c0 0x7f4e3c075c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4e480af000 tx=0x7f4e4801b060 comp rx=0 tx=0).stop 2024-09-13T08:43:19.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 -- 172.21.15.44:0/1010274622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e600afb10 msgr2=0x7f4e6006f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:19.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e600afb10 0x7f4e6006f3b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4e50001ec0 tx=0x7f4e5000c890 comp rx=0 tx=0).stop 2024-09-13T08:43:19.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 -- 172.21.15.44:0/1010274622 shutdown_connections 2024-09-13T08:43:19.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e3c0737c0 0x7f4e3c075c80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:19.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e601605d0 0x7f4e600cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:19.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e600b1650 0x7f4e6006e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:19.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 --2- 172.21.15.44:0/1010274622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e600afb10 0x7f4e6006f3b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:19.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 -- 172.21.15.44:0/1010274622 >> 172.21.15.44:0/1010274622 conn(0x7f4e600d1dd0 msgr2=0x7f4e601662c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:19.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 -- 172.21.15.44:0/1010274622 shutdown_connections 2024-09-13T08:43:19.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.823+0000 7f4e65a00640 1 -- 172.21.15.44:0/1010274622 wait complete. 2024-09-13T08:43:19.849 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T08:43:19.849 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:43:20.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.995+0000 7fc60a600640 1 -- 172.21.15.44:0/2739946668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5e80039f0 msgr2=0x7fc5e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.995+0000 7fc60a600640 1 --2- 172.21.15.44:0/2739946668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5e80039f0 0x7fc5e8003dd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009b30 tx=0x7fc5f401d210 comp rx=0 tx=0).stop 2024-09-13T08:43:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.995+0000 7fc603e00640 1 -- 172.21.15.44:0/2739946668 reap_dead start 2024-09-13T08:43:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.995+0000 7fc60a600640 1 -- 172.21.15.44:0/2739946668 shutdown_connections 2024-09-13T08:43:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 -- 172.21.15.44:0/2739946668 >> 172.21.15.44:0/2739946668 conn(0x7fc6040d1dc0 msgr2=0x7fc6040d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 -- 172.21.15.44:0/2739946668 shutdown_connections 2024-09-13T08:43:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 -- 172.21.15.44:0/2739946668 wait complete. 2024-09-13T08:43:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 Processor -- start 2024-09-13T08:43:20.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 -- start start 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e80039f0 0x7fc6041f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6040c8700 0x7fc6040caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6040cb3e0 0x7fc60414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc60415c400 con 0x7fc6040c8700 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc60415c280 con 0x7fc5e80039f0 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc6040ce670 con 0x7fc6040cb3e0 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6040c8700 0x7fc6040caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6040c8700 0x7fc6040caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46812/0 (socket says 172.21.15.44:46812) 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fbe00640 1 -- 172.21.15.44:0/1250072745 learned_addr learned my addr 172.21.15.44:0/1250072745 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc603400640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e80039f0 0x7fc6041f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc603e00640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6040cb3e0 0x7fc60414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fbe00640 1 -- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6040cb3e0 msgr2=0x7fc60414efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fbe00640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6040cb3e0 0x7fc60414efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fbe00640 1 -- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e80039f0 msgr2=0x7fc6041f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fbe00640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e80039f0 0x7fc6041f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fbe00640 1 -- 172.21.15.44:0/1250072745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60414cec0 con 0x7fc6040c8700 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fbe00640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6040c8700 0x7fc6040caea0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0115a0 tx=0x7fc5ec011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fb400640 1 -- 172.21.15.44:0/1250072745 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec013720 con 0x7fc6040c8700 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fb400640 1 -- 172.21.15.44:0/1250072745 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5ec0138c0 con 0x7fc6040c8700 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc60a600640 1 -- 172.21.15.44:0/1250072745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60414d1a0 con 0x7fc6040c8700 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:19.999+0000 7fc5fb400640 1 -- 172.21.15.44:0/1250072745 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec013d10 con 0x7fc6040c8700 2024-09-13T08:43:20.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.003+0000 7fc60a600640 1 -- 172.21.15.44:0/1250072745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60414d760 con 0x7fc6040c8700 2024-09-13T08:43:20.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.003+0000 7fc60a600640 1 -- 172.21.15.44:0/1250072745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60414c990 con 0x7fc6040c8700 2024-09-13T08:43:20.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.003+0000 7fc5fb400640 1 -- 172.21.15.44:0/1250072745 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fc5ec024440 con 0x7fc6040c8700 2024-09-13T08:43:20.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.003+0000 7fc5fb400640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5e00737c0 0x7fc5e0075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.003+0000 7fc5fb400640 1 -- 172.21.15.44:0/1250072745 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fc5ec095e50 con 0x7fc6040c8700 2024-09-13T08:43:20.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.003+0000 7fc603400640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5e00737c0 0x7fc5e0075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.007+0000 7fc603400640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5e00737c0 0x7fc5e0075c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009b30 tx=0x7fc5f4019fb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:20.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.007+0000 7fc5fb400640 1 -- 172.21.15.44:0/1250072745 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc60414c990 con 0x7fc6040c8700 2024-09-13T08:43:20.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.219+0000 7fc60a600640 1 -- 172.21.15.44:0/1250072745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6040cec00 con 0x7fc6040c8700 2024-09-13T08:43:20.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.219+0000 7fc5fb400640 1 -- 172.21.15.44:0/1250072745 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5223 (secure 0 0 0) 0x7fc5ec017070 con 0x7fc6040c8700 2024-09-13T08:43:20.223 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:20.224 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":37,"btime":"2024-09-13T08:43:19:822213+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1613950025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1613950025},{"type":"v1","addr":"172.21.15.129:6836","nonce":1613950025}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5402,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/10747232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":10747232},{"type":"v1","addr":"172.21.15.129:6837","nonce":10747232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3945609265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3945609265},{"type":"v1","addr":"172.21.15.44:6834","nonce":3945609265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:43:16.187506+0000","modified":"2024-09-13T08:43:19.249207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2755626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2755626228},{"type":"v1","addr":"172.21.15.44:6835","nonce":2755626228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5877,"qdb_cluster":[5877]},"id":4}]} 2024-09-13T08:43:20.224 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 37 2024-09-13T08:43:20.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 -- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5e00737c0 msgr2=0x7fc5e0075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5e00737c0 0x7fc5e0075c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009b30 tx=0x7fc5f4019fb0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 -- 172.21.15.44:0/1250072745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6040c8700 msgr2=0x7fc6040caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6040c8700 0x7fc6040caea0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0115a0 tx=0x7fc5ec011a60 comp rx=0 tx=0).stop 2024-09-13T08:43:20.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 -- 172.21.15.44:0/1250072745 shutdown_connections 2024-09-13T08:43:20.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5e00737c0 0x7fc5e0075c80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6040cb3e0 0x7fc60414efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6040c8700 0x7fc6040caea0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 --2- 172.21.15.44:0/1250072745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e80039f0 0x7fc6041f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 -- 172.21.15.44:0/1250072745 >> 172.21.15.44:0/1250072745 conn(0x7fc6040d1dc0 msgr2=0x7fc6040cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:20.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 -- 172.21.15.44:0/1250072745 shutdown_connections 2024-09-13T08:43:20.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.223+0000 7fc60a600640 1 -- 172.21.15.44:0/1250072745 wait complete. 2024-09-13T08:43:20.246 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:43:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 -- 172.21.15.44:0/2598500964 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a400cd540 msgr2=0x7f1a4006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 --2- 172.21.15.44:0/2598500964 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a400cd540 0x7f1a4006e7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009bc0 tx=0x7f1a3c01f3b0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 -- 172.21.15.44:0/2598500964 shutdown_connections 2024-09-13T08:43:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 --2- 172.21.15.44:0/2598500964 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a400cd540 0x7f1a4006e7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 --2- 172.21.15.44:0/2598500964 >> v2:172.21.15.129:3300/0 conn(0x7f1a4015bfa0 0x7f1a400cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 -- 172.21.15.44:0/2598500964 >> 172.21.15.44:0/2598500964 conn(0x7f1a400b18f0 msgr2=0x7f1a400b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 -- 172.21.15.44:0/2598500964 shutdown_connections 2024-09-13T08:43:20.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 -- 172.21.15.44:0/2598500964 wait complete. 2024-09-13T08:43:20.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 Processor -- start 2024-09-13T08:43:20.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 -- start start 2024-09-13T08:43:20.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a400cd540 0x7f1a400ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a4015bfa0 0x7f1a40156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4015c3e0 0x7f1a4014f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4006f070 con 0x7f1a400cd540 2024-09-13T08:43:20.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4006eef0 con 0x7f1a4015c3e0 2024-09-13T08:43:20.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a48800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4006f1f0 con 0x7f1a4015bfa0 2024-09-13T08:43:20.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a45e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a400cd540 0x7f1a400ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a45e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a400cd540 0x7f1a400ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46824/0 (socket says 172.21.15.44:46824) 2024-09-13T08:43:20.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a45e00640 1 -- 172.21.15.44:0/2276462743 learned_addr learned my addr 172.21.15.44:0/2276462743 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:20.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a46800640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4015c3e0 0x7f1a4014f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a45400640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a4015bfa0 0x7f1a40156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a45e00640 1 -- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a4015bfa0 msgr2=0x7f1a40156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a45e00640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a4015bfa0 0x7f1a40156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a45e00640 1 -- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4015c3e0 msgr2=0x7f1a4014f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a45e00640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4015c3e0 0x7f1a4014f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a45e00640 1 -- 172.21.15.44:0/2276462743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a40156b80 con 0x7f1a400cd540 2024-09-13T08:43:20.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a46800640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4015c3e0 0x7f1a4014f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:20.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.391+0000 7f1a45e00640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a400cd540 0x7f1a400ad970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c460 tx=0x7f1a3400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:20.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.395+0000 7f1a3a000640 1 -- 172.21.15.44:0/2276462743 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34016020 con 0x7f1a400cd540 2024-09-13T08:43:20.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.395+0000 7f1a3a000640 1 -- 172.21.15.44:0/2276462743 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a34012940 con 0x7f1a400cd540 2024-09-13T08:43:20.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.395+0000 7f1a48800640 1 -- 172.21.15.44:0/2276462743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a400b2ff0 con 0x7f1a400cd540 2024-09-13T08:43:20.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.395+0000 7f1a3a000640 1 -- 172.21.15.44:0/2276462743 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34012d90 con 0x7f1a400cd540 2024-09-13T08:43:20.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.395+0000 7f1a48800640 1 -- 172.21.15.44:0/2276462743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a401f8960 con 0x7f1a400cd540 2024-09-13T08:43:20.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.395+0000 7f1a48800640 1 -- 172.21.15.44:0/2276462743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a40069410 con 0x7f1a400cd540 2024-09-13T08:43:20.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.395+0000 7f1a3a000640 1 -- 172.21.15.44:0/2276462743 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f1a34019020 con 0x7f1a400cd540 2024-09-13T08:43:20.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.395+0000 7f1a3a000640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a240737c0 0x7f1a24075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.395+0000 7f1a3a000640 1 -- 172.21.15.44:0/2276462743 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f1a340925a0 con 0x7f1a400cd540 2024-09-13T08:43:20.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.395+0000 7f1a45400640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a240737c0 0x7f1a24075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.399+0000 7f1a45400640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a240737c0 0x7f1a24075c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1a400af060 tx=0x7f1a300091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:20.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.399+0000 7f1a3a000640 1 -- 172.21.15.44:0/2276462743 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1a34020020 con 0x7f1a400cd540 2024-09-13T08:43:20.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.575+0000 7f1a48800640 1 -- 172.21.15.44:0/2276462743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a400cbf00 con 0x7f1a400cd540 2024-09-13T08:43:20.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.575+0000 7f1a3a000640 1 -- 172.21.15.44:0/2276462743 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16780 (secure 0 0 0) 0x7f1a3405f300 con 0x7f1a400cd540 2024-09-13T08:43:20.581 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:20.581 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":61,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:43:17.784977+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:43:13.893284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-13T08:43:15.154456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 -- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a240737c0 msgr2=0x7f1a24075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a240737c0 0x7f1a24075c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1a400af060 tx=0x7f1a300091e0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 -- 172.21.15.44:0/2276462743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a400cd540 msgr2=0x7f1a400ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a400cd540 0x7f1a400ad970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c460 tx=0x7f1a3400c920 comp rx=0 tx=0).stop 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 -- 172.21.15.44:0/2276462743 shutdown_connections 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a240737c0 0x7f1a24075c80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4015c3e0 0x7f1a4014f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a4015bfa0 0x7f1a40156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 --2- 172.21.15.44:0/2276462743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a400cd540 0x7f1a400ad970 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 -- 172.21.15.44:0/2276462743 >> 172.21.15.44:0/2276462743 conn(0x7f1a400b18f0 msgr2=0x7f1a400cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:20.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 -- 172.21.15.44:0/2276462743 shutdown_connections 2024-09-13T08:43:20.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.579+0000 7f1a48800640 1 -- 172.21.15.44:0/2276462743 wait complete. 2024-09-13T08:43:20.603 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:43:20.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:43:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 -- 172.21.15.44:0/2378030549 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25f00cf300 msgr2=0x7f25f006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 --2- 172.21.15.44:0/2378030549 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25f00cf300 0x7f25f006ecb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009900 tx=0x7f25dc01c8b0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 -- 172.21.15.44:0/2378030549 shutdown_connections 2024-09-13T08:43:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 --2- 172.21.15.44:0/2378030549 >> v2:172.21.15.129:3300/0 conn(0x7f25f006e810 0x7f25f006f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 --2- 172.21.15.44:0/2378030549 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25f00cf300 0x7f25f006ecb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 -- 172.21.15.44:0/2378030549 >> 172.21.15.44:0/2378030549 conn(0x7f25f00b18f0 msgr2=0x7f25f00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 -- 172.21.15.44:0/2378030549 shutdown_connections 2024-09-13T08:43:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 -- 172.21.15.44:0/2378030549 wait complete. 2024-09-13T08:43:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 Processor -- start 2024-09-13T08:43:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 -- start start 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25f006e810 0x7f25f014d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25f00cf300 0x7f25f0154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f015ade0 0x7f25f01551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25f00cde70 con 0x7f25f015ade0 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25f00cdcf0 con 0x7f25f00cf300 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f8600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f25f00cdff0 con 0x7f25f006e810 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f6600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f015ade0 0x7f25f01551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f6600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f015ade0 0x7f25f01551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46826/0 (socket says 172.21.15.44:46826) 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f6600640 1 -- 172.21.15.44:0/1263369445 learned_addr learned my addr 172.21.15.44:0/1263369445 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f5200640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25f00cf300 0x7f25f0154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f6600640 1 -- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25f006e810 msgr2=0x7f25f014d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f6600640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25f006e810 0x7f25f014d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f6600640 1 -- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25f00cf300 msgr2=0x7f25f0154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f6600640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25f00cf300 0x7f25f0154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.747+0000 7f25f6600640 1 -- 172.21.15.44:0/1263369445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f0073a60 con 0x7f25f015ade0 2024-09-13T08:43:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.751+0000 7f25f6600640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f015ade0 0x7f25f01551d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f25e800ead0 tx=0x7f25e800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.751+0000 7f25ed600640 1 -- 172.21.15.44:0/1263369445 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8004b30 con 0x7f25f015ade0 2024-09-13T08:43:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.751+0000 7f25ed600640 1 -- 172.21.15.44:0/1263369445 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25e8004dc0 con 0x7f25f015ade0 2024-09-13T08:43:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.751+0000 7f25f8600640 1 -- 172.21.15.44:0/1263369445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f0073d40 con 0x7f25f015ade0 2024-09-13T08:43:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.751+0000 7f25ed600640 1 -- 172.21.15.44:0/1263369445 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8005280 con 0x7f25f015ade0 2024-09-13T08:43:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.751+0000 7f25f8600640 1 -- 172.21.15.44:0/1263369445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f014dab0 con 0x7f25f015ade0 2024-09-13T08:43:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.751+0000 7f25f8600640 1 -- 172.21.15.44:0/1263369445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f0069410 con 0x7f25f015ade0 2024-09-13T08:43:20.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.751+0000 7f25ed600640 1 -- 172.21.15.44:0/1263369445 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f25e8005420 con 0x7f25f015ade0 2024-09-13T08:43:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.751+0000 7f25ed600640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25d00737c0 0x7f25d0075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.751+0000 7f25ed600640 1 -- 172.21.15.44:0/1263369445 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f25e8093550 con 0x7f25f015ade0 2024-09-13T08:43:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.755+0000 7f25f5c00640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25d00737c0 0x7f25d0075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.755+0000 7f25f5c00640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25d00737c0 0x7f25d0075c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f25e4007fd0 tx=0x7f25e4007940 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:20.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.755+0000 7f25ed600640 1 -- 172.21.15.44:0/1263369445 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f25e80601f0 con 0x7f25f015ade0 2024-09-13T08:43:20.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.931+0000 7f25f8600640 1 -- 172.21.15.44:0/1263369445 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25f0152ca0 con 0x7f25d00737c0 2024-09-13T08:43:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.939+0000 7f25ed600640 1 -- 172.21.15.44:0/1263369445 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f25f0152ca0 con 0x7f25d00737c0 2024-09-13T08:43:20.946 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:20.947 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:43:20.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 -- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25d00737c0 msgr2=0x7f25d0075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25d00737c0 0x7f25d0075c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f25e4007fd0 tx=0x7f25e4007940 comp rx=0 tx=0).stop 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 -- 172.21.15.44:0/1263369445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f015ade0 msgr2=0x7f25f01551d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f015ade0 0x7f25f01551d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f25e800ead0 tx=0x7f25e800ef90 comp rx=0 tx=0).stop 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 -- 172.21.15.44:0/1263369445 shutdown_connections 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f25d00737c0 0x7f25d0075c80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f015ade0 0x7f25f01551d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25f00cf300 0x7f25f0154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 --2- 172.21.15.44:0/1263369445 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25f006e810 0x7f25f014d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 -- 172.21.15.44:0/1263369445 >> 172.21.15.44:0/1263369445 conn(0x7f25f00b18f0 msgr2=0x7f25f0158450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 -- 172.21.15.44:0/1263369445 shutdown_connections 2024-09-13T08:43:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:20.943+0000 7f25f8600640 1 -- 172.21.15.44:0/1263369445 wait complete. 2024-09-13T08:43:20.966 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-13T08:43:20.604967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":374344,"kb_used_data":6828,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754600376,"statfs":{"total":773094113280,"available":772710785024,"internally_reserved":0,"allocated":6991872,"data_stored":3579468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":376098353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.840529"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.818943+0000","last_change":"2024-09-13T08:43:16.817882+0000","last_active":"2024-09-13T08:43:16.818943+0000","last_peered":"2024-09-13T08:43:16.818943+0000","last_clean":"2024-09-13T08:43:16.818943+0000","last_became_active":"2024-09-13T08:43:16.817105+0000","last_became_peered":"2024-09-13T08:43:16.817105+0000","last_unstale":"2024-09-13T08:43:16.818943+0000","last_undegraded":"2024-09-13T08:43:16.818943+0000","last_fullsized":"2024-09-13T08:43:16.818943+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:56:37.249907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861394+0000","last_change":"2024-09-13T08:43:16.818673+0000","last_active":"2024-09-13T08:43:18.861394+0000","last_peered":"2024-09-13T08:43:18.861394+0000","last_clean":"2024-09-13T08:43:18.861394+0000","last_became_active":"2024-09-13T08:43:16.818278+0000","last_became_peered":"2024-09-13T08:43:16.818278+0000","last_unstale":"2024-09-13T08:43:18.861394+0000","last_undegraded":"2024-09-13T08:43:18.861394+0000","last_fullsized":"2024-09-13T08:43:18.861394+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:00.403728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794916+0000","last_change":"2024-09-13T08:43:16.813551+0000","last_active":"2024-09-13T08:43:18.794916+0000","last_peered":"2024-09-13T08:43:18.794916+0000","last_clean":"2024-09-13T08:43:18.794916+0000","last_became_active":"2024-09-13T08:43:16.812940+0000","last_became_peered":"2024-09-13T08:43:16.812940+0000","last_unstale":"2024-09-13T08:43:18.794916+0000","last_undegraded":"2024-09-13T08:43:18.794916+0000","last_fullsized":"2024-09-13T08:43:18.794916+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:48.617459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791472+0000","last_change":"2024-09-13T08:43:16.819995+0000","last_active":"2024-09-13T08:43:17.791472+0000","last_peered":"2024-09-13T08:43:17.791472+0000","last_clean":"2024-09-13T08:43:17.791472+0000","last_became_active":"2024-09-13T08:43:16.818190+0000","last_became_peered":"2024-09-13T08:43:16.818190+0000","last_unstale":"2024-09-13T08:43:17.791472+0000","last_undegraded":"2024-09-13T08:43:17.791472+0000","last_fullsized":"2024-09-13T08:43:17.791472+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:27:56.292419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.816335+0000","last_change":"2024-09-13T08:43:16.808958+0000","last_active":"2024-09-13T08:43:16.816335+0000","last_peered":"2024-09-13T08:43:16.816335+0000","last_clean":"2024-09-13T08:43:16.816335+0000","last_became_active":"2024-09-13T08:43:16.808685+0000","last_became_peered":"2024-09-13T08:43:16.808685+0000","last_unstale":"2024-09-13T08:43:16.816335+0000","last_undegraded":"2024-09-13T08:43:16.816335+0000","last_fullsized":"2024-09-13T08:43:16.816335+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:04:09.314912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.817862+0000","last_change":"2024-09-13T08:43:16.813201+0000","last_active":"2024-09-13T08:43:16.817862+0000","last_peered":"2024-09-13T08:43:16.817862+0000","last_clean":"2024-09-13T08:43:16.817862+0000","last_became_active":"2024-09-13T08:43:16.812933+0000","last_became_peered":"2024-09-13T08:43:16.812933+0000","last_unstale":"2024-09-13T08:43:16.817862+0000","last_undegraded":"2024-09-13T08:43:16.817862+0000","last_fullsized":"2024-09-13T08:43:16.817862+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:22.975460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791108+0000","last_change":"2024-09-13T08:43:16.818209+0000","last_active":"2024-09-13T08:43:17.791108+0000","last_peered":"2024-09-13T08:43:17.791108+0000","last_clean":"2024-09-13T08:43:17.791108+0000","last_became_active":"2024-09-13T08:43:16.817923+0000","last_became_peered":"2024-09-13T08:43:16.817923+0000","last_unstale":"2024-09-13T08:43:17.791108+0000","last_undegraded":"2024-09-13T08:43:17.791108+0000","last_fullsized":"2024-09-13T08:43:17.791108+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:37.310222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791999+0000","last_change":"2024-09-13T08:43:16.809360+0000","last_active":"2024-09-13T08:43:17.791999+0000","last_peered":"2024-09-13T08:43:17.791999+0000","last_clean":"2024-09-13T08:43:17.791999+0000","last_became_active":"2024-09-13T08:43:16.808345+0000","last_became_peered":"2024-09-13T08:43:16.808345+0000","last_unstale":"2024-09-13T08:43:17.791999+0000","last_undegraded":"2024-09-13T08:43:17.791999+0000","last_fullsized":"2024-09-13T08:43:17.791999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:55.030189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.663000+0000","last_change":"2024-09-13T08:43:16.822144+0000","last_active":"2024-09-13T08:43:18.663000+0000","last_peered":"2024-09-13T08:43:18.663000+0000","last_clean":"2024-09-13T08:43:18.663000+0000","last_became_active":"2024-09-13T08:43:16.821864+0000","last_became_peered":"2024-09-13T08:43:16.821864+0000","last_unstale":"2024-09-13T08:43:18.663000+0000","last_undegraded":"2024-09-13T08:43:18.663000+0000","last_fullsized":"2024-09-13T08:43:18.663000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:38.477323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.829154+0000","last_change":"2024-09-13T08:43:16.827402+0000","last_active":"2024-09-13T08:43:16.829154+0000","last_peered":"2024-09-13T08:43:16.829154+0000","last_clean":"2024-09-13T08:43:16.829154+0000","last_became_active":"2024-09-13T08:43:16.827089+0000","last_became_peered":"2024-09-13T08:43:16.827089+0000","last_unstale":"2024-09-13T08:43:16.829154+0000","last_undegraded":"2024-09-13T08:43:16.829154+0000","last_fullsized":"2024-09-13T08:43:16.829154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:14:16.051866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791576+0000","last_change":"2024-09-13T08:43:16.803299+0000","last_active":"2024-09-13T08:43:17.791576+0000","last_peered":"2024-09-13T08:43:17.791576+0000","last_clean":"2024-09-13T08:43:17.791576+0000","last_became_active":"2024-09-13T08:43:16.803009+0000","last_became_peered":"2024-09-13T08:43:16.803009+0000","last_unstale":"2024-09-13T08:43:17.791576+0000","last_undegraded":"2024-09-13T08:43:17.791576+0000","last_fullsized":"2024-09-13T08:43:17.791576+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:22.005050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662774+0000","last_change":"2024-09-13T08:43:16.812577+0000","last_active":"2024-09-13T08:43:18.662774+0000","last_peered":"2024-09-13T08:43:18.662774+0000","last_clean":"2024-09-13T08:43:18.662774+0000","last_became_active":"2024-09-13T08:43:16.812313+0000","last_became_peered":"2024-09-13T08:43:16.812313+0000","last_unstale":"2024-09-13T08:43:18.662774+0000","last_undegraded":"2024-09-13T08:43:18.662774+0000","last_fullsized":"2024-09-13T08:43:18.662774+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:04:13.261593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860593+0000","last_change":"2024-09-13T08:43:16.817172+0000","last_active":"2024-09-13T08:43:18.860593+0000","last_peered":"2024-09-13T08:43:18.860593+0000","last_clean":"2024-09-13T08:43:18.860593+0000","last_became_active":"2024-09-13T08:43:16.816869+0000","last_became_peered":"2024-09-13T08:43:16.816869+0000","last_unstale":"2024-09-13T08:43:18.860593+0000","last_undegraded":"2024-09-13T08:43:18.860593+0000","last_fullsized":"2024-09-13T08:43:18.860593+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:08.354996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794463+0000","last_change":"2024-09-13T08:43:16.806479+0000","last_active":"2024-09-13T08:43:18.794463+0000","last_peered":"2024-09-13T08:43:18.794463+0000","last_clean":"2024-09-13T08:43:18.794463+0000","last_became_active":"2024-09-13T08:43:16.806208+0000","last_became_peered":"2024-09-13T08:43:16.806208+0000","last_unstale":"2024-09-13T08:43:18.794463+0000","last_undegraded":"2024-09-13T08:43:18.794463+0000","last_fullsized":"2024-09-13T08:43:18.794463+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:13.700936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860240+0000","last_change":"2024-09-13T08:43:16.817320+0000","last_active":"2024-09-13T08:43:18.860240+0000","last_peered":"2024-09-13T08:43:18.860240+0000","last_clean":"2024-09-13T08:43:18.860240+0000","last_became_active":"2024-09-13T08:43:16.816380+0000","last_became_peered":"2024-09-13T08:43:16.816380+0000","last_unstale":"2024-09-13T08:43:18.860240+0000","last_undegraded":"2024-09-13T08:43:18.860240+0000","last_fullsized":"2024-09-13T08:43:18.860240+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:51:59.644256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.819706+0000","last_change":"2024-09-13T08:43:16.815202+0000","last_active":"2024-09-13T08:43:16.819706+0000","last_peered":"2024-09-13T08:43:16.819706+0000","last_clean":"2024-09-13T08:43:16.819706+0000","last_became_active":"2024-09-13T08:43:16.813944+0000","last_became_peered":"2024-09-13T08:43:16.813944+0000","last_unstale":"2024-09-13T08:43:16.819706+0000","last_undegraded":"2024-09-13T08:43:16.819706+0000","last_fullsized":"2024-09-13T08:43:16.819706+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:17.821216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.818966+0000","last_change":"2024-09-13T08:43:16.817920+0000","last_active":"2024-09-13T08:43:16.818966+0000","last_peered":"2024-09-13T08:43:16.818966+0000","last_clean":"2024-09-13T08:43:16.818966+0000","last_became_active":"2024-09-13T08:43:16.817609+0000","last_became_peered":"2024-09-13T08:43:16.817609+0000","last_unstale":"2024-09-13T08:43:16.818966+0000","last_undegraded":"2024-09-13T08:43:16.818966+0000","last_fullsized":"2024-09-13T08:43:16.818966+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:21.956413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794771+0000","last_change":"2024-09-13T08:43:16.809067+0000","last_active":"2024-09-13T08:43:18.794771+0000","last_peered":"2024-09-13T08:43:18.794771+0000","last_clean":"2024-09-13T08:43:18.794771+0000","last_became_active":"2024-09-13T08:43:16.808770+0000","last_became_peered":"2024-09-13T08:43:16.808770+0000","last_unstale":"2024-09-13T08:43:18.794771+0000","last_undegraded":"2024-09-13T08:43:18.794771+0000","last_fullsized":"2024-09-13T08:43:18.794771+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:06:51.560999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861119+0000","last_change":"2024-09-13T08:43:16.815008+0000","last_active":"2024-09-13T08:43:18.861119+0000","last_peered":"2024-09-13T08:43:18.861119+0000","last_clean":"2024-09-13T08:43:18.861119+0000","last_became_active":"2024-09-13T08:43:16.814780+0000","last_became_peered":"2024-09-13T08:43:16.814780+0000","last_unstale":"2024-09-13T08:43:18.861119+0000","last_undegraded":"2024-09-13T08:43:18.861119+0000","last_fullsized":"2024-09-13T08:43:18.861119+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:23:52.086003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791307+0000","last_change":"2024-09-13T08:43:16.818745+0000","last_active":"2024-09-13T08:43:17.791307+0000","last_peered":"2024-09-13T08:43:17.791307+0000","last_clean":"2024-09-13T08:43:17.791307+0000","last_became_active":"2024-09-13T08:43:16.812424+0000","last_became_peered":"2024-09-13T08:43:16.812424+0000","last_unstale":"2024-09-13T08:43:17.791307+0000","last_undegraded":"2024-09-13T08:43:17.791307+0000","last_fullsized":"2024-09-13T08:43:17.791307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:28:12.706783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860518+0000","last_change":"2024-09-13T08:43:16.810671+0000","last_active":"2024-09-13T08:43:18.860518+0000","last_peered":"2024-09-13T08:43:18.860518+0000","last_clean":"2024-09-13T08:43:18.860518+0000","last_became_active":"2024-09-13T08:43:16.810290+0000","last_became_peered":"2024-09-13T08:43:16.810290+0000","last_unstale":"2024-09-13T08:43:18.860518+0000","last_undegraded":"2024-09-13T08:43:18.860518+0000","last_fullsized":"2024-09-13T08:43:18.860518+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:08.673621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791257+0000","last_change":"2024-09-13T08:43:16.810346+0000","last_active":"2024-09-13T08:43:17.791257+0000","last_peered":"2024-09-13T08:43:17.791257+0000","last_clean":"2024-09-13T08:43:17.791257+0000","last_became_active":"2024-09-13T08:43:16.808958+0000","last_became_peered":"2024-09-13T08:43:16.808958+0000","last_unstale":"2024-09-13T08:43:17.791257+0000","last_undegraded":"2024-09-13T08:43:17.791257+0000","last_fullsized":"2024-09-13T08:43:17.791257+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:50.393357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791175+0000","last_change":"2024-09-13T08:43:16.815624+0000","last_active":"2024-09-13T08:43:17.791175+0000","last_peered":"2024-09-13T08:43:17.791175+0000","last_clean":"2024-09-13T08:43:17.791175+0000","last_became_active":"2024-09-13T08:43:16.815388+0000","last_became_peered":"2024-09-13T08:43:16.815388+0000","last_unstale":"2024-09-13T08:43:17.791175+0000","last_undegraded":"2024-09-13T08:43:17.791175+0000","last_fullsized":"2024-09-13T08:43:17.791175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:24:19.128614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.663090+0000","last_change":"2024-09-13T08:43:16.822958+0000","last_active":"2024-09-13T08:43:18.663090+0000","last_peered":"2024-09-13T08:43:18.663090+0000","last_clean":"2024-09-13T08:43:18.663090+0000","last_became_active":"2024-09-13T08:43:16.822701+0000","last_became_peered":"2024-09-13T08:43:16.822701+0000","last_unstale":"2024-09-13T08:43:18.663090+0000","last_undegraded":"2024-09-13T08:43:18.663090+0000","last_fullsized":"2024-09-13T08:43:18.663090+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:18.835674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861393+0000","last_change":"2024-09-13T08:43:16.818229+0000","last_active":"2024-09-13T08:43:18.861393+0000","last_peered":"2024-09-13T08:43:18.861393+0000","last_clean":"2024-09-13T08:43:18.861393+0000","last_became_active":"2024-09-13T08:43:16.817978+0000","last_became_peered":"2024-09-13T08:43:16.817978+0000","last_unstale":"2024-09-13T08:43:18.861393+0000","last_undegraded":"2024-09-13T08:43:18.861393+0000","last_fullsized":"2024-09-13T08:43:18.861393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:24.066893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860721+0000","last_change":"2024-09-13T08:43:16.807144+0000","last_active":"2024-09-13T08:43:18.860721+0000","last_peered":"2024-09-13T08:43:18.860721+0000","last_clean":"2024-09-13T08:43:18.860721+0000","last_became_active":"2024-09-13T08:43:16.806751+0000","last_became_peered":"2024-09-13T08:43:16.806751+0000","last_unstale":"2024-09-13T08:43:18.860721+0000","last_undegraded":"2024-09-13T08:43:18.860721+0000","last_fullsized":"2024-09-13T08:43:18.860721+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:43.503649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861285+0000","last_change":"2024-09-13T08:43:16.824234+0000","last_active":"2024-09-13T08:43:18.861285+0000","last_peered":"2024-09-13T08:43:18.861285+0000","last_clean":"2024-09-13T08:43:18.861285+0000","last_became_active":"2024-09-13T08:43:16.822388+0000","last_became_peered":"2024-09-13T08:43:16.822388+0000","last_unstale":"2024-09-13T08:43:18.861285+0000","last_undegraded":"2024-09-13T08:43:18.861285+0000","last_fullsized":"2024-09-13T08:43:18.861285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:10:45.208508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791528+0000","last_change":"2024-09-13T08:43:16.815164+0000","last_active":"2024-09-13T08:43:17.791528+0000","last_peered":"2024-09-13T08:43:17.791528+0000","last_clean":"2024-09-13T08:43:17.791528+0000","last_became_active":"2024-09-13T08:43:16.814905+0000","last_became_peered":"2024-09-13T08:43:16.814905+0000","last_unstale":"2024-09-13T08:43:17.791528+0000","last_undegraded":"2024-09-13T08:43:17.791528+0000","last_fullsized":"2024-09-13T08:43:17.791528+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:17.220615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790840+0000","last_change":"2024-09-13T08:43:16.810533+0000","last_active":"2024-09-13T08:43:17.790840+0000","last_peered":"2024-09-13T08:43:17.790840+0000","last_clean":"2024-09-13T08:43:17.790840+0000","last_became_active":"2024-09-13T08:43:16.810186+0000","last_became_peered":"2024-09-13T08:43:16.810186+0000","last_unstale":"2024-09-13T08:43:17.790840+0000","last_undegraded":"2024-09-13T08:43:17.790840+0000","last_fullsized":"2024-09-13T08:43:17.790840+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:49:43.333978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791138+0000","last_change":"2024-09-13T08:43:16.813821+0000","last_active":"2024-09-13T08:43:17.791138+0000","last_peered":"2024-09-13T08:43:17.791138+0000","last_clean":"2024-09-13T08:43:17.791138+0000","last_became_active":"2024-09-13T08:43:16.813564+0000","last_became_peered":"2024-09-13T08:43:16.813564+0000","last_unstale":"2024-09-13T08:43:17.791138+0000","last_undegraded":"2024-09-13T08:43:17.791138+0000","last_fullsized":"2024-09-13T08:43:17.791138+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:08:00.544785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860832+0000","last_change":"2024-09-13T08:43:16.808390+0000","last_active":"2024-09-13T08:43:18.860832+0000","last_peered":"2024-09-13T08:43:18.860832+0000","last_clean":"2024-09-13T08:43:18.860832+0000","last_became_active":"2024-09-13T08:43:16.808078+0000","last_became_peered":"2024-09-13T08:43:16.808078+0000","last_unstale":"2024-09-13T08:43:18.860832+0000","last_undegraded":"2024-09-13T08:43:18.860832+0000","last_fullsized":"2024-09-13T08:43:18.860832+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:48.635890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790519+0000","last_change":"2024-09-13T08:43:16.805571+0000","last_active":"2024-09-13T08:43:17.790519+0000","last_peered":"2024-09-13T08:43:17.790519+0000","last_clean":"2024-09-13T08:43:17.790519+0000","last_became_active":"2024-09-13T08:43:16.805209+0000","last_became_peered":"2024-09-13T08:43:16.805209+0000","last_unstale":"2024-09-13T08:43:17.790519+0000","last_undegraded":"2024-09-13T08:43:17.790519+0000","last_fullsized":"2024-09-13T08:43:17.790519+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:50.997778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.818642+0000","last_change":"2024-09-13T08:43:16.810204+0000","last_active":"2024-09-13T08:43:16.818642+0000","last_peered":"2024-09-13T08:43:16.818642+0000","last_clean":"2024-09-13T08:43:16.818642+0000","last_became_active":"2024-09-13T08:43:16.809948+0000","last_became_peered":"2024-09-13T08:43:16.809948+0000","last_unstale":"2024-09-13T08:43:16.818642+0000","last_undegraded":"2024-09-13T08:43:16.818642+0000","last_fullsized":"2024-09-13T08:43:16.818642+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:14:54.111787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"60'4","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791001+0000","last_change":"2024-09-13T08:43:15.786799+0000","last_active":"2024-09-13T08:43:17.791001+0000","last_peered":"2024-09-13T08:43:17.791001+0000","last_clean":"2024-09-13T08:43:17.791001+0000","last_became_active":"2024-09-13T08:43:15.786517+0000","last_became_peered":"2024-09-13T08:43:15.786517+0000","last_unstale":"2024-09-13T08:43:17.791001+0000","last_undegraded":"2024-09-13T08:43:17.791001+0000","last_fullsized":"2024-09-13T08:43:17.791001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:57.379125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.818177+0000","last_change":"2024-09-13T08:43:16.808004+0000","last_active":"2024-09-13T08:43:16.818177+0000","last_peered":"2024-09-13T08:43:16.818177+0000","last_clean":"2024-09-13T08:43:16.818177+0000","last_became_active":"2024-09-13T08:43:16.807740+0000","last_became_peered":"2024-09-13T08:43:16.807740+0000","last_unstale":"2024-09-13T08:43:16.818177+0000","last_undegraded":"2024-09-13T08:43:16.818177+0000","last_fullsized":"2024-09-13T08:43:16.818177+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:09.440313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861226+0000","last_change":"2024-09-13T08:43:15.800841+0000","last_active":"2024-09-13T08:43:18.861226+0000","last_peered":"2024-09-13T08:43:18.861226+0000","last_clean":"2024-09-13T08:43:18.861226+0000","last_became_active":"2024-09-13T08:43:15.800649+0000","last_became_peered":"2024-09-13T08:43:15.800649+0000","last_unstale":"2024-09-13T08:43:18.861226+0000","last_undegraded":"2024-09-13T08:43:18.861226+0000","last_fullsized":"2024-09-13T08:43:18.861226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:24.902615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791218+0000","last_change":"2024-09-13T08:43:16.818858+0000","last_active":"2024-09-13T08:43:17.791218+0000","last_peered":"2024-09-13T08:43:17.791218+0000","last_clean":"2024-09-13T08:43:17.791218+0000","last_became_active":"2024-09-13T08:43:16.818554+0000","last_became_peered":"2024-09-13T08:43:16.818554+0000","last_unstale":"2024-09-13T08:43:17.791218+0000","last_undegraded":"2024-09-13T08:43:17.791218+0000","last_fullsized":"2024-09-13T08:43:17.791218+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:30.063946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"60'2","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.828229+0000","last_change":"2024-09-13T08:43:15.793173+0000","last_active":"2024-09-13T08:43:16.828229+0000","last_peered":"2024-09-13T08:43:16.828229+0000","last_clean":"2024-09-13T08:43:16.828229+0000","last_became_active":"2024-09-13T08:43:15.792801+0000","last_became_peered":"2024-09-13T08:43:15.792801+0000","last_unstale":"2024-09-13T08:43:16.828229+0000","last_undegraded":"2024-09-13T08:43:16.828229+0000","last_fullsized":"2024-09-13T08:43:16.828229+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:25.488120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.819662+0000","last_change":"2024-09-13T08:43:16.809830+0000","last_active":"2024-09-13T08:43:16.819662+0000","last_peered":"2024-09-13T08:43:16.819662+0000","last_clean":"2024-09-13T08:43:16.819662+0000","last_became_active":"2024-09-13T08:43:16.809608+0000","last_became_peered":"2024-09-13T08:43:16.809608+0000","last_unstale":"2024-09-13T08:43:16.819662+0000","last_undegraded":"2024-09-13T08:43:16.819662+0000","last_fullsized":"2024-09-13T08:43:16.819662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:01:27.492994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791484+0000","last_change":"2024-09-13T08:43:15.790858+0000","last_active":"2024-09-13T08:43:17.791484+0000","last_peered":"2024-09-13T08:43:17.791484+0000","last_clean":"2024-09-13T08:43:17.791484+0000","last_became_active":"2024-09-13T08:43:15.790574+0000","last_became_peered":"2024-09-13T08:43:15.790574+0000","last_unstale":"2024-09-13T08:43:17.791484+0000","last_undegraded":"2024-09-13T08:43:17.791484+0000","last_fullsized":"2024-09-13T08:43:17.791484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:52.809472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790759+0000","last_change":"2024-09-13T08:43:16.795036+0000","last_active":"2024-09-13T08:43:17.790759+0000","last_peered":"2024-09-13T08:43:17.790759+0000","last_clean":"2024-09-13T08:43:17.790759+0000","last_became_active":"2024-09-13T08:43:16.794723+0000","last_became_peered":"2024-09-13T08:43:16.794723+0000","last_unstale":"2024-09-13T08:43:17.790759+0000","last_undegraded":"2024-09-13T08:43:17.790759+0000","last_fullsized":"2024-09-13T08:43:17.790759+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:24.811313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794662+0000","last_change":"2024-09-13T08:43:15.790947+0000","last_active":"2024-09-13T08:43:18.794662+0000","last_peered":"2024-09-13T08:43:18.794662+0000","last_clean":"2024-09-13T08:43:18.794662+0000","last_became_active":"2024-09-13T08:43:15.790165+0000","last_became_peered":"2024-09-13T08:43:15.790165+0000","last_unstale":"2024-09-13T08:43:18.794662+0000","last_undegraded":"2024-09-13T08:43:18.794662+0000","last_fullsized":"2024-09-13T08:43:18.794662+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:51:27.879550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790780+0000","last_change":"2024-09-13T08:43:16.809236+0000","last_active":"2024-09-13T08:43:17.790780+0000","last_peered":"2024-09-13T08:43:17.790780+0000","last_clean":"2024-09-13T08:43:17.790780+0000","last_became_active":"2024-09-13T08:43:16.808889+0000","last_became_peered":"2024-09-13T08:43:16.808889+0000","last_unstale":"2024-09-13T08:43:17.790780+0000","last_undegraded":"2024-09-13T08:43:17.790780+0000","last_fullsized":"2024-09-13T08:43:17.790780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:07:44.255227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"60'1","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.827511+0000","last_change":"2024-09-13T08:43:15.791105+0000","last_active":"2024-09-13T08:43:16.827511+0000","last_peered":"2024-09-13T08:43:16.827511+0000","last_clean":"2024-09-13T08:43:16.827511+0000","last_became_active":"2024-09-13T08:43:15.790803+0000","last_became_peered":"2024-09-13T08:43:15.790803+0000","last_unstale":"2024-09-13T08:43:16.827511+0000","last_undegraded":"2024-09-13T08:43:16.827511+0000","last_fullsized":"2024-09-13T08:43:16.827511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:49.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794393+0000","last_change":"2024-09-13T08:43:16.805584+0000","last_active":"2024-09-13T08:43:18.794393+0000","last_peered":"2024-09-13T08:43:18.794393+0000","last_clean":"2024-09-13T08:43:18.794393+0000","last_became_active":"2024-09-13T08:43:16.805259+0000","last_became_peered":"2024-09-13T08:43:16.805259+0000","last_unstale":"2024-09-13T08:43:18.794393+0000","last_undegraded":"2024-09-13T08:43:18.794393+0000","last_fullsized":"2024-09-13T08:43:18.794393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:24:35.034287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662622+0000","last_change":"2024-09-13T08:43:15.785688+0000","last_active":"2024-09-13T08:43:18.662622+0000","last_peered":"2024-09-13T08:43:18.662622+0000","last_clean":"2024-09-13T08:43:18.662622+0000","last_became_active":"2024-09-13T08:43:15.785388+0000","last_became_peered":"2024-09-13T08:43:15.785388+0000","last_unstale":"2024-09-13T08:43:18.662622+0000","last_undegraded":"2024-09-13T08:43:18.662622+0000","last_fullsized":"2024-09-13T08:43:18.662622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:12.431789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.811773+0000","last_change":"2024-09-13T08:43:16.805154+0000","last_active":"2024-09-13T08:43:16.811773+0000","last_peered":"2024-09-13T08:43:16.811773+0000","last_clean":"2024-09-13T08:43:16.811773+0000","last_became_active":"2024-09-13T08:43:16.804886+0000","last_became_peered":"2024-09-13T08:43:16.804886+0000","last_unstale":"2024-09-13T08:43:16.811773+0000","last_undegraded":"2024-09-13T08:43:16.811773+0000","last_fullsized":"2024-09-13T08:43:16.811773+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:14.772801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860103+0000","last_change":"2024-09-13T08:43:15.787515+0000","last_active":"2024-09-13T08:43:18.860103+0000","last_peered":"2024-09-13T08:43:18.860103+0000","last_clean":"2024-09-13T08:43:18.860103+0000","last_became_active":"2024-09-13T08:43:15.786997+0000","last_became_peered":"2024-09-13T08:43:15.786997+0000","last_unstale":"2024-09-13T08:43:18.860103+0000","last_undegraded":"2024-09-13T08:43:18.860103+0000","last_fullsized":"2024-09-13T08:43:18.860103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:00.138886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860801+0000","last_change":"2024-09-13T08:43:16.798116+0000","last_active":"2024-09-13T08:43:18.860801+0000","last_peered":"2024-09-13T08:43:18.860801+0000","last_clean":"2024-09-13T08:43:18.860801+0000","last_became_active":"2024-09-13T08:43:16.797800+0000","last_became_peered":"2024-09-13T08:43:16.797800+0000","last_unstale":"2024-09-13T08:43:18.860801+0000","last_undegraded":"2024-09-13T08:43:18.860801+0000","last_fullsized":"2024-09-13T08:43:18.860801+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:27.320983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662790+0000","last_change":"2024-09-13T08:43:15.792500+0000","last_active":"2024-09-13T08:43:18.662790+0000","last_peered":"2024-09-13T08:43:18.662790+0000","last_clean":"2024-09-13T08:43:18.662790+0000","last_became_active":"2024-09-13T08:43:15.792200+0000","last_became_peered":"2024-09-13T08:43:15.792200+0000","last_unstale":"2024-09-13T08:43:18.662790+0000","last_undegraded":"2024-09-13T08:43:18.662790+0000","last_fullsized":"2024-09-13T08:43:18.662790+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:23.815682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794921+0000","last_change":"2024-09-13T08:43:16.805873+0000","last_active":"2024-09-13T08:43:18.794921+0000","last_peered":"2024-09-13T08:43:18.794921+0000","last_clean":"2024-09-13T08:43:18.794921+0000","last_became_active":"2024-09-13T08:43:16.805545+0000","last_became_peered":"2024-09-13T08:43:16.805545+0000","last_unstale":"2024-09-13T08:43:18.794921+0000","last_undegraded":"2024-09-13T08:43:18.794921+0000","last_fullsized":"2024-09-13T08:43:18.794921+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:11:21.886945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791685+0000","last_change":"2024-09-13T08:43:15.788513+0000","last_active":"2024-09-13T08:43:17.791685+0000","last_peered":"2024-09-13T08:43:17.791685+0000","last_clean":"2024-09-13T08:43:17.791685+0000","last_became_active":"2024-09-13T08:43:15.787923+0000","last_became_peered":"2024-09-13T08:43:15.787923+0000","last_unstale":"2024-09-13T08:43:17.791685+0000","last_undegraded":"2024-09-13T08:43:17.791685+0000","last_fullsized":"2024-09-13T08:43:17.791685+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:47.151058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791816+0000","last_change":"2024-09-13T08:43:16.811398+0000","last_active":"2024-09-13T08:43:17.791816+0000","last_peered":"2024-09-13T08:43:17.791816+0000","last_clean":"2024-09-13T08:43:17.791816+0000","last_became_active":"2024-09-13T08:43:16.811081+0000","last_became_peered":"2024-09-13T08:43:16.811081+0000","last_unstale":"2024-09-13T08:43:17.791816+0000","last_undegraded":"2024-09-13T08:43:17.791816+0000","last_fullsized":"2024-09-13T08:43:17.791816+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:32:12.639464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"60'3","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791708+0000","last_change":"2024-09-13T08:43:15.792355+0000","last_active":"2024-09-13T08:43:17.791708+0000","last_peered":"2024-09-13T08:43:17.791708+0000","last_clean":"2024-09-13T08:43:17.791708+0000","last_became_active":"2024-09-13T08:43:15.791972+0000","last_became_peered":"2024-09-13T08:43:15.791972+0000","last_unstale":"2024-09-13T08:43:17.791708+0000","last_undegraded":"2024-09-13T08:43:17.791708+0000","last_fullsized":"2024-09-13T08:43:17.791708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:54.861136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791510+0000","last_change":"2024-09-13T08:43:16.821625+0000","last_active":"2024-09-13T08:43:17.791510+0000","last_peered":"2024-09-13T08:43:17.791510+0000","last_clean":"2024-09-13T08:43:17.791510+0000","last_became_active":"2024-09-13T08:43:16.821277+0000","last_became_peered":"2024-09-13T08:43:16.821277+0000","last_unstale":"2024-09-13T08:43:17.791510+0000","last_undegraded":"2024-09-13T08:43:17.791510+0000","last_fullsized":"2024-09-13T08:43:17.791510+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:15.920121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"60'6","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860561+0000","last_change":"2024-09-13T08:43:15.788489+0000","last_active":"2024-09-13T08:43:18.860561+0000","last_peered":"2024-09-13T08:43:18.860561+0000","last_clean":"2024-09-13T08:43:18.860561+0000","last_became_active":"2024-09-13T08:43:15.788176+0000","last_became_peered":"2024-09-13T08:43:15.788176+0000","last_unstale":"2024-09-13T08:43:18.860561+0000","last_undegraded":"2024-09-13T08:43:18.860561+0000","last_fullsized":"2024-09-13T08:43:18.860561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:51:27.879550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662727+0000","last_change":"2024-09-13T08:43:16.809196+0000","last_active":"2024-09-13T08:43:18.662727+0000","last_peered":"2024-09-13T08:43:18.662727+0000","last_clean":"2024-09-13T08:43:18.662727+0000","last_became_active":"2024-09-13T08:43:16.808921+0000","last_became_peered":"2024-09-13T08:43:16.808921+0000","last_unstale":"2024-09-13T08:43:18.662727+0000","last_undegraded":"2024-09-13T08:43:18.662727+0000","last_fullsized":"2024-09-13T08:43:18.662727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:39.435646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790950+0000","last_change":"2024-09-13T08:43:15.783835+0000","last_active":"2024-09-13T08:43:17.790950+0000","last_peered":"2024-09-13T08:43:17.790950+0000","last_clean":"2024-09-13T08:43:17.790950+0000","last_became_active":"2024-09-13T08:43:15.783522+0000","last_became_peered":"2024-09-13T08:43:15.783522+0000","last_unstale":"2024-09-13T08:43:17.790950+0000","last_undegraded":"2024-09-13T08:43:17.790950+0000","last_fullsized":"2024-09-13T08:43:17.790950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:28:11.697090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794837+0000","last_change":"2024-09-13T08:43:16.820336+0000","last_active":"2024-09-13T08:43:18.794837+0000","last_peered":"2024-09-13T08:43:18.794837+0000","last_clean":"2024-09-13T08:43:18.794837+0000","last_became_active":"2024-09-13T08:43:16.820002+0000","last_became_peered":"2024-09-13T08:43:16.820002+0000","last_unstale":"2024-09-13T08:43:18.794837+0000","last_undegraded":"2024-09-13T08:43:18.794837+0000","last_fullsized":"2024-09-13T08:43:18.794837+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:44:51.897731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791068+0000","last_change":"2024-09-13T08:43:15.792918+0000","last_active":"2024-09-13T08:43:17.791068+0000","last_peered":"2024-09-13T08:43:17.791068+0000","last_clean":"2024-09-13T08:43:17.791068+0000","last_became_active":"2024-09-13T08:43:15.791822+0000","last_became_peered":"2024-09-13T08:43:15.791822+0000","last_unstale":"2024-09-13T08:43:17.791068+0000","last_undegraded":"2024-09-13T08:43:17.791068+0000","last_fullsized":"2024-09-13T08:43:17.791068+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:51:41.415569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662981+0000","last_change":"2024-09-13T08:43:16.810442+0000","last_active":"2024-09-13T08:43:18.662981+0000","last_peered":"2024-09-13T08:43:18.662981+0000","last_clean":"2024-09-13T08:43:18.662981+0000","last_became_active":"2024-09-13T08:43:16.810152+0000","last_became_peered":"2024-09-13T08:43:16.810152+0000","last_unstale":"2024-09-13T08:43:18.662981+0000","last_undegraded":"2024-09-13T08:43:18.662981+0000","last_fullsized":"2024-09-13T08:43:18.662981+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:48.635890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791285+0000","last_change":"2024-09-13T08:43:15.786181+0000","last_active":"2024-09-13T08:43:17.791285+0000","last_peered":"2024-09-13T08:43:17.791285+0000","last_clean":"2024-09-13T08:43:17.791285+0000","last_became_active":"2024-09-13T08:43:15.785695+0000","last_became_peered":"2024-09-13T08:43:15.785695+0000","last_unstale":"2024-09-13T08:43:17.791285+0000","last_undegraded":"2024-09-13T08:43:17.791285+0000","last_fullsized":"2024-09-13T08:43:17.791285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:08.478225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791328+0000","last_change":"2024-09-13T08:43:16.815274+0000","last_active":"2024-09-13T08:43:17.791328+0000","last_peered":"2024-09-13T08:43:17.791328+0000","last_clean":"2024-09-13T08:43:17.791328+0000","last_became_active":"2024-09-13T08:43:16.804806+0000","last_became_peered":"2024-09-13T08:43:16.804806+0000","last_unstale":"2024-09-13T08:43:17.791328+0000","last_undegraded":"2024-09-13T08:43:17.791328+0000","last_fullsized":"2024-09-13T08:43:17.791328+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:33.793602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"60'8","reported_seq":23,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791049+0000","last_change":"2024-09-13T08:43:15.786318+0000","last_active":"2024-09-13T08:43:17.791049+0000","last_peered":"2024-09-13T08:43:17.791049+0000","last_clean":"2024-09-13T08:43:17.791049+0000","last_became_active":"2024-09-13T08:43:15.785846+0000","last_became_peered":"2024-09-13T08:43:15.785846+0000","last_unstale":"2024-09-13T08:43:17.791049+0000","last_undegraded":"2024-09-13T08:43:17.791049+0000","last_fullsized":"2024-09-13T08:43:17.791049+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:10.086776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.795016+0000","last_change":"2024-09-13T08:43:16.803765+0000","last_active":"2024-09-13T08:43:18.795016+0000","last_peered":"2024-09-13T08:43:18.795016+0000","last_clean":"2024-09-13T08:43:18.795016+0000","last_became_active":"2024-09-13T08:43:16.803487+0000","last_became_peered":"2024-09-13T08:43:16.803487+0000","last_unstale":"2024-09-13T08:43:18.795016+0000","last_undegraded":"2024-09-13T08:43:18.795016+0000","last_fullsized":"2024-09-13T08:43:18.795016+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:45.977000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861251+0000","last_change":"2024-09-13T08:43:15.795460+0000","last_active":"2024-09-13T08:43:18.861251+0000","last_peered":"2024-09-13T08:43:18.861251+0000","last_clean":"2024-09-13T08:43:18.861251+0000","last_became_active":"2024-09-13T08:43:15.795165+0000","last_became_peered":"2024-09-13T08:43:15.795165+0000","last_unstale":"2024-09-13T08:43:18.861251+0000","last_undegraded":"2024-09-13T08:43:18.861251+0000","last_fullsized":"2024-09-13T08:43:18.861251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:23.815682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794926+0000","last_change":"2024-09-13T08:43:16.806639+0000","last_active":"2024-09-13T08:43:18.794926+0000","last_peered":"2024-09-13T08:43:18.794926+0000","last_clean":"2024-09-13T08:43:18.794926+0000","last_became_active":"2024-09-13T08:43:16.806341+0000","last_became_peered":"2024-09-13T08:43:16.806341+0000","last_unstale":"2024-09-13T08:43:18.794926+0000","last_undegraded":"2024-09-13T08:43:18.794926+0000","last_fullsized":"2024-09-13T08:43:18.794926+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:13:28.124095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791809+0000","last_change":"2024-09-13T08:43:15.788192+0000","last_active":"2024-09-13T08:43:17.791809+0000","last_peered":"2024-09-13T08:43:17.791809+0000","last_clean":"2024-09-13T08:43:17.791809+0000","last_became_active":"2024-09-13T08:43:15.787581+0000","last_became_peered":"2024-09-13T08:43:15.787581+0000","last_unstale":"2024-09-13T08:43:17.791809+0000","last_undegraded":"2024-09-13T08:43:17.791809+0000","last_fullsized":"2024-09-13T08:43:17.791809+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:22:42.515576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860836+0000","last_change":"2024-09-13T08:43:16.814575+0000","last_active":"2024-09-13T08:43:18.860836+0000","last_peered":"2024-09-13T08:43:18.860836+0000","last_clean":"2024-09-13T08:43:18.860836+0000","last_became_active":"2024-09-13T08:43:16.814038+0000","last_became_peered":"2024-09-13T08:43:16.814038+0000","last_unstale":"2024-09-13T08:43:18.860836+0000","last_undegraded":"2024-09-13T08:43:18.860836+0000","last_fullsized":"2024-09-13T08:43:18.860836+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:56.549276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794851+0000","last_change":"2024-09-13T08:43:15.793141+0000","last_active":"2024-09-13T08:43:18.794851+0000","last_peered":"2024-09-13T08:43:18.794851+0000","last_clean":"2024-09-13T08:43:18.794851+0000","last_became_active":"2024-09-13T08:43:15.792406+0000","last_became_peered":"2024-09-13T08:43:15.792406+0000","last_unstale":"2024-09-13T08:43:18.794851+0000","last_undegraded":"2024-09-13T08:43:18.794851+0000","last_fullsized":"2024-09-13T08:43:18.794851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:31:46.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791703+0000","last_change":"2024-09-13T08:43:16.814229+0000","last_active":"2024-09-13T08:43:17.791703+0000","last_peered":"2024-09-13T08:43:17.791703+0000","last_clean":"2024-09-13T08:43:17.791703+0000","last_became_active":"2024-09-13T08:43:16.813894+0000","last_became_peered":"2024-09-13T08:43:16.813894+0000","last_unstale":"2024-09-13T08:43:17.791703+0000","last_undegraded":"2024-09-13T08:43:17.791703+0000","last_fullsized":"2024-09-13T08:43:17.791703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:11.285883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.792270+0000","last_change":"2024-09-13T08:43:15.784641+0000","last_active":"2024-09-13T08:43:17.792270+0000","last_peered":"2024-09-13T08:43:17.792270+0000","last_clean":"2024-09-13T08:43:17.792270+0000","last_became_active":"2024-09-13T08:43:15.783695+0000","last_became_peered":"2024-09-13T08:43:15.783695+0000","last_unstale":"2024-09-13T08:43:17.792270+0000","last_undegraded":"2024-09-13T08:43:17.792270+0000","last_fullsized":"2024-09-13T08:43:17.792270+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:20.995357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662790+0000","last_change":"2024-09-13T08:43:16.822104+0000","last_active":"2024-09-13T08:43:18.662790+0000","last_peered":"2024-09-13T08:43:18.662790+0000","last_clean":"2024-09-13T08:43:18.662790+0000","last_became_active":"2024-09-13T08:43:16.821813+0000","last_became_peered":"2024-09-13T08:43:16.821813+0000","last_unstale":"2024-09-13T08:43:18.662790+0000","last_undegraded":"2024-09-13T08:43:18.662790+0000","last_fullsized":"2024-09-13T08:43:18.662790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:35.031802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791006+0000","last_change":"2024-09-13T08:43:15.797489+0000","last_active":"2024-09-13T08:43:17.791006+0000","last_peered":"2024-09-13T08:43:17.791006+0000","last_clean":"2024-09-13T08:43:17.791006+0000","last_became_active":"2024-09-13T08:43:15.797117+0000","last_became_peered":"2024-09-13T08:43:15.797117+0000","last_unstale":"2024-09-13T08:43:17.791006+0000","last_undegraded":"2024-09-13T08:43:17.791006+0000","last_fullsized":"2024-09-13T08:43:17.791006+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:32.605971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791095+0000","last_change":"2024-09-13T08:43:16.806921+0000","last_active":"2024-09-13T08:43:17.791095+0000","last_peered":"2024-09-13T08:43:17.791095+0000","last_clean":"2024-09-13T08:43:17.791095+0000","last_became_active":"2024-09-13T08:43:16.806475+0000","last_became_peered":"2024-09-13T08:43:16.806475+0000","last_unstale":"2024-09-13T08:43:17.791095+0000","last_undegraded":"2024-09-13T08:43:17.791095+0000","last_fullsized":"2024-09-13T08:43:17.791095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:02.071955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790869+0000","last_change":"2024-09-13T08:43:15.783802+0000","last_active":"2024-09-13T08:43:17.790869+0000","last_peered":"2024-09-13T08:43:17.790869+0000","last_clean":"2024-09-13T08:43:17.790869+0000","last_became_active":"2024-09-13T08:43:15.783504+0000","last_became_peered":"2024-09-13T08:43:15.783504+0000","last_unstale":"2024-09-13T08:43:17.790869+0000","last_undegraded":"2024-09-13T08:43:17.790869+0000","last_fullsized":"2024-09-13T08:43:17.790869+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:49.383664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790695+0000","last_change":"2024-09-13T08:43:16.802620+0000","last_active":"2024-09-13T08:43:17.790695+0000","last_peered":"2024-09-13T08:43:17.790695+0000","last_clean":"2024-09-13T08:43:17.790695+0000","last_became_active":"2024-09-13T08:43:16.802170+0000","last_became_peered":"2024-09-13T08:43:16.802170+0000","last_unstale":"2024-09-13T08:43:17.790695+0000","last_undegraded":"2024-09-13T08:43:17.790695+0000","last_fullsized":"2024-09-13T08:43:17.790695+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:19.694731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.663107+0000","last_change":"2024-09-13T08:43:15.784382+0000","last_active":"2024-09-13T08:43:18.663107+0000","last_peered":"2024-09-13T08:43:18.663107+0000","last_clean":"2024-09-13T08:43:18.663107+0000","last_became_active":"2024-09-13T08:43:15.784098+0000","last_became_peered":"2024-09-13T08:43:15.784098+0000","last_unstale":"2024-09-13T08:43:18.663107+0000","last_undegraded":"2024-09-13T08:43:18.663107+0000","last_fullsized":"2024-09-13T08:43:18.663107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:34.681324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.795050+0000","last_change":"2024-09-13T08:43:16.808006+0000","last_active":"2024-09-13T08:43:18.795050+0000","last_peered":"2024-09-13T08:43:18.795050+0000","last_clean":"2024-09-13T08:43:18.795050+0000","last_became_active":"2024-09-13T08:43:16.807637+0000","last_became_peered":"2024-09-13T08:43:16.807637+0000","last_unstale":"2024-09-13T08:43:18.795050+0000","last_undegraded":"2024-09-13T08:43:18.795050+0000","last_fullsized":"2024-09-13T08:43:18.795050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:16.973803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861325+0000","last_change":"2024-09-13T08:43:15.795287+0000","last_active":"2024-09-13T08:43:18.861325+0000","last_peered":"2024-09-13T08:43:18.861325+0000","last_clean":"2024-09-13T08:43:18.861325+0000","last_became_active":"2024-09-13T08:43:15.794985+0000","last_became_peered":"2024-09-13T08:43:15.794985+0000","last_unstale":"2024-09-13T08:43:18.861325+0000","last_undegraded":"2024-09-13T08:43:18.861325+0000","last_fullsized":"2024-09-13T08:43:18.861325+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:53.611470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.795085+0000","last_change":"2024-09-13T08:43:16.823426+0000","last_active":"2024-09-13T08:43:18.795085+0000","last_peered":"2024-09-13T08:43:18.795085+0000","last_clean":"2024-09-13T08:43:18.795085+0000","last_became_active":"2024-09-13T08:43:16.823172+0000","last_became_peered":"2024-09-13T08:43:16.823172+0000","last_unstale":"2024-09-13T08:43:18.795085+0000","last_undegraded":"2024-09-13T08:43:18.795085+0000","last_fullsized":"2024-09-13T08:43:18.795085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:32.141439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.818461+0000","last_change":"2024-09-13T08:43:15.799101+0000","last_active":"2024-09-13T08:43:16.818461+0000","last_peered":"2024-09-13T08:43:16.818461+0000","last_clean":"2024-09-13T08:43:16.818461+0000","last_became_active":"2024-09-13T08:43:15.798891+0000","last_became_peered":"2024-09-13T08:43:15.798891+0000","last_unstale":"2024-09-13T08:43:16.818461+0000","last_undegraded":"2024-09-13T08:43:16.818461+0000","last_fullsized":"2024-09-13T08:43:16.818461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:15.743527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860903+0000","last_change":"2024-09-13T08:43:16.815521+0000","last_active":"2024-09-13T08:43:18.860903+0000","last_peered":"2024-09-13T08:43:18.860903+0000","last_clean":"2024-09-13T08:43:18.860903+0000","last_became_active":"2024-09-13T08:43:16.814348+0000","last_became_peered":"2024-09-13T08:43:16.814348+0000","last_unstale":"2024-09-13T08:43:18.860903+0000","last_undegraded":"2024-09-13T08:43:18.860903+0000","last_fullsized":"2024-09-13T08:43:18.860903+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:18.835674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662810+0000","last_change":"2024-09-13T08:43:15.798380+0000","last_active":"2024-09-13T08:43:18.662810+0000","last_peered":"2024-09-13T08:43:18.662810+0000","last_clean":"2024-09-13T08:43:18.662810+0000","last_became_active":"2024-09-13T08:43:15.798062+0000","last_became_peered":"2024-09-13T08:43:15.798062+0000","last_unstale":"2024-09-13T08:43:18.662810+0000","last_undegraded":"2024-09-13T08:43:18.662810+0000","last_fullsized":"2024-09-13T08:43:18.662810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:24.902615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791774+0000","last_change":"2024-09-13T08:43:16.803663+0000","last_active":"2024-09-13T08:43:17.791774+0000","last_peered":"2024-09-13T08:43:17.791774+0000","last_clean":"2024-09-13T08:43:17.791774+0000","last_became_active":"2024-09-13T08:43:16.802142+0000","last_became_peered":"2024-09-13T08:43:16.802142+0000","last_unstale":"2024-09-13T08:43:17.791774+0000","last_undegraded":"2024-09-13T08:43:17.791774+0000","last_fullsized":"2024-09-13T08:43:17.791774+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:59.529940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794629+0000","last_change":"2024-09-13T08:43:15.794144+0000","last_active":"2024-09-13T08:43:18.794629+0000","last_peered":"2024-09-13T08:43:18.794629+0000","last_clean":"2024-09-13T08:43:18.794629+0000","last_became_active":"2024-09-13T08:43:15.793769+0000","last_became_peered":"2024-09-13T08:43:15.793769+0000","last_unstale":"2024-09-13T08:43:18.794629+0000","last_undegraded":"2024-09-13T08:43:18.794629+0000","last_fullsized":"2024-09-13T08:43:18.794629+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:06.717335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.811998+0000","last_change":"2024-09-13T08:43:16.802991+0000","last_active":"2024-09-13T08:43:16.811998+0000","last_peered":"2024-09-13T08:43:16.811998+0000","last_clean":"2024-09-13T08:43:16.811998+0000","last_became_active":"2024-09-13T08:43:16.802719+0000","last_became_peered":"2024-09-13T08:43:16.802719+0000","last_unstale":"2024-09-13T08:43:16.811998+0000","last_undegraded":"2024-09-13T08:43:16.811998+0000","last_fullsized":"2024-09-13T08:43:16.811998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:37.786564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860730+0000","last_change":"2024-09-13T08:43:15.795297+0000","last_active":"2024-09-13T08:43:18.860730+0000","last_peered":"2024-09-13T08:43:18.860730+0000","last_clean":"2024-09-13T08:43:18.860730+0000","last_became_active":"2024-09-13T08:43:15.795009+0000","last_became_peered":"2024-09-13T08:43:15.795009+0000","last_unstale":"2024-09-13T08:43:18.860730+0000","last_undegraded":"2024-09-13T08:43:18.860730+0000","last_fullsized":"2024-09-13T08:43:18.860730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:12.431789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860729+0000","last_change":"2024-09-13T08:43:16.799262+0000","last_active":"2024-09-13T08:43:18.860729+0000","last_peered":"2024-09-13T08:43:18.860729+0000","last_clean":"2024-09-13T08:43:18.860729+0000","last_became_active":"2024-09-13T08:43:16.798985+0000","last_became_peered":"2024-09-13T08:43:16.798985+0000","last_unstale":"2024-09-13T08:43:18.860729+0000","last_undegraded":"2024-09-13T08:43:18.860729+0000","last_fullsized":"2024-09-13T08:43:18.860729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:05.875443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794360+0000","last_change":"2024-09-13T08:43:15.782349+0000","last_active":"2024-09-13T08:43:18.794360+0000","last_peered":"2024-09-13T08:43:18.794360+0000","last_clean":"2024-09-13T08:43:18.794360+0000","last_became_active":"2024-09-13T08:43:15.782098+0000","last_became_peered":"2024-09-13T08:43:15.782098+0000","last_unstale":"2024-09-13T08:43:18.794360+0000","last_undegraded":"2024-09-13T08:43:18.794360+0000","last_fullsized":"2024-09-13T08:43:18.794360+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:23.057200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.792236+0000","last_change":"2024-09-13T08:43:16.809938+0000","last_active":"2024-09-13T08:43:17.792236+0000","last_peered":"2024-09-13T08:43:17.792236+0000","last_clean":"2024-09-13T08:43:17.792236+0000","last_became_active":"2024-09-13T08:43:16.809699+0000","last_became_peered":"2024-09-13T08:43:16.809699+0000","last_unstale":"2024-09-13T08:43:17.792236+0000","last_undegraded":"2024-09-13T08:43:17.792236+0000","last_fullsized":"2024-09-13T08:43:17.792236+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:46:12.249470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.809427+0000","last_change":"2024-09-13T08:43:15.785423+0000","last_active":"2024-09-13T08:43:16.809427+0000","last_peered":"2024-09-13T08:43:16.809427+0000","last_clean":"2024-09-13T08:43:16.809427+0000","last_became_active":"2024-09-13T08:43:15.784926+0000","last_became_peered":"2024-09-13T08:43:15.784926+0000","last_unstale":"2024-09-13T08:43:16.809427+0000","last_undegraded":"2024-09-13T08:43:16.809427+0000","last_fullsized":"2024-09-13T08:43:16.809427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:46.548513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860918+0000","last_change":"2024-09-13T08:43:16.806772+0000","last_active":"2024-09-13T08:43:18.860918+0000","last_peered":"2024-09-13T08:43:18.860918+0000","last_clean":"2024-09-13T08:43:18.860918+0000","last_became_active":"2024-09-13T08:43:16.806371+0000","last_became_peered":"2024-09-13T08:43:16.806371+0000","last_unstale":"2024-09-13T08:43:18.860918+0000","last_undegraded":"2024-09-13T08:43:18.860918+0000","last_fullsized":"2024-09-13T08:43:18.860918+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:27:50.131143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.792310+0000","last_change":"2024-09-13T08:43:15.791213+0000","last_active":"2024-09-13T08:43:17.792310+0000","last_peered":"2024-09-13T08:43:17.792310+0000","last_clean":"2024-09-13T08:43:17.792310+0000","last_became_active":"2024-09-13T08:43:15.790898+0000","last_became_peered":"2024-09-13T08:43:15.790898+0000","last_unstale":"2024-09-13T08:43:17.792310+0000","last_undegraded":"2024-09-13T08:43:17.792310+0000","last_fullsized":"2024-09-13T08:43:17.792310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:07:43.245534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.792452+0000","last_change":"2024-09-13T08:43:16.806147+0000","last_active":"2024-09-13T08:43:17.792452+0000","last_peered":"2024-09-13T08:43:17.792452+0000","last_clean":"2024-09-13T08:43:17.792452+0000","last_became_active":"2024-09-13T08:43:16.805854+0000","last_became_peered":"2024-09-13T08:43:16.805854+0000","last_unstale":"2024-09-13T08:43:17.792452+0000","last_undegraded":"2024-09-13T08:43:17.792452+0000","last_fullsized":"2024-09-13T08:43:17.792452+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:46:13.648919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860251+0000","last_change":"2024-09-13T08:43:15.794923+0000","last_active":"2024-09-13T08:43:18.860251+0000","last_peered":"2024-09-13T08:43:18.860251+0000","last_clean":"2024-09-13T08:43:18.860251+0000","last_became_active":"2024-09-13T08:43:15.794651+0000","last_became_peered":"2024-09-13T08:43:15.794651+0000","last_unstale":"2024-09-13T08:43:18.860251+0000","last_undegraded":"2024-09-13T08:43:18.860251+0000","last_fullsized":"2024-09-13T08:43:18.860251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:23.057200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":185,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662358+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:43:18.662358+0000","last_peered":"2024-09-13T08:43:18.662358+0000","last_clean":"2024-09-13T08:43:18.662358+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:43:18.662358+0000","last_undegraded":"2024-09-13T08:43:18.662358+0000","last_fullsized":"2024-09-13T08:43:18.662358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40220,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331620,"statfs":{"total":96636764160,"available":96595578880,"internally_reserved":0,"allocated":720896,"data_stored":296958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509513,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46372,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325468,"statfs":{"total":96636764160,"available":96589279232,"internally_reserved":0,"allocated":1323008,"data_stored":893596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509514,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45800,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326040,"statfs":{"total":96636764160,"available":96589864960,"internally_reserved":0,"allocated":733184,"data_stored":303810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509514,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40228,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331612,"statfs":{"total":96636764160,"available":96595570688,"internally_reserved":0,"allocated":729088,"data_stored":303284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509513,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62440,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309400,"statfs":{"total":96636764160,"available":96572825600,"internally_reserved":0,"allocated":733184,"data_stored":298638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509515,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40796,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331044,"statfs":{"total":96636764160,"available":96594989056,"internally_reserved":0,"allocated":1310720,"data_stored":887816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40224,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331616,"statfs":{"total":96636764160,"available":96595574784,"internally_reserved":0,"allocated":724992,"data_stored":298478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542219,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58264,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313576,"statfs":{"total":96636764160,"available":96577101824,"internally_reserved":0,"allocated":716800,"data_stored":296888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:43:20.971 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:43:21.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.155+0000 7effc1600640 1 -- 172.21.15.44:0/2258422360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effbc15e330 msgr2=0x7effbc0b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:21.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.155+0000 7effc1600640 1 --2- 172.21.15.44:0/2258422360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effbc15e330 0x7effbc0b0210 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7effbc161c00 tx=0x7effb401ccf0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.155+0000 7effc1600640 1 -- 172.21.15.44:0/2258422360 shutdown_connections 2024-09-13T08:43:21.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.155+0000 7effc1600640 1 --2- 172.21.15.44:0/2258422360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effbc15e330 0x7effbc0b0210 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.155+0000 7effc1600640 1 --2- 172.21.15.44:0/2258422360 >> v2:172.21.15.129:3300/0 conn(0x7effbc06ee50 0x7effbc0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.155+0000 7effc1600640 1 -- 172.21.15.44:0/2258422360 >> 172.21.15.44:0/2258422360 conn(0x7effbc0d1dc0 msgr2=0x7effbc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:21.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.155+0000 7effc1600640 1 -- 172.21.15.44:0/2258422360 shutdown_connections 2024-09-13T08:43:21.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.155+0000 7effc1600640 1 -- 172.21.15.44:0/2258422360 wait complete. 2024-09-13T08:43:21.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effc1600640 1 Processor -- start 2024-09-13T08:43:21.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effc1600640 1 -- start start 2024-09-13T08:43:21.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effc1600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7effbc06ee50 0x7effbc0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:21.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effc1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effbc15df50 0x7effbc156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effc1600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effbc15e330 0x7effbc156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effc1600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0d0fd0 con 0x7effbc15df50 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effc1600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0d0e50 con 0x7effbc15e330 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effc1600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7effbc0d1150 con 0x7effbc06ee50 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effbc15df50 0x7effbc156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effbc15df50 0x7effbc156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46846/0 (socket says 172.21.15.44:46846) 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb2000640 1 -- 172.21.15.44:0/1689279130 learned_addr learned my addr 172.21.15.44:0/1689279130 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effbaa00640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7effbc06ee50 0x7effbc0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effbb400640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effbc15e330 0x7effbc156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb2000640 1 -- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7effbc06ee50 msgr2=0x7effbc0c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb2000640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7effbc06ee50 0x7effbc0c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb2000640 1 -- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effbc15e330 msgr2=0x7effbc156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb2000640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effbc15e330 0x7effbc156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb2000640 1 -- 172.21.15.44:0/1689279130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc156a50 con 0x7effbc15df50 2024-09-13T08:43:21.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb2000640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effbc15df50 0x7effbc156510 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7effa400d980 tx=0x7effa400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:21.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb3400640 1 -- 172.21.15.44:0/1689279130 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4017070 con 0x7effbc15df50 2024-09-13T08:43:21.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb3400640 1 -- 172.21.15.44:0/1689279130 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effa4009d60 con 0x7effbc15df50 2024-09-13T08:43:21.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effc1600640 1 -- 172.21.15.44:0/1689279130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc09d180 con 0x7effbc15df50 2024-09-13T08:43:21.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effb3400640 1 -- 172.21.15.44:0/1689279130 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4005280 con 0x7effbc15df50 2024-09-13T08:43:21.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.159+0000 7effc1600640 1 -- 172.21.15.44:0/1689279130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc0c7cf0 con 0x7effbc15df50 2024-09-13T08:43:21.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.163+0000 7effc1600640 1 -- 172.21.15.44:0/1689279130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effbc09d350 con 0x7effbc15df50 2024-09-13T08:43:21.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.163+0000 7effb3400640 1 -- 172.21.15.44:0/1689279130 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7effa4005420 con 0x7effbc15df50 2024-09-13T08:43:21.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.163+0000 7effb3400640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff980737c0 0x7eff98075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:21.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.163+0000 7effb3400640 1 -- 172.21.15.44:0/1689279130 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7effa40928b0 con 0x7effbc15df50 2024-09-13T08:43:21.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.163+0000 7effbaa00640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff980737c0 0x7eff98075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:21.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.163+0000 7effbaa00640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff980737c0 0x7eff98075c80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7effac007fd0 tx=0x7effac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:21.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.167+0000 7effb3400640 1 -- 172.21.15.44:0/1689279130 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7effbc156a50 con 0x7effbc15df50 2024-09-13T08:43:21.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.339+0000 7effc1600640 1 -- 172.21.15.44:0/1689279130 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7effbc071430 con 0x7eff980737c0 2024-09-13T08:43:21.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.347+0000 7effb3400640 1 -- 172.21.15.44:0/1689279130 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7effbc071430 con 0x7eff980737c0 2024-09-13T08:43:21.353 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:21.354 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:43:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 -- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff980737c0 msgr2=0x7eff98075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff980737c0 0x7eff98075c80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7effac007fd0 tx=0x7effac009040 comp rx=0 tx=0).stop 2024-09-13T08:43:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 -- 172.21.15.44:0/1689279130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effbc15df50 msgr2=0x7effbc156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effbc15df50 0x7effbc156510 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7effa400d980 tx=0x7effa400de40 comp rx=0 tx=0).stop 2024-09-13T08:43:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 -- 172.21.15.44:0/1689279130 shutdown_connections 2024-09-13T08:43:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff980737c0 0x7eff98075c80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effbc15e330 0x7effbc156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effbc15df50 0x7effbc156510 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 --2- 172.21.15.44:0/1689279130 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7effbc06ee50 0x7effbc0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 -- 172.21.15.44:0/1689279130 >> 172.21.15.44:0/1689279130 conn(0x7effbc0d1dc0 msgr2=0x7effbc071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 -- 172.21.15.44:0/1689279130 shutdown_connections 2024-09-13T08:43:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.351+0000 7effc1600640 1 -- 172.21.15.44:0/1689279130 wait complete. 2024-09-13T08:43:21.374 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-13T08:43:20.604967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":374344,"kb_used_data":6828,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754600376,"statfs":{"total":773094113280,"available":772710785024,"internally_reserved":0,"allocated":6991872,"data_stored":3579468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":376098353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.840529"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.818943+0000","last_change":"2024-09-13T08:43:16.817882+0000","last_active":"2024-09-13T08:43:16.818943+0000","last_peered":"2024-09-13T08:43:16.818943+0000","last_clean":"2024-09-13T08:43:16.818943+0000","last_became_active":"2024-09-13T08:43:16.817105+0000","last_became_peered":"2024-09-13T08:43:16.817105+0000","last_unstale":"2024-09-13T08:43:16.818943+0000","last_undegraded":"2024-09-13T08:43:16.818943+0000","last_fullsized":"2024-09-13T08:43:16.818943+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:56:37.249907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861394+0000","last_change":"2024-09-13T08:43:16.818673+0000","last_active":"2024-09-13T08:43:18.861394+0000","last_peered":"2024-09-13T08:43:18.861394+0000","last_clean":"2024-09-13T08:43:18.861394+0000","last_became_active":"2024-09-13T08:43:16.818278+0000","last_became_peered":"2024-09-13T08:43:16.818278+0000","last_unstale":"2024-09-13T08:43:18.861394+0000","last_undegraded":"2024-09-13T08:43:18.861394+0000","last_fullsized":"2024-09-13T08:43:18.861394+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:00.403728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794916+0000","last_change":"2024-09-13T08:43:16.813551+0000","last_active":"2024-09-13T08:43:18.794916+0000","last_peered":"2024-09-13T08:43:18.794916+0000","last_clean":"2024-09-13T08:43:18.794916+0000","last_became_active":"2024-09-13T08:43:16.812940+0000","last_became_peered":"2024-09-13T08:43:16.812940+0000","last_unstale":"2024-09-13T08:43:18.794916+0000","last_undegraded":"2024-09-13T08:43:18.794916+0000","last_fullsized":"2024-09-13T08:43:18.794916+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:48.617459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791472+0000","last_change":"2024-09-13T08:43:16.819995+0000","last_active":"2024-09-13T08:43:17.791472+0000","last_peered":"2024-09-13T08:43:17.791472+0000","last_clean":"2024-09-13T08:43:17.791472+0000","last_became_active":"2024-09-13T08:43:16.818190+0000","last_became_peered":"2024-09-13T08:43:16.818190+0000","last_unstale":"2024-09-13T08:43:17.791472+0000","last_undegraded":"2024-09-13T08:43:17.791472+0000","last_fullsized":"2024-09-13T08:43:17.791472+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:27:56.292419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.816335+0000","last_change":"2024-09-13T08:43:16.808958+0000","last_active":"2024-09-13T08:43:16.816335+0000","last_peered":"2024-09-13T08:43:16.816335+0000","last_clean":"2024-09-13T08:43:16.816335+0000","last_became_active":"2024-09-13T08:43:16.808685+0000","last_became_peered":"2024-09-13T08:43:16.808685+0000","last_unstale":"2024-09-13T08:43:16.816335+0000","last_undegraded":"2024-09-13T08:43:16.816335+0000","last_fullsized":"2024-09-13T08:43:16.816335+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:04:09.314912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.817862+0000","last_change":"2024-09-13T08:43:16.813201+0000","last_active":"2024-09-13T08:43:16.817862+0000","last_peered":"2024-09-13T08:43:16.817862+0000","last_clean":"2024-09-13T08:43:16.817862+0000","last_became_active":"2024-09-13T08:43:16.812933+0000","last_became_peered":"2024-09-13T08:43:16.812933+0000","last_unstale":"2024-09-13T08:43:16.817862+0000","last_undegraded":"2024-09-13T08:43:16.817862+0000","last_fullsized":"2024-09-13T08:43:16.817862+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:22.975460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791108+0000","last_change":"2024-09-13T08:43:16.818209+0000","last_active":"2024-09-13T08:43:17.791108+0000","last_peered":"2024-09-13T08:43:17.791108+0000","last_clean":"2024-09-13T08:43:17.791108+0000","last_became_active":"2024-09-13T08:43:16.817923+0000","last_became_peered":"2024-09-13T08:43:16.817923+0000","last_unstale":"2024-09-13T08:43:17.791108+0000","last_undegraded":"2024-09-13T08:43:17.791108+0000","last_fullsized":"2024-09-13T08:43:17.791108+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:37.310222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791999+0000","last_change":"2024-09-13T08:43:16.809360+0000","last_active":"2024-09-13T08:43:17.791999+0000","last_peered":"2024-09-13T08:43:17.791999+0000","last_clean":"2024-09-13T08:43:17.791999+0000","last_became_active":"2024-09-13T08:43:16.808345+0000","last_became_peered":"2024-09-13T08:43:16.808345+0000","last_unstale":"2024-09-13T08:43:17.791999+0000","last_undegraded":"2024-09-13T08:43:17.791999+0000","last_fullsized":"2024-09-13T08:43:17.791999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:55.030189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.663000+0000","last_change":"2024-09-13T08:43:16.822144+0000","last_active":"2024-09-13T08:43:18.663000+0000","last_peered":"2024-09-13T08:43:18.663000+0000","last_clean":"2024-09-13T08:43:18.663000+0000","last_became_active":"2024-09-13T08:43:16.821864+0000","last_became_peered":"2024-09-13T08:43:16.821864+0000","last_unstale":"2024-09-13T08:43:18.663000+0000","last_undegraded":"2024-09-13T08:43:18.663000+0000","last_fullsized":"2024-09-13T08:43:18.663000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:38.477323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.829154+0000","last_change":"2024-09-13T08:43:16.827402+0000","last_active":"2024-09-13T08:43:16.829154+0000","last_peered":"2024-09-13T08:43:16.829154+0000","last_clean":"2024-09-13T08:43:16.829154+0000","last_became_active":"2024-09-13T08:43:16.827089+0000","last_became_peered":"2024-09-13T08:43:16.827089+0000","last_unstale":"2024-09-13T08:43:16.829154+0000","last_undegraded":"2024-09-13T08:43:16.829154+0000","last_fullsized":"2024-09-13T08:43:16.829154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:14:16.051866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791576+0000","last_change":"2024-09-13T08:43:16.803299+0000","last_active":"2024-09-13T08:43:17.791576+0000","last_peered":"2024-09-13T08:43:17.791576+0000","last_clean":"2024-09-13T08:43:17.791576+0000","last_became_active":"2024-09-13T08:43:16.803009+0000","last_became_peered":"2024-09-13T08:43:16.803009+0000","last_unstale":"2024-09-13T08:43:17.791576+0000","last_undegraded":"2024-09-13T08:43:17.791576+0000","last_fullsized":"2024-09-13T08:43:17.791576+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:22.005050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662774+0000","last_change":"2024-09-13T08:43:16.812577+0000","last_active":"2024-09-13T08:43:18.662774+0000","last_peered":"2024-09-13T08:43:18.662774+0000","last_clean":"2024-09-13T08:43:18.662774+0000","last_became_active":"2024-09-13T08:43:16.812313+0000","last_became_peered":"2024-09-13T08:43:16.812313+0000","last_unstale":"2024-09-13T08:43:18.662774+0000","last_undegraded":"2024-09-13T08:43:18.662774+0000","last_fullsized":"2024-09-13T08:43:18.662774+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:04:13.261593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860593+0000","last_change":"2024-09-13T08:43:16.817172+0000","last_active":"2024-09-13T08:43:18.860593+0000","last_peered":"2024-09-13T08:43:18.860593+0000","last_clean":"2024-09-13T08:43:18.860593+0000","last_became_active":"2024-09-13T08:43:16.816869+0000","last_became_peered":"2024-09-13T08:43:16.816869+0000","last_unstale":"2024-09-13T08:43:18.860593+0000","last_undegraded":"2024-09-13T08:43:18.860593+0000","last_fullsized":"2024-09-13T08:43:18.860593+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:08.354996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794463+0000","last_change":"2024-09-13T08:43:16.806479+0000","last_active":"2024-09-13T08:43:18.794463+0000","last_peered":"2024-09-13T08:43:18.794463+0000","last_clean":"2024-09-13T08:43:18.794463+0000","last_became_active":"2024-09-13T08:43:16.806208+0000","last_became_peered":"2024-09-13T08:43:16.806208+0000","last_unstale":"2024-09-13T08:43:18.794463+0000","last_undegraded":"2024-09-13T08:43:18.794463+0000","last_fullsized":"2024-09-13T08:43:18.794463+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:13.700936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860240+0000","last_change":"2024-09-13T08:43:16.817320+0000","last_active":"2024-09-13T08:43:18.860240+0000","last_peered":"2024-09-13T08:43:18.860240+0000","last_clean":"2024-09-13T08:43:18.860240+0000","last_became_active":"2024-09-13T08:43:16.816380+0000","last_became_peered":"2024-09-13T08:43:16.816380+0000","last_unstale":"2024-09-13T08:43:18.860240+0000","last_undegraded":"2024-09-13T08:43:18.860240+0000","last_fullsized":"2024-09-13T08:43:18.860240+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:51:59.644256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.819706+0000","last_change":"2024-09-13T08:43:16.815202+0000","last_active":"2024-09-13T08:43:16.819706+0000","last_peered":"2024-09-13T08:43:16.819706+0000","last_clean":"2024-09-13T08:43:16.819706+0000","last_became_active":"2024-09-13T08:43:16.813944+0000","last_became_peered":"2024-09-13T08:43:16.813944+0000","last_unstale":"2024-09-13T08:43:16.819706+0000","last_undegraded":"2024-09-13T08:43:16.819706+0000","last_fullsized":"2024-09-13T08:43:16.819706+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:17.821216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.818966+0000","last_change":"2024-09-13T08:43:16.817920+0000","last_active":"2024-09-13T08:43:16.818966+0000","last_peered":"2024-09-13T08:43:16.818966+0000","last_clean":"2024-09-13T08:43:16.818966+0000","last_became_active":"2024-09-13T08:43:16.817609+0000","last_became_peered":"2024-09-13T08:43:16.817609+0000","last_unstale":"2024-09-13T08:43:16.818966+0000","last_undegraded":"2024-09-13T08:43:16.818966+0000","last_fullsized":"2024-09-13T08:43:16.818966+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:21.956413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794771+0000","last_change":"2024-09-13T08:43:16.809067+0000","last_active":"2024-09-13T08:43:18.794771+0000","last_peered":"2024-09-13T08:43:18.794771+0000","last_clean":"2024-09-13T08:43:18.794771+0000","last_became_active":"2024-09-13T08:43:16.808770+0000","last_became_peered":"2024-09-13T08:43:16.808770+0000","last_unstale":"2024-09-13T08:43:18.794771+0000","last_undegraded":"2024-09-13T08:43:18.794771+0000","last_fullsized":"2024-09-13T08:43:18.794771+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:06:51.560999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861119+0000","last_change":"2024-09-13T08:43:16.815008+0000","last_active":"2024-09-13T08:43:18.861119+0000","last_peered":"2024-09-13T08:43:18.861119+0000","last_clean":"2024-09-13T08:43:18.861119+0000","last_became_active":"2024-09-13T08:43:16.814780+0000","last_became_peered":"2024-09-13T08:43:16.814780+0000","last_unstale":"2024-09-13T08:43:18.861119+0000","last_undegraded":"2024-09-13T08:43:18.861119+0000","last_fullsized":"2024-09-13T08:43:18.861119+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:23:52.086003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791307+0000","last_change":"2024-09-13T08:43:16.818745+0000","last_active":"2024-09-13T08:43:17.791307+0000","last_peered":"2024-09-13T08:43:17.791307+0000","last_clean":"2024-09-13T08:43:17.791307+0000","last_became_active":"2024-09-13T08:43:16.812424+0000","last_became_peered":"2024-09-13T08:43:16.812424+0000","last_unstale":"2024-09-13T08:43:17.791307+0000","last_undegraded":"2024-09-13T08:43:17.791307+0000","last_fullsized":"2024-09-13T08:43:17.791307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:28:12.706783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860518+0000","last_change":"2024-09-13T08:43:16.810671+0000","last_active":"2024-09-13T08:43:18.860518+0000","last_peered":"2024-09-13T08:43:18.860518+0000","last_clean":"2024-09-13T08:43:18.860518+0000","last_became_active":"2024-09-13T08:43:16.810290+0000","last_became_peered":"2024-09-13T08:43:16.810290+0000","last_unstale":"2024-09-13T08:43:18.860518+0000","last_undegraded":"2024-09-13T08:43:18.860518+0000","last_fullsized":"2024-09-13T08:43:18.860518+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:08.673621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791257+0000","last_change":"2024-09-13T08:43:16.810346+0000","last_active":"2024-09-13T08:43:17.791257+0000","last_peered":"2024-09-13T08:43:17.791257+0000","last_clean":"2024-09-13T08:43:17.791257+0000","last_became_active":"2024-09-13T08:43:16.808958+0000","last_became_peered":"2024-09-13T08:43:16.808958+0000","last_unstale":"2024-09-13T08:43:17.791257+0000","last_undegraded":"2024-09-13T08:43:17.791257+0000","last_fullsized":"2024-09-13T08:43:17.791257+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:50.393357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791175+0000","last_change":"2024-09-13T08:43:16.815624+0000","last_active":"2024-09-13T08:43:17.791175+0000","last_peered":"2024-09-13T08:43:17.791175+0000","last_clean":"2024-09-13T08:43:17.791175+0000","last_became_active":"2024-09-13T08:43:16.815388+0000","last_became_peered":"2024-09-13T08:43:16.815388+0000","last_unstale":"2024-09-13T08:43:17.791175+0000","last_undegraded":"2024-09-13T08:43:17.791175+0000","last_fullsized":"2024-09-13T08:43:17.791175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:24:19.128614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.663090+0000","last_change":"2024-09-13T08:43:16.822958+0000","last_active":"2024-09-13T08:43:18.663090+0000","last_peered":"2024-09-13T08:43:18.663090+0000","last_clean":"2024-09-13T08:43:18.663090+0000","last_became_active":"2024-09-13T08:43:16.822701+0000","last_became_peered":"2024-09-13T08:43:16.822701+0000","last_unstale":"2024-09-13T08:43:18.663090+0000","last_undegraded":"2024-09-13T08:43:18.663090+0000","last_fullsized":"2024-09-13T08:43:18.663090+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:18.835674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861393+0000","last_change":"2024-09-13T08:43:16.818229+0000","last_active":"2024-09-13T08:43:18.861393+0000","last_peered":"2024-09-13T08:43:18.861393+0000","last_clean":"2024-09-13T08:43:18.861393+0000","last_became_active":"2024-09-13T08:43:16.817978+0000","last_became_peered":"2024-09-13T08:43:16.817978+0000","last_unstale":"2024-09-13T08:43:18.861393+0000","last_undegraded":"2024-09-13T08:43:18.861393+0000","last_fullsized":"2024-09-13T08:43:18.861393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:24.066893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860721+0000","last_change":"2024-09-13T08:43:16.807144+0000","last_active":"2024-09-13T08:43:18.860721+0000","last_peered":"2024-09-13T08:43:18.860721+0000","last_clean":"2024-09-13T08:43:18.860721+0000","last_became_active":"2024-09-13T08:43:16.806751+0000","last_became_peered":"2024-09-13T08:43:16.806751+0000","last_unstale":"2024-09-13T08:43:18.860721+0000","last_undegraded":"2024-09-13T08:43:18.860721+0000","last_fullsized":"2024-09-13T08:43:18.860721+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:43.503649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861285+0000","last_change":"2024-09-13T08:43:16.824234+0000","last_active":"2024-09-13T08:43:18.861285+0000","last_peered":"2024-09-13T08:43:18.861285+0000","last_clean":"2024-09-13T08:43:18.861285+0000","last_became_active":"2024-09-13T08:43:16.822388+0000","last_became_peered":"2024-09-13T08:43:16.822388+0000","last_unstale":"2024-09-13T08:43:18.861285+0000","last_undegraded":"2024-09-13T08:43:18.861285+0000","last_fullsized":"2024-09-13T08:43:18.861285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:10:45.208508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791528+0000","last_change":"2024-09-13T08:43:16.815164+0000","last_active":"2024-09-13T08:43:17.791528+0000","last_peered":"2024-09-13T08:43:17.791528+0000","last_clean":"2024-09-13T08:43:17.791528+0000","last_became_active":"2024-09-13T08:43:16.814905+0000","last_became_peered":"2024-09-13T08:43:16.814905+0000","last_unstale":"2024-09-13T08:43:17.791528+0000","last_undegraded":"2024-09-13T08:43:17.791528+0000","last_fullsized":"2024-09-13T08:43:17.791528+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:17.220615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790840+0000","last_change":"2024-09-13T08:43:16.810533+0000","last_active":"2024-09-13T08:43:17.790840+0000","last_peered":"2024-09-13T08:43:17.790840+0000","last_clean":"2024-09-13T08:43:17.790840+0000","last_became_active":"2024-09-13T08:43:16.810186+0000","last_became_peered":"2024-09-13T08:43:16.810186+0000","last_unstale":"2024-09-13T08:43:17.790840+0000","last_undegraded":"2024-09-13T08:43:17.790840+0000","last_fullsized":"2024-09-13T08:43:17.790840+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:49:43.333978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791138+0000","last_change":"2024-09-13T08:43:16.813821+0000","last_active":"2024-09-13T08:43:17.791138+0000","last_peered":"2024-09-13T08:43:17.791138+0000","last_clean":"2024-09-13T08:43:17.791138+0000","last_became_active":"2024-09-13T08:43:16.813564+0000","last_became_peered":"2024-09-13T08:43:16.813564+0000","last_unstale":"2024-09-13T08:43:17.791138+0000","last_undegraded":"2024-09-13T08:43:17.791138+0000","last_fullsized":"2024-09-13T08:43:17.791138+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:08:00.544785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860832+0000","last_change":"2024-09-13T08:43:16.808390+0000","last_active":"2024-09-13T08:43:18.860832+0000","last_peered":"2024-09-13T08:43:18.860832+0000","last_clean":"2024-09-13T08:43:18.860832+0000","last_became_active":"2024-09-13T08:43:16.808078+0000","last_became_peered":"2024-09-13T08:43:16.808078+0000","last_unstale":"2024-09-13T08:43:18.860832+0000","last_undegraded":"2024-09-13T08:43:18.860832+0000","last_fullsized":"2024-09-13T08:43:18.860832+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:48.635890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790519+0000","last_change":"2024-09-13T08:43:16.805571+0000","last_active":"2024-09-13T08:43:17.790519+0000","last_peered":"2024-09-13T08:43:17.790519+0000","last_clean":"2024-09-13T08:43:17.790519+0000","last_became_active":"2024-09-13T08:43:16.805209+0000","last_became_peered":"2024-09-13T08:43:16.805209+0000","last_unstale":"2024-09-13T08:43:17.790519+0000","last_undegraded":"2024-09-13T08:43:17.790519+0000","last_fullsized":"2024-09-13T08:43:17.790519+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:50.997778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.818642+0000","last_change":"2024-09-13T08:43:16.810204+0000","last_active":"2024-09-13T08:43:16.818642+0000","last_peered":"2024-09-13T08:43:16.818642+0000","last_clean":"2024-09-13T08:43:16.818642+0000","last_became_active":"2024-09-13T08:43:16.809948+0000","last_became_peered":"2024-09-13T08:43:16.809948+0000","last_unstale":"2024-09-13T08:43:16.818642+0000","last_undegraded":"2024-09-13T08:43:16.818642+0000","last_fullsized":"2024-09-13T08:43:16.818642+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:14:54.111787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"60'4","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791001+0000","last_change":"2024-09-13T08:43:15.786799+0000","last_active":"2024-09-13T08:43:17.791001+0000","last_peered":"2024-09-13T08:43:17.791001+0000","last_clean":"2024-09-13T08:43:17.791001+0000","last_became_active":"2024-09-13T08:43:15.786517+0000","last_became_peered":"2024-09-13T08:43:15.786517+0000","last_unstale":"2024-09-13T08:43:17.791001+0000","last_undegraded":"2024-09-13T08:43:17.791001+0000","last_fullsized":"2024-09-13T08:43:17.791001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:57.379125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.818177+0000","last_change":"2024-09-13T08:43:16.808004+0000","last_active":"2024-09-13T08:43:16.818177+0000","last_peered":"2024-09-13T08:43:16.818177+0000","last_clean":"2024-09-13T08:43:16.818177+0000","last_became_active":"2024-09-13T08:43:16.807740+0000","last_became_peered":"2024-09-13T08:43:16.807740+0000","last_unstale":"2024-09-13T08:43:16.818177+0000","last_undegraded":"2024-09-13T08:43:16.818177+0000","last_fullsized":"2024-09-13T08:43:16.818177+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:09.440313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861226+0000","last_change":"2024-09-13T08:43:15.800841+0000","last_active":"2024-09-13T08:43:18.861226+0000","last_peered":"2024-09-13T08:43:18.861226+0000","last_clean":"2024-09-13T08:43:18.861226+0000","last_became_active":"2024-09-13T08:43:15.800649+0000","last_became_peered":"2024-09-13T08:43:15.800649+0000","last_unstale":"2024-09-13T08:43:18.861226+0000","last_undegraded":"2024-09-13T08:43:18.861226+0000","last_fullsized":"2024-09-13T08:43:18.861226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:24.902615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791218+0000","last_change":"2024-09-13T08:43:16.818858+0000","last_active":"2024-09-13T08:43:17.791218+0000","last_peered":"2024-09-13T08:43:17.791218+0000","last_clean":"2024-09-13T08:43:17.791218+0000","last_became_active":"2024-09-13T08:43:16.818554+0000","last_became_peered":"2024-09-13T08:43:16.818554+0000","last_unstale":"2024-09-13T08:43:17.791218+0000","last_undegraded":"2024-09-13T08:43:17.791218+0000","last_fullsized":"2024-09-13T08:43:17.791218+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:30.063946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"60'2","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.828229+0000","last_change":"2024-09-13T08:43:15.793173+0000","last_active":"2024-09-13T08:43:16.828229+0000","last_peered":"2024-09-13T08:43:16.828229+0000","last_clean":"2024-09-13T08:43:16.828229+0000","last_became_active":"2024-09-13T08:43:15.792801+0000","last_became_peered":"2024-09-13T08:43:15.792801+0000","last_unstale":"2024-09-13T08:43:16.828229+0000","last_undegraded":"2024-09-13T08:43:16.828229+0000","last_fullsized":"2024-09-13T08:43:16.828229+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:25.488120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.819662+0000","last_change":"2024-09-13T08:43:16.809830+0000","last_active":"2024-09-13T08:43:16.819662+0000","last_peered":"2024-09-13T08:43:16.819662+0000","last_clean":"2024-09-13T08:43:16.819662+0000","last_became_active":"2024-09-13T08:43:16.809608+0000","last_became_peered":"2024-09-13T08:43:16.809608+0000","last_unstale":"2024-09-13T08:43:16.819662+0000","last_undegraded":"2024-09-13T08:43:16.819662+0000","last_fullsized":"2024-09-13T08:43:16.819662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:01:27.492994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791484+0000","last_change":"2024-09-13T08:43:15.790858+0000","last_active":"2024-09-13T08:43:17.791484+0000","last_peered":"2024-09-13T08:43:17.791484+0000","last_clean":"2024-09-13T08:43:17.791484+0000","last_became_active":"2024-09-13T08:43:15.790574+0000","last_became_peered":"2024-09-13T08:43:15.790574+0000","last_unstale":"2024-09-13T08:43:17.791484+0000","last_undegraded":"2024-09-13T08:43:17.791484+0000","last_fullsized":"2024-09-13T08:43:17.791484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:52.809472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790759+0000","last_change":"2024-09-13T08:43:16.795036+0000","last_active":"2024-09-13T08:43:17.790759+0000","last_peered":"2024-09-13T08:43:17.790759+0000","last_clean":"2024-09-13T08:43:17.790759+0000","last_became_active":"2024-09-13T08:43:16.794723+0000","last_became_peered":"2024-09-13T08:43:16.794723+0000","last_unstale":"2024-09-13T08:43:17.790759+0000","last_undegraded":"2024-09-13T08:43:17.790759+0000","last_fullsized":"2024-09-13T08:43:17.790759+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:24.811313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794662+0000","last_change":"2024-09-13T08:43:15.790947+0000","last_active":"2024-09-13T08:43:18.794662+0000","last_peered":"2024-09-13T08:43:18.794662+0000","last_clean":"2024-09-13T08:43:18.794662+0000","last_became_active":"2024-09-13T08:43:15.790165+0000","last_became_peered":"2024-09-13T08:43:15.790165+0000","last_unstale":"2024-09-13T08:43:18.794662+0000","last_undegraded":"2024-09-13T08:43:18.794662+0000","last_fullsized":"2024-09-13T08:43:18.794662+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:51:27.879550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790780+0000","last_change":"2024-09-13T08:43:16.809236+0000","last_active":"2024-09-13T08:43:17.790780+0000","last_peered":"2024-09-13T08:43:17.790780+0000","last_clean":"2024-09-13T08:43:17.790780+0000","last_became_active":"2024-09-13T08:43:16.808889+0000","last_became_peered":"2024-09-13T08:43:16.808889+0000","last_unstale":"2024-09-13T08:43:17.790780+0000","last_undegraded":"2024-09-13T08:43:17.790780+0000","last_fullsized":"2024-09-13T08:43:17.790780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:07:44.255227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"60'1","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.827511+0000","last_change":"2024-09-13T08:43:15.791105+0000","last_active":"2024-09-13T08:43:16.827511+0000","last_peered":"2024-09-13T08:43:16.827511+0000","last_clean":"2024-09-13T08:43:16.827511+0000","last_became_active":"2024-09-13T08:43:15.790803+0000","last_became_peered":"2024-09-13T08:43:15.790803+0000","last_unstale":"2024-09-13T08:43:16.827511+0000","last_undegraded":"2024-09-13T08:43:16.827511+0000","last_fullsized":"2024-09-13T08:43:16.827511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:49.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794393+0000","last_change":"2024-09-13T08:43:16.805584+0000","last_active":"2024-09-13T08:43:18.794393+0000","last_peered":"2024-09-13T08:43:18.794393+0000","last_clean":"2024-09-13T08:43:18.794393+0000","last_became_active":"2024-09-13T08:43:16.805259+0000","last_became_peered":"2024-09-13T08:43:16.805259+0000","last_unstale":"2024-09-13T08:43:18.794393+0000","last_undegraded":"2024-09-13T08:43:18.794393+0000","last_fullsized":"2024-09-13T08:43:18.794393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:24:35.034287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662622+0000","last_change":"2024-09-13T08:43:15.785688+0000","last_active":"2024-09-13T08:43:18.662622+0000","last_peered":"2024-09-13T08:43:18.662622+0000","last_clean":"2024-09-13T08:43:18.662622+0000","last_became_active":"2024-09-13T08:43:15.785388+0000","last_became_peered":"2024-09-13T08:43:15.785388+0000","last_unstale":"2024-09-13T08:43:18.662622+0000","last_undegraded":"2024-09-13T08:43:18.662622+0000","last_fullsized":"2024-09-13T08:43:18.662622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:12.431789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.811773+0000","last_change":"2024-09-13T08:43:16.805154+0000","last_active":"2024-09-13T08:43:16.811773+0000","last_peered":"2024-09-13T08:43:16.811773+0000","last_clean":"2024-09-13T08:43:16.811773+0000","last_became_active":"2024-09-13T08:43:16.804886+0000","last_became_peered":"2024-09-13T08:43:16.804886+0000","last_unstale":"2024-09-13T08:43:16.811773+0000","last_undegraded":"2024-09-13T08:43:16.811773+0000","last_fullsized":"2024-09-13T08:43:16.811773+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:14.772801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860103+0000","last_change":"2024-09-13T08:43:15.787515+0000","last_active":"2024-09-13T08:43:18.860103+0000","last_peered":"2024-09-13T08:43:18.860103+0000","last_clean":"2024-09-13T08:43:18.860103+0000","last_became_active":"2024-09-13T08:43:15.786997+0000","last_became_peered":"2024-09-13T08:43:15.786997+0000","last_unstale":"2024-09-13T08:43:18.860103+0000","last_undegraded":"2024-09-13T08:43:18.860103+0000","last_fullsized":"2024-09-13T08:43:18.860103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:00.138886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860801+0000","last_change":"2024-09-13T08:43:16.798116+0000","last_active":"2024-09-13T08:43:18.860801+0000","last_peered":"2024-09-13T08:43:18.860801+0000","last_clean":"2024-09-13T08:43:18.860801+0000","last_became_active":"2024-09-13T08:43:16.797800+0000","last_became_peered":"2024-09-13T08:43:16.797800+0000","last_unstale":"2024-09-13T08:43:18.860801+0000","last_undegraded":"2024-09-13T08:43:18.860801+0000","last_fullsized":"2024-09-13T08:43:18.860801+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:27.320983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662790+0000","last_change":"2024-09-13T08:43:15.792500+0000","last_active":"2024-09-13T08:43:18.662790+0000","last_peered":"2024-09-13T08:43:18.662790+0000","last_clean":"2024-09-13T08:43:18.662790+0000","last_became_active":"2024-09-13T08:43:15.792200+0000","last_became_peered":"2024-09-13T08:43:15.792200+0000","last_unstale":"2024-09-13T08:43:18.662790+0000","last_undegraded":"2024-09-13T08:43:18.662790+0000","last_fullsized":"2024-09-13T08:43:18.662790+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:23.815682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794921+0000","last_change":"2024-09-13T08:43:16.805873+0000","last_active":"2024-09-13T08:43:18.794921+0000","last_peered":"2024-09-13T08:43:18.794921+0000","last_clean":"2024-09-13T08:43:18.794921+0000","last_became_active":"2024-09-13T08:43:16.805545+0000","last_became_peered":"2024-09-13T08:43:16.805545+0000","last_unstale":"2024-09-13T08:43:18.794921+0000","last_undegraded":"2024-09-13T08:43:18.794921+0000","last_fullsized":"2024-09-13T08:43:18.794921+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:11:21.886945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791685+0000","last_change":"2024-09-13T08:43:15.788513+0000","last_active":"2024-09-13T08:43:17.791685+0000","last_peered":"2024-09-13T08:43:17.791685+0000","last_clean":"2024-09-13T08:43:17.791685+0000","last_became_active":"2024-09-13T08:43:15.787923+0000","last_became_peered":"2024-09-13T08:43:15.787923+0000","last_unstale":"2024-09-13T08:43:17.791685+0000","last_undegraded":"2024-09-13T08:43:17.791685+0000","last_fullsized":"2024-09-13T08:43:17.791685+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:47.151058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791816+0000","last_change":"2024-09-13T08:43:16.811398+0000","last_active":"2024-09-13T08:43:17.791816+0000","last_peered":"2024-09-13T08:43:17.791816+0000","last_clean":"2024-09-13T08:43:17.791816+0000","last_became_active":"2024-09-13T08:43:16.811081+0000","last_became_peered":"2024-09-13T08:43:16.811081+0000","last_unstale":"2024-09-13T08:43:17.791816+0000","last_undegraded":"2024-09-13T08:43:17.791816+0000","last_fullsized":"2024-09-13T08:43:17.791816+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:32:12.639464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"60'3","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791708+0000","last_change":"2024-09-13T08:43:15.792355+0000","last_active":"2024-09-13T08:43:17.791708+0000","last_peered":"2024-09-13T08:43:17.791708+0000","last_clean":"2024-09-13T08:43:17.791708+0000","last_became_active":"2024-09-13T08:43:15.791972+0000","last_became_peered":"2024-09-13T08:43:15.791972+0000","last_unstale":"2024-09-13T08:43:17.791708+0000","last_undegraded":"2024-09-13T08:43:17.791708+0000","last_fullsized":"2024-09-13T08:43:17.791708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:54.861136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791510+0000","last_change":"2024-09-13T08:43:16.821625+0000","last_active":"2024-09-13T08:43:17.791510+0000","last_peered":"2024-09-13T08:43:17.791510+0000","last_clean":"2024-09-13T08:43:17.791510+0000","last_became_active":"2024-09-13T08:43:16.821277+0000","last_became_peered":"2024-09-13T08:43:16.821277+0000","last_unstale":"2024-09-13T08:43:17.791510+0000","last_undegraded":"2024-09-13T08:43:17.791510+0000","last_fullsized":"2024-09-13T08:43:17.791510+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:15.920121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"60'6","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860561+0000","last_change":"2024-09-13T08:43:15.788489+0000","last_active":"2024-09-13T08:43:18.860561+0000","last_peered":"2024-09-13T08:43:18.860561+0000","last_clean":"2024-09-13T08:43:18.860561+0000","last_became_active":"2024-09-13T08:43:15.788176+0000","last_became_peered":"2024-09-13T08:43:15.788176+0000","last_unstale":"2024-09-13T08:43:18.860561+0000","last_undegraded":"2024-09-13T08:43:18.860561+0000","last_fullsized":"2024-09-13T08:43:18.860561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:51:27.879550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662727+0000","last_change":"2024-09-13T08:43:16.809196+0000","last_active":"2024-09-13T08:43:18.662727+0000","last_peered":"2024-09-13T08:43:18.662727+0000","last_clean":"2024-09-13T08:43:18.662727+0000","last_became_active":"2024-09-13T08:43:16.808921+0000","last_became_peered":"2024-09-13T08:43:16.808921+0000","last_unstale":"2024-09-13T08:43:18.662727+0000","last_undegraded":"2024-09-13T08:43:18.662727+0000","last_fullsized":"2024-09-13T08:43:18.662727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:39.435646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790950+0000","last_change":"2024-09-13T08:43:15.783835+0000","last_active":"2024-09-13T08:43:17.790950+0000","last_peered":"2024-09-13T08:43:17.790950+0000","last_clean":"2024-09-13T08:43:17.790950+0000","last_became_active":"2024-09-13T08:43:15.783522+0000","last_became_peered":"2024-09-13T08:43:15.783522+0000","last_unstale":"2024-09-13T08:43:17.790950+0000","last_undegraded":"2024-09-13T08:43:17.790950+0000","last_fullsized":"2024-09-13T08:43:17.790950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:28:11.697090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794837+0000","last_change":"2024-09-13T08:43:16.820336+0000","last_active":"2024-09-13T08:43:18.794837+0000","last_peered":"2024-09-13T08:43:18.794837+0000","last_clean":"2024-09-13T08:43:18.794837+0000","last_became_active":"2024-09-13T08:43:16.820002+0000","last_became_peered":"2024-09-13T08:43:16.820002+0000","last_unstale":"2024-09-13T08:43:18.794837+0000","last_undegraded":"2024-09-13T08:43:18.794837+0000","last_fullsized":"2024-09-13T08:43:18.794837+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:44:51.897731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791068+0000","last_change":"2024-09-13T08:43:15.792918+0000","last_active":"2024-09-13T08:43:17.791068+0000","last_peered":"2024-09-13T08:43:17.791068+0000","last_clean":"2024-09-13T08:43:17.791068+0000","last_became_active":"2024-09-13T08:43:15.791822+0000","last_became_peered":"2024-09-13T08:43:15.791822+0000","last_unstale":"2024-09-13T08:43:17.791068+0000","last_undegraded":"2024-09-13T08:43:17.791068+0000","last_fullsized":"2024-09-13T08:43:17.791068+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:51:41.415569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662981+0000","last_change":"2024-09-13T08:43:16.810442+0000","last_active":"2024-09-13T08:43:18.662981+0000","last_peered":"2024-09-13T08:43:18.662981+0000","last_clean":"2024-09-13T08:43:18.662981+0000","last_became_active":"2024-09-13T08:43:16.810152+0000","last_became_peered":"2024-09-13T08:43:16.810152+0000","last_unstale":"2024-09-13T08:43:18.662981+0000","last_undegraded":"2024-09-13T08:43:18.662981+0000","last_fullsized":"2024-09-13T08:43:18.662981+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:48.635890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791285+0000","last_change":"2024-09-13T08:43:15.786181+0000","last_active":"2024-09-13T08:43:17.791285+0000","last_peered":"2024-09-13T08:43:17.791285+0000","last_clean":"2024-09-13T08:43:17.791285+0000","last_became_active":"2024-09-13T08:43:15.785695+0000","last_became_peered":"2024-09-13T08:43:15.785695+0000","last_unstale":"2024-09-13T08:43:17.791285+0000","last_undegraded":"2024-09-13T08:43:17.791285+0000","last_fullsized":"2024-09-13T08:43:17.791285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:08.478225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791328+0000","last_change":"2024-09-13T08:43:16.815274+0000","last_active":"2024-09-13T08:43:17.791328+0000","last_peered":"2024-09-13T08:43:17.791328+0000","last_clean":"2024-09-13T08:43:17.791328+0000","last_became_active":"2024-09-13T08:43:16.804806+0000","last_became_peered":"2024-09-13T08:43:16.804806+0000","last_unstale":"2024-09-13T08:43:17.791328+0000","last_undegraded":"2024-09-13T08:43:17.791328+0000","last_fullsized":"2024-09-13T08:43:17.791328+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:33.793602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"60'8","reported_seq":23,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791049+0000","last_change":"2024-09-13T08:43:15.786318+0000","last_active":"2024-09-13T08:43:17.791049+0000","last_peered":"2024-09-13T08:43:17.791049+0000","last_clean":"2024-09-13T08:43:17.791049+0000","last_became_active":"2024-09-13T08:43:15.785846+0000","last_became_peered":"2024-09-13T08:43:15.785846+0000","last_unstale":"2024-09-13T08:43:17.791049+0000","last_undegraded":"2024-09-13T08:43:17.791049+0000","last_fullsized":"2024-09-13T08:43:17.791049+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:10.086776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.795016+0000","last_change":"2024-09-13T08:43:16.803765+0000","last_active":"2024-09-13T08:43:18.795016+0000","last_peered":"2024-09-13T08:43:18.795016+0000","last_clean":"2024-09-13T08:43:18.795016+0000","last_became_active":"2024-09-13T08:43:16.803487+0000","last_became_peered":"2024-09-13T08:43:16.803487+0000","last_unstale":"2024-09-13T08:43:18.795016+0000","last_undegraded":"2024-09-13T08:43:18.795016+0000","last_fullsized":"2024-09-13T08:43:18.795016+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:45.977000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861251+0000","last_change":"2024-09-13T08:43:15.795460+0000","last_active":"2024-09-13T08:43:18.861251+0000","last_peered":"2024-09-13T08:43:18.861251+0000","last_clean":"2024-09-13T08:43:18.861251+0000","last_became_active":"2024-09-13T08:43:15.795165+0000","last_became_peered":"2024-09-13T08:43:15.795165+0000","last_unstale":"2024-09-13T08:43:18.861251+0000","last_undegraded":"2024-09-13T08:43:18.861251+0000","last_fullsized":"2024-09-13T08:43:18.861251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:23.815682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794926+0000","last_change":"2024-09-13T08:43:16.806639+0000","last_active":"2024-09-13T08:43:18.794926+0000","last_peered":"2024-09-13T08:43:18.794926+0000","last_clean":"2024-09-13T08:43:18.794926+0000","last_became_active":"2024-09-13T08:43:16.806341+0000","last_became_peered":"2024-09-13T08:43:16.806341+0000","last_unstale":"2024-09-13T08:43:18.794926+0000","last_undegraded":"2024-09-13T08:43:18.794926+0000","last_fullsized":"2024-09-13T08:43:18.794926+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:13:28.124095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791809+0000","last_change":"2024-09-13T08:43:15.788192+0000","last_active":"2024-09-13T08:43:17.791809+0000","last_peered":"2024-09-13T08:43:17.791809+0000","last_clean":"2024-09-13T08:43:17.791809+0000","last_became_active":"2024-09-13T08:43:15.787581+0000","last_became_peered":"2024-09-13T08:43:15.787581+0000","last_unstale":"2024-09-13T08:43:17.791809+0000","last_undegraded":"2024-09-13T08:43:17.791809+0000","last_fullsized":"2024-09-13T08:43:17.791809+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:22:42.515576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860836+0000","last_change":"2024-09-13T08:43:16.814575+0000","last_active":"2024-09-13T08:43:18.860836+0000","last_peered":"2024-09-13T08:43:18.860836+0000","last_clean":"2024-09-13T08:43:18.860836+0000","last_became_active":"2024-09-13T08:43:16.814038+0000","last_became_peered":"2024-09-13T08:43:16.814038+0000","last_unstale":"2024-09-13T08:43:18.860836+0000","last_undegraded":"2024-09-13T08:43:18.860836+0000","last_fullsized":"2024-09-13T08:43:18.860836+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:56.549276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794851+0000","last_change":"2024-09-13T08:43:15.793141+0000","last_active":"2024-09-13T08:43:18.794851+0000","last_peered":"2024-09-13T08:43:18.794851+0000","last_clean":"2024-09-13T08:43:18.794851+0000","last_became_active":"2024-09-13T08:43:15.792406+0000","last_became_peered":"2024-09-13T08:43:15.792406+0000","last_unstale":"2024-09-13T08:43:18.794851+0000","last_undegraded":"2024-09-13T08:43:18.794851+0000","last_fullsized":"2024-09-13T08:43:18.794851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:31:46.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791703+0000","last_change":"2024-09-13T08:43:16.814229+0000","last_active":"2024-09-13T08:43:17.791703+0000","last_peered":"2024-09-13T08:43:17.791703+0000","last_clean":"2024-09-13T08:43:17.791703+0000","last_became_active":"2024-09-13T08:43:16.813894+0000","last_became_peered":"2024-09-13T08:43:16.813894+0000","last_unstale":"2024-09-13T08:43:17.791703+0000","last_undegraded":"2024-09-13T08:43:17.791703+0000","last_fullsized":"2024-09-13T08:43:17.791703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:11.285883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.792270+0000","last_change":"2024-09-13T08:43:15.784641+0000","last_active":"2024-09-13T08:43:17.792270+0000","last_peered":"2024-09-13T08:43:17.792270+0000","last_clean":"2024-09-13T08:43:17.792270+0000","last_became_active":"2024-09-13T08:43:15.783695+0000","last_became_peered":"2024-09-13T08:43:15.783695+0000","last_unstale":"2024-09-13T08:43:17.792270+0000","last_undegraded":"2024-09-13T08:43:17.792270+0000","last_fullsized":"2024-09-13T08:43:17.792270+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:20.995357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662790+0000","last_change":"2024-09-13T08:43:16.822104+0000","last_active":"2024-09-13T08:43:18.662790+0000","last_peered":"2024-09-13T08:43:18.662790+0000","last_clean":"2024-09-13T08:43:18.662790+0000","last_became_active":"2024-09-13T08:43:16.821813+0000","last_became_peered":"2024-09-13T08:43:16.821813+0000","last_unstale":"2024-09-13T08:43:18.662790+0000","last_undegraded":"2024-09-13T08:43:18.662790+0000","last_fullsized":"2024-09-13T08:43:18.662790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:35.031802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791006+0000","last_change":"2024-09-13T08:43:15.797489+0000","last_active":"2024-09-13T08:43:17.791006+0000","last_peered":"2024-09-13T08:43:17.791006+0000","last_clean":"2024-09-13T08:43:17.791006+0000","last_became_active":"2024-09-13T08:43:15.797117+0000","last_became_peered":"2024-09-13T08:43:15.797117+0000","last_unstale":"2024-09-13T08:43:17.791006+0000","last_undegraded":"2024-09-13T08:43:17.791006+0000","last_fullsized":"2024-09-13T08:43:17.791006+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:32.605971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791095+0000","last_change":"2024-09-13T08:43:16.806921+0000","last_active":"2024-09-13T08:43:17.791095+0000","last_peered":"2024-09-13T08:43:17.791095+0000","last_clean":"2024-09-13T08:43:17.791095+0000","last_became_active":"2024-09-13T08:43:16.806475+0000","last_became_peered":"2024-09-13T08:43:16.806475+0000","last_unstale":"2024-09-13T08:43:17.791095+0000","last_undegraded":"2024-09-13T08:43:17.791095+0000","last_fullsized":"2024-09-13T08:43:17.791095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:02.071955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790869+0000","last_change":"2024-09-13T08:43:15.783802+0000","last_active":"2024-09-13T08:43:17.790869+0000","last_peered":"2024-09-13T08:43:17.790869+0000","last_clean":"2024-09-13T08:43:17.790869+0000","last_became_active":"2024-09-13T08:43:15.783504+0000","last_became_peered":"2024-09-13T08:43:15.783504+0000","last_unstale":"2024-09-13T08:43:17.790869+0000","last_undegraded":"2024-09-13T08:43:17.790869+0000","last_fullsized":"2024-09-13T08:43:17.790869+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:49.383664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.790695+0000","last_change":"2024-09-13T08:43:16.802620+0000","last_active":"2024-09-13T08:43:17.790695+0000","last_peered":"2024-09-13T08:43:17.790695+0000","last_clean":"2024-09-13T08:43:17.790695+0000","last_became_active":"2024-09-13T08:43:16.802170+0000","last_became_peered":"2024-09-13T08:43:16.802170+0000","last_unstale":"2024-09-13T08:43:17.790695+0000","last_undegraded":"2024-09-13T08:43:17.790695+0000","last_fullsized":"2024-09-13T08:43:17.790695+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:19.694731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.663107+0000","last_change":"2024-09-13T08:43:15.784382+0000","last_active":"2024-09-13T08:43:18.663107+0000","last_peered":"2024-09-13T08:43:18.663107+0000","last_clean":"2024-09-13T08:43:18.663107+0000","last_became_active":"2024-09-13T08:43:15.784098+0000","last_became_peered":"2024-09-13T08:43:15.784098+0000","last_unstale":"2024-09-13T08:43:18.663107+0000","last_undegraded":"2024-09-13T08:43:18.663107+0000","last_fullsized":"2024-09-13T08:43:18.663107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:34.681324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.795050+0000","last_change":"2024-09-13T08:43:16.808006+0000","last_active":"2024-09-13T08:43:18.795050+0000","last_peered":"2024-09-13T08:43:18.795050+0000","last_clean":"2024-09-13T08:43:18.795050+0000","last_became_active":"2024-09-13T08:43:16.807637+0000","last_became_peered":"2024-09-13T08:43:16.807637+0000","last_unstale":"2024-09-13T08:43:18.795050+0000","last_undegraded":"2024-09-13T08:43:18.795050+0000","last_fullsized":"2024-09-13T08:43:18.795050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:16.973803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.861325+0000","last_change":"2024-09-13T08:43:15.795287+0000","last_active":"2024-09-13T08:43:18.861325+0000","last_peered":"2024-09-13T08:43:18.861325+0000","last_clean":"2024-09-13T08:43:18.861325+0000","last_became_active":"2024-09-13T08:43:15.794985+0000","last_became_peered":"2024-09-13T08:43:15.794985+0000","last_unstale":"2024-09-13T08:43:18.861325+0000","last_undegraded":"2024-09-13T08:43:18.861325+0000","last_fullsized":"2024-09-13T08:43:18.861325+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:53.611470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.795085+0000","last_change":"2024-09-13T08:43:16.823426+0000","last_active":"2024-09-13T08:43:18.795085+0000","last_peered":"2024-09-13T08:43:18.795085+0000","last_clean":"2024-09-13T08:43:18.795085+0000","last_became_active":"2024-09-13T08:43:16.823172+0000","last_became_peered":"2024-09-13T08:43:16.823172+0000","last_unstale":"2024-09-13T08:43:18.795085+0000","last_undegraded":"2024-09-13T08:43:18.795085+0000","last_fullsized":"2024-09-13T08:43:18.795085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:32.141439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.818461+0000","last_change":"2024-09-13T08:43:15.799101+0000","last_active":"2024-09-13T08:43:16.818461+0000","last_peered":"2024-09-13T08:43:16.818461+0000","last_clean":"2024-09-13T08:43:16.818461+0000","last_became_active":"2024-09-13T08:43:15.798891+0000","last_became_peered":"2024-09-13T08:43:15.798891+0000","last_unstale":"2024-09-13T08:43:16.818461+0000","last_undegraded":"2024-09-13T08:43:16.818461+0000","last_fullsized":"2024-09-13T08:43:16.818461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:15.743527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860903+0000","last_change":"2024-09-13T08:43:16.815521+0000","last_active":"2024-09-13T08:43:18.860903+0000","last_peered":"2024-09-13T08:43:18.860903+0000","last_clean":"2024-09-13T08:43:18.860903+0000","last_became_active":"2024-09-13T08:43:16.814348+0000","last_became_peered":"2024-09-13T08:43:16.814348+0000","last_unstale":"2024-09-13T08:43:18.860903+0000","last_undegraded":"2024-09-13T08:43:18.860903+0000","last_fullsized":"2024-09-13T08:43:18.860903+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:18.835674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662810+0000","last_change":"2024-09-13T08:43:15.798380+0000","last_active":"2024-09-13T08:43:18.662810+0000","last_peered":"2024-09-13T08:43:18.662810+0000","last_clean":"2024-09-13T08:43:18.662810+0000","last_became_active":"2024-09-13T08:43:15.798062+0000","last_became_peered":"2024-09-13T08:43:15.798062+0000","last_unstale":"2024-09-13T08:43:18.662810+0000","last_undegraded":"2024-09-13T08:43:18.662810+0000","last_fullsized":"2024-09-13T08:43:18.662810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:24.902615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.791774+0000","last_change":"2024-09-13T08:43:16.803663+0000","last_active":"2024-09-13T08:43:17.791774+0000","last_peered":"2024-09-13T08:43:17.791774+0000","last_clean":"2024-09-13T08:43:17.791774+0000","last_became_active":"2024-09-13T08:43:16.802142+0000","last_became_peered":"2024-09-13T08:43:16.802142+0000","last_unstale":"2024-09-13T08:43:17.791774+0000","last_undegraded":"2024-09-13T08:43:17.791774+0000","last_fullsized":"2024-09-13T08:43:17.791774+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:59.529940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794629+0000","last_change":"2024-09-13T08:43:15.794144+0000","last_active":"2024-09-13T08:43:18.794629+0000","last_peered":"2024-09-13T08:43:18.794629+0000","last_clean":"2024-09-13T08:43:18.794629+0000","last_became_active":"2024-09-13T08:43:15.793769+0000","last_became_peered":"2024-09-13T08:43:15.793769+0000","last_unstale":"2024-09-13T08:43:18.794629+0000","last_undegraded":"2024-09-13T08:43:18.794629+0000","last_fullsized":"2024-09-13T08:43:18.794629+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:06.717335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.811998+0000","last_change":"2024-09-13T08:43:16.802991+0000","last_active":"2024-09-13T08:43:16.811998+0000","last_peered":"2024-09-13T08:43:16.811998+0000","last_clean":"2024-09-13T08:43:16.811998+0000","last_became_active":"2024-09-13T08:43:16.802719+0000","last_became_peered":"2024-09-13T08:43:16.802719+0000","last_unstale":"2024-09-13T08:43:16.811998+0000","last_undegraded":"2024-09-13T08:43:16.811998+0000","last_fullsized":"2024-09-13T08:43:16.811998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:37.786564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860730+0000","last_change":"2024-09-13T08:43:15.795297+0000","last_active":"2024-09-13T08:43:18.860730+0000","last_peered":"2024-09-13T08:43:18.860730+0000","last_clean":"2024-09-13T08:43:18.860730+0000","last_became_active":"2024-09-13T08:43:15.795009+0000","last_became_peered":"2024-09-13T08:43:15.795009+0000","last_unstale":"2024-09-13T08:43:18.860730+0000","last_undegraded":"2024-09-13T08:43:18.860730+0000","last_fullsized":"2024-09-13T08:43:18.860730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:12.431789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860729+0000","last_change":"2024-09-13T08:43:16.799262+0000","last_active":"2024-09-13T08:43:18.860729+0000","last_peered":"2024-09-13T08:43:18.860729+0000","last_clean":"2024-09-13T08:43:18.860729+0000","last_became_active":"2024-09-13T08:43:16.798985+0000","last_became_peered":"2024-09-13T08:43:16.798985+0000","last_unstale":"2024-09-13T08:43:18.860729+0000","last_undegraded":"2024-09-13T08:43:18.860729+0000","last_fullsized":"2024-09-13T08:43:18.860729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:05.875443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.794360+0000","last_change":"2024-09-13T08:43:15.782349+0000","last_active":"2024-09-13T08:43:18.794360+0000","last_peered":"2024-09-13T08:43:18.794360+0000","last_clean":"2024-09-13T08:43:18.794360+0000","last_became_active":"2024-09-13T08:43:15.782098+0000","last_became_peered":"2024-09-13T08:43:15.782098+0000","last_unstale":"2024-09-13T08:43:18.794360+0000","last_undegraded":"2024-09-13T08:43:18.794360+0000","last_fullsized":"2024-09-13T08:43:18.794360+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:23.057200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.792236+0000","last_change":"2024-09-13T08:43:16.809938+0000","last_active":"2024-09-13T08:43:17.792236+0000","last_peered":"2024-09-13T08:43:17.792236+0000","last_clean":"2024-09-13T08:43:17.792236+0000","last_became_active":"2024-09-13T08:43:16.809699+0000","last_became_peered":"2024-09-13T08:43:16.809699+0000","last_unstale":"2024-09-13T08:43:17.792236+0000","last_undegraded":"2024-09-13T08:43:17.792236+0000","last_fullsized":"2024-09-13T08:43:17.792236+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:46:12.249470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T08:43:16.809427+0000","last_change":"2024-09-13T08:43:15.785423+0000","last_active":"2024-09-13T08:43:16.809427+0000","last_peered":"2024-09-13T08:43:16.809427+0000","last_clean":"2024-09-13T08:43:16.809427+0000","last_became_active":"2024-09-13T08:43:15.784926+0000","last_became_peered":"2024-09-13T08:43:15.784926+0000","last_unstale":"2024-09-13T08:43:16.809427+0000","last_undegraded":"2024-09-13T08:43:16.809427+0000","last_fullsized":"2024-09-13T08:43:16.809427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:46.548513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860918+0000","last_change":"2024-09-13T08:43:16.806772+0000","last_active":"2024-09-13T08:43:18.860918+0000","last_peered":"2024-09-13T08:43:18.860918+0000","last_clean":"2024-09-13T08:43:18.860918+0000","last_became_active":"2024-09-13T08:43:16.806371+0000","last_became_peered":"2024-09-13T08:43:16.806371+0000","last_unstale":"2024-09-13T08:43:18.860918+0000","last_undegraded":"2024-09-13T08:43:18.860918+0000","last_fullsized":"2024-09-13T08:43:18.860918+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:27:50.131143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.792310+0000","last_change":"2024-09-13T08:43:15.791213+0000","last_active":"2024-09-13T08:43:17.792310+0000","last_peered":"2024-09-13T08:43:17.792310+0000","last_clean":"2024-09-13T08:43:17.792310+0000","last_became_active":"2024-09-13T08:43:15.790898+0000","last_became_peered":"2024-09-13T08:43:15.790898+0000","last_unstale":"2024-09-13T08:43:17.792310+0000","last_undegraded":"2024-09-13T08:43:17.792310+0000","last_fullsized":"2024-09-13T08:43:17.792310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:07:43.245534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:17.792452+0000","last_change":"2024-09-13T08:43:16.806147+0000","last_active":"2024-09-13T08:43:17.792452+0000","last_peered":"2024-09-13T08:43:17.792452+0000","last_clean":"2024-09-13T08:43:17.792452+0000","last_became_active":"2024-09-13T08:43:16.805854+0000","last_became_peered":"2024-09-13T08:43:16.805854+0000","last_unstale":"2024-09-13T08:43:17.792452+0000","last_undegraded":"2024-09-13T08:43:17.792452+0000","last_fullsized":"2024-09-13T08:43:17.792452+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:15.762779+0000","last_clean_scrub_stamp":"2024-09-13T08:43:15.762779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:46:13.648919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.860251+0000","last_change":"2024-09-13T08:43:15.794923+0000","last_active":"2024-09-13T08:43:18.860251+0000","last_peered":"2024-09-13T08:43:18.860251+0000","last_clean":"2024-09-13T08:43:18.860251+0000","last_became_active":"2024-09-13T08:43:15.794651+0000","last_became_peered":"2024-09-13T08:43:15.794651+0000","last_unstale":"2024-09-13T08:43:18.860251+0000","last_undegraded":"2024-09-13T08:43:18.860251+0000","last_fullsized":"2024-09-13T08:43:18.860251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:43:14.753086+0000","last_clean_scrub_stamp":"2024-09-13T08:43:14.753086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:23.057200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":185,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T08:43:18.662358+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:43:18.662358+0000","last_peered":"2024-09-13T08:43:18.662358+0000","last_clean":"2024-09-13T08:43:18.662358+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:43:18.662358+0000","last_undegraded":"2024-09-13T08:43:18.662358+0000","last_fullsized":"2024-09-13T08:43:18.662358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40220,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331620,"statfs":{"total":96636764160,"available":96595578880,"internally_reserved":0,"allocated":720896,"data_stored":296958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509513,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46372,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325468,"statfs":{"total":96636764160,"available":96589279232,"internally_reserved":0,"allocated":1323008,"data_stored":893596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509514,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45800,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326040,"statfs":{"total":96636764160,"available":96589864960,"internally_reserved":0,"allocated":733184,"data_stored":303810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509514,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40228,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331612,"statfs":{"total":96636764160,"available":96595570688,"internally_reserved":0,"allocated":729088,"data_stored":303284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509513,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62440,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309400,"statfs":{"total":96636764160,"available":96572825600,"internally_reserved":0,"allocated":733184,"data_stored":298638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509515,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40796,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331044,"statfs":{"total":96636764160,"available":96594989056,"internally_reserved":0,"allocated":1310720,"data_stored":887816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40224,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331616,"statfs":{"total":96636764160,"available":96595574784,"internally_reserved":0,"allocated":724992,"data_stored":298478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542219,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58264,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313576,"statfs":{"total":96636764160,"available":96577101824,"internally_reserved":0,"allocated":716800,"data_stored":296888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:43:21.378 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:43:21.379 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T08:43:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.571+0000 7fdcf3600640 1 -- 172.21.15.44:0/2741392240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcd4003a40 msgr2=0x7fdcd4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.571+0000 7fdcf3600640 1 --2- 172.21.15.44:0/2741392240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcd4003a40 0x7fdcd4003e60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdce00016f0 tx=0x7fdce001c770 comp rx=0 tx=0).stop 2024-09-13T08:43:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.571+0000 7fdcf1600640 1 -- 172.21.15.44:0/2741392240 reap_dead start 2024-09-13T08:43:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.571+0000 7fdcf3600640 1 -- 172.21.15.44:0/2741392240 shutdown_connections 2024-09-13T08:43:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.571+0000 7fdcf3600640 1 -- 172.21.15.44:0/2741392240 >> 172.21.15.44:0/2741392240 conn(0x7fdcec0b18f0 msgr2=0x7fdcec0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.571+0000 7fdcf3600640 1 -- 172.21.15.44:0/2741392240 shutdown_connections 2024-09-13T08:43:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.571+0000 7fdcf3600640 1 -- 172.21.15.44:0/2741392240 wait complete. 2024-09-13T08:43:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf3600640 1 Processor -- start 2024-09-13T08:43:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf3600640 1 -- start start 2024-09-13T08:43:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf3600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd4003a40 0x7fdcec1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf3600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdcec1fb190 0x7fdcec1fb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf3600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcec151d30 0x7fdcec1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf3600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec0ccd50 con 0x7fdcec151d30 2024-09-13T08:43:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf3600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec0ccbd0 con 0x7fdcd4003a40 2024-09-13T08:43:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf3600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdcec0cced0 con 0x7fdcec1fb190 2024-09-13T08:43:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcec151d30 0x7fdcec1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcec151d30 0x7fdcec1521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46864/0 (socket says 172.21.15.44:46864) 2024-09-13T08:43:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf1600640 1 -- 172.21.15.44:0/551802826 learned_addr learned my addr 172.21.15.44:0/551802826 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcebe00640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdcec1fb190 0x7fdcec1fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf0c00640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd4003a40 0x7fdcec1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf1600640 1 -- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdcec1fb190 msgr2=0x7fdcec1fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf1600640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdcec1fb190 0x7fdcec1fb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf1600640 1 -- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd4003a40 msgr2=0x7fdcec1426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf1600640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd4003a40 0x7fdcec1426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf1600640 1 -- 172.21.15.44:0/551802826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec152a60 con 0x7fdcec151d30 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf1600640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcec151d30 0x7fdcec1521b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0b2830 tx=0x7fdce400fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdce8c00640 1 -- 172.21.15.44:0/551802826 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4004b30 con 0x7fdcec151d30 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdce8c00640 1 -- 172.21.15.44:0/551802826 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdce4004cd0 con 0x7fdcec151d30 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf3600640 1 -- 172.21.15.44:0/551802826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec152d40 con 0x7fdcec151d30 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdce8c00640 1 -- 172.21.15.44:0/551802826 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4003be0 con 0x7fdcec151d30 2024-09-13T08:43:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.575+0000 7fdcf3600640 1 -- 172.21.15.44:0/551802826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec142be0 con 0x7fdcec151d30 2024-09-13T08:43:21.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.579+0000 7fdcf3600640 1 -- 172.21.15.44:0/551802826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec069410 con 0x7fdcec151d30 2024-09-13T08:43:21.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.579+0000 7fdce8c00640 1 -- 172.21.15.44:0/551802826 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fdce4004030 con 0x7fdcec151d30 2024-09-13T08:43:21.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.579+0000 7fdce8c00640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdccc0737c0 0x7fdccc075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:21.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.579+0000 7fdce8c00640 1 -- 172.21.15.44:0/551802826 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fdce4095690 con 0x7fdcec151d30 2024-09-13T08:43:21.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.579+0000 7fdcf0c00640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdccc0737c0 0x7fdccc075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:21.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.579+0000 7fdcf0c00640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdccc0737c0 0x7fdccc075c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdce00016c0 tx=0x7fdce00a8040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:21.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.583+0000 7fdce8c00640 1 -- 172.21.15.44:0/551802826 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdce405f140 con 0x7fdcec151d30 2024-09-13T08:43:21.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.827+0000 7fdcf3600640 1 -- 172.21.15.44:0/551802826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fdcec15a8b0 con 0x7fdcec151d30 2024-09-13T08:43:21.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.835+0000 7fdce8c00640 1 -- 172.21.15.44:0/551802826 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fdce4063b70 con 0x7fdcec151d30 2024-09-13T08:43:21.840 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T08:43:21.840 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:43:21.840 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-09-13T08:43:21.840 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-09-13T08:43:21.840 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-13T08:43:21.841 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-09-13T08:43:21.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 -- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdccc0737c0 msgr2=0x7fdccc075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:21.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdccc0737c0 0x7fdccc075c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdce00016c0 tx=0x7fdce00a8040 comp rx=0 tx=0).stop 2024-09-13T08:43:21.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 -- 172.21.15.44:0/551802826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcec151d30 msgr2=0x7fdcec1521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcec151d30 0x7fdcec1521b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0b2830 tx=0x7fdce400fea0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 -- 172.21.15.44:0/551802826 shutdown_connections 2024-09-13T08:43:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdccc0737c0 0x7fdccc075c80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcec151d30 0x7fdcec1521b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdcec1fb190 0x7fdcec1fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 --2- 172.21.15.44:0/551802826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd4003a40 0x7fdcec1426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 -- 172.21.15.44:0/551802826 >> 172.21.15.44:0/551802826 conn(0x7fdcec0b18f0 msgr2=0x7fdcec073ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 -- 172.21.15.44:0/551802826 shutdown_connections 2024-09-13T08:43:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:21.839+0000 7fdcf3600640 1 -- 172.21.15.44:0/551802826 wait complete. 2024-09-13T08:43:21.862 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:43:22.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.003+0000 7fa6b3200640 1 -- 172.21.15.44:0/3645924993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa698004810 msgr2=0x7fa698004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.003+0000 7fa6b3200640 1 --2- 172.21.15.44:0/3645924993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa698004810 0x7fa698004c90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00ac40 tx=0x7fa69c01e050 comp rx=0 tx=0).stop 2024-09-13T08:43:22.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.003+0000 7fa6b2800640 1 -- 172.21.15.44:0/3645924993 reap_dead start 2024-09-13T08:43:22.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.003+0000 7fa6b3200640 1 -- 172.21.15.44:0/3645924993 shutdown_connections 2024-09-13T08:43:22.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.003+0000 7fa6b3200640 1 -- 172.21.15.44:0/3645924993 >> 172.21.15.44:0/3645924993 conn(0x7fa6b4073490 msgr2=0x7fa6b40d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:22.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.003+0000 7fa6b3200640 1 -- 172.21.15.44:0/3645924993 shutdown_connections 2024-09-13T08:43:22.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.003+0000 7fa6b3200640 1 -- 172.21.15.44:0/3645924993 wait complete. 2024-09-13T08:43:22.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b3200640 1 Processor -- start 2024-09-13T08:43:22.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b3200640 1 -- start start 2024-09-13T08:43:22.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa698004810 0x7fa6b409a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b3200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6b41f7980 0x7fa6b409bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b3200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b4070190 0x7fa6b41f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b40cd4a0 con 0x7fa698004810 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b3200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b40cd320 con 0x7fa6b4070190 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b3200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b415a6e0 con 0x7fa6b41f7980 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa698004810 0x7fa6b409a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa698004810 0x7fa6b409a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46888/0 (socket says 172.21.15.44:46888) 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b1e00640 1 -- 172.21.15.44:0/3161424062 learned_addr learned my addr 172.21.15.44:0/3161424062 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b2800640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b4070190 0x7fa6b41f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6abe00640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6b41f7980 0x7fa6b409bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b1e00640 1 -- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6b41f7980 msgr2=0x7fa6b409bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b1e00640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6b41f7980 0x7fa6b409bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b1e00640 1 -- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b4070190 msgr2=0x7fa6b41f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b1e00640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b4070190 0x7fa6b41f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b1e00640 1 -- 172.21.15.44:0/3161424062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b40b0fa0 con 0x7fa698004810 2024-09-13T08:43:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b1e00640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa698004810 0x7fa6b409a310 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4012620 tx=0x7fa6a4012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6aa000640 1 -- 172.21.15.44:0/3161424062 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40109f0 con 0x7fa698004810 2024-09-13T08:43:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6aa000640 1 -- 172.21.15.44:0/3161424062 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6a4017030 con 0x7fa698004810 2024-09-13T08:43:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6aa000640 1 -- 172.21.15.44:0/3161424062 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4010e30 con 0x7fa698004810 2024-09-13T08:43:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b3200640 1 -- 172.21.15.44:0/3161424062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b41f7ef0 con 0x7fa698004810 2024-09-13T08:43:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.007+0000 7fa6b3200640 1 -- 172.21.15.44:0/3161424062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b41f84b0 con 0x7fa698004810 2024-09-13T08:43:22.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.011+0000 7fa6b3200640 1 -- 172.21.15.44:0/3161424062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b40b1170 con 0x7fa698004810 2024-09-13T08:43:22.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.011+0000 7fa6aa000640 1 -- 172.21.15.44:0/3161424062 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fa6a4013450 con 0x7fa698004810 2024-09-13T08:43:22.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.011+0000 7fa6aa000640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6900737f0 0x7fa690075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.011+0000 7fa6aa000640 1 -- 172.21.15.44:0/3161424062 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fa6a4096310 con 0x7fa698004810 2024-09-13T08:43:22.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.011+0000 7fa6abe00640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6900737f0 0x7fa690075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.015+0000 7fa6abe00640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6900737f0 0x7fa690075cb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0af000 tx=0x7fa69c01ae50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:22.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.015+0000 7fa6aa000640 1 -- 172.21.15.44:0/3161424062 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa6a4062fb0 con 0x7fa698004810 2024-09-13T08:43:22.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 -- 172.21.15.44:0/3161424062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6b40b2800 con 0x7fa698004810 2024-09-13T08:43:22.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6aa000640 1 -- 172.21.15.44:0/3161424062 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5223 (secure 0 0 0) 0x7fa6a40628b0 con 0x7fa698004810 2024-09-13T08:43:22.235 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:43:22.235 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":37,"btime":"2024-09-13T08:43:19:822213+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1613950025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1613950025},{"type":"v1","addr":"172.21.15.129:6836","nonce":1613950025}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5402,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/10747232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":10747232},{"type":"v1","addr":"172.21.15.129:6837","nonce":10747232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3945609265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3945609265},{"type":"v1","addr":"172.21.15.44:6834","nonce":3945609265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:43:16.187506+0000","modified":"2024-09-13T08:43:19.249207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2755626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2755626228},{"type":"v1","addr":"172.21.15.44:6835","nonce":2755626228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5877,"qdb_cluster":[5877]},"id":4}]} 2024-09-13T08:43:22.235 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 37 2024-09-13T08:43:22.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 -- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6900737f0 msgr2=0x7fa690075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6900737f0 0x7fa690075cb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0af000 tx=0x7fa69c01ae50 comp rx=0 tx=0).stop 2024-09-13T08:43:22.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 -- 172.21.15.44:0/3161424062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa698004810 msgr2=0x7fa6b409a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa698004810 0x7fa6b409a310 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4012620 tx=0x7fa6a4012ae0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 -- 172.21.15.44:0/3161424062 shutdown_connections 2024-09-13T08:43:22.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6900737f0 0x7fa690075cb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b4070190 0x7fa6b41f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6b41f7980 0x7fa6b409bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 --2- 172.21.15.44:0/3161424062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa698004810 0x7fa6b409a310 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 -- 172.21.15.44:0/3161424062 >> 172.21.15.44:0/3161424062 conn(0x7fa6b4073490 msgr2=0x7fa6b40ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:22.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 -- 172.21.15.44:0/3161424062 shutdown_connections 2024-09-13T08:43:22.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.231+0000 7fa6b3200640 1 -- 172.21.15.44:0/3161424062 wait complete. 2024-09-13T08:43:22.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T08:43:16.187506+0000', 'modified': '2024-09-13T08:43:19.249207+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5877}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5877': {'gid': 5877, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/2755626228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 2755626228}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2755626228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5877, 'qdb_cluster': [5877]} 2024-09-13T08:43:22.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T08:43:22.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T08:43:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.811+0000 7fe80a400640 1 -- 172.21.15.44:0/3578858222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e8004810 msgr2=0x7fe7e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.811+0000 7fe80a400640 1 --2- 172.21.15.44:0/3578858222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e8004810 0x7fe7e8004c90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009b70 tx=0x7fe7ec01d290 comp rx=0 tx=0).stop 2024-09-13T08:43:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.811+0000 7fe803e00640 1 -- 172.21.15.44:0/3578858222 reap_dead start 2024-09-13T08:43:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.811+0000 7fe80a400640 1 -- 172.21.15.44:0/3578858222 shutdown_connections 2024-09-13T08:43:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.811+0000 7fe80a400640 1 -- 172.21.15.44:0/3578858222 >> 172.21.15.44:0/3578858222 conn(0x7fe8040d1dc0 msgr2=0x7fe8040d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.811+0000 7fe80a400640 1 -- 172.21.15.44:0/3578858222 shutdown_connections 2024-09-13T08:43:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.811+0000 7fe80a400640 1 -- 172.21.15.44:0/3578858222 wait complete. 2024-09-13T08:43:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe80a400640 1 Processor -- start 2024-09-13T08:43:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe80a400640 1 -- start start 2024-09-13T08:43:22.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe80a400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7e8004810 0x7fe804158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe80a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8041555d0 0x7fe804154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe80a400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe804155110 0x7fe8040ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe80a400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe80415c470 con 0x7fe8041555d0 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe80a400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8040d2af0 con 0x7fe804155110 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe80a400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe8040aeef0 con 0x7fe7e8004810 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8041555d0 0x7fe804154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8041555d0 0x7fe804154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49594/0 (socket says 172.21.15.44:49594) 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fbe00640 1 -- 172.21.15.44:0/1778469503 learned_addr learned my addr 172.21.15.44:0/1778469503 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe803400640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7e8004810 0x7fe804158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe803e00640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe804155110 0x7fe8040ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fbe00640 1 -- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7e8004810 msgr2=0x7fe804158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7e8004810 0x7fe804158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fbe00640 1 -- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe804155110 msgr2=0x7fe8040ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe804155110 0x7fe8040ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fbe00640 1 -- 172.21.15.44:0/1778469503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8040b16a0 con 0x7fe8041555d0 2024-09-13T08:43:22.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe803e00640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe804155110 0x7fe8040ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:22.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8041555d0 0x7fe804154bd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0ae040 tx=0x7fe7ec0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:22.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fb400640 1 -- 172.21.15.44:0/1778469503 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec004490 con 0x7fe8041555d0 2024-09-13T08:43:22.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fb400640 1 -- 172.21.15.44:0/1778469503 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7ec0be070 con 0x7fe8041555d0 2024-09-13T08:43:22.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe7fb400640 1 -- 172.21.15.44:0/1778469503 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec004c90 con 0x7fe8041555d0 2024-09-13T08:43:22.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe80a400640 1 -- 172.21.15.44:0/1778469503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8040b1980 con 0x7fe8041555d0 2024-09-13T08:43:22.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.815+0000 7fe80a400640 1 -- 172.21.15.44:0/1778469503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8040b1f40 con 0x7fe8041555d0 2024-09-13T08:43:22.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe7fb400640 1 -- 172.21.15.44:0/1778469503 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fe7ec0b5040 con 0x7fe8041555d0 2024-09-13T08:43:22.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe7fb400640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe7e0073820 0x7fe7e0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe7fb400640 1 -- 172.21.15.44:0/1778469503 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fe7ec12dab0 con 0x7fe8041555d0 2024-09-13T08:43:22.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe80d870000 1 Processor -- start 2024-09-13T08:43:22.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803400640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe7e0073820 0x7fe7e0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe80d870000 1 -- start start 2024-09-13T08:43:22.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803400640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe7e0073820 0x7fe7e0075ce0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4001170 tx=0x7fe7f4001d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe80d870000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 0x556ba2a27db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe80d870000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a282f0 0x556ba29db8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe80d870000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29dc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe80d870000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556ba28fc3f0 con 0x556ba29dbe10 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe80d870000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ba2839b20 con 0x556ba2a279d0 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe80d870000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x556ba2828f50 con 0x556ba2a282f0 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29dc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29dc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49596/0 (socket says 172.21.15.44:49596) 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803e00640 1 -- 172.21.15.44:0/2382615815 learned_addr learned my addr 172.21.15.44:0/2382615815 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803400640 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 0x556ba2a27db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803e00640 1 -- 172.21.15.44:0/2382615815 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a282f0 msgr2=0x556ba29db8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:43:22.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a282f0 0x556ba29db8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803e00640 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a282f0 0x556ba29db8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803e00640 1 -- 172.21.15.44:0/2382615815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 msgr2=0x556ba2a27db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:43:22.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803e00640 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 0x556ba2a27db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803e00640 1 -- 172.21.15.44:0/2382615815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba27f0590 con 0x556ba29dbe10 2024-09-13T08:43:22.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe803400640 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 0x556ba2a27db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:43:22.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.819+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a282f0 0x556ba29db8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:43:22.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe803e00640 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29dc1f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe804074d10 tx=0x7fe7fc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71743a5d45b4f93 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:22.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe7f3e00640 1 -- 172.21.15.44:0/2382615815 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc012110 con 0x556ba29dbe10 2024-09-13T08:43:22.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe7f3e00640 1 -- 172.21.15.44:0/2382615815 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7fc003980 con 0x556ba29dbe10 2024-09-13T08:43:22.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe7f3e00640 1 -- 172.21.15.44:0/2382615815 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc0122b0 con 0x556ba29dbe10 2024-09-13T08:43:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 -- 172.21.15.44:0/2382615815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 msgr2=0x556ba29dc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29dc1f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe804074d10 tx=0x7fe7fc012820 comp rx=0 tx=0).stop 2024-09-13T08:43:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 -- 172.21.15.44:0/2382615815 shutdown_connections 2024-09-13T08:43:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29dc1f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a282f0 0x556ba29db8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 --2- 172.21.15.44:0/2382615815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 0x556ba2a27db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 -- 172.21.15.44:0/2382615815 >> 172.21.15.44:0/2382615815 conn(0x556ba28c5130 msgr2=0x556ba2a0ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 -- 172.21.15.44:0/2382615815 shutdown_connections 2024-09-13T08:43:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 -- 172.21.15.44:0/2382615815 wait complete. 2024-09-13T08:43:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 -- start start 2024-09-13T08:43:22.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 Processor -- start 2024-09-13T08:43:22.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba29dbe10 0x556ba2a086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 0x556ba2a08e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a282f0 0x556ba2a093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556ba27c35d0 con 0x556ba2a282f0 2024-09-13T08:43:22.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ba282cd60 con 0x556ba2a279d0 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x556ba27cf500 con 0x556ba29dbe10 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe80d870000 10 client.0 _ll_register_callbacks cb 0x556ba27d3810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe803e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a282f0 0x556ba2a093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe803e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a282f0 0x556ba2a093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49608/0 (socket says 172.21.15.44:49608) 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe803e00640 1 -- 172.21.15.44:0/3235312076 learned_addr learned my addr 172.21.15.44:0/3235312076 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe803400640 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba29dbe10 0x556ba2a086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 0x556ba2a08e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe803e00640 1 -- 172.21.15.44:0/3235312076 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba29dbe10 msgr2=0x556ba2a086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe803e00640 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba29dbe10 0x556ba2a086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe803e00640 1 -- 172.21.15.44:0/3235312076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 msgr2=0x556ba2a08e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe803e00640 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 0x556ba2a08e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.823+0000 7fe803e00640 1 -- 172.21.15.44:0/3235312076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba27c7170 con 0x556ba2a282f0 2024-09-13T08:43:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe803e00640 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a282f0 0x556ba2a093b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe8040732d0 tx=0x7fe7fc01cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ec3b316c962a3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe7f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3235312076 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc003980 con 0x556ba2a282f0 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3235312076 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fe7fc012070 con 0x556ba2a282f0 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 1 -- 172.21.15.44:0/3235312076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ba27c10f0 con 0x556ba2a282f0 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3235312076 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe7fc012270 con 0x556ba2a282f0 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 10 client.5991 fetch_fsmap learned FSMap version 37 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 1 -- 172.21.15.44:0/3235312076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x556ba28adef0 con 0x556ba2a282f0 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3235312076 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556ba27c10f0 con 0x556ba2a282f0 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3235312076 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7fe7fc023070 con 0x556ba2a282f0 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 10 client.5991 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 10 client.5991 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 20 client.5991 populate_metadata read hostname 'smithi044' 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.44:6833/2755626228,v1:172.21.15.44:6835/2755626228] conn(0x556ba2a0a060 0x556ba2a0a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 4 client.5991 mds_command: new command op to 5877 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T08:43:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 1 -- 172.21.15.44:0/3235312076 --> [v2:172.21.15.44:6833/2755626228,v1:172.21.15.44:6835/2755626228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ba283ba70 con 0x556ba2a0a060 2024-09-13T08:43:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe803400640 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.44:6833/2755626228,v1:172.21.15.44:6835/2755626228] conn(0x556ba2a0a060 0x556ba2a0a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe803400640 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.44:6833/2755626228,v1:172.21.15.44:6835/2755626228] conn(0x556ba2a0a060 0x556ba2a0a440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b00c63453dfc61 server_cookie=5074176c298c6257 in_seq=0 out_seq=0 2024-09-13T08:43:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe7f0c00640 10 client.5991 ms_handle_connect on v2:172.21.15.44:6833/2755626228 2024-09-13T08:43:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3235312076 <== mds.0 v2:172.21.15.44:6833/2755626228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ba283ba70 con 0x556ba2a0a060 2024-09-13T08:43:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe7f0c00640 10 client.5991 handle_command_reply: tid=0 2024-09-13T08:43:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 1 client.5991 shutdown 2024-09-13T08:43:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 1 -- 172.21.15.44:0/3235312076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a282f0 msgr2=0x556ba2a093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.827+0000 7fe80d870000 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a282f0 0x556ba2a093b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe8040732d0 tx=0x7fe7fc01cb60 comp rx=0 tx=0).stop 2024-09-13T08:43:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.831+0000 7fe80d870000 1 -- 172.21.15.44:0/3235312076 shutdown_connections 2024-09-13T08:43:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.831+0000 7fe80d870000 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.44:6833/2755626228,v1:172.21.15.44:6835/2755626228] conn(0x556ba2a0a060 0x556ba2a0a440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.831+0000 7fe80d870000 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a282f0 0x556ba2a093b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.831+0000 7fe80d870000 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a279d0 0x556ba2a08e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.831+0000 7fe80d870000 1 --2- 172.21.15.44:0/3235312076 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba29dbe10 0x556ba2a086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.831+0000 7fe80d870000 1 -- 172.21.15.44:0/3235312076 >> 172.21.15.44:0/3235312076 conn(0x556ba28c5130 msgr2=0x556ba29dafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.831+0000 7fe7f0c00640 0 client.5991 ms_handle_reset on v2:172.21.15.44:6833/2755626228 2024-09-13T08:43:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.831+0000 7fe80d870000 1 -- 172.21.15.44:0/3235312076 shutdown_connections 2024-09-13T08:43:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.831+0000 7fe80d870000 1 -- 172.21.15.44:0/3235312076 wait complete. 2024-09-13T08:43:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.831+0000 7fe80d870000 20 client.5991 trim_cache size 0 max 16384 2024-09-13T08:43:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 Processor -- start 2024-09-13T08:43:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 -- start start 2024-09-13T08:43:22.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a279d0 0x556ba29db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 0x556ba2a39c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8042018b0 con 0x556ba29dbe10 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041f51e0 con 0x556ba2a282f0 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe8041f4b90 con 0x556ba2a279d0 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29db0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49620/0 (socket says 172.21.15.44:49620) 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803400640 1 -- 172.21.15.44:0/640432396 learned_addr learned my addr 172.21.15.44:0/640432396 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a279d0 0x556ba29db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803e00640 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 0x556ba2a39c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803400640 1 -- 172.21.15.44:0/640432396 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a279d0 msgr2=0x556ba29db5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:43:22.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803400640 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a279d0 0x556ba29db5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803400640 1 -- 172.21.15.44:0/640432396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 msgr2=0x556ba2a39c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:43:22.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803400640 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 0x556ba2a39c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803400640 1 -- 172.21.15.44:0/640432396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba2884d90 con 0x556ba29dbe10 2024-09-13T08:43:22.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a279d0 0x556ba29db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:22.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803e00640 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 0x556ba2a39c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:22.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe803400640 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29db0b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4013b30 tx=0x7fe7f4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f108be35260388e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:22.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe7f3400640 1 -- 172.21.15.44:0/640432396 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4020b60 con 0x556ba29dbe10 2024-09-13T08:43:22.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe7f3400640 1 -- 172.21.15.44:0/640432396 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7f4020d00 con 0x556ba29dbe10 2024-09-13T08:43:22.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe7f3400640 1 -- 172.21.15.44:0/640432396 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f402a610 con 0x556ba29dbe10 2024-09-13T08:43:22.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 -- 172.21.15.44:0/640432396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 msgr2=0x556ba29db0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29db0b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4013b30 tx=0x7fe7f4016000 comp rx=0 tx=0).stop 2024-09-13T08:43:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 -- 172.21.15.44:0/640432396 shutdown_connections 2024-09-13T08:43:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 0x556ba2a39c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba2a279d0 0x556ba29db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 --2- 172.21.15.44:0/640432396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba29dbe10 0x556ba29db0b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 -- 172.21.15.44:0/640432396 >> 172.21.15.44:0/640432396 conn(0x556ba28c5130 msgr2=0x556ba29e3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 -- 172.21.15.44:0/640432396 shutdown_connections 2024-09-13T08:43:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 -- 172.21.15.44:0/640432396 wait complete. 2024-09-13T08:43:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 -- start start 2024-09-13T08:43:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.851+0000 7fe80d870000 1 Processor -- start 2024-09-13T08:43:22.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba29dbe10 0x556ba2a9df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a279d0 0x556ba2a9e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 0x556ba2a9e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556ba28388f0 con 0x556ba2a279d0 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ba28ada60 con 0x556ba2a282f0 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x556ba28e8730 con 0x556ba29dbe10 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 10 client.0 _ll_register_callbacks cb 0x7fe7fc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a279d0 0x556ba2a9e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a279d0 0x556ba2a9e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49628/0 (socket says 172.21.15.44:49628) 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7fbe00640 1 -- 172.21.15.44:0/3655747719 learned_addr learned my addr 172.21.15.44:0/3655747719 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe803400640 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba29dbe10 0x556ba2a9df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe803e00640 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 0x556ba2a9e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7fbe00640 1 -- 172.21.15.44:0/3655747719 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba29dbe10 msgr2=0x556ba2a9df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba29dbe10 0x556ba2a9df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7fbe00640 1 -- 172.21.15.44:0/3655747719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 msgr2=0x556ba2a9e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 0x556ba2a9e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7fbe00640 1 -- 172.21.15.44:0/3655747719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba27efa90 con 0x556ba2a279d0 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe803e00640 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 0x556ba2a9e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7fbe00640 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a279d0 0x556ba2a9e440 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009b70 tx=0x7fe7ec1012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9d93eaf43eb1f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3655747719 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0f9ac0 con 0x556ba2a279d0 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3655747719 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fe7ec103870 con 0x556ba2a279d0 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 -- 172.21.15.44:0/3655747719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f4020d00 con 0x556ba2a279d0 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3655747719 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe7ec0f98e0 con 0x556ba2a279d0 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 10 client.5997 fetch_fsmap learned FSMap version 37 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 -- 172.21.15.44:0/3655747719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x556ba27ce050 con 0x556ba2a279d0 2024-09-13T08:43:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3655747719 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7f4020d00 con 0x556ba2a279d0 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3655747719 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7fe7ec0f93d0 con 0x556ba2a279d0 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 10 client.5997 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 10 client.5997 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 20 client.5997 populate_metadata read hostname 'smithi044' 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.44:6833/2755626228,v1:172.21.15.44:6835/2755626228] conn(0x556ba2a3cd30 0x556ba2a3d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 4 client.5997 mds_command: new command op to 5877 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 -- 172.21.15.44:0/3655747719 --> [v2:172.21.15.44:6833/2755626228,v1:172.21.15.44:6835/2755626228] -- command(tid 0: {"prefix": "status"}) -- 0x556ba2837520 con 0x556ba2a3cd30 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe803400640 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.44:6833/2755626228,v1:172.21.15.44:6835/2755626228] conn(0x556ba2a3cd30 0x556ba2a3d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7f0c00640 10 client.5997 ms_handle_connect on v2:172.21.15.44:6833/2755626228 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe803400640 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.44:6833/2755626228,v1:172.21.15.44:6835/2755626228] conn(0x556ba2a3cd30 0x556ba2a3d110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1407e903d6b4a47 server_cookie=2e92486d5159ab39 in_seq=0 out_seq=0 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7f0c00640 1 -- 172.21.15.44:0/3655747719 <== mds.0 v2:172.21.15.44:6833/2755626228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x556ba2837520 con 0x556ba2a3cd30 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7f0c00640 10 client.5997 handle_command_reply: tid=0 2024-09-13T08:43:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 client.5997 shutdown 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 -- 172.21.15.44:0/3655747719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a279d0 msgr2=0x556ba2a9e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a279d0 0x556ba2a9e440 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009b70 tx=0x7fe7ec1012e0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 -- 172.21.15.44:0/3655747719 shutdown_connections 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.44:6833/2755626228,v1:172.21.15.44:6835/2755626228] conn(0x556ba2a3cd30 0x556ba2a3d110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ba2a282f0 0x556ba2a9e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ba2a279d0 0x556ba2a9e440 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 --2- 172.21.15.44:0/3655747719 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x556ba29dbe10 0x556ba2a9df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe7f0c00640 0 client.5997 ms_handle_reset on v2:172.21.15.44:6833/2755626228 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 -- 172.21.15.44:0/3655747719 >> 172.21.15.44:0/3655747719 conn(0x556ba28c5130 msgr2=0x556ba29da0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.855+0000 7fe80d870000 1 -- 172.21.15.44:0/3655747719 shutdown_connections 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80d870000 1 -- 172.21.15.44:0/3655747719 wait complete. 2024-09-13T08:43:22.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80d870000 20 client.5997 trim_cache size 0 max 16384 2024-09-13T08:43:22.866 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:43:22.866 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:43:22.866 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T08:43:22.866 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 5877, 2024-09-13T08:43:22.866 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T08:43:22.866 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T08:43:22.866 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T08:43:22.866 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.0760592400000002, 2024-09-13T08:43:22.867 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 37, 2024-09-13T08:43:22.867 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 61, 2024-09-13T08:43:22.867 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 61, 2024-09-13T08:43:22.867 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 10.491794621 2024-09-13T08:43:22.867 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:43:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 -- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe7e0073820 msgr2=0x7fe7e0075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe7e0073820 0x7fe7e0075ce0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4001170 tx=0x7fe7f4001d10 comp rx=0 tx=0).stop 2024-09-13T08:43:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 -- 172.21.15.44:0/1778469503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8041555d0 msgr2=0x7fe804154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8041555d0 0x7fe804154bd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0ae040 tx=0x7fe7ec0097d0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 -- 172.21.15.44:0/1778469503 shutdown_connections 2024-09-13T08:43:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe7e0073820 0x7fe7e0075ce0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe804155110 0x7fe8040ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8041555d0 0x7fe804154bd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 --2- 172.21.15.44:0/1778469503 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7e8004810 0x7fe804158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:22.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 -- 172.21.15.44:0/1778469503 >> 172.21.15.44:0/1778469503 conn(0x7fe8040d1dc0 msgr2=0x7fe8040720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:22.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 -- 172.21.15.44:0/1778469503 shutdown_connections 2024-09-13T08:43:22.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:22.859+0000 7fe80a400640 1 -- 172.21.15.44:0/1778469503 wait complete. 2024-09-13T08:43:22.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:43:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 -- 172.21.15.44:0/4014420352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62415dae0 msgr2=0x7fa6240acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 --2- 172.21.15.44:0/4014420352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62415dae0 0x7fa6240acd40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa624070d30 tx=0x7fa61401dad0 comp rx=0 tx=0).stop 2024-09-13T08:43:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 -- 172.21.15.44:0/4014420352 shutdown_connections 2024-09-13T08:43:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 --2- 172.21.15.44:0/4014420352 >> v2:172.21.15.129:3300/0 conn(0x7fa6240b49e0 0x7fa62406f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 --2- 172.21.15.44:0/4014420352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62415dae0 0x7fa6240acd40 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 -- 172.21.15.44:0/4014420352 >> 172.21.15.44:0/4014420352 conn(0x7fa6240b18f0 msgr2=0x7fa6240b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 -- 172.21.15.44:0/4014420352 shutdown_connections 2024-09-13T08:43:23.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 -- 172.21.15.44:0/4014420352 wait complete. 2024-09-13T08:43:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 Processor -- start 2024-09-13T08:43:23.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 -- start start 2024-09-13T08:43:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6240b49e0 0x7fa62409d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6240cd100 0x7fa624151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62415dae0 0x7fa6241f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240cf190 con 0x7fa62415dae0 2024-09-13T08:43:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240cf010 con 0x7fa6240cd100 2024-09-13T08:43:23.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa62a000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa6240cf310 con 0x7fa6240b49e0 2024-09-13T08:43:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa623e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62415dae0 0x7fa6241f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa623e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62415dae0 0x7fa6241f5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49644/0 (socket says 172.21.15.44:49644) 2024-09-13T08:43:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa623e00640 1 -- 172.21.15.44:0/2502202578 learned_addr learned my addr 172.21.15.44:0/2502202578 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa623400640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6240b49e0 0x7fa62409d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:23.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa61be00640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6240cd100 0x7fa624151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa623e00640 1 -- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6240b49e0 msgr2=0x7fa62409d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa623e00640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6240b49e0 0x7fa62409d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa623e00640 1 -- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6240cd100 msgr2=0x7fa624151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa623e00640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6240cd100 0x7fa624151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.031+0000 7fa623e00640 1 -- 172.21.15.44:0/2502202578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa624153790 con 0x7fa62415dae0 2024-09-13T08:43:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa623e00640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62415dae0 0x7fa6241f5670 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00e9f0 tx=0x7fa61c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:23.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa61b400640 1 -- 172.21.15.44:0/2502202578 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c00cd80 con 0x7fa62415dae0 2024-09-13T08:43:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa61b400640 1 -- 172.21.15.44:0/2502202578 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa61c004c20 con 0x7fa62415dae0 2024-09-13T08:43:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa62a000640 1 -- 172.21.15.44:0/2502202578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa624153a30 con 0x7fa62415dae0 2024-09-13T08:43:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa61b400640 1 -- 172.21.15.44:0/2502202578 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c005020 con 0x7fa62415dae0 2024-09-13T08:43:23.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa62a000640 1 -- 172.21.15.44:0/2502202578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6241542d0 con 0x7fa62415dae0 2024-09-13T08:43:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa62a000640 1 -- 172.21.15.44:0/2502202578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa624069410 con 0x7fa62415dae0 2024-09-13T08:43:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa61b400640 1 -- 172.21.15.44:0/2502202578 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7fa61c0051c0 con 0x7fa62415dae0 2024-09-13T08:43:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa61b400640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6000737c0 0x7fa600075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa61b400640 1 -- 172.21.15.44:0/2502202578 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fa61c092f60 con 0x7fa62415dae0 2024-09-13T08:43:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.035+0000 7fa623400640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6000737c0 0x7fa600075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.039+0000 7fa623400640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6000737c0 0x7fa600075c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa6140b0000 tx=0x7fa61401c310 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:23.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.039+0000 7fa61b400640 1 -- 172.21.15.44:0/2502202578 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa61c016030 con 0x7fa62415dae0 2024-09-13T08:43:23.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 -- 172.21.15.44:0/2502202578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa6240acd40 con 0x7fa62415dae0 2024-09-13T08:43:23.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa61b400640 1 -- 172.21.15.44:0/2502202578 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) ==== 53+0+73 (secure 0 0 0) 0x7fa61c05f5b0 con 0x7fa62415dae0 2024-09-13T08:43:23.254 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:43:23.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 -- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6000737c0 msgr2=0x7fa600075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:23.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6000737c0 0x7fa600075c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa6140b0000 tx=0x7fa61401c310 comp rx=0 tx=0).stop 2024-09-13T08:43:23.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 -- 172.21.15.44:0/2502202578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62415dae0 msgr2=0x7fa6241f5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:23.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62415dae0 0x7fa6241f5670 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00e9f0 tx=0x7fa61c00eeb0 comp rx=0 tx=0).stop 2024-09-13T08:43:23.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 -- 172.21.15.44:0/2502202578 shutdown_connections 2024-09-13T08:43:23.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6000737c0 0x7fa600075c80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:23.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62415dae0 0x7fa6241f5670 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:23.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6240cd100 0x7fa624151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:23.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 --2- 172.21.15.44:0/2502202578 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6240b49e0 0x7fa62409d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:23.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 -- 172.21.15.44:0/2502202578 >> 172.21.15.44:0/2502202578 conn(0x7fa6240b18f0 msgr2=0x7fa62406f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:23.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 -- 172.21.15.44:0/2502202578 shutdown_connections 2024-09-13T08:43:23.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23.251+0000 7fa62a000640 1 -- 172.21.15.44:0/2502202578 wait complete. 2024-09-13T08:43:23.274 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T08:43:23.274 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:43:23.274 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:43:23.274 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T08:43:23.274 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:43:23.274 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T08:43:23.274 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T08:43:23.275 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:43:23.275 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:43:23.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:23.275 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:43:23.284 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:43:23.284 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42079sec preferred_lft 42079sec 2024-09-13T08:43:23.285 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:43:23.286 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T08:43:23.287 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:43:23.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:23.288 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:43:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:43:23.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:23.349 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:43:23.406 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:43:23.406 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:43:23.406 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:43:23.406 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:43:23.407 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:43:23.407 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:43:23.407 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:43:23.407 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:43:23.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:43:23.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:43:23.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:43:23.407 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:43:23.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:43:23.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:43:23.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:23.544 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:43:23.557 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:43:23.557 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T08:43:23.557 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:43:23.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:23.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T08:43:23.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:43:23.620 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T08:43:23.676 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:43:23.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:43:23.677 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:43:23.728 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:43:23.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:23.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:43:23.823 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:43:23.823 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:43:23.823 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:43:23.838 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:43:23.839 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:43:23.839 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:43:23.839 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:43:23.839 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:43:23.851 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:43:23.851 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:43:23.851 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:43:23.851 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:43:23.851 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:43:23.852 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:43:23.852 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:43:23.852 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:43:23.855 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:43:23.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:23.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:43:23.875 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:43:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:43:23.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:43:23.983 INFO:teuthology.orchestra.run.smithi044.stdout:client5914 2024-09-13T08:43:23.984 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:43:23.984 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:23.993 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-13T08:43:24.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.187+0000 7f3494600640 1 -- 172.21.15.44:0/1081013911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c068960 msgr2=0x7f348c0ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:24.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.187+0000 7f3494600640 1 --2- 172.21.15.44:0/1081013911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c068960 0x7f348c0ccdc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3478009840 tx=0x7f347801c790 comp rx=0 tx=0).stop 2024-09-13T08:43:24.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.187+0000 7f3494600640 1 -- 172.21.15.44:0/1081013911 shutdown_connections 2024-09-13T08:43:24.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.187+0000 7f3494600640 1 --2- 172.21.15.44:0/1081013911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c068960 0x7f348c0ccdc0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:24.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.187+0000 7f3494600640 1 -- 172.21.15.44:0/1081013911 >> 172.21.15.44:0/1081013911 conn(0x7f348c0b2140 msgr2=0x7f348c0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:24.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.187+0000 7f3494600640 1 -- 172.21.15.44:0/1081013911 shutdown_connections 2024-09-13T08:43:24.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.187+0000 7f3494600640 1 -- 172.21.15.44:0/1081013911 wait complete. 2024-09-13T08:43:24.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 Processor -- start 2024-09-13T08:43:24.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 -- start start 2024-09-13T08:43:24.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f348c068960 0x7f348c0aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:24.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f348c0b3440 0x7f348c14eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:24.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c15be90 0x7f348c159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:24.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f348c16a6b0 con 0x7f348c15be90 2024-09-13T08:43:24.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f348c16a530 con 0x7f348c0b3440 2024-09-13T08:43:24.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f348c16a830 con 0x7f348c068960 2024-09-13T08:43:24.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3492600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c15be90 0x7f348c159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:24.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3492600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c15be90 0x7f348c159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49668/0 (socket says 172.21.15.44:49668) 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3492600640 1 -- 172.21.15.44:0/4243612397 learned_addr learned my addr 172.21.15.44:0/4243612397 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3491200640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f348c0b3440 0x7f348c14eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3491c00640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f348c068960 0x7f348c0aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3492600640 1 -- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f348c068960 msgr2=0x7f348c0aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3492600640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f348c068960 0x7f348c0aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3492600640 1 -- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f348c0b3440 msgr2=0x7f348c14eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3492600640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f348c0b3440 0x7f348c14eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3492600640 1 -- 172.21.15.44:0/4243612397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c0adad0 con 0x7f348c15be90 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3491c00640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f348c068960 0x7f348c0aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3491200640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f348c0b3440 0x7f348c14eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3492600640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c15be90 0x7f348c159490 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f348400c730 tx=0x7f348400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3489600640 1 -- 172.21.15.44:0/4243612397 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484003db0 con 0x7f348c15be90 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3489600640 1 -- 172.21.15.44:0/4243612397 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f348400ce40 con 0x7f348c15be90 2024-09-13T08:43:24.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3489600640 1 -- 172.21.15.44:0/4243612397 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34840052d0 con 0x7f348c15be90 2024-09-13T08:43:24.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 -- 172.21.15.44:0/4243612397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348c0addb0 con 0x7f348c15be90 2024-09-13T08:43:24.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 -- 172.21.15.44:0/4243612397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348c0ae370 con 0x7f348c15be90 2024-09-13T08:43:24.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.191+0000 7f3494600640 1 -- 172.21.15.44:0/4243612397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348c0ccdc0 con 0x7f348c15be90 2024-09-13T08:43:24.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.195+0000 7f3489600640 1 -- 172.21.15.44:0/4243612397 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f3484005470 con 0x7f348c15be90 2024-09-13T08:43:24.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.195+0000 7f3489600640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f346c0737c0 0x7f346c075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:24.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.195+0000 7f3489600640 1 -- 172.21.15.44:0/4243612397 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f3484092520 con 0x7f348c15be90 2024-09-13T08:43:24.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.195+0000 7f3491c00640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f346c0737c0 0x7f346c075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:24.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.195+0000 7f3491c00640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f346c0737c0 0x7f346c075c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3480000f70 tx=0x7f3480009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:24.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.199+0000 7f3489600640 1 -- 172.21.15.44:0/4243612397 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f348405f220 con 0x7f348c15be90 2024-09-13T08:43:24.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.411+0000 7f3494600640 1 -- 172.21.15.44:0/4243612397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f348c159230 con 0x7f348c15be90 2024-09-13T08:43:24.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.843+0000 7f3489600640 1 -- 172.21.15.44:0/4243612397 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v38) ==== 136+0+0 (secure 0 0 0) 0x7f348405eb20 con 0x7f348c15be90 2024-09-13T08:43:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 -- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f346c0737c0 msgr2=0x7f346c075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f346c0737c0 0x7f346c075c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3480000f70 tx=0x7f3480009040 comp rx=0 tx=0).stop 2024-09-13T08:43:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 -- 172.21.15.44:0/4243612397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c15be90 msgr2=0x7f348c159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c15be90 0x7f348c159490 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f348400c730 tx=0x7f348400cbf0 comp rx=0 tx=0).stop 2024-09-13T08:43:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 -- 172.21.15.44:0/4243612397 shutdown_connections 2024-09-13T08:43:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f346c0737c0 0x7f346c075c80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c15be90 0x7f348c159490 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f348c0b3440 0x7f348c14eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 --2- 172.21.15.44:0/4243612397 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f348c068960 0x7f348c0aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 -- 172.21.15.44:0/4243612397 >> 172.21.15.44:0/4243612397 conn(0x7f348c0b2140 msgr2=0x7f348c158360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 -- 172.21.15.44:0/4243612397 shutdown_connections 2024-09-13T08:43:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:24.847+0000 7f3494600640 1 -- 172.21.15.44:0/4243612397 wait complete. 2024-09-13T08:43:24.873 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:43:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 -- 172.21.15.44:0/870033526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af80b49e0 msgr2=0x7f3af80acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 --2- 172.21.15.44:0/870033526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af80b49e0 0x7f3af80acd40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009c80 tx=0x7f3af001cd10 comp rx=0 tx=0).stop 2024-09-13T08:43:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 -- 172.21.15.44:0/870033526 shutdown_connections 2024-09-13T08:43:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 --2- 172.21.15.44:0/870033526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af80b49e0 0x7f3af80acd40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 --2- 172.21.15.44:0/870033526 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3af80cd100 0x7f3af806f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 -- 172.21.15.44:0/870033526 >> 172.21.15.44:0/870033526 conn(0x7f3af80b18f0 msgr2=0x7f3af80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 -- 172.21.15.44:0/870033526 shutdown_connections 2024-09-13T08:43:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 -- 172.21.15.44:0/870033526 wait complete. 2024-09-13T08:43:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 Processor -- start 2024-09-13T08:43:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 -- start start 2024-09-13T08:43:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3af80b49e0 0x7f3af814a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80cd100 0x7f3af814cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af815dae0 0x7f3af809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3af81674b0 con 0x7f3af815dae0 2024-09-13T08:43:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3af8167330 con 0x7f3af80cd100 2024-09-13T08:43:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3b00000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3af8167630 con 0x7f3af80b49e0 2024-09-13T08:43:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3afe000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af815dae0 0x7f3af809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3afe000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af815dae0 0x7f3af809cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49678/0 (socket says 172.21.15.44:49678) 2024-09-13T08:43:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3afe000640 1 -- 172.21.15.44:0/4247949209 learned_addr learned my addr 172.21.15.44:0/4247949209 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3afd600640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3af80b49e0 0x7f3af814a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.019+0000 7f3afcc00640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80cd100 0x7f3af814cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3afe000640 1 -- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3af80b49e0 msgr2=0x7f3af814a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3afe000640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3af80b49e0 0x7f3af814a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3afe000640 1 -- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80cd100 msgr2=0x7f3af814cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3afe000640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80cd100 0x7f3af814cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3afe000640 1 -- 172.21.15.44:0/4247949209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af8070b90 con 0x7f3af815dae0 2024-09-13T08:43:25.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3afe000640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af815dae0 0x7f3af809cc20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009bd0 tx=0x7f3af0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:25.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3af5600640 1 -- 172.21.15.44:0/4247949209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af00bb070 con 0x7f3af815dae0 2024-09-13T08:43:25.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3af5600640 1 -- 172.21.15.44:0/4247949209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3af0004d40 con 0x7f3af815dae0 2024-09-13T08:43:25.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3b00000640 1 -- 172.21.15.44:0/4247949209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af814d130 con 0x7f3af815dae0 2024-09-13T08:43:25.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3af5600640 1 -- 172.21.15.44:0/4247949209 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af00b2040 con 0x7f3af815dae0 2024-09-13T08:43:25.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3b00000640 1 -- 172.21.15.44:0/4247949209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af8155a80 con 0x7f3af815dae0 2024-09-13T08:43:25.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3b00000640 1 -- 172.21.15.44:0/4247949209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af80acd40 con 0x7f3af815dae0 2024-09-13T08:43:25.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.023+0000 7f3af5600640 1 -- 172.21.15.44:0/4247949209 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f3af00be070 con 0x7f3af815dae0 2024-09-13T08:43:25.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.027+0000 7f3af5600640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ad80737c0 0x7f3ad8075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.027+0000 7f3af5600640 1 -- 172.21.15.44:0/4247949209 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f3af012c2f0 con 0x7f3af815dae0 2024-09-13T08:43:25.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.027+0000 7f3afd600640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ad80737c0 0x7f3ad8075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.027+0000 7f3afd600640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ad80737c0 0x7f3ad8075c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3af8153f00 tx=0x7f3aec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.031+0000 7f3af5600640 1 -- 172.21.15.44:0/4247949209 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3af0016990 con 0x7f3af815dae0 2024-09-13T08:43:25.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.227+0000 7f3b00000640 1 -- 172.21.15.44:0/4247949209 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f3af8156440 con 0x7f3ad80737c0 2024-09-13T08:43:25.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3af5600640 1 -- 172.21.15.44:0/4247949209 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3af8156440 con 0x7f3ad80737c0 2024-09-13T08:43:25.235 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:43:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:43:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:43:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:43:25.235 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:43:25.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 -- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ad80737c0 msgr2=0x7f3ad8075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ad80737c0 0x7f3ad8075c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3af8153f00 tx=0x7f3aec009040 comp rx=0 tx=0).stop 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 -- 172.21.15.44:0/4247949209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af815dae0 msgr2=0x7f3af809cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af815dae0 0x7f3af809cc20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009bd0 tx=0x7f3af0003f60 comp rx=0 tx=0).stop 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 -- 172.21.15.44:0/4247949209 shutdown_connections 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3ad80737c0 0x7f3ad8075c80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af815dae0 0x7f3af809cc20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80cd100 0x7f3af814cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 --2- 172.21.15.44:0/4247949209 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3af80b49e0 0x7f3af814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 -- 172.21.15.44:0/4247949209 >> 172.21.15.44:0/4247949209 conn(0x7f3af80b18f0 msgr2=0x7f3af815c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 -- 172.21.15.44:0/4247949209 shutdown_connections 2024-09-13T08:43:25.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.231+0000 7f3b00000640 1 -- 172.21.15.44:0/4247949209 wait complete. 2024-09-13T08:43:25.256 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:43:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.403+0000 7f72bae00640 1 -- 172.21.15.44:0/170128960 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f72b415dae0 msgr2=0x7f72b40acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.403+0000 7f72bae00640 1 --2- 172.21.15.44:0/170128960 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f72b415dae0 0x7f72b40acd40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f72a400aa10 tx=0x7f72a401dba0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.403+0000 7f72bae00640 1 -- 172.21.15.44:0/170128960 shutdown_connections 2024-09-13T08:43:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.403+0000 7f72bae00640 1 --2- 172.21.15.44:0/170128960 >> v2:172.21.15.129:3300/0 conn(0x7f72b40cd100 0x7f72b40b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.403+0000 7f72bae00640 1 --2- 172.21.15.44:0/170128960 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f72b415dae0 0x7f72b40acd40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.403+0000 7f72bae00640 1 -- 172.21.15.44:0/170128960 >> 172.21.15.44:0/170128960 conn(0x7f72b40b18f0 msgr2=0x7f72b40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.403+0000 7f72bae00640 1 -- 172.21.15.44:0/170128960 shutdown_connections 2024-09-13T08:43:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.403+0000 7f72bae00640 1 -- 172.21.15.44:0/170128960 wait complete. 2024-09-13T08:43:25.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.403+0000 7f72bae00640 1 Processor -- start 2024-09-13T08:43:25.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.403+0000 7f72bae00640 1 -- start start 2024-09-13T08:43:25.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72bae00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f72b40b4810 0x7f72b415c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72bae00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b40cd100 0x7f72b41f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72bae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72b415dae0 0x7f72b4153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72bae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41674b0 con 0x7f72b415dae0 2024-09-13T08:43:25.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72bae00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4167330 con 0x7f72b40cd100 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72bae00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f72b4167630 con 0x7f72b40b4810 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72b415dae0 0x7f72b4153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f72b40b4810 0x7f72b415c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72b415dae0 0x7f72b4153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49694/0 (socket says 172.21.15.44:49694) 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b8e00640 1 -- 172.21.15.44:0/266922074 learned_addr learned my addr 172.21.15.44:0/266922074 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72abe00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b40cd100 0x7f72b41f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b8e00640 1 -- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f72b40b4810 msgr2=0x7f72b415c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b8e00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f72b40b4810 0x7f72b415c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b8e00640 1 -- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b40cd100 msgr2=0x7f72b41f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b8e00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b40cd100 0x7f72b41f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b8e00640 1 -- 172.21.15.44:0/266922074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b415c970 con 0x7f72b415dae0 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b3e00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f72b40b4810 0x7f72b415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:43:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b8e00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72b415dae0 0x7f72b4153660 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00c7c0 tx=0x7f72ac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b0c00640 1 -- 172.21.15.44:0/266922074 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac003b80 con 0x7f72b415dae0 2024-09-13T08:43:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b0c00640 1 -- 172.21.15.44:0/266922074 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72ac003d20 con 0x7f72b415dae0 2024-09-13T08:43:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72b0c00640 1 -- 172.21.15.44:0/266922074 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00aa50 con 0x7f72b415dae0 2024-09-13T08:43:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72bae00640 1 -- 172.21.15.44:0/266922074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b41f8710 con 0x7f72b415dae0 2024-09-13T08:43:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72bae00640 1 -- 172.21.15.44:0/266922074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b41f8cd0 con 0x7f72b415dae0 2024-09-13T08:43:25.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.407+0000 7f72bae00640 1 -- 172.21.15.44:0/266922074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b4075be0 con 0x7f72b415dae0 2024-09-13T08:43:25.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.411+0000 7f72b0c00640 1 -- 172.21.15.44:0/266922074 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f72ac00abf0 con 0x7f72b415dae0 2024-09-13T08:43:25.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.411+0000 7f72b0c00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f72900737c0 0x7f7290075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.411+0000 7f72b0c00640 1 -- 172.21.15.44:0/266922074 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f72ac092440 con 0x7f72b415dae0 2024-09-13T08:43:25.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.411+0000 7f72b3e00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f72900737c0 0x7f7290075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.411+0000 7f72b3e00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f72900737c0 0x7f7290075c80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72a40b0000 tx=0x7f72a401c3e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.415+0000 7f72b0c00640 1 -- 172.21.15.44:0/266922074 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f72ac09a020 con 0x7f72b415dae0 2024-09-13T08:43:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 -- 172.21.15.44:0/266922074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f72b40adeb0 con 0x7f72b415dae0 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72b0c00640 1 -- 172.21.15.44:0/266922074 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f72ac05ea10 con 0x7f72b415dae0 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 -- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f72900737c0 msgr2=0x7f7290075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f72900737c0 0x7f7290075c80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72a40b0000 tx=0x7f72a401c3e0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 -- 172.21.15.44:0/266922074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72b415dae0 msgr2=0x7f72b4153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72b415dae0 0x7f72b4153660 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00c7c0 tx=0x7f72ac00cc80 comp rx=0 tx=0).stop 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 -- 172.21.15.44:0/266922074 shutdown_connections 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f72900737c0 0x7f7290075c80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72b415dae0 0x7f72b4153660 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b40cd100 0x7f72b41f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 --2- 172.21.15.44:0/266922074 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f72b40b4810 0x7f72b415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 -- 172.21.15.44:0/266922074 >> 172.21.15.44:0/266922074 conn(0x7f72b40b18f0 msgr2=0x7f72b4158880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:25.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 -- 172.21.15.44:0/266922074 shutdown_connections 2024-09-13T08:43:25.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.591+0000 7f72bae00640 1 -- 172.21.15.44:0/266922074 wait complete. 2024-09-13T08:43:25.617 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-13T08:43:25.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 -- 172.21.15.44:0/710665891 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c1c0ca480 msgr2=0x7f9c1c0ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 --2- 172.21.15.44:0/710665891 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c1c0ca480 0x7f9c1c0ca910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009bc0 tx=0x7f9c1401cb10 comp rx=0 tx=0).stop 2024-09-13T08:43:25.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 -- 172.21.15.44:0/710665891 shutdown_connections 2024-09-13T08:43:25.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 --2- 172.21.15.44:0/710665891 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c1c0ca480 0x7f9c1c0ca910 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 --2- 172.21.15.44:0/710665891 >> v2:172.21.15.129:3300/0 conn(0x7f9c1c0ce3b0 0x7f9c1c0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 -- 172.21.15.44:0/710665891 >> 172.21.15.44:0/710665891 conn(0x7f9c1c0cfc20 msgr2=0x7f9c1c0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:25.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 -- 172.21.15.44:0/710665891 shutdown_connections 2024-09-13T08:43:25.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 -- 172.21.15.44:0/710665891 wait complete. 2024-09-13T08:43:25.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 Processor -- start 2024-09-13T08:43:25.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 -- start start 2024-09-13T08:43:25.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.763+0000 7f9c22800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c1c0ca480 0x7f9c1c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c22800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c0ce3b0 0x7f9c1c14bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c22800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c151410 0x7f9c1c0738a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c22800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c0ad580 con 0x7f9c1c151410 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c22800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c0ad400 con 0x7f9c1c0ce3b0 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c22800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1c0ad700 con 0x7f9c1c0ca480 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c151410 0x7f9c1c0738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c13e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c0ce3b0 0x7f9c1c14bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c13e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c0ce3b0 0x7f9c1c14bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:34764/0 (socket says 172.21.15.44:34764) 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c1b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c1c0ca480 0x7f9c1c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c13e00640 1 -- 172.21.15.44:0/1640782214 learned_addr learned my addr 172.21.15.44:0/1640782214 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c151410 0x7f9c1c0738a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49698/0 (socket says 172.21.15.44:49698) 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c1be00640 1 -- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c1c0ca480 msgr2=0x7f9c1c154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c1be00640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c1c0ca480 0x7f9c1c154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c1be00640 1 -- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c0ce3b0 msgr2=0x7f9c1c14bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c1be00640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c0ce3b0 0x7f9c1c14bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c1be00640 1 -- 172.21.15.44:0/1640782214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c154860 con 0x7f9c1c151410 2024-09-13T08:43:25.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c1be00640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c151410 0x7f9c1c0738a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9c140ac000 tx=0x7f9c14009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:25.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c13400640 1 -- 172.21.15.44:0/1640782214 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c140bb070 con 0x7f9c1c151410 2024-09-13T08:43:25.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c13400640 1 -- 172.21.15.44:0/1640782214 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c14004d40 con 0x7f9c1c151410 2024-09-13T08:43:25.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c13400640 1 -- 172.21.15.44:0/1640782214 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c140b2040 con 0x7f9c1c151410 2024-09-13T08:43:25.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c22800640 1 -- 172.21.15.44:0/1640782214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c14c520 con 0x7f9c1c151410 2024-09-13T08:43:25.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c22800640 1 -- 172.21.15.44:0/1640782214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c152bf0 con 0x7f9c1c151410 2024-09-13T08:43:25.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c22800640 1 -- 172.21.15.44:0/1640782214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1c0ca910 con 0x7f9c1c151410 2024-09-13T08:43:25.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.767+0000 7f9c13400640 1 -- 172.21.15.44:0/1640782214 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f9c14004220 con 0x7f9c1c151410 2024-09-13T08:43:25.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.771+0000 7f9c13400640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bf80737c0 0x7f9bf8075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:25.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.771+0000 7f9c13400640 1 -- 172.21.15.44:0/1640782214 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f9c1412c450 con 0x7f9c1c151410 2024-09-13T08:43:25.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.771+0000 7f9c1b400640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bf80737c0 0x7f9bf8075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:25.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.771+0000 7f9c1b400640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bf80737c0 0x7f9bf8075c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c1f5430 tx=0x7f9c0c0013a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.775+0000 7f9c13400640 1 -- 172.21.15.44:0/1640782214 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9c14016790 con 0x7f9c1c151410 2024-09-13T08:43:25.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.951+0000 7f9c22800640 1 -- 172.21.15.44:0/1640782214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f9c1c069b00 con 0x7f9c1c151410 2024-09-13T08:43:25.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.951+0000 7f9c13400640 1 -- 172.21.15.44:0/1640782214 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v34) ==== 116+0+2 (secure 0 0 0) 0x7f9c140f8a20 con 0x7f9c1c151410 2024-09-13T08:43:25.955 INFO:teuthology.orchestra.run.smithi044.stdout:4 2024-09-13T08:43:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.951+0000 7f9c22800640 1 -- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bf80737c0 msgr2=0x7f9bf8075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.951+0000 7f9c22800640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bf80737c0 0x7f9bf8075c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c1f5430 tx=0x7f9c0c0013a0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.951+0000 7f9c22800640 1 -- 172.21.15.44:0/1640782214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c151410 msgr2=0x7f9c1c0738a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.951+0000 7f9c22800640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c151410 0x7f9c1c0738a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9c140ac000 tx=0x7f9c14009260 comp rx=0 tx=0).stop 2024-09-13T08:43:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.951+0000 7f9c22800640 1 -- 172.21.15.44:0/1640782214 shutdown_connections 2024-09-13T08:43:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.951+0000 7f9c22800640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9bf80737c0 0x7f9bf8075c80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.951+0000 7f9c22800640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c151410 0x7f9c1c0738a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.955+0000 7f9c22800640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c0ce3b0 0x7f9c1c14bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.955+0000 7f9c22800640 1 --2- 172.21.15.44:0/1640782214 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c1c0ca480 0x7f9c1c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.955+0000 7f9c22800640 1 -- 172.21.15.44:0/1640782214 >> 172.21.15.44:0/1640782214 conn(0x7f9c1c0cfc20 msgr2=0x7f9c1c0c8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.955+0000 7f9c22800640 1 -- 172.21.15.44:0/1640782214 shutdown_connections 2024-09-13T08:43:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:25.955+0000 7f9c22800640 1 -- 172.21.15.44:0/1640782214 wait complete. 2024-09-13T08:43:25.977 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:43:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.123+0000 7f2326c00640 1 -- 172.21.15.44:0/932496751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23201605d0 msgr2=0x7f23200d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.123+0000 7f2326c00640 1 --2- 172.21.15.44:0/932496751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23201605d0 0x7f23200d0690 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2310009840 tx=0x7f231001c820 comp rx=0 tx=0).stop 2024-09-13T08:43:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.123+0000 7f2326c00640 1 -- 172.21.15.44:0/932496751 shutdown_connections 2024-09-13T08:43:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.123+0000 7f2326c00640 1 --2- 172.21.15.44:0/932496751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23201605d0 0x7f23200d0690 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.123+0000 7f2326c00640 1 -- 172.21.15.44:0/932496751 >> 172.21.15.44:0/932496751 conn(0x7f23200d1dd0 msgr2=0x7f23200d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.123+0000 7f2326c00640 1 -- 172.21.15.44:0/932496751 shutdown_connections 2024-09-13T08:43:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.123+0000 7f2326c00640 1 -- 172.21.15.44:0/932496751 wait complete. 2024-09-13T08:43:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 Processor -- start 2024-09-13T08:43:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 -- start start 2024-09-13T08:43:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23200afb10 0x7f232006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23200b1650 0x7f23200b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23201605d0 0x7f23200b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23200b1220 con 0x7f23201605d0 2024-09-13T08:43:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23200b0ee0 con 0x7f23200afb10 2024-09-13T08:43:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f23200b13a0 con 0x7f23200b1650 2024-09-13T08:43:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2324c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23201605d0 0x7f23200b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2324c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23201605d0 0x7f23200b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49720/0 (socket says 172.21.15.44:49720) 2024-09-13T08:43:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2324c00640 1 -- 172.21.15.44:0/3069019255 learned_addr learned my addr 172.21.15.44:0/3069019255 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f231fe00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23200afb10 0x7f232006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2317e00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23200b1650 0x7f23200b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2324c00640 1 -- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23200b1650 msgr2=0x7f23200b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2324c00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23200b1650 0x7f23200b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2324c00640 1 -- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23200afb10 msgr2=0x7f232006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2324c00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23200afb10 0x7f232006f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2324c00640 1 -- 172.21.15.44:0/3069019255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23200b6c50 con 0x7f23201605d0 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2317e00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23200b1650 0x7f23200b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2324c00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23201605d0 0x7f23200b3d90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f231800c7c0 tx=0x7f231800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f231cc00640 1 -- 172.21.15.44:0/3069019255 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318003b80 con 0x7f23201605d0 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f231cc00640 1 -- 172.21.15.44:0/3069019255 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2318003d20 con 0x7f23201605d0 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f231cc00640 1 -- 172.21.15.44:0/3069019255 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231800aab0 con 0x7f23201605d0 2024-09-13T08:43:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 -- 172.21.15.44:0/3069019255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23200b6f30 con 0x7f23201605d0 2024-09-13T08:43:26.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 -- 172.21.15.44:0/3069019255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232015d1c0 con 0x7f23201605d0 2024-09-13T08:43:26.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.127+0000 7f2326c00640 1 -- 172.21.15.44:0/3069019255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232015d3c0 con 0x7f23201605d0 2024-09-13T08:43:26.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.131+0000 7f231cc00640 1 -- 172.21.15.44:0/3069019255 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f231800ac50 con 0x7f23201605d0 2024-09-13T08:43:26.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.131+0000 7f231cc00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22fc0737c0 0x7f22fc075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.131+0000 7f231cc00640 1 -- 172.21.15.44:0/3069019255 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f23180924c0 con 0x7f23201605d0 2024-09-13T08:43:26.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.131+0000 7f231fe00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22fc0737c0 0x7f22fc075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:26.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.131+0000 7f231fe00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22fc0737c0 0x7f22fc075c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f23100af000 tx=0x7f231001b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:26.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.135+0000 7f231cc00640 1 -- 172.21.15.44:0/3069019255 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f232015d3c0 con 0x7f23201605d0 2024-09-13T08:43:26.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.311+0000 7f2326c00640 1 -- 172.21.15.44:0/3069019255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f23200d0690 con 0x7f23201605d0 2024-09-13T08:43:26.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.315+0000 7f231cc00640 1 -- 172.21.15.44:0/3069019255 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v35)=0 v35) ==== 131+0+0 (secure 0 0 0) 0x7f231805f160 con 0x7f23201605d0 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 -- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22fc0737c0 msgr2=0x7f22fc075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22fc0737c0 0x7f22fc075c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f23100af000 tx=0x7f231001b060 comp rx=0 tx=0).stop 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 -- 172.21.15.44:0/3069019255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23201605d0 msgr2=0x7f23200b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23201605d0 0x7f23200b3d90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f231800c7c0 tx=0x7f231800cc80 comp rx=0 tx=0).stop 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 -- 172.21.15.44:0/3069019255 shutdown_connections 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22fc0737c0 0x7f22fc075c80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23201605d0 0x7f23200b3d90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f23200b1650 0x7f23200b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 --2- 172.21.15.44:0/3069019255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23200afb10 0x7f232006f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 -- 172.21.15.44:0/3069019255 >> 172.21.15.44:0/3069019255 conn(0x7f23200d1dd0 msgr2=0x7f23200b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 -- 172.21.15.44:0/3069019255 shutdown_connections 2024-09-13T08:43:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.319+0000 7f2326c00640 1 -- 172.21.15.44:0/3069019255 wait complete. 2024-09-13T08:43:26.340 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:43:26.338+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:43:26.340 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:43:26.338+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:43:26.340 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:43:26.338+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:43:26.340 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:43:26.338+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:43:26.340 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:43:26.338+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:43:26.342 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-13T08:43:26.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 -- 172.21.15.44:0/2502117489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153815aca0 msgr2=0x7f1538073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 --2- 172.21.15.44:0/2502117489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153815aca0 0x7f1538073120 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1520009900 tx=0x7f152001c8e0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 -- 172.21.15.44:0/2502117489 shutdown_connections 2024-09-13T08:43:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 --2- 172.21.15.44:0/2502117489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153815aca0 0x7f1538073120 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 --2- 172.21.15.44:0/2502117489 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153815a8c0 0x7f153815f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 -- 172.21.15.44:0/2502117489 >> 172.21.15.44:0/2502117489 conn(0x7f15380b2140 msgr2=0x7f15380b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 -- 172.21.15.44:0/2502117489 shutdown_connections 2024-09-13T08:43:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 -- 172.21.15.44:0/2502117489 wait complete. 2024-09-13T08:43:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 Processor -- start 2024-09-13T08:43:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 -- start start 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538073660 0x7f1538159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153815a8c0 0x7f15380b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153815aca0 0x7f15380b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1538167e80 con 0x7f153815aca0 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1538167d00 con 0x7f1538073660 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f153da00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1538168000 con 0x7f153815a8c0 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f1537400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153815aca0 0x7f15380b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f1537400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153815aca0 0x7f15380b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49734/0 (socket says 172.21.15.44:49734) 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f1537400640 1 -- 172.21.15.44:0/1796289294 learned_addr learned my addr 172.21.15.44:0/1796289294 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f1536a00640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538073660 0x7f1538159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.491+0000 7f152e000640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153815a8c0 0x7f15380b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f1537400640 1 -- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153815a8c0 msgr2=0x7f15380b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f1537400640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153815a8c0 0x7f15380b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f1537400640 1 -- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538073660 msgr2=0x7f1538159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f1537400640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538073660 0x7f1538159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f1537400640 1 -- 172.21.15.44:0/1796289294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1538158ca0 con 0x7f153815aca0 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f1537400640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153815aca0 0x7f15380b3d40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f153000c7c0 tx=0x7f153000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f152f400640 1 -- 172.21.15.44:0/1796289294 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530003b80 con 0x7f153815aca0 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f152f400640 1 -- 172.21.15.44:0/1796289294 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1530003d20 con 0x7f153815aca0 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f153da00640 1 -- 172.21.15.44:0/1796289294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15381557d0 con 0x7f153815aca0 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f152f400640 1 -- 172.21.15.44:0/1796289294 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153000aab0 con 0x7f153815aca0 2024-09-13T08:43:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f153da00640 1 -- 172.21.15.44:0/1796289294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1538155070 con 0x7f153815aca0 2024-09-13T08:43:26.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f153da00640 1 -- 172.21.15.44:0/1796289294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153815f100 con 0x7f153815aca0 2024-09-13T08:43:26.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f152f400640 1 -- 172.21.15.44:0/1796289294 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f153000ac50 con 0x7f153815aca0 2024-09-13T08:43:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.495+0000 7f152f400640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f15140737c0 0x7f1514075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.499+0000 7f152f400640 1 -- 172.21.15.44:0/1796289294 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f1530092410 con 0x7f153815aca0 2024-09-13T08:43:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.499+0000 7f1536a00640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f15140737c0 0x7f1514075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:26.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.499+0000 7f1536a00640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f15140737c0 0x7f1514075c80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1538151490 tx=0x7f152800b410 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:26.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.499+0000 7f152f400640 1 -- 172.21.15.44:0/1796289294 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f153815f640 con 0x7f153815aca0 2024-09-13T08:43:26.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.703+0000 7f153da00640 1 -- 172.21.15.44:0/1796289294 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f1538068e90 con 0x7f15140737c0 2024-09-13T08:43:26.718 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:43:26.718+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:43:26.718 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:43:26.718+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:43:26.718 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:43:26.718+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:43:26.719 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:43:26.718+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:43:26.719 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:43:26.718+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:43:26.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.747+0000 7f152f400640 1 -- 172.21.15.44:0/1796289294 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1538068e90 con 0x7f15140737c0 2024-09-13T08:43:26.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 -- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f15140737c0 msgr2=0x7f1514075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f15140737c0 0x7f1514075c80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1538151490 tx=0x7f152800b410 comp rx=0 tx=0).stop 2024-09-13T08:43:26.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 -- 172.21.15.44:0/1796289294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153815aca0 msgr2=0x7f15380b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153815aca0 0x7f15380b3d40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f153000c7c0 tx=0x7f153000cc80 comp rx=0 tx=0).stop 2024-09-13T08:43:26.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 -- 172.21.15.44:0/1796289294 shutdown_connections 2024-09-13T08:43:26.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f15140737c0 0x7f1514075c80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153815aca0 0x7f15380b3d40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153815a8c0 0x7f15380b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 --2- 172.21.15.44:0/1796289294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538073660 0x7f1538159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 -- 172.21.15.44:0/1796289294 >> 172.21.15.44:0/1796289294 conn(0x7f15380b2140 msgr2=0x7f15380cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:26.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 -- 172.21.15.44:0/1796289294 shutdown_connections 2024-09-13T08:43:26.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.751+0000 7f153da00640 1 -- 172.21.15.44:0/1796289294 wait complete. 2024-09-13T08:43:26.774 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:43:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.919+0000 7f857ca00640 1 -- 172.21.15.44:0/3825207308 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85780cd860 msgr2=0x7f85780cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.919+0000 7f857ca00640 1 --2- 172.21.15.44:0/3825207308 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85780cd860 0x7f85780cb520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8570009c80 tx=0x7f857001cd10 comp rx=0 tx=0).stop 2024-09-13T08:43:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.919+0000 7f857ca00640 1 -- 172.21.15.44:0/3825207308 shutdown_connections 2024-09-13T08:43:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.919+0000 7f857ca00640 1 --2- 172.21.15.44:0/3825207308 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85780cd860 0x7f85780cb520 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.919+0000 7f857ca00640 1 --2- 172.21.15.44:0/3825207308 >> v2:172.21.15.129:3300/0 conn(0x7f85781518a0 0x7f857815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.919+0000 7f857ca00640 1 -- 172.21.15.44:0/3825207308 >> 172.21.15.44:0/3825207308 conn(0x7f85780b2680 msgr2=0x7f85780b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.919+0000 7f857ca00640 1 -- 172.21.15.44:0/3825207308 shutdown_connections 2024-09-13T08:43:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.919+0000 7f857ca00640 1 -- 172.21.15.44:0/3825207308 wait complete. 2024-09-13T08:43:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f857ca00640 1 Processor -- start 2024-09-13T08:43:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f857ca00640 1 -- start start 2024-09-13T08:43:26.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f857ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85780cd860 0x7f857814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f857ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85781518a0 0x7f857815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f857ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f857815e420 0x7f857815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f857ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8578167e50 con 0x7f85780cd860 2024-09-13T08:43:26.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f857ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8578167cd0 con 0x7f857815e420 2024-09-13T08:43:26.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f857ca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8578167fd0 con 0x7f85781518a0 2024-09-13T08:43:26.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f8576800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85780cd860 0x7f857814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:26.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f8576800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85780cd860 0x7f857814d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49738/0 (socket says 172.21.15.44:49738) 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f8576800640 1 -- 172.21.15.44:0/2389026516 learned_addr learned my addr 172.21.15.44:0/2389026516 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f8576800640 1 -- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85781518a0 msgr2=0x7f857815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f8576800640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85781518a0 0x7f857815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f8576800640 1 -- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f857815e420 msgr2=0x7f857815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f8576800640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f857815e420 0x7f857815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f8576800640 1 -- 172.21.15.44:0/2389026516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85781f8d70 con 0x7f85780cd860 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f8576800640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85780cd860 0x7f857814d860 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f856800ea10 tx=0x7f856800eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f856ea00640 1 -- 172.21.15.44:0/2389026516 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856800cde0 con 0x7f85780cd860 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f856ea00640 1 -- 172.21.15.44:0/2389026516 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8568004020 con 0x7f85780cd860 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f857ca00640 1 -- 172.21.15.44:0/2389026516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85781f9050 con 0x7f85780cd860 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f856ea00640 1 -- 172.21.15.44:0/2389026516 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568004470 con 0x7f85780cd860 2024-09-13T08:43:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.923+0000 7f857ca00640 1 -- 172.21.15.44:0/2389026516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85781f9610 con 0x7f85780cd860 2024-09-13T08:43:26.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.927+0000 7f857ca00640 1 -- 172.21.15.44:0/2389026516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85780cb520 con 0x7f85780cd860 2024-09-13T08:43:26.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.927+0000 7f856ea00640 1 -- 172.21.15.44:0/2389026516 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 97896+0+0 (secure 0 0 0) 0x7f8568003600 con 0x7f85780cd860 2024-09-13T08:43:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.927+0000 7f856ea00640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f85540737c0 0x7f8554075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:43:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.927+0000 7f856ea00640 1 -- 172.21.15.44:0/2389026516 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f8568093160 con 0x7f85780cd860 2024-09-13T08:43:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.927+0000 7f856fe00640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f85540737c0 0x7f8554075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:43:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.931+0000 7f856fe00640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f85540737c0 0x7f8554075c80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8560000f70 tx=0x7f85600079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:43:26.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:26.931+0000 7f856ea00640 1 -- 172.21.15.44:0/2389026516 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f856805fe00 con 0x7f85780cd860 2024-09-13T08:43:27.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.131+0000 7f857ca00640 1 -- 172.21.15.44:0/2389026516 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f857815d960 con 0x7f85540737c0 2024-09-13T08:43:27.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f856ea00640 1 -- 172.21.15.44:0/2389026516 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f857815d960 con 0x7f85540737c0 2024-09-13T08:43:27.142 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe 2024-09-13T08:43:27.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 -- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f85540737c0 msgr2=0x7f8554075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:27.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f85540737c0 0x7f8554075c80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8560000f70 tx=0x7f85600079c0 comp rx=0 tx=0).stop 2024-09-13T08:43:27.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 -- 172.21.15.44:0/2389026516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85780cd860 msgr2=0x7f857814d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:43:27.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85780cd860 0x7f857814d860 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f856800ea10 tx=0x7f856800eed0 comp rx=0 tx=0).stop 2024-09-13T08:43:27.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 -- 172.21.15.44:0/2389026516 shutdown_connections 2024-09-13T08:43:27.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f85540737c0 0x7f8554075c80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:27.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f857815e420 0x7f857815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:27.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85781518a0 0x7f857815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 --2- 172.21.15.44:0/2389026516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85780cd860 0x7f857814d860 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:43:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 -- 172.21.15.44:0/2389026516 >> 172.21.15.44:0/2389026516 conn(0x7f85780b2680 msgr2=0x7f85780b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:43:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 -- 172.21.15.44:0/2389026516 shutdown_connections 2024-09-13T08:43:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:43:27.139+0000 7f857ca00640 1 -- 172.21.15.44:0/2389026516 wait complete. 2024-09-13T08:43:27.163 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe 2024-09-13T08:43:27.163 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:43:27.163 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:27.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:27.172 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:43:31.720 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:43:31.724 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:43:36.700 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:43:36.700 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:43:36.700 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.47233 s, 113 MB/s 2024-09-13T08:43:36.701 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:43:36.701 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:36.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:36.710 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:43:46.373 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:43:46.374 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:43:46.374 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.60746 s, 112 MB/s 2024-09-13T08:43:46.375 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:43:46.375 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:46.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:46.384 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:43:55.996 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:43:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:43:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.56257 s, 112 MB/s 2024-09-13T08:43:55.998 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:43:55.998 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:43:56.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:43:56.007 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:44:01.769 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:44:01.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:44:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:44:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:44:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.50538 s, 113 MB/s 2024-09-13T08:44:05.564 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:44:05.565 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:44:05.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:44:05.574 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:44:15.396 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:44:15.396 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:44:15.396 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.77032 s, 110 MB/s 2024-09-13T08:44:15.397 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:44:15.397 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:44:15.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:44:15.407 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:44:24.871 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:44:24.871 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:44:24.871 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.41341 s, 114 MB/s 2024-09-13T08:44:24.872 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:44:24.873 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:44:24.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:44:24.882 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:44:31.840 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:44:31.845 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:44:34.249 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:44:34.249 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:44:34.250 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.3109 s, 115 MB/s 2024-09-13T08:44:34.251 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:44:34.251 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:44:34.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:44:34.260 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:44:43.804 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:44:43.804 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:44:43.804 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.49086 s, 113 MB/s 2024-09-13T08:44:43.805 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:44:43.806 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:44:43.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:44:43.815 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:44:53.789 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:44:53.790 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:44:53.790 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.91989 s, 108 MB/s 2024-09-13T08:44:53.791 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:44:53.791 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:44:53.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:44:53.800 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:45:01.926 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:45:01.930 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:45:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:45:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:45:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.4072 s, 114 MB/s 2024-09-13T08:45:03.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-13T08:45:03.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.409+0000 7f5333c00640 1 -- 172.21.15.44:0/995510510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f532c0b0b60 msgr2=0x7f532c166eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:03.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.409+0000 7f5333c00640 1 --2- 172.21.15.44:0/995510510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f532c0b0b60 0x7f532c166eb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5324009bc0 tx=0x7f532401f3b0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.409+0000 7f5333c00640 1 -- 172.21.15.44:0/995510510 shutdown_connections 2024-09-13T08:45:03.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.409+0000 7f5333c00640 1 --2- 172.21.15.44:0/995510510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f532c0b0b60 0x7f532c166eb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.409+0000 7f5333c00640 1 -- 172.21.15.44:0/995510510 >> 172.21.15.44:0/995510510 conn(0x7f532c0d1da0 msgr2=0x7f532c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:03.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.409+0000 7f5333c00640 1 -- 172.21.15.44:0/995510510 shutdown_connections 2024-09-13T08:45:03.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.409+0000 7f5333c00640 1 -- 172.21.15.44:0/995510510 wait complete. 2024-09-13T08:45:03.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 Processor -- start 2024-09-13T08:45:03.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 -- start start 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f532c0b0b60 0x7f532c06f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f532c15be10 0x7f532c1488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f532c15c1f0 0x7f532c0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f532c167570 con 0x7f532c15c1f0 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1673f0 con 0x7f532c15be10 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f532c1676f0 con 0x7f532c0b0b60 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5331c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f532c15c1f0 0x7f532c0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5331c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f532c15c1f0 0x7f532c0c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53834/0 (socket says 172.21.15.44:53834) 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5331c00640 1 -- 172.21.15.44:0/1791479520 learned_addr learned my addr 172.21.15.44:0/1791479520 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f532be00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f532c15be10 0x7f532c1488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:03.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5331200640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f532c0b0b60 0x7f532c06f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:03.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5331c00640 1 -- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f532c0b0b60 msgr2=0x7f532c06f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:03.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5331c00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f532c0b0b60 0x7f532c06f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5331c00640 1 -- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f532c15be10 msgr2=0x7f532c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:03.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5331c00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f532c15be10 0x7f532c1488c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5331c00640 1 -- 172.21.15.44:0/1791479520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532c149470 con 0x7f532c15c1f0 2024-09-13T08:45:03.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5331c00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f532c15c1f0 0x7f532c0c78d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f532401cfe0 tx=0x7f53240af580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:03.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5328c00640 1 -- 172.21.15.44:0/1791479520 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53240b4040 con 0x7f532c15c1f0 2024-09-13T08:45:03.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5328c00640 1 -- 172.21.15.44:0/1791479520 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53240bf070 con 0x7f532c15c1f0 2024-09-13T08:45:03.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 -- 172.21.15.44:0/1791479520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532c06f7f0 con 0x7f532c15c1f0 2024-09-13T08:45:03.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5328c00640 1 -- 172.21.15.44:0/1791479520 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53240b09b0 con 0x7f532c15c1f0 2024-09-13T08:45:03.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 -- 172.21.15.44:0/1791479520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532c0b1780 con 0x7f532c15c1f0 2024-09-13T08:45:03.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.413+0000 7f5333c00640 1 -- 172.21.15.44:0/1791479520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f532c0ce700 con 0x7f532c15c1f0 2024-09-13T08:45:03.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.417+0000 7f5328c00640 1 -- 172.21.15.44:0/1791479520 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 97896+0+0 (secure 0 0 0) 0x7f53240b0b50 con 0x7f532c15c1f0 2024-09-13T08:45:03.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.417+0000 7f5328c00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f530c0737f0 0x7f530c075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:03.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.417+0000 7f5328c00640 1 -- 172.21.15.44:0/1791479520 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7f5324130870 con 0x7f532c15c1f0 2024-09-13T08:45:03.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.417+0000 7f5331200640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f530c0737f0 0x7f530c075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:03.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.417+0000 7f5331200640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f530c0737f0 0x7f530c075cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5320007fd0 tx=0x7f5320007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:03.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.421+0000 7f5328c00640 1 -- 172.21.15.44:0/1791479520 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5324018f10 con 0x7f532c15c1f0 2024-09-13T08:45:03.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.629+0000 7f5333c00640 1 -- 172.21.15.44:0/1791479520 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f532c070350 con 0x7f530c0737f0 2024-09-13T08:45:03.635 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:03.633+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:03.635 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:03.633+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:03.635 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:03.633+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:03.635 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:03.633+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:03.635 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:03.633+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:03.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.657+0000 7f5328c00640 1 -- 172.21.15.44:0/1791479520 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f532c070350 con 0x7f530c0737f0 2024-09-13T08:45:03.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 -- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f530c0737f0 msgr2=0x7f530c075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:03.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f530c0737f0 0x7f530c075cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5320007fd0 tx=0x7f5320007ea0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 -- 172.21.15.44:0/1791479520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f532c15c1f0 msgr2=0x7f532c0c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:03.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f532c15c1f0 0x7f532c0c78d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f532401cfe0 tx=0x7f53240af580 comp rx=0 tx=0).stop 2024-09-13T08:45:03.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 -- 172.21.15.44:0/1791479520 shutdown_connections 2024-09-13T08:45:03.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f530c0737f0 0x7f530c075cb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f532c15c1f0 0x7f532c0c78d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f532c15be10 0x7f532c1488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 --2- 172.21.15.44:0/1791479520 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f532c0b0b60 0x7f532c06f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 -- 172.21.15.44:0/1791479520 >> 172.21.15.44:0/1791479520 conn(0x7f532c0d1da0 msgr2=0x7f532c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:03.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 -- 172.21.15.44:0/1791479520 shutdown_connections 2024-09-13T08:45:03.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.661+0000 7f5333c00640 1 -- 172.21.15.44:0/1791479520 wait complete. 2024-09-13T08:45:03.683 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:45:03.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 -- 172.21.15.44:0/2148942211 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6be40cf1d0 msgr2=0x7f6be415fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:03.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 --2- 172.21.15.44:0/2148942211 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6be40cf1d0 0x7f6be415fd80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009840 tx=0x7f6bcc01c7f0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 -- 172.21.15.44:0/2148942211 shutdown_connections 2024-09-13T08:45:03.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 --2- 172.21.15.44:0/2148942211 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6be40cf1d0 0x7f6be415fd80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 --2- 172.21.15.44:0/2148942211 >> v2:172.21.15.129:3300/0 conn(0x7f6be415f9a0 0x7f6be406e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 -- 172.21.15.44:0/2148942211 >> 172.21.15.44:0/2148942211 conn(0x7f6be40b18f0 msgr2=0x7f6be40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:03.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 -- 172.21.15.44:0/2148942211 shutdown_connections 2024-09-13T08:45:03.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 -- 172.21.15.44:0/2148942211 wait complete. 2024-09-13T08:45:03.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 Processor -- start 2024-09-13T08:45:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 -- start start 2024-09-13T08:45:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6be40cf1d0 0x7f6be40aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6be415f9a0 0x7f6be4154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6be41602c0 0x7f6be41fb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4169720 con 0x7f6be41602c0 2024-09-13T08:45:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6be41695a0 con 0x7f6be415f9a0 2024-09-13T08:45:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.833+0000 7f6be9e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6be41698a0 con 0x7f6be40cf1d0 2024-09-13T08:45:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6be41602c0 0x7f6be41fb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6be41602c0 0x7f6be41fb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53836/0 (socket says 172.21.15.44:53836) 2024-09-13T08:45:03.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be3400640 1 -- 172.21.15.44:0/2453902872 learned_addr learned my addr 172.21.15.44:0/2453902872 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:03.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6bdbe00640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6be415f9a0 0x7f6be4154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:03.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be2a00640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6be40cf1d0 0x7f6be40aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:03.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be3400640 1 -- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6be40cf1d0 msgr2=0x7f6be40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:03.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be3400640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6be40cf1d0 0x7f6be40aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be3400640 1 -- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6be415f9a0 msgr2=0x7f6be4154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:03.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be3400640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6be415f9a0 0x7f6be4154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:03.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be3400640 1 -- 172.21.15.44:0/2453902872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be409cc30 con 0x7f6be41602c0 2024-09-13T08:45:03.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be3400640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6be41602c0 0x7f6be41fb4c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00c740 tx=0x7f6bdc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:03.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6bdb400640 1 -- 172.21.15.44:0/2453902872 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc003b80 con 0x7f6be41602c0 2024-09-13T08:45:03.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6bdb400640 1 -- 172.21.15.44:0/2453902872 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bdc00ce50 con 0x7f6be41602c0 2024-09-13T08:45:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6bdb400640 1 -- 172.21.15.44:0/2453902872 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00a7f0 con 0x7f6be41602c0 2024-09-13T08:45:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be9e00640 1 -- 172.21.15.44:0/2453902872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be409ac60 con 0x7f6be41602c0 2024-09-13T08:45:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be9e00640 1 -- 172.21.15.44:0/2453902872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be409b220 con 0x7f6be41602c0 2024-09-13T08:45:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6be9e00640 1 -- 172.21.15.44:0/2453902872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be4069410 con 0x7f6be41602c0 2024-09-13T08:45:03.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.837+0000 7f6bdb400640 1 -- 172.21.15.44:0/2453902872 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 97896+0+0 (secure 0 0 0) 0x7f6bdc00a990 con 0x7f6be41602c0 2024-09-13T08:45:03.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.841+0000 7f6bdb400640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6bc00737c0 0x7f6bc0075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.841+0000 7f6bdb400640 1 -- 172.21.15.44:0/2453902872 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7f6bdc092520 con 0x7f6be41602c0 2024-09-13T08:45:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.841+0000 7f6be2a00640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6bc00737c0 0x7f6bc0075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.841+0000 7f6be2a00640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6bc00737c0 0x7f6bc0075c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40096c0 tx=0x7f6bd4009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:03.845+0000 7f6bdb400640 1 -- 172.21.15.44:0/2453902872 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6be409cc30 con 0x7f6be41602c0 2024-09-13T08:45:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.045+0000 7f6be9e00640 1 -- 172.21.15.44:0/2453902872 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f6be409aa10 con 0x7f6bc00737c0 2024-09-13T08:45:04.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6bdb400640 1 -- 172.21.15.44:0/2453902872 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f6be409aa10 con 0x7f6bc00737c0 2024-09-13T08:45:04.054 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe 2024-09-13T08:45:04.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 -- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6bc00737c0 msgr2=0x7f6bc0075c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:04.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6bc00737c0 0x7f6bc0075c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40096c0 tx=0x7f6bd4009260 comp rx=0 tx=0).stop 2024-09-13T08:45:04.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 -- 172.21.15.44:0/2453902872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6be41602c0 msgr2=0x7f6be41fb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:04.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6be41602c0 0x7f6be41fb4c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00c740 tx=0x7f6bdc00cc00 comp rx=0 tx=0).stop 2024-09-13T08:45:04.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 -- 172.21.15.44:0/2453902872 shutdown_connections 2024-09-13T08:45:04.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6bc00737c0 0x7f6bc0075c80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:04.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6be41602c0 0x7f6be41fb4c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:04.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6be415f9a0 0x7f6be4154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:04.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 --2- 172.21.15.44:0/2453902872 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6be40cf1d0 0x7f6be40aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:04.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 -- 172.21.15.44:0/2453902872 >> 172.21.15.44:0/2453902872 conn(0x7f6be40b18f0 msgr2=0x7f6be415a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:04.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 -- 172.21.15.44:0/2453902872 shutdown_connections 2024-09-13T08:45:04.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:04.053+0000 7f6be9e00640 1 -- 172.21.15.44:0/2453902872 wait complete. 2024-09-13T08:45:04.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:04.076 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe 2024-09-13T08:45:04.076 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:45:04.085 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe 2024-09-13T08:45:04.085 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="10488365056" 2024-09-13T08:45:04.085 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:04.087 INFO:tasks.cephfs.test_volumes:r_size = 10488365056 exp_size = 10737418240 2024-09-13T08:45:06.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:06.088 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe 2024-09-13T08:45:06.088 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:45:06.099 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe 2024-09-13T08:45:06.099 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="10488365056" 2024-09-13T08:45:06.099 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:06.100 INFO:tasks.cephfs.test_volumes:r_size = 10488365056 exp_size = 10737418240 2024-09-13T08:45:08.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:08.101 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe 2024-09-13T08:45:08.101 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:45:08.112 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/1a3fa11e-765f-4bf2-872c-b6dd78fdf0fe 2024-09-13T08:45:08.112 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="10737418240" 2024-09-13T08:45:08.112 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:08.113 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-13T08:45:08.113 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-13T08:45:08.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.317+0000 7fd8bfe00640 1 -- 172.21.15.44:0/3159496006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c00d2810 msgr2=0x7fd8c0160350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:08.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.317+0000 7fd8bfe00640 1 --2- 172.21.15.44:0/3159496006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c00d2810 0x7fd8c0160350 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00d14c0 tx=0x7fd8a801c7d0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.317+0000 7fd8bfe00640 1 -- 172.21.15.44:0/3159496006 shutdown_connections 2024-09-13T08:45:08.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.317+0000 7fd8bfe00640 1 --2- 172.21.15.44:0/3159496006 >> v2:172.21.15.129:3300/0 conn(0x7fd8c00b25e0 0x7fd8c00af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.317+0000 7fd8bfe00640 1 --2- 172.21.15.44:0/3159496006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c00d2810 0x7fd8c0160350 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.317+0000 7fd8bfe00640 1 -- 172.21.15.44:0/3159496006 >> 172.21.15.44:0/3159496006 conn(0x7fd8c00d1cf0 msgr2=0x7fd8c00d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:08.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 -- 172.21.15.44:0/3159496006 shutdown_connections 2024-09-13T08:45:08.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 -- 172.21.15.44:0/3159496006 wait complete. 2024-09-13T08:45:08.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 Processor -- start 2024-09-13T08:45:08.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 -- start start 2024-09-13T08:45:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c00b25e0 0x7fd8c014aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c00cfd30 0x7fd8c015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd8c00d2810 0x7fd8c015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0168f10 con 0x7fd8c00b25e0 2024-09-13T08:45:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c00b07a0 con 0x7fd8c00cfd30 2024-09-13T08:45:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c0169090 con 0x7fd8c00d2810 2024-09-13T08:45:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c00b25e0 0x7fd8c014aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c00b25e0 0x7fd8c014aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53862/0 (socket says 172.21.15.44:53862) 2024-09-13T08:45:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bea00640 1 -- 172.21.15.44:0/1069157984 learned_addr learned my addr 172.21.15.44:0/1069157984 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:08.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8b7e00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c00cfd30 0x7fd8c015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:08.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bf400640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd8c00d2810 0x7fd8c015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:08.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bea00640 1 -- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd8c00d2810 msgr2=0x7fd8c015d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:08.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bea00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd8c00d2810 0x7fd8c015d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bea00640 1 -- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c00cfd30 msgr2=0x7fd8c015f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:08.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bea00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c00cfd30 0x7fd8c015f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bea00640 1 -- 172.21.15.44:0/1069157984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c014b9d0 con 0x7fd8c00b25e0 2024-09-13T08:45:08.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8b7e00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c00cfd30 0x7fd8c015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bea00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c00b25e0 0x7fd8c014aa90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000c490 tx=0x7fd8b000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8b6a00640 1 -- 172.21.15.44:0/1069157984 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0016020 con 0x7fd8c00b25e0 2024-09-13T08:45:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8b6a00640 1 -- 172.21.15.44:0/1069157984 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8b0012940 con 0x7fd8c00b25e0 2024-09-13T08:45:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8b6a00640 1 -- 172.21.15.44:0/1069157984 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0012d90 con 0x7fd8c00b25e0 2024-09-13T08:45:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.321+0000 7fd8bfe00640 1 -- 172.21.15.44:0/1069157984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c015c770 con 0x7fd8c00b25e0 2024-09-13T08:45:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.325+0000 7fd8bfe00640 1 -- 172.21.15.44:0/1069157984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c015cd30 con 0x7fd8c00b25e0 2024-09-13T08:45:08.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.325+0000 7fd8bfe00640 1 -- 172.21.15.44:0/1069157984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c00af540 con 0x7fd8c00b25e0 2024-09-13T08:45:08.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.325+0000 7fd8b6a00640 1 -- 172.21.15.44:0/1069157984 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 97947+0+0 (secure 0 0 0) 0x7fd8b0019020 con 0x7fd8c00b25e0 2024-09-13T08:45:08.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.325+0000 7fd8b6a00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd89c073850 0x7fd89c075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:08.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.325+0000 7fd8b6a00640 1 -- 172.21.15.44:0/1069157984 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7fd8b0092250 con 0x7fd8c00b25e0 2024-09-13T08:45:08.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.325+0000 7fd8b7e00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd89c073850 0x7fd89c075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:08.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.329+0000 7fd8b7e00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd89c073850 0x7fd89c075d10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80af000 tx=0x7fd8a801b010 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:08.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.329+0000 7fd8b6a00640 1 -- 172.21.15.44:0/1069157984 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd8b0020030 con 0x7fd8c00b25e0 2024-09-13T08:45:08.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.537+0000 7fd8bfe00640 1 -- 172.21.15.44:0/1069157984 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd8c00b4480 con 0x7fd89c073850 2024-09-13T08:45:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8b6a00640 1 -- 172.21.15.44:0/1069157984 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd8c00b4480 con 0x7fd89c073850 2024-09-13T08:45:08.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 -- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd89c073850 msgr2=0x7fd89c075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:08.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd89c073850 0x7fd89c075d10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80af000 tx=0x7fd8a801b010 comp rx=0 tx=0).stop 2024-09-13T08:45:08.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 -- 172.21.15.44:0/1069157984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c00b25e0 msgr2=0x7fd8c014aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:08.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c00b25e0 0x7fd8c014aa90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000c490 tx=0x7fd8b000c950 comp rx=0 tx=0).stop 2024-09-13T08:45:08.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 -- 172.21.15.44:0/1069157984 shutdown_connections 2024-09-13T08:45:08.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd89c073850 0x7fd89c075d10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd8c00d2810 0x7fd8c015d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c00cfd30 0x7fd8c015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 --2- 172.21.15.44:0/1069157984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c00b25e0 0x7fd8c014aa90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 -- 172.21.15.44:0/1069157984 >> 172.21.15.44:0/1069157984 conn(0x7fd8c00d1cf0 msgr2=0x7fd8c00b72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:08.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 -- 172.21.15.44:0/1069157984 shutdown_connections 2024-09-13T08:45:08.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.609+0000 7fd8bfe00640 1 -- 172.21.15.44:0/1069157984 wait complete. 2024-09-13T08:45:08.613 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:08.613+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:08.613 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:08.613+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:08.613 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:08.613+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:08.613 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:08.613+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:08.613 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:08.613+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:08.633 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:08.629+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:08.633 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:08.629+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:08.633 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:08.629+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:08.634 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:08.629+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:08.634 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:08.629+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:08.635 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:45:08.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.785+0000 7fb01da00640 1 -- 172.21.15.44:0/3180744331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc004810 msgr2=0x7faffc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:08.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.785+0000 7fb01da00640 1 --2- 172.21.15.44:0/3180744331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc004810 0x7faffc004c90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb000009d10 tx=0x7fb00001d3f0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.785+0000 7fb01da00640 1 -- 172.21.15.44:0/3180744331 shutdown_connections 2024-09-13T08:45:08.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.785+0000 7fb01da00640 1 --2- 172.21.15.44:0/3180744331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc004810 0x7faffc004c90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.785+0000 7fb01da00640 1 -- 172.21.15.44:0/3180744331 >> 172.21.15.44:0/3180744331 conn(0x7fb0180d1da0 msgr2=0x7fb0180d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:08.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.785+0000 7fb01da00640 1 -- 172.21.15.44:0/3180744331 shutdown_connections 2024-09-13T08:45:08.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.785+0000 7fb01da00640 1 -- 172.21.15.44:0/3180744331 wait complete. 2024-09-13T08:45:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.785+0000 7fb01da00640 1 Processor -- start 2024-09-13T08:45:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.785+0000 7fb01da00640 1 -- start start 2024-09-13T08:45:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb01da00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faffc004810 0x7fb0180b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb01da00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0181f0860 0x7fb0181f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb01da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0180b73f0 0x7fb0181563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb01da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb0180ceca0 con 0x7fb0180b73f0 2024-09-13T08:45:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb01da00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb0180ceb20 con 0x7faffc004810 2024-09-13T08:45:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb01da00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb0180cee20 con 0x7fb0181f0860 2024-09-13T08:45:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb017400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0180b73f0 0x7fb0181563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb017400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0180b73f0 0x7fb0181563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53878/0 (socket says 172.21.15.44:53878) 2024-09-13T08:45:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb017400640 1 -- 172.21.15.44:0/49089599 learned_addr learned my addr 172.21.15.44:0/49089599 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb016a00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faffc004810 0x7fb0180b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb00fe00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0181f0860 0x7fb0181f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb017400640 1 -- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0181f0860 msgr2=0x7fb0181f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb017400640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0181f0860 0x7fb0181f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb017400640 1 -- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faffc004810 msgr2=0x7fb0180b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb017400640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faffc004810 0x7fb0180b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb017400640 1 -- 172.21.15.44:0/49089599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0181f0dc0 con 0x7fb0180b73f0 2024-09-13T08:45:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb017400640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0180b73f0 0x7fb0181563e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb018074e00 tx=0x7fb010006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb00ea00640 1 -- 172.21.15.44:0/49089599 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010006910 con 0x7fb0180b73f0 2024-09-13T08:45:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb00ea00640 1 -- 172.21.15.44:0/49089599 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb010018070 con 0x7fb0180b73f0 2024-09-13T08:45:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb01da00640 1 -- 172.21.15.44:0/49089599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0181f10a0 con 0x7fb0180b73f0 2024-09-13T08:45:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb00ea00640 1 -- 172.21.15.44:0/49089599 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010011430 con 0x7fb0180b73f0 2024-09-13T08:45:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb01da00640 1 -- 172.21.15.44:0/49089599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0181f1660 con 0x7fb0180b73f0 2024-09-13T08:45:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb01da00640 1 -- 172.21.15.44:0/49089599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0180cf380 con 0x7fb0180b73f0 2024-09-13T08:45:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.789+0000 7fb00ea00640 1 -- 172.21.15.44:0/49089599 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 97947+0+0 (secure 0 0 0) 0x7fb0100115d0 con 0x7fb0180b73f0 2024-09-13T08:45:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.793+0000 7fb00ea00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faff4073880 0x7faff4075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.793+0000 7fb00ea00640 1 -- 172.21.15.44:0/49089599 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7fb010092bc0 con 0x7fb0180b73f0 2024-09-13T08:45:08.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.793+0000 7fb016a00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faff4073880 0x7faff4075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:08.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.793+0000 7fb016a00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faff4073880 0x7faff4075d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb008000c10 tx=0x7fb008006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:08.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.797+0000 7fb00ea00640 1 -- 172.21.15.44:0/49089599 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb01005f3c0 con 0x7fb0180b73f0 2024-09-13T08:45:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:08.873+0000 7fb00ea00640 1 -- 172.21.15.44:0/49089599 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 27) ==== 98049+0+0 (secure 0 0 0) 0x7fb01005b630 con 0x7fb0180b73f0 2024-09-13T08:45:09.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.041+0000 7fb01da00640 1 -- 172.21.15.44:0/49089599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb018070350 con 0x7fb0180b73f0 2024-09-13T08:45:09.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.041+0000 7fb00ea00640 1 -- 172.21.15.44:0/49089599 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1912 (secure 0 0 0) 0x7fb01007ef20 con 0x7fb0180b73f0 2024-09-13T08:45:09.044 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:09.044 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:45:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 479, 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:45:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 63, 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 2585, 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 10738572125, 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 21844377600, 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 751249735680, 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 85, 2024-09-13T08:45:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 92256206, 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 0, 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 23 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 38, 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:43:24:844013+0000", 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4, 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:45:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 4, 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 5877 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:45:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 13, 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:44:24.622159+0000", 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": {} 2024-09-13T08:45:09.052 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:45:09.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 -- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faff4073880 msgr2=0x7faff4075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faff4073880 0x7faff4075d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb008000c10 tx=0x7fb008006b80 comp rx=0 tx=0).stop 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 -- 172.21.15.44:0/49089599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0180b73f0 msgr2=0x7fb0181563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0180b73f0 0x7fb0181563e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb018074e00 tx=0x7fb010006720 comp rx=0 tx=0).stop 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 -- 172.21.15.44:0/49089599 shutdown_connections 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faff4073880 0x7faff4075d40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0180b73f0 0x7fb0181563e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0181f0860 0x7fb0181f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 --2- 172.21.15.44:0/49089599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faffc004810 0x7fb0180b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 -- 172.21.15.44:0/49089599 >> 172.21.15.44:0/49089599 conn(0x7fb0180d1da0 msgr2=0x7fb0180b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 -- 172.21.15.44:0/49089599 shutdown_connections 2024-09-13T08:45:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:09.045+0000 7fb01da00640 1 -- 172.21.15.44:0/49089599 wait complete. 2024-09-13T08:45:09.618 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:09.617+0000 7f8a89c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:09.618 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:09.617+0000 7f8a89c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:09.618 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:09.617+0000 7f8a89c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:09.619 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:09.617+0000 7f8a89c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:09.619 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:09.617+0000 7f8a89c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:10.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:45:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.297+0000 7fe528a00640 1 -- 172.21.15.44:0/2162291296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5240cc4e0 msgr2=0x7fe52406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.297+0000 7fe528a00640 1 --2- 172.21.15.44:0/2162291296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5240cc4e0 0x7fe52406d330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009900 tx=0x7fe50c01c8b0 comp rx=0 tx=0).stop 2024-09-13T08:45:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.297+0000 7fe528a00640 1 -- 172.21.15.44:0/2162291296 shutdown_connections 2024-09-13T08:45:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.297+0000 7fe528a00640 1 --2- 172.21.15.44:0/2162291296 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe5240ce880 0x7fe5240ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.297+0000 7fe528a00640 1 --2- 172.21.15.44:0/2162291296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5240cc4e0 0x7fe52406d330 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.297+0000 7fe528a00640 1 -- 172.21.15.44:0/2162291296 >> 172.21.15.44:0/2162291296 conn(0x7fe5240d0b00 msgr2=0x7fe5240d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.297+0000 7fe528a00640 1 -- 172.21.15.44:0/2162291296 shutdown_connections 2024-09-13T08:45:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.297+0000 7fe528a00640 1 -- 172.21.15.44:0/2162291296 wait complete. 2024-09-13T08:45:10.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.297+0000 7fe528a00640 1 Processor -- start 2024-09-13T08:45:10.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.297+0000 7fe528a00640 1 -- start start 2024-09-13T08:45:10.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe528a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5240cc4e0 0x7fe524160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:10.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe528a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe5240ce880 0x7fe5240aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:10.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe528a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5240d03a0 0x7fe5240adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:10.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe528a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe52416b050 con 0x7fe5240cc4e0 2024-09-13T08:45:10.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe528a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe52416aed0 con 0x7fe5240d03a0 2024-09-13T08:45:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe528a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe52416b1d0 con 0x7fe5240ce880 2024-09-13T08:45:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe522800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5240cc4e0 0x7fe524160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe522800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5240cc4e0 0x7fe524160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53890/0 (socket says 172.21.15.44:53890) 2024-09-13T08:45:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe522800640 1 -- 172.21.15.44:0/2275293567 learned_addr learned my addr 172.21.15.44:0/2275293567 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe522800640 1 -- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe5240ce880 msgr2=0x7fe5240aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe522800640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe5240ce880 0x7fe5240aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe522800640 1 -- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5240d03a0 msgr2=0x7fe5240adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe522800640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5240d03a0 0x7fe5240adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe522800640 1 -- 172.21.15.44:0/2275293567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5240af6b0 con 0x7fe5240cc4e0 2024-09-13T08:45:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe522800640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5240cc4e0 0x7fe524160390 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c3c0 tx=0x7fe51400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe51aa00640 1 -- 172.21.15.44:0/2275293567 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5140052d0 con 0x7fe5240cc4e0 2024-09-13T08:45:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe51aa00640 1 -- 172.21.15.44:0/2275293567 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe514007020 con 0x7fe5240cc4e0 2024-09-13T08:45:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe528a00640 1 -- 172.21.15.44:0/2275293567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5240af990 con 0x7fe5240cc4e0 2024-09-13T08:45:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe51aa00640 1 -- 172.21.15.44:0/2275293567 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514007470 con 0x7fe5240cc4e0 2024-09-13T08:45:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe528a00640 1 -- 172.21.15.44:0/2275293567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5241f7e50 con 0x7fe5240cc4e0 2024-09-13T08:45:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.301+0000 7fe528a00640 1 -- 172.21.15.44:0/2275293567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5240afeb0 con 0x7fe5240cc4e0 2024-09-13T08:45:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.305+0000 7fe51aa00640 1 -- 172.21.15.44:0/2275293567 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 98049+0+0 (secure 0 0 0) 0x7fe5140067e0 con 0x7fe5240cc4e0 2024-09-13T08:45:10.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.305+0000 7fe51aa00640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe500073970 0x7fe500075e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:10.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.305+0000 7fe51aa00640 1 -- 172.21.15.44:0/2275293567 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7fe514091e10 con 0x7fe5240cc4e0 2024-09-13T08:45:10.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.309+0000 7fe51be00640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe500073970 0x7fe500075e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:10.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.313+0000 7fe51be00640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe500073970 0x7fe500075e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0af000 tx=0x7fe50c019650 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:10.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.313+0000 7fe51aa00640 1 -- 172.21.15.44:0/2275293567 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe5240afeb0 con 0x7fe5240cc4e0 2024-09-13T08:45:10.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.649+0000 7fe528a00640 1 -- 172.21.15.44:0/2275293567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe524074810 con 0x7fe5240cc4e0 2024-09-13T08:45:10.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe51aa00640 1 -- 172.21.15.44:0/2275293567 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1912 (secure 0 0 0) 0x7fe51405e880 con 0x7fe5240cc4e0 2024-09-13T08:45:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:10.653 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:45:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:45:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 481, 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 63, 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:45:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 2585, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 10738572125, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 21844377600, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 751249735680, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 85, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 92256206, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 0, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 23 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:45:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 38, 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:43:24:844013+0000", 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4, 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 4, 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 5877 2024-09-13T08:45:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:45:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:45:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 13, 2024-09-13T08:45:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:44:24.622159+0000", 2024-09-13T08:45:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:45:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": {} 2024-09-13T08:45:10.660 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:45:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 -- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe500073970 msgr2=0x7fe500075e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe500073970 0x7fe500075e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0af000 tx=0x7fe50c019650 comp rx=0 tx=0).stop 2024-09-13T08:45:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 -- 172.21.15.44:0/2275293567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5240cc4e0 msgr2=0x7fe524160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5240cc4e0 0x7fe524160390 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c3c0 tx=0x7fe51400c880 comp rx=0 tx=0).stop 2024-09-13T08:45:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 -- 172.21.15.44:0/2275293567 shutdown_connections 2024-09-13T08:45:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe500073970 0x7fe500075e30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5240d03a0 0x7fe5240adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe5240ce880 0x7fe5240aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 --2- 172.21.15.44:0/2275293567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5240cc4e0 0x7fe524160390 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 -- 172.21.15.44:0/2275293567 >> 172.21.15.44:0/2275293567 conn(0x7fe5240d0b00 msgr2=0x7fe52406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 -- 172.21.15.44:0/2275293567 shutdown_connections 2024-09-13T08:45:10.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:10.653+0000 7fe528a00640 1 -- 172.21.15.44:0/2275293567 wait complete. 2024-09-13T08:45:11.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:45:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.881+0000 7fc66ae00640 1 -- 172.21.15.44:0/3587999436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc64c0039f0 msgr2=0x7fc64c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.881+0000 7fc66ae00640 1 --2- 172.21.15.44:0/3587999436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc64c0039f0 0x7fc64c003df0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc65800c2e0 tx=0x7fc65801cfb0 comp rx=0 tx=0).stop 2024-09-13T08:45:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.881+0000 7fc66a400640 1 -- 172.21.15.44:0/3587999436 reap_dead start 2024-09-13T08:45:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.881+0000 7fc66ae00640 1 -- 172.21.15.44:0/3587999436 shutdown_connections 2024-09-13T08:45:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.881+0000 7fc66ae00640 1 -- 172.21.15.44:0/3587999436 >> 172.21.15.44:0/3587999436 conn(0x7fc6640d0c20 msgr2=0x7fc6640d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.881+0000 7fc66ae00640 1 -- 172.21.15.44:0/3587999436 shutdown_connections 2024-09-13T08:45:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.881+0000 7fc66ae00640 1 -- 172.21.15.44:0/3587999436 wait complete. 2024-09-13T08:45:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc66ae00640 1 Processor -- start 2024-09-13T08:45:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc66ae00640 1 -- start start 2024-09-13T08:45:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc66ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc64c0039f0 0x7fc6641f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc66ae00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6640c3a00 0x7fc664071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc66ae00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc664071840 0x7fc6641f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc66ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641672c0 con 0x7fc64c0039f0 2024-09-13T08:45:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc66ae00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6640cf410 con 0x7fc6640c3a00 2024-09-13T08:45:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc66ae00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc664167440 con 0x7fc664071840 2024-09-13T08:45:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc669a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc64c0039f0 0x7fc6641f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc669a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc64c0039f0 0x7fc6641f8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53906/0 (socket says 172.21.15.44:53906) 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc669a00640 1 -- 172.21.15.44:0/1256579112 learned_addr learned my addr 172.21.15.44:0/1256579112 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc669a00640 1 -- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc664071840 msgr2=0x7fc6641f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.885+0000 7fc669a00640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc664071840 0x7fc6641f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc669a00640 1 -- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6640c3a00 msgr2=0x7fc664071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc669a00640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6640c3a00 0x7fc664071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc669a00640 1 -- 172.21.15.44:0/1256579112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6641f7ac0 con 0x7fc64c0039f0 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc669a00640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc64c0039f0 0x7fc6641f8b60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc65801ca70 tx=0x7fc658001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc661600640 1 -- 172.21.15.44:0/1256579112 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658003600 con 0x7fc64c0039f0 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc661600640 1 -- 172.21.15.44:0/1256579112 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc658003890 con 0x7fc64c0039f0 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc661600640 1 -- 172.21.15.44:0/1256579112 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658003ca0 con 0x7fc64c0039f0 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc66ae00640 1 -- 172.21.15.44:0/1256579112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6641f7da0 con 0x7fc64c0039f0 2024-09-13T08:45:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc66ae00640 1 -- 172.21.15.44:0/1256579112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6641f8360 con 0x7fc64c0039f0 2024-09-13T08:45:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc66ae00640 1 -- 172.21.15.44:0/1256579112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc664071c60 con 0x7fc64c0039f0 2024-09-13T08:45:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.889+0000 7fc661600640 1 -- 172.21.15.44:0/1256579112 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 28) ==== 98100+0+0 (secure 0 0 0) 0x7fc6580bf060 con 0x7fc64c0039f0 2024-09-13T08:45:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.893+0000 7fc661600640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc644073a30 0x7fc644075ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.893+0000 7fc661600640 1 -- 172.21.15.44:0/1256579112 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7fc65812cb60 con 0x7fc64c0039f0 2024-09-13T08:45:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.893+0000 7fc669000640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc644073a30 0x7fc644075ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.897+0000 7fc661600640 1 -- 172.21.15.44:0/1256579112 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc658016d90 con 0x7fc64c0039f0 2024-09-13T08:45:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:11.897+0000 7fc669000640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc644073a30 0x7fc644075ef0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc650007fd0 tx=0x7fc6500079a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:12.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.245+0000 7fc66ae00640 1 -- 172.21.15.44:0/1256579112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc6640ce7f0 con 0x7fc64c0039f0 2024-09-13T08:45:12.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.245+0000 7fc661600640 1 -- 172.21.15.44:0/1256579112 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2138 (secure 0 0 0) 0x7fc6580f9800 con 0x7fc64c0039f0 2024-09-13T08:45:12.248 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:12.248 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:45:12.248 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:45:12.248 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:45:12.248 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:45:12.248 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:45:12.248 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:45:12.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 482, 2024-09-13T08:45:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 63, 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:12.250 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 2586, 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 10738646955, 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 21873770496, 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 751220342784, 2024-09-13T08:45:12.251 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 85, 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 89466857, 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 0, 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 23 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 38, 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:43:24:844013+0000", 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4, 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:45:12.252 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 4, 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 5877 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:45:12.253 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:45:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:45:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:45:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:45:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:45:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:45:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:45:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:45:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:45:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 13, 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:44:24.622159+0000", 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": { 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-ongoing-clones": { 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "1 ongoing clones - average progress is 0.0% (0s)\n [............................] ", 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0, 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:12.255 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:45:12.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 -- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc644073a30 msgr2=0x7fc644075ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:12.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc644073a30 0x7fc644075ef0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc650007fd0 tx=0x7fc6500079a0 comp rx=0 tx=0).stop 2024-09-13T08:45:12.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 -- 172.21.15.44:0/1256579112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc64c0039f0 msgr2=0x7fc6641f8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:12.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc64c0039f0 0x7fc6641f8b60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc65801ca70 tx=0x7fc658001da0 comp rx=0 tx=0).stop 2024-09-13T08:45:12.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 -- 172.21.15.44:0/1256579112 shutdown_connections 2024-09-13T08:45:12.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc644073a30 0x7fc644075ef0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:12.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc664071840 0x7fc6641f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:12.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6640c3a00 0x7fc664071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:12.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 --2- 172.21.15.44:0/1256579112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc64c0039f0 0x7fc6641f8b60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:12.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 -- 172.21.15.44:0/1256579112 >> 172.21.15.44:0/1256579112 conn(0x7fc6640d0c20 msgr2=0x7fc6640d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:12.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 -- 172.21.15.44:0/1256579112 shutdown_connections 2024-09-13T08:45:12.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.249+0000 7fc66ae00640 1 -- 172.21.15.44:0/1256579112 wait complete. 2024-09-13T08:45:12.295 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-13T08:45:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.513+0000 7f98de000640 1 -- 172.21.15.44:0/2912428068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d815a1f0 msgr2=0x7f98d815df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.513+0000 7f98de000640 1 --2- 172.21.15.44:0/2912428068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d815a1f0 0x7f98d815df00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f98d80cdc80 tx=0x7f98cc01db30 comp rx=0 tx=0).stop 2024-09-13T08:45:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.513+0000 7f98de000640 1 -- 172.21.15.44:0/2912428068 shutdown_connections 2024-09-13T08:45:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.513+0000 7f98de000640 1 --2- 172.21.15.44:0/2912428068 >> v2:172.21.15.129:3300/0 conn(0x7f98d815a5d0 0x7f98d815e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.513+0000 7f98de000640 1 --2- 172.21.15.44:0/2912428068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d815a1f0 0x7f98d815df00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.513+0000 7f98de000640 1 -- 172.21.15.44:0/2912428068 >> 172.21.15.44:0/2912428068 conn(0x7f98d80742f0 msgr2=0x7f98d8074700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.513+0000 7f98de000640 1 -- 172.21.15.44:0/2912428068 shutdown_connections 2024-09-13T08:45:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.513+0000 7f98de000640 1 -- 172.21.15.44:0/2912428068 wait complete. 2024-09-13T08:45:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.517+0000 7f98de000640 1 Processor -- start 2024-09-13T08:45:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.517+0000 7f98de000640 1 -- start start 2024-09-13T08:45:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.517+0000 7f98de000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98d8099a70 0x7f98d809dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.517+0000 7f98de000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d815a1f0 0x7f98d8098940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.517+0000 7f98de000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d815a5d0 0x7f98d809c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.517+0000 7f98de000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80d0f10 con 0x7f98d815a1f0 2024-09-13T08:45:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.517+0000 7f98de000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80d0d90 con 0x7f98d815a5d0 2024-09-13T08:45:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.517+0000 7f98de000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f98d8167240 con 0x7f98d8099a70 2024-09-13T08:45:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d815a1f0 0x7f98d8098940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d815a1f0 0x7f98d8098940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53764/0 (socket says 172.21.15.44:53764) 2024-09-13T08:45:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d7e00640 1 -- 172.21.15.44:0/2533574358 learned_addr learned my addr 172.21.15.44:0/2533574358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98dd600640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d815a5d0 0x7f98d809c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d7e00640 1 -- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98d8099a70 msgr2=0x7f98d809dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d7e00640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98d8099a70 0x7f98d809dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d7e00640 1 -- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d815a5d0 msgr2=0x7f98d809c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d7e00640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d815a5d0 0x7f98d809c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d7e00640 1 -- 172.21.15.44:0/2533574358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d8159a70 con 0x7f98d815a1f0 2024-09-13T08:45:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d7e00640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d815a1f0 0x7f98d8098940 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f98c400e7a0 tx=0x7f98c400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:12.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d4c00640 1 -- 172.21.15.44:0/2533574358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c400cc60 con 0x7f98d815a1f0 2024-09-13T08:45:12.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d4c00640 1 -- 172.21.15.44:0/2533574358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98c4010430 con 0x7f98d815a1f0 2024-09-13T08:45:12.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98d4c00640 1 -- 172.21.15.44:0/2533574358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4010860 con 0x7f98d815a1f0 2024-09-13T08:45:12.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98de000640 1 -- 172.21.15.44:0/2533574358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d809cc20 con 0x7f98d815a1f0 2024-09-13T08:45:12.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.521+0000 7f98de000640 1 -- 172.21.15.44:0/2533574358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d809d1e0 con 0x7f98d815a1f0 2024-09-13T08:45:12.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.525+0000 7f98de000640 1 -- 172.21.15.44:0/2533574358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d80d01d0 con 0x7f98d815a1f0 2024-09-13T08:45:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.525+0000 7f98d4c00640 1 -- 172.21.15.44:0/2533574358 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 28) ==== 98100+0+0 (secure 0 0 0) 0x7f98c4010a00 con 0x7f98d815a1f0 2024-09-13T08:45:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.529+0000 7f98d4c00640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b8073a50 0x7f98b8075f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.529+0000 7f98d4c00640 1 -- 172.21.15.44:0/2533574358 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7f98c4093280 con 0x7f98d815a1f0 2024-09-13T08:45:12.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.529+0000 7f98dcc00640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b8073a50 0x7f98b8075f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:12.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.533+0000 7f98dcc00640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b8073a50 0x7f98b8075f10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0b0000 tx=0x7f98cc01a8d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:12.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.533+0000 7f98d4c00640 1 -- 172.21.15.44:0/2533574358 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f98c405f6c0 con 0x7f98d815a1f0 2024-09-13T08:45:12.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:12.857+0000 7f98de000640 1 -- 172.21.15.44:0/2533574358 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f98d80b61c0 con 0x7f98b8073a50 2024-09-13T08:45:13.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98d4c00640 1 -- 172.21.15.44:0/2533574358 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98d80b61c0 con 0x7f98b8073a50 2024-09-13T08:45:13.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 -- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b8073a50 msgr2=0x7f98b8075f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b8073a50 0x7f98b8075f10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0b0000 tx=0x7f98cc01a8d0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 -- 172.21.15.44:0/2533574358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d815a1f0 msgr2=0x7f98d8098940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d815a1f0 0x7f98d8098940 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f98c400e7a0 tx=0x7f98c400ec60 comp rx=0 tx=0).stop 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 -- 172.21.15.44:0/2533574358 shutdown_connections 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b8073a50 0x7f98b8075f10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d815a5d0 0x7f98d809c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d815a1f0 0x7f98d8098940 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 --2- 172.21.15.44:0/2533574358 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98d8099a70 0x7f98d809dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 -- 172.21.15.44:0/2533574358 >> 172.21.15.44:0/2533574358 conn(0x7f98d80742f0 msgr2=0x7f98d80c5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 -- 172.21.15.44:0/2533574358 shutdown_connections 2024-09-13T08:45:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.029+0000 7f98de000640 1 -- 172.21.15.44:0/2533574358 wait complete. 2024-09-13T08:45:13.052 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:45:13.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.197+0000 7fad7f800640 1 -- 172.21.15.44:0/2687852541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad78073170 msgr2=0x7fad7806f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.197+0000 7fad7f800640 1 --2- 172.21.15.44:0/2687852541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad78073170 0x7fad7806f520 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fad78069190 tx=0x7fad6401c7e0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.197+0000 7fad7f800640 1 -- 172.21.15.44:0/2687852541 shutdown_connections 2024-09-13T08:45:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.197+0000 7fad7f800640 1 --2- 172.21.15.44:0/2687852541 >> v2:172.21.15.129:3300/0 conn(0x7fad7806d330 0x7fad7806fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.197+0000 7fad7f800640 1 --2- 172.21.15.44:0/2687852541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad78073170 0x7fad7806f520 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.197+0000 7fad7f800640 1 -- 172.21.15.44:0/2687852541 >> 172.21.15.44:0/2687852541 conn(0x7fad780b2240 msgr2=0x7fad780b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.197+0000 7fad7f800640 1 -- 172.21.15.44:0/2687852541 shutdown_connections 2024-09-13T08:45:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7f800640 1 -- 172.21.15.44:0/2687852541 wait complete. 2024-09-13T08:45:13.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7f800640 1 Processor -- start 2024-09-13T08:45:13.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7f800640 1 -- start start 2024-09-13T08:45:13.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7f800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad7806d330 0x7fad7815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7f800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad78073170 0x7fad781f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7f800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad780ce800 0x7fad780aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7f800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad78168340 con 0x7fad7806d330 2024-09-13T08:45:13.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7f800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad781681c0 con 0x7fad780ce800 2024-09-13T08:45:13.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7f800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fad781684c0 con 0x7fad78073170 2024-09-13T08:45:13.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad7806d330 0x7fad7815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad7806d330 0x7fad7815d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53790/0 (socket says 172.21.15.44:53790) 2024-09-13T08:45:13.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7ce00640 1 -- 172.21.15.44:0/2074053676 learned_addr learned my addr 172.21.15.44:0/2074053676 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7d800640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad780ce800 0x7fad780aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad77e00640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad78073170 0x7fad781f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7ce00640 1 -- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad78073170 msgr2=0x7fad781f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7ce00640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad78073170 0x7fad781f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7ce00640 1 -- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad780ce800 msgr2=0x7fad780aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7ce00640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad780ce800 0x7fad780aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7ce00640 1 -- 172.21.15.44:0/2074053676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad781f8d40 con 0x7fad7806d330 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad77e00640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad78073170 0x7fad781f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7ce00640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad7806d330 0x7fad7815d790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c430 tx=0x7fad6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad74c00640 1 -- 172.21.15.44:0/2074053676 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c011790 con 0x7fad7806d330 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad74c00640 1 -- 172.21.15.44:0/2074053676 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad6c017070 con 0x7fad7806d330 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad7f800640 1 -- 172.21.15.44:0/2074053676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad780af280 con 0x7fad7806d330 2024-09-13T08:45:13.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.201+0000 7fad74c00640 1 -- 172.21.15.44:0/2074053676 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c011cc0 con 0x7fad7806d330 2024-09-13T08:45:13.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.205+0000 7fad7f800640 1 -- 172.21.15.44:0/2074053676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7815c9d0 con 0x7fad7806d330 2024-09-13T08:45:13.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.205+0000 7fad7f800640 1 -- 172.21.15.44:0/2074053676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7806cc20 con 0x7fad7806d330 2024-09-13T08:45:13.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.205+0000 7fad74c00640 1 -- 172.21.15.44:0/2074053676 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 28) ==== 98100+0+0 (secure 0 0 0) 0x7fad6c01c460 con 0x7fad7806d330 2024-09-13T08:45:13.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.205+0000 7fad74c00640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad58073a00 0x7fad58075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.205+0000 7fad74c00640 1 -- 172.21.15.44:0/2074053676 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7fad6c092540 con 0x7fad7806d330 2024-09-13T08:45:13.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.205+0000 7fad77e00640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad58073a00 0x7fad58075ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.205+0000 7fad77e00640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad58073a00 0x7fad58075ec0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fad640ae000 tx=0x7fad6401b020 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:13.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.209+0000 7fad74c00640 1 -- 172.21.15.44:0/2074053676 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fad781f8d40 con 0x7fad7806d330 2024-09-13T08:45:13.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.409+0000 7fad7f800640 1 -- 172.21.15.44:0/2074053676 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fad78144860 con 0x7fad58073a00 2024-09-13T08:45:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.409+0000 7fad74c00640 1 -- 172.21.15.44:0/2074053676 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fad78144860 con 0x7fad58073a00 2024-09-13T08:45:13.410 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:45:13.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:13.411 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:45:13.411 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:13.411 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.409+0000 7fad7f800640 1 -- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad58073a00 msgr2=0x7fad58075ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.409+0000 7fad7f800640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad58073a00 0x7fad58075ec0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fad640ae000 tx=0x7fad6401b020 comp rx=0 tx=0).stop 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.409+0000 7fad7f800640 1 -- 172.21.15.44:0/2074053676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad7806d330 msgr2=0x7fad7815d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.409+0000 7fad7f800640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad7806d330 0x7fad7815d790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c430 tx=0x7fad6c00c8f0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.409+0000 7fad7f800640 1 -- 172.21.15.44:0/2074053676 shutdown_connections 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.409+0000 7fad7f800640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad58073a00 0x7fad58075ec0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.409+0000 7fad7f800640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad780ce800 0x7fad780aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.409+0000 7fad7f800640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad78073170 0x7fad781f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.413+0000 7fad7f800640 1 --2- 172.21.15.44:0/2074053676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad7806d330 0x7fad7815d790 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.413+0000 7fad7f800640 1 -- 172.21.15.44:0/2074053676 >> 172.21.15.44:0/2074053676 conn(0x7fad780b2240 msgr2=0x7fad780b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.413+0000 7fad7f800640 1 -- 172.21.15.44:0/2074053676 shutdown_connections 2024-09-13T08:45:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.413+0000 7fad7f800640 1 -- 172.21.15.44:0/2074053676 wait complete. 2024-09-13T08:45:13.431 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-13T08:45:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 -- 172.21.15.44:0/3997787341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c0b3d00 msgr2=0x7fc65c0b5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 --2- 172.21.15.44:0/3997787341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c0b3d00 0x7fc65c0b5310 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc65c076f00 tx=0x7fc64401c800 comp rx=0 tx=0).stop 2024-09-13T08:45:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 -- 172.21.15.44:0/3997787341 shutdown_connections 2024-09-13T08:45:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 --2- 172.21.15.44:0/3997787341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c0b3d00 0x7fc65c0b5310 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 --2- 172.21.15.44:0/3997787341 >> v2:172.21.15.129:3300/0 conn(0x7fc65c15adf0 0x7fc65c15b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 -- 172.21.15.44:0/3997787341 >> 172.21.15.44:0/3997787341 conn(0x7fc65c0d0d90 msgr2=0x7fc65c0d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 -- 172.21.15.44:0/3997787341 shutdown_connections 2024-09-13T08:45:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 -- 172.21.15.44:0/3997787341 wait complete. 2024-09-13T08:45:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 Processor -- start 2024-09-13T08:45:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 -- start start 2024-09-13T08:45:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc65c0b3d00 0x7fc65c15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c0b5850 0x7fc65c0711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc65c15adf0 0x7fc65c0716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c167c30 con 0x7fc65c0b5850 2024-09-13T08:45:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c167ab0 con 0x7fc65c15adf0 2024-09-13T08:45:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc661a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc65c167db0 con 0x7fc65c0b3d00 2024-09-13T08:45:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc652000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c0b5850 0x7fc65c0711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc652000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c0b5850 0x7fc65c0711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53814/0 (socket says 172.21.15.44:53814) 2024-09-13T08:45:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc65aa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc65c0b3d00 0x7fc65c15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc652000640 1 -- 172.21.15.44:0/1777278941 learned_addr learned my addr 172.21.15.44:0/1777278941 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc65b400640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc65c15adf0 0x7fc65c0716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc652000640 1 -- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc65c0b3d00 msgr2=0x7fc65c15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc652000640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc65c0b3d00 0x7fc65c15ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc652000640 1 -- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc65c15adf0 msgr2=0x7fc65c0716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc652000640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc65c15adf0 0x7fc65c0716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.581+0000 7fc652000640 1 -- 172.21.15.44:0/1777278941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c15cfa0 con 0x7fc65c0b5850 2024-09-13T08:45:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.585+0000 7fc652000640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c0b5850 0x7fc65c0711b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc6440b0040 tx=0x7fc6440a55d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.585+0000 7fc653400640 1 -- 172.21.15.44:0/1777278941 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6440b2040 con 0x7fc65c0b5850 2024-09-13T08:45:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.585+0000 7fc653400640 1 -- 172.21.15.44:0/1777278941 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc644004490 con 0x7fc65c0b5850 2024-09-13T08:45:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.585+0000 7fc661a00640 1 -- 172.21.15.44:0/1777278941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c1fab20 con 0x7fc65c0b5850 2024-09-13T08:45:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.585+0000 7fc653400640 1 -- 172.21.15.44:0/1777278941 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644005280 con 0x7fc65c0b5850 2024-09-13T08:45:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.585+0000 7fc661a00640 1 -- 172.21.15.44:0/1777278941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c1fb0e0 con 0x7fc65c0b5850 2024-09-13T08:45:13.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.585+0000 7fc661a00640 1 -- 172.21.15.44:0/1777278941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc65c0b5310 con 0x7fc65c0b5850 2024-09-13T08:45:13.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.585+0000 7fc653400640 1 -- 172.21.15.44:0/1777278941 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 28) ==== 98100+0+0 (secure 0 0 0) 0x7fc644005420 con 0x7fc65c0b5850 2024-09-13T08:45:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.589+0000 7fc653400640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc638073a00 0x7fc638075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.589+0000 7fc653400640 1 -- 172.21.15.44:0/1777278941 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7fc64412c3c0 con 0x7fc65c0b5850 2024-09-13T08:45:13.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.589+0000 7fc65aa00640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc638073a00 0x7fc638075ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.589+0000 7fc65aa00640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc638073a00 0x7fc638075ec0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009b40 tx=0x7fc64c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:13.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.593+0000 7fc653400640 1 -- 172.21.15.44:0/1777278941 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc6440164b0 con 0x7fc65c0b5850 2024-09-13T08:45:13.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.793+0000 7fc661a00640 1 -- 172.21.15.44:0/1777278941 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc65c0ae9b0 con 0x7fc638073a00 2024-09-13T08:45:13.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.797+0000 7fc653400640 1 -- 172.21.15.44:0/1777278941 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7fc65c0ae9b0 con 0x7fc638073a00 2024-09-13T08:45:13.799 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:13.800 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:45:13.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:13.800 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sv1" 2024-09-13T08:45:13.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:13.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:13.800 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1clone1" 2024-09-13T08:45:13.800 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:13.800 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:45:13.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 -- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc638073a00 msgr2=0x7fc638075ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc638073a00 0x7fc638075ec0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009b40 tx=0x7fc64c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 -- 172.21.15.44:0/1777278941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c0b5850 msgr2=0x7fc65c0711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c0b5850 0x7fc65c0711b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc6440b0040 tx=0x7fc6440a55d0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 -- 172.21.15.44:0/1777278941 shutdown_connections 2024-09-13T08:45:13.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc638073a00 0x7fc638075ec0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc65c15adf0 0x7fc65c0716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c0b5850 0x7fc65c0711b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 --2- 172.21.15.44:0/1777278941 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc65c0b3d00 0x7fc65c15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 -- 172.21.15.44:0/1777278941 >> 172.21.15.44:0/1777278941 conn(0x7fc65c0d0d90 msgr2=0x7fc65c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 -- 172.21.15.44:0/1777278941 shutdown_connections 2024-09-13T08:45:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.801+0000 7fc661a00640 1 -- 172.21.15.44:0/1777278941 wait complete. 2024-09-13T08:45:13.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-13T08:45:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.973+0000 7f49d6000640 1 -- 172.21.15.44:0/2725532264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49d015a7e0 msgr2=0x7f49d00ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.973+0000 7f49d6000640 1 --2- 172.21.15.44:0/2725532264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49d015a7e0 0x7f49d00ae4d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009840 tx=0x7f49c001c820 comp rx=0 tx=0).stop 2024-09-13T08:45:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.973+0000 7f49d6000640 1 -- 172.21.15.44:0/2725532264 shutdown_connections 2024-09-13T08:45:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.973+0000 7f49d6000640 1 --2- 172.21.15.44:0/2725532264 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49d00aea10 0x7f49d00d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.973+0000 7f49d6000640 1 --2- 172.21.15.44:0/2725532264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49d015a7e0 0x7f49d00ae4d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.973+0000 7f49d6000640 1 -- 172.21.15.44:0/2725532264 >> 172.21.15.44:0/2725532264 conn(0x7f49d006dd30 msgr2=0x7f49d006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.973+0000 7f49d6000640 1 -- 172.21.15.44:0/2725532264 shutdown_connections 2024-09-13T08:45:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.973+0000 7f49d6000640 1 -- 172.21.15.44:0/2725532264 wait complete. 2024-09-13T08:45:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.973+0000 7f49d6000640 1 Processor -- start 2024-09-13T08:45:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.973+0000 7f49d6000640 1 -- start start 2024-09-13T08:45:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49d6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49d00aea10 0x7f49d01f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49d6000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49d015a7e0 0x7f49d01642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49d6000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49d015abc0 0x7f49d0144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49d6000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0167f60 con 0x7f49d00aea10 2024-09-13T08:45:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49d6000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0167de0 con 0x7f49d015abc0 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49d6000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f49d01680e0 con 0x7f49d015a7e0 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49d00aea10 0x7f49d01f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49d00aea10 0x7f49d01f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53820/0 (socket says 172.21.15.44:53820) 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cf400640 1 -- 172.21.15.44:0/3061054330 learned_addr learned my addr 172.21.15.44:0/3061054330 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49c7e00640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49d015a7e0 0x7f49d01642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cfe00640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49d015abc0 0x7f49d0144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cf400640 1 -- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49d015a7e0 msgr2=0x7f49d01642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cf400640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49d015a7e0 0x7f49d01642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cf400640 1 -- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49d015abc0 msgr2=0x7f49d0144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cf400640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49d015abc0 0x7f49d0144060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cf400640 1 -- 172.21.15.44:0/3061054330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d01fbe50 con 0x7f49d00aea10 2024-09-13T08:45:13.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49c7e00640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49d015a7e0 0x7f49d01642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:13.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cfe00640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49d015abc0 0x7f49d0144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:13.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49cf400640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49d00aea10 0x7f49d01f7920 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009790 tx=0x7f49c0005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:13.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49c7400640 1 -- 172.21.15.44:0/3061054330 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00b2040 con 0x7f49d00aea10 2024-09-13T08:45:13.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49c7400640 1 -- 172.21.15.44:0/3061054330 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c0004330 con 0x7f49d00aea10 2024-09-13T08:45:13.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49d6000640 1 -- 172.21.15.44:0/3061054330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d0156da0 con 0x7f49d00aea10 2024-09-13T08:45:13.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49c7400640 1 -- 172.21.15.44:0/3061054330 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0005220 con 0x7f49d00aea10 2024-09-13T08:45:13.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49d6000640 1 -- 172.21.15.44:0/3061054330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d01fb5b0 con 0x7f49d00aea10 2024-09-13T08:45:13.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49d6000640 1 -- 172.21.15.44:0/3061054330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d0146af0 con 0x7f49d00aea10 2024-09-13T08:45:13.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.977+0000 7f49c7400640 1 -- 172.21.15.44:0/3061054330 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 28) ==== 98100+0+0 (secure 0 0 0) 0x7f49c00053c0 con 0x7f49d00aea10 2024-09-13T08:45:13.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.981+0000 7f49c7400640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f49ac073a00 0x7f49ac075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:13.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.981+0000 7f49c7400640 1 -- 172.21.15.44:0/3061054330 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7f49c012ba80 con 0x7f49d00aea10 2024-09-13T08:45:13.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.981+0000 7f49c7e00640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f49ac073a00 0x7f49ac075ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:13.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.981+0000 7f49c7e00640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f49ac073a00 0x7f49ac075ec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f49d01f8a70 tx=0x7f49b8009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:13.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:13.985+0000 7f49c7400640 1 -- 172.21.15.44:0/3061054330 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f49d0146af0 con 0x7f49d00aea10 2024-09-13T08:45:14.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.189+0000 7f49d6000640 1 -- 172.21.15.44:0/3061054330 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49d006adb0 con 0x7f49ac073a00 2024-09-13T08:45:14.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.205+0000 7f49c7400640 1 -- 172.21.15.44:0/3061054330 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f49d006adb0 con 0x7f49ac073a00 2024-09-13T08:45:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:14.207 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:45:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1" 2024-09-13T08:45:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:14.207 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:45:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 -- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f49ac073a00 msgr2=0x7f49ac075ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f49ac073a00 0x7f49ac075ec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f49d01f8a70 tx=0x7f49b8009260 comp rx=0 tx=0).stop 2024-09-13T08:45:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 -- 172.21.15.44:0/3061054330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49d00aea10 msgr2=0x7f49d01f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49d00aea10 0x7f49d01f7920 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009790 tx=0x7f49c0005790 comp rx=0 tx=0).stop 2024-09-13T08:45:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 -- 172.21.15.44:0/3061054330 shutdown_connections 2024-09-13T08:45:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f49ac073a00 0x7f49ac075ec0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49d015abc0 0x7f49d0144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f49d015a7e0 0x7f49d01642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 --2- 172.21.15.44:0/3061054330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49d00aea10 0x7f49d01f7920 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 -- 172.21.15.44:0/3061054330 >> 172.21.15.44:0/3061054330 conn(0x7f49d006dd30 msgr2=0x7f49d00aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:14.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 -- 172.21.15.44:0/3061054330 shutdown_connections 2024-09-13T08:45:14.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.209+0000 7f49d6000640 1 -- 172.21.15.44:0/3061054330 wait complete. 2024-09-13T08:45:14.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-13T08:45:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 -- 172.21.15.44:0/179315614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c080b1650 msgr2=0x7f9c080b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 --2- 172.21.15.44:0/179315614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c080b1650 0x7f9c080b1a30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009900 tx=0x7f9bf401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 -- 172.21.15.44:0/179315614 shutdown_connections 2024-09-13T08:45:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 --2- 172.21.15.44:0/179315614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c080b1650 0x7f9c080b1a30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 --2- 172.21.15.44:0/179315614 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c081605d0 0x7f9c080d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 -- 172.21.15.44:0/179315614 >> 172.21.15.44:0/179315614 conn(0x7f9c080d1dd0 msgr2=0x7f9c080d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 -- 172.21.15.44:0/179315614 shutdown_connections 2024-09-13T08:45:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 -- 172.21.15.44:0/179315614 wait complete. 2024-09-13T08:45:14.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 Processor -- start 2024-09-13T08:45:14.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 -- start start 2024-09-13T08:45:14.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c080aeb00 0x7f9c080cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:14.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c080b1650 0x7f9c081f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:14.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c081605d0 0x7f9c080cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:14.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c080b1220 con 0x7f9c081605d0 2024-09-13T08:45:14.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c080b0ee0 con 0x7f9c080aeb00 2024-09-13T08:45:14.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9c080b13a0 con 0x7f9c080b1650 2024-09-13T08:45:14.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c081605d0 0x7f9c080cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:14.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c080aeb00 0x7f9c080cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c081605d0 0x7f9c080cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53838/0 (socket says 172.21.15.44:53838) 2024-09-13T08:45:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0de00640 1 -- 172.21.15.44:0/1626586330 learned_addr learned my addr 172.21.15.44:0/1626586330 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c080b1650 0x7f9c081f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0de00640 1 -- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c080b1650 msgr2=0x7f9c081f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0de00640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c080b1650 0x7f9c081f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0de00640 1 -- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c080aeb00 msgr2=0x7f9c080cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0de00640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c080aeb00 0x7f9c080cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.381+0000 7f9c0de00640 1 -- 172.21.15.44:0/1626586330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c080cf5b0 con 0x7f9c081605d0 2024-09-13T08:45:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.385+0000 7f9c0de00640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c081605d0 0x7f9c080cca40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000c780 tx=0x7f9c0000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.385+0000 7f9c05600640 1 -- 172.21.15.44:0/1626586330 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00004b70 con 0x7f9c081605d0 2024-09-13T08:45:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.385+0000 7f9c05600640 1 -- 172.21.15.44:0/1626586330 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c00004e00 con 0x7f9c081605d0 2024-09-13T08:45:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.385+0000 7f9c05600640 1 -- 172.21.15.44:0/1626586330 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000aa90 con 0x7f9c081605d0 2024-09-13T08:45:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.385+0000 7f9c0fe00640 1 -- 172.21.15.44:0/1626586330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c081f6360 con 0x7f9c081605d0 2024-09-13T08:45:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.385+0000 7f9c0fe00640 1 -- 172.21.15.44:0/1626586330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c081f6920 con 0x7f9c081605d0 2024-09-13T08:45:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.385+0000 7f9c0fe00640 1 -- 172.21.15.44:0/1626586330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c080b1a30 con 0x7f9c081605d0 2024-09-13T08:45:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.385+0000 7f9c05600640 1 -- 172.21.15.44:0/1626586330 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 28) ==== 98100+0+0 (secure 0 0 0) 0x7f9c0000ac30 con 0x7f9c081605d0 2024-09-13T08:45:14.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.385+0000 7f9c05600640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be8073a00 0x7f9be8075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:14.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.389+0000 7f9c05600640 1 -- 172.21.15.44:0/1626586330 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7f9c00092b80 con 0x7f9c081605d0 2024-09-13T08:45:14.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.389+0000 7f9c0d400640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be8073a00 0x7f9be8075ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:14.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.389+0000 7f9c0d400640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be8073a00 0x7f9be8075ec0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00ab40 tx=0x7f9bfc00a250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:14.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.389+0000 7f9c05600640 1 -- 172.21.15.44:0/1626586330 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9c00020070 con 0x7f9c081605d0 2024-09-13T08:45:14.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.597+0000 7f9c0fe00640 1 -- 172.21.15.44:0/1626586330 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c080d1080 con 0x7f9be8073a00 2024-09-13T08:45:14.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c05600640 1 -- 172.21.15.44:0/1626586330 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9c080d1080 con 0x7f9be8073a00 2024-09-13T08:45:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:14.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 -- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be8073a00 msgr2=0x7f9be8075ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:14.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be8073a00 0x7f9be8075ec0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00ab40 tx=0x7f9bfc00a250 comp rx=0 tx=0).stop 2024-09-13T08:45:14.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 -- 172.21.15.44:0/1626586330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c081605d0 msgr2=0x7f9c080cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:14.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c081605d0 0x7f9c080cca40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000c780 tx=0x7f9c0000cc40 comp rx=0 tx=0).stop 2024-09-13T08:45:14.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 -- 172.21.15.44:0/1626586330 shutdown_connections 2024-09-13T08:45:14.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be8073a00 0x7f9be8075ec0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c081605d0 0x7f9c080cca40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c080b1650 0x7f9c081f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 --2- 172.21.15.44:0/1626586330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c080aeb00 0x7f9c080cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 -- 172.21.15.44:0/1626586330 >> 172.21.15.44:0/1626586330 conn(0x7f9c080d1dd0 msgr2=0x7f9c080b7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:14.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 -- 172.21.15.44:0/1626586330 shutdown_connections 2024-09-13T08:45:14.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.633+0000 7f9c0fe00640 1 -- 172.21.15.44:0/1626586330 wait complete. 2024-09-13T08:45:14.656 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-13T08:45:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 -- 172.21.15.44:0/1554499115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde240ce620 msgr2=0x7fde240cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 --2- 172.21.15.44:0/1554499115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde240ce620 0x7fde240cfd30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fde240b7e60 tx=0x7fde1c01cc60 comp rx=0 tx=0).stop 2024-09-13T08:45:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 -- 172.21.15.44:0/1554499115 shutdown_connections 2024-09-13T08:45:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 --2- 172.21.15.44:0/1554499115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde240ce620 0x7fde240cfd30 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 --2- 172.21.15.44:0/1554499115 >> v2:172.21.15.129:3300/0 conn(0x7fde240b27a0 0x7fde240ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 -- 172.21.15.44:0/1554499115 >> 172.21.15.44:0/1554499115 conn(0x7fde240d1e30 msgr2=0x7fde240d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 -- 172.21.15.44:0/1554499115 shutdown_connections 2024-09-13T08:45:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 -- 172.21.15.44:0/1554499115 wait complete. 2024-09-13T08:45:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 Processor -- start 2024-09-13T08:45:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 -- start start 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde240b27a0 0x7fde2415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde240ce620 0x7fde241660e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde2415f640 0x7fde240c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fde240b1010 con 0x7fde240b27a0 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fde240b0e90 con 0x7fde2415f640 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde29a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fde24169390 con 0x7fde240ce620 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde22a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde240b27a0 0x7fde2415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde22a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde240b27a0 0x7fde2415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53858/0 (socket says 172.21.15.44:53858) 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde22a00640 1 -- 172.21.15.44:0/1850290215 learned_addr learned my addr 172.21.15.44:0/1850290215 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde1a000640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde240ce620 0x7fde241660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde23400640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde2415f640 0x7fde240c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde22a00640 1 -- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde240ce620 msgr2=0x7fde241660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde22a00640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde240ce620 0x7fde241660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde22a00640 1 -- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde2415f640 msgr2=0x7fde240c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde22a00640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde2415f640 0x7fde240c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde22a00640 1 -- 172.21.15.44:0/1850290215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde24165500 con 0x7fde240b27a0 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.805+0000 7fde1a000640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde240ce620 0x7fde241660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde22a00640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde240b27a0 0x7fde2415a1a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fde1400c3c0 tx=0x7fde1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde1b400640 1 -- 172.21.15.44:0/1850290215 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14017070 con 0x7fde240b27a0 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde1b400640 1 -- 172.21.15.44:0/1850290215 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde14003600 con 0x7fde240b27a0 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde29a00640 1 -- 172.21.15.44:0/1850290215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde241657e0 con 0x7fde240b27a0 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde1b400640 1 -- 172.21.15.44:0/1850290215 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14003a50 con 0x7fde240b27a0 2024-09-13T08:45:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde29a00640 1 -- 172.21.15.44:0/1850290215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde241640e0 con 0x7fde240b27a0 2024-09-13T08:45:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde29a00640 1 -- 172.21.15.44:0/1850290215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde24163ed0 con 0x7fde240b27a0 2024-09-13T08:45:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde1b400640 1 -- 172.21.15.44:0/1850290215 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 28) ==== 98100+0+0 (secure 0 0 0) 0x7fde1401a070 con 0x7fde240b27a0 2024-09-13T08:45:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde1b400640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde00073a00 0x7fde00075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde1b400640 1 -- 172.21.15.44:0/1850290215 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7fde14091c40 con 0x7fde240b27a0 2024-09-13T08:45:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.809+0000 7fde1a000640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde00073a00 0x7fde00075ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:14.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.813+0000 7fde1a000640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde00073a00 0x7fde00075ec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005fd0 tx=0x7fde0c005da0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:14.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:14.813+0000 7fde1b400640 1 -- 172.21.15.44:0/1850290215 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fde1405e690 con 0x7fde240b27a0 2024-09-13T08:45:15.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.017+0000 7fde29a00640 1 -- 172.21.15.44:0/1850290215 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fde240d1560 con 0x7fde00073a00 2024-09-13T08:45:15.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde1b400640 1 -- 172.21.15.44:0/1850290215 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde240d1560 con 0x7fde00073a00 2024-09-13T08:45:15.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde29a00640 1 -- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde00073a00 msgr2=0x7fde00075ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:15.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde29a00640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde00073a00 0x7fde00075ec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005fd0 tx=0x7fde0c005da0 comp rx=0 tx=0).stop 2024-09-13T08:45:15.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde29a00640 1 -- 172.21.15.44:0/1850290215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde240b27a0 msgr2=0x7fde2415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:15.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde29a00640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde240b27a0 0x7fde2415a1a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fde1400c3c0 tx=0x7fde1400c880 comp rx=0 tx=0).stop 2024-09-13T08:45:15.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde29a00640 1 -- 172.21.15.44:0/1850290215 shutdown_connections 2024-09-13T08:45:15.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde29a00640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fde00073a00 0x7fde00075ec0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:15.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde29a00640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde2415f640 0x7fde240c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:15.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde29a00640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fde240ce620 0x7fde241660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:15.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde29a00640 1 --2- 172.21.15.44:0/1850290215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde240b27a0 0x7fde2415a1a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:15.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.037+0000 7fde29a00640 1 -- 172.21.15.44:0/1850290215 >> 172.21.15.44:0/1850290215 conn(0x7fde240d1e30 msgr2=0x7fde2415b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:15.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.041+0000 7fde29a00640 1 -- 172.21.15.44:0/1850290215 shutdown_connections 2024-09-13T08:45:15.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.041+0000 7fde29a00640 1 -- 172.21.15.44:0/1850290215 wait complete. 2024-09-13T08:45:15.044 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:15.041+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:15.044 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:15.041+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:15.044 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:15.041+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:15.044 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:15.041+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:15.044 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:15.041+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:15.061 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-13T08:45:15.062 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:15.057+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:15.062 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:15.057+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:15.062 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:15.057+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:15.062 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:15.057+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:15.062 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:15.057+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.217+0000 7f4675200640 1 -- 172.21.15.44:0/3673655963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467015c1f0 msgr2=0x7f46700cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:15.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.217+0000 7f4675200640 1 --2- 172.21.15.44:0/3673655963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467015c1f0 0x7f46700cee10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f46700b0fb0 tx=0x7f465801eff0 comp rx=0 tx=0).stop 2024-09-13T08:45:15.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.217+0000 7f4675200640 1 -- 172.21.15.44:0/3673655963 shutdown_connections 2024-09-13T08:45:15.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.217+0000 7f4675200640 1 --2- 172.21.15.44:0/3673655963 >> v2:172.21.15.129:3300/0 conn(0x7f46700b0b60 0x7f46700ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:15.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.217+0000 7f4675200640 1 --2- 172.21.15.44:0/3673655963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467015c1f0 0x7f46700cee10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:15.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.217+0000 7f4675200640 1 -- 172.21.15.44:0/3673655963 >> 172.21.15.44:0/3673655963 conn(0x7f46700d1da0 msgr2=0x7f46700d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:15.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.217+0000 7f4675200640 1 -- 172.21.15.44:0/3673655963 shutdown_connections 2024-09-13T08:45:15.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.217+0000 7f4675200640 1 -- 172.21.15.44:0/3673655963 wait complete. 2024-09-13T08:45:15.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.217+0000 7f4675200640 1 Processor -- start 2024-09-13T08:45:15.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.217+0000 7f4675200640 1 -- start start 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4675200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46700b0b60 0x7f46700ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4675200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f467015be10 0x7f46700c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4675200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f467015c1f0 0x7f46700c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4675200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46700cd4a0 con 0x7f46700b0b60 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4675200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46700cd320 con 0x7f467015c1f0 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4675200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f46700cd620 con 0x7f467015be10 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f466de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46700b0b60 0x7f46700ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4667e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f467015be10 0x7f46700c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f466de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46700b0b60 0x7f46700ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53880/0 (socket says 172.21.15.44:53880) 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f466de00640 1 -- 172.21.15.44:0/96871746 learned_addr learned my addr 172.21.15.44:0/96871746 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f466e800640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f467015c1f0 0x7f46700c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f466de00640 1 -- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f467015be10 msgr2=0x7f46700c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:15.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f466de00640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f467015be10 0x7f46700c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:15.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f466de00640 1 -- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f467015c1f0 msgr2=0x7f46700c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:15.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f466de00640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f467015c1f0 0x7f46700c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:15.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f466de00640 1 -- 172.21.15.44:0/96871746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46700c6f00 con 0x7f46700b0b60 2024-09-13T08:45:15.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f466de00640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46700b0b60 0x7f46700ca720 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f466000c530 tx=0x7f466000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:15.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4666000640 1 -- 172.21.15.44:0/96871746 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660017070 con 0x7f46700b0b60 2024-09-13T08:45:15.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4666000640 1 -- 172.21.15.44:0/96871746 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4660003600 con 0x7f46700b0b60 2024-09-13T08:45:15.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4675200640 1 -- 172.21.15.44:0/96871746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46700c71e0 con 0x7f46700b0b60 2024-09-13T08:45:15.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4666000640 1 -- 172.21.15.44:0/96871746 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660003a50 con 0x7f46700b0b60 2024-09-13T08:45:15.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4675200640 1 -- 172.21.15.44:0/96871746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46700c77a0 con 0x7f46700b0b60 2024-09-13T08:45:15.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.221+0000 7f4675200640 1 -- 172.21.15.44:0/96871746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46700cdb40 con 0x7f46700b0b60 2024-09-13T08:45:15.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.225+0000 7f4666000640 1 -- 172.21.15.44:0/96871746 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 28) ==== 98100+0+0 (secure 0 0 0) 0x7f466001a070 con 0x7f46700b0b60 2024-09-13T08:45:15.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.225+0000 7f4666000640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f464c073a00 0x7f464c075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:15.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.225+0000 7f4666000640 1 -- 172.21.15.44:0/96871746 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7f4660091cd0 con 0x7f46700b0b60 2024-09-13T08:45:15.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.225+0000 7f4667e00640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f464c073a00 0x7f464c075ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:15.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.225+0000 7f4667e00640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f464c073a00 0x7f464c075ec0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f46580b2000 tx=0x7f465801d830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:15.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.229+0000 7f4666000640 1 -- 172.21.15.44:0/96871746 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f466005e1f0 con 0x7f46700b0b60 2024-09-13T08:45:15.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:15.429+0000 7f4675200640 1 -- 172.21.15.44:0/96871746 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f4670070350 con 0x7f464c073a00 2024-09-13T08:45:16.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.981+0000 7f4666000640 1 -- 172.21.15.44:0/96871746 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4670070350 con 0x7f464c073a00 2024-09-13T08:45:16.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 -- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f464c073a00 msgr2=0x7f464c075ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:16.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f464c073a00 0x7f464c075ec0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f46580b2000 tx=0x7f465801d830 comp rx=0 tx=0).stop 2024-09-13T08:45:16.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 -- 172.21.15.44:0/96871746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46700b0b60 msgr2=0x7f46700ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:16.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46700b0b60 0x7f46700ca720 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f466000c530 tx=0x7f466000c9f0 comp rx=0 tx=0).stop 2024-09-13T08:45:16.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 -- 172.21.15.44:0/96871746 shutdown_connections 2024-09-13T08:45:16.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f464c073a00 0x7f464c075ec0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:16.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f467015c1f0 0x7f46700c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:16.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f467015be10 0x7f46700c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:16.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 --2- 172.21.15.44:0/96871746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46700b0b60 0x7f46700ca720 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:16.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 -- 172.21.15.44:0/96871746 >> 172.21.15.44:0/96871746 conn(0x7f46700d1da0 msgr2=0x7f4670071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:16.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 -- 172.21.15.44:0/96871746 shutdown_connections 2024-09-13T08:45:16.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:16.985+0000 7f4675200640 1 -- 172.21.15.44:0/96871746 wait complete. 2024-09-13T08:45:17.015 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-13T08:45:17.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:45:17.073 INFO:teuthology.orchestra.run.smithi044.stdout:2 2024-09-13T08:45:17.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:45:17.080 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:45:17.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.233+0000 7fe913200640 1 -- 172.21.15.44:0/1126349369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9140b0a90 msgr2=0x7fe9140d2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:17.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.233+0000 7fe913200640 1 --2- 172.21.15.44:0/1126349369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9140b0a90 0x7fe9140d2240 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009900 tx=0x7fe8fc01c820 comp rx=0 tx=0).stop 2024-09-13T08:45:17.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.233+0000 7fe913200640 1 -- 172.21.15.44:0/1126349369 shutdown_connections 2024-09-13T08:45:17.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.233+0000 7fe913200640 1 --2- 172.21.15.44:0/1126349369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9140b0a90 0x7fe9140d2240 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:17.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.233+0000 7fe913200640 1 -- 172.21.15.44:0/1126349369 >> 172.21.15.44:0/1126349369 conn(0x7fe91409d0c0 msgr2=0x7fe9140d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:17.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.233+0000 7fe913200640 1 -- 172.21.15.44:0/1126349369 shutdown_connections 2024-09-13T08:45:17.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.233+0000 7fe913200640 1 -- 172.21.15.44:0/1126349369 wait complete. 2024-09-13T08:45:17.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 Processor -- start 2024-09-13T08:45:17.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 -- start start 2024-09-13T08:45:17.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9140b0a90 0x7fe9140b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:17.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9140d2780 0x7fe9140b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:17.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914161230 0x7fe91415f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:17.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe91416c0f0 con 0x7fe914161230 2024-09-13T08:45:17.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe91416bf70 con 0x7fe9140d2780 2024-09-13T08:45:17.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe91416c270 con 0x7fe9140b0a90 2024-09-13T08:45:17.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe912800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914161230 0x7fe91415f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:17.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe912800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914161230 0x7fe91415f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53910/0 (socket says 172.21.15.44:53910) 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe912800640 1 -- 172.21.15.44:0/2291299500 learned_addr learned my addr 172.21.15.44:0/2291299500 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe90be00640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9140d2780 0x7fe9140b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe911e00640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9140b0a90 0x7fe9140b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe912800640 1 -- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9140b0a90 msgr2=0x7fe9140b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe912800640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9140b0a90 0x7fe9140b6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe912800640 1 -- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9140d2780 msgr2=0x7fe9140b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe912800640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9140d2780 0x7fe9140b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe912800640 1 -- 172.21.15.44:0/2291299500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9140b8860 con 0x7fe914161230 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe912800640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914161230 0x7fe91415f130 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00c7a0 tx=0x7fe90c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe90a000640 1 -- 172.21.15.44:0/2291299500 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c017070 con 0x7fe914161230 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe90a000640 1 -- 172.21.15.44:0/2291299500 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe90c005070 con 0x7fe914161230 2024-09-13T08:45:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe90a000640 1 -- 172.21.15.44:0/2291299500 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0054c0 con 0x7fe914161230 2024-09-13T08:45:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 -- 172.21.15.44:0/2291299500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9140ce1a0 con 0x7fe914161230 2024-09-13T08:45:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 -- 172.21.15.44:0/2291299500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9140b7d80 con 0x7fe914161230 2024-09-13T08:45:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.237+0000 7fe913200640 1 -- 172.21.15.44:0/2291299500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9140d2240 con 0x7fe914161230 2024-09-13T08:45:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.241+0000 7fe90a000640 1 -- 172.21.15.44:0/2291299500 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fe90c01a070 con 0x7fe914161230 2024-09-13T08:45:17.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.241+0000 7fe90a000640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe8f0073b20 0x7fe8f0075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:17.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.241+0000 7fe90a000640 1 -- 172.21.15.44:0/2291299500 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7fe90c0921e0 con 0x7fe914161230 2024-09-13T08:45:17.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.241+0000 7fe911e00640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe8f0073b20 0x7fe8f0075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:17.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.241+0000 7fe911e00640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe8f0073b20 0x7fe8f0075fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe90400b810 tx=0x7fe904008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:17.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.245+0000 7fe90a000640 1 -- 172.21.15.44:0/2291299500 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe90c05eb70 con 0x7fe914161230 2024-09-13T08:45:17.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.421+0000 7fe913200640 1 -- 172.21.15.44:0/2291299500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fe91408d540 con 0x7fe914161230 2024-09-13T08:45:17.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.421+0000 7fe90a000640 1 -- 172.21.15.44:0/2291299500 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v35)=0 v35) ==== 131+0+0 (secure 0 0 0) 0x7fe90c05e470 con 0x7fe914161230 2024-09-13T08:45:17.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 -- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe8f0073b20 msgr2=0x7fe8f0075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:17.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe8f0073b20 0x7fe8f0075fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe90400b810 tx=0x7fe904008040 comp rx=0 tx=0).stop 2024-09-13T08:45:17.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 -- 172.21.15.44:0/2291299500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914161230 msgr2=0x7fe91415f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:17.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914161230 0x7fe91415f130 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00c7a0 tx=0x7fe90c00cc60 comp rx=0 tx=0).stop 2024-09-13T08:45:17.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 -- 172.21.15.44:0/2291299500 shutdown_connections 2024-09-13T08:45:17.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe8f0073b20 0x7fe8f0075fe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:17.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914161230 0x7fe91415f130 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:17.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9140d2780 0x7fe9140b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:17.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 --2- 172.21.15.44:0/2291299500 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9140b0a90 0x7fe9140b6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:17.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 -- 172.21.15.44:0/2291299500 >> 172.21.15.44:0/2291299500 conn(0x7fe91409d0c0 msgr2=0x7fe91406da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:17.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 -- 172.21.15.44:0/2291299500 shutdown_connections 2024-09-13T08:45:17.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.425+0000 7fe913200640 1 -- 172.21.15.44:0/2291299500 wait complete. 2024-09-13T08:45:17.446 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:45:17.446 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:45:17.571 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:45:17.676 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:45:17.676 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:45:17.676 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:45:17.676 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:17.685 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:45:17.685 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:17.737 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:45:17.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.965+0000 7f8e0f800640 1 -- 172.21.15.44:0/731409629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8df0004da0 msgr2=0x7f8df0005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:17.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.965+0000 7f8e0f800640 1 --2- 172.21.15.44:0/731409629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8df0004da0 0x7f8df0005220 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000bfb0 tx=0x7f8e0001ee20 comp rx=0 tx=0).stop 2024-09-13T08:45:17.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.965+0000 7f8e0d800640 1 -- 172.21.15.44:0/731409629 reap_dead start 2024-09-13T08:45:17.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.965+0000 7f8e0f800640 1 -- 172.21.15.44:0/731409629 shutdown_connections 2024-09-13T08:45:17.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.965+0000 7f8e0f800640 1 -- 172.21.15.44:0/731409629 >> 172.21.15.44:0/731409629 conn(0x7f8e080b18f0 msgr2=0x7f8e080b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:17.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.969+0000 7f8e0f800640 1 -- 172.21.15.44:0/731409629 shutdown_connections 2024-09-13T08:45:17.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.969+0000 7f8e0f800640 1 -- 172.21.15.44:0/731409629 wait complete. 2024-09-13T08:45:17.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.969+0000 7f8e0f800640 1 Processor -- start 2024-09-13T08:45:17.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.969+0000 7f8e0f800640 1 -- start start 2024-09-13T08:45:17.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0f800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e08144830 0x7f8e081f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:17.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0f800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e08144c10 0x7f8e0809d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:17.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0f800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e081f8e30 0x7f8e0809d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:17.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0f800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8e080cdbd0 con 0x7f8e081f8e30 2024-09-13T08:45:17.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0f800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08070b70 con 0x7f8e08144c10 2024-09-13T08:45:17.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0f800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8e080ccb80 con 0x7f8e08144830 2024-09-13T08:45:17.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e07e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e08144c10 0x7f8e0809d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:17.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e081f8e30 0x7f8e0809d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:17.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e081f8e30 0x7f8e0809d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53928/0 (socket says 172.21.15.44:53928) 2024-09-13T08:45:17.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0d800640 1 -- 172.21.15.44:0/1715023845 learned_addr learned my addr 172.21.15.44:0/1715023845 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:17.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0ce00640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e08144830 0x7f8e081f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:17.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0d800640 1 -- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e08144830 msgr2=0x7f8e081f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:17.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0d800640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e08144830 0x7f8e081f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:17.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0d800640 1 -- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e08144c10 msgr2=0x7f8e0809d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:17.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0d800640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e08144c10 0x7f8e0809d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:17.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0d800640 1 -- 172.21.15.44:0/1715023845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e081f5500 con 0x7f8e081f8e30 2024-09-13T08:45:17.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0d800640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e081f8e30 0x7f8e0809d5f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8e00002240 tx=0x7f8e000b0bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:17.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e04c00640 1 -- 172.21.15.44:0/1715023845 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000b4070 con 0x7f8e081f8e30 2024-09-13T08:45:17.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e04c00640 1 -- 172.21.15.44:0/1715023845 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e000b6070 con 0x7f8e081f8e30 2024-09-13T08:45:17.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e04c00640 1 -- 172.21.15.44:0/1715023845 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0000bb80 con 0x7f8e081f8e30 2024-09-13T08:45:17.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0f800640 1 -- 172.21.15.44:0/1715023845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e081f57e0 con 0x7f8e081f8e30 2024-09-13T08:45:17.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.973+0000 7f8e0f800640 1 -- 172.21.15.44:0/1715023845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e08150ed0 con 0x7f8e081f8e30 2024-09-13T08:45:17.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.977+0000 7f8e0f800640 1 -- 172.21.15.44:0/1715023845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e08069410 con 0x7f8e081f8e30 2024-09-13T08:45:17.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.981+0000 7f8e04c00640 1 -- 172.21.15.44:0/1715023845 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f8e00004a50 con 0x7f8e081f8e30 2024-09-13T08:45:17.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.981+0000 7f8e04c00640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8de8073b50 0x7f8de8076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:17.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.981+0000 7f8e04c00640 1 -- 172.21.15.44:0/1715023845 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7f8e0012cc60 con 0x7f8e081f8e30 2024-09-13T08:45:17.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.981+0000 7f8e0ce00640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8de8073b50 0x7f8de8076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:17.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.985+0000 7f8e0ce00640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8de8073b50 0x7f8de8076010 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8df4001870 tx=0x7f8df4001ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:17.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:17.985+0000 7f8e04c00640 1 -- 172.21.15.44:0/1715023845 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8e0015c060 con 0x7f8e081f8e30 2024-09-13T08:45:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.361+0000 7f8e0f800640 1 -- 172.21.15.44:0/1715023845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e080cd3d0 con 0x7f8e081f8e30 2024-09-13T08:45:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.361+0000 7f8e04c00640 1 -- 172.21.15.44:0/1715023845 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v64) ==== 80+0+0 (secure 0 0 0) 0x7f8e00018c10 con 0x7f8e081f8e30 2024-09-13T08:45:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:45:18.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 -- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8de8073b50 msgr2=0x7f8de8076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:18.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8de8073b50 0x7f8de8076010 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8df4001870 tx=0x7f8df4001ba0 comp rx=0 tx=0).stop 2024-09-13T08:45:18.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 -- 172.21.15.44:0/1715023845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e081f8e30 msgr2=0x7f8e0809d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:18.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e081f8e30 0x7f8e0809d5f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8e00002240 tx=0x7f8e000b0bf0 comp rx=0 tx=0).stop 2024-09-13T08:45:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 -- 172.21.15.44:0/1715023845 shutdown_connections 2024-09-13T08:45:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8de8073b50 0x7f8de8076010 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e081f8e30 0x7f8e0809d5f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e08144c10 0x7f8e0809d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 --2- 172.21.15.44:0/1715023845 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e08144830 0x7f8e081f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 -- 172.21.15.44:0/1715023845 >> 172.21.15.44:0/1715023845 conn(0x7f8e080b18f0 msgr2=0x7f8e0815e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 -- 172.21.15.44:0/1715023845 shutdown_connections 2024-09-13T08:45:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.365+0000 7f8e0f800640 1 -- 172.21.15.44:0/1715023845 wait complete. 2024-09-13T08:45:18.401 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T08:45:18.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:18.402 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T08:45:18.479 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:45:18.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:18.480 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T08:45:18.499 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:45:18.499 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:45:18.499 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:18.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:45:18.514 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:45:18.514 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:45:18.514 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:18.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:45:18.554 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:45:18.554 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:45:18.554 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:18.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:45:18.571 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:45:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.833+0000 7f80f2a00640 1 -- 172.21.15.44:0/2805639323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80ec15ade0 msgr2=0x7f80ec0cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:18.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.833+0000 7f80f2a00640 1 --2- 172.21.15.44:0/2805639323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80ec15ade0 0x7f80ec0cebf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009840 tx=0x7f80dc01c820 comp rx=0 tx=0).stop 2024-09-13T08:45:18.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.833+0000 7f80f2a00640 1 -- 172.21.15.44:0/2805639323 shutdown_connections 2024-09-13T08:45:18.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.833+0000 7f80f2a00640 1 --2- 172.21.15.44:0/2805639323 >> v2:172.21.15.129:3300/0 conn(0x7f80ec06e810 0x7f80ec0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:18.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.833+0000 7f80f2a00640 1 --2- 172.21.15.44:0/2805639323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80ec15ade0 0x7f80ec0cebf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:18.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.833+0000 7f80f2a00640 1 -- 172.21.15.44:0/2805639323 >> 172.21.15.44:0/2805639323 conn(0x7f80ec0b18f0 msgr2=0x7f80ec0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:18.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.833+0000 7f80f2a00640 1 -- 172.21.15.44:0/2805639323 shutdown_connections 2024-09-13T08:45:18.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.833+0000 7f80f2a00640 1 -- 172.21.15.44:0/2805639323 wait complete. 2024-09-13T08:45:18.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.837+0000 7f80f2a00640 1 Processor -- start 2024-09-13T08:45:18.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.837+0000 7f80f2a00640 1 -- start start 2024-09-13T08:45:18.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80f2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec06e810 0x7f80ec15b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:18.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80f2a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80ec0b4bd0 0x7f80ec1423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:18.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80f2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80ec15ade0 0x7f80ec141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:18.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80f2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec0ce030 con 0x7f80ec06e810 2024-09-13T08:45:18.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80f2a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec0cdeb0 con 0x7f80ec15ade0 2024-09-13T08:45:18.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80f2a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f80ec0ce1b0 con 0x7f80ec0b4bd0 2024-09-13T08:45:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80eb400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80ec0b4bd0 0x7f80ec1423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80ebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec06e810 0x7f80ec15b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80ebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec06e810 0x7f80ec15b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53934/0 (socket says 172.21.15.44:53934) 2024-09-13T08:45:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80ebe00640 1 -- 172.21.15.44:0/1135556590 learned_addr learned my addr 172.21.15.44:0/1135556590 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80f0a00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80ec15ade0 0x7f80ec141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80ebe00640 1 -- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80ec0b4bd0 msgr2=0x7f80ec1423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80ebe00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80ec0b4bd0 0x7f80ec1423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80ebe00640 1 -- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80ec15ade0 msgr2=0x7f80ec141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80ebe00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80ec15ade0 0x7f80ec141820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80ebe00640 1 -- 172.21.15.44:0/1135556590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec0ae380 con 0x7f80ec06e810 2024-09-13T08:45:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80eb400640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80ec0b4bd0 0x7f80ec1423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80f0a00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80ec15ade0 0x7f80ec141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.841+0000 7f80ebe00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec06e810 0x7f80ec15b7c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009790 tx=0x7f80dc005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.845+0000 7f80e3e00640 1 -- 172.21.15.44:0/1135556590 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc0ac050 con 0x7f80ec06e810 2024-09-13T08:45:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.845+0000 7f80e3e00640 1 -- 172.21.15.44:0/1135556590 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80dc004320 con 0x7f80ec06e810 2024-09-13T08:45:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.845+0000 7f80f2a00640 1 -- 172.21.15.44:0/1135556590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ec158d90 con 0x7f80ec06e810 2024-09-13T08:45:18.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.845+0000 7f80e3e00640 1 -- 172.21.15.44:0/1135556590 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc0a6ad0 con 0x7f80ec06e810 2024-09-13T08:45:18.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.845+0000 7f80f2a00640 1 -- 172.21.15.44:0/1135556590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ec159350 con 0x7f80ec06e810 2024-09-13T08:45:18.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.845+0000 7f80f2a00640 1 -- 172.21.15.44:0/1135556590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ec069410 con 0x7f80ec06e810 2024-09-13T08:45:18.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.849+0000 7f80e3e00640 1 -- 172.21.15.44:0/1135556590 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f80dc0af070 con 0x7f80ec06e810 2024-09-13T08:45:18.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.849+0000 7f80e3e00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80cc073b20 0x7f80cc075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:18.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.849+0000 7f80e3e00640 1 -- 172.21.15.44:0/1135556590 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5851+0+0 (secure 0 0 0) 0x7f80dc12c6e0 con 0x7f80ec06e810 2024-09-13T08:45:18.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.853+0000 7f80eb400640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80cc073b20 0x7f80cc075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:18.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.853+0000 7f80eb400640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80cc073b20 0x7f80cc075fe0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80ec15e1b0 tx=0x7f80d80091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:18.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:18.857+0000 7f80e3e00640 1 -- 172.21.15.44:0/1135556590 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f80dc0b7070 con 0x7f80ec06e810 2024-09-13T08:45:19.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.249+0000 7f80f2a00640 1 -- 172.21.15.44:0/1135556590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80ec0cf720 con 0x7f80ec06e810 2024-09-13T08:45:19.253 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:19.253 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":38,"btime":"2024-09-13T08:43:24:844013+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1613950025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1613950025},{"type":"v1","addr":"172.21.15.129:6836","nonce":1613950025}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5402,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/10747232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":10747232},{"type":"v1","addr":"172.21.15.129:6837","nonce":10747232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3945609265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3945609265},{"type":"v1","addr":"172.21.15.44:6834","nonce":3945609265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:43:16.187506+0000","modified":"2024-09-13T08:43:19.249207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2755626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2755626228},{"type":"v1","addr":"172.21.15.44:6835","nonce":2755626228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5877,"qdb_cluster":[5877]},"id":4}]} 2024-09-13T08:45:19.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.253+0000 7f80e3e00640 1 -- 172.21.15.44:0/1135556590 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5223 (secure 0 0 0) 0x7f80dc016410 con 0x7f80ec06e810 2024-09-13T08:45:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 38 2024-09-13T08:45:19.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 -- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80cc073b20 msgr2=0x7f80cc075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:19.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80cc073b20 0x7f80cc075fe0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80ec15e1b0 tx=0x7f80d80091e0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 -- 172.21.15.44:0/1135556590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec06e810 msgr2=0x7f80ec15b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:19.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec06e810 0x7f80ec15b7c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009790 tx=0x7f80dc005710 comp rx=0 tx=0).stop 2024-09-13T08:45:19.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 -- 172.21.15.44:0/1135556590 shutdown_connections 2024-09-13T08:45:19.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f80cc073b20 0x7f80cc075fe0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80ec15ade0 0x7f80ec141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f80ec0b4bd0 0x7f80ec1423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 --2- 172.21.15.44:0/1135556590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec06e810 0x7f80ec15b7c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 -- 172.21.15.44:0/1135556590 >> 172.21.15.44:0/1135556590 conn(0x7f80ec0b18f0 msgr2=0x7f80ec0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:19.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 -- 172.21.15.44:0/1135556590 shutdown_connections 2024-09-13T08:45:19.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.257+0000 7f80f2a00640 1 -- 172.21.15.44:0/1135556590 wait complete. 2024-09-13T08:45:19.292 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:45:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.453+0000 7f2b83000640 1 -- 172.21.15.44:0/808941774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b7c0cdb30 msgr2=0x7f2b7c0af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.453+0000 7f2b83000640 1 --2- 172.21.15.44:0/808941774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b7c0cdb30 0x7f2b7c0af0c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009c80 tx=0x7f2b7401cda0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.453+0000 7f2b83000640 1 -- 172.21.15.44:0/808941774 shutdown_connections 2024-09-13T08:45:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.453+0000 7f2b83000640 1 --2- 172.21.15.44:0/808941774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b7c0cdb30 0x7f2b7c0af0c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.453+0000 7f2b83000640 1 -- 172.21.15.44:0/808941774 >> 172.21.15.44:0/808941774 conn(0x7f2b7c0d1dc0 msgr2=0x7f2b7c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.453+0000 7f2b83000640 1 -- 172.21.15.44:0/808941774 shutdown_connections 2024-09-13T08:45:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.453+0000 7f2b83000640 1 -- 172.21.15.44:0/808941774 wait complete. 2024-09-13T08:45:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 Processor -- start 2024-09-13T08:45:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 -- start start 2024-09-13T08:45:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b7c0cdb30 0x7f2b7c167050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b7c0ced80 0x7f2b7c0ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b7c15bea0 0x7f2b7c157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c16a8a0 con 0x7f2b7c0ced80 2024-09-13T08:45:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c16a720 con 0x7f2b7c0cdb30 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c16aa20 con 0x7f2b7c15bea0 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b73e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b7c0ced80 0x7f2b7c0ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b73e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b7c0ced80 0x7f2b7c0ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53948/0 (socket says 172.21.15.44:53948) 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b73e00640 1 -- 172.21.15.44:0/4203532950 learned_addr learned my addr 172.21.15.44:0/4203532950 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b81000640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b7c15bea0 0x7f2b7c157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b7be00640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b7c0cdb30 0x7f2b7c167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b73e00640 1 -- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b7c15bea0 msgr2=0x7f2b7c157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b73e00640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b7c15bea0 0x7f2b7c157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b73e00640 1 -- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b7c0cdb30 msgr2=0x7f2b7c167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b73e00640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b7c0cdb30 0x7f2b7c167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b73e00640 1 -- 172.21.15.44:0/4203532950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c0cae20 con 0x7f2b7c0ced80 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b7be00640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b7c0cdb30 0x7f2b7c167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b73e00640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b7c0ced80 0x7f2b7c0ca3f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400c4b0 tx=0x7f2b6400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b78c00640 1 -- 172.21.15.44:0/4203532950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b640052d0 con 0x7f2b7c0ced80 2024-09-13T08:45:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b78c00640 1 -- 172.21.15.44:0/4203532950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b64005470 con 0x7f2b7c0ced80 2024-09-13T08:45:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b78c00640 1 -- 172.21.15.44:0/4203532950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64012870 con 0x7f2b7c0ced80 2024-09-13T08:45:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 -- 172.21.15.44:0/4203532950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c06e7e0 con 0x7f2b7c0ced80 2024-09-13T08:45:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 -- 172.21.15.44:0/4203532950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c06eda0 con 0x7f2b7c0ced80 2024-09-13T08:45:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.457+0000 7f2b83000640 1 -- 172.21.15.44:0/4203532950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c0b01a0 con 0x7f2b7c0ced80 2024-09-13T08:45:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.461+0000 7f2b78c00640 1 -- 172.21.15.44:0/4203532950 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f2b64003600 con 0x7f2b7c0ced80 2024-09-13T08:45:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.461+0000 7f2b78c00640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b58073b20 0x7f2b58075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.461+0000 7f2b78c00640 1 -- 172.21.15.44:0/4203532950 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5851+0+0 (secure 0 0 0) 0x7f2b64092390 con 0x7f2b7c0ced80 2024-09-13T08:45:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.461+0000 7f2b7be00640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b58073b20 0x7f2b58075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.461+0000 7f2b7be00640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b58073b20 0x7f2b58075fe0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0016b0 tx=0x7f2b6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.465+0000 7f2b78c00640 1 -- 172.21.15.44:0/4203532950 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2b6405f030 con 0x7f2b7c0ced80 2024-09-13T08:45:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 -- 172.21.15.44:0/4203532950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b7c0762f0 con 0x7f2b7c0ced80 2024-09-13T08:45:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b78c00640 1 -- 172.21.15.44:0/4203532950 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5223 (secure 0 0 0) 0x7f2b6405e930 con 0x7f2b7c0ced80 2024-09-13T08:45:19.678 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:19.678 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":38,"btime":"2024-09-13T08:43:24:844013+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1613950025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1613950025},{"type":"v1","addr":"172.21.15.129:6836","nonce":1613950025}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5402,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/10747232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":10747232},{"type":"v1","addr":"172.21.15.129:6837","nonce":10747232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3945609265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3945609265},{"type":"v1","addr":"172.21.15.44:6834","nonce":3945609265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:43:16.187506+0000","modified":"2024-09-13T08:43:19.249207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2755626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2755626228},{"type":"v1","addr":"172.21.15.44:6835","nonce":2755626228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5877,"qdb_cluster":[5877]},"id":4}]} 2024-09-13T08:45:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 38 2024-09-13T08:45:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 -- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b58073b20 msgr2=0x7f2b58075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b58073b20 0x7f2b58075fe0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0016b0 tx=0x7f2b6c009040 comp rx=0 tx=0).stop 2024-09-13T08:45:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 -- 172.21.15.44:0/4203532950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b7c0ced80 msgr2=0x7f2b7c0ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b7c0ced80 0x7f2b7c0ca3f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400c4b0 tx=0x7f2b6400c970 comp rx=0 tx=0).stop 2024-09-13T08:45:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 -- 172.21.15.44:0/4203532950 shutdown_connections 2024-09-13T08:45:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b58073b20 0x7f2b58075fe0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b7c15bea0 0x7f2b7c157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b7c0ced80 0x7f2b7c0ca3f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 --2- 172.21.15.44:0/4203532950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b7c0cdb30 0x7f2b7c167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 -- 172.21.15.44:0/4203532950 >> 172.21.15.44:0/4203532950 conn(0x7f2b7c0d1dc0 msgr2=0x7f2b7c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 -- 172.21.15.44:0/4203532950 shutdown_connections 2024-09-13T08:45:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.677+0000 7f2b83000640 1 -- 172.21.15.44:0/4203532950 wait complete. 2024-09-13T08:45:19.700 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:45:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.845+0000 7f7740000640 1 -- 172.21.15.44:0/1178460207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7720004810 msgr2=0x7f7720004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.845+0000 7f7740000640 1 --2- 172.21.15.44:0/1178460207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7720004810 0x7f7720004c90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7724001ec0 tx=0x7f772401cf40 comp rx=0 tx=0).stop 2024-09-13T08:45:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.845+0000 7f773e000640 1 -- 172.21.15.44:0/1178460207 reap_dead start 2024-09-13T08:45:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.845+0000 7f7740000640 1 -- 172.21.15.44:0/1178460207 shutdown_connections 2024-09-13T08:45:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.845+0000 7f7740000640 1 -- 172.21.15.44:0/1178460207 >> 172.21.15.44:0/1178460207 conn(0x7f77380d1dc0 msgr2=0x7f77380d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.845+0000 7f7740000640 1 -- 172.21.15.44:0/1178460207 shutdown_connections 2024-09-13T08:45:19.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.845+0000 7f7740000640 1 -- 172.21.15.44:0/1178460207 wait complete. 2024-09-13T08:45:19.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 Processor -- start 2024-09-13T08:45:19.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 -- start start 2024-09-13T08:45:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7720004810 0x7f77380b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77381e7670 0x7f77380b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f773815a3f0 0x7f773815a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77380d2af0 con 0x7f773815a3f0 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77380af0c0 con 0x7f7720004810 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f77381596c0 con 0x7f77381e7670 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f773815a3f0 0x7f773815a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f773815a3f0 0x7f773815a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53962/0 (socket says 172.21.15.44:53962) 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773e000640 1 -- 172.21.15.44:0/150711946 learned_addr learned my addr 172.21.15.44:0/150711946 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773d600640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7720004810 0x7f77380b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773cc00640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77381e7670 0x7f77380b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773e000640 1 -- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77381e7670 msgr2=0x7f77380b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773e000640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77381e7670 0x7f77380b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773e000640 1 -- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7720004810 msgr2=0x7f77380b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773e000640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7720004810 0x7f77380b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773e000640 1 -- 172.21.15.44:0/150711946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77380b7b60 con 0x7f773815a3f0 2024-09-13T08:45:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f773e000640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f773815a3f0 0x7f773815a850 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7738074d10 tx=0x7f773000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7735600640 1 -- 172.21.15.44:0/150711946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730004b30 con 0x7f773815a3f0 2024-09-13T08:45:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7735600640 1 -- 172.21.15.44:0/150711946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7730004cd0 con 0x7f773815a3f0 2024-09-13T08:45:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7735600640 1 -- 172.21.15.44:0/150711946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730003be0 con 0x7f773815a3f0 2024-09-13T08:45:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 -- 172.21.15.44:0/150711946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77380b1a90 con 0x7f773815a3f0 2024-09-13T08:45:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 -- 172.21.15.44:0/150711946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7738159910 con 0x7f773815a3f0 2024-09-13T08:45:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.849+0000 7f7740000640 1 -- 172.21.15.44:0/150711946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77380af650 con 0x7f773815a3f0 2024-09-13T08:45:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.853+0000 7f7735600640 1 -- 172.21.15.44:0/150711946 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f7730004030 con 0x7f773815a3f0 2024-09-13T08:45:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.853+0000 7f7735600640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7718073b50 0x7f7718076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.853+0000 7f7735600640 1 -- 172.21.15.44:0/150711946 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5851+0+0 (secure 0 0 0) 0x7f7730092ff0 con 0x7f773815a3f0 2024-09-13T08:45:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.853+0000 7f773d600640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7718073b50 0x7f7718076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:19.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.853+0000 7f773d600640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7718073b50 0x7f7718076010 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f772c000c10 tx=0x7f772c006b20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:19.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:19.857+0000 7f7735600640 1 -- 172.21.15.44:0/150711946 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f773005f640 con 0x7f773815a3f0 2024-09-13T08:45:20.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.033+0000 7f7740000640 1 -- 172.21.15.44:0/150711946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77380b6c50 con 0x7f773815a3f0 2024-09-13T08:45:20.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.033+0000 7f7735600640 1 -- 172.21.15.44:0/150711946 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16985 (secure 0 0 0) 0x7f7730064070 con 0x7f773815a3f0 2024-09-13T08:45:20.035 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:20.036 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":64,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:45:18.118716+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:43:13.893284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-13T08:43:15.154456+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1c","mappings":[{"from":6,"to":1}]},{"pgid":"9.1e","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:45:20.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 -- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7718073b50 msgr2=0x7f7718076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7718073b50 0x7f7718076010 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f772c000c10 tx=0x7f772c006b20 comp rx=0 tx=0).stop 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 -- 172.21.15.44:0/150711946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f773815a3f0 msgr2=0x7f773815a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f773815a3f0 0x7f773815a850 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7738074d10 tx=0x7f773000f3c0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 -- 172.21.15.44:0/150711946 shutdown_connections 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7718073b50 0x7f7718076010 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f773815a3f0 0x7f773815a850 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f77381e7670 0x7f77380b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 --2- 172.21.15.44:0/150711946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7720004810 0x7f77380b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 -- 172.21.15.44:0/150711946 >> 172.21.15.44:0/150711946 conn(0x7f77380d1dc0 msgr2=0x7f77380b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 -- 172.21.15.44:0/150711946 shutdown_connections 2024-09-13T08:45:20.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.037+0000 7f7740000640 1 -- 172.21.15.44:0/150711946 wait complete. 2024-09-13T08:45:20.057 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:45:20.057 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:45:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 -- 172.21.15.44:0/330320784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f540051d0 msgr2=0x7f0f5400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 --2- 172.21.15.44:0/330320784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f540051d0 0x7f0f5400b9d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800c640 tx=0x7f0f6801f9c0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 -- 172.21.15.44:0/330320784 shutdown_connections 2024-09-13T08:45:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 --2- 172.21.15.44:0/330320784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f540051d0 0x7f0f5400b9d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 -- 172.21.15.44:0/330320784 >> 172.21.15.44:0/330320784 conn(0x7f0f700d0d50 msgr2=0x7f0f700d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 -- 172.21.15.44:0/330320784 shutdown_connections 2024-09-13T08:45:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 -- 172.21.15.44:0/330320784 wait complete. 2024-09-13T08:45:20.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 Processor -- start 2024-09-13T08:45:20.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 -- start start 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f540051d0 0x7f0f700b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f70151b10 0x7f0f701f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f7015db80 0x7f0f7015dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0f700cb7e0 con 0x7f0f70151b10 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0f700cb660 con 0x7f0f540051d0 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f75000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0f700ae1a0 con 0x7f0f7015db80 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f67e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f70151b10 0x7f0f701f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f67e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f70151b10 0x7f0f701f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53978/0 (socket says 172.21.15.44:53978) 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f67e00640 1 -- 172.21.15.44:0/2432666735 learned_addr learned my addr 172.21.15.44:0/2432666735 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f6de00640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f540051d0 0x7f0f700b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f6e800640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f7015db80 0x7f0f7015dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f67e00640 1 -- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f7015db80 msgr2=0x7f0f7015dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f67e00640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f7015db80 0x7f0f7015dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f67e00640 1 -- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f540051d0 msgr2=0x7f0f700b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f67e00640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f540051d0 0x7f0f700b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f67e00640 1 -- 172.21.15.44:0/2432666735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f701f4950 con 0x7f0f70151b10 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f6e800640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f7015db80 0x7f0f7015dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.205+0000 7f0f67e00640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f70151b10 0x7f0f701f42d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0f58011900 tx=0x7f0f58011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.209+0000 7f0f66000640 1 -- 172.21.15.44:0/2432666735 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5800fe70 con 0x7f0f70151b10 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.209+0000 7f0f66000640 1 -- 172.21.15.44:0/2432666735 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f58004050 con 0x7f0f70151b10 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.209+0000 7f0f66000640 1 -- 172.21.15.44:0/2432666735 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f580044a0 con 0x7f0f70151b10 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.209+0000 7f0f75000640 1 -- 172.21.15.44:0/2432666735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f701f4c30 con 0x7f0f70151b10 2024-09-13T08:45:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.209+0000 7f0f75000640 1 -- 172.21.15.44:0/2432666735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f701f51f0 con 0x7f0f70151b10 2024-09-13T08:45:20.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.209+0000 7f0f75000640 1 -- 172.21.15.44:0/2432666735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f700afed0 con 0x7f0f70151b10 2024-09-13T08:45:20.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.209+0000 7f0f66000640 1 -- 172.21.15.44:0/2432666735 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f0f58003600 con 0x7f0f70151b10 2024-09-13T08:45:20.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.209+0000 7f0f66000640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f4c073b20 0x7f0f4c075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.209+0000 7f0f66000640 1 -- 172.21.15.44:0/2432666735 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5851+0+0 (secure 0 0 0) 0x7f0f58093770 con 0x7f0f70151b10 2024-09-13T08:45:20.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.209+0000 7f0f6de00640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f4c073b20 0x7f0f4c075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.213+0000 7f0f6de00640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f4c073b20 0x7f0f4c075fe0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0f70075150 tx=0x7f0f60006b20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:20.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.213+0000 7f0f66000640 1 -- 172.21.15.44:0/2432666735 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0f5805fdc0 con 0x7f0f70151b10 2024-09-13T08:45:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 -- 172.21.15.44:0/2432666735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f70152bb0 con 0x7f0f70151b10 2024-09-13T08:45:20.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f66000640 1 -- 172.21.15.44:0/2432666735 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5223 (secure 0 0 0) 0x7f0f580647f0 con 0x7f0f70151b10 2024-09-13T08:45:20.426 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:20.426 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":38,"btime":"2024-09-13T08:43:24:844013+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1613950025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1613950025},{"type":"v1","addr":"172.21.15.129:6836","nonce":1613950025}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5402,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/10747232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":10747232},{"type":"v1","addr":"172.21.15.129:6837","nonce":10747232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3945609265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3945609265},{"type":"v1","addr":"172.21.15.44:6834","nonce":3945609265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:43:16.187506+0000","modified":"2024-09-13T08:43:19.249207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2755626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2755626228},{"type":"v1","addr":"172.21.15.44:6835","nonce":2755626228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5877,"qdb_cluster":[5877]},"id":4}]} 2024-09-13T08:45:20.426 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 38 2024-09-13T08:45:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 -- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f4c073b20 msgr2=0x7f0f4c075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f4c073b20 0x7f0f4c075fe0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0f70075150 tx=0x7f0f60006b20 comp rx=0 tx=0).stop 2024-09-13T08:45:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 -- 172.21.15.44:0/2432666735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f70151b10 msgr2=0x7f0f701f42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f70151b10 0x7f0f701f42d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0f58011900 tx=0x7f0f58011dc0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 -- 172.21.15.44:0/2432666735 shutdown_connections 2024-09-13T08:45:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0f4c073b20 0x7f0f4c075fe0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f7015db80 0x7f0f7015dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f70151b10 0x7f0f701f42d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 --2- 172.21.15.44:0/2432666735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f540051d0 0x7f0f700b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 -- 172.21.15.44:0/2432666735 >> 172.21.15.44:0/2432666735 conn(0x7f0f700d0d50 msgr2=0x7f0f7015f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 -- 172.21.15.44:0/2432666735 shutdown_connections 2024-09-13T08:45:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.425+0000 7f0f75000640 1 -- 172.21.15.44:0/2432666735 wait complete. 2024-09-13T08:45:20.447 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:45:20.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 -- 172.21.15.44:0/896549966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c0cc8c0 msgr2=0x7fe19c0cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 --2- 172.21.15.44:0/896549966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c0cc8c0 0x7fe19c0cccf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe194009c80 tx=0x7fe19401cd70 comp rx=0 tx=0).stop 2024-09-13T08:45:20.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 -- 172.21.15.44:0/896549966 shutdown_connections 2024-09-13T08:45:20.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 --2- 172.21.15.44:0/896549966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c0cc8c0 0x7fe19c0cccf0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 --2- 172.21.15.44:0/896549966 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe19c158240 0x7fe19c15ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 -- 172.21.15.44:0/896549966 >> 172.21.15.44:0/896549966 conn(0x7fe19c0d0c90 msgr2=0x7fe19c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:20.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 -- 172.21.15.44:0/896549966 shutdown_connections 2024-09-13T08:45:20.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 -- 172.21.15.44:0/896549966 wait complete. 2024-09-13T08:45:20.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 Processor -- start 2024-09-13T08:45:20.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 -- start start 2024-09-13T08:45:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.593+0000 7fe1a3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe19c0cc8c0 0x7fe19c156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a3400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe19c158240 0x7fe19c0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c158620 0x7fe19c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a3400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c0ce860 con 0x7fe19c158620 2024-09-13T08:45:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a3400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c072190 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a3400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c0ce9e0 con 0x7fe19c158240 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a0a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe19c0cc8c0 0x7fe19c156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c158620 0x7fe19c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a0a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe19c0cc8c0 0x7fe19c156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:48728/0 (socket says 172.21.15.44:48728) 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a0a00640 1 -- 172.21.15.44:0/3461137066 learned_addr learned my addr 172.21.15.44:0/3461137066 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe19be00640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe19c158240 0x7fe19c0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a0a00640 1 -- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe19c158240 msgr2=0x7fe19c0b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a0a00640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe19c158240 0x7fe19c0b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a0a00640 1 -- 172.21.15.44:0/3461137066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c158620 msgr2=0x7fe19c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a0a00640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c158620 0x7fe19c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a0a00640 1 -- 172.21.15.44:0/3461137066 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c0b1090 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a1400640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c158620 0x7fe19c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a0a00640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe19c0cc8c0 0x7fe19c156d70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe19000d950 tx=0x7fe19000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe198c00640 1 -- 172.21.15.44:0/3461137066 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190017070 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe198c00640 1 -- 172.21.15.44:0/3461137066 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe190009d20 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe198c00640 1 -- 172.21.15.44:0/3461137066 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1900106c0 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a3400640 1 -- 172.21.15.44:0/3461137066 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c1574c0 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a3400640 1 -- 172.21.15.44:0/3461137066 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c157a80 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe1a3400640 1 -- 172.21.15.44:0/3461137066 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c0cccf0 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.597+0000 7fe198c00640 1 -- 172.21.15.44:0/3461137066 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fe190010860 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.601+0000 7fe198c00640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe17c073b20 0x7fe17c075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.601+0000 7fe198c00640 1 -- 172.21.15.44:0/3461137066 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5851+0+0 (secure 0 0 0) 0x7fe190092b50 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.601+0000 7fe19be00640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe17c073b20 0x7fe17c075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.601+0000 7fe19be00640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe17c073b20 0x7fe17c075fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0cb200 tx=0x7fe188009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:20.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.605+0000 7fe198c00640 1 -- 172.21.15.44:0/3461137066 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe19005f1a0 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.813+0000 7fe1a3400640 1 -- 172.21.15.44:0/3461137066 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe19c0a1c30 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe198c00640 1 -- 172.21.15.44:0/3461137066 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5223 (secure 0 0 0) 0x7fe190063bd0 con 0x7fe19c0cc8c0 2024-09-13T08:45:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:20.817 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":38,"btime":"2024-09-13T08:43:24:844013+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1613950025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1613950025},{"type":"v1","addr":"172.21.15.129:6836","nonce":1613950025}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5402,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/10747232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":10747232},{"type":"v1","addr":"172.21.15.129:6837","nonce":10747232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3945609265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3945609265},{"type":"v1","addr":"172.21.15.44:6834","nonce":3945609265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:43:16.187506+0000","modified":"2024-09-13T08:43:19.249207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2755626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2755626228},{"type":"v1","addr":"172.21.15.44:6835","nonce":2755626228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5877,"qdb_cluster":[5877]},"id":4}]} 2024-09-13T08:45:20.818 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 38 2024-09-13T08:45:20.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 -- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe17c073b20 msgr2=0x7fe17c075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe17c073b20 0x7fe17c075fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0cb200 tx=0x7fe188009040 comp rx=0 tx=0).stop 2024-09-13T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 -- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe19c0cc8c0 msgr2=0x7fe19c156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe19c0cc8c0 0x7fe19c156d70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe19000d950 tx=0x7fe19000de10 comp rx=0 tx=0).stop 2024-09-13T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 -- 172.21.15.44:0/3461137066 shutdown_connections 2024-09-13T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe17c073b20 0x7fe17c075fe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c158620 0x7fe19c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe19c158240 0x7fe19c0b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 --2- 172.21.15.44:0/3461137066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe19c0cc8c0 0x7fe19c156d70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 -- 172.21.15.44:0/3461137066 >> 172.21.15.44:0/3461137066 conn(0x7fe19c0d0c90 msgr2=0x7fe19c1513d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 -- 172.21.15.44:0/3461137066 shutdown_connections 2024-09-13T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.817+0000 7fe1a3400640 1 -- 172.21.15.44:0/3461137066 wait complete. 2024-09-13T08:45:20.840 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:45:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 -- 172.21.15.44:0/1761613655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb00cf4e0 msgr2=0x7fddb00cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 --2- 172.21.15.44:0/1761613655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb00cf4e0 0x7fddb00cd950 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fddb0073a20 tx=0x7fdda801cc60 comp rx=0 tx=0).stop 2024-09-13T08:45:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 -- 172.21.15.44:0/1761613655 shutdown_connections 2024-09-13T08:45:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 --2- 172.21.15.44:0/1761613655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb00cf4e0 0x7fddb00cd950 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 --2- 172.21.15.44:0/1761613655 >> v2:172.21.15.129:3300/0 conn(0x7fddb015bd50 0x7fddb00ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 -- 172.21.15.44:0/1761613655 >> 172.21.15.44:0/1761613655 conn(0x7fddb00d1ea0 msgr2=0x7fddb00d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 -- 172.21.15.44:0/1761613655 shutdown_connections 2024-09-13T08:45:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 -- 172.21.15.44:0/1761613655 wait complete. 2024-09-13T08:45:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 Processor -- start 2024-09-13T08:45:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 -- start start 2024-09-13T08:45:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb00cf4e0 0x7fddb0154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddb015bd50 0x7fddb014b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddb015c130 0x7fddb00c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fddb0168ea0 con 0x7fddb00cf4e0 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fddb0168d20 con 0x7fddb015bd50 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb8200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fddb0169020 con 0x7fddb015c130 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb00cf4e0 0x7fddb0154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb00cf4e0 0x7fddb0154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54010/0 (socket says 172.21.15.44:54010) 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb5800640 1 -- 172.21.15.44:0/2223429996 learned_addr learned my addr 172.21.15.44:0/2223429996 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb4e00640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddb015bd50 0x7fddb014b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.985+0000 7fddb6200640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddb015c130 0x7fddb00c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddb5800640 1 -- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddb015c130 msgr2=0x7fddb00c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddb5800640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddb015c130 0x7fddb00c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddb5800640 1 -- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddb015bd50 msgr2=0x7fddb014b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddb5800640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddb015bd50 0x7fddb014b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddb5800640 1 -- 172.21.15.44:0/2223429996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb0155100 con 0x7fddb00cf4e0 2024-09-13T08:45:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddb5800640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb00cf4e0 0x7fddb0154610 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdda400d9b0 tx=0x7fdda400de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddad600640 1 -- 172.21.15.44:0/2223429996 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4017070 con 0x7fddb00cf4e0 2024-09-13T08:45:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddad600640 1 -- 172.21.15.44:0/2223429996 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdda4009cc0 con 0x7fddb00cf4e0 2024-09-13T08:45:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddad600640 1 -- 172.21.15.44:0/2223429996 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4005280 con 0x7fddb00cf4e0 2024-09-13T08:45:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddb8200640 1 -- 172.21.15.44:0/2223429996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb0154b50 con 0x7fddb00cf4e0 2024-09-13T08:45:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddb8200640 1 -- 172.21.15.44:0/2223429996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb01585e0 con 0x7fddb00cf4e0 2024-09-13T08:45:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddb8200640 1 -- 172.21.15.44:0/2223429996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb00ce5e0 con 0x7fddb00cf4e0 2024-09-13T08:45:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.989+0000 7fddad600640 1 -- 172.21.15.44:0/2223429996 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fdda4005420 con 0x7fddb00cf4e0 2024-09-13T08:45:20.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.993+0000 7fddad600640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd90073b20 0x7fdd90075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:20.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.993+0000 7fddad600640 1 -- 172.21.15.44:0/2223429996 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5851+0+0 (secure 0 0 0) 0x7fdda4092c80 con 0x7fddb00cf4e0 2024-09-13T08:45:20.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.993+0000 7fddb4e00640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd90073b20 0x7fdd90075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.993+0000 7fddb4e00640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd90073b20 0x7fdd90075fe0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fddb01f8c50 tx=0x7fdd9c009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:20.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:20.997+0000 7fddad600640 1 -- 172.21.15.44:0/2223429996 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdda405f0b0 con 0x7fddb00cf4e0 2024-09-13T08:45:21.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.173+0000 7fddb8200640 1 -- 172.21.15.44:0/2223429996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddb01595c0 con 0x7fddb00cf4e0 2024-09-13T08:45:21.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.173+0000 7fddad600640 1 -- 172.21.15.44:0/2223429996 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16985 (secure 0 0 0) 0x7fdda4063ae0 con 0x7fddb00cf4e0 2024-09-13T08:45:21.176 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:21.176 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":64,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:45:18.118716+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:43:13.893284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-13T08:43:15.154456+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1c","mappings":[{"from":6,"to":1}]},{"pgid":"9.1e","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:45:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 -- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd90073b20 msgr2=0x7fdd90075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd90073b20 0x7fdd90075fe0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fddb01f8c50 tx=0x7fdd9c009060 comp rx=0 tx=0).stop 2024-09-13T08:45:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 -- 172.21.15.44:0/2223429996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb00cf4e0 msgr2=0x7fddb0154610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb00cf4e0 0x7fddb0154610 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdda400d9b0 tx=0x7fdda400de70 comp rx=0 tx=0).stop 2024-09-13T08:45:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 -- 172.21.15.44:0/2223429996 shutdown_connections 2024-09-13T08:45:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdd90073b20 0x7fdd90075fe0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fddb015c130 0x7fddb00c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddb015bd50 0x7fddb014b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 --2- 172.21.15.44:0/2223429996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb00cf4e0 0x7fddb0154610 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 -- 172.21.15.44:0/2223429996 >> 172.21.15.44:0/2223429996 conn(0x7fddb00d1ea0 msgr2=0x7fddb00cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 -- 172.21.15.44:0/2223429996 shutdown_connections 2024-09-13T08:45:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.177+0000 7fddb8200640 1 -- 172.21.15.44:0/2223429996 wait complete. 2024-09-13T08:45:21.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T08:45:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.345+0000 7f52e7a00640 1 -- 172.21.15.44:0/953961411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52e00cdb30 msgr2=0x7f52e00aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.345+0000 7f52e7a00640 1 --2- 172.21.15.44:0/953961411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52e00cdb30 0x7f52e00aeef0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009bc0 tx=0x7f52d801cba0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.345+0000 7f52e7a00640 1 -- 172.21.15.44:0/953961411 shutdown_connections 2024-09-13T08:45:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.345+0000 7f52e7a00640 1 --2- 172.21.15.44:0/953961411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52e00cdb30 0x7f52e00aeef0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.345+0000 7f52e7a00640 1 --2- 172.21.15.44:0/953961411 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f52e00ced80 0x7f52e00ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.345+0000 7f52e7a00640 1 -- 172.21.15.44:0/953961411 >> 172.21.15.44:0/953961411 conn(0x7f52e00d1dc0 msgr2=0x7f52e00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.345+0000 7f52e7a00640 1 -- 172.21.15.44:0/953961411 shutdown_connections 2024-09-13T08:45:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.345+0000 7f52e7a00640 1 -- 172.21.15.44:0/953961411 wait complete. 2024-09-13T08:45:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.345+0000 7f52e7a00640 1 Processor -- start 2024-09-13T08:45:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.345+0000 7f52e7a00640 1 -- start start 2024-09-13T08:45:21.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e7a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f52e00cdb30 0x7f52e0155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:21.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e7a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52e00ced80 0x7f52e015b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:21.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e7a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e015bea0 0x7f52e00b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:21.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e7a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52e0169340 con 0x7f52e00ced80 2024-09-13T08:45:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e7a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01691c0 con 0x7f52e015bea0 2024-09-13T08:45:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e7a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f52e01694c0 con 0x7f52e00cdb30 2024-09-13T08:45:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52e00ced80 0x7f52e015b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52e00ced80 0x7f52e015b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54016/0 (socket says 172.21.15.44:54016) 2024-09-13T08:45:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dfe00640 1 -- 172.21.15.44:0/7554210 learned_addr learned my addr 172.21.15.44:0/7554210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e5000640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f52e00cdb30 0x7f52e0155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e5a00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e015bea0 0x7f52e00b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dfe00640 1 -- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f52e00cdb30 msgr2=0x7f52e0155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dfe00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f52e00cdb30 0x7f52e0155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dfe00640 1 -- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e015bea0 msgr2=0x7f52e00b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dfe00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e015bea0 0x7f52e00b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dfe00640 1 -- 172.21.15.44:0/7554210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e00cb380 con 0x7f52e00ced80 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dfe00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52e00ced80 0x7f52e015b340 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00c970 tx=0x7f52cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dcc00640 1 -- 172.21.15.44:0/7554210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc007c70 con 0x7f52e00ced80 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dcc00640 1 -- 172.21.15.44:0/7554210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52cc00f450 con 0x7f52e00ced80 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e7a00640 1 -- 172.21.15.44:0/7554210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e00c8b80 con 0x7f52e00ced80 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52dcc00640 1 -- 172.21.15.44:0/7554210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc00f8a0 con 0x7f52e00ced80 2024-09-13T08:45:21.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e7a00640 1 -- 172.21.15.44:0/7554210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e00cab80 con 0x7f52e00ced80 2024-09-13T08:45:21.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.349+0000 7f52e7a00640 1 -- 172.21.15.44:0/7554210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e00ce670 con 0x7f52e00ced80 2024-09-13T08:45:21.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.353+0000 7f52dcc00640 1 -- 172.21.15.44:0/7554210 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f52cc00fa40 con 0x7f52e00ced80 2024-09-13T08:45:21.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.353+0000 7f52dcc00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f52c0073b20 0x7f52c0075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:21.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.353+0000 7f52dcc00640 1 -- 172.21.15.44:0/7554210 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5851+0+0 (secure 0 0 0) 0x7f52cc092ff0 con 0x7f52e00ced80 2024-09-13T08:45:21.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.353+0000 7f52e5000640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f52c0073b20 0x7f52c0075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:21.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.353+0000 7f52e5000640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f52c0073b20 0x7f52c0075fe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f52d4007fd0 tx=0x7f52d4007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.357+0000 7f52dcc00640 1 -- 172.21.15.44:0/7554210 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f52cc014030 con 0x7f52e00ced80 2024-09-13T08:45:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.573+0000 7f52e7a00640 1 -- 172.21.15.44:0/7554210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f52e00c8570 con 0x7f52e00ced80 2024-09-13T08:45:21.574 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T08:45:21.573+0000 7f03b3c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T08:45:21.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.573+0000 7f52dcc00640 1 -- 172.21.15.44:0/7554210 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7f52cc05f640 con 0x7f52e00ced80 2024-09-13T08:45:21.576 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 -- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f52c0073b20 msgr2=0x7f52c0075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f52c0073b20 0x7f52c0075fe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f52d4007fd0 tx=0x7f52d4007ea0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 -- 172.21.15.44:0/7554210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52e00ced80 msgr2=0x7f52e015b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52e00ced80 0x7f52e015b340 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00c970 tx=0x7f52cc00ce30 comp rx=0 tx=0).stop 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 -- 172.21.15.44:0/7554210 shutdown_connections 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f52c0073b20 0x7f52c0075fe0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e015bea0 0x7f52e00b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52e00ced80 0x7f52e015b340 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 --2- 172.21.15.44:0/7554210 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f52e00cdb30 0x7f52e0155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 -- 172.21.15.44:0/7554210 >> 172.21.15.44:0/7554210 conn(0x7f52e00d1dc0 msgr2=0x7f52e00ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 -- 172.21.15.44:0/7554210 shutdown_connections 2024-09-13T08:45:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.577+0000 7f52e7a00640 1 -- 172.21.15.44:0/7554210 wait complete. 2024-09-13T08:45:21.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T08:45:21.660 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:45:21.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.761+0000 7fdf64800640 1 -- 172.21.15.44:0/3307132703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c0d04d0 msgr2=0x7fdf5c0af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:21.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.761+0000 7fdf64800640 1 --2- 172.21.15.44:0/3307132703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c0d04d0 0x7fdf5c0af200 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009c80 tx=0x7fdf5801f5a0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.761+0000 7fdf64800640 1 -- 172.21.15.44:0/3307132703 shutdown_connections 2024-09-13T08:45:21.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.761+0000 7fdf64800640 1 --2- 172.21.15.44:0/3307132703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c0d04d0 0x7fdf5c0af200 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.761+0000 7fdf64800640 1 --2- 172.21.15.44:0/3307132703 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf5c0d29b0 0x7fdf5c0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.761+0000 7fdf64800640 1 -- 172.21.15.44:0/3307132703 >> 172.21.15.44:0/3307132703 conn(0x7fdf5c0d1e90 msgr2=0x7fdf5c0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.761+0000 7fdf64800640 1 -- 172.21.15.44:0/3307132703 shutdown_connections 2024-09-13T08:45:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.761+0000 7fdf64800640 1 -- 172.21.15.44:0/3307132703 wait complete. 2024-09-13T08:45:21.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 Processor -- start 2024-09-13T08:45:21.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 -- start start 2024-09-13T08:45:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf5c0d00f0 0x7fdf5c1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf5c0d04d0 0x7fdf5c0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c0d29b0 0x7fdf5c071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c0b0ed0 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c0b0d50 con 0x7fdf5c0d04d0 2024-09-13T08:45:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c0af740 con 0x7fdf5c0d00f0 2024-09-13T08:45:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf62800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c0d29b0 0x7fdf5c071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf62800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c0d29b0 0x7fdf5c071d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54056/0 (socket says 172.21.15.44:54056) 2024-09-13T08:45:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf62800640 1 -- 172.21.15.44:0/77294183 learned_addr learned my addr 172.21.15.44:0/77294183 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf61400640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf5c0d04d0 0x7fdf5c0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf61e00640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf5c0d00f0 0x7fdf5c1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf62800640 1 -- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf5c0d00f0 msgr2=0x7fdf5c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf62800640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf5c0d00f0 0x7fdf5c1540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf62800640 1 -- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf5c0d04d0 msgr2=0x7fdf5c0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf62800640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf5c0d04d0 0x7fdf5c0c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf62800640 1 -- 172.21.15.44:0/77294183 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c0b4d60 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf62800640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c0d29b0 0x7fdf5c071d10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdf5801dc50 tx=0x7fdf580af410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf56000640 1 -- 172.21.15.44:0/77294183 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58004e60 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf56000640 1 -- 172.21.15.44:0/77294183 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf580af8a0 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf56000640 1 -- 172.21.15.44:0/77294183 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf580afcf0 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 -- 172.21.15.44:0/77294183 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c0b5040 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 -- 172.21.15.44:0/77294183 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c0b5600 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.765+0000 7fdf64800640 1 -- 172.21.15.44:0/77294183 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c0af200 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.769+0000 7fdf56000640 1 -- 172.21.15.44:0/77294183 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fdf58004000 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.769+0000 7fdf56000640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf40073b20 0x7fdf40075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.769+0000 7fdf56000640 1 -- 172.21.15.44:0/77294183 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5909+0+0 (secure 0 0 0) 0x7fdf58131640 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.769+0000 7fdf61e00640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf40073b20 0x7fdf40075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:21.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.769+0000 7fdf61e00640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf40073b20 0x7fdf40075fe0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdf50007fd0 tx=0x7fdf50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:21.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.773+0000 7fdf56000640 1 -- 172.21.15.44:0/77294183 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdf580ac050 con 0x7fdf5c0d29b0 2024-09-13T08:45:21.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:21.981+0000 7fdf64800640 1 -- 172.21.15.44:0/77294183 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdf5c06fbc0 con 0x7fdf5c0d29b0 2024-09-13T08:45:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.577+0000 7fdf56000640 1 -- 172.21.15.44:0/77294183 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) ==== 104+0+0 (secure 0 0 0) 0x7fdf58019100 con 0x7fdf5c0d29b0 2024-09-13T08:45:22.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 -- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf40073b20 msgr2=0x7fdf40075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:22.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf40073b20 0x7fdf40075fe0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdf50007fd0 tx=0x7fdf50009040 comp rx=0 tx=0).stop 2024-09-13T08:45:22.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 -- 172.21.15.44:0/77294183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c0d29b0 msgr2=0x7fdf5c071d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:22.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c0d29b0 0x7fdf5c071d10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdf5801dc50 tx=0x7fdf580af410 comp rx=0 tx=0).stop 2024-09-13T08:45:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 -- 172.21.15.44:0/77294183 shutdown_connections 2024-09-13T08:45:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf40073b20 0x7fdf40075fe0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c0d29b0 0x7fdf5c071d10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf5c0d04d0 0x7fdf5c0c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 --2- 172.21.15.44:0/77294183 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf5c0d00f0 0x7fdf5c1540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 -- 172.21.15.44:0/77294183 >> 172.21.15.44:0/77294183 conn(0x7fdf5c0d1e90 msgr2=0x7fdf5c06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 -- 172.21.15.44:0/77294183 shutdown_connections 2024-09-13T08:45:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.581+0000 7fdf64800640 1 -- 172.21.15.44:0/77294183 wait complete. 2024-09-13T08:45:22.605 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T08:45:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 -- 172.21.15.44:0/1814631168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1015be10 msgr2=0x7efc100ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 --2- 172.21.15.44:0/1814631168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1015be10 0x7efc100ce700 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efc04009900 tx=0x7efc0401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:45:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 -- 172.21.15.44:0/1814631168 shutdown_connections 2024-09-13T08:45:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 --2- 172.21.15.44:0/1814631168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc100b0b60 0x7efc100ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 --2- 172.21.15.44:0/1814631168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1015be10 0x7efc100ce700 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:22.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 -- 172.21.15.44:0/1814631168 >> 172.21.15.44:0/1814631168 conn(0x7efc100d1da0 msgr2=0x7efc100d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:22.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 -- 172.21.15.44:0/1814631168 shutdown_connections 2024-09-13T08:45:22.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 -- 172.21.15.44:0/1814631168 wait complete. 2024-09-13T08:45:22.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 Processor -- start 2024-09-13T08:45:22.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 -- start start 2024-09-13T08:45:22.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc100b0b60 0x7efc100b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:22.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1015be10 0x7efc100c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:22.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc1015c1f0 0x7efc100c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:22.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efc10169040 con 0x7efc1015be10 2024-09-13T08:45:22.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc10168ec0 con 0x7efc1015c1f0 2024-09-13T08:45:22.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.761+0000 7efc17400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7efc101691c0 con 0x7efc100b0b60 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1015be10 0x7efc100c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1015be10 0x7efc100c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59748/0 (socket says 172.21.15.44:59748) 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0fe00640 1 -- 172.21.15.44:0/1682156078 learned_addr learned my addr 172.21.15.44:0/1682156078 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc14a00640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc100b0b60 0x7efc100b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc15400640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc1015c1f0 0x7efc100c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0fe00640 1 -- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc100b0b60 msgr2=0x7efc100b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0fe00640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc100b0b60 0x7efc100b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0fe00640 1 -- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc1015c1f0 msgr2=0x7efc100c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0fe00640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc1015c1f0 0x7efc100c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0fe00640 1 -- 172.21.15.44:0/1682156078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc100b34f0 con 0x7efc1015be10 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0fe00640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1015be10 0x7efc100c7c40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00b410 tx=0x7efbfc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0cc00640 1 -- 172.21.15.44:0/1682156078 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc015030 con 0x7efc1015be10 2024-09-13T08:45:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0cc00640 1 -- 172.21.15.44:0/1682156078 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efbfc004020 con 0x7efc1015be10 2024-09-13T08:45:22.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0cc00640 1 -- 172.21.15.44:0/1682156078 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0044b0 con 0x7efc1015be10 2024-09-13T08:45:22.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc17400640 1 -- 172.21.15.44:0/1682156078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc100b5960 con 0x7efc1015be10 2024-09-13T08:45:22.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc17400640 1 -- 172.21.15.44:0/1682156078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc100b5f20 con 0x7efc1015be10 2024-09-13T08:45:22.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc17400640 1 -- 172.21.15.44:0/1682156078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc100ccbd0 con 0x7efc1015be10 2024-09-13T08:45:22.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.765+0000 7efc0cc00640 1 -- 172.21.15.44:0/1682156078 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7efbfc018070 con 0x7efc1015be10 2024-09-13T08:45:22.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.769+0000 7efc0cc00640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efbf0073b50 0x7efbf0076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:22.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.769+0000 7efc0cc00640 1 -- 172.21.15.44:0/1682156078 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5909+0+0 (secure 0 0 0) 0x7efbfc092410 con 0x7efc1015be10 2024-09-13T08:45:22.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.769+0000 7efc14a00640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efbf0073b50 0x7efbf0076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.769+0000 7efc14a00640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efbf0073b50 0x7efbf0076010 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efc040af000 tx=0x7efc0401b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:22.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.773+0000 7efc0cc00640 1 -- 172.21.15.44:0/1682156078 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7efbfc05e840 con 0x7efc1015be10 2024-09-13T08:45:22.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:22.949+0000 7efc17400640 1 -- 172.21.15.44:0/1682156078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efc10070350 con 0x7efc1015be10 2024-09-13T08:45:23.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.585+0000 7efc0cc00640 1 -- 172.21.15.44:0/1682156078 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) ==== 181+0+0 (secure 0 0 0) 0x7efbfc063270 con 0x7efc1015be10 2024-09-13T08:45:23.586 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T08:45:23.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 -- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efbf0073b50 msgr2=0x7efbf0076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:23.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efbf0073b50 0x7efbf0076010 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efc040af000 tx=0x7efc0401b120 comp rx=0 tx=0).stop 2024-09-13T08:45:23.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 -- 172.21.15.44:0/1682156078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1015be10 msgr2=0x7efc100c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:23.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1015be10 0x7efc100c7c40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00b410 tx=0x7efbfc00b8d0 comp rx=0 tx=0).stop 2024-09-13T08:45:23.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 -- 172.21.15.44:0/1682156078 shutdown_connections 2024-09-13T08:45:23.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efbf0073b50 0x7efbf0076010 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:23.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc1015c1f0 0x7efc100c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:23.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1015be10 0x7efc100c7c40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:23.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 --2- 172.21.15.44:0/1682156078 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc100b0b60 0x7efc100b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:23.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 -- 172.21.15.44:0/1682156078 >> 172.21.15.44:0/1682156078 conn(0x7efc100d1da0 msgr2=0x7efc100af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:23.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 -- 172.21.15.44:0/1682156078 shutdown_connections 2024-09-13T08:45:23.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.589+0000 7efc17400640 1 -- 172.21.15.44:0/1682156078 wait complete. 2024-09-13T08:45:23.614 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.765+0000 7f6bb9400640 1 -- 172.21.15.44:0/2176767009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bb415aca0 msgr2=0x7f6bb4073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.765+0000 7f6bb9400640 1 --2- 172.21.15.44:0/2176767009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bb415aca0 0x7f6bb4073120 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40d1510 tx=0x7f6b9c01c600 comp rx=0 tx=0).stop 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.765+0000 7f6bb9400640 1 -- 172.21.15.44:0/2176767009 shutdown_connections 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.765+0000 7f6bb9400640 1 --2- 172.21.15.44:0/2176767009 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6bb4073660 0x7f6bb40ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.765+0000 7f6bb9400640 1 --2- 172.21.15.44:0/2176767009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bb415aca0 0x7f6bb4073120 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.765+0000 7f6bb9400640 1 --2- 172.21.15.44:0/2176767009 >> v2:172.21.15.129:3300/0 conn(0x7f6bb415a8c0 0x7f6bb415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.765+0000 7f6bb9400640 1 -- 172.21.15.44:0/2176767009 >> 172.21.15.44:0/2176767009 conn(0x7f6bb40b2140 msgr2=0x7f6bb40b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.765+0000 7f6bb9400640 1 -- 172.21.15.44:0/2176767009 shutdown_connections 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.765+0000 7f6bb9400640 1 -- 172.21.15.44:0/2176767009 wait complete. 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb9400640 1 Processor -- start 2024-09-13T08:45:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb9400640 1 -- start start 2024-09-13T08:45:23.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb4073660 0x7f6bb4150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:23.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bb415a8c0 0x7f6bb40b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:23.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb9400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6bb415aca0 0x7f6bb40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:23.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb9400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb40701b0 con 0x7f6bb415a8c0 2024-09-13T08:45:23.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb9400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4070030 con 0x7f6bb4073660 2024-09-13T08:45:23.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb9400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb4070330 con 0x7f6bb415aca0 2024-09-13T08:45:23.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6babe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bb415a8c0 0x7f6bb40b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:23.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6babe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bb415a8c0 0x7f6bb40b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59764/0 (socket says 172.21.15.44:59764) 2024-09-13T08:45:23.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6babe00640 1 -- 172.21.15.44:0/3533323264 learned_addr learned my addr 172.21.15.44:0/3533323264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb3200640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6bb415aca0 0x7f6bb40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb2800640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb4073660 0x7f6bb4150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6babe00640 1 -- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6bb415aca0 msgr2=0x7f6bb40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6babe00640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6bb415aca0 0x7f6bb40b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6babe00640 1 -- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb4073660 msgr2=0x7f6bb4150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6babe00640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb4073660 0x7f6bb4150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6babe00640 1 -- 172.21.15.44:0/3533323264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb415e0c0 con 0x7f6bb415a8c0 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6babe00640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bb415a8c0 0x7f6bb40b00b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009850 tx=0x7f6b9c0a9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6baaa00640 1 -- 172.21.15.44:0/3533323264 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0ad070 con 0x7f6bb415a8c0 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6baaa00640 1 -- 172.21.15.44:0/3533323264 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b9c0a9d50 con 0x7f6bb415a8c0 2024-09-13T08:45:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb9400640 1 -- 172.21.15.44:0/3533323264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb41fb540 con 0x7f6bb415a8c0 2024-09-13T08:45:23.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6baaa00640 1 -- 172.21.15.44:0/3533323264 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c005290 con 0x7f6bb415a8c0 2024-09-13T08:45:23.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.769+0000 7f6bb9400640 1 -- 172.21.15.44:0/3533323264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb41fbb00 con 0x7f6bb415a8c0 2024-09-13T08:45:23.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.773+0000 7f6bb9400640 1 -- 172.21.15.44:0/3533323264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb40b5f80 con 0x7f6bb415a8c0 2024-09-13T08:45:23.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.773+0000 7f6baaa00640 1 -- 172.21.15.44:0/3533323264 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f6b9c0b0070 con 0x7f6bb415a8c0 2024-09-13T08:45:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.773+0000 7f6baaa00640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6b90073b20 0x7f6b90075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.773+0000 7f6baaa00640 1 -- 172.21.15.44:0/3533323264 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5471+0+0 (secure 0 0 0) 0x7f6b9c12cd90 con 0x7f6bb415a8c0 2024-09-13T08:45:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.773+0000 7f6bb2800640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6b90073b20 0x7f6b90075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.773+0000 7f6bb2800640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6b90073b20 0x7f6b90075fe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40b5180 tx=0x7f6ba400bf10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:23.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.777+0000 7f6baaa00640 1 -- 172.21.15.44:0/3533323264 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6bb40b5f80 con 0x7f6bb415a8c0 2024-09-13T08:45:23.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:23.957+0000 7f6bb9400640 1 -- 172.21.15.44:0/3533323264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6bb4151400 con 0x7f6bb415a8c0 2024-09-13T08:45:24.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.585+0000 7f6baaa00640 1 -- 172.21.15.44:0/3533323264 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) ==== 169+0+0 (secure 0 0 0) 0x7f6bb415e0c0 con 0x7f6bb415a8c0 2024-09-13T08:45:24.588 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T08:45:24.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 -- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6b90073b20 msgr2=0x7f6b90075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:24.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6b90073b20 0x7f6b90075fe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40b5180 tx=0x7f6ba400bf10 comp rx=0 tx=0).stop 2024-09-13T08:45:24.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 -- 172.21.15.44:0/3533323264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bb415a8c0 msgr2=0x7f6bb40b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:24.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bb415a8c0 0x7f6bb40b00b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009850 tx=0x7f6b9c0a9500 comp rx=0 tx=0).stop 2024-09-13T08:45:24.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 -- 172.21.15.44:0/3533323264 shutdown_connections 2024-09-13T08:45:24.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6b90073b20 0x7f6b90075fe0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6bb415aca0 0x7f6bb40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bb415a8c0 0x7f6bb40b00b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 --2- 172.21.15.44:0/3533323264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb4073660 0x7f6bb4150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 -- 172.21.15.44:0/3533323264 >> 172.21.15.44:0/3533323264 conn(0x7f6bb40b2140 msgr2=0x7f6bb40cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:24.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 -- 172.21.15.44:0/3533323264 shutdown_connections 2024-09-13T08:45:24.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.589+0000 7f6bb9400640 1 -- 172.21.15.44:0/3533323264 wait complete. 2024-09-13T08:45:24.620 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-13T08:45:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.769+0000 7f7ab2400640 1 -- 172.21.15.44:0/3666762957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aac073820 msgr2=0x7f7aac073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.769+0000 7f7ab2400640 1 --2- 172.21.15.44:0/3666762957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aac073820 0x7f7aac073c70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009c80 tx=0x7f7aa401cd10 comp rx=0 tx=0).stop 2024-09-13T08:45:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.769+0000 7f7ab2400640 1 -- 172.21.15.44:0/3666762957 shutdown_connections 2024-09-13T08:45:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.769+0000 7f7ab2400640 1 --2- 172.21.15.44:0/3666762957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aac073820 0x7f7aac073c70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.769+0000 7f7ab2400640 1 --2- 172.21.15.44:0/3666762957 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7aac070ba0 0x7f7aac070fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.769+0000 7f7ab2400640 1 -- 172.21.15.44:0/3666762957 >> 172.21.15.44:0/3666762957 conn(0x7f7aac06dd30 msgr2=0x7f7aac06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.769+0000 7f7ab2400640 1 -- 172.21.15.44:0/3666762957 shutdown_connections 2024-09-13T08:45:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.769+0000 7f7ab2400640 1 -- 172.21.15.44:0/3666762957 wait complete. 2024-09-13T08:45:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 Processor -- start 2024-09-13T08:45:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 -- start start 2024-09-13T08:45:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aac070ba0 0x7f7aac14b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7aac073820 0x7f7aac159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7aac1518b0 0x7f7aac159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac167820 con 0x7f7aac070ba0 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1676a0 con 0x7f7aac073820 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7aac1679a0 con 0x7f7aac1518b0 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aac070ba0 0x7f7aac14b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aac070ba0 0x7f7aac14b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59784/0 (socket says 172.21.15.44:59784) 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab1000640 1 -- 172.21.15.44:0/579781506 learned_addr learned my addr 172.21.15.44:0/579781506 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab1a00640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7aac1518b0 0x7f7aac159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7aa3e00640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7aac073820 0x7f7aac159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab1000640 1 -- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7aac1518b0 msgr2=0x7f7aac159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab1000640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7aac1518b0 0x7f7aac159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab1000640 1 -- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7aac073820 msgr2=0x7f7aac159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:24.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab1000640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7aac073820 0x7f7aac159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab1000640 1 -- 172.21.15.44:0/579781506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac0b0970 con 0x7f7aac070ba0 2024-09-13T08:45:24.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab1000640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aac070ba0 0x7f7aac14b290 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00b440 tx=0x7f7a9c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:24.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7aa9600640 1 -- 172.21.15.44:0/579781506 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c015020 con 0x7f7aac070ba0 2024-09-13T08:45:24.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7aa9600640 1 -- 172.21.15.44:0/579781506 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a9c011980 con 0x7f7aac070ba0 2024-09-13T08:45:24.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7aa9600640 1 -- 172.21.15.44:0/579781506 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c011dd0 con 0x7f7aac070ba0 2024-09-13T08:45:24.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 -- 172.21.15.44:0/579781506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac0b0c50 con 0x7f7aac070ba0 2024-09-13T08:45:24.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 -- 172.21.15.44:0/579781506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac15cb40 con 0x7f7aac070ba0 2024-09-13T08:45:24.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.773+0000 7f7ab2400640 1 -- 172.21.15.44:0/579781506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aac070fc0 con 0x7f7aac070ba0 2024-09-13T08:45:24.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.777+0000 7f7aa9600640 1 -- 172.21.15.44:0/579781506 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f7a9c018020 con 0x7f7aac070ba0 2024-09-13T08:45:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.777+0000 7f7aa9600640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a88073b50 0x7f7a88076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.777+0000 7f7aa9600640 1 -- 172.21.15.44:0/579781506 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4999+0+0 (secure 0 0 0) 0x7f7a9c092700 con 0x7f7aac070ba0 2024-09-13T08:45:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.777+0000 7f7aa3e00640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a88073b50 0x7f7a88076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.777+0000 7f7aa3e00640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a88073b50 0x7f7a88076010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7aac14dae0 tx=0x7f7a9400b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.781+0000 7f7aa9600640 1 -- 172.21.15.44:0/579781506 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7a9c01f030 con 0x7f7aac070ba0 2024-09-13T08:45:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.957+0000 7f7ab2400640 1 -- 172.21.15.44:0/579781506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f7aac073e50 con 0x7f7aac070ba0 2024-09-13T08:45:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.961+0000 7f7aa9600640 1 -- 172.21.15.44:0/579781506 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v36) ==== 71+0+0 (secure 0 0 0) 0x7f7a9c00d040 con 0x7f7aac070ba0 2024-09-13T08:45:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 -- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a88073b50 msgr2=0x7f7a88076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a88073b50 0x7f7a88076010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7aac14dae0 tx=0x7f7a9400b390 comp rx=0 tx=0).stop 2024-09-13T08:45:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 -- 172.21.15.44:0/579781506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aac070ba0 msgr2=0x7f7aac14b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aac070ba0 0x7f7aac14b290 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00b440 tx=0x7f7a9c00b900 comp rx=0 tx=0).stop 2024-09-13T08:45:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 -- 172.21.15.44:0/579781506 shutdown_connections 2024-09-13T08:45:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7a88073b50 0x7f7a88076010 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7aac1518b0 0x7f7aac159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7aac073820 0x7f7aac159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 --2- 172.21.15.44:0/579781506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aac070ba0 0x7f7aac14b290 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 -- 172.21.15.44:0/579781506 >> 172.21.15.44:0/579781506 conn(0x7f7aac06dd30 msgr2=0x7f7aac15a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 -- 172.21.15.44:0/579781506 shutdown_connections 2024-09-13T08:45:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:24.965+0000 7f7ab2400640 1 -- 172.21.15.44:0/579781506 wait complete. 2024-09-13T08:45:24.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-13T08:45:25.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 -- 172.21.15.44:0/528154183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b015df50 msgr2=0x7f81b006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 --2- 172.21.15.44:0/528154183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b015df50 0x7f81b006e740 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f81b0069eb0 tx=0x7f81a001c830 comp rx=0 tx=0).stop 2024-09-13T08:45:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 -- 172.21.15.44:0/528154183 shutdown_connections 2024-09-13T08:45:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 --2- 172.21.15.44:0/528154183 >> v2:172.21.15.129:3300/0 conn(0x7f81b00b11d0 0x7f81b006ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 --2- 172.21.15.44:0/528154183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b015df50 0x7f81b006e740 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 -- 172.21.15.44:0/528154183 >> 172.21.15.44:0/528154183 conn(0x7f81b00d1dc0 msgr2=0x7f81b00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 -- 172.21.15.44:0/528154183 shutdown_connections 2024-09-13T08:45:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 -- 172.21.15.44:0/528154183 wait complete. 2024-09-13T08:45:25.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 Processor -- start 2024-09-13T08:45:25.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 -- start start 2024-09-13T08:45:25.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b00b11d0 0x7f81b01549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:25.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81b015df50 0x7f81b0155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:25.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81b015e330 0x7f81b00b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:25.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81b00d0860 con 0x7f81b00b11d0 2024-09-13T08:45:25.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f81b00d06e0 con 0x7f81b015e330 2024-09-13T08:45:25.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b6c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f81b00d09e0 con 0x7f81b015df50 2024-09-13T08:45:25.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81afe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b00b11d0 0x7f81b01549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81afe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b00b11d0 0x7f81b01549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59812/0 (socket says 172.21.15.44:59812) 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81afe00640 1 -- 172.21.15.44:0/77083999 learned_addr learned my addr 172.21.15.44:0/77083999 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81a7400640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81b015df50 0x7f81b0155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.133+0000 7f81b4c00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81b015e330 0x7f81b00b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81afe00640 1 -- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81b015df50 msgr2=0x7f81b0155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81afe00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81b015df50 0x7f81b0155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81afe00640 1 -- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81b015e330 msgr2=0x7f81b00b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81afe00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81b015e330 0x7f81b00b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81afe00640 1 -- 172.21.15.44:0/77083999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b01558c0 con 0x7f81b00b11d0 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81afe00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b00b11d0 0x7f81b01549c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f81a00b0040 tx=0x7f81a00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81acc00640 1 -- 172.21.15.44:0/77083999 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a00b2040 con 0x7f81b00b11d0 2024-09-13T08:45:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81acc00640 1 -- 172.21.15.44:0/77083999 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81a0003600 con 0x7f81b00b11d0 2024-09-13T08:45:25.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81b6c00640 1 -- 172.21.15.44:0/77083999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b0155ba0 con 0x7f81b00b11d0 2024-09-13T08:45:25.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81acc00640 1 -- 172.21.15.44:0/77083999 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0003a50 con 0x7f81b00b11d0 2024-09-13T08:45:25.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81b6c00640 1 -- 172.21.15.44:0/77083999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b0156160 con 0x7f81b00b11d0 2024-09-13T08:45:25.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81b6c00640 1 -- 172.21.15.44:0/77083999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b00afcd0 con 0x7f81b00b11d0 2024-09-13T08:45:25.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81acc00640 1 -- 172.21.15.44:0/77083999 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f81a00bd060 con 0x7f81b00b11d0 2024-09-13T08:45:25.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81acc00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c073b20 0x7f818c075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:25.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.137+0000 7f81acc00640 1 -- 172.21.15.44:0/77083999 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4999+0+0 (secure 0 0 0) 0x7f81a012c330 con 0x7f81b00b11d0 2024-09-13T08:45:25.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.141+0000 7f81a7400640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c073b20 0x7f818c075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:25.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.141+0000 7f81a7400640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c073b20 0x7f818c075fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f81b01578b0 tx=0x7f8198008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:25.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.141+0000 7f81acc00640 1 -- 172.21.15.44:0/77083999 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f81a00164b0 con 0x7f81b00b11d0 2024-09-13T08:45:25.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.381+0000 7f81b6c00640 1 -- 172.21.15.44:0/77083999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7f81b015ac40 con 0x7f81b00b11d0 2024-09-13T08:45:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81acc00640 1 -- 172.21.15.44:0/77083999 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v341) ==== 167+0+0 (secure 0 0 0) 0x7f81a00f8db0 con 0x7f81b00b11d0 2024-09-13T08:45:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 -- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c073b20 msgr2=0x7f818c075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c073b20 0x7f818c075fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f81b01578b0 tx=0x7f8198008040 comp rx=0 tx=0).stop 2024-09-13T08:45:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 -- 172.21.15.44:0/77083999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b00b11d0 msgr2=0x7f81b01549c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b00b11d0 0x7f81b01549c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f81a00b0040 tx=0x7f81a00040c0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 -- 172.21.15.44:0/77083999 shutdown_connections 2024-09-13T08:45:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c073b20 0x7f818c075fe0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81b015e330 0x7f81b00b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81b015df50 0x7f81b0155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 --2- 172.21.15.44:0/77083999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b00b11d0 0x7f81b01549c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 -- 172.21.15.44:0/77083999 >> 172.21.15.44:0/77083999 conn(0x7f81b00d1dc0 msgr2=0x7f81b0071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 -- 172.21.15.44:0/77083999 shutdown_connections 2024-09-13T08:45:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.593+0000 7f81b6c00640 1 -- 172.21.15.44:0/77083999 wait complete. 2024-09-13T08:45:25.615 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-13T08:45:25.615 INFO:tasks.cephfs_test_runner:Starting test: test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that 2 progress bars are printed in output of "ceph status" 2024-09-13T08:45:25.615 INFO:tasks.cephfs_test_runner:test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-13T08:45:25.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:45:25.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 -- 172.21.15.44:0/2715689689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b015c5e0 msgr2=0x7f86b006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:25.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 --2- 172.21.15.44:0/2715689689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b015c5e0 0x7f86b006ec50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f86b0075140 tx=0x7f86a001c8c0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 -- 172.21.15.44:0/2715689689 shutdown_connections 2024-09-13T08:45:25.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 --2- 172.21.15.44:0/2715689689 >> v2:172.21.15.129:3300/0 conn(0x7f86b00744c0 0x7f86b0074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 --2- 172.21.15.44:0/2715689689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b015c5e0 0x7f86b006ec50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 -- 172.21.15.44:0/2715689689 >> 172.21.15.44:0/2715689689 conn(0x7f86b00b1c20 msgr2=0x7f86b00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:25.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 -- 172.21.15.44:0/2715689689 shutdown_connections 2024-09-13T08:45:25.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 -- 172.21.15.44:0/2715689689 wait complete. 2024-09-13T08:45:25.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 Processor -- start 2024-09-13T08:45:25.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 -- start start 2024-09-13T08:45:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86b0072c90 0x7f86b014dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86b00744c0 0x7f86b014a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b015c5e0 0x7f86b00ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01523e0 con 0x7f86b015c5e0 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0152260 con 0x7f86b0072c90 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86b4a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f86b0152560 con 0x7f86b00744c0 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86ae800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b015c5e0 0x7f86b00ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86ae800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b015c5e0 0x7f86b00ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59830/0 (socket says 172.21.15.44:59830) 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86ae800640 1 -- 172.21.15.44:0/4068138249 learned_addr learned my addr 172.21.15.44:0/4068138249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86ade00640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86b0072c90 0x7f86b014dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86a5400640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86b00744c0 0x7f86b014a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86ae800640 1 -- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86b00744c0 msgr2=0x7f86b014a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86ae800640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86b00744c0 0x7f86b014a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86ae800640 1 -- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86b0072c90 msgr2=0x7f86b014dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86ae800640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86b0072c90 0x7f86b014dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86ae800640 1 -- 172.21.15.44:0/4068138249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b014fc70 con 0x7f86b015c5e0 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86a5400640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86b00744c0 0x7f86b014a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.773+0000 7f86ae800640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b015c5e0 0x7f86b00ceb30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f86a800ea20 tx=0x7f86a800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.777+0000 7f86a6a00640 1 -- 172.21.15.44:0/4068138249 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a800cde0 con 0x7f86b015c5e0 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.777+0000 7f86a6a00640 1 -- 172.21.15.44:0/4068138249 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86a8004c40 con 0x7f86b015c5e0 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.777+0000 7f86b4a00640 1 -- 172.21.15.44:0/4068138249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b01535d0 con 0x7f86b015c5e0 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.777+0000 7f86a6a00640 1 -- 172.21.15.44:0/4068138249 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8005280 con 0x7f86b015c5e0 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.777+0000 7f86b4a00640 1 -- 172.21.15.44:0/4068138249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b014b9a0 con 0x7f86b015c5e0 2024-09-13T08:45:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.777+0000 7f86b4a00640 1 -- 172.21.15.44:0/4068138249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b01533c0 con 0x7f86b015c5e0 2024-09-13T08:45:25.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.777+0000 7f86a6a00640 1 -- 172.21.15.44:0/4068138249 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f86a8005420 con 0x7f86b015c5e0 2024-09-13T08:45:25.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.777+0000 7f86a6a00640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f868c073b20 0x7f868c075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:25.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.777+0000 7f86a6a00640 1 -- 172.21.15.44:0/4068138249 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4999+0+0 (secure 0 0 0) 0x7f86a8093110 con 0x7f86b015c5e0 2024-09-13T08:45:25.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.777+0000 7f86ade00640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f868c073b20 0x7f868c075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:25.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.781+0000 7f86ade00640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f868c073b20 0x7f868c075fe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f86a00af000 tx=0x7f86a00a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:25.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.781+0000 7f86a6a00640 1 -- 172.21.15.44:0/4068138249 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f86a8016030 con 0x7f86b015c5e0 2024-09-13T08:45:25.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.957+0000 7f86b4a00640 1 -- 172.21.15.44:0/4068138249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86b0074930 con 0x7f86b015c5e0 2024-09-13T08:45:25.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86a6a00640 1 -- 172.21.15.44:0/4068138249 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+13131 (secure 0 0 0) 0x7f86a805f760 con 0x7f86b015c5e0 2024-09-13T08:45:25.962 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:25.962 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":67,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:45:24.587135+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/2755626228":"2024-09-14T08:45:21.575624+0000","172.21.15.44:6833/2755626228":"2024-09-14T08:45:21.575624+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:45:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 -- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f868c073b20 msgr2=0x7f868c075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f868c073b20 0x7f868c075fe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f86a00af000 tx=0x7f86a00a7040 comp rx=0 tx=0).stop 2024-09-13T08:45:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 -- 172.21.15.44:0/4068138249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b015c5e0 msgr2=0x7f86b00ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b015c5e0 0x7f86b00ceb30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f86a800ea20 tx=0x7f86a800eee0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 -- 172.21.15.44:0/4068138249 shutdown_connections 2024-09-13T08:45:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f868c073b20 0x7f868c075fe0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b015c5e0 0x7f86b00ceb30 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86b00744c0 0x7f86b014a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 --2- 172.21.15.44:0/4068138249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86b0072c90 0x7f86b014dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 -- 172.21.15.44:0/4068138249 >> 172.21.15.44:0/4068138249 conn(0x7f86b00b1c20 msgr2=0x7f86b009b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 -- 172.21.15.44:0/4068138249 shutdown_connections 2024-09-13T08:45:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:25.961+0000 7f86b4a00640 1 -- 172.21.15.44:0/4068138249 wait complete. 2024-09-13T08:45:25.987 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T08:37:24.563344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T08:45:25.987 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T08:45:26.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 -- 172.21.15.44:0/4240372848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73781614f0 msgr2=0x7f73780d4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:26.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 --2- 172.21.15.44:0/4240372848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73781614f0 0x7f73780d4ab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7368005fd0 tx=0x7f736801c760 comp rx=0 tx=0).stop 2024-09-13T08:45:26.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 -- 172.21.15.44:0/4240372848 shutdown_connections 2024-09-13T08:45:26.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 --2- 172.21.15.44:0/4240372848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73781614f0 0x7f73780d4ab0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 -- 172.21.15.44:0/4240372848 >> 172.21.15.44:0/4240372848 conn(0x7f73780d3f90 msgr2=0x7f73780d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:26.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 -- 172.21.15.44:0/4240372848 shutdown_connections 2024-09-13T08:45:26.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 -- 172.21.15.44:0/4240372848 wait complete. 2024-09-13T08:45:26.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 Processor -- start 2024-09-13T08:45:26.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 -- start start 2024-09-13T08:45:26.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73780d2570 0x7f737815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:26.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73781614f0 0x7f7378160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:26.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7378162530 0x7f73780b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:26.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f737816ca70 con 0x7f73781614f0 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f737816c8f0 con 0x7f73780d2570 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f737ca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f737816cbf0 con 0x7f7378162530 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7375400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73781614f0 0x7f7378160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7375400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73781614f0 0x7f7378160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59852/0 (socket says 172.21.15.44:59852) 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7375400640 1 -- 172.21.15.44:0/4236656236 learned_addr learned my addr 172.21.15.44:0/4236656236 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7375e00640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73780d2570 0x7f737815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7376800640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7378162530 0x7f73780b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7375400640 1 -- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7378162530 msgr2=0x7f73780b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7375400640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7378162530 0x7f73780b9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7375400640 1 -- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73780d2570 msgr2=0x7f737815fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7375400640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73780d2570 0x7f737815fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7375400640 1 -- 172.21.15.44:0/4236656236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737815baa0 con 0x7f73781614f0 2024-09-13T08:45:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.137+0000 7f7376800640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7378162530 0x7f73780b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:26.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.141+0000 7f7375400640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73781614f0 0x7f7378160550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f736400c3c0 tx=0x7f736400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:26.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.141+0000 7f736e000640 1 -- 172.21.15.44:0/4236656236 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364017070 con 0x7f73781614f0 2024-09-13T08:45:26.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.141+0000 7f736e000640 1 -- 172.21.15.44:0/4236656236 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73640053c0 con 0x7f73781614f0 2024-09-13T08:45:26.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.141+0000 7f736e000640 1 -- 172.21.15.44:0/4236656236 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364003860 con 0x7f73781614f0 2024-09-13T08:45:26.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.141+0000 7f737ca00640 1 -- 172.21.15.44:0/4236656236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737815bd80 con 0x7f73781614f0 2024-09-13T08:45:26.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.141+0000 7f737ca00640 1 -- 172.21.15.44:0/4236656236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737815c340 con 0x7f73781614f0 2024-09-13T08:45:26.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.141+0000 7f737ca00640 1 -- 172.21.15.44:0/4236656236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737815c540 con 0x7f73781614f0 2024-09-13T08:45:26.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.141+0000 7f736e000640 1 -- 172.21.15.44:0/4236656236 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f736401a070 con 0x7f73781614f0 2024-09-13T08:45:26.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.141+0000 7f736e000640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7358073b50 0x7f7358076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:26.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.141+0000 7f736e000640 1 -- 172.21.15.44:0/4236656236 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4999+0+0 (secure 0 0 0) 0x7f7364091c60 con 0x7f73781614f0 2024-09-13T08:45:26.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.145+0000 7f7375e00640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7358073b50 0x7f7358076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:26.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.145+0000 7f7375e00640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7358073b50 0x7f7358076010 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f73680ae000 tx=0x7f736801afa0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:26.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.145+0000 7f736e000640 1 -- 172.21.15.44:0/4236656236 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f737815c540 con 0x7f73781614f0 2024-09-13T08:45:26.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 -- 172.21.15.44:0/4236656236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73780b28d0 con 0x7f73781614f0 2024-09-13T08:45:26.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f736e000640 1 -- 172.21.15.44:0/4236656236 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7f736405e090 con 0x7f73781614f0 2024-09-13T08:45:26.326 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T08:45:26.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 -- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7358073b50 msgr2=0x7f7358076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:26.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7358073b50 0x7f7358076010 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f73680ae000 tx=0x7f736801afa0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 -- 172.21.15.44:0/4236656236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73781614f0 msgr2=0x7f7378160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:26.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73781614f0 0x7f7378160550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f736400c3c0 tx=0x7f736400c880 comp rx=0 tx=0).stop 2024-09-13T08:45:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 -- 172.21.15.44:0/4236656236 shutdown_connections 2024-09-13T08:45:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7358073b50 0x7f7358076010 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7378162530 0x7f73780b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73781614f0 0x7f7378160550 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 --2- 172.21.15.44:0/4236656236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73780d2570 0x7f737815fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 -- 172.21.15.44:0/4236656236 >> 172.21.15.44:0/4236656236 conn(0x7f73780d3f90 msgr2=0x7f737806ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 -- 172.21.15.44:0/4236656236 shutdown_connections 2024-09-13T08:45:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.325+0000 7f737ca00640 1 -- 172.21.15.44:0/4236656236 wait complete. 2024-09-13T08:45:26.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-13T08:45:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.493+0000 7ff3ddc00640 1 -- 172.21.15.44:0/62962796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3d8071740 msgr2=0x7ff3d80aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.493+0000 7ff3ddc00640 1 --2- 172.21.15.44:0/62962796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3d8071740 0x7ff3d80aeb00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80732c0 tx=0x7ff3c001eff0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.493+0000 7ff3ddc00640 1 -- 172.21.15.44:0/62962796 shutdown_connections 2024-09-13T08:45:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.493+0000 7ff3ddc00640 1 --2- 172.21.15.44:0/62962796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3d8071740 0x7ff3d80aeb00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.493+0000 7ff3ddc00640 1 --2- 172.21.15.44:0/62962796 >> v2:172.21.15.129:3300/0 conn(0x7ff3d8071360 0x7ff3d806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.493+0000 7ff3ddc00640 1 -- 172.21.15.44:0/62962796 >> 172.21.15.44:0/62962796 conn(0x7ff3d809aa10 msgr2=0x7ff3d809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.493+0000 7ff3ddc00640 1 -- 172.21.15.44:0/62962796 shutdown_connections 2024-09-13T08:45:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.493+0000 7ff3ddc00640 1 -- 172.21.15.44:0/62962796 wait complete. 2024-09-13T08:45:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 Processor -- start 2024-09-13T08:45:26.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 -- start start 2024-09-13T08:45:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3d8071360 0x7ff3d815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3d8071740 0x7ff3d80ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d80af040 0x7ff3d809ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d80b6b80 con 0x7ff3d8071360 2024-09-13T08:45:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d815e040 con 0x7ff3d80af040 2024-09-13T08:45:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d809f9c0 con 0x7ff3d8071740 2024-09-13T08:45:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3d8071360 0x7ff3d815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3d8071360 0x7ff3d815b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59868/0 (socket says 172.21.15.44:59868) 2024-09-13T08:45:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d7e00640 1 -- 172.21.15.44:0/474228368 learned_addr learned my addr 172.21.15.44:0/474228368 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3dd200640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d80af040 0x7ff3d809ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3cf400640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3d8071740 0x7ff3d80ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d7e00640 1 -- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3d8071740 msgr2=0x7ff3d80ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d7e00640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3d8071740 0x7ff3d80ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d7e00640 1 -- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d80af040 msgr2=0x7ff3d809ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:26.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d7e00640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d80af040 0x7ff3d809ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:26.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d7e00640 1 -- 172.21.15.44:0/474228368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d80cbf90 con 0x7ff3d8071360 2024-09-13T08:45:26.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d7e00640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3d8071360 0x7ff3d815b780 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800d950 tx=0x7ff3c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d4c00640 1 -- 172.21.15.44:0/474228368 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8017070 con 0x7ff3d8071360 2024-09-13T08:45:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d4c00640 1 -- 172.21.15.44:0/474228368 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3c8009d20 con 0x7ff3d8071360 2024-09-13T08:45:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3d4c00640 1 -- 172.21.15.44:0/474228368 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c80106c0 con 0x7ff3d8071360 2024-09-13T08:45:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 -- 172.21.15.44:0/474228368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d80cc2d0 con 0x7ff3d8071360 2024-09-13T08:45:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 -- 172.21.15.44:0/474228368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d815bcc0 con 0x7ff3d8071360 2024-09-13T08:45:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.497+0000 7ff3ddc00640 1 -- 172.21.15.44:0/474228368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d80af5b0 con 0x7ff3d8071360 2024-09-13T08:45:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.501+0000 7ff3d4c00640 1 -- 172.21.15.44:0/474228368 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7ff3c8010860 con 0x7ff3d8071360 2024-09-13T08:45:26.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.501+0000 7ff3d4c00640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3b4073b20 0x7ff3b4075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:26.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.501+0000 7ff3d4c00640 1 -- 172.21.15.44:0/474228368 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4999+0+0 (secure 0 0 0) 0x7ff3c8092c70 con 0x7ff3d8071360 2024-09-13T08:45:26.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.501+0000 7ff3cf400640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3b4073b20 0x7ff3b4075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:26.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.501+0000 7ff3cf400640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3b4073b20 0x7ff3b4075fe0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00b2000 tx=0x7ff3c001d830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:26.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.505+0000 7ff3d4c00640 1 -- 172.21.15.44:0/474228368 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff3d80cc2d0 con 0x7ff3d8071360 2024-09-13T08:45:26.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:26.745+0000 7ff3ddc00640 1 -- 172.21.15.44:0/474228368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7ff3d806e1b0 con 0x7ff3d8071360 2024-09-13T08:45:27.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3d4c00640 1 -- 172.21.15.44:0/474228368 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v343) ==== 170+0+0 (secure 0 0 0) 0x7ff3c805f910 con 0x7ff3d8071360 2024-09-13T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 -- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3b4073b20 msgr2=0x7ff3b4075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3b4073b20 0x7ff3b4075fe0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00b2000 tx=0x7ff3c001d830 comp rx=0 tx=0).stop 2024-09-13T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 -- 172.21.15.44:0/474228368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3d8071360 msgr2=0x7ff3d815b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3d8071360 0x7ff3d815b780 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800d950 tx=0x7ff3c800de10 comp rx=0 tx=0).stop 2024-09-13T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 -- 172.21.15.44:0/474228368 shutdown_connections 2024-09-13T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3b4073b20 0x7ff3b4075fe0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d80af040 0x7ff3d809ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3d8071740 0x7ff3d80ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 --2- 172.21.15.44:0/474228368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3d8071360 0x7ff3d815b780 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 -- 172.21.15.44:0/474228368 >> 172.21.15.44:0/474228368 conn(0x7ff3d809aa10 msgr2=0x7ff3d80afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 -- 172.21.15.44:0/474228368 shutdown_connections 2024-09-13T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.601+0000 7ff3ddc00640 1 -- 172.21.15.44:0/474228368 wait complete. 2024-09-13T08:45:27.627 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-13T08:45:27.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.777+0000 7f1f93200640 1 -- 172.21.15.44:0/3778536455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f78004810 msgr2=0x7f1f78004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:27.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.777+0000 7f1f93200640 1 --2- 172.21.15.44:0/3778536455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f78004810 0x7f1f78004c90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009b60 tx=0x7f1f8001d230 comp rx=0 tx=0).stop 2024-09-13T08:45:27.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.777+0000 7f1f92800640 1 -- 172.21.15.44:0/3778536455 reap_dead start 2024-09-13T08:45:27.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.777+0000 7f1f93200640 1 -- 172.21.15.44:0/3778536455 shutdown_connections 2024-09-13T08:45:27.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.777+0000 7f1f93200640 1 -- 172.21.15.44:0/3778536455 >> 172.21.15.44:0/3778536455 conn(0x7f1f940d3e00 msgr2=0x7f1f940d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:27.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.777+0000 7f1f93200640 1 -- 172.21.15.44:0/3778536455 shutdown_connections 2024-09-13T08:45:27.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.777+0000 7f1f93200640 1 -- 172.21.15.44:0/3778536455 wait complete. 2024-09-13T08:45:27.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 Processor -- start 2024-09-13T08:45:27.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 -- start start 2024-09-13T08:45:27.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f78004810 0x7f1f941f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:27.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f941f5890 0x7f1f9415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f9415a610 0x7f1f941f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f940d10d0 con 0x7f1f941f5890 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f940d0f50 con 0x7f1f9415a610 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1f940d1250 con 0x7f1f78004810 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f941f5890 0x7f1f9415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f941f5890 0x7f1f9415a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59886/0 (socket says 172.21.15.44:59886) 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91400640 1 -- 172.21.15.44:0/3717469537 learned_addr learned my addr 172.21.15.44:0/3717469537 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f92800640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f9415a610 0x7f1f941f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91e00640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f78004810 0x7f1f941f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91400640 1 -- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f78004810 msgr2=0x7f1f941f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91400640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f78004810 0x7f1f941f74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91400640 1 -- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f9415a610 msgr2=0x7f1f941f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:27.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91400640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f9415a610 0x7f1f941f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91400640 1 -- 172.21.15.44:0/3717469537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f941f8bd0 con 0x7f1f941f5890 2024-09-13T08:45:27.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91e00640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f78004810 0x7f1f941f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:27.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f91400640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f941f5890 0x7f1f9415a0d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009b60 tx=0x7f1f800acd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:27.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f8a000640 1 -- 172.21.15.44:0/3717469537 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f800bc030 con 0x7f1f941f5890 2024-09-13T08:45:27.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f8a000640 1 -- 172.21.15.44:0/3717469537 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f800ac400 con 0x7f1f941f5890 2024-09-13T08:45:27.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f8a000640 1 -- 172.21.15.44:0/3717469537 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f800b1040 con 0x7f1f941f5890 2024-09-13T08:45:27.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 -- 172.21.15.44:0/3717469537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f941f7a30 con 0x7f1f941f5890 2024-09-13T08:45:27.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 -- 172.21.15.44:0/3717469537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f940caf70 con 0x7f1f941f5890 2024-09-13T08:45:27.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.781+0000 7f1f93200640 1 -- 172.21.15.44:0/3717469537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f940b6f80 con 0x7f1f941f5890 2024-09-13T08:45:27.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.785+0000 7f1f8a000640 1 -- 172.21.15.44:0/3717469537 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f1f800bf070 con 0x7f1f941f5890 2024-09-13T08:45:27.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.785+0000 7f1f8a000640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f74073b50 0x7f1f74076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:27.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.785+0000 7f1f8a000640 1 -- 172.21.15.44:0/3717469537 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4999+0+0 (secure 0 0 0) 0x7f1f8012d860 con 0x7f1f941f5890 2024-09-13T08:45:27.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.785+0000 7f1f91e00640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f74073b50 0x7f1f74076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:27.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.785+0000 7f1f91e00640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f74073b50 0x7f1f74076010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400afd0 tx=0x7f1f84000ee0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:27.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.789+0000 7f1f8a000640 1 -- 172.21.15.44:0/3717469537 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1f80016f80 con 0x7f1f941f5890 2024-09-13T08:45:27.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.965+0000 7f1f93200640 1 -- 172.21.15.44:0/3717469537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f1f9415a9f0 con 0x7f1f941f5890 2024-09-13T08:45:27.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.965+0000 7f1f8a000640 1 -- 172.21.15.44:0/3717469537 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v36) ==== 59+0+1998 (secure 0 0 0) 0x7f1f800fa500 con 0x7f1f941f5890 2024-09-13T08:45:27.969 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-13T08:45:27.969 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-09-13T08:45:27.969 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-13T08:45:27.970 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-13T08:45:27.970 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4094.620852 2024-09-13T08:45:27.970 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3964.014083 2024-09-13T08:45:27.970 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3978.263337 2024-09-13T08:45:27.970 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4002.298162 2024-09-13T08:45:27.971 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4667.000623 2024-09-13T08:45:27.971 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4664.046790 2024-09-13T08:45:27.971 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6403.016405 2024-09-13T08:45:27.971 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4696.908546 2024-09-13T08:45:27.971 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-09-13T08:45:27.971 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-09-13T08:45:27.972 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 2/2 2024-09-13T08:45:27.972 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-09-13T08:45:27.972 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-09-13T08:45:27.972 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-13T08:45:27.972 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-09-13T08:45:27.973 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-13T08:45:27.973 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-09-13T08:45:27.973 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-09-13T08:45:27.973 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-09-13T08:45:27.973 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-09-13T08:45:27.973 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-09-13T08:45:27.974 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-09-13T08:45:27.974 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-09-13T08:45:27.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 -- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f74073b50 msgr2=0x7f1f74076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f74073b50 0x7f1f74076010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400afd0 tx=0x7f1f84000ee0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 -- 172.21.15.44:0/3717469537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f941f5890 msgr2=0x7f1f9415a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f941f5890 0x7f1f9415a0d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009b60 tx=0x7f1f800acd00 comp rx=0 tx=0).stop 2024-09-13T08:45:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 -- 172.21.15.44:0/3717469537 shutdown_connections 2024-09-13T08:45:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f74073b50 0x7f1f74076010 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f9415a610 0x7f1f941f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f941f5890 0x7f1f9415a0d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 --2- 172.21.15.44:0/3717469537 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f78004810 0x7f1f941f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 -- 172.21.15.44:0/3717469537 >> 172.21.15.44:0/3717469537 conn(0x7f1f940d3e00 msgr2=0x7f1f940b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 -- 172.21.15.44:0/3717469537 shutdown_connections 2024-09-13T08:45:27.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:27.969+0000 7f1f93200640 1 -- 172.21.15.44:0/3717469537 wait complete. 2024-09-13T08:45:27.990 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:45:28.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 -- 172.21.15.44:0/86852547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0b1650 msgr2=0x7fc32c0b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:28.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 --2- 172.21.15.44:0/86852547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0b1650 0x7fc32c0b1a30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc314009840 tx=0x7fc31401c820 comp rx=0 tx=0).stop 2024-09-13T08:45:28.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 -- 172.21.15.44:0/86852547 shutdown_connections 2024-09-13T08:45:28.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 --2- 172.21.15.44:0/86852547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0b1650 0x7fc32c0b1a30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 -- 172.21.15.44:0/86852547 >> 172.21.15.44:0/86852547 conn(0x7fc32c0d1dd0 msgr2=0x7fc32c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 -- 172.21.15.44:0/86852547 shutdown_connections 2024-09-13T08:45:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 -- 172.21.15.44:0/86852547 wait complete. 2024-09-13T08:45:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 Processor -- start 2024-09-13T08:45:28.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 -- start start 2024-09-13T08:45:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0afb10 0x7fc32c158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b1650 0x7fc32c158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c1605d0 0x7fc32c1faaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c0b1220 con 0x7fc32c0b1650 2024-09-13T08:45:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c0b0ee0 con 0x7fc32c0afb10 2024-09-13T08:45:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc332c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c0b13a0 con 0x7fc32c1605d0 2024-09-13T08:45:28.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc323e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b1650 0x7fc32c158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc323e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b1650 0x7fc32c158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59890/0 (socket says 172.21.15.44:59890) 2024-09-13T08:45:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc323e00640 1 -- 172.21.15.44:0/1573022000 learned_addr learned my addr 172.21.15.44:0/1573022000 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc32be00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0afb10 0x7fc32c158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc330c00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c1605d0 0x7fc32c1faaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc323e00640 1 -- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c1605d0 msgr2=0x7fc32c1faaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc323e00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c1605d0 0x7fc32c1faaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc323e00640 1 -- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0afb10 msgr2=0x7fc32c158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc323e00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0afb10 0x7fc32c158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.141+0000 7fc323e00640 1 -- 172.21.15.44:0/1573022000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c158770 con 0x7fc32c0b1650 2024-09-13T08:45:28.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.145+0000 7fc323e00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b1650 0x7fc32c158230 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc31401aed0 tx=0x7fc3140a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.145+0000 7fc328c00640 1 -- 172.21.15.44:0/1573022000 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314004040 con 0x7fc32c0b1650 2024-09-13T08:45:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.145+0000 7fc328c00640 1 -- 172.21.15.44:0/1573022000 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3140042d0 con 0x7fc32c0b1650 2024-09-13T08:45:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.145+0000 7fc328c00640 1 -- 172.21.15.44:0/1573022000 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314005050 con 0x7fc32c0b1650 2024-09-13T08:45:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.145+0000 7fc332c00640 1 -- 172.21.15.44:0/1573022000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c06f7f0 con 0x7fc32c0b1650 2024-09-13T08:45:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.145+0000 7fc332c00640 1 -- 172.21.15.44:0/1573022000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c06e880 con 0x7fc32c0b1650 2024-09-13T08:45:28.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.145+0000 7fc332c00640 1 -- 172.21.15.44:0/1573022000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c06fd10 con 0x7fc32c0b1650 2024-09-13T08:45:28.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.145+0000 7fc328c00640 1 -- 172.21.15.44:0/1573022000 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fc3140c1020 con 0x7fc32c0b1650 2024-09-13T08:45:28.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.145+0000 7fc328c00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073b50 0x7fc308076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:28.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.149+0000 7fc328c00640 1 -- 172.21.15.44:0/1573022000 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4999+0+0 (secure 0 0 0) 0x7fc31412d060 con 0x7fc32c0b1650 2024-09-13T08:45:28.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.149+0000 7fc32be00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073b50 0x7fc308076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:28.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.149+0000 7fc32be00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073b50 0x7fc308076010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc31c001130 tx=0x7fc31c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:28.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.149+0000 7fc328c00640 1 -- 172.21.15.44:0/1573022000 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc32c06fd10 con 0x7fc32c0b1650 2024-09-13T08:45:28.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.325+0000 7fc332c00640 1 -- 172.21.15.44:0/1573022000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc32c0b1a30 con 0x7fc32c0b1650 2024-09-13T08:45:28.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc328c00640 1 -- 172.21.15.44:0/1573022000 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7fc314102020 con 0x7fc32c0b1650 2024-09-13T08:45:28.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 -- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073b50 msgr2=0x7fc308076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:28.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073b50 0x7fc308076010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc31c001130 tx=0x7fc31c009040 comp rx=0 tx=0).stop 2024-09-13T08:45:28.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 -- 172.21.15.44:0/1573022000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b1650 msgr2=0x7fc32c158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:28.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b1650 0x7fc32c158230 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc31401aed0 tx=0x7fc3140a6570 comp rx=0 tx=0).stop 2024-09-13T08:45:28.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 -- 172.21.15.44:0/1573022000 shutdown_connections 2024-09-13T08:45:28.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc308073b50 0x7fc308076010 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc32c1605d0 0x7fc32c1faaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc32c0b1650 0x7fc32c158230 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 --2- 172.21.15.44:0/1573022000 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c0afb10 0x7fc32c158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 -- 172.21.15.44:0/1573022000 >> 172.21.15.44:0/1573022000 conn(0x7fc32c0d1dd0 msgr2=0x7fc32c0ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:28.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 -- 172.21.15.44:0/1573022000 shutdown_connections 2024-09-13T08:45:28.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.329+0000 7fc332c00640 1 -- 172.21.15.44:0/1573022000 wait complete. 2024-09-13T08:45:28.352 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:45:28.352 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:45:28.447 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:45:28.548 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:45:28.548 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:45:28.548 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:28.558 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:45:28.559 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:45:28.559 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:28.568 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:45:28.569 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:45:28.569 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:28.608 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:45:28.609 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:45:28.609 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:28.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:45:28.620 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T08:45:28.620 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:45:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.801+0000 7f0d54400640 1 -- 172.21.15.44:0/461527305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d340052f0 msgr2=0x7f0d3400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.801+0000 7f0d54400640 1 --2- 172.21.15.44:0/461527305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d340052f0 0x7f0d3400b9d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009eb0 tx=0x7f0d4401d5f0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.801+0000 7f0d54400640 1 -- 172.21.15.44:0/461527305 shutdown_connections 2024-09-13T08:45:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.801+0000 7f0d54400640 1 --2- 172.21.15.44:0/461527305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d340052f0 0x7f0d3400b9d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.801+0000 7f0d54400640 1 -- 172.21.15.44:0/461527305 >> 172.21.15.44:0/461527305 conn(0x7f0d4c0d1dc0 msgr2=0x7f0d4c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.801+0000 7f0d54400640 1 -- 172.21.15.44:0/461527305 shutdown_connections 2024-09-13T08:45:28.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.801+0000 7f0d54400640 1 -- 172.21.15.44:0/461527305 wait complete. 2024-09-13T08:45:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d54400640 1 Processor -- start 2024-09-13T08:45:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d54400640 1 -- start start 2024-09-13T08:45:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d54400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d340052f0 0x7f0d4c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d54400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c157070 0x7f0d4c1f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d54400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d4c0c7af0 0x7f0d4c151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d54400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c06e740 con 0x7f0d4c157070 2024-09-13T08:45:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d54400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c0b13e0 con 0x7f0d340052f0 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d54400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4c06e8c0 con 0x7f0d4c0c7af0 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c157070 0x7f0d4c1f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c157070 0x7f0d4c1f86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59904/0 (socket says 172.21.15.44:59904) 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51000640 1 -- 172.21.15.44:0/372505064 learned_addr learned my addr 172.21.15.44:0/372505064 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d52400640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d4c0c7af0 0x7f0d4c151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51000640 1 -- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d4c0c7af0 msgr2=0x7f0d4c151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51a00640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d340052f0 0x7f0d4c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51000640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d4c0c7af0 0x7f0d4c151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51000640 1 -- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d340052f0 msgr2=0x7f0d4c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51000640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d340052f0 0x7f0d4c0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51000640 1 -- 172.21.15.44:0/372505064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c0ceec0 con 0x7f0d4c157070 2024-09-13T08:45:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d52400640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d4c0c7af0 0x7f0d4c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51a00640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d340052f0 0x7f0d4c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d51000640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c157070 0x7f0d4c1f86f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0d38003f00 tx=0x7f0d38001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d49600640 1 -- 172.21.15.44:0/372505064 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38015070 con 0x7f0d4c157070 2024-09-13T08:45:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d49600640 1 -- 172.21.15.44:0/372505064 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d38011840 con 0x7f0d4c157070 2024-09-13T08:45:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d49600640 1 -- 172.21.15.44:0/372505064 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38011c90 con 0x7f0d4c157070 2024-09-13T08:45:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d54400640 1 -- 172.21.15.44:0/372505064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c1f8c30 con 0x7f0d4c157070 2024-09-13T08:45:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.805+0000 7f0d54400640 1 -- 172.21.15.44:0/372505064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c0c6d30 con 0x7f0d4c157070 2024-09-13T08:45:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.809+0000 7f0d54400640 1 -- 172.21.15.44:0/372505064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4c0afcd0 con 0x7f0d4c157070 2024-09-13T08:45:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.809+0000 7f0d49600640 1 -- 172.21.15.44:0/372505064 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f0d38018070 con 0x7f0d4c157070 2024-09-13T08:45:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.809+0000 7f0d49600640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d2c073b20 0x7f0d2c075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.809+0000 7f0d49600640 1 -- 172.21.15.44:0/372505064 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4999+0+0 (secure 0 0 0) 0x7f0d38092fc0 con 0x7f0d4c157070 2024-09-13T08:45:28.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.809+0000 7f0d51a00640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d2c073b20 0x7f0d2c075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:28.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.813+0000 7f0d51a00640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d2c073b20 0x7f0d2c075fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000fc0 tx=0x7f0d400067e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:28.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:28.813+0000 7f0d49600640 1 -- 172.21.15.44:0/372505064 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0d4c1f8c30 con 0x7f0d4c157070 2024-09-13T08:45:29.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.025+0000 7f0d54400640 1 -- 172.21.15.44:0/372505064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d4c0b4650 con 0x7f0d4c157070 2024-09-13T08:45:29.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.025+0000 7f0d49600640 1 -- 172.21.15.44:0/372505064 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+3911 (secure 0 0 0) 0x7f0d3805fcc0 con 0x7f0d4c157070 2024-09-13T08:45:29.027 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:29.027 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":40,"btime":"2024-09-13T08:45:22:577844+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1613950025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1613950025},{"type":"v1","addr":"172.21.15.129:6836","nonce":1613950025}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5402,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/10747232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":10747232},{"type":"v1","addr":"172.21.15.129:6837","nonce":10747232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3945609265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3945609265},{"type":"v1","addr":"172.21.15.44:6834","nonce":3945609265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2845713246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2845713246},{"type":"v1","addr":"172.21.15.44:6835","nonce":2845713246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[]} 2024-09-13T08:45:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 40 2024-09-13T08:45:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 -- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d2c073b20 msgr2=0x7f0d2c075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d2c073b20 0x7f0d2c075fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000fc0 tx=0x7f0d400067e0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 -- 172.21.15.44:0/372505064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c157070 msgr2=0x7f0d4c1f86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c157070 0x7f0d4c1f86f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0d38003f00 tx=0x7f0d38001640 comp rx=0 tx=0).stop 2024-09-13T08:45:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 -- 172.21.15.44:0/372505064 shutdown_connections 2024-09-13T08:45:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d2c073b20 0x7f0d2c075fe0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d4c0c7af0 0x7f0d4c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c157070 0x7f0d4c1f86f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 --2- 172.21.15.44:0/372505064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d340052f0 0x7f0d4c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 -- 172.21.15.44:0/372505064 >> 172.21.15.44:0/372505064 conn(0x7f0d4c0d1dc0 msgr2=0x7f0d4c0b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 -- 172.21.15.44:0/372505064 shutdown_connections 2024-09-13T08:45:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.029+0000 7f0d54400640 1 -- 172.21.15.44:0/372505064 wait complete. 2024-09-13T08:45:29.051 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T08:45:29.051 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T08:45:29.051 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T08:45:29.051 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:45:29.052 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T08:45:29.052 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T08:45:29.052 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T08:45:29.052 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:45:29.052 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T08:45:29.052 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T08:45:29.053 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T08:45:29.053 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:45:29.053 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T08:45:29.053 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T08:45:29.053 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T08:45:29.053 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:45:29.063 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T08:45:29.063 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T08:45:29.064 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T08:45:29.064 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T08:45:29.067 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T08:45:29.067 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T08:45:29.068 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T08:45:29.068 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T08:45:29.111 INFO:tasks.ceph.mds.b:Started 2024-09-13T08:45:29.112 DEBUG:teuthology.parallel:result is None 2024-09-13T08:45:29.112 INFO:tasks.ceph.mds.a:Started 2024-09-13T08:45:29.113 INFO:tasks.ceph.mds.c:Started 2024-09-13T08:45:29.113 DEBUG:teuthology.parallel:result is None 2024-09-13T08:45:29.113 DEBUG:teuthology.parallel:result is None 2024-09-13T08:45:29.114 INFO:tasks.ceph.mds.d:Started 2024-09-13T08:45:29.114 DEBUG:teuthology.parallel:result is None 2024-09-13T08:45:29.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T08:45:29.198 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:45:29.199 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:45:29.206 INFO:tasks.ceph.mds.b.smithi129.stdout:starting mds.b at 2024-09-13T08:45:29.207 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:45:29.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.265+0000 7f7e58800640 1 -- 172.21.15.44:0/487862332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e5015bd50 msgr2=0x7f7e500ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:29.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.265+0000 7f7e58800640 1 --2- 172.21.15.44:0/487862332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e5015bd50 0x7f7e500ce5e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7e5006d870 tx=0x7f7e4401c6e0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.265+0000 7f7e58800640 1 -- 172.21.15.44:0/487862332 shutdown_connections 2024-09-13T08:45:29.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.265+0000 7f7e58800640 1 --2- 172.21.15.44:0/487862332 >> v2:172.21.15.129:3300/0 conn(0x7f7e5015c130 0x7f7e500ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.265+0000 7f7e58800640 1 --2- 172.21.15.44:0/487862332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e5015bd50 0x7f7e500ce5e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.265+0000 7f7e58800640 1 -- 172.21.15.44:0/487862332 >> 172.21.15.44:0/487862332 conn(0x7f7e500d1ea0 msgr2=0x7f7e500d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:29.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.265+0000 7f7e58800640 1 -- 172.21.15.44:0/487862332 shutdown_connections 2024-09-13T08:45:29.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.265+0000 7f7e58800640 1 -- 172.21.15.44:0/487862332 wait complete. 2024-09-13T08:45:29.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.265+0000 7f7e58800640 1 Processor -- start 2024-09-13T08:45:29.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.265+0000 7f7e58800640 1 -- start start 2024-09-13T08:45:29.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e58800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500cf4e0 0x7f7e50150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:29.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e58800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e5015bd50 0x7f7e5014cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:29.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e58800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e5015c130 0x7f7e501548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:29.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e58800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50168e50 con 0x7f7e5015c130 2024-09-13T08:45:29.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e58800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50168cd0 con 0x7f7e5015bd50 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e58800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7e50168fd0 con 0x7f7e500cf4e0 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e56800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e5015c130 0x7f7e501548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e56800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e5015c130 0x7f7e501548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59954/0 (socket says 172.21.15.44:59954) 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e56800640 1 -- 172.21.15.44:0/2102828368 learned_addr learned my addr 172.21.15.44:0/2102828368 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e55e00640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500cf4e0 0x7f7e50150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e55400640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e5015bd50 0x7f7e5014cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e56800640 1 -- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500cf4e0 msgr2=0x7f7e50150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e56800640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500cf4e0 0x7f7e50150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e56800640 1 -- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e5015bd50 msgr2=0x7f7e5014cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e56800640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e5015bd50 0x7f7e5014cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e56800640 1 -- 172.21.15.44:0/2102828368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e50154e10 con 0x7f7e5015c130 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e56800640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e5015c130 0x7f7e501548d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00c730 tx=0x7f7e4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e4a000640 1 -- 172.21.15.44:0/2102828368 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c003db0 con 0x7f7e5015c130 2024-09-13T08:45:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e4a000640 1 -- 172.21.15.44:0/2102828368 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e4c00ce40 con 0x7f7e5015c130 2024-09-13T08:45:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e58800640 1 -- 172.21.15.44:0/2102828368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e50156200 con 0x7f7e5015c130 2024-09-13T08:45:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e4a000640 1 -- 172.21.15.44:0/2102828368 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0052d0 con 0x7f7e5015c130 2024-09-13T08:45:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e58800640 1 -- 172.21.15.44:0/2102828368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e50151680 con 0x7f7e5015c130 2024-09-13T08:45:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.269+0000 7f7e58800640 1 -- 172.21.15.44:0/2102828368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e500c5dd0 con 0x7f7e5015c130 2024-09-13T08:45:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.273+0000 7f7e4a000640 1 -- 172.21.15.44:0/2102828368 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f7e4c005470 con 0x7f7e5015c130 2024-09-13T08:45:29.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.273+0000 7f7e4a000640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e34073b20 0x7f7e34075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:29.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.273+0000 7f7e4a000640 1 -- 172.21.15.44:0/2102828368 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4999+0+0 (secure 0 0 0) 0x7f7e4c0924f0 con 0x7f7e5015c130 2024-09-13T08:45:29.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.273+0000 7f7e55e00640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e34073b20 0x7f7e34075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:29.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.273+0000 7f7e55e00640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e34073b20 0x7f7e34075fe0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7e440af000 tx=0x7f7e4401af20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:29.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.277+0000 7f7e4a000640 1 -- 172.21.15.44:0/2102828368 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7e4c05eba0 con 0x7f7e5015c130 2024-09-13T08:45:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.469+0000 7f7e58800640 1 -- 172.21.15.44:0/2102828368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7e500721a0 con 0x7f7e5015c130 2024-09-13T08:45:29.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.613+0000 7f7e4a000640 1 -- 172.21.15.44:0/2102828368 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v69) ==== 97+0+0 (secure 0 0 0) 0x7f7e4c0635d0 con 0x7f7e5015c130 2024-09-13T08:45:29.614 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-09-13T08:45:29.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 -- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e34073b20 msgr2=0x7f7e34075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:29.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e34073b20 0x7f7e34075fe0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7e440af000 tx=0x7f7e4401af20 comp rx=0 tx=0).stop 2024-09-13T08:45:29.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 -- 172.21.15.44:0/2102828368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e5015c130 msgr2=0x7f7e501548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:29.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e5015c130 0x7f7e501548d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00c730 tx=0x7f7e4c00cbf0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 -- 172.21.15.44:0/2102828368 shutdown_connections 2024-09-13T08:45:29.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e34073b20 0x7f7e34075fe0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e5015c130 0x7f7e501548d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e5015bd50 0x7f7e5014cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 --2- 172.21.15.44:0/2102828368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500cf4e0 0x7f7e50150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 -- 172.21.15.44:0/2102828368 >> 172.21.15.44:0/2102828368 conn(0x7f7e500d1ea0 msgr2=0x7f7e500b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:29.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 -- 172.21.15.44:0/2102828368 shutdown_connections 2024-09-13T08:45:29.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.617+0000 7f7e58800640 1 -- 172.21.15.44:0/2102828368 wait complete. 2024-09-13T08:45:29.640 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T08:45:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 -- 172.21.15.44:0/2171572753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbbc0ced80 msgr2=0x7fbbbc0ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 --2- 172.21.15.44:0/2171572753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbbc0ced80 0x7fbbbc0ae9b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009840 tx=0x7fbba801c820 comp rx=0 tx=0).stop 2024-09-13T08:45:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 -- 172.21.15.44:0/2171572753 shutdown_connections 2024-09-13T08:45:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 --2- 172.21.15.44:0/2171572753 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbbbc0cdb30 0x7fbbbc0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 --2- 172.21.15.44:0/2171572753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbbc0ced80 0x7fbbbc0ae9b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 -- 172.21.15.44:0/2171572753 >> 172.21.15.44:0/2171572753 conn(0x7fbbbc0d1dc0 msgr2=0x7fbbbc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 -- 172.21.15.44:0/2171572753 shutdown_connections 2024-09-13T08:45:29.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 -- 172.21.15.44:0/2171572753 wait complete. 2024-09-13T08:45:29.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 Processor -- start 2024-09-13T08:45:29.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 -- start start 2024-09-13T08:45:29.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc0cdb30 0x7fbbbc1fb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:29.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbbc0ced80 0x7fbbbc0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbbbc15bea0 0x7fbbbc070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc169340 con 0x7fbbbc0cdb30 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc1691c0 con 0x7fbbbc0ced80 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.789+0000 7fbbc0a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fbbbc1694c0 con 0x7fbbbc15bea0 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc0cdb30 0x7fbbbc1fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc0cdb30 0x7fbbbc1fb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59964/0 (socket says 172.21.15.44:59964) 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb9e00640 1 -- 172.21.15.44:0/931372996 learned_addr learned my addr 172.21.15.44:0/931372996 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb9400640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbbc0ced80 0x7fbbbc0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbba800640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbbbc15bea0 0x7fbbbc070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb9e00640 1 -- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbbbc15bea0 msgr2=0x7fbbbc070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb9e00640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbbbc15bea0 0x7fbbbc070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb9e00640 1 -- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbbc0ced80 msgr2=0x7fbbbc0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb9e00640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbbc0ced80 0x7fbbbc0ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb9e00640 1 -- 172.21.15.44:0/931372996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbbc1fba70 con 0x7fbbbc0cdb30 2024-09-13T08:45:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb9e00640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc0cdb30 0x7fbbbc1fb530 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00c430 tx=0x7fbbac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb2000640 1 -- 172.21.15.44:0/931372996 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac011790 con 0x7fbbbc0cdb30 2024-09-13T08:45:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb2000640 1 -- 172.21.15.44:0/931372996 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbac017070 con 0x7fbbbc0cdb30 2024-09-13T08:45:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb2000640 1 -- 172.21.15.44:0/931372996 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac011ce0 con 0x7fbbbc0cdb30 2024-09-13T08:45:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbc0a00640 1 -- 172.21.15.44:0/931372996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbbc1f76d0 con 0x7fbbbc0cdb30 2024-09-13T08:45:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbc0a00640 1 -- 172.21.15.44:0/931372996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbbc1f7c90 con 0x7fbbbc0cdb30 2024-09-13T08:45:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbc0a00640 1 -- 172.21.15.44:0/931372996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbbc0ce670 con 0x7fbbbc0cdb30 2024-09-13T08:45:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.793+0000 7fbbb2000640 1 -- 172.21.15.44:0/931372996 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fbbac01c460 con 0x7fbbbc0cdb30 2024-09-13T08:45:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.797+0000 7fbbb2000640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbb9c073b20 0x7fbb9c075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.797+0000 7fbbb2000640 1 -- 172.21.15.44:0/931372996 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4913+0+0 (secure 0 0 0) 0x7fbbac092460 con 0x7fbbbc0cdb30 2024-09-13T08:45:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.797+0000 7fbbb9400640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbb9c073b20 0x7fbb9c075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.797+0000 7fbbb9400640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbb9c073b20 0x7fbb9c075fe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbba80af000 tx=0x7fbba801b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:29.801+0000 7fbbb2000640 1 -- 172.21.15.44:0/931372996 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fbbbc1fba70 con 0x7fbbbc0cdb30 2024-09-13T08:45:30.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.033+0000 7fbbc0a00640 1 -- 172.21.15.44:0/931372996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbbbc1f82e0 con 0x7fbbbc0cdb30 2024-09-13T08:45:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.033+0000 7fbbb2000640 1 -- 172.21.15.44:0/931372996 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fbbac05f1c0 con 0x7fbbbc0cdb30 2024-09-13T08:45:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:30.036 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:45:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-09-13T08:45:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw==", 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-09-13T08:45:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ==", 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA==", 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:45:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw==", 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg==", 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg==", 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw==", 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig==", 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ==", 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g==", 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg==", 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg==", 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:45:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA==", 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w==", 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA==", 2024-09-13T08:45:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w==", 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:45:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmDD2LJxAA3s0jEb2EfPsLcu0w145G3Q==", 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ==", 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg==", 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:45:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:45:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T08:45:30.050 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:45:30.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 -- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbb9c073b20 msgr2=0x7fbb9c075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:30.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbb9c073b20 0x7fbb9c075fe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbba80af000 tx=0x7fbba801b060 comp rx=0 tx=0).stop 2024-09-13T08:45:30.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 -- 172.21.15.44:0/931372996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc0cdb30 msgr2=0x7fbbbc1fb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:30.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc0cdb30 0x7fbbbc1fb530 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00c430 tx=0x7fbbac00c830 comp rx=0 tx=0).stop 2024-09-13T08:45:30.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 -- 172.21.15.44:0/931372996 shutdown_connections 2024-09-13T08:45:30.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbb9c073b20 0x7fbb9c075fe0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbbbc15bea0 0x7fbbbc070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbbc0ced80 0x7fbbbc0ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 --2- 172.21.15.44:0/931372996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc0cdb30 0x7fbbbc1fb530 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 -- 172.21.15.44:0/931372996 >> 172.21.15.44:0/931372996 conn(0x7fbbbc0d1dc0 msgr2=0x7fbbbc0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:30.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 -- 172.21.15.44:0/931372996 shutdown_connections 2024-09-13T08:45:30.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.037+0000 7fbbc0a00640 1 -- 172.21.15.44:0/931372996 wait complete. 2024-09-13T08:45:30.058 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T08:45:30.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 -- 172.21.15.44:0/864922307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0000d28c0 msgr2=0x7fd0000cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:30.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 --2- 172.21.15.44:0/864922307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0000d28c0 0x7fd0000cea20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd000074290 tx=0x7fcfe801c8c0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 -- 172.21.15.44:0/864922307 shutdown_connections 2024-09-13T08:45:30.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 --2- 172.21.15.44:0/864922307 >> v2:172.21.15.129:3300/0 conn(0x7fd0000b53f0 0x7fd0000b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 --2- 172.21.15.44:0/864922307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0000d28c0 0x7fd0000cea20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 --2- 172.21.15.44:0/864922307 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd00015be80 0x7fd0000ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 -- 172.21.15.44:0/864922307 >> 172.21.15.44:0/864922307 conn(0x7fd0000d1da0 msgr2=0x7fd0000d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:30.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 -- 172.21.15.44:0/864922307 shutdown_connections 2024-09-13T08:45:30.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 -- 172.21.15.44:0/864922307 wait complete. 2024-09-13T08:45:30.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 Processor -- start 2024-09-13T08:45:30.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 -- start start 2024-09-13T08:45:30.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd0000b53f0 0x7fd000146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:30.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0000d28c0 0x7fd0000caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:30.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd00015be80 0x7fd0000c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:30.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0000ccfa0 con 0x7fd0000d28c0 2024-09-13T08:45:30.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd0000cce20 con 0x7fd00015be80 2024-09-13T08:45:30.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fd005a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd0000cd120 con 0x7fd0000b53f0 2024-09-13T08:45:30.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fcff6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0000d28c0 0x7fd0000caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:30.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fcff6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0000d28c0 0x7fd0000caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59978/0 (socket says 172.21.15.44:59978) 2024-09-13T08:45:30.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fcff6000640 1 -- 172.21.15.44:0/1977348212 learned_addr learned my addr 172.21.15.44:0/1977348212 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:30.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.205+0000 7fcffea00640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd0000b53f0 0x7fd000146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:30.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcfff400640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd00015be80 0x7fd0000c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:30.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcff6000640 1 -- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd0000b53f0 msgr2=0x7fd000146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:30.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcff6000640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd0000b53f0 0x7fd000146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcff6000640 1 -- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd00015be80 msgr2=0x7fd0000c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:30.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcff6000640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd00015be80 0x7fd0000c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcff6000640 1 -- 172.21.15.44:0/1977348212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0000cb580 con 0x7fd0000d28c0 2024-09-13T08:45:30.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcff6000640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0000d28c0 0x7fd0000caae0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80b0040 tx=0x7fcfe80a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:30.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcff7400640 1 -- 172.21.15.44:0/1977348212 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80b2040 con 0x7fd0000d28c0 2024-09-13T08:45:30.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcff7400640 1 -- 172.21.15.44:0/1977348212 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe80042e0 con 0x7fd0000d28c0 2024-09-13T08:45:30.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcff7400640 1 -- 172.21.15.44:0/1977348212 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8005290 con 0x7fd0000d28c0 2024-09-13T08:45:30.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fd005a00640 1 -- 172.21.15.44:0/1977348212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00006df40 con 0x7fd0000d28c0 2024-09-13T08:45:30.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fd005a00640 1 -- 172.21.15.44:0/1977348212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0001534f0 con 0x7fd0000d28c0 2024-09-13T08:45:30.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fd005a00640 1 -- 172.21.15.44:0/1977348212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00006dd30 con 0x7fd0000d28c0 2024-09-13T08:45:30.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.209+0000 7fcff7400640 1 -- 172.21.15.44:0/1977348212 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fd0000b65d0 con 0x7fd0000d28c0 2024-09-13T08:45:30.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.213+0000 7fcff7400640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfdc073b20 0x7fcfdc075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:30.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.213+0000 7fcff7400640 1 -- 172.21.15.44:0/1977348212 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4913+0+0 (secure 0 0 0) 0x7fcfe812c210 con 0x7fd0000d28c0 2024-09-13T08:45:30.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.213+0000 7fcffea00640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfdc073b20 0x7fcfdc075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:30.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.213+0000 7fcffea00640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfdc073b20 0x7fcfdc075fe0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcff0005fd0 tx=0x7fcff000b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:30.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.217+0000 7fcff7400640 1 -- 172.21.15.44:0/1977348212 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcfe80164b0 con 0x7fd0000d28c0 2024-09-13T08:45:30.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.397+0000 7fd005a00640 1 -- 172.21.15.44:0/1977348212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd0000b5880 con 0x7fd0000d28c0 2024-09-13T08:45:30.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.397+0000 7fcff7400640 1 -- 172.21.15.44:0/1977348212 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7fcfe80f8bb0 con 0x7fd0000d28c0 2024-09-13T08:45:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:30.399 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:45:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T08:45:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T08:45:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:30.400 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:45:30.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 -- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfdc073b20 msgr2=0x7fcfdc075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:30.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfdc073b20 0x7fcfdc075fe0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcff0005fd0 tx=0x7fcff000b390 comp rx=0 tx=0).stop 2024-09-13T08:45:30.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 -- 172.21.15.44:0/1977348212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0000d28c0 msgr2=0x7fd0000caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:30.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0000d28c0 0x7fd0000caae0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80b0040 tx=0x7fcfe80a56d0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 -- 172.21.15.44:0/1977348212 shutdown_connections 2024-09-13T08:45:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfdc073b20 0x7fcfdc075fe0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd00015be80 0x7fd0000c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0000d28c0 0x7fd0000caae0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 --2- 172.21.15.44:0/1977348212 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd0000b53f0 0x7fd000146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 -- 172.21.15.44:0/1977348212 >> 172.21.15.44:0/1977348212 conn(0x7fd0000d1da0 msgr2=0x7fd0000c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 -- 172.21.15.44:0/1977348212 shutdown_connections 2024-09-13T08:45:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.401+0000 7fd005a00640 1 -- 172.21.15.44:0/1977348212 wait complete. 2024-09-13T08:45:30.421 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T08:45:30.421 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T08:45:30.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.569+0000 7f5411800640 1 -- 172.21.15.44:0/4034974067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f540c0cea60 msgr2=0x7f540c06eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:30.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.569+0000 7f5411800640 1 --2- 172.21.15.44:0/4034974067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f540c0cea60 0x7f540c06eb40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f540c051480 tx=0x7f540401ddf0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.569+0000 7f5411800640 1 -- 172.21.15.44:0/4034974067 shutdown_connections 2024-09-13T08:45:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.569+0000 7f5411800640 1 --2- 172.21.15.44:0/4034974067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f540c0cea60 0x7f540c06eb40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.569+0000 7f5411800640 1 --2- 172.21.15.44:0/4034974067 >> v2:172.21.15.129:3300/0 conn(0x7f540c0734f0 0x7f540c15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.569+0000 7f5411800640 1 -- 172.21.15.44:0/4034974067 >> 172.21.15.44:0/4034974067 conn(0x7f540c0cfbc0 msgr2=0x7f540c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.569+0000 7f5411800640 1 -- 172.21.15.44:0/4034974067 shutdown_connections 2024-09-13T08:45:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.569+0000 7f5411800640 1 -- 172.21.15.44:0/4034974067 wait complete. 2024-09-13T08:45:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5411800640 1 Processor -- start 2024-09-13T08:45:30.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5411800640 1 -- start start 2024-09-13T08:45:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5411800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f540c0734f0 0x7f540c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5411800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f540c0cea60 0x7f540c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5411800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f540c0cf440 0x7f540c154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5411800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f540c06faa0 con 0x7f540c0734f0 2024-09-13T08:45:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5411800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f540c06f920 con 0x7f540c0cea60 2024-09-13T08:45:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5411800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f540c06fc20 con 0x7f540c0cf440 2024-09-13T08:45:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f540aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f540c0734f0 0x7f540c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f540aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f540c0734f0 0x7f540c15d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59990/0 (socket says 172.21.15.44:59990) 2024-09-13T08:45:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f540aa00640 1 -- 172.21.15.44:0/3505724202 learned_addr learned my addr 172.21.15.44:0/3505724202 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f540b400640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f540c0cf440 0x7f540c154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5403e00640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f540c0cea60 0x7f540c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f540aa00640 1 -- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f540c0cf440 msgr2=0x7f540c154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f540aa00640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f540c0cf440 0x7f540c154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f540aa00640 1 -- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f540c0cea60 msgr2=0x7f540c153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f540aa00640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f540c0cea60 0x7f540c153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f540aa00640 1 -- 172.21.15.44:0/3505724202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c15c410 con 0x7f540c0734f0 2024-09-13T08:45:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5403e00640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f540c0cea60 0x7f540c153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f540aa00640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f540c0734f0 0x7f540c15d4b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00c970 tx=0x7f53fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:30.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5402a00640 1 -- 172.21.15.44:0/3505724202 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc007bb0 con 0x7f540c0734f0 2024-09-13T08:45:30.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5402a00640 1 -- 172.21.15.44:0/3505724202 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53fc007e40 con 0x7f540c0734f0 2024-09-13T08:45:30.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5402a00640 1 -- 172.21.15.44:0/3505724202 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc005260 con 0x7f540c0734f0 2024-09-13T08:45:30.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5411800640 1 -- 172.21.15.44:0/3505724202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c15c6f0 con 0x7f540c0734f0 2024-09-13T08:45:30.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.573+0000 7f5411800640 1 -- 172.21.15.44:0/3505724202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c15ccb0 con 0x7f540c0734f0 2024-09-13T08:45:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.577+0000 7f5411800640 1 -- 172.21.15.44:0/3505724202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c069410 con 0x7f540c0734f0 2024-09-13T08:45:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.577+0000 7f5402a00640 1 -- 172.21.15.44:0/3505724202 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f53fc005400 con 0x7f540c0734f0 2024-09-13T08:45:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.577+0000 7f5402a00640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53e8073b20 0x7f53e8075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.577+0000 7f5402a00640 1 -- 172.21.15.44:0/3505724202 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4913+0+0 (secure 0 0 0) 0x7f53fc092e20 con 0x7f540c0734f0 2024-09-13T08:45:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.577+0000 7f5403e00640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53e8073b20 0x7f53e8075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:30.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.577+0000 7f5403e00640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53e8073b20 0x7f53e8075fe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f540c15bfc0 tx=0x7f53f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:30.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.581+0000 7f5402a00640 1 -- 172.21.15.44:0/3505724202 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f53fc014030 con 0x7f540c0734f0 2024-09-13T08:45:30.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:30.757+0000 7f5411800640 1 -- 172.21.15.44:0/3505724202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f540c15a9d0 con 0x7f540c0734f0 2024-09-13T08:45:31.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.625+0000 7f5402a00640 1 -- 172.21.15.44:0/3505724202 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f53fc05fac0 con 0x7f540c0734f0 2024-09-13T08:45:31.626 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T08:45:31.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 -- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53e8073b20 msgr2=0x7f53e8075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:31.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53e8073b20 0x7f53e8075fe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f540c15bfc0 tx=0x7f53f4009040 comp rx=0 tx=0).stop 2024-09-13T08:45:31.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 -- 172.21.15.44:0/3505724202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f540c0734f0 msgr2=0x7f540c15d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f540c0734f0 0x7f540c15d4b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00c970 tx=0x7f53fc00ce30 comp rx=0 tx=0).stop 2024-09-13T08:45:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 -- 172.21.15.44:0/3505724202 shutdown_connections 2024-09-13T08:45:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53e8073b20 0x7f53e8075fe0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f540c0cf440 0x7f540c154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f540c0cea60 0x7f540c153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 --2- 172.21.15.44:0/3505724202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f540c0734f0 0x7f540c15d4b0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 -- 172.21.15.44:0/3505724202 >> 172.21.15.44:0/3505724202 conn(0x7f540c0cfbc0 msgr2=0x7f540c14c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 -- 172.21.15.44:0/3505724202 shutdown_connections 2024-09-13T08:45:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.629+0000 7f5411800640 1 -- 172.21.15.44:0/3505724202 wait complete. 2024-09-13T08:45:31.659 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T08:45:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 -- 172.21.15.44:0/244961539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8e40afb10 msgr2=0x7ff8e40affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 --2- 172.21.15.44:0/244961539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8e40afb10 0x7ff8e40affc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009c80 tx=0x7ff8dc01cda0 comp rx=0 tx=0).stop 2024-09-13T08:45:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 -- 172.21.15.44:0/244961539 shutdown_connections 2024-09-13T08:45:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 --2- 172.21.15.44:0/244961539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8e40afb10 0x7ff8e40affc0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 -- 172.21.15.44:0/244961539 >> 172.21.15.44:0/244961539 conn(0x7ff8e40d1dd0 msgr2=0x7ff8e40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 -- 172.21.15.44:0/244961539 shutdown_connections 2024-09-13T08:45:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 -- 172.21.15.44:0/244961539 wait complete. 2024-09-13T08:45:31.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 Processor -- start 2024-09-13T08:45:31.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 -- start start 2024-09-13T08:45:31.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8e40afb10 0x7ff8e415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8e40b1650 0x7ff8e40b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8e41605d0 0x7ff8e40b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e416a6d0 con 0x7ff8e40b1650 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e416a550 con 0x7ff8e41605d0 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8e9200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e416a850 con 0x7ff8e40afb10 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8e40b1650 0x7ff8e40b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8e40b1650 0x7ff8e40b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60002/0 (socket says 172.21.15.44:60002) 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.817+0000 7ff8dbe00640 1 -- 172.21.15.44:0/1720216648 learned_addr learned my addr 172.21.15.44:0/1720216648 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8e2800640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8e41605d0 0x7ff8e40b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8e1e00640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8e40afb10 0x7ff8e415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8dbe00640 1 -- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8e40afb10 msgr2=0x7ff8e415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8dbe00640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8e40afb10 0x7ff8e415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:31.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8dbe00640 1 -- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8e41605d0 msgr2=0x7ff8e40b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:31.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8dbe00640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8e41605d0 0x7ff8e40b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:31.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8dbe00640 1 -- 172.21.15.44:0/1720216648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e4072430 con 0x7ff8e40b1650 2024-09-13T08:45:31.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8dbe00640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8e40b1650 0x7ff8e40b3850 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00c450 tx=0x7ff8cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:31.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8da000640 1 -- 172.21.15.44:0/1720216648 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc005480 con 0x7ff8e40b1650 2024-09-13T08:45:31.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8da000640 1 -- 172.21.15.44:0/1720216648 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8cc003600 con 0x7ff8e40b1650 2024-09-13T08:45:31.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8da000640 1 -- 172.21.15.44:0/1720216648 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc003a30 con 0x7ff8e40b1650 2024-09-13T08:45:31.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8e9200640 1 -- 172.21.15.44:0/1720216648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e40b5e10 con 0x7ff8e40b1650 2024-09-13T08:45:31.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8e9200640 1 -- 172.21.15.44:0/1720216648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e40b63d0 con 0x7ff8e40b1650 2024-09-13T08:45:31.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8e9200640 1 -- 172.21.15.44:0/1720216648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e40affc0 con 0x7ff8e40b1650 2024-09-13T08:45:31.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.821+0000 7ff8da000640 1 -- 172.21.15.44:0/1720216648 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7ff8cc003bd0 con 0x7ff8e40b1650 2024-09-13T08:45:31.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.825+0000 7ff8da000640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8c0073b20 0x7ff8c0075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:31.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.825+0000 7ff8da000640 1 -- 172.21.15.44:0/1720216648 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5283+0+0 (secure 0 0 0) 0x7ff8cc092d50 con 0x7ff8e40b1650 2024-09-13T08:45:31.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.825+0000 7ff8e1e00640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8c0073b20 0x7ff8c0075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:31.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.825+0000 7ff8e1e00640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8c0073b20 0x7ff8c0075fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4007fd0 tx=0x7ff8d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:31.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:31.825+0000 7ff8da000640 1 -- 172.21.15.44:0/1720216648 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff8cc05f3a0 con 0x7ff8e40b1650 2024-09-13T08:45:32.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.001+0000 7ff8e9200640 1 -- 172.21.15.44:0/1720216648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff8e40d1080 con 0x7ff8e40b1650 2024-09-13T08:45:32.020 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:45:32.024 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:45:32.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.629+0000 7ff8da000640 1 -- 172.21.15.44:0/1720216648 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7ff8cc063dd0 con 0x7ff8e40b1650 2024-09-13T08:45:32.629 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T08:45:32.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 -- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8c0073b20 msgr2=0x7ff8c0075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:32.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8c0073b20 0x7ff8c0075fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4007fd0 tx=0x7ff8d4009040 comp rx=0 tx=0).stop 2024-09-13T08:45:32.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 -- 172.21.15.44:0/1720216648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8e40b1650 msgr2=0x7ff8e40b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:32.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8e40b1650 0x7ff8e40b3850 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00c450 tx=0x7ff8cc00c910 comp rx=0 tx=0).stop 2024-09-13T08:45:32.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 -- 172.21.15.44:0/1720216648 shutdown_connections 2024-09-13T08:45:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8c0073b20 0x7ff8c0075fe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8e41605d0 0x7ff8e40b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8e40b1650 0x7ff8e40b3850 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 --2- 172.21.15.44:0/1720216648 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8e40afb10 0x7ff8e415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 -- 172.21.15.44:0/1720216648 >> 172.21.15.44:0/1720216648 conn(0x7ff8e40d1dd0 msgr2=0x7ff8e4166ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 -- 172.21.15.44:0/1720216648 shutdown_connections 2024-09-13T08:45:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.633+0000 7ff8e9200640 1 -- 172.21.15.44:0/1720216648 wait complete. 2024-09-13T08:45:32.676 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T08:45:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd18b600640 1 -- 172.21.15.44:0/519761019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c003a10 msgr2=0x7fd16c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd18b600640 1 --2- 172.21.15.44:0/519761019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c003a10 0x7fd16c003e30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c2e0 tx=0x7fd17801cfb0 comp rx=0 tx=0).stop 2024-09-13T08:45:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd189600640 1 -- 172.21.15.44:0/519761019 reap_dead start 2024-09-13T08:45:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd18b600640 1 -- 172.21.15.44:0/519761019 shutdown_connections 2024-09-13T08:45:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd18b600640 1 -- 172.21.15.44:0/519761019 >> 172.21.15.44:0/519761019 conn(0x7fd1840d1dd0 msgr2=0x7fd1840d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd18b600640 1 -- 172.21.15.44:0/519761019 shutdown_connections 2024-09-13T08:45:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd18b600640 1 -- 172.21.15.44:0/519761019 wait complete. 2024-09-13T08:45:32.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd18b600640 1 Processor -- start 2024-09-13T08:45:32.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd18b600640 1 -- start start 2024-09-13T08:45:32.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd18b600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd16c003a10 0x7fd1840cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:32.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.829+0000 7fd18b600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1840b75d0 0x7fd1841ec870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:32.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd18b600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1841f5fb0 0x7fd18415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:32.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd18b600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd184166d20 con 0x7fd1840b75d0 2024-09-13T08:45:32.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd18b600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd1840b1430 con 0x7fd16c003a10 2024-09-13T08:45:32.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd18b600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd184166ea0 con 0x7fd1841f5fb0 2024-09-13T08:45:32.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd183e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1840b75d0 0x7fd1841ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd183e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1840b75d0 0x7fd1841ec870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37904/0 (socket says 172.21.15.44:37904) 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd183e00640 1 -- 172.21.15.44:0/1534813321 learned_addr learned my addr 172.21.15.44:0/1534813321 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd188c00640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd16c003a10 0x7fd1840cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd189600640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1841f5fb0 0x7fd18415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd183e00640 1 -- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1841f5fb0 msgr2=0x7fd18415a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd183e00640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1841f5fb0 0x7fd18415a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd183e00640 1 -- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd16c003a10 msgr2=0x7fd1840cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd183e00640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd16c003a10 0x7fd1840cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd183e00640 1 -- 172.21.15.44:0/1534813321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1841f6980 con 0x7fd1840b75d0 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd189600640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1841f5fb0 0x7fd18415a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:32.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd183e00640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1840b75d0 0x7fd1841ec870 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd170011a50 tx=0x7fd170011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:32.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd180c00640 1 -- 172.21.15.44:0/1534813321 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17001f070 con 0x7fd1840b75d0 2024-09-13T08:45:32.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd180c00640 1 -- 172.21.15.44:0/1534813321 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd170012450 con 0x7fd1840b75d0 2024-09-13T08:45:32.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd180c00640 1 -- 172.21.15.44:0/1534813321 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170012800 con 0x7fd1840b75d0 2024-09-13T08:45:32.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd18b600640 1 -- 172.21.15.44:0/1534813321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1841f6c60 con 0x7fd1840b75d0 2024-09-13T08:45:32.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd18b600640 1 -- 172.21.15.44:0/1534813321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1841ec350 con 0x7fd1840b75d0 2024-09-13T08:45:32.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd18b600640 1 -- 172.21.15.44:0/1534813321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1840b1a30 con 0x7fd1840b75d0 2024-09-13T08:45:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.833+0000 7fd180c00640 1 -- 172.21.15.44:0/1534813321 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fd170022070 con 0x7fd1840b75d0 2024-09-13T08:45:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.837+0000 7fd180c00640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd164073b50 0x7fd164076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.837+0000 7fd180c00640 1 -- 172.21.15.44:0/1534813321 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5665+0+0 (secure 0 0 0) 0x7fd170093b70 con 0x7fd1840b75d0 2024-09-13T08:45:32.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.837+0000 7fd188c00640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd164073b50 0x7fd164076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:32.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.837+0000 7fd188c00640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd164073b50 0x7fd164076010 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd1780ac000 tx=0x7fd178019e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:32.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:32.841+0000 7fd180c00640 1 -- 172.21.15.44:0/1534813321 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd1700601c0 con 0x7fd1840b75d0 2024-09-13T08:45:33.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.053+0000 7fd18b600640 1 -- 172.21.15.44:0/1534813321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd184156190 con 0x7fd1840b75d0 2024-09-13T08:45:33.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd180c00640 1 -- 172.21.15.44:0/1534813321 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v43) ==== 174+0+0 (secure 0 0 0) 0x7fd170064bf0 con 0x7fd1840b75d0 2024-09-13T08:45:33.645 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 10 and data pool 11 2024-09-13T08:45:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 -- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd164073b50 msgr2=0x7fd164076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd164073b50 0x7fd164076010 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd1780ac000 tx=0x7fd178019e00 comp rx=0 tx=0).stop 2024-09-13T08:45:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 -- 172.21.15.44:0/1534813321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1840b75d0 msgr2=0x7fd1841ec870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1840b75d0 0x7fd1841ec870 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd170011a50 tx=0x7fd170011f10 comp rx=0 tx=0).stop 2024-09-13T08:45:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 -- 172.21.15.44:0/1534813321 shutdown_connections 2024-09-13T08:45:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd164073b50 0x7fd164076010 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1841f5fb0 0x7fd18415a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1840b75d0 0x7fd1841ec870 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 --2- 172.21.15.44:0/1534813321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd16c003a10 0x7fd1840cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 -- 172.21.15.44:0/1534813321 >> 172.21.15.44:0/1534813321 conn(0x7fd1840d1dd0 msgr2=0x7fd184158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:33.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 -- 172.21.15.44:0/1534813321 shutdown_connections 2024-09-13T08:45:33.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.645+0000 7fd18b600640 1 -- 172.21.15.44:0/1534813321 wait complete. 2024-09-13T08:45:33.695 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:45:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.841+0000 7fe19aa00640 1 -- 172.21.15.44:0/1575933970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19415e260 msgr2=0x7fe1940b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.841+0000 7fe19aa00640 1 --2- 172.21.15.44:0/1575933970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19415e260 0x7fe1940b2bf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe184009840 tx=0x7fe18401c790 comp rx=0 tx=0).stop 2024-09-13T08:45:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.841+0000 7fe19aa00640 1 -- 172.21.15.44:0/1575933970 shutdown_connections 2024-09-13T08:45:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.841+0000 7fe19aa00640 1 --2- 172.21.15.44:0/1575933970 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1940b4cf0 0x7fe1940b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.841+0000 7fe19aa00640 1 --2- 172.21.15.44:0/1575933970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19415e260 0x7fe1940b2bf0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.841+0000 7fe19aa00640 1 -- 172.21.15.44:0/1575933970 >> 172.21.15.44:0/1575933970 conn(0x7fe1940b18f0 msgr2=0x7fe1940b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.841+0000 7fe19aa00640 1 -- 172.21.15.44:0/1575933970 shutdown_connections 2024-09-13T08:45:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.841+0000 7fe19aa00640 1 -- 172.21.15.44:0/1575933970 wait complete. 2024-09-13T08:45:33.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.841+0000 7fe19aa00640 1 Processor -- start 2024-09-13T08:45:33.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.841+0000 7fe19aa00640 1 -- start start 2024-09-13T08:45:33.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe19aa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1940b4cf0 0x7fe1940adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:33.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe19aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19415e260 0x7fe19414a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:33.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe19aa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe194155910 0x7fe194151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:33.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe19aa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe194168380 con 0x7fe19415e260 2024-09-13T08:45:33.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe19aa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe194168200 con 0x7fe194155910 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe19aa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe194168500 con 0x7fe1940b4cf0 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe18be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19415e260 0x7fe19414a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe18be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19415e260 0x7fe19414a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37920/0 (socket says 172.21.15.44:37920) 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe18be00640 1 -- 172.21.15.44:0/1917986074 learned_addr learned my addr 172.21.15.44:0/1917986074 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe193e00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1940b4cf0 0x7fe1940adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe198a00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe194155910 0x7fe194151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe18be00640 1 -- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1940b4cf0 msgr2=0x7fe1940adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe18be00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1940b4cf0 0x7fe1940adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe18be00640 1 -- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe194155910 msgr2=0x7fe194151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe18be00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe194155910 0x7fe194151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe18be00640 1 -- 172.21.15.44:0/1917986074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe194153740 con 0x7fe19415e260 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe18be00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19415e260 0x7fe19414a240 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c450 tx=0x7fe17c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe190c00640 1 -- 172.21.15.44:0/1917986074 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c017070 con 0x7fe19415e260 2024-09-13T08:45:33.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe190c00640 1 -- 172.21.15.44:0/1917986074 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe17c003600 con 0x7fe19415e260 2024-09-13T08:45:33.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe19aa00640 1 -- 172.21.15.44:0/1917986074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1941fb8a0 con 0x7fe19415e260 2024-09-13T08:45:33.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe190c00640 1 -- 172.21.15.44:0/1917986074 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c003a50 con 0x7fe19415e260 2024-09-13T08:45:33.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe19aa00640 1 -- 172.21.15.44:0/1917986074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1941fbb60 con 0x7fe19415e260 2024-09-13T08:45:33.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.845+0000 7fe19aa00640 1 -- 172.21.15.44:0/1917986074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe194069410 con 0x7fe19415e260 2024-09-13T08:45:33.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.849+0000 7fe190c00640 1 -- 172.21.15.44:0/1917986074 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fe17c01a070 con 0x7fe19415e260 2024-09-13T08:45:33.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.849+0000 7fe190c00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170073b20 0x7fe170075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:33.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.849+0000 7fe190c00640 1 -- 172.21.15.44:0/1917986074 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fe17c091e30 con 0x7fe19415e260 2024-09-13T08:45:33.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.849+0000 7fe193e00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170073b20 0x7fe170075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:33.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.849+0000 7fe193e00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170073b20 0x7fe170075fe0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe1840af000 tx=0x7fe18401afd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:33.853+0000 7fe190c00640 1 -- 172.21.15.44:0/1917986074 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe17c096e20 con 0x7fe19415e260 2024-09-13T08:45:34.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.029+0000 7fe19aa00640 1 -- 172.21.15.44:0/1917986074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe19415e7d0 con 0x7fe19415e260 2024-09-13T08:45:34.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.029+0000 7fe190c00640 1 -- 172.21.15.44:0/1917986074 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16780 (secure 0 0 0) 0x7fe17c05e910 con 0x7fe19415e260 2024-09-13T08:45:34.032 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:34.032 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":74,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:45:33.055643+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:45:30.771920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T08:45:32.014594+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:45:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 -- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170073b20 msgr2=0x7fe170075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170073b20 0x7fe170075fe0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe1840af000 tx=0x7fe18401afd0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 -- 172.21.15.44:0/1917986074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19415e260 msgr2=0x7fe19414a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19415e260 0x7fe19414a240 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c450 tx=0x7fe17c00c910 comp rx=0 tx=0).stop 2024-09-13T08:45:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 -- 172.21.15.44:0/1917986074 shutdown_connections 2024-09-13T08:45:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170073b20 0x7fe170075fe0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe194155910 0x7fe194151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19415e260 0x7fe19414a240 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 --2- 172.21.15.44:0/1917986074 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1940b4cf0 0x7fe1940adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 -- 172.21.15.44:0/1917986074 >> 172.21.15.44:0/1917986074 conn(0x7fe1940b18f0 msgr2=0x7fe1940cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 -- 172.21.15.44:0/1917986074 shutdown_connections 2024-09-13T08:45:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.033+0000 7fe19aa00640 1 -- 172.21.15.44:0/1917986074 wait complete. 2024-09-13T08:45:34.056 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:45:34.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.201+0000 7f69d2200640 1 -- 172.21.15.44:0/3482687873 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69cc15e330 msgr2=0x7f69cc0b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.201+0000 7f69d2200640 1 --2- 172.21.15.44:0/3482687873 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69cc15e330 0x7f69cc0b0210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009bc0 tx=0x7f69c401cb10 comp rx=0 tx=0).stop 2024-09-13T08:45:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.201+0000 7f69d2200640 1 -- 172.21.15.44:0/3482687873 shutdown_connections 2024-09-13T08:45:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.201+0000 7f69d2200640 1 --2- 172.21.15.44:0/3482687873 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69cc15e330 0x7f69cc0b0210 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.201+0000 7f69d2200640 1 --2- 172.21.15.44:0/3482687873 >> v2:172.21.15.129:3300/0 conn(0x7f69cc06ee50 0x7f69cc0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.201+0000 7f69d2200640 1 -- 172.21.15.44:0/3482687873 >> 172.21.15.44:0/3482687873 conn(0x7f69cc0d1dc0 msgr2=0x7f69cc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.201+0000 7f69d2200640 1 -- 172.21.15.44:0/3482687873 shutdown_connections 2024-09-13T08:45:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.201+0000 7f69d2200640 1 -- 172.21.15.44:0/3482687873 wait complete. 2024-09-13T08:45:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 Processor -- start 2024-09-13T08:45:34.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 -- start start 2024-09-13T08:45:34.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69cc06ee50 0x7f69cc1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:34.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc15df50 0x7f69cc13ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:34.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69cc15e330 0x7f69cc1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:34.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc0d1080 con 0x7f69cc15df50 2024-09-13T08:45:34.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc0d0f00 con 0x7f69cc15e330 2024-09-13T08:45:34.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f69cc0d1200 con 0x7f69cc06ee50 2024-09-13T08:45:34.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc15df50 0x7f69cc13ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:34.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc15df50 0x7f69cc13ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37924/0 (socket says 172.21.15.44:37924) 2024-09-13T08:45:34.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3e00640 1 -- 172.21.15.44:0/104840440 learned_addr learned my addr 172.21.15.44:0/104840440 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:34.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69cb400640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69cc06ee50 0x7f69cc1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69cbe00640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69cc15e330 0x7f69cc1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3e00640 1 -- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69cc06ee50 msgr2=0x7f69cc1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3e00640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69cc06ee50 0x7f69cc1588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3e00640 1 -- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69cc15e330 msgr2=0x7f69cc1556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3e00640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69cc15e330 0x7f69cc1556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3e00640 1 -- 172.21.15.44:0/104840440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc13f8e0 con 0x7f69cc15df50 2024-09-13T08:45:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69cb400640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69cc06ee50 0x7f69cc1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69cbe00640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69cc15e330 0x7f69cc1556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3e00640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc15df50 0x7f69cc13ee20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f69b400e9f0 tx=0x7f69b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3400640 1 -- 172.21.15.44:0/104840440 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b400cd80 con 0x7f69cc15df50 2024-09-13T08:45:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3400640 1 -- 172.21.15.44:0/104840440 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69b4010430 con 0x7f69cc15df50 2024-09-13T08:45:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69c3400640 1 -- 172.21.15.44:0/104840440 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4010860 con 0x7f69cc15df50 2024-09-13T08:45:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 -- 172.21.15.44:0/104840440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69cc13fbc0 con 0x7f69cc15df50 2024-09-13T08:45:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 -- 172.21.15.44:0/104840440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69cc154bc0 con 0x7f69cc15df50 2024-09-13T08:45:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.205+0000 7f69d2200640 1 -- 172.21.15.44:0/104840440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69cc1553a0 con 0x7f69cc15df50 2024-09-13T08:45:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.209+0000 7f69c3400640 1 -- 172.21.15.44:0/104840440 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f69b4010a00 con 0x7f69cc15df50 2024-09-13T08:45:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.209+0000 7f69c3400640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69a8073b20 0x7f69a8075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.209+0000 7f69c3400640 1 -- 172.21.15.44:0/104840440 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f69b40933b0 con 0x7f69cc15df50 2024-09-13T08:45:34.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.209+0000 7f69cb400640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69a8073b20 0x7f69a8075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:34.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.209+0000 7f69cb400640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69a8073b20 0x7f69a8075fe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0b7da0 tx=0x7f69bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:34.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.213+0000 7f69c3400640 1 -- 172.21.15.44:0/104840440 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f69b4060110 con 0x7f69cc15df50 2024-09-13T08:45:34.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.389+0000 7f69d2200640 1 -- 172.21.15.44:0/104840440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69cc152680 con 0x7f69cc15df50 2024-09-13T08:45:34.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69c3400640 1 -- 172.21.15.44:0/104840440 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16780 (secure 0 0 0) 0x7f69b405fa10 con 0x7f69cc15df50 2024-09-13T08:45:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:34.394 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":74,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:45:33.055643+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:45:30.771920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T08:45:32.014594+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:45:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 -- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69a8073b20 msgr2=0x7f69a8075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69a8073b20 0x7f69a8075fe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0b7da0 tx=0x7f69bc009040 comp rx=0 tx=0).stop 2024-09-13T08:45:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 -- 172.21.15.44:0/104840440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc15df50 msgr2=0x7f69cc13ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc15df50 0x7f69cc13ee20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f69b400e9f0 tx=0x7f69b400eeb0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 -- 172.21.15.44:0/104840440 shutdown_connections 2024-09-13T08:45:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69a8073b20 0x7f69a8075fe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69cc15e330 0x7f69cc1556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc15df50 0x7f69cc13ee20 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 --2- 172.21.15.44:0/104840440 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69cc06ee50 0x7f69cc1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 -- 172.21.15.44:0/104840440 >> 172.21.15.44:0/104840440 conn(0x7f69cc0d1dc0 msgr2=0x7f69cc070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 -- 172.21.15.44:0/104840440 shutdown_connections 2024-09-13T08:45:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.393+0000 7f69d2200640 1 -- 172.21.15.44:0/104840440 wait complete. 2024-09-13T08:45:34.417 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T08:45:34.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.565+0000 7f30a0800640 1 -- 172.21.15.44:0/3808070661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f309815dde0 msgr2=0x7f3098161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.565+0000 7f30a0800640 1 --2- 172.21.15.44:0/3808070661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f309815dde0 0x7f3098161b70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3098162b90 tx=0x7f308c01c680 comp rx=0 tx=0).stop 2024-09-13T08:45:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.565+0000 7f30a0800640 1 -- 172.21.15.44:0/3808070661 shutdown_connections 2024-09-13T08:45:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.565+0000 7f30a0800640 1 --2- 172.21.15.44:0/3808070661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f309815dde0 0x7f3098161b70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.565+0000 7f30a0800640 1 -- 172.21.15.44:0/3808070661 >> 172.21.15.44:0/3808070661 conn(0x7f30980d0cd0 msgr2=0x7f30980d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.565+0000 7f30a0800640 1 -- 172.21.15.44:0/3808070661 shutdown_connections 2024-09-13T08:45:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.565+0000 7f30a0800640 1 -- 172.21.15.44:0/3808070661 wait complete. 2024-09-13T08:45:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 Processor -- start 2024-09-13T08:45:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 -- start start 2024-09-13T08:45:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30980d02e0 0x7f3098159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f309815dde0 0x7f30980cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f309815e1c0 0x7f309815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f30980cb6b0 con 0x7f309815e1c0 2024-09-13T08:45:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30980cb530 con 0x7f30980d02e0 2024-09-13T08:45:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f30980cb830 con 0x7f309815dde0 2024-09-13T08:45:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f309815e1c0 0x7f309815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f309815e1c0 0x7f309815af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37938/0 (socket says 172.21.15.44:37938) 2024-09-13T08:45:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309e800640 1 -- 172.21.15.44:0/874084634 learned_addr learned my addr 172.21.15.44:0/874084634 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309de00640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30980d02e0 0x7f3098159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309d400640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f309815dde0 0x7f30980cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309e800640 1 -- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f309815dde0 msgr2=0x7f30980cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309e800640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f309815dde0 0x7f30980cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309e800640 1 -- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30980d02e0 msgr2=0x7f3098159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309e800640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30980d02e0 0x7f3098159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309e800640 1 -- 172.21.15.44:0/874084634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3098159a80 con 0x7f309815e1c0 2024-09-13T08:45:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f309e800640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f309815e1c0 0x7f309815af60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f30940015b0 tx=0x7f309400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f3092000640 1 -- 172.21.15.44:0/874084634 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094016020 con 0x7f309815e1c0 2024-09-13T08:45:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 -- 172.21.15.44:0/874084634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309815e880 con 0x7f309815e1c0 2024-09-13T08:45:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f3092000640 1 -- 172.21.15.44:0/874084634 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f309400ab60 con 0x7f309815e1c0 2024-09-13T08:45:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f3092000640 1 -- 172.21.15.44:0/874084634 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094012420 con 0x7f309815e1c0 2024-09-13T08:45:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 -- 172.21.15.44:0/874084634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309815ee40 con 0x7f309815e1c0 2024-09-13T08:45:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.569+0000 7f30a0800640 1 -- 172.21.15.44:0/874084634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30980af660 con 0x7f309815e1c0 2024-09-13T08:45:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.573+0000 7f3092000640 1 -- 172.21.15.44:0/874084634 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f3094019020 con 0x7f309815e1c0 2024-09-13T08:45:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.573+0000 7f3092000640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f307c073b20 0x7f307c075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.573+0000 7f3092000640 1 -- 172.21.15.44:0/874084634 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f3094092dc0 con 0x7f309815e1c0 2024-09-13T08:45:34.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.573+0000 7f309de00640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f307c073b20 0x7f307c075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:34.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.573+0000 7f309de00640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f307c073b20 0x7f307c075fe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f308c0af000 tx=0x7f308c01aec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:34.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.577+0000 7f3092000640 1 -- 172.21.15.44:0/874084634 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3094020020 con 0x7f309815e1c0 2024-09-13T08:45:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:34.789+0000 7f30a0800640 1 -- 172.21.15.44:0/874084634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f30980b0ab0 con 0x7f309815e1c0 2024-09-13T08:45:35.623 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-09-13T08:45:35.621+0000 7fe1b4800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:45:35.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.669+0000 7f3092000640 1 -- 172.21.15.44:0/874084634 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f3098159a80 con 0x7f309815e1c0 2024-09-13T08:45:35.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 -- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f307c073b20 msgr2=0x7f307c075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:35.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f307c073b20 0x7f307c075fe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f308c0af000 tx=0x7f308c01aec0 comp rx=0 tx=0).stop 2024-09-13T08:45:35.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 -- 172.21.15.44:0/874084634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f309815e1c0 msgr2=0x7f309815af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f309815e1c0 0x7f309815af60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f30940015b0 tx=0x7f309400cd20 comp rx=0 tx=0).stop 2024-09-13T08:45:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 -- 172.21.15.44:0/874084634 shutdown_connections 2024-09-13T08:45:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f307c073b20 0x7f307c075fe0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f309815e1c0 0x7f309815af60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f309815dde0 0x7f30980cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 --2- 172.21.15.44:0/874084634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30980d02e0 0x7f3098159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 -- 172.21.15.44:0/874084634 >> 172.21.15.44:0/874084634 conn(0x7f30980d0cd0 msgr2=0x7f309815d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 -- 172.21.15.44:0/874084634 shutdown_connections 2024-09-13T08:45:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.673+0000 7f30a0800640 1 -- 172.21.15.44:0/874084634 wait complete. 2024-09-13T08:45:35.697 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T08:45:35.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T08:45:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.849+0000 7f06efe00640 1 -- 172.21.15.44:0/2776411685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00d22c0 msgr2=0x7f06f00cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.849+0000 7f06efe00640 1 --2- 172.21.15.44:0/2776411685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00d22c0 0x7f06f00cc620 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009c80 tx=0x7f06e801cda0 comp rx=0 tx=0).stop 2024-09-13T08:45:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.849+0000 7f06efe00640 1 -- 172.21.15.44:0/2776411685 shutdown_connections 2024-09-13T08:45:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.849+0000 7f06efe00640 1 --2- 172.21.15.44:0/2776411685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00d22c0 0x7f06f00cc620 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.849+0000 7f06efe00640 1 --2- 172.21.15.44:0/2776411685 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f015bd40 0x7f06f00d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:35.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.849+0000 7f06efe00640 1 -- 172.21.15.44:0/2776411685 >> 172.21.15.44:0/2776411685 conn(0x7f06f009aa10 msgr2=0x7f06f009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:35.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.849+0000 7f06efe00640 1 -- 172.21.15.44:0/2776411685 shutdown_connections 2024-09-13T08:45:35.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.849+0000 7f06efe00640 1 -- 172.21.15.44:0/2776411685 wait complete. 2024-09-13T08:45:35.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06efe00640 1 Processor -- start 2024-09-13T08:45:35.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06efe00640 1 -- start start 2024-09-13T08:45:35.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06efe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06f00b0d10 0x7f06f0147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:35.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00d22c0 0x7f06f0148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:35.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06efe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f015bd40 0x7f06f0148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:35.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06efe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0168d00 con 0x7f06f00d22c0 2024-09-13T08:45:35.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06efe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0168b80 con 0x7f06f00b0d10 2024-09-13T08:45:35.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06efe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f06f0168e80 con 0x7f06f015bd40 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00d22c0 0x7f06f0148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00d22c0 0x7f06f0148450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37962/0 (socket says 172.21.15.44:37962) 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e7e00640 1 -- 172.21.15.44:0/1124560178 learned_addr learned my addr 172.21.15.44:0/1124560178 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06eea00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06f00b0d10 0x7f06f0147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06ef400640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f015bd40 0x7f06f0148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e7e00640 1 -- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f015bd40 msgr2=0x7f06f0148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e7e00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f015bd40 0x7f06f0148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e7e00640 1 -- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06f00b0d10 msgr2=0x7f06f0147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e7e00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06f00b0d10 0x7f06f0147f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e7e00640 1 -- 172.21.15.44:0/1124560178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f00c77a0 con 0x7f06f00d22c0 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e7e00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00d22c0 0x7f06f0148450 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f06d800c530 tx=0x7f06d800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e6a00640 1 -- 172.21.15.44:0/1124560178 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8017070 con 0x7f06f00d22c0 2024-09-13T08:45:35.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e6a00640 1 -- 172.21.15.44:0/1124560178 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06d80053c0 con 0x7f06f00d22c0 2024-09-13T08:45:35.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06e6a00640 1 -- 172.21.15.44:0/1124560178 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8012890 con 0x7f06f00d22c0 2024-09-13T08:45:35.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06efe00640 1 -- 172.21.15.44:0/1124560178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f00c7a80 con 0x7f06f00d22c0 2024-09-13T08:45:35.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.853+0000 7f06efe00640 1 -- 172.21.15.44:0/1124560178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f00c8040 con 0x7f06f00d22c0 2024-09-13T08:45:35.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.857+0000 7f06efe00640 1 -- 172.21.15.44:0/1124560178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f00cc620 con 0x7f06f00d22c0 2024-09-13T08:45:35.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.857+0000 7f06e6a00640 1 -- 172.21.15.44:0/1124560178 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f06d801a070 con 0x7f06f00d22c0 2024-09-13T08:45:35.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.857+0000 7f06e6a00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06cc073b20 0x7f06cc075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:35.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.857+0000 7f06e6a00640 1 -- 172.21.15.44:0/1124560178 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f06d8091c80 con 0x7f06f00d22c0 2024-09-13T08:45:35.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.857+0000 7f06eea00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06cc073b20 0x7f06cc075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:35.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.857+0000 7f06eea00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06cc073b20 0x7f06cc075fe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f06e000bb50 tx=0x7f06e000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:35.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:35.861+0000 7f06e6a00640 1 -- 172.21.15.44:0/1124560178 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f06d805e090 con 0x7f06f00d22c0 2024-09-13T08:45:36.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.073+0000 7f06efe00640 1 -- 172.21.15.44:0/1124560178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f06f00cdc00 con 0x7f06f00d22c0 2024-09-13T08:45:36.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.673+0000 7f06e6a00640 1 -- 172.21.15.44:0/1124560178 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f06d8062ac0 con 0x7f06f00d22c0 2024-09-13T08:45:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 -- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06cc073b20 msgr2=0x7f06cc075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06cc073b20 0x7f06cc075fe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f06e000bb50 tx=0x7f06e000b2e0 comp rx=0 tx=0).stop 2024-09-13T08:45:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 -- 172.21.15.44:0/1124560178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00d22c0 msgr2=0x7f06f0148450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00d22c0 0x7f06f0148450 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f06d800c530 tx=0x7f06d800c9f0 comp rx=0 tx=0).stop 2024-09-13T08:45:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 -- 172.21.15.44:0/1124560178 shutdown_connections 2024-09-13T08:45:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06cc073b20 0x7f06cc075fe0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f015bd40 0x7f06f0148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00d22c0 0x7f06f0148450 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 --2- 172.21.15.44:0/1124560178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06f00b0d10 0x7f06f0147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 -- 172.21.15.44:0/1124560178 >> 172.21.15.44:0/1124560178 conn(0x7f06f009aa10 msgr2=0x7f06f00b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 -- 172.21.15.44:0/1124560178 shutdown_connections 2024-09-13T08:45:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.677+0000 7f06efe00640 1 -- 172.21.15.44:0/1124560178 wait complete. 2024-09-13T08:45:36.702 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T08:45:36.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:45:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.849+0000 7f3548600640 1 -- 172.21.15.44:0/610337306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354006ee50 msgr2=0x7f35400afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.849+0000 7f3548600640 1 --2- 172.21.15.44:0/610337306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354006ee50 0x7f35400afcd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3540069eb0 tx=0x7f352c01c830 comp rx=0 tx=0).stop 2024-09-13T08:45:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.849+0000 7f3548600640 1 -- 172.21.15.44:0/610337306 shutdown_connections 2024-09-13T08:45:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.849+0000 7f3548600640 1 --2- 172.21.15.44:0/610337306 >> v2:172.21.15.129:3300/0 conn(0x7f354015e330 0x7f35400b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.849+0000 7f3548600640 1 --2- 172.21.15.44:0/610337306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354006ee50 0x7f35400afcd0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.849+0000 7f3548600640 1 -- 172.21.15.44:0/610337306 >> 172.21.15.44:0/610337306 conn(0x7f35400d1dc0 msgr2=0x7f35400d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.849+0000 7f3548600640 1 -- 172.21.15.44:0/610337306 shutdown_connections 2024-09-13T08:45:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.849+0000 7f3548600640 1 -- 172.21.15.44:0/610337306 wait complete. 2024-09-13T08:45:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.849+0000 7f3548600640 1 Processor -- start 2024-09-13T08:45:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3548600640 1 -- start start 2024-09-13T08:45:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3548600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354006ee50 0x7f35401fb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3548600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f354015df50 0x7f35400c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3548600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f354015e330 0x7f3540145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3548600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f35400d0fd0 con 0x7f354006ee50 2024-09-13T08:45:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3548600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f35400d0e50 con 0x7f354015e330 2024-09-13T08:45:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3548600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f35400d1150 con 0x7f354015df50 2024-09-13T08:45:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3545c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354006ee50 0x7f35401fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3545c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354006ee50 0x7f35401fb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37966/0 (socket says 172.21.15.44:37966) 2024-09-13T08:45:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3545c00640 1 -- 172.21.15.44:0/3132193673 learned_addr learned my addr 172.21.15.44:0/3132193673 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3546600640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f354015e330 0x7f3540145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3545c00640 1 -- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f354015df50 msgr2=0x7f35400c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3545c00640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f354015df50 0x7f35400c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3545c00640 1 -- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f354015e330 msgr2=0x7f3540145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3545c00640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f354015e330 0x7f3540145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3545c00640 1 -- 172.21.15.44:0/3132193673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3540146050 con 0x7f354006ee50 2024-09-13T08:45:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3546600640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f354015e330 0x7f3540145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3545c00640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354006ee50 0x7f35401fb600 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f353400c500 tx=0x7f353400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:36.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f353d600640 1 -- 172.21.15.44:0/3132193673 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534017070 con 0x7f354006ee50 2024-09-13T08:45:36.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f353d600640 1 -- 172.21.15.44:0/3132193673 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35340053c0 con 0x7f354006ee50 2024-09-13T08:45:36.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f353d600640 1 -- 172.21.15.44:0/3132193673 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534012890 con 0x7f354006ee50 2024-09-13T08:45:36.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3548600640 1 -- 172.21.15.44:0/3132193673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35401fbc00 con 0x7f354006ee50 2024-09-13T08:45:36.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3548600640 1 -- 172.21.15.44:0/3132193673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35401fc1c0 con 0x7f354006ee50 2024-09-13T08:45:36.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.853+0000 7f3548600640 1 -- 172.21.15.44:0/3132193673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35400c5680 con 0x7f354006ee50 2024-09-13T08:45:36.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.857+0000 7f353d600640 1 -- 172.21.15.44:0/3132193673 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f353401a070 con 0x7f354006ee50 2024-09-13T08:45:36.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.857+0000 7f353d600640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3520073b20 0x7f3520075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:36.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.857+0000 7f353d600640 1 -- 172.21.15.44:0/3132193673 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f3534091e20 con 0x7f354006ee50 2024-09-13T08:45:36.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.857+0000 7f3545200640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3520073b20 0x7f3520075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:36.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.857+0000 7f3545200640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3520073b20 0x7f3520075fe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f352c0af000 tx=0x7f352c0195d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:36.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:36.861+0000 7f353d600640 1 -- 172.21.15.44:0/3132193673 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3540146050 con 0x7f354006ee50 2024-09-13T08:45:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 -- 172.21.15.44:0/3132193673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35400af850 con 0x7f354006ee50 2024-09-13T08:45:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f353d600640 1 -- 172.21.15.44:0/3132193673 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5228 (secure 0 0 0) 0x7f353405e7c0 con 0x7f354006ee50 2024-09-13T08:45:37.074 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:37.074 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":46,"btime":"2024-09-13T08:45:36:672438+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3854096181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3854096181},{"type":"v1","addr":"172.21.15.129:6836","nonce":3854096181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2183755853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":2183755853},{"type":"v1","addr":"172.21.15.129:6837","nonce":2183755853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6330,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1061181525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1061181525},{"type":"v1","addr":"172.21.15.44:6834","nonce":1061181525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:45:33.055598+0000","modified":"2024-09-13T08:45:36.075582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/651900305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":651900305},{"type":"v1","addr":"172.21.15.44:6835","nonce":651900305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6333,"qdb_cluster":[6333]},"id":5}]} 2024-09-13T08:45:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 46 2024-09-13T08:45:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 -- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3520073b20 msgr2=0x7f3520075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3520073b20 0x7f3520075fe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f352c0af000 tx=0x7f352c0195d0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 -- 172.21.15.44:0/3132193673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354006ee50 msgr2=0x7f35401fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354006ee50 0x7f35401fb600 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f353400c500 tx=0x7f353400c9c0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 -- 172.21.15.44:0/3132193673 shutdown_connections 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3520073b20 0x7f3520075fe0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f354015e330 0x7f3540145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f354015df50 0x7f35400c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 --2- 172.21.15.44:0/3132193673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354006ee50 0x7f35401fb600 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.073+0000 7f3548600640 1 -- 172.21.15.44:0/3132193673 >> 172.21.15.44:0/3132193673 conn(0x7f35400d1dc0 msgr2=0x7f3540072230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.077+0000 7f3548600640 1 -- 172.21.15.44:0/3132193673 shutdown_connections 2024-09-13T08:45:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.077+0000 7f3548600640 1 -- 172.21.15.44:0/3132193673 wait complete. 2024-09-13T08:45:37.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:45:37.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.241+0000 7fce0e400640 1 -- 172.21.15.44:0/445500197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce0806f8b0 msgr2=0x7fce0815dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.241+0000 7fce0e400640 1 --2- 172.21.15.44:0/445500197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce0806f8b0 0x7fce0815dae0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fce080af800 tx=0x7fcdf801c7a0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.241+0000 7fce0e400640 1 -- 172.21.15.44:0/445500197 shutdown_connections 2024-09-13T08:45:37.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.241+0000 7fce0e400640 1 --2- 172.21.15.44:0/445500197 >> v2:172.21.15.129:3300/0 conn(0x7fce080cd100 0x7fce080acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.241+0000 7fce0e400640 1 --2- 172.21.15.44:0/445500197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce0806f8b0 0x7fce0815dae0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.241+0000 7fce0e400640 1 -- 172.21.15.44:0/445500197 >> 172.21.15.44:0/445500197 conn(0x7fce080b18f0 msgr2=0x7fce080b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:37.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.241+0000 7fce0e400640 1 -- 172.21.15.44:0/445500197 shutdown_connections 2024-09-13T08:45:37.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 -- 172.21.15.44:0/445500197 wait complete. 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 Processor -- start 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 -- start start 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce0806f8b0 0x7fce08145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fce080cc620 0x7fce0815a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce080cd100 0x7fce0815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce081684c0 con 0x7fce0806f8b0 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fce08168340 con 0x7fce080cd100 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fce08168640 con 0x7fce080cc620 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce07400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce0806f8b0 0x7fce08145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce07400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce0806f8b0 0x7fce08145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37982/0 (socket says 172.21.15.44:37982) 2024-09-13T08:45:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce07400640 1 -- 172.21.15.44:0/2965514146 learned_addr learned my addr 172.21.15.44:0/2965514146 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce07e00640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce080cd100 0x7fce0815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fcdffe00640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fce080cc620 0x7fce0815a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce07400640 1 -- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fce080cc620 msgr2=0x7fce0815a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce07400640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fce080cc620 0x7fce0815a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce07400640 1 -- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce080cd100 msgr2=0x7fce0815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce07400640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce080cd100 0x7fce0815a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce07400640 1 -- 172.21.15.44:0/2965514146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce08148ad0 con 0x7fce0806f8b0 2024-09-13T08:45:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce07400640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce0806f8b0 0x7fce08145380 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80193b0 tx=0x7fcdf80a6660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fcdff400640 1 -- 172.21.15.44:0/2965514146 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8004300 con 0x7fce0806f8b0 2024-09-13T08:45:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fcdff400640 1 -- 172.21.15.44:0/2965514146 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdf80044a0 con 0x7fce0806f8b0 2024-09-13T08:45:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fcdff400640 1 -- 172.21.15.44:0/2965514146 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8005270 con 0x7fce0806f8b0 2024-09-13T08:45:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 -- 172.21.15.44:0/2965514146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce08147d10 con 0x7fce0806f8b0 2024-09-13T08:45:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.245+0000 7fce0e400640 1 -- 172.21.15.44:0/2965514146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce081482d0 con 0x7fce0806f8b0 2024-09-13T08:45:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.249+0000 7fce0e400640 1 -- 172.21.15.44:0/2965514146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce08069410 con 0x7fce0806f8b0 2024-09-13T08:45:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.249+0000 7fcdff400640 1 -- 172.21.15.44:0/2965514146 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fcdf80b7070 con 0x7fce0806f8b0 2024-09-13T08:45:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.249+0000 7fcdff400640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcde4073b50 0x7fcde4076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.249+0000 7fcdff400640 1 -- 172.21.15.44:0/2965514146 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7fcdf812cd70 con 0x7fce0806f8b0 2024-09-13T08:45:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.249+0000 7fcdffe00640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcde4073b50 0x7fcde4076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:37.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.253+0000 7fcdffe00640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcde4073b50 0x7fcde4076010 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00058d0 tx=0x7fcdf0005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:37.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.253+0000 7fcdff400640 1 -- 172.21.15.44:0/2965514146 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcdf80ac030 con 0x7fce0806f8b0 2024-09-13T08:45:37.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.429+0000 7fce0e400640 1 -- 172.21.15.44:0/2965514146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce0815dae0 con 0x7fce0806f8b0 2024-09-13T08:45:37.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.429+0000 7fcdff400640 1 -- 172.21.15.44:0/2965514146 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16767 (secure 0 0 0) 0x7fcdf80164b0 con 0x7fce0806f8b0 2024-09-13T08:45:37.432 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:37.432 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:45:34.646621+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:45:30.771920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T08:45:32.014594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 -- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcde4073b50 msgr2=0x7fcde4076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcde4073b50 0x7fcde4076010 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00058d0 tx=0x7fcdf0005860 comp rx=0 tx=0).stop 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 -- 172.21.15.44:0/2965514146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce0806f8b0 msgr2=0x7fce08145380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce0806f8b0 0x7fce08145380 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80193b0 tx=0x7fcdf80a6660 comp rx=0 tx=0).stop 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 -- 172.21.15.44:0/2965514146 shutdown_connections 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcde4073b50 0x7fcde4076010 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce080cd100 0x7fce0815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fce080cc620 0x7fce0815a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 --2- 172.21.15.44:0/2965514146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce0806f8b0 0x7fce08145380 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 -- 172.21.15.44:0/2965514146 >> 172.21.15.44:0/2965514146 conn(0x7fce080b18f0 msgr2=0x7fce08073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:37.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 -- 172.21.15.44:0/2965514146 shutdown_connections 2024-09-13T08:45:37.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.433+0000 7fce0e400640 1 -- 172.21.15.44:0/2965514146 wait complete. 2024-09-13T08:45:37.456 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:45:37.456 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:45:37.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 -- 172.21.15.44:0/657496330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ec0afca0 msgr2=0x7f12ec0b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 --2- 172.21.15.44:0/657496330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ec0afca0 0x7f12ec0b00d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0cfb60 tx=0x7f12e401cac0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 -- 172.21.15.44:0/657496330 shutdown_connections 2024-09-13T08:45:37.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 --2- 172.21.15.44:0/657496330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ec0afca0 0x7f12ec0b00d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 --2- 172.21.15.44:0/657496330 >> v2:172.21.15.129:3300/0 conn(0x7f12ec15f490 0x7f12ec0b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 -- 172.21.15.44:0/657496330 >> 172.21.15.44:0/657496330 conn(0x7f12ec0d1e30 msgr2=0x7f12ec0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:37.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 -- 172.21.15.44:0/657496330 shutdown_connections 2024-09-13T08:45:37.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 -- 172.21.15.44:0/657496330 wait complete. 2024-09-13T08:45:37.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 Processor -- start 2024-09-13T08:45:37.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 -- start start 2024-09-13T08:45:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ec0afca0 0x7f12ec15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f12ec0b2e80 0x7f12ec0ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12ec15f490 0x7f12ec0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec168e00 con 0x7f12ec0afca0 2024-09-13T08:45:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec168c80 con 0x7f12ec15f490 2024-09-13T08:45:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.601+0000 7f12f0e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f12ec168f80 con 0x7f12ec0b2e80 2024-09-13T08:45:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ec0afca0 0x7f12ec15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ec0afca0 0x7f12ec15aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38000/0 (socket says 172.21.15.44:38000) 2024-09-13T08:45:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e9e00640 1 -- 172.21.15.44:0/963860891 learned_addr learned my addr 172.21.15.44:0/963860891 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12ea800640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12ec15f490 0x7f12ec0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e9400640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f12ec0b2e80 0x7f12ec0ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e9e00640 1 -- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f12ec0b2e80 msgr2=0x7f12ec0ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e9e00640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f12ec0b2e80 0x7f12ec0ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e9e00640 1 -- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12ec15f490 msgr2=0x7f12ec0c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e9e00640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12ec15f490 0x7f12ec0c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e9e00640 1 -- 172.21.15.44:0/963860891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ec0c9730 con 0x7f12ec0afca0 2024-09-13T08:45:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12ea800640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12ec15f490 0x7f12ec0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e9e00640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ec0afca0 0x7f12ec15aef0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00d750 tx=0x7f12dc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:37.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e2000640 1 -- 172.21.15.44:0/963860891 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc017070 con 0x7f12ec0afca0 2024-09-13T08:45:37.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e2000640 1 -- 172.21.15.44:0/963860891 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12dc009d20 con 0x7f12ec0afca0 2024-09-13T08:45:37.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e2000640 1 -- 172.21.15.44:0/963860891 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc00de60 con 0x7f12ec0afca0 2024-09-13T08:45:37.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12f0e00640 1 -- 172.21.15.44:0/963860891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ec0c9a10 con 0x7f12ec0afca0 2024-09-13T08:45:37.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12f0e00640 1 -- 172.21.15.44:0/963860891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ec157580 con 0x7f12ec0afca0 2024-09-13T08:45:37.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12f0e00640 1 -- 172.21.15.44:0/963860891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ec0b00d0 con 0x7f12ec0afca0 2024-09-13T08:45:37.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.605+0000 7f12e2000640 1 -- 172.21.15.44:0/963860891 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f12dc004280 con 0x7f12ec0afca0 2024-09-13T08:45:37.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.609+0000 7f12e2000640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f12cc073b20 0x7f12cc075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:37.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.609+0000 7f12e2000640 1 -- 172.21.15.44:0/963860891 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f12dc092e80 con 0x7f12ec0afca0 2024-09-13T08:45:37.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.609+0000 7f12e9400640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f12cc073b20 0x7f12cc075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:37.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.609+0000 7f12e9400640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f12cc073b20 0x7f12cc075fe0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f12ec15a0d0 tx=0x7f12d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:37.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.613+0000 7f12e2000640 1 -- 172.21.15.44:0/963860891 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f12dc020020 con 0x7f12ec0afca0 2024-09-13T08:45:37.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.785+0000 7f12f0e00640 1 -- 172.21.15.44:0/963860891 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12ec09fc50 con 0x7f12cc073b20 2024-09-13T08:45:37.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.789+0000 7f12e2000640 1 -- 172.21.15.44:0/963860891 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251714 (secure 0 0 0) 0x7f12ec09fc50 con 0x7f12cc073b20 2024-09-13T08:45:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:37.796 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:45:37.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 -- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f12cc073b20 msgr2=0x7f12cc075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f12cc073b20 0x7f12cc075fe0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f12ec15a0d0 tx=0x7f12d8009040 comp rx=0 tx=0).stop 2024-09-13T08:45:37.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 -- 172.21.15.44:0/963860891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ec0afca0 msgr2=0x7f12ec15aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:37.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ec0afca0 0x7f12ec15aef0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00d750 tx=0x7f12dc00dc10 comp rx=0 tx=0).stop 2024-09-13T08:45:37.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 -- 172.21.15.44:0/963860891 shutdown_connections 2024-09-13T08:45:37.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f12cc073b20 0x7f12cc075fe0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12ec15f490 0x7f12ec0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f12ec0b2e80 0x7f12ec0ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 --2- 172.21.15.44:0/963860891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ec0afca0 0x7f12ec15aef0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:37.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 -- 172.21.15.44:0/963860891 >> 172.21.15.44:0/963860891 conn(0x7f12ec0d1e30 msgr2=0x7f12ec158860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:37.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 -- 172.21.15.44:0/963860891 shutdown_connections 2024-09-13T08:45:37.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:37.793+0000 7f12f0e00640 1 -- 172.21.15.44:0/963860891 wait complete. 2024-09-13T08:45:37.814 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-13T08:45:36.640901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":136,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402060,"kb_used_data":8148,"kb_used_omap":12,"kb_used_meta":393715,"kb_avail":754572660,"statfs":{"total":773094113280,"available":772682403840,"internally_reserved":0,"allocated":8343552,"data_stored":4204762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":403164721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.010718"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689535+0000","last_change":"2024-09-13T08:45:33.684143+0000","last_active":"2024-09-13T08:45:33.689535+0000","last_peered":"2024-09-13T08:45:33.689535+0000","last_clean":"2024-09-13T08:45:33.689535+0000","last_became_active":"2024-09-13T08:45:33.683889+0000","last_became_peered":"2024-09-13T08:45:33.683889+0000","last_unstale":"2024-09-13T08:45:33.689535+0000","last_undegraded":"2024-09-13T08:45:33.689535+0000","last_fullsized":"2024-09-13T08:45:33.689535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:07.001116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688670+0000","last_change":"2024-09-13T08:45:33.685799+0000","last_active":"2024-09-13T08:45:33.688670+0000","last_peered":"2024-09-13T08:45:33.688670+0000","last_clean":"2024-09-13T08:45:33.688670+0000","last_became_active":"2024-09-13T08:45:33.685475+0000","last_became_peered":"2024-09-13T08:45:33.685475+0000","last_unstale":"2024-09-13T08:45:33.688670+0000","last_undegraded":"2024-09-13T08:45:33.688670+0000","last_fullsized":"2024-09-13T08:45:33.688670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:35.321800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684692+0000","last_change":"2024-09-13T08:45:33.683135+0000","last_active":"2024-09-13T08:45:33.684692+0000","last_peered":"2024-09-13T08:45:33.684692+0000","last_clean":"2024-09-13T08:45:33.684692+0000","last_became_active":"2024-09-13T08:45:33.682789+0000","last_became_peered":"2024-09-13T08:45:33.682789+0000","last_unstale":"2024-09-13T08:45:33.684692+0000","last_undegraded":"2024-09-13T08:45:33.684692+0000","last_fullsized":"2024-09-13T08:45:33.684692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:47:16.394704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684656+0000","last_change":"2024-09-13T08:45:33.682881+0000","last_active":"2024-09-13T08:45:33.684656+0000","last_peered":"2024-09-13T08:45:33.684656+0000","last_clean":"2024-09-13T08:45:33.684656+0000","last_became_active":"2024-09-13T08:45:33.681260+0000","last_became_peered":"2024-09-13T08:45:33.681260+0000","last_unstale":"2024-09-13T08:45:33.684656+0000","last_undegraded":"2024-09-13T08:45:33.684656+0000","last_fullsized":"2024-09-13T08:45:33.684656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:34.085379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675098+0000","last_change":"2024-09-13T08:45:33.673961+0000","last_active":"2024-09-13T08:45:33.675098+0000","last_peered":"2024-09-13T08:45:33.675098+0000","last_clean":"2024-09-13T08:45:33.675098+0000","last_became_active":"2024-09-13T08:45:33.673661+0000","last_became_peered":"2024-09-13T08:45:33.673661+0000","last_unstale":"2024-09-13T08:45:33.675098+0000","last_undegraded":"2024-09-13T08:45:33.675098+0000","last_fullsized":"2024-09-13T08:45:33.675098+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:05.467156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689608+0000","last_change":"2024-09-13T08:45:33.686731+0000","last_active":"2024-09-13T08:45:33.689608+0000","last_peered":"2024-09-13T08:45:33.689608+0000","last_clean":"2024-09-13T08:45:33.689608+0000","last_became_active":"2024-09-13T08:45:33.686352+0000","last_became_peered":"2024-09-13T08:45:33.686352+0000","last_unstale":"2024-09-13T08:45:33.689608+0000","last_undegraded":"2024-09-13T08:45:33.689608+0000","last_fullsized":"2024-09-13T08:45:33.689608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:43:22.048135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.666172+0000","last_change":"2024-09-13T08:45:33.660192+0000","last_active":"2024-09-13T08:45:33.666172+0000","last_peered":"2024-09-13T08:45:33.666172+0000","last_clean":"2024-09-13T08:45:33.666172+0000","last_became_active":"2024-09-13T08:45:33.659425+0000","last_became_peered":"2024-09-13T08:45:33.659425+0000","last_unstale":"2024-09-13T08:45:33.666172+0000","last_undegraded":"2024-09-13T08:45:33.666172+0000","last_fullsized":"2024-09-13T08:45:33.666172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:27.961233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688123+0000","last_change":"2024-09-13T08:45:33.671839+0000","last_active":"2024-09-13T08:45:33.688123+0000","last_peered":"2024-09-13T08:45:33.688123+0000","last_clean":"2024-09-13T08:45:33.688123+0000","last_became_active":"2024-09-13T08:45:33.671538+0000","last_became_peered":"2024-09-13T08:45:33.671538+0000","last_unstale":"2024-09-13T08:45:33.688123+0000","last_undegraded":"2024-09-13T08:45:33.688123+0000","last_fullsized":"2024-09-13T08:45:33.688123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:51.161859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684700+0000","last_change":"2024-09-13T08:45:33.683295+0000","last_active":"2024-09-13T08:45:33.684700+0000","last_peered":"2024-09-13T08:45:33.684700+0000","last_clean":"2024-09-13T08:45:33.684700+0000","last_became_active":"2024-09-13T08:45:33.681988+0000","last_became_peered":"2024-09-13T08:45:33.681988+0000","last_unstale":"2024-09-13T08:45:33.684700+0000","last_undegraded":"2024-09-13T08:45:33.684700+0000","last_fullsized":"2024-09-13T08:45:33.684700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:34:29.504228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681352+0000","last_change":"2024-09-13T08:45:33.667868+0000","last_active":"2024-09-13T08:45:33.681352+0000","last_peered":"2024-09-13T08:45:33.681352+0000","last_clean":"2024-09-13T08:45:33.681352+0000","last_became_active":"2024-09-13T08:45:33.667543+0000","last_became_peered":"2024-09-13T08:45:33.667543+0000","last_unstale":"2024-09-13T08:45:33.681352+0000","last_undegraded":"2024-09-13T08:45:33.681352+0000","last_fullsized":"2024-09-13T08:45:33.681352+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:10.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691398+0000","last_change":"2024-09-13T08:45:33.689087+0000","last_active":"2024-09-13T08:45:33.691398+0000","last_peered":"2024-09-13T08:45:33.691398+0000","last_clean":"2024-09-13T08:45:33.691398+0000","last_became_active":"2024-09-13T08:45:33.688839+0000","last_became_peered":"2024-09-13T08:45:33.688839+0000","last_unstale":"2024-09-13T08:45:33.691398+0000","last_undegraded":"2024-09-13T08:45:33.691398+0000","last_fullsized":"2024-09-13T08:45:33.691398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:20:46.695925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.666207+0000","last_change":"2024-09-13T08:45:33.664987+0000","last_active":"2024-09-13T08:45:33.666207+0000","last_peered":"2024-09-13T08:45:33.666207+0000","last_clean":"2024-09-13T08:45:33.666207+0000","last_became_active":"2024-09-13T08:45:33.664707+0000","last_became_peered":"2024-09-13T08:45:33.664707+0000","last_unstale":"2024-09-13T08:45:33.666207+0000","last_undegraded":"2024-09-13T08:45:33.666207+0000","last_fullsized":"2024-09-13T08:45:33.666207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:03.021478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683740+0000","last_change":"2024-09-13T08:45:33.681174+0000","last_active":"2024-09-13T08:45:33.683740+0000","last_peered":"2024-09-13T08:45:33.683740+0000","last_clean":"2024-09-13T08:45:33.683740+0000","last_became_active":"2024-09-13T08:45:33.680680+0000","last_became_peered":"2024-09-13T08:45:33.680680+0000","last_unstale":"2024-09-13T08:45:33.683740+0000","last_undegraded":"2024-09-13T08:45:33.683740+0000","last_fullsized":"2024-09-13T08:45:33.683740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:46.928710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671724+0000","last_change":"2024-09-13T08:45:33.668789+0000","last_active":"2024-09-13T08:45:33.671724+0000","last_peered":"2024-09-13T08:45:33.671724+0000","last_clean":"2024-09-13T08:45:33.671724+0000","last_became_active":"2024-09-13T08:45:33.668462+0000","last_became_peered":"2024-09-13T08:45:33.668462+0000","last_unstale":"2024-09-13T08:45:33.671724+0000","last_undegraded":"2024-09-13T08:45:33.671724+0000","last_fullsized":"2024-09-13T08:45:33.671724+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:32:11.095202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.673865+0000","last_change":"2024-09-13T08:45:33.671105+0000","last_active":"2024-09-13T08:45:33.673865+0000","last_peered":"2024-09-13T08:45:33.673865+0000","last_clean":"2024-09-13T08:45:33.673865+0000","last_became_active":"2024-09-13T08:45:33.670718+0000","last_became_peered":"2024-09-13T08:45:33.670718+0000","last_unstale":"2024-09-13T08:45:33.673865+0000","last_undegraded":"2024-09-13T08:45:33.673865+0000","last_fullsized":"2024-09-13T08:45:33.673865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:34.021306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691685+0000","last_change":"2024-09-13T08:45:33.686654+0000","last_active":"2024-09-13T08:45:33.691685+0000","last_peered":"2024-09-13T08:45:33.691685+0000","last_clean":"2024-09-13T08:45:33.691685+0000","last_became_active":"2024-09-13T08:45:33.686375+0000","last_became_peered":"2024-09-13T08:45:33.686375+0000","last_unstale":"2024-09-13T08:45:33.691685+0000","last_undegraded":"2024-09-13T08:45:33.691685+0000","last_fullsized":"2024-09-13T08:45:33.691685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:03.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.672216+0000","last_change":"2024-09-13T08:45:33.670204+0000","last_active":"2024-09-13T08:45:33.672216+0000","last_peered":"2024-09-13T08:45:33.672216+0000","last_clean":"2024-09-13T08:45:33.672216+0000","last_became_active":"2024-09-13T08:45:33.669936+0000","last_became_peered":"2024-09-13T08:45:33.669936+0000","last_unstale":"2024-09-13T08:45:33.672216+0000","last_undegraded":"2024-09-13T08:45:33.672216+0000","last_fullsized":"2024-09-13T08:45:33.672216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:34.875650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667097+0000","last_change":"2024-09-13T08:45:33.663820+0000","last_active":"2024-09-13T08:45:33.667097+0000","last_peered":"2024-09-13T08:45:33.667097+0000","last_clean":"2024-09-13T08:45:33.667097+0000","last_became_active":"2024-09-13T08:45:33.663147+0000","last_became_peered":"2024-09-13T08:45:33.663147+0000","last_unstale":"2024-09-13T08:45:33.667097+0000","last_undegraded":"2024-09-13T08:45:33.667097+0000","last_fullsized":"2024-09-13T08:45:33.667097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:16:16.642146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.687335+0000","last_change":"2024-09-13T08:45:33.680218+0000","last_active":"2024-09-13T08:45:33.687335+0000","last_peered":"2024-09-13T08:45:33.687335+0000","last_clean":"2024-09-13T08:45:33.687335+0000","last_became_active":"2024-09-13T08:45:33.679938+0000","last_became_peered":"2024-09-13T08:45:33.679938+0000","last_unstale":"2024-09-13T08:45:33.687335+0000","last_undegraded":"2024-09-13T08:45:33.687335+0000","last_fullsized":"2024-09-13T08:45:33.687335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:05.467156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675520+0000","last_change":"2024-09-13T08:45:33.674349+0000","last_active":"2024-09-13T08:45:33.675520+0000","last_peered":"2024-09-13T08:45:33.675520+0000","last_clean":"2024-09-13T08:45:33.675520+0000","last_became_active":"2024-09-13T08:45:33.674004+0000","last_became_peered":"2024-09-13T08:45:33.674004+0000","last_unstale":"2024-09-13T08:45:33.675520+0000","last_undegraded":"2024-09-13T08:45:33.675520+0000","last_fullsized":"2024-09-13T08:45:33.675520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:06.280244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683524+0000","last_change":"2024-09-13T08:45:33.679260+0000","last_active":"2024-09-13T08:45:33.683524+0000","last_peered":"2024-09-13T08:45:33.683524+0000","last_clean":"2024-09-13T08:45:33.683524+0000","last_became_active":"2024-09-13T08:45:33.678976+0000","last_became_peered":"2024-09-13T08:45:33.678976+0000","last_unstale":"2024-09-13T08:45:33.683524+0000","last_undegraded":"2024-09-13T08:45:33.683524+0000","last_fullsized":"2024-09-13T08:45:33.683524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:41:27.288006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.673433+0000","last_change":"2024-09-13T08:45:33.666891+0000","last_active":"2024-09-13T08:45:33.673433+0000","last_peered":"2024-09-13T08:45:33.673433+0000","last_clean":"2024-09-13T08:45:33.673433+0000","last_became_active":"2024-09-13T08:45:33.666585+0000","last_became_peered":"2024-09-13T08:45:33.666585+0000","last_unstale":"2024-09-13T08:45:33.673433+0000","last_undegraded":"2024-09-13T08:45:33.673433+0000","last_fullsized":"2024-09-13T08:45:33.673433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:26:12.036228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.678998+0000","last_change":"2024-09-13T08:45:33.670178+0000","last_active":"2024-09-13T08:45:33.678998+0000","last_peered":"2024-09-13T08:45:33.678998+0000","last_clean":"2024-09-13T08:45:33.678998+0000","last_became_active":"2024-09-13T08:45:33.669687+0000","last_became_peered":"2024-09-13T08:45:33.669687+0000","last_unstale":"2024-09-13T08:45:33.678998+0000","last_undegraded":"2024-09-13T08:45:33.678998+0000","last_fullsized":"2024-09-13T08:45:33.678998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:11:12.735593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688146+0000","last_change":"2024-09-13T08:45:33.677052+0000","last_active":"2024-09-13T08:45:33.688146+0000","last_peered":"2024-09-13T08:45:33.688146+0000","last_clean":"2024-09-13T08:45:33.688146+0000","last_became_active":"2024-09-13T08:45:33.676767+0000","last_became_peered":"2024-09-13T08:45:33.676767+0000","last_unstale":"2024-09-13T08:45:33.688146+0000","last_undegraded":"2024-09-13T08:45:33.688146+0000","last_fullsized":"2024-09-13T08:45:33.688146+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:08.841526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.679899+0000","last_change":"2024-09-13T08:45:33.670240+0000","last_active":"2024-09-13T08:45:33.679899+0000","last_peered":"2024-09-13T08:45:33.679899+0000","last_clean":"2024-09-13T08:45:33.679899+0000","last_became_active":"2024-09-13T08:45:33.669952+0000","last_became_peered":"2024-09-13T08:45:33.669952+0000","last_unstale":"2024-09-13T08:45:33.679899+0000","last_undegraded":"2024-09-13T08:45:33.679899+0000","last_fullsized":"2024-09-13T08:45:33.679899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:15.070109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.692180+0000","last_change":"2024-09-13T08:45:33.690462+0000","last_active":"2024-09-13T08:45:33.692180+0000","last_peered":"2024-09-13T08:45:33.692180+0000","last_clean":"2024-09-13T08:45:33.692180+0000","last_became_active":"2024-09-13T08:45:33.690197+0000","last_became_peered":"2024-09-13T08:45:33.690197+0000","last_unstale":"2024-09-13T08:45:33.692180+0000","last_undegraded":"2024-09-13T08:45:33.692180+0000","last_fullsized":"2024-09-13T08:45:33.692180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:14.833887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.668085+0000","last_change":"2024-09-13T08:45:33.663806+0000","last_active":"2024-09-13T08:45:33.668085+0000","last_peered":"2024-09-13T08:45:33.668085+0000","last_clean":"2024-09-13T08:45:33.668085+0000","last_became_active":"2024-09-13T08:45:33.663498+0000","last_became_peered":"2024-09-13T08:45:33.663498+0000","last_unstale":"2024-09-13T08:45:33.668085+0000","last_undegraded":"2024-09-13T08:45:33.668085+0000","last_fullsized":"2024-09-13T08:45:33.668085+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:40:14.829023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683182+0000","last_change":"2024-09-13T08:45:33.674360+0000","last_active":"2024-09-13T08:45:33.683182+0000","last_peered":"2024-09-13T08:45:33.683182+0000","last_clean":"2024-09-13T08:45:33.683182+0000","last_became_active":"2024-09-13T08:45:33.673982+0000","last_became_peered":"2024-09-13T08:45:33.673982+0000","last_unstale":"2024-09-13T08:45:33.683182+0000","last_undegraded":"2024-09-13T08:45:33.683182+0000","last_fullsized":"2024-09-13T08:45:33.683182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:16:14.468554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689721+0000","last_change":"2024-09-13T08:45:33.689213+0000","last_active":"2024-09-13T08:45:33.689721+0000","last_peered":"2024-09-13T08:45:33.689721+0000","last_clean":"2024-09-13T08:45:33.689721+0000","last_became_active":"2024-09-13T08:45:33.688971+0000","last_became_peered":"2024-09-13T08:45:33.688971+0000","last_unstale":"2024-09-13T08:45:33.689721+0000","last_undegraded":"2024-09-13T08:45:33.689721+0000","last_fullsized":"2024-09-13T08:45:33.689721+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:48:29.114234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.687983+0000","last_change":"2024-09-13T08:45:33.672566+0000","last_active":"2024-09-13T08:45:33.687983+0000","last_peered":"2024-09-13T08:45:33.687983+0000","last_clean":"2024-09-13T08:45:33.687983+0000","last_became_active":"2024-09-13T08:45:33.672302+0000","last_became_peered":"2024-09-13T08:45:33.672302+0000","last_unstale":"2024-09-13T08:45:33.687983+0000","last_undegraded":"2024-09-13T08:45:33.687983+0000","last_fullsized":"2024-09-13T08:45:33.687983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:51:14.557452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684957+0000","last_change":"2024-09-13T08:45:33.675233+0000","last_active":"2024-09-13T08:45:33.684957+0000","last_peered":"2024-09-13T08:45:33.684957+0000","last_clean":"2024-09-13T08:45:33.684957+0000","last_became_active":"2024-09-13T08:45:33.674920+0000","last_became_peered":"2024-09-13T08:45:33.674920+0000","last_unstale":"2024-09-13T08:45:33.684957+0000","last_undegraded":"2024-09-13T08:45:33.684957+0000","last_fullsized":"2024-09-13T08:45:33.684957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:10.521583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683642+0000","last_change":"2024-09-13T08:45:33.679637+0000","last_active":"2024-09-13T08:45:33.683642+0000","last_peered":"2024-09-13T08:45:33.683642+0000","last_clean":"2024-09-13T08:45:33.683642+0000","last_became_active":"2024-09-13T08:45:33.679391+0000","last_became_peered":"2024-09-13T08:45:33.679391+0000","last_unstale":"2024-09-13T08:45:33.683642+0000","last_undegraded":"2024-09-13T08:45:33.683642+0000","last_fullsized":"2024-09-13T08:45:33.683642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:56.161329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691281+0000","last_change":"2024-09-13T08:45:33.688757+0000","last_active":"2024-09-13T08:45:33.691281+0000","last_peered":"2024-09-13T08:45:33.691281+0000","last_clean":"2024-09-13T08:45:33.691281+0000","last_became_active":"2024-09-13T08:45:33.687386+0000","last_became_peered":"2024-09-13T08:45:33.687386+0000","last_unstale":"2024-09-13T08:45:33.691281+0000","last_undegraded":"2024-09-13T08:45:33.691281+0000","last_fullsized":"2024-09-13T08:45:33.691281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:27.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"74'4","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689227+0000","last_change":"2024-09-13T08:45:32.657493+0000","last_active":"2024-09-13T08:45:33.689227+0000","last_peered":"2024-09-13T08:45:33.689227+0000","last_clean":"2024-09-13T08:45:33.689227+0000","last_became_active":"2024-09-13T08:45:32.656217+0000","last_became_peered":"2024-09-13T08:45:32.656217+0000","last_unstale":"2024-09-13T08:45:33.689227+0000","last_undegraded":"2024-09-13T08:45:33.689227+0000","last_fullsized":"2024-09-13T08:45:33.689227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:13.023500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683491+0000","last_change":"2024-09-13T08:45:33.678452+0000","last_active":"2024-09-13T08:45:33.683491+0000","last_peered":"2024-09-13T08:45:33.683491+0000","last_clean":"2024-09-13T08:45:33.683491+0000","last_became_active":"2024-09-13T08:45:33.678224+0000","last_became_peered":"2024-09-13T08:45:33.678224+0000","last_unstale":"2024-09-13T08:45:33.683491+0000","last_undegraded":"2024-09-13T08:45:33.683491+0000","last_fullsized":"2024-09-13T08:45:33.683491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:52:00.198742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684555+0000","last_change":"2024-09-13T08:45:32.654382+0000","last_active":"2024-09-13T08:45:33.684555+0000","last_peered":"2024-09-13T08:45:33.684555+0000","last_clean":"2024-09-13T08:45:33.684555+0000","last_became_active":"2024-09-13T08:45:32.654090+0000","last_became_peered":"2024-09-13T08:45:32.654090+0000","last_unstale":"2024-09-13T08:45:33.684555+0000","last_undegraded":"2024-09-13T08:45:33.684555+0000","last_fullsized":"2024-09-13T08:45:33.684555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:42.438703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.690499+0000","last_change":"2024-09-13T08:45:33.686795+0000","last_active":"2024-09-13T08:45:33.690499+0000","last_peered":"2024-09-13T08:45:33.690499+0000","last_clean":"2024-09-13T08:45:33.690499+0000","last_became_active":"2024-09-13T08:45:33.686497+0000","last_became_peered":"2024-09-13T08:45:33.686497+0000","last_unstale":"2024-09-13T08:45:33.690499+0000","last_undegraded":"2024-09-13T08:45:33.690499+0000","last_fullsized":"2024-09-13T08:45:33.690499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:27.050847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694008+0000","last_change":"2024-09-13T08:45:32.656780+0000","last_active":"2024-09-13T08:45:33.694008+0000","last_peered":"2024-09-13T08:45:33.694008+0000","last_clean":"2024-09-13T08:45:33.694008+0000","last_became_active":"2024-09-13T08:45:32.653941+0000","last_became_peered":"2024-09-13T08:45:32.653941+0000","last_unstale":"2024-09-13T08:45:33.694008+0000","last_undegraded":"2024-09-13T08:45:33.694008+0000","last_fullsized":"2024-09-13T08:45:33.694008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:12.824731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667910+0000","last_change":"2024-09-13T08:45:33.663007+0000","last_active":"2024-09-13T08:45:33.667910+0000","last_peered":"2024-09-13T08:45:33.667910+0000","last_clean":"2024-09-13T08:45:33.667910+0000","last_became_active":"2024-09-13T08:45:33.662591+0000","last_became_peered":"2024-09-13T08:45:33.662591+0000","last_unstale":"2024-09-13T08:45:33.667910+0000","last_undegraded":"2024-09-13T08:45:33.667910+0000","last_fullsized":"2024-09-13T08:45:33.667910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:59.290026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.690456+0000","last_change":"2024-09-13T08:45:32.655814+0000","last_active":"2024-09-13T08:45:33.690456+0000","last_peered":"2024-09-13T08:45:33.690456+0000","last_clean":"2024-09-13T08:45:33.690456+0000","last_became_active":"2024-09-13T08:45:32.655509+0000","last_became_peered":"2024-09-13T08:45:32.655509+0000","last_unstale":"2024-09-13T08:45:33.690456+0000","last_undegraded":"2024-09-13T08:45:33.690456+0000","last_fullsized":"2024-09-13T08:45:33.690456+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:56:39.873532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688008+0000","last_change":"2024-09-13T08:45:33.676002+0000","last_active":"2024-09-13T08:45:33.688008+0000","last_peered":"2024-09-13T08:45:33.688008+0000","last_clean":"2024-09-13T08:45:33.688008+0000","last_became_active":"2024-09-13T08:45:33.675711+0000","last_became_peered":"2024-09-13T08:45:33.675711+0000","last_unstale":"2024-09-13T08:45:33.688008+0000","last_undegraded":"2024-09-13T08:45:33.688008+0000","last_fullsized":"2024-09-13T08:45:33.688008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:34.875650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685399+0000","last_change":"2024-09-13T08:45:32.650357+0000","last_active":"2024-09-13T08:45:33.685399+0000","last_peered":"2024-09-13T08:45:33.685399+0000","last_clean":"2024-09-13T08:45:33.685399+0000","last_became_active":"2024-09-13T08:45:32.650092+0000","last_became_peered":"2024-09-13T08:45:32.650092+0000","last_unstale":"2024-09-13T08:45:33.685399+0000","last_undegraded":"2024-09-13T08:45:33.685399+0000","last_fullsized":"2024-09-13T08:45:33.685399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:11.131195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689533+0000","last_change":"2024-09-13T08:45:33.686668+0000","last_active":"2024-09-13T08:45:33.689533+0000","last_peered":"2024-09-13T08:45:33.689533+0000","last_clean":"2024-09-13T08:45:33.689533+0000","last_became_active":"2024-09-13T08:45:33.686340+0000","last_became_peered":"2024-09-13T08:45:33.686340+0000","last_unstale":"2024-09-13T08:45:33.689533+0000","last_undegraded":"2024-09-13T08:45:33.689533+0000","last_fullsized":"2024-09-13T08:45:33.689533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:45:11.714466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685672+0000","last_change":"2024-09-13T08:45:32.652970+0000","last_active":"2024-09-13T08:45:33.685672+0000","last_peered":"2024-09-13T08:45:33.685672+0000","last_clean":"2024-09-13T08:45:33.685672+0000","last_became_active":"2024-09-13T08:45:32.652708+0000","last_became_peered":"2024-09-13T08:45:32.652708+0000","last_unstale":"2024-09-13T08:45:33.685672+0000","last_undegraded":"2024-09-13T08:45:33.685672+0000","last_fullsized":"2024-09-13T08:45:33.685672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:26.448081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.644840+0000","last_change":"2024-09-13T08:45:32.644840+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.644840+0000","last_undegraded":"2024-09-13T08:45:32.644840+0000","last_fullsized":"2024-09-13T08:45:32.644840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T08:45:32.660378+0000","last_change":"2024-09-13T08:45:32.659746+0000","last_active":"2024-09-13T08:45:32.660378+0000","last_peered":"2024-09-13T08:45:32.660378+0000","last_clean":"2024-09-13T08:45:32.660378+0000","last_became_active":"2024-09-13T08:45:32.659314+0000","last_became_peered":"2024-09-13T08:45:32.659314+0000","last_unstale":"2024-09-13T08:45:32.660378+0000","last_undegraded":"2024-09-13T08:45:32.660378+0000","last_fullsized":"2024-09-13T08:45:32.660378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:08:43.439003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671291+0000","last_change":"2024-09-13T08:45:33.668090+0000","last_active":"2024-09-13T08:45:33.671291+0000","last_peered":"2024-09-13T08:45:33.671291+0000","last_clean":"2024-09-13T08:45:33.671291+0000","last_became_active":"2024-09-13T08:45:33.667802+0000","last_became_peered":"2024-09-13T08:45:33.667802+0000","last_unstale":"2024-09-13T08:45:33.671291+0000","last_undegraded":"2024-09-13T08:45:33.671291+0000","last_fullsized":"2024-09-13T08:45:33.671291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:02:31.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660673+0000","last_change":"2024-09-13T08:45:32.654860+0000","last_active":"2024-09-13T08:45:33.660673+0000","last_peered":"2024-09-13T08:45:33.660673+0000","last_clean":"2024-09-13T08:45:33.660673+0000","last_became_active":"2024-09-13T08:45:32.652510+0000","last_became_peered":"2024-09-13T08:45:32.652510+0000","last_unstale":"2024-09-13T08:45:33.660673+0000","last_undegraded":"2024-09-13T08:45:33.660673+0000","last_fullsized":"2024-09-13T08:45:33.660673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:16.016602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681459+0000","last_change":"2024-09-13T08:45:33.679316+0000","last_active":"2024-09-13T08:45:33.681459+0000","last_peered":"2024-09-13T08:45:33.681459+0000","last_clean":"2024-09-13T08:45:33.681459+0000","last_became_active":"2024-09-13T08:45:33.678605+0000","last_became_peered":"2024-09-13T08:45:33.678605+0000","last_unstale":"2024-09-13T08:45:33.681459+0000","last_undegraded":"2024-09-13T08:45:33.681459+0000","last_fullsized":"2024-09-13T08:45:33.681459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:01:38.403717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685240+0000","last_change":"2024-09-13T08:45:32.655976+0000","last_active":"2024-09-13T08:45:33.685240+0000","last_peered":"2024-09-13T08:45:33.685240+0000","last_clean":"2024-09-13T08:45:33.685240+0000","last_became_active":"2024-09-13T08:45:32.654678+0000","last_became_peered":"2024-09-13T08:45:32.654678+0000","last_unstale":"2024-09-13T08:45:33.685240+0000","last_undegraded":"2024-09-13T08:45:33.685240+0000","last_fullsized":"2024-09-13T08:45:33.685240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:07.445463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684924+0000","last_change":"2024-09-13T08:45:33.682397+0000","last_active":"2024-09-13T08:45:33.684924+0000","last_peered":"2024-09-13T08:45:33.684924+0000","last_clean":"2024-09-13T08:45:33.684924+0000","last_became_active":"2024-09-13T08:45:33.682010+0000","last_became_peered":"2024-09-13T08:45:33.682010+0000","last_unstale":"2024-09-13T08:45:33.684924+0000","last_undegraded":"2024-09-13T08:45:33.684924+0000","last_fullsized":"2024-09-13T08:45:33.684924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:39.373555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.670069+0000","last_change":"2024-09-13T08:45:32.654977+0000","last_active":"2024-09-13T08:45:33.670069+0000","last_peered":"2024-09-13T08:45:33.670069+0000","last_clean":"2024-09-13T08:45:33.670069+0000","last_became_active":"2024-09-13T08:45:32.654656+0000","last_became_peered":"2024-09-13T08:45:32.654656+0000","last_unstale":"2024-09-13T08:45:33.670069+0000","last_undegraded":"2024-09-13T08:45:33.670069+0000","last_fullsized":"2024-09-13T08:45:33.670069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:09.832815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689499+0000","last_change":"2024-09-13T08:45:33.682432+0000","last_active":"2024-09-13T08:45:33.689499+0000","last_peered":"2024-09-13T08:45:33.689499+0000","last_clean":"2024-09-13T08:45:33.689499+0000","last_became_active":"2024-09-13T08:45:33.681551+0000","last_became_peered":"2024-09-13T08:45:33.681551+0000","last_unstale":"2024-09-13T08:45:33.689499+0000","last_undegraded":"2024-09-13T08:45:33.689499+0000","last_fullsized":"2024-09-13T08:45:33.689499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:06.280244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"74'3","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682838+0000","last_change":"2024-09-13T08:45:32.644983+0000","last_active":"2024-09-13T08:45:33.682838+0000","last_peered":"2024-09-13T08:45:33.682838+0000","last_clean":"2024-09-13T08:45:33.682838+0000","last_became_active":"2024-09-13T08:45:32.644649+0000","last_became_peered":"2024-09-13T08:45:32.644649+0000","last_unstale":"2024-09-13T08:45:33.682838+0000","last_undegraded":"2024-09-13T08:45:33.682838+0000","last_fullsized":"2024-09-13T08:45:33.682838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:16:03.736952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.669128+0000","last_change":"2024-09-13T08:45:33.663848+0000","last_active":"2024-09-13T08:45:33.669128+0000","last_peered":"2024-09-13T08:45:33.669128+0000","last_clean":"2024-09-13T08:45:33.669128+0000","last_became_active":"2024-09-13T08:45:33.663559+0000","last_became_peered":"2024-09-13T08:45:33.663559+0000","last_unstale":"2024-09-13T08:45:33.669128+0000","last_undegraded":"2024-09-13T08:45:33.669128+0000","last_fullsized":"2024-09-13T08:45:33.669128+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:51.161859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T08:45:32.657443+0000","last_change":"2024-09-13T08:45:32.656020+0000","last_active":"2024-09-13T08:45:32.657443+0000","last_peered":"2024-09-13T08:45:32.657443+0000","last_clean":"2024-09-13T08:45:32.657443+0000","last_became_active":"2024-09-13T08:45:32.655215+0000","last_became_peered":"2024-09-13T08:45:32.655215+0000","last_unstale":"2024-09-13T08:45:32.657443+0000","last_undegraded":"2024-09-13T08:45:32.657443+0000","last_fullsized":"2024-09-13T08:45:32.657443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:44.310136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.677142+0000","last_change":"2024-09-13T08:45:33.671033+0000","last_active":"2024-09-13T08:45:33.677142+0000","last_peered":"2024-09-13T08:45:33.677142+0000","last_clean":"2024-09-13T08:45:33.677142+0000","last_became_active":"2024-09-13T08:45:33.670779+0000","last_became_peered":"2024-09-13T08:45:33.670779+0000","last_unstale":"2024-09-13T08:45:33.677142+0000","last_undegraded":"2024-09-13T08:45:33.677142+0000","last_fullsized":"2024-09-13T08:45:33.677142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:47:52.333905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T08:45:32.657464+0000","last_change":"2024-09-13T08:45:32.656334+0000","last_active":"2024-09-13T08:45:32.657464+0000","last_peered":"2024-09-13T08:45:32.657464+0000","last_clean":"2024-09-13T08:45:32.657464+0000","last_became_active":"2024-09-13T08:45:32.656007+0000","last_became_peered":"2024-09-13T08:45:32.656007+0000","last_unstale":"2024-09-13T08:45:32.657464+0000","last_undegraded":"2024-09-13T08:45:32.657464+0000","last_fullsized":"2024-09-13T08:45:32.657464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:45.983114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682359+0000","last_change":"2024-09-13T08:45:33.681389+0000","last_active":"2024-09-13T08:45:33.682359+0000","last_peered":"2024-09-13T08:45:33.682359+0000","last_clean":"2024-09-13T08:45:33.682359+0000","last_became_active":"2024-09-13T08:45:33.681085+0000","last_became_peered":"2024-09-13T08:45:33.681085+0000","last_unstale":"2024-09-13T08:45:33.682359+0000","last_undegraded":"2024-09-13T08:45:33.682359+0000","last_fullsized":"2024-09-13T08:45:33.682359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:38.549474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.652883+0000","last_change":"2024-09-13T08:45:32.641547+0000","last_active":"2024-09-13T08:45:33.652883+0000","last_peered":"2024-09-13T08:45:33.652883+0000","last_clean":"2024-09-13T08:45:33.652883+0000","last_became_active":"2024-09-13T08:45:32.641221+0000","last_became_peered":"2024-09-13T08:45:32.641221+0000","last_unstale":"2024-09-13T08:45:33.652883+0000","last_undegraded":"2024-09-13T08:45:33.652883+0000","last_fullsized":"2024-09-13T08:45:33.652883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:37.612881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671771+0000","last_change":"2024-09-13T08:45:33.668342+0000","last_active":"2024-09-13T08:45:33.671771+0000","last_peered":"2024-09-13T08:45:33.671771+0000","last_clean":"2024-09-13T08:45:33.671771+0000","last_became_active":"2024-09-13T08:45:33.668068+0000","last_became_peered":"2024-09-13T08:45:33.668068+0000","last_unstale":"2024-09-13T08:45:33.671771+0000","last_undegraded":"2024-09-13T08:45:33.671771+0000","last_fullsized":"2024-09-13T08:45:33.671771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:51:14.557452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.653649+0000","last_change":"2024-09-13T08:45:32.645856+0000","last_active":"2024-09-13T08:45:33.653649+0000","last_peered":"2024-09-13T08:45:33.653649+0000","last_clean":"2024-09-13T08:45:33.653649+0000","last_became_active":"2024-09-13T08:45:32.645551+0000","last_became_peered":"2024-09-13T08:45:32.645551+0000","last_unstale":"2024-09-13T08:45:33.653649+0000","last_undegraded":"2024-09-13T08:45:33.653649+0000","last_fullsized":"2024-09-13T08:45:33.653649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:13.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682542+0000","last_change":"2024-09-13T08:45:33.679240+0000","last_active":"2024-09-13T08:45:33.682542+0000","last_peered":"2024-09-13T08:45:33.682542+0000","last_clean":"2024-09-13T08:45:33.682542+0000","last_became_active":"2024-09-13T08:45:33.678945+0000","last_became_peered":"2024-09-13T08:45:33.678945+0000","last_unstale":"2024-09-13T08:45:33.682542+0000","last_undegraded":"2024-09-13T08:45:33.682542+0000","last_fullsized":"2024-09-13T08:45:33.682542+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:18.936719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"74'8","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691071+0000","last_change":"2024-09-13T08:45:32.644799+0000","last_active":"2024-09-13T08:45:33.691071+0000","last_peered":"2024-09-13T08:45:33.691071+0000","last_clean":"2024-09-13T08:45:33.691071+0000","last_became_active":"2024-09-13T08:45:32.644559+0000","last_became_peered":"2024-09-13T08:45:32.644559+0000","last_unstale":"2024-09-13T08:45:33.691071+0000","last_undegraded":"2024-09-13T08:45:33.691071+0000","last_fullsized":"2024-09-13T08:45:33.691071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:23.392065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680801+0000","last_change":"2024-09-13T08:45:33.676706+0000","last_active":"2024-09-13T08:45:33.680801+0000","last_peered":"2024-09-13T08:45:33.680801+0000","last_clean":"2024-09-13T08:45:33.680801+0000","last_became_active":"2024-09-13T08:45:33.676442+0000","last_became_peered":"2024-09-13T08:45:33.676442+0000","last_unstale":"2024-09-13T08:45:33.680801+0000","last_undegraded":"2024-09-13T08:45:33.680801+0000","last_fullsized":"2024-09-13T08:45:33.680801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:00.210640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694996+0000","last_change":"2024-09-13T08:45:32.655505+0000","last_active":"2024-09-13T08:45:33.694996+0000","last_peered":"2024-09-13T08:45:33.694996+0000","last_clean":"2024-09-13T08:45:33.694996+0000","last_became_active":"2024-09-13T08:45:32.655218+0000","last_became_peered":"2024-09-13T08:45:32.655218+0000","last_unstale":"2024-09-13T08:45:33.694996+0000","last_undegraded":"2024-09-13T08:45:33.694996+0000","last_fullsized":"2024-09-13T08:45:33.694996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:04.883497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680842+0000","last_change":"2024-09-13T08:45:33.675626+0000","last_active":"2024-09-13T08:45:33.680842+0000","last_peered":"2024-09-13T08:45:33.680842+0000","last_clean":"2024-09-13T08:45:33.680842+0000","last_became_active":"2024-09-13T08:45:33.675346+0000","last_became_peered":"2024-09-13T08:45:33.675346+0000","last_unstale":"2024-09-13T08:45:33.680842+0000","last_undegraded":"2024-09-13T08:45:33.680842+0000","last_fullsized":"2024-09-13T08:45:33.680842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:22.946990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685223+0000","last_change":"2024-09-13T08:45:32.658779+0000","last_active":"2024-09-13T08:45:33.685223+0000","last_peered":"2024-09-13T08:45:33.685223+0000","last_clean":"2024-09-13T08:45:33.685223+0000","last_became_active":"2024-09-13T08:45:32.658420+0000","last_became_peered":"2024-09-13T08:45:32.658420+0000","last_unstale":"2024-09-13T08:45:33.685223+0000","last_undegraded":"2024-09-13T08:45:33.685223+0000","last_fullsized":"2024-09-13T08:45:33.685223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:18.130462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680052+0000","last_change":"2024-09-13T08:45:33.675553+0000","last_active":"2024-09-13T08:45:33.680052+0000","last_peered":"2024-09-13T08:45:33.680052+0000","last_clean":"2024-09-13T08:45:33.680052+0000","last_became_active":"2024-09-13T08:45:33.675233+0000","last_became_peered":"2024-09-13T08:45:33.675233+0000","last_unstale":"2024-09-13T08:45:33.680052+0000","last_undegraded":"2024-09-13T08:45:33.680052+0000","last_fullsized":"2024-09-13T08:45:33.680052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:41.676077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.692093+0000","last_change":"2024-09-13T08:45:32.659506+0000","last_active":"2024-09-13T08:45:33.692093+0000","last_peered":"2024-09-13T08:45:33.692093+0000","last_clean":"2024-09-13T08:45:33.692093+0000","last_became_active":"2024-09-13T08:45:32.659220+0000","last_became_peered":"2024-09-13T08:45:32.659220+0000","last_unstale":"2024-09-13T08:45:33.692093+0000","last_undegraded":"2024-09-13T08:45:33.692093+0000","last_fullsized":"2024-09-13T08:45:33.692093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:30.318592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689473+0000","last_change":"2024-09-13T08:45:33.688805+0000","last_active":"2024-09-13T08:45:33.689473+0000","last_peered":"2024-09-13T08:45:33.689473+0000","last_clean":"2024-09-13T08:45:33.689473+0000","last_became_active":"2024-09-13T08:45:33.688534+0000","last_became_peered":"2024-09-13T08:45:33.688534+0000","last_unstale":"2024-09-13T08:45:33.689473+0000","last_undegraded":"2024-09-13T08:45:33.689473+0000","last_fullsized":"2024-09-13T08:45:33.689473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:11.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660382+0000","last_change":"2024-09-13T08:45:32.655084+0000","last_active":"2024-09-13T08:45:33.660382+0000","last_peered":"2024-09-13T08:45:33.660382+0000","last_clean":"2024-09-13T08:45:33.660382+0000","last_became_active":"2024-09-13T08:45:32.654795+0000","last_became_peered":"2024-09-13T08:45:32.654795+0000","last_unstale":"2024-09-13T08:45:33.660382+0000","last_undegraded":"2024-09-13T08:45:33.660382+0000","last_fullsized":"2024-09-13T08:45:33.660382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:23:00.745264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.641979+0000","last_change":"2024-09-13T08:45:32.641979+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.641979+0000","last_undegraded":"2024-09-13T08:45:32.641979+0000","last_fullsized":"2024-09-13T08:45:32.641979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.661857+0000","last_change":"2024-09-13T08:45:32.648855+0000","last_active":"2024-09-13T08:45:33.661857+0000","last_peered":"2024-09-13T08:45:33.661857+0000","last_clean":"2024-09-13T08:45:33.661857+0000","last_became_active":"2024-09-13T08:45:32.648572+0000","last_became_peered":"2024-09-13T08:45:32.648572+0000","last_unstale":"2024-09-13T08:45:33.661857+0000","last_undegraded":"2024-09-13T08:45:33.661857+0000","last_fullsized":"2024-09-13T08:45:33.661857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:11.383879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.644401+0000","last_change":"2024-09-13T08:45:32.644401+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.644401+0000","last_undegraded":"2024-09-13T08:45:32.644401+0000","last_fullsized":"2024-09-13T08:45:32.644401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.664419+0000","last_change":"2024-09-13T08:45:32.651171+0000","last_active":"2024-09-13T08:45:33.664419+0000","last_peered":"2024-09-13T08:45:33.664419+0000","last_clean":"2024-09-13T08:45:33.664419+0000","last_became_active":"2024-09-13T08:45:32.650437+0000","last_became_peered":"2024-09-13T08:45:32.650437+0000","last_unstale":"2024-09-13T08:45:33.664419+0000","last_undegraded":"2024-09-13T08:45:33.664419+0000","last_fullsized":"2024-09-13T08:45:33.664419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:43.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671316+0000","last_change":"2024-09-13T08:45:33.670268+0000","last_active":"2024-09-13T08:45:33.671316+0000","last_peered":"2024-09-13T08:45:33.671316+0000","last_clean":"2024-09-13T08:45:33.671316+0000","last_became_active":"2024-09-13T08:45:33.669974+0000","last_became_peered":"2024-09-13T08:45:33.669974+0000","last_unstale":"2024-09-13T08:45:33.671316+0000","last_undegraded":"2024-09-13T08:45:33.671316+0000","last_fullsized":"2024-09-13T08:45:33.671316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:14.031957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.657855+0000","last_change":"2024-09-13T08:45:32.652196+0000","last_active":"2024-09-13T08:45:33.657855+0000","last_peered":"2024-09-13T08:45:33.657855+0000","last_clean":"2024-09-13T08:45:33.657855+0000","last_became_active":"2024-09-13T08:45:32.651851+0000","last_became_peered":"2024-09-13T08:45:32.651851+0000","last_unstale":"2024-09-13T08:45:33.657855+0000","last_undegraded":"2024-09-13T08:45:33.657855+0000","last_fullsized":"2024-09-13T08:45:33.657855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:48:29.415637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.668002+0000","last_change":"2024-09-13T08:45:33.664149+0000","last_active":"2024-09-13T08:45:33.668002+0000","last_peered":"2024-09-13T08:45:33.668002+0000","last_clean":"2024-09-13T08:45:33.668002+0000","last_became_active":"2024-09-13T08:45:33.663531+0000","last_became_peered":"2024-09-13T08:45:33.663531+0000","last_unstale":"2024-09-13T08:45:33.668002+0000","last_undegraded":"2024-09-13T08:45:33.668002+0000","last_fullsized":"2024-09-13T08:45:33.668002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:18:36.348488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694733+0000","last_change":"2024-09-13T08:45:32.660012+0000","last_active":"2024-09-13T08:45:33.694733+0000","last_peered":"2024-09-13T08:45:33.694733+0000","last_clean":"2024-09-13T08:45:33.694733+0000","last_became_active":"2024-09-13T08:45:32.659398+0000","last_became_peered":"2024-09-13T08:45:32.659398+0000","last_unstale":"2024-09-13T08:45:33.694733+0000","last_undegraded":"2024-09-13T08:45:33.694733+0000","last_fullsized":"2024-09-13T08:45:33.694733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:12.195364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667819+0000","last_change":"2024-09-13T08:45:33.660680+0000","last_active":"2024-09-13T08:45:33.667819+0000","last_peered":"2024-09-13T08:45:33.667819+0000","last_clean":"2024-09-13T08:45:33.667819+0000","last_became_active":"2024-09-13T08:45:33.660407+0000","last_became_peered":"2024-09-13T08:45:33.660407+0000","last_unstale":"2024-09-13T08:45:33.667819+0000","last_undegraded":"2024-09-13T08:45:33.667819+0000","last_fullsized":"2024-09-13T08:45:33.667819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:21:26.352682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684746+0000","last_change":"2024-09-13T08:45:32.661277+0000","last_active":"2024-09-13T08:45:33.684746+0000","last_peered":"2024-09-13T08:45:33.684746+0000","last_clean":"2024-09-13T08:45:33.684746+0000","last_became_active":"2024-09-13T08:45:32.660211+0000","last_became_peered":"2024-09-13T08:45:32.660211+0000","last_unstale":"2024-09-13T08:45:33.684746+0000","last_undegraded":"2024-09-13T08:45:33.684746+0000","last_fullsized":"2024-09-13T08:45:33.684746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:01.561248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684943+0000","last_change":"2024-09-13T08:45:33.683345+0000","last_active":"2024-09-13T08:45:33.684943+0000","last_peered":"2024-09-13T08:45:33.684943+0000","last_clean":"2024-09-13T08:45:33.684943+0000","last_became_active":"2024-09-13T08:45:33.682992+0000","last_became_peered":"2024-09-13T08:45:33.682992+0000","last_unstale":"2024-09-13T08:45:33.684943+0000","last_undegraded":"2024-09-13T08:45:33.684943+0000","last_fullsized":"2024-09-13T08:45:33.684943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:47.803484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660890+0000","last_change":"2024-09-13T08:45:32.646650+0000","last_active":"2024-09-13T08:45:33.660890+0000","last_peered":"2024-09-13T08:45:33.660890+0000","last_clean":"2024-09-13T08:45:33.660890+0000","last_became_active":"2024-09-13T08:45:32.646362+0000","last_became_peered":"2024-09-13T08:45:32.646362+0000","last_unstale":"2024-09-13T08:45:33.660890+0000","last_undegraded":"2024-09-13T08:45:33.660890+0000","last_fullsized":"2024-09-13T08:45:33.660890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:48:47.580467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680628+0000","last_change":"2024-09-13T08:45:33.678066+0000","last_active":"2024-09-13T08:45:33.680628+0000","last_peered":"2024-09-13T08:45:33.680628+0000","last_clean":"2024-09-13T08:45:33.680628+0000","last_became_active":"2024-09-13T08:45:33.674778+0000","last_became_peered":"2024-09-13T08:45:33.674778+0000","last_unstale":"2024-09-13T08:45:33.680628+0000","last_undegraded":"2024-09-13T08:45:33.680628+0000","last_fullsized":"2024-09-13T08:45:33.680628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:56.987435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.661122+0000","last_change":"2024-09-13T08:45:32.646318+0000","last_active":"2024-09-13T08:45:33.661122+0000","last_peered":"2024-09-13T08:45:33.661122+0000","last_clean":"2024-09-13T08:45:33.661122+0000","last_became_active":"2024-09-13T08:45:32.645801+0000","last_became_peered":"2024-09-13T08:45:32.645801+0000","last_unstale":"2024-09-13T08:45:33.661122+0000","last_undegraded":"2024-09-13T08:45:33.661122+0000","last_fullsized":"2024-09-13T08:45:33.661122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:40.323104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.672925+0000","last_change":"2024-09-13T08:45:33.670611+0000","last_active":"2024-09-13T08:45:33.672925+0000","last_peered":"2024-09-13T08:45:33.672925+0000","last_clean":"2024-09-13T08:45:33.672925+0000","last_became_active":"2024-09-13T08:45:33.670334+0000","last_became_peered":"2024-09-13T08:45:33.670334+0000","last_unstale":"2024-09-13T08:45:33.672925+0000","last_undegraded":"2024-09-13T08:45:33.672925+0000","last_fullsized":"2024-09-13T08:45:33.672925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:08.841526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.662228+0000","last_change":"2024-09-13T08:45:32.650061+0000","last_active":"2024-09-13T08:45:33.662228+0000","last_peered":"2024-09-13T08:45:33.662228+0000","last_clean":"2024-09-13T08:45:33.662228+0000","last_became_active":"2024-09-13T08:45:32.649743+0000","last_became_peered":"2024-09-13T08:45:32.649743+0000","last_unstale":"2024-09-13T08:45:33.662228+0000","last_undegraded":"2024-09-13T08:45:33.662228+0000","last_fullsized":"2024-09-13T08:45:33.662228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:16.817675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691205+0000","last_change":"2024-09-13T08:45:33.682926+0000","last_active":"2024-09-13T08:45:33.691205+0000","last_peered":"2024-09-13T08:45:33.691205+0000","last_clean":"2024-09-13T08:45:33.691205+0000","last_became_active":"2024-09-13T08:45:33.681036+0000","last_became_peered":"2024-09-13T08:45:33.681036+0000","last_unstale":"2024-09-13T08:45:33.691205+0000","last_undegraded":"2024-09-13T08:45:33.691205+0000","last_fullsized":"2024-09-13T08:45:33.691205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:31.247460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.655978+0000","last_change":"2024-09-13T08:45:32.651908+0000","last_active":"2024-09-13T08:45:33.655978+0000","last_peered":"2024-09-13T08:45:33.655978+0000","last_clean":"2024-09-13T08:45:33.655978+0000","last_became_active":"2024-09-13T08:45:32.651622+0000","last_became_peered":"2024-09-13T08:45:32.651622+0000","last_unstale":"2024-09-13T08:45:33.655978+0000","last_undegraded":"2024-09-13T08:45:33.655978+0000","last_fullsized":"2024-09-13T08:45:33.655978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:32.504688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.642635+0000","last_change":"2024-09-13T08:45:32.642635+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.642635+0000","last_undegraded":"2024-09-13T08:45:32.642635+0000","last_fullsized":"2024-09-13T08:45:32.642635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.664624+0000","last_change":"2024-09-13T08:45:32.656890+0000","last_active":"2024-09-13T08:45:33.664624+0000","last_peered":"2024-09-13T08:45:33.664624+0000","last_clean":"2024-09-13T08:45:33.664624+0000","last_became_active":"2024-09-13T08:45:32.656592+0000","last_became_peered":"2024-09-13T08:45:32.656592+0000","last_unstale":"2024-09-13T08:45:33.664624+0000","last_undegraded":"2024-09-13T08:45:33.664624+0000","last_fullsized":"2024-09-13T08:45:33.664624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:38.811210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675107+0000","last_change":"2024-09-13T08:45:33.669976+0000","last_active":"2024-09-13T08:45:33.675107+0000","last_peered":"2024-09-13T08:45:33.675107+0000","last_clean":"2024-09-13T08:45:33.675107+0000","last_became_active":"2024-09-13T08:45:33.669693+0000","last_became_peered":"2024-09-13T08:45:33.669693+0000","last_unstale":"2024-09-13T08:45:33.675107+0000","last_undegraded":"2024-09-13T08:45:33.675107+0000","last_fullsized":"2024-09-13T08:45:33.675107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:27:36.696332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.658907+0000","last_change":"2024-09-13T08:45:32.654218+0000","last_active":"2024-09-13T08:45:33.658907+0000","last_peered":"2024-09-13T08:45:33.658907+0000","last_clean":"2024-09-13T08:45:33.658907+0000","last_became_active":"2024-09-13T08:45:32.653519+0000","last_became_peered":"2024-09-13T08:45:32.653519+0000","last_unstale":"2024-09-13T08:45:33.658907+0000","last_undegraded":"2024-09-13T08:45:33.658907+0000","last_fullsized":"2024-09-13T08:45:33.658907+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:33.012849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681797+0000","last_change":"2024-09-13T08:45:33.679432+0000","last_active":"2024-09-13T08:45:33.681797+0000","last_peered":"2024-09-13T08:45:33.681797+0000","last_clean":"2024-09-13T08:45:33.681797+0000","last_became_active":"2024-09-13T08:45:33.679129+0000","last_became_peered":"2024-09-13T08:45:33.679129+0000","last_unstale":"2024-09-13T08:45:33.681797+0000","last_undegraded":"2024-09-13T08:45:33.681797+0000","last_fullsized":"2024-09-13T08:45:33.681797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:31.247460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681998+0000","last_change":"2024-09-13T08:45:32.647002+0000","last_active":"2024-09-13T08:45:33.681998+0000","last_peered":"2024-09-13T08:45:33.681998+0000","last_clean":"2024-09-13T08:45:33.681998+0000","last_became_active":"2024-09-13T08:45:32.645456+0000","last_became_peered":"2024-09-13T08:45:32.645456+0000","last_unstale":"2024-09-13T08:45:33.681998+0000","last_undegraded":"2024-09-13T08:45:33.681998+0000","last_fullsized":"2024-09-13T08:45:33.681998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:41.793310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":212,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.652555+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:45:33.652555+0000","last_peered":"2024-09-13T08:45:33.652555+0000","last_clean":"2024-09-13T08:45:33.652555+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:45:33.652555+0000","last_undegraded":"2024-09-13T08:45:33.652555+0000","last_fullsized":"2024-09-13T08:45:33.652555+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509540,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50120,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321720,"statfs":{"total":96636764160,"available":96585441280,"internally_reserved":0,"allocated":905216,"data_stored":380942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50696,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321144,"statfs":{"total":96636764160,"available":96584851456,"internally_reserved":0,"allocated":1490944,"data_stored":969788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509541,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54288,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317552,"statfs":{"total":96636764160,"available":96581173248,"internally_reserved":0,"allocated":905216,"data_stored":380024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":880640,"data_stored":372499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":897024,"data_stored":379912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509542,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46532,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325308,"statfs":{"total":96636764160,"available":96589115392,"internally_reserved":0,"allocated":1486848,"data_stored":969698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":897024,"data_stored":379400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542246,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":880640,"data_stored":372499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:45:37.819 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:45:38.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.005+0000 7fb71d600640 1 -- 172.21.15.44:0/1214905298 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb7180b2590 msgr2=0x7fb7180b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.005+0000 7fb71d600640 1 --2- 172.21.15.44:0/1214905298 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb7180b2590 0x7fb7180b29e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb710009c80 tx=0x7fb71001cd10 comp rx=0 tx=0).stop 2024-09-13T08:45:38.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.005+0000 7fb71d600640 1 -- 172.21.15.44:0/1214905298 shutdown_connections 2024-09-13T08:45:38.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.005+0000 7fb71d600640 1 --2- 172.21.15.44:0/1214905298 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb7180b2590 0x7fb7180b29e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.005+0000 7fb71d600640 1 --2- 172.21.15.44:0/1214905298 >> v2:172.21.15.129:3300/0 conn(0x7fb71815f000 0x7fb7180d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.005+0000 7fb71d600640 1 -- 172.21.15.44:0/1214905298 >> 172.21.15.44:0/1214905298 conn(0x7fb71809aa10 msgr2=0x7fb71809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:38.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.005+0000 7fb71d600640 1 -- 172.21.15.44:0/1214905298 shutdown_connections 2024-09-13T08:45:38.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.005+0000 7fb71d600640 1 -- 172.21.15.44:0/1214905298 wait complete. 2024-09-13T08:45:38.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.005+0000 7fb71d600640 1 Processor -- start 2024-09-13T08:45:38.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71d600640 1 -- start start 2024-09-13T08:45:38.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7180b2590 0x7fb7181509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71d600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7180cefb0 0x7fb718150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71d600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb71815f000 0x7fb7180aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb718168f10 con 0x7fb7180b2590 2024-09-13T08:45:38.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71d600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb718168d90 con 0x7fb7180cefb0 2024-09-13T08:45:38.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71d600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb718169090 con 0x7fb71815f000 2024-09-13T08:45:38.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb717e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7180b2590 0x7fb7181509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb717e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7180b2590 0x7fb7181509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38010/0 (socket says 172.21.15.44:38010) 2024-09-13T08:45:38.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb717e00640 1 -- 172.21.15.44:0/3860451954 learned_addr learned my addr 172.21.15.44:0/3860451954 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:38.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb70fe00640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7180cefb0 0x7fb718150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71cc00640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb71815f000 0x7fb7180aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb717e00640 1 -- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb71815f000 msgr2=0x7fb7180aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb717e00640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb71815f000 0x7fb7180aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb717e00640 1 -- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7180cefb0 msgr2=0x7fb718150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb717e00640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7180cefb0 0x7fb718150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb717e00640 1 -- 172.21.15.44:0/3860451954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7181518e0 con 0x7fb7180b2590 2024-09-13T08:45:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb717e00640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7180b2590 0x7fb7181509d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb70800e7a0 tx=0x7fb70800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb714c00640 1 -- 172.21.15.44:0/3860451954 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70800ccc0 con 0x7fb7180b2590 2024-09-13T08:45:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb714c00640 1 -- 172.21.15.44:0/3860451954 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb708010430 con 0x7fb7180b2590 2024-09-13T08:45:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb714c00640 1 -- 172.21.15.44:0/3860451954 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708010860 con 0x7fb7180b2590 2024-09-13T08:45:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71d600640 1 -- 172.21.15.44:0/3860451954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb718151bc0 con 0x7fb7180b2590 2024-09-13T08:45:38.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71d600640 1 -- 172.21.15.44:0/3860451954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb718152180 con 0x7fb7180b2590 2024-09-13T08:45:38.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.009+0000 7fb71d600640 1 -- 172.21.15.44:0/3860451954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7180b29e0 con 0x7fb7180b2590 2024-09-13T08:45:38.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.013+0000 7fb714c00640 1 -- 172.21.15.44:0/3860451954 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fb708010a00 con 0x7fb7180b2590 2024-09-13T08:45:38.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.013+0000 7fb714c00640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6f4073b20 0x7fb6f4075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.013+0000 7fb714c00640 1 -- 172.21.15.44:0/3860451954 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7fb7080930f0 con 0x7fb7180b2590 2024-09-13T08:45:38.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.013+0000 7fb70fe00640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6f4073b20 0x7fb6f4075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.013+0000 7fb70fe00640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6f4073b20 0x7fb6f4075fe0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb7181527d0 tx=0x7fb700009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:38.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.017+0000 7fb714c00640 1 -- 172.21.15.44:0/3860451954 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb70805f780 con 0x7fb7180b2590 2024-09-13T08:45:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.189+0000 7fb71d600640 1 -- 172.21.15.44:0/3860451954 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb7180759e0 con 0x7fb6f4073b20 2024-09-13T08:45:38.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.197+0000 7fb714c00640 1 -- 172.21.15.44:0/3860451954 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251714 (secure 0 0 0) 0x7fb7180759e0 con 0x7fb6f4073b20 2024-09-13T08:45:38.200 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:38.201 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:45:38.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 -- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6f4073b20 msgr2=0x7fb6f4075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6f4073b20 0x7fb6f4075fe0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb7181527d0 tx=0x7fb700009040 comp rx=0 tx=0).stop 2024-09-13T08:45:38.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 -- 172.21.15.44:0/3860451954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7180b2590 msgr2=0x7fb7181509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7180b2590 0x7fb7181509d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb70800e7a0 tx=0x7fb70800ec60 comp rx=0 tx=0).stop 2024-09-13T08:45:38.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 -- 172.21.15.44:0/3860451954 shutdown_connections 2024-09-13T08:45:38.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6f4073b20 0x7fb6f4075fe0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb71815f000 0x7fb7180aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7180cefb0 0x7fb718150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 --2- 172.21.15.44:0/3860451954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7180b2590 0x7fb7181509d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 -- 172.21.15.44:0/3860451954 >> 172.21.15.44:0/3860451954 conn(0x7fb71809aa10 msgr2=0x7fb7180b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:38.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 -- 172.21.15.44:0/3860451954 shutdown_connections 2024-09-13T08:45:38.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.201+0000 7fb71d600640 1 -- 172.21.15.44:0/3860451954 wait complete. 2024-09-13T08:45:38.221 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-13T08:45:36.640901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":136,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402060,"kb_used_data":8148,"kb_used_omap":12,"kb_used_meta":393715,"kb_avail":754572660,"statfs":{"total":773094113280,"available":772682403840,"internally_reserved":0,"allocated":8343552,"data_stored":4204762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":403164721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.010718"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689535+0000","last_change":"2024-09-13T08:45:33.684143+0000","last_active":"2024-09-13T08:45:33.689535+0000","last_peered":"2024-09-13T08:45:33.689535+0000","last_clean":"2024-09-13T08:45:33.689535+0000","last_became_active":"2024-09-13T08:45:33.683889+0000","last_became_peered":"2024-09-13T08:45:33.683889+0000","last_unstale":"2024-09-13T08:45:33.689535+0000","last_undegraded":"2024-09-13T08:45:33.689535+0000","last_fullsized":"2024-09-13T08:45:33.689535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:07.001116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688670+0000","last_change":"2024-09-13T08:45:33.685799+0000","last_active":"2024-09-13T08:45:33.688670+0000","last_peered":"2024-09-13T08:45:33.688670+0000","last_clean":"2024-09-13T08:45:33.688670+0000","last_became_active":"2024-09-13T08:45:33.685475+0000","last_became_peered":"2024-09-13T08:45:33.685475+0000","last_unstale":"2024-09-13T08:45:33.688670+0000","last_undegraded":"2024-09-13T08:45:33.688670+0000","last_fullsized":"2024-09-13T08:45:33.688670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:35.321800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684692+0000","last_change":"2024-09-13T08:45:33.683135+0000","last_active":"2024-09-13T08:45:33.684692+0000","last_peered":"2024-09-13T08:45:33.684692+0000","last_clean":"2024-09-13T08:45:33.684692+0000","last_became_active":"2024-09-13T08:45:33.682789+0000","last_became_peered":"2024-09-13T08:45:33.682789+0000","last_unstale":"2024-09-13T08:45:33.684692+0000","last_undegraded":"2024-09-13T08:45:33.684692+0000","last_fullsized":"2024-09-13T08:45:33.684692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:47:16.394704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684656+0000","last_change":"2024-09-13T08:45:33.682881+0000","last_active":"2024-09-13T08:45:33.684656+0000","last_peered":"2024-09-13T08:45:33.684656+0000","last_clean":"2024-09-13T08:45:33.684656+0000","last_became_active":"2024-09-13T08:45:33.681260+0000","last_became_peered":"2024-09-13T08:45:33.681260+0000","last_unstale":"2024-09-13T08:45:33.684656+0000","last_undegraded":"2024-09-13T08:45:33.684656+0000","last_fullsized":"2024-09-13T08:45:33.684656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:34.085379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675098+0000","last_change":"2024-09-13T08:45:33.673961+0000","last_active":"2024-09-13T08:45:33.675098+0000","last_peered":"2024-09-13T08:45:33.675098+0000","last_clean":"2024-09-13T08:45:33.675098+0000","last_became_active":"2024-09-13T08:45:33.673661+0000","last_became_peered":"2024-09-13T08:45:33.673661+0000","last_unstale":"2024-09-13T08:45:33.675098+0000","last_undegraded":"2024-09-13T08:45:33.675098+0000","last_fullsized":"2024-09-13T08:45:33.675098+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:05.467156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689608+0000","last_change":"2024-09-13T08:45:33.686731+0000","last_active":"2024-09-13T08:45:33.689608+0000","last_peered":"2024-09-13T08:45:33.689608+0000","last_clean":"2024-09-13T08:45:33.689608+0000","last_became_active":"2024-09-13T08:45:33.686352+0000","last_became_peered":"2024-09-13T08:45:33.686352+0000","last_unstale":"2024-09-13T08:45:33.689608+0000","last_undegraded":"2024-09-13T08:45:33.689608+0000","last_fullsized":"2024-09-13T08:45:33.689608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:43:22.048135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.666172+0000","last_change":"2024-09-13T08:45:33.660192+0000","last_active":"2024-09-13T08:45:33.666172+0000","last_peered":"2024-09-13T08:45:33.666172+0000","last_clean":"2024-09-13T08:45:33.666172+0000","last_became_active":"2024-09-13T08:45:33.659425+0000","last_became_peered":"2024-09-13T08:45:33.659425+0000","last_unstale":"2024-09-13T08:45:33.666172+0000","last_undegraded":"2024-09-13T08:45:33.666172+0000","last_fullsized":"2024-09-13T08:45:33.666172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:27.961233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688123+0000","last_change":"2024-09-13T08:45:33.671839+0000","last_active":"2024-09-13T08:45:33.688123+0000","last_peered":"2024-09-13T08:45:33.688123+0000","last_clean":"2024-09-13T08:45:33.688123+0000","last_became_active":"2024-09-13T08:45:33.671538+0000","last_became_peered":"2024-09-13T08:45:33.671538+0000","last_unstale":"2024-09-13T08:45:33.688123+0000","last_undegraded":"2024-09-13T08:45:33.688123+0000","last_fullsized":"2024-09-13T08:45:33.688123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:51.161859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684700+0000","last_change":"2024-09-13T08:45:33.683295+0000","last_active":"2024-09-13T08:45:33.684700+0000","last_peered":"2024-09-13T08:45:33.684700+0000","last_clean":"2024-09-13T08:45:33.684700+0000","last_became_active":"2024-09-13T08:45:33.681988+0000","last_became_peered":"2024-09-13T08:45:33.681988+0000","last_unstale":"2024-09-13T08:45:33.684700+0000","last_undegraded":"2024-09-13T08:45:33.684700+0000","last_fullsized":"2024-09-13T08:45:33.684700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:34:29.504228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681352+0000","last_change":"2024-09-13T08:45:33.667868+0000","last_active":"2024-09-13T08:45:33.681352+0000","last_peered":"2024-09-13T08:45:33.681352+0000","last_clean":"2024-09-13T08:45:33.681352+0000","last_became_active":"2024-09-13T08:45:33.667543+0000","last_became_peered":"2024-09-13T08:45:33.667543+0000","last_unstale":"2024-09-13T08:45:33.681352+0000","last_undegraded":"2024-09-13T08:45:33.681352+0000","last_fullsized":"2024-09-13T08:45:33.681352+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:10.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691398+0000","last_change":"2024-09-13T08:45:33.689087+0000","last_active":"2024-09-13T08:45:33.691398+0000","last_peered":"2024-09-13T08:45:33.691398+0000","last_clean":"2024-09-13T08:45:33.691398+0000","last_became_active":"2024-09-13T08:45:33.688839+0000","last_became_peered":"2024-09-13T08:45:33.688839+0000","last_unstale":"2024-09-13T08:45:33.691398+0000","last_undegraded":"2024-09-13T08:45:33.691398+0000","last_fullsized":"2024-09-13T08:45:33.691398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:20:46.695925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.666207+0000","last_change":"2024-09-13T08:45:33.664987+0000","last_active":"2024-09-13T08:45:33.666207+0000","last_peered":"2024-09-13T08:45:33.666207+0000","last_clean":"2024-09-13T08:45:33.666207+0000","last_became_active":"2024-09-13T08:45:33.664707+0000","last_became_peered":"2024-09-13T08:45:33.664707+0000","last_unstale":"2024-09-13T08:45:33.666207+0000","last_undegraded":"2024-09-13T08:45:33.666207+0000","last_fullsized":"2024-09-13T08:45:33.666207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:03.021478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683740+0000","last_change":"2024-09-13T08:45:33.681174+0000","last_active":"2024-09-13T08:45:33.683740+0000","last_peered":"2024-09-13T08:45:33.683740+0000","last_clean":"2024-09-13T08:45:33.683740+0000","last_became_active":"2024-09-13T08:45:33.680680+0000","last_became_peered":"2024-09-13T08:45:33.680680+0000","last_unstale":"2024-09-13T08:45:33.683740+0000","last_undegraded":"2024-09-13T08:45:33.683740+0000","last_fullsized":"2024-09-13T08:45:33.683740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:46.928710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671724+0000","last_change":"2024-09-13T08:45:33.668789+0000","last_active":"2024-09-13T08:45:33.671724+0000","last_peered":"2024-09-13T08:45:33.671724+0000","last_clean":"2024-09-13T08:45:33.671724+0000","last_became_active":"2024-09-13T08:45:33.668462+0000","last_became_peered":"2024-09-13T08:45:33.668462+0000","last_unstale":"2024-09-13T08:45:33.671724+0000","last_undegraded":"2024-09-13T08:45:33.671724+0000","last_fullsized":"2024-09-13T08:45:33.671724+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:32:11.095202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.673865+0000","last_change":"2024-09-13T08:45:33.671105+0000","last_active":"2024-09-13T08:45:33.673865+0000","last_peered":"2024-09-13T08:45:33.673865+0000","last_clean":"2024-09-13T08:45:33.673865+0000","last_became_active":"2024-09-13T08:45:33.670718+0000","last_became_peered":"2024-09-13T08:45:33.670718+0000","last_unstale":"2024-09-13T08:45:33.673865+0000","last_undegraded":"2024-09-13T08:45:33.673865+0000","last_fullsized":"2024-09-13T08:45:33.673865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:34.021306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691685+0000","last_change":"2024-09-13T08:45:33.686654+0000","last_active":"2024-09-13T08:45:33.691685+0000","last_peered":"2024-09-13T08:45:33.691685+0000","last_clean":"2024-09-13T08:45:33.691685+0000","last_became_active":"2024-09-13T08:45:33.686375+0000","last_became_peered":"2024-09-13T08:45:33.686375+0000","last_unstale":"2024-09-13T08:45:33.691685+0000","last_undegraded":"2024-09-13T08:45:33.691685+0000","last_fullsized":"2024-09-13T08:45:33.691685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:03.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.672216+0000","last_change":"2024-09-13T08:45:33.670204+0000","last_active":"2024-09-13T08:45:33.672216+0000","last_peered":"2024-09-13T08:45:33.672216+0000","last_clean":"2024-09-13T08:45:33.672216+0000","last_became_active":"2024-09-13T08:45:33.669936+0000","last_became_peered":"2024-09-13T08:45:33.669936+0000","last_unstale":"2024-09-13T08:45:33.672216+0000","last_undegraded":"2024-09-13T08:45:33.672216+0000","last_fullsized":"2024-09-13T08:45:33.672216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:34.875650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667097+0000","last_change":"2024-09-13T08:45:33.663820+0000","last_active":"2024-09-13T08:45:33.667097+0000","last_peered":"2024-09-13T08:45:33.667097+0000","last_clean":"2024-09-13T08:45:33.667097+0000","last_became_active":"2024-09-13T08:45:33.663147+0000","last_became_peered":"2024-09-13T08:45:33.663147+0000","last_unstale":"2024-09-13T08:45:33.667097+0000","last_undegraded":"2024-09-13T08:45:33.667097+0000","last_fullsized":"2024-09-13T08:45:33.667097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:16:16.642146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.687335+0000","last_change":"2024-09-13T08:45:33.680218+0000","last_active":"2024-09-13T08:45:33.687335+0000","last_peered":"2024-09-13T08:45:33.687335+0000","last_clean":"2024-09-13T08:45:33.687335+0000","last_became_active":"2024-09-13T08:45:33.679938+0000","last_became_peered":"2024-09-13T08:45:33.679938+0000","last_unstale":"2024-09-13T08:45:33.687335+0000","last_undegraded":"2024-09-13T08:45:33.687335+0000","last_fullsized":"2024-09-13T08:45:33.687335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:05.467156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675520+0000","last_change":"2024-09-13T08:45:33.674349+0000","last_active":"2024-09-13T08:45:33.675520+0000","last_peered":"2024-09-13T08:45:33.675520+0000","last_clean":"2024-09-13T08:45:33.675520+0000","last_became_active":"2024-09-13T08:45:33.674004+0000","last_became_peered":"2024-09-13T08:45:33.674004+0000","last_unstale":"2024-09-13T08:45:33.675520+0000","last_undegraded":"2024-09-13T08:45:33.675520+0000","last_fullsized":"2024-09-13T08:45:33.675520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:06.280244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683524+0000","last_change":"2024-09-13T08:45:33.679260+0000","last_active":"2024-09-13T08:45:33.683524+0000","last_peered":"2024-09-13T08:45:33.683524+0000","last_clean":"2024-09-13T08:45:33.683524+0000","last_became_active":"2024-09-13T08:45:33.678976+0000","last_became_peered":"2024-09-13T08:45:33.678976+0000","last_unstale":"2024-09-13T08:45:33.683524+0000","last_undegraded":"2024-09-13T08:45:33.683524+0000","last_fullsized":"2024-09-13T08:45:33.683524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:41:27.288006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.673433+0000","last_change":"2024-09-13T08:45:33.666891+0000","last_active":"2024-09-13T08:45:33.673433+0000","last_peered":"2024-09-13T08:45:33.673433+0000","last_clean":"2024-09-13T08:45:33.673433+0000","last_became_active":"2024-09-13T08:45:33.666585+0000","last_became_peered":"2024-09-13T08:45:33.666585+0000","last_unstale":"2024-09-13T08:45:33.673433+0000","last_undegraded":"2024-09-13T08:45:33.673433+0000","last_fullsized":"2024-09-13T08:45:33.673433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:26:12.036228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.678998+0000","last_change":"2024-09-13T08:45:33.670178+0000","last_active":"2024-09-13T08:45:33.678998+0000","last_peered":"2024-09-13T08:45:33.678998+0000","last_clean":"2024-09-13T08:45:33.678998+0000","last_became_active":"2024-09-13T08:45:33.669687+0000","last_became_peered":"2024-09-13T08:45:33.669687+0000","last_unstale":"2024-09-13T08:45:33.678998+0000","last_undegraded":"2024-09-13T08:45:33.678998+0000","last_fullsized":"2024-09-13T08:45:33.678998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:11:12.735593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688146+0000","last_change":"2024-09-13T08:45:33.677052+0000","last_active":"2024-09-13T08:45:33.688146+0000","last_peered":"2024-09-13T08:45:33.688146+0000","last_clean":"2024-09-13T08:45:33.688146+0000","last_became_active":"2024-09-13T08:45:33.676767+0000","last_became_peered":"2024-09-13T08:45:33.676767+0000","last_unstale":"2024-09-13T08:45:33.688146+0000","last_undegraded":"2024-09-13T08:45:33.688146+0000","last_fullsized":"2024-09-13T08:45:33.688146+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:08.841526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.679899+0000","last_change":"2024-09-13T08:45:33.670240+0000","last_active":"2024-09-13T08:45:33.679899+0000","last_peered":"2024-09-13T08:45:33.679899+0000","last_clean":"2024-09-13T08:45:33.679899+0000","last_became_active":"2024-09-13T08:45:33.669952+0000","last_became_peered":"2024-09-13T08:45:33.669952+0000","last_unstale":"2024-09-13T08:45:33.679899+0000","last_undegraded":"2024-09-13T08:45:33.679899+0000","last_fullsized":"2024-09-13T08:45:33.679899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:15.070109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.692180+0000","last_change":"2024-09-13T08:45:33.690462+0000","last_active":"2024-09-13T08:45:33.692180+0000","last_peered":"2024-09-13T08:45:33.692180+0000","last_clean":"2024-09-13T08:45:33.692180+0000","last_became_active":"2024-09-13T08:45:33.690197+0000","last_became_peered":"2024-09-13T08:45:33.690197+0000","last_unstale":"2024-09-13T08:45:33.692180+0000","last_undegraded":"2024-09-13T08:45:33.692180+0000","last_fullsized":"2024-09-13T08:45:33.692180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:14.833887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.668085+0000","last_change":"2024-09-13T08:45:33.663806+0000","last_active":"2024-09-13T08:45:33.668085+0000","last_peered":"2024-09-13T08:45:33.668085+0000","last_clean":"2024-09-13T08:45:33.668085+0000","last_became_active":"2024-09-13T08:45:33.663498+0000","last_became_peered":"2024-09-13T08:45:33.663498+0000","last_unstale":"2024-09-13T08:45:33.668085+0000","last_undegraded":"2024-09-13T08:45:33.668085+0000","last_fullsized":"2024-09-13T08:45:33.668085+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:40:14.829023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683182+0000","last_change":"2024-09-13T08:45:33.674360+0000","last_active":"2024-09-13T08:45:33.683182+0000","last_peered":"2024-09-13T08:45:33.683182+0000","last_clean":"2024-09-13T08:45:33.683182+0000","last_became_active":"2024-09-13T08:45:33.673982+0000","last_became_peered":"2024-09-13T08:45:33.673982+0000","last_unstale":"2024-09-13T08:45:33.683182+0000","last_undegraded":"2024-09-13T08:45:33.683182+0000","last_fullsized":"2024-09-13T08:45:33.683182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:16:14.468554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689721+0000","last_change":"2024-09-13T08:45:33.689213+0000","last_active":"2024-09-13T08:45:33.689721+0000","last_peered":"2024-09-13T08:45:33.689721+0000","last_clean":"2024-09-13T08:45:33.689721+0000","last_became_active":"2024-09-13T08:45:33.688971+0000","last_became_peered":"2024-09-13T08:45:33.688971+0000","last_unstale":"2024-09-13T08:45:33.689721+0000","last_undegraded":"2024-09-13T08:45:33.689721+0000","last_fullsized":"2024-09-13T08:45:33.689721+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:48:29.114234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.687983+0000","last_change":"2024-09-13T08:45:33.672566+0000","last_active":"2024-09-13T08:45:33.687983+0000","last_peered":"2024-09-13T08:45:33.687983+0000","last_clean":"2024-09-13T08:45:33.687983+0000","last_became_active":"2024-09-13T08:45:33.672302+0000","last_became_peered":"2024-09-13T08:45:33.672302+0000","last_unstale":"2024-09-13T08:45:33.687983+0000","last_undegraded":"2024-09-13T08:45:33.687983+0000","last_fullsized":"2024-09-13T08:45:33.687983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:51:14.557452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684957+0000","last_change":"2024-09-13T08:45:33.675233+0000","last_active":"2024-09-13T08:45:33.684957+0000","last_peered":"2024-09-13T08:45:33.684957+0000","last_clean":"2024-09-13T08:45:33.684957+0000","last_became_active":"2024-09-13T08:45:33.674920+0000","last_became_peered":"2024-09-13T08:45:33.674920+0000","last_unstale":"2024-09-13T08:45:33.684957+0000","last_undegraded":"2024-09-13T08:45:33.684957+0000","last_fullsized":"2024-09-13T08:45:33.684957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:10.521583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683642+0000","last_change":"2024-09-13T08:45:33.679637+0000","last_active":"2024-09-13T08:45:33.683642+0000","last_peered":"2024-09-13T08:45:33.683642+0000","last_clean":"2024-09-13T08:45:33.683642+0000","last_became_active":"2024-09-13T08:45:33.679391+0000","last_became_peered":"2024-09-13T08:45:33.679391+0000","last_unstale":"2024-09-13T08:45:33.683642+0000","last_undegraded":"2024-09-13T08:45:33.683642+0000","last_fullsized":"2024-09-13T08:45:33.683642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:56.161329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691281+0000","last_change":"2024-09-13T08:45:33.688757+0000","last_active":"2024-09-13T08:45:33.691281+0000","last_peered":"2024-09-13T08:45:33.691281+0000","last_clean":"2024-09-13T08:45:33.691281+0000","last_became_active":"2024-09-13T08:45:33.687386+0000","last_became_peered":"2024-09-13T08:45:33.687386+0000","last_unstale":"2024-09-13T08:45:33.691281+0000","last_undegraded":"2024-09-13T08:45:33.691281+0000","last_fullsized":"2024-09-13T08:45:33.691281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:27.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"74'4","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689227+0000","last_change":"2024-09-13T08:45:32.657493+0000","last_active":"2024-09-13T08:45:33.689227+0000","last_peered":"2024-09-13T08:45:33.689227+0000","last_clean":"2024-09-13T08:45:33.689227+0000","last_became_active":"2024-09-13T08:45:32.656217+0000","last_became_peered":"2024-09-13T08:45:32.656217+0000","last_unstale":"2024-09-13T08:45:33.689227+0000","last_undegraded":"2024-09-13T08:45:33.689227+0000","last_fullsized":"2024-09-13T08:45:33.689227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:13.023500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683491+0000","last_change":"2024-09-13T08:45:33.678452+0000","last_active":"2024-09-13T08:45:33.683491+0000","last_peered":"2024-09-13T08:45:33.683491+0000","last_clean":"2024-09-13T08:45:33.683491+0000","last_became_active":"2024-09-13T08:45:33.678224+0000","last_became_peered":"2024-09-13T08:45:33.678224+0000","last_unstale":"2024-09-13T08:45:33.683491+0000","last_undegraded":"2024-09-13T08:45:33.683491+0000","last_fullsized":"2024-09-13T08:45:33.683491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:52:00.198742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684555+0000","last_change":"2024-09-13T08:45:32.654382+0000","last_active":"2024-09-13T08:45:33.684555+0000","last_peered":"2024-09-13T08:45:33.684555+0000","last_clean":"2024-09-13T08:45:33.684555+0000","last_became_active":"2024-09-13T08:45:32.654090+0000","last_became_peered":"2024-09-13T08:45:32.654090+0000","last_unstale":"2024-09-13T08:45:33.684555+0000","last_undegraded":"2024-09-13T08:45:33.684555+0000","last_fullsized":"2024-09-13T08:45:33.684555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:42.438703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.690499+0000","last_change":"2024-09-13T08:45:33.686795+0000","last_active":"2024-09-13T08:45:33.690499+0000","last_peered":"2024-09-13T08:45:33.690499+0000","last_clean":"2024-09-13T08:45:33.690499+0000","last_became_active":"2024-09-13T08:45:33.686497+0000","last_became_peered":"2024-09-13T08:45:33.686497+0000","last_unstale":"2024-09-13T08:45:33.690499+0000","last_undegraded":"2024-09-13T08:45:33.690499+0000","last_fullsized":"2024-09-13T08:45:33.690499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:27.050847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694008+0000","last_change":"2024-09-13T08:45:32.656780+0000","last_active":"2024-09-13T08:45:33.694008+0000","last_peered":"2024-09-13T08:45:33.694008+0000","last_clean":"2024-09-13T08:45:33.694008+0000","last_became_active":"2024-09-13T08:45:32.653941+0000","last_became_peered":"2024-09-13T08:45:32.653941+0000","last_unstale":"2024-09-13T08:45:33.694008+0000","last_undegraded":"2024-09-13T08:45:33.694008+0000","last_fullsized":"2024-09-13T08:45:33.694008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:12.824731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667910+0000","last_change":"2024-09-13T08:45:33.663007+0000","last_active":"2024-09-13T08:45:33.667910+0000","last_peered":"2024-09-13T08:45:33.667910+0000","last_clean":"2024-09-13T08:45:33.667910+0000","last_became_active":"2024-09-13T08:45:33.662591+0000","last_became_peered":"2024-09-13T08:45:33.662591+0000","last_unstale":"2024-09-13T08:45:33.667910+0000","last_undegraded":"2024-09-13T08:45:33.667910+0000","last_fullsized":"2024-09-13T08:45:33.667910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:59.290026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.690456+0000","last_change":"2024-09-13T08:45:32.655814+0000","last_active":"2024-09-13T08:45:33.690456+0000","last_peered":"2024-09-13T08:45:33.690456+0000","last_clean":"2024-09-13T08:45:33.690456+0000","last_became_active":"2024-09-13T08:45:32.655509+0000","last_became_peered":"2024-09-13T08:45:32.655509+0000","last_unstale":"2024-09-13T08:45:33.690456+0000","last_undegraded":"2024-09-13T08:45:33.690456+0000","last_fullsized":"2024-09-13T08:45:33.690456+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:56:39.873532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688008+0000","last_change":"2024-09-13T08:45:33.676002+0000","last_active":"2024-09-13T08:45:33.688008+0000","last_peered":"2024-09-13T08:45:33.688008+0000","last_clean":"2024-09-13T08:45:33.688008+0000","last_became_active":"2024-09-13T08:45:33.675711+0000","last_became_peered":"2024-09-13T08:45:33.675711+0000","last_unstale":"2024-09-13T08:45:33.688008+0000","last_undegraded":"2024-09-13T08:45:33.688008+0000","last_fullsized":"2024-09-13T08:45:33.688008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:34.875650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685399+0000","last_change":"2024-09-13T08:45:32.650357+0000","last_active":"2024-09-13T08:45:33.685399+0000","last_peered":"2024-09-13T08:45:33.685399+0000","last_clean":"2024-09-13T08:45:33.685399+0000","last_became_active":"2024-09-13T08:45:32.650092+0000","last_became_peered":"2024-09-13T08:45:32.650092+0000","last_unstale":"2024-09-13T08:45:33.685399+0000","last_undegraded":"2024-09-13T08:45:33.685399+0000","last_fullsized":"2024-09-13T08:45:33.685399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:11.131195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689533+0000","last_change":"2024-09-13T08:45:33.686668+0000","last_active":"2024-09-13T08:45:33.689533+0000","last_peered":"2024-09-13T08:45:33.689533+0000","last_clean":"2024-09-13T08:45:33.689533+0000","last_became_active":"2024-09-13T08:45:33.686340+0000","last_became_peered":"2024-09-13T08:45:33.686340+0000","last_unstale":"2024-09-13T08:45:33.689533+0000","last_undegraded":"2024-09-13T08:45:33.689533+0000","last_fullsized":"2024-09-13T08:45:33.689533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:45:11.714466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685672+0000","last_change":"2024-09-13T08:45:32.652970+0000","last_active":"2024-09-13T08:45:33.685672+0000","last_peered":"2024-09-13T08:45:33.685672+0000","last_clean":"2024-09-13T08:45:33.685672+0000","last_became_active":"2024-09-13T08:45:32.652708+0000","last_became_peered":"2024-09-13T08:45:32.652708+0000","last_unstale":"2024-09-13T08:45:33.685672+0000","last_undegraded":"2024-09-13T08:45:33.685672+0000","last_fullsized":"2024-09-13T08:45:33.685672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:26.448081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.644840+0000","last_change":"2024-09-13T08:45:32.644840+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.644840+0000","last_undegraded":"2024-09-13T08:45:32.644840+0000","last_fullsized":"2024-09-13T08:45:32.644840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T08:45:32.660378+0000","last_change":"2024-09-13T08:45:32.659746+0000","last_active":"2024-09-13T08:45:32.660378+0000","last_peered":"2024-09-13T08:45:32.660378+0000","last_clean":"2024-09-13T08:45:32.660378+0000","last_became_active":"2024-09-13T08:45:32.659314+0000","last_became_peered":"2024-09-13T08:45:32.659314+0000","last_unstale":"2024-09-13T08:45:32.660378+0000","last_undegraded":"2024-09-13T08:45:32.660378+0000","last_fullsized":"2024-09-13T08:45:32.660378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:08:43.439003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671291+0000","last_change":"2024-09-13T08:45:33.668090+0000","last_active":"2024-09-13T08:45:33.671291+0000","last_peered":"2024-09-13T08:45:33.671291+0000","last_clean":"2024-09-13T08:45:33.671291+0000","last_became_active":"2024-09-13T08:45:33.667802+0000","last_became_peered":"2024-09-13T08:45:33.667802+0000","last_unstale":"2024-09-13T08:45:33.671291+0000","last_undegraded":"2024-09-13T08:45:33.671291+0000","last_fullsized":"2024-09-13T08:45:33.671291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:02:31.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660673+0000","last_change":"2024-09-13T08:45:32.654860+0000","last_active":"2024-09-13T08:45:33.660673+0000","last_peered":"2024-09-13T08:45:33.660673+0000","last_clean":"2024-09-13T08:45:33.660673+0000","last_became_active":"2024-09-13T08:45:32.652510+0000","last_became_peered":"2024-09-13T08:45:32.652510+0000","last_unstale":"2024-09-13T08:45:33.660673+0000","last_undegraded":"2024-09-13T08:45:33.660673+0000","last_fullsized":"2024-09-13T08:45:33.660673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:16.016602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681459+0000","last_change":"2024-09-13T08:45:33.679316+0000","last_active":"2024-09-13T08:45:33.681459+0000","last_peered":"2024-09-13T08:45:33.681459+0000","last_clean":"2024-09-13T08:45:33.681459+0000","last_became_active":"2024-09-13T08:45:33.678605+0000","last_became_peered":"2024-09-13T08:45:33.678605+0000","last_unstale":"2024-09-13T08:45:33.681459+0000","last_undegraded":"2024-09-13T08:45:33.681459+0000","last_fullsized":"2024-09-13T08:45:33.681459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:01:38.403717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685240+0000","last_change":"2024-09-13T08:45:32.655976+0000","last_active":"2024-09-13T08:45:33.685240+0000","last_peered":"2024-09-13T08:45:33.685240+0000","last_clean":"2024-09-13T08:45:33.685240+0000","last_became_active":"2024-09-13T08:45:32.654678+0000","last_became_peered":"2024-09-13T08:45:32.654678+0000","last_unstale":"2024-09-13T08:45:33.685240+0000","last_undegraded":"2024-09-13T08:45:33.685240+0000","last_fullsized":"2024-09-13T08:45:33.685240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:07.445463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684924+0000","last_change":"2024-09-13T08:45:33.682397+0000","last_active":"2024-09-13T08:45:33.684924+0000","last_peered":"2024-09-13T08:45:33.684924+0000","last_clean":"2024-09-13T08:45:33.684924+0000","last_became_active":"2024-09-13T08:45:33.682010+0000","last_became_peered":"2024-09-13T08:45:33.682010+0000","last_unstale":"2024-09-13T08:45:33.684924+0000","last_undegraded":"2024-09-13T08:45:33.684924+0000","last_fullsized":"2024-09-13T08:45:33.684924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:39.373555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.670069+0000","last_change":"2024-09-13T08:45:32.654977+0000","last_active":"2024-09-13T08:45:33.670069+0000","last_peered":"2024-09-13T08:45:33.670069+0000","last_clean":"2024-09-13T08:45:33.670069+0000","last_became_active":"2024-09-13T08:45:32.654656+0000","last_became_peered":"2024-09-13T08:45:32.654656+0000","last_unstale":"2024-09-13T08:45:33.670069+0000","last_undegraded":"2024-09-13T08:45:33.670069+0000","last_fullsized":"2024-09-13T08:45:33.670069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:09.832815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689499+0000","last_change":"2024-09-13T08:45:33.682432+0000","last_active":"2024-09-13T08:45:33.689499+0000","last_peered":"2024-09-13T08:45:33.689499+0000","last_clean":"2024-09-13T08:45:33.689499+0000","last_became_active":"2024-09-13T08:45:33.681551+0000","last_became_peered":"2024-09-13T08:45:33.681551+0000","last_unstale":"2024-09-13T08:45:33.689499+0000","last_undegraded":"2024-09-13T08:45:33.689499+0000","last_fullsized":"2024-09-13T08:45:33.689499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:06.280244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"74'3","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682838+0000","last_change":"2024-09-13T08:45:32.644983+0000","last_active":"2024-09-13T08:45:33.682838+0000","last_peered":"2024-09-13T08:45:33.682838+0000","last_clean":"2024-09-13T08:45:33.682838+0000","last_became_active":"2024-09-13T08:45:32.644649+0000","last_became_peered":"2024-09-13T08:45:32.644649+0000","last_unstale":"2024-09-13T08:45:33.682838+0000","last_undegraded":"2024-09-13T08:45:33.682838+0000","last_fullsized":"2024-09-13T08:45:33.682838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:16:03.736952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.669128+0000","last_change":"2024-09-13T08:45:33.663848+0000","last_active":"2024-09-13T08:45:33.669128+0000","last_peered":"2024-09-13T08:45:33.669128+0000","last_clean":"2024-09-13T08:45:33.669128+0000","last_became_active":"2024-09-13T08:45:33.663559+0000","last_became_peered":"2024-09-13T08:45:33.663559+0000","last_unstale":"2024-09-13T08:45:33.669128+0000","last_undegraded":"2024-09-13T08:45:33.669128+0000","last_fullsized":"2024-09-13T08:45:33.669128+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:51.161859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T08:45:32.657443+0000","last_change":"2024-09-13T08:45:32.656020+0000","last_active":"2024-09-13T08:45:32.657443+0000","last_peered":"2024-09-13T08:45:32.657443+0000","last_clean":"2024-09-13T08:45:32.657443+0000","last_became_active":"2024-09-13T08:45:32.655215+0000","last_became_peered":"2024-09-13T08:45:32.655215+0000","last_unstale":"2024-09-13T08:45:32.657443+0000","last_undegraded":"2024-09-13T08:45:32.657443+0000","last_fullsized":"2024-09-13T08:45:32.657443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:44.310136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.677142+0000","last_change":"2024-09-13T08:45:33.671033+0000","last_active":"2024-09-13T08:45:33.677142+0000","last_peered":"2024-09-13T08:45:33.677142+0000","last_clean":"2024-09-13T08:45:33.677142+0000","last_became_active":"2024-09-13T08:45:33.670779+0000","last_became_peered":"2024-09-13T08:45:33.670779+0000","last_unstale":"2024-09-13T08:45:33.677142+0000","last_undegraded":"2024-09-13T08:45:33.677142+0000","last_fullsized":"2024-09-13T08:45:33.677142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:47:52.333905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T08:45:32.657464+0000","last_change":"2024-09-13T08:45:32.656334+0000","last_active":"2024-09-13T08:45:32.657464+0000","last_peered":"2024-09-13T08:45:32.657464+0000","last_clean":"2024-09-13T08:45:32.657464+0000","last_became_active":"2024-09-13T08:45:32.656007+0000","last_became_peered":"2024-09-13T08:45:32.656007+0000","last_unstale":"2024-09-13T08:45:32.657464+0000","last_undegraded":"2024-09-13T08:45:32.657464+0000","last_fullsized":"2024-09-13T08:45:32.657464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:45.983114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682359+0000","last_change":"2024-09-13T08:45:33.681389+0000","last_active":"2024-09-13T08:45:33.682359+0000","last_peered":"2024-09-13T08:45:33.682359+0000","last_clean":"2024-09-13T08:45:33.682359+0000","last_became_active":"2024-09-13T08:45:33.681085+0000","last_became_peered":"2024-09-13T08:45:33.681085+0000","last_unstale":"2024-09-13T08:45:33.682359+0000","last_undegraded":"2024-09-13T08:45:33.682359+0000","last_fullsized":"2024-09-13T08:45:33.682359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:38.549474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.652883+0000","last_change":"2024-09-13T08:45:32.641547+0000","last_active":"2024-09-13T08:45:33.652883+0000","last_peered":"2024-09-13T08:45:33.652883+0000","last_clean":"2024-09-13T08:45:33.652883+0000","last_became_active":"2024-09-13T08:45:32.641221+0000","last_became_peered":"2024-09-13T08:45:32.641221+0000","last_unstale":"2024-09-13T08:45:33.652883+0000","last_undegraded":"2024-09-13T08:45:33.652883+0000","last_fullsized":"2024-09-13T08:45:33.652883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:37.612881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671771+0000","last_change":"2024-09-13T08:45:33.668342+0000","last_active":"2024-09-13T08:45:33.671771+0000","last_peered":"2024-09-13T08:45:33.671771+0000","last_clean":"2024-09-13T08:45:33.671771+0000","last_became_active":"2024-09-13T08:45:33.668068+0000","last_became_peered":"2024-09-13T08:45:33.668068+0000","last_unstale":"2024-09-13T08:45:33.671771+0000","last_undegraded":"2024-09-13T08:45:33.671771+0000","last_fullsized":"2024-09-13T08:45:33.671771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:51:14.557452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.653649+0000","last_change":"2024-09-13T08:45:32.645856+0000","last_active":"2024-09-13T08:45:33.653649+0000","last_peered":"2024-09-13T08:45:33.653649+0000","last_clean":"2024-09-13T08:45:33.653649+0000","last_became_active":"2024-09-13T08:45:32.645551+0000","last_became_peered":"2024-09-13T08:45:32.645551+0000","last_unstale":"2024-09-13T08:45:33.653649+0000","last_undegraded":"2024-09-13T08:45:33.653649+0000","last_fullsized":"2024-09-13T08:45:33.653649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:13.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682542+0000","last_change":"2024-09-13T08:45:33.679240+0000","last_active":"2024-09-13T08:45:33.682542+0000","last_peered":"2024-09-13T08:45:33.682542+0000","last_clean":"2024-09-13T08:45:33.682542+0000","last_became_active":"2024-09-13T08:45:33.678945+0000","last_became_peered":"2024-09-13T08:45:33.678945+0000","last_unstale":"2024-09-13T08:45:33.682542+0000","last_undegraded":"2024-09-13T08:45:33.682542+0000","last_fullsized":"2024-09-13T08:45:33.682542+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:18.936719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"74'8","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691071+0000","last_change":"2024-09-13T08:45:32.644799+0000","last_active":"2024-09-13T08:45:33.691071+0000","last_peered":"2024-09-13T08:45:33.691071+0000","last_clean":"2024-09-13T08:45:33.691071+0000","last_became_active":"2024-09-13T08:45:32.644559+0000","last_became_peered":"2024-09-13T08:45:32.644559+0000","last_unstale":"2024-09-13T08:45:33.691071+0000","last_undegraded":"2024-09-13T08:45:33.691071+0000","last_fullsized":"2024-09-13T08:45:33.691071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:23.392065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680801+0000","last_change":"2024-09-13T08:45:33.676706+0000","last_active":"2024-09-13T08:45:33.680801+0000","last_peered":"2024-09-13T08:45:33.680801+0000","last_clean":"2024-09-13T08:45:33.680801+0000","last_became_active":"2024-09-13T08:45:33.676442+0000","last_became_peered":"2024-09-13T08:45:33.676442+0000","last_unstale":"2024-09-13T08:45:33.680801+0000","last_undegraded":"2024-09-13T08:45:33.680801+0000","last_fullsized":"2024-09-13T08:45:33.680801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:00.210640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694996+0000","last_change":"2024-09-13T08:45:32.655505+0000","last_active":"2024-09-13T08:45:33.694996+0000","last_peered":"2024-09-13T08:45:33.694996+0000","last_clean":"2024-09-13T08:45:33.694996+0000","last_became_active":"2024-09-13T08:45:32.655218+0000","last_became_peered":"2024-09-13T08:45:32.655218+0000","last_unstale":"2024-09-13T08:45:33.694996+0000","last_undegraded":"2024-09-13T08:45:33.694996+0000","last_fullsized":"2024-09-13T08:45:33.694996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:04.883497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680842+0000","last_change":"2024-09-13T08:45:33.675626+0000","last_active":"2024-09-13T08:45:33.680842+0000","last_peered":"2024-09-13T08:45:33.680842+0000","last_clean":"2024-09-13T08:45:33.680842+0000","last_became_active":"2024-09-13T08:45:33.675346+0000","last_became_peered":"2024-09-13T08:45:33.675346+0000","last_unstale":"2024-09-13T08:45:33.680842+0000","last_undegraded":"2024-09-13T08:45:33.680842+0000","last_fullsized":"2024-09-13T08:45:33.680842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:22.946990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685223+0000","last_change":"2024-09-13T08:45:32.658779+0000","last_active":"2024-09-13T08:45:33.685223+0000","last_peered":"2024-09-13T08:45:33.685223+0000","last_clean":"2024-09-13T08:45:33.685223+0000","last_became_active":"2024-09-13T08:45:32.658420+0000","last_became_peered":"2024-09-13T08:45:32.658420+0000","last_unstale":"2024-09-13T08:45:33.685223+0000","last_undegraded":"2024-09-13T08:45:33.685223+0000","last_fullsized":"2024-09-13T08:45:33.685223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:18.130462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680052+0000","last_change":"2024-09-13T08:45:33.675553+0000","last_active":"2024-09-13T08:45:33.680052+0000","last_peered":"2024-09-13T08:45:33.680052+0000","last_clean":"2024-09-13T08:45:33.680052+0000","last_became_active":"2024-09-13T08:45:33.675233+0000","last_became_peered":"2024-09-13T08:45:33.675233+0000","last_unstale":"2024-09-13T08:45:33.680052+0000","last_undegraded":"2024-09-13T08:45:33.680052+0000","last_fullsized":"2024-09-13T08:45:33.680052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:41.676077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.692093+0000","last_change":"2024-09-13T08:45:32.659506+0000","last_active":"2024-09-13T08:45:33.692093+0000","last_peered":"2024-09-13T08:45:33.692093+0000","last_clean":"2024-09-13T08:45:33.692093+0000","last_became_active":"2024-09-13T08:45:32.659220+0000","last_became_peered":"2024-09-13T08:45:32.659220+0000","last_unstale":"2024-09-13T08:45:33.692093+0000","last_undegraded":"2024-09-13T08:45:33.692093+0000","last_fullsized":"2024-09-13T08:45:33.692093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:30.318592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689473+0000","last_change":"2024-09-13T08:45:33.688805+0000","last_active":"2024-09-13T08:45:33.689473+0000","last_peered":"2024-09-13T08:45:33.689473+0000","last_clean":"2024-09-13T08:45:33.689473+0000","last_became_active":"2024-09-13T08:45:33.688534+0000","last_became_peered":"2024-09-13T08:45:33.688534+0000","last_unstale":"2024-09-13T08:45:33.689473+0000","last_undegraded":"2024-09-13T08:45:33.689473+0000","last_fullsized":"2024-09-13T08:45:33.689473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:11.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660382+0000","last_change":"2024-09-13T08:45:32.655084+0000","last_active":"2024-09-13T08:45:33.660382+0000","last_peered":"2024-09-13T08:45:33.660382+0000","last_clean":"2024-09-13T08:45:33.660382+0000","last_became_active":"2024-09-13T08:45:32.654795+0000","last_became_peered":"2024-09-13T08:45:32.654795+0000","last_unstale":"2024-09-13T08:45:33.660382+0000","last_undegraded":"2024-09-13T08:45:33.660382+0000","last_fullsized":"2024-09-13T08:45:33.660382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:23:00.745264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.641979+0000","last_change":"2024-09-13T08:45:32.641979+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.641979+0000","last_undegraded":"2024-09-13T08:45:32.641979+0000","last_fullsized":"2024-09-13T08:45:32.641979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.661857+0000","last_change":"2024-09-13T08:45:32.648855+0000","last_active":"2024-09-13T08:45:33.661857+0000","last_peered":"2024-09-13T08:45:33.661857+0000","last_clean":"2024-09-13T08:45:33.661857+0000","last_became_active":"2024-09-13T08:45:32.648572+0000","last_became_peered":"2024-09-13T08:45:32.648572+0000","last_unstale":"2024-09-13T08:45:33.661857+0000","last_undegraded":"2024-09-13T08:45:33.661857+0000","last_fullsized":"2024-09-13T08:45:33.661857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:11.383879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.644401+0000","last_change":"2024-09-13T08:45:32.644401+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.644401+0000","last_undegraded":"2024-09-13T08:45:32.644401+0000","last_fullsized":"2024-09-13T08:45:32.644401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.664419+0000","last_change":"2024-09-13T08:45:32.651171+0000","last_active":"2024-09-13T08:45:33.664419+0000","last_peered":"2024-09-13T08:45:33.664419+0000","last_clean":"2024-09-13T08:45:33.664419+0000","last_became_active":"2024-09-13T08:45:32.650437+0000","last_became_peered":"2024-09-13T08:45:32.650437+0000","last_unstale":"2024-09-13T08:45:33.664419+0000","last_undegraded":"2024-09-13T08:45:33.664419+0000","last_fullsized":"2024-09-13T08:45:33.664419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:43.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671316+0000","last_change":"2024-09-13T08:45:33.670268+0000","last_active":"2024-09-13T08:45:33.671316+0000","last_peered":"2024-09-13T08:45:33.671316+0000","last_clean":"2024-09-13T08:45:33.671316+0000","last_became_active":"2024-09-13T08:45:33.669974+0000","last_became_peered":"2024-09-13T08:45:33.669974+0000","last_unstale":"2024-09-13T08:45:33.671316+0000","last_undegraded":"2024-09-13T08:45:33.671316+0000","last_fullsized":"2024-09-13T08:45:33.671316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:14.031957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.657855+0000","last_change":"2024-09-13T08:45:32.652196+0000","last_active":"2024-09-13T08:45:33.657855+0000","last_peered":"2024-09-13T08:45:33.657855+0000","last_clean":"2024-09-13T08:45:33.657855+0000","last_became_active":"2024-09-13T08:45:32.651851+0000","last_became_peered":"2024-09-13T08:45:32.651851+0000","last_unstale":"2024-09-13T08:45:33.657855+0000","last_undegraded":"2024-09-13T08:45:33.657855+0000","last_fullsized":"2024-09-13T08:45:33.657855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:48:29.415637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.668002+0000","last_change":"2024-09-13T08:45:33.664149+0000","last_active":"2024-09-13T08:45:33.668002+0000","last_peered":"2024-09-13T08:45:33.668002+0000","last_clean":"2024-09-13T08:45:33.668002+0000","last_became_active":"2024-09-13T08:45:33.663531+0000","last_became_peered":"2024-09-13T08:45:33.663531+0000","last_unstale":"2024-09-13T08:45:33.668002+0000","last_undegraded":"2024-09-13T08:45:33.668002+0000","last_fullsized":"2024-09-13T08:45:33.668002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:18:36.348488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694733+0000","last_change":"2024-09-13T08:45:32.660012+0000","last_active":"2024-09-13T08:45:33.694733+0000","last_peered":"2024-09-13T08:45:33.694733+0000","last_clean":"2024-09-13T08:45:33.694733+0000","last_became_active":"2024-09-13T08:45:32.659398+0000","last_became_peered":"2024-09-13T08:45:32.659398+0000","last_unstale":"2024-09-13T08:45:33.694733+0000","last_undegraded":"2024-09-13T08:45:33.694733+0000","last_fullsized":"2024-09-13T08:45:33.694733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:12.195364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667819+0000","last_change":"2024-09-13T08:45:33.660680+0000","last_active":"2024-09-13T08:45:33.667819+0000","last_peered":"2024-09-13T08:45:33.667819+0000","last_clean":"2024-09-13T08:45:33.667819+0000","last_became_active":"2024-09-13T08:45:33.660407+0000","last_became_peered":"2024-09-13T08:45:33.660407+0000","last_unstale":"2024-09-13T08:45:33.667819+0000","last_undegraded":"2024-09-13T08:45:33.667819+0000","last_fullsized":"2024-09-13T08:45:33.667819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:21:26.352682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684746+0000","last_change":"2024-09-13T08:45:32.661277+0000","last_active":"2024-09-13T08:45:33.684746+0000","last_peered":"2024-09-13T08:45:33.684746+0000","last_clean":"2024-09-13T08:45:33.684746+0000","last_became_active":"2024-09-13T08:45:32.660211+0000","last_became_peered":"2024-09-13T08:45:32.660211+0000","last_unstale":"2024-09-13T08:45:33.684746+0000","last_undegraded":"2024-09-13T08:45:33.684746+0000","last_fullsized":"2024-09-13T08:45:33.684746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:01.561248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684943+0000","last_change":"2024-09-13T08:45:33.683345+0000","last_active":"2024-09-13T08:45:33.684943+0000","last_peered":"2024-09-13T08:45:33.684943+0000","last_clean":"2024-09-13T08:45:33.684943+0000","last_became_active":"2024-09-13T08:45:33.682992+0000","last_became_peered":"2024-09-13T08:45:33.682992+0000","last_unstale":"2024-09-13T08:45:33.684943+0000","last_undegraded":"2024-09-13T08:45:33.684943+0000","last_fullsized":"2024-09-13T08:45:33.684943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:47.803484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660890+0000","last_change":"2024-09-13T08:45:32.646650+0000","last_active":"2024-09-13T08:45:33.660890+0000","last_peered":"2024-09-13T08:45:33.660890+0000","last_clean":"2024-09-13T08:45:33.660890+0000","last_became_active":"2024-09-13T08:45:32.646362+0000","last_became_peered":"2024-09-13T08:45:32.646362+0000","last_unstale":"2024-09-13T08:45:33.660890+0000","last_undegraded":"2024-09-13T08:45:33.660890+0000","last_fullsized":"2024-09-13T08:45:33.660890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:48:47.580467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680628+0000","last_change":"2024-09-13T08:45:33.678066+0000","last_active":"2024-09-13T08:45:33.680628+0000","last_peered":"2024-09-13T08:45:33.680628+0000","last_clean":"2024-09-13T08:45:33.680628+0000","last_became_active":"2024-09-13T08:45:33.674778+0000","last_became_peered":"2024-09-13T08:45:33.674778+0000","last_unstale":"2024-09-13T08:45:33.680628+0000","last_undegraded":"2024-09-13T08:45:33.680628+0000","last_fullsized":"2024-09-13T08:45:33.680628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:56.987435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.661122+0000","last_change":"2024-09-13T08:45:32.646318+0000","last_active":"2024-09-13T08:45:33.661122+0000","last_peered":"2024-09-13T08:45:33.661122+0000","last_clean":"2024-09-13T08:45:33.661122+0000","last_became_active":"2024-09-13T08:45:32.645801+0000","last_became_peered":"2024-09-13T08:45:32.645801+0000","last_unstale":"2024-09-13T08:45:33.661122+0000","last_undegraded":"2024-09-13T08:45:33.661122+0000","last_fullsized":"2024-09-13T08:45:33.661122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:40.323104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.672925+0000","last_change":"2024-09-13T08:45:33.670611+0000","last_active":"2024-09-13T08:45:33.672925+0000","last_peered":"2024-09-13T08:45:33.672925+0000","last_clean":"2024-09-13T08:45:33.672925+0000","last_became_active":"2024-09-13T08:45:33.670334+0000","last_became_peered":"2024-09-13T08:45:33.670334+0000","last_unstale":"2024-09-13T08:45:33.672925+0000","last_undegraded":"2024-09-13T08:45:33.672925+0000","last_fullsized":"2024-09-13T08:45:33.672925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:08.841526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.662228+0000","last_change":"2024-09-13T08:45:32.650061+0000","last_active":"2024-09-13T08:45:33.662228+0000","last_peered":"2024-09-13T08:45:33.662228+0000","last_clean":"2024-09-13T08:45:33.662228+0000","last_became_active":"2024-09-13T08:45:32.649743+0000","last_became_peered":"2024-09-13T08:45:32.649743+0000","last_unstale":"2024-09-13T08:45:33.662228+0000","last_undegraded":"2024-09-13T08:45:33.662228+0000","last_fullsized":"2024-09-13T08:45:33.662228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:16.817675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691205+0000","last_change":"2024-09-13T08:45:33.682926+0000","last_active":"2024-09-13T08:45:33.691205+0000","last_peered":"2024-09-13T08:45:33.691205+0000","last_clean":"2024-09-13T08:45:33.691205+0000","last_became_active":"2024-09-13T08:45:33.681036+0000","last_became_peered":"2024-09-13T08:45:33.681036+0000","last_unstale":"2024-09-13T08:45:33.691205+0000","last_undegraded":"2024-09-13T08:45:33.691205+0000","last_fullsized":"2024-09-13T08:45:33.691205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:31.247460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.655978+0000","last_change":"2024-09-13T08:45:32.651908+0000","last_active":"2024-09-13T08:45:33.655978+0000","last_peered":"2024-09-13T08:45:33.655978+0000","last_clean":"2024-09-13T08:45:33.655978+0000","last_became_active":"2024-09-13T08:45:32.651622+0000","last_became_peered":"2024-09-13T08:45:32.651622+0000","last_unstale":"2024-09-13T08:45:33.655978+0000","last_undegraded":"2024-09-13T08:45:33.655978+0000","last_fullsized":"2024-09-13T08:45:33.655978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:32.504688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.642635+0000","last_change":"2024-09-13T08:45:32.642635+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.642635+0000","last_undegraded":"2024-09-13T08:45:32.642635+0000","last_fullsized":"2024-09-13T08:45:32.642635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.664624+0000","last_change":"2024-09-13T08:45:32.656890+0000","last_active":"2024-09-13T08:45:33.664624+0000","last_peered":"2024-09-13T08:45:33.664624+0000","last_clean":"2024-09-13T08:45:33.664624+0000","last_became_active":"2024-09-13T08:45:32.656592+0000","last_became_peered":"2024-09-13T08:45:32.656592+0000","last_unstale":"2024-09-13T08:45:33.664624+0000","last_undegraded":"2024-09-13T08:45:33.664624+0000","last_fullsized":"2024-09-13T08:45:33.664624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:38.811210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675107+0000","last_change":"2024-09-13T08:45:33.669976+0000","last_active":"2024-09-13T08:45:33.675107+0000","last_peered":"2024-09-13T08:45:33.675107+0000","last_clean":"2024-09-13T08:45:33.675107+0000","last_became_active":"2024-09-13T08:45:33.669693+0000","last_became_peered":"2024-09-13T08:45:33.669693+0000","last_unstale":"2024-09-13T08:45:33.675107+0000","last_undegraded":"2024-09-13T08:45:33.675107+0000","last_fullsized":"2024-09-13T08:45:33.675107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:27:36.696332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.658907+0000","last_change":"2024-09-13T08:45:32.654218+0000","last_active":"2024-09-13T08:45:33.658907+0000","last_peered":"2024-09-13T08:45:33.658907+0000","last_clean":"2024-09-13T08:45:33.658907+0000","last_became_active":"2024-09-13T08:45:32.653519+0000","last_became_peered":"2024-09-13T08:45:32.653519+0000","last_unstale":"2024-09-13T08:45:33.658907+0000","last_undegraded":"2024-09-13T08:45:33.658907+0000","last_fullsized":"2024-09-13T08:45:33.658907+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:33.012849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681797+0000","last_change":"2024-09-13T08:45:33.679432+0000","last_active":"2024-09-13T08:45:33.681797+0000","last_peered":"2024-09-13T08:45:33.681797+0000","last_clean":"2024-09-13T08:45:33.681797+0000","last_became_active":"2024-09-13T08:45:33.679129+0000","last_became_peered":"2024-09-13T08:45:33.679129+0000","last_unstale":"2024-09-13T08:45:33.681797+0000","last_undegraded":"2024-09-13T08:45:33.681797+0000","last_fullsized":"2024-09-13T08:45:33.681797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:31.247460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681998+0000","last_change":"2024-09-13T08:45:32.647002+0000","last_active":"2024-09-13T08:45:33.681998+0000","last_peered":"2024-09-13T08:45:33.681998+0000","last_clean":"2024-09-13T08:45:33.681998+0000","last_became_active":"2024-09-13T08:45:32.645456+0000","last_became_peered":"2024-09-13T08:45:32.645456+0000","last_unstale":"2024-09-13T08:45:33.681998+0000","last_undegraded":"2024-09-13T08:45:33.681998+0000","last_fullsized":"2024-09-13T08:45:33.681998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:41.793310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":212,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.652555+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:45:33.652555+0000","last_peered":"2024-09-13T08:45:33.652555+0000","last_clean":"2024-09-13T08:45:33.652555+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:45:33.652555+0000","last_undegraded":"2024-09-13T08:45:33.652555+0000","last_fullsized":"2024-09-13T08:45:33.652555+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509540,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50120,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321720,"statfs":{"total":96636764160,"available":96585441280,"internally_reserved":0,"allocated":905216,"data_stored":380942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50696,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321144,"statfs":{"total":96636764160,"available":96584851456,"internally_reserved":0,"allocated":1490944,"data_stored":969788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509541,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54288,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317552,"statfs":{"total":96636764160,"available":96581173248,"internally_reserved":0,"allocated":905216,"data_stored":380024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":880640,"data_stored":372499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":897024,"data_stored":379912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509542,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46532,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325308,"statfs":{"total":96636764160,"available":96589115392,"internally_reserved":0,"allocated":1486848,"data_stored":969698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":897024,"data_stored":379400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542246,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":880640,"data_stored":372499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:45:38.225 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:45:38.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.413+0000 7f0665e00640 1 -- 172.21.15.44:0/2881251643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0648004810 msgr2=0x7f0648004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.413+0000 7f0665e00640 1 --2- 172.21.15.44:0/2881251643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0648004810 0x7f0648004c90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c310 tx=0x7f064c01cf60 comp rx=0 tx=0).stop 2024-09-13T08:45:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.413+0000 7f0665400640 1 -- 172.21.15.44:0/2881251643 reap_dead start 2024-09-13T08:45:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.413+0000 7f0665e00640 1 -- 172.21.15.44:0/2881251643 shutdown_connections 2024-09-13T08:45:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.413+0000 7f0665e00640 1 -- 172.21.15.44:0/2881251643 >> 172.21.15.44:0/2881251643 conn(0x7f066009aa10 msgr2=0x7f066009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.413+0000 7f0665e00640 1 -- 172.21.15.44:0/2881251643 shutdown_connections 2024-09-13T08:45:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.413+0000 7f0665e00640 1 -- 172.21.15.44:0/2881251643 wait complete. 2024-09-13T08:45:38.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.413+0000 7f0665e00640 1 Processor -- start 2024-09-13T08:45:38.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.413+0000 7f0665e00640 1 -- start start 2024-09-13T08:45:38.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0665e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0648004810 0x7f06601f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0665e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06600cb940 0x7f06600b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0665e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06600c7f80 0x7f06600c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0665e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0660167f60 con 0x7f06600cb940 2024-09-13T08:45:38.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0665e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0660167de0 con 0x7f06600c7f80 2024-09-13T08:45:38.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0665e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f06601680e0 con 0x7f0648004810 2024-09-13T08:45:38.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06600cb940 0x7f06600b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06600cb940 0x7f06600b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38026/0 (socket says 172.21.15.44:38026) 2024-09-13T08:45:38.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0664a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0648004810 0x7f06601f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065fe00640 1 -- 172.21.15.44:0/24830421 learned_addr learned my addr 172.21.15.44:0/24830421 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:38.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0665400640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06600c7f80 0x7f06600c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065fe00640 1 -- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0648004810 msgr2=0x7f06601f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065fe00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0648004810 0x7f06601f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065fe00640 1 -- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06600c7f80 msgr2=0x7f06600c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065fe00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06600c7f80 0x7f06600c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065fe00640 1 -- 172.21.15.44:0/24830421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06600c8940 con 0x7f06600cb940 2024-09-13T08:45:38.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0664a00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0648004810 0x7f06601f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:45:38.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065fe00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06600cb940 0x7f06600b5a50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f064c01ca20 tx=0x7f064c001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:38.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065cc00640 1 -- 172.21.15.44:0/24830421 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c0014c0 con 0x7f06600cb940 2024-09-13T08:45:38.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065cc00640 1 -- 172.21.15.44:0/24830421 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f064c001660 con 0x7f06600cb940 2024-09-13T08:45:38.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f065cc00640 1 -- 172.21.15.44:0/24830421 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c006330 con 0x7f06600cb940 2024-09-13T08:45:38.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0665e00640 1 -- 172.21.15.44:0/24830421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06600b62a0 con 0x7f06600cb940 2024-09-13T08:45:38.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0665e00640 1 -- 172.21.15.44:0/24830421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06600b6860 con 0x7f06600cb940 2024-09-13T08:45:38.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.417+0000 7f0665e00640 1 -- 172.21.15.44:0/24830421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06600b29e0 con 0x7f06600cb940 2024-09-13T08:45:38.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.421+0000 7f065cc00640 1 -- 172.21.15.44:0/24830421 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f064c0b7030 con 0x7f06600cb940 2024-09-13T08:45:38.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.421+0000 7f065cc00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0640073b50 0x7f0640076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.421+0000 7f065cc00640 1 -- 172.21.15.44:0/24830421 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f064c12bce0 con 0x7f06600cb940 2024-09-13T08:45:38.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.421+0000 7f0664a00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0640073b50 0x7f0640076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.421+0000 7f0664a00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0640073b50 0x7f0640076010 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f06540124a0 tx=0x7f06540091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:38.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.425+0000 7f065cc00640 1 -- 172.21.15.44:0/24830421 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f064c016d10 con 0x7f06600cb940 2024-09-13T08:45:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.597+0000 7f0665e00640 1 -- 172.21.15.44:0/24830421 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f066015fd20 con 0x7f0640073b50 2024-09-13T08:45:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.601+0000 7f065cc00640 1 -- 172.21.15.44:0/24830421 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251714 (secure 0 0 0) 0x7f065405e760 con 0x7f0640073b50 2024-09-13T08:45:38.606 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:38.607 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:45:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 -- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0640073b50 msgr2=0x7f0640076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0640073b50 0x7f0640076010 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f06540124a0 tx=0x7f06540091e0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 -- 172.21.15.44:0/24830421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06600cb940 msgr2=0x7f06600b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06600cb940 0x7f06600b5a50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f064c01ca20 tx=0x7f064c001d00 comp rx=0 tx=0).stop 2024-09-13T08:45:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 -- 172.21.15.44:0/24830421 shutdown_connections 2024-09-13T08:45:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0640073b50 0x7f0640076010 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06600c7f80 0x7f06600c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06600cb940 0x7f06600b5a50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 --2- 172.21.15.44:0/24830421 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0648004810 0x7f06601f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 -- 172.21.15.44:0/24830421 >> 172.21.15.44:0/24830421 conn(0x7f066009aa10 msgr2=0x7f0660071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 -- 172.21.15.44:0/24830421 shutdown_connections 2024-09-13T08:45:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.605+0000 7f0665e00640 1 -- 172.21.15.44:0/24830421 wait complete. 2024-09-13T08:45:38.626 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-13T08:45:36.640901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":136,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402060,"kb_used_data":8148,"kb_used_omap":12,"kb_used_meta":393715,"kb_avail":754572660,"statfs":{"total":773094113280,"available":772682403840,"internally_reserved":0,"allocated":8343552,"data_stored":4204762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":403164721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.010718"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689535+0000","last_change":"2024-09-13T08:45:33.684143+0000","last_active":"2024-09-13T08:45:33.689535+0000","last_peered":"2024-09-13T08:45:33.689535+0000","last_clean":"2024-09-13T08:45:33.689535+0000","last_became_active":"2024-09-13T08:45:33.683889+0000","last_became_peered":"2024-09-13T08:45:33.683889+0000","last_unstale":"2024-09-13T08:45:33.689535+0000","last_undegraded":"2024-09-13T08:45:33.689535+0000","last_fullsized":"2024-09-13T08:45:33.689535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:07.001116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688670+0000","last_change":"2024-09-13T08:45:33.685799+0000","last_active":"2024-09-13T08:45:33.688670+0000","last_peered":"2024-09-13T08:45:33.688670+0000","last_clean":"2024-09-13T08:45:33.688670+0000","last_became_active":"2024-09-13T08:45:33.685475+0000","last_became_peered":"2024-09-13T08:45:33.685475+0000","last_unstale":"2024-09-13T08:45:33.688670+0000","last_undegraded":"2024-09-13T08:45:33.688670+0000","last_fullsized":"2024-09-13T08:45:33.688670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:35.321800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684692+0000","last_change":"2024-09-13T08:45:33.683135+0000","last_active":"2024-09-13T08:45:33.684692+0000","last_peered":"2024-09-13T08:45:33.684692+0000","last_clean":"2024-09-13T08:45:33.684692+0000","last_became_active":"2024-09-13T08:45:33.682789+0000","last_became_peered":"2024-09-13T08:45:33.682789+0000","last_unstale":"2024-09-13T08:45:33.684692+0000","last_undegraded":"2024-09-13T08:45:33.684692+0000","last_fullsized":"2024-09-13T08:45:33.684692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:47:16.394704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684656+0000","last_change":"2024-09-13T08:45:33.682881+0000","last_active":"2024-09-13T08:45:33.684656+0000","last_peered":"2024-09-13T08:45:33.684656+0000","last_clean":"2024-09-13T08:45:33.684656+0000","last_became_active":"2024-09-13T08:45:33.681260+0000","last_became_peered":"2024-09-13T08:45:33.681260+0000","last_unstale":"2024-09-13T08:45:33.684656+0000","last_undegraded":"2024-09-13T08:45:33.684656+0000","last_fullsized":"2024-09-13T08:45:33.684656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:34.085379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675098+0000","last_change":"2024-09-13T08:45:33.673961+0000","last_active":"2024-09-13T08:45:33.675098+0000","last_peered":"2024-09-13T08:45:33.675098+0000","last_clean":"2024-09-13T08:45:33.675098+0000","last_became_active":"2024-09-13T08:45:33.673661+0000","last_became_peered":"2024-09-13T08:45:33.673661+0000","last_unstale":"2024-09-13T08:45:33.675098+0000","last_undegraded":"2024-09-13T08:45:33.675098+0000","last_fullsized":"2024-09-13T08:45:33.675098+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:05.467156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689608+0000","last_change":"2024-09-13T08:45:33.686731+0000","last_active":"2024-09-13T08:45:33.689608+0000","last_peered":"2024-09-13T08:45:33.689608+0000","last_clean":"2024-09-13T08:45:33.689608+0000","last_became_active":"2024-09-13T08:45:33.686352+0000","last_became_peered":"2024-09-13T08:45:33.686352+0000","last_unstale":"2024-09-13T08:45:33.689608+0000","last_undegraded":"2024-09-13T08:45:33.689608+0000","last_fullsized":"2024-09-13T08:45:33.689608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:43:22.048135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.666172+0000","last_change":"2024-09-13T08:45:33.660192+0000","last_active":"2024-09-13T08:45:33.666172+0000","last_peered":"2024-09-13T08:45:33.666172+0000","last_clean":"2024-09-13T08:45:33.666172+0000","last_became_active":"2024-09-13T08:45:33.659425+0000","last_became_peered":"2024-09-13T08:45:33.659425+0000","last_unstale":"2024-09-13T08:45:33.666172+0000","last_undegraded":"2024-09-13T08:45:33.666172+0000","last_fullsized":"2024-09-13T08:45:33.666172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:27.961233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688123+0000","last_change":"2024-09-13T08:45:33.671839+0000","last_active":"2024-09-13T08:45:33.688123+0000","last_peered":"2024-09-13T08:45:33.688123+0000","last_clean":"2024-09-13T08:45:33.688123+0000","last_became_active":"2024-09-13T08:45:33.671538+0000","last_became_peered":"2024-09-13T08:45:33.671538+0000","last_unstale":"2024-09-13T08:45:33.688123+0000","last_undegraded":"2024-09-13T08:45:33.688123+0000","last_fullsized":"2024-09-13T08:45:33.688123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:51.161859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684700+0000","last_change":"2024-09-13T08:45:33.683295+0000","last_active":"2024-09-13T08:45:33.684700+0000","last_peered":"2024-09-13T08:45:33.684700+0000","last_clean":"2024-09-13T08:45:33.684700+0000","last_became_active":"2024-09-13T08:45:33.681988+0000","last_became_peered":"2024-09-13T08:45:33.681988+0000","last_unstale":"2024-09-13T08:45:33.684700+0000","last_undegraded":"2024-09-13T08:45:33.684700+0000","last_fullsized":"2024-09-13T08:45:33.684700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:34:29.504228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681352+0000","last_change":"2024-09-13T08:45:33.667868+0000","last_active":"2024-09-13T08:45:33.681352+0000","last_peered":"2024-09-13T08:45:33.681352+0000","last_clean":"2024-09-13T08:45:33.681352+0000","last_became_active":"2024-09-13T08:45:33.667543+0000","last_became_peered":"2024-09-13T08:45:33.667543+0000","last_unstale":"2024-09-13T08:45:33.681352+0000","last_undegraded":"2024-09-13T08:45:33.681352+0000","last_fullsized":"2024-09-13T08:45:33.681352+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:10.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691398+0000","last_change":"2024-09-13T08:45:33.689087+0000","last_active":"2024-09-13T08:45:33.691398+0000","last_peered":"2024-09-13T08:45:33.691398+0000","last_clean":"2024-09-13T08:45:33.691398+0000","last_became_active":"2024-09-13T08:45:33.688839+0000","last_became_peered":"2024-09-13T08:45:33.688839+0000","last_unstale":"2024-09-13T08:45:33.691398+0000","last_undegraded":"2024-09-13T08:45:33.691398+0000","last_fullsized":"2024-09-13T08:45:33.691398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:20:46.695925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.666207+0000","last_change":"2024-09-13T08:45:33.664987+0000","last_active":"2024-09-13T08:45:33.666207+0000","last_peered":"2024-09-13T08:45:33.666207+0000","last_clean":"2024-09-13T08:45:33.666207+0000","last_became_active":"2024-09-13T08:45:33.664707+0000","last_became_peered":"2024-09-13T08:45:33.664707+0000","last_unstale":"2024-09-13T08:45:33.666207+0000","last_undegraded":"2024-09-13T08:45:33.666207+0000","last_fullsized":"2024-09-13T08:45:33.666207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:03.021478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683740+0000","last_change":"2024-09-13T08:45:33.681174+0000","last_active":"2024-09-13T08:45:33.683740+0000","last_peered":"2024-09-13T08:45:33.683740+0000","last_clean":"2024-09-13T08:45:33.683740+0000","last_became_active":"2024-09-13T08:45:33.680680+0000","last_became_peered":"2024-09-13T08:45:33.680680+0000","last_unstale":"2024-09-13T08:45:33.683740+0000","last_undegraded":"2024-09-13T08:45:33.683740+0000","last_fullsized":"2024-09-13T08:45:33.683740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:46.928710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671724+0000","last_change":"2024-09-13T08:45:33.668789+0000","last_active":"2024-09-13T08:45:33.671724+0000","last_peered":"2024-09-13T08:45:33.671724+0000","last_clean":"2024-09-13T08:45:33.671724+0000","last_became_active":"2024-09-13T08:45:33.668462+0000","last_became_peered":"2024-09-13T08:45:33.668462+0000","last_unstale":"2024-09-13T08:45:33.671724+0000","last_undegraded":"2024-09-13T08:45:33.671724+0000","last_fullsized":"2024-09-13T08:45:33.671724+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:32:11.095202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.673865+0000","last_change":"2024-09-13T08:45:33.671105+0000","last_active":"2024-09-13T08:45:33.673865+0000","last_peered":"2024-09-13T08:45:33.673865+0000","last_clean":"2024-09-13T08:45:33.673865+0000","last_became_active":"2024-09-13T08:45:33.670718+0000","last_became_peered":"2024-09-13T08:45:33.670718+0000","last_unstale":"2024-09-13T08:45:33.673865+0000","last_undegraded":"2024-09-13T08:45:33.673865+0000","last_fullsized":"2024-09-13T08:45:33.673865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:34.021306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691685+0000","last_change":"2024-09-13T08:45:33.686654+0000","last_active":"2024-09-13T08:45:33.691685+0000","last_peered":"2024-09-13T08:45:33.691685+0000","last_clean":"2024-09-13T08:45:33.691685+0000","last_became_active":"2024-09-13T08:45:33.686375+0000","last_became_peered":"2024-09-13T08:45:33.686375+0000","last_unstale":"2024-09-13T08:45:33.691685+0000","last_undegraded":"2024-09-13T08:45:33.691685+0000","last_fullsized":"2024-09-13T08:45:33.691685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:03.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.672216+0000","last_change":"2024-09-13T08:45:33.670204+0000","last_active":"2024-09-13T08:45:33.672216+0000","last_peered":"2024-09-13T08:45:33.672216+0000","last_clean":"2024-09-13T08:45:33.672216+0000","last_became_active":"2024-09-13T08:45:33.669936+0000","last_became_peered":"2024-09-13T08:45:33.669936+0000","last_unstale":"2024-09-13T08:45:33.672216+0000","last_undegraded":"2024-09-13T08:45:33.672216+0000","last_fullsized":"2024-09-13T08:45:33.672216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:34.875650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667097+0000","last_change":"2024-09-13T08:45:33.663820+0000","last_active":"2024-09-13T08:45:33.667097+0000","last_peered":"2024-09-13T08:45:33.667097+0000","last_clean":"2024-09-13T08:45:33.667097+0000","last_became_active":"2024-09-13T08:45:33.663147+0000","last_became_peered":"2024-09-13T08:45:33.663147+0000","last_unstale":"2024-09-13T08:45:33.667097+0000","last_undegraded":"2024-09-13T08:45:33.667097+0000","last_fullsized":"2024-09-13T08:45:33.667097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:16:16.642146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.687335+0000","last_change":"2024-09-13T08:45:33.680218+0000","last_active":"2024-09-13T08:45:33.687335+0000","last_peered":"2024-09-13T08:45:33.687335+0000","last_clean":"2024-09-13T08:45:33.687335+0000","last_became_active":"2024-09-13T08:45:33.679938+0000","last_became_peered":"2024-09-13T08:45:33.679938+0000","last_unstale":"2024-09-13T08:45:33.687335+0000","last_undegraded":"2024-09-13T08:45:33.687335+0000","last_fullsized":"2024-09-13T08:45:33.687335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:05.467156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675520+0000","last_change":"2024-09-13T08:45:33.674349+0000","last_active":"2024-09-13T08:45:33.675520+0000","last_peered":"2024-09-13T08:45:33.675520+0000","last_clean":"2024-09-13T08:45:33.675520+0000","last_became_active":"2024-09-13T08:45:33.674004+0000","last_became_peered":"2024-09-13T08:45:33.674004+0000","last_unstale":"2024-09-13T08:45:33.675520+0000","last_undegraded":"2024-09-13T08:45:33.675520+0000","last_fullsized":"2024-09-13T08:45:33.675520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:06.280244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683524+0000","last_change":"2024-09-13T08:45:33.679260+0000","last_active":"2024-09-13T08:45:33.683524+0000","last_peered":"2024-09-13T08:45:33.683524+0000","last_clean":"2024-09-13T08:45:33.683524+0000","last_became_active":"2024-09-13T08:45:33.678976+0000","last_became_peered":"2024-09-13T08:45:33.678976+0000","last_unstale":"2024-09-13T08:45:33.683524+0000","last_undegraded":"2024-09-13T08:45:33.683524+0000","last_fullsized":"2024-09-13T08:45:33.683524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:41:27.288006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.673433+0000","last_change":"2024-09-13T08:45:33.666891+0000","last_active":"2024-09-13T08:45:33.673433+0000","last_peered":"2024-09-13T08:45:33.673433+0000","last_clean":"2024-09-13T08:45:33.673433+0000","last_became_active":"2024-09-13T08:45:33.666585+0000","last_became_peered":"2024-09-13T08:45:33.666585+0000","last_unstale":"2024-09-13T08:45:33.673433+0000","last_undegraded":"2024-09-13T08:45:33.673433+0000","last_fullsized":"2024-09-13T08:45:33.673433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:26:12.036228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.678998+0000","last_change":"2024-09-13T08:45:33.670178+0000","last_active":"2024-09-13T08:45:33.678998+0000","last_peered":"2024-09-13T08:45:33.678998+0000","last_clean":"2024-09-13T08:45:33.678998+0000","last_became_active":"2024-09-13T08:45:33.669687+0000","last_became_peered":"2024-09-13T08:45:33.669687+0000","last_unstale":"2024-09-13T08:45:33.678998+0000","last_undegraded":"2024-09-13T08:45:33.678998+0000","last_fullsized":"2024-09-13T08:45:33.678998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:11:12.735593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688146+0000","last_change":"2024-09-13T08:45:33.677052+0000","last_active":"2024-09-13T08:45:33.688146+0000","last_peered":"2024-09-13T08:45:33.688146+0000","last_clean":"2024-09-13T08:45:33.688146+0000","last_became_active":"2024-09-13T08:45:33.676767+0000","last_became_peered":"2024-09-13T08:45:33.676767+0000","last_unstale":"2024-09-13T08:45:33.688146+0000","last_undegraded":"2024-09-13T08:45:33.688146+0000","last_fullsized":"2024-09-13T08:45:33.688146+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:08.841526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.679899+0000","last_change":"2024-09-13T08:45:33.670240+0000","last_active":"2024-09-13T08:45:33.679899+0000","last_peered":"2024-09-13T08:45:33.679899+0000","last_clean":"2024-09-13T08:45:33.679899+0000","last_became_active":"2024-09-13T08:45:33.669952+0000","last_became_peered":"2024-09-13T08:45:33.669952+0000","last_unstale":"2024-09-13T08:45:33.679899+0000","last_undegraded":"2024-09-13T08:45:33.679899+0000","last_fullsized":"2024-09-13T08:45:33.679899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:15.070109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.692180+0000","last_change":"2024-09-13T08:45:33.690462+0000","last_active":"2024-09-13T08:45:33.692180+0000","last_peered":"2024-09-13T08:45:33.692180+0000","last_clean":"2024-09-13T08:45:33.692180+0000","last_became_active":"2024-09-13T08:45:33.690197+0000","last_became_peered":"2024-09-13T08:45:33.690197+0000","last_unstale":"2024-09-13T08:45:33.692180+0000","last_undegraded":"2024-09-13T08:45:33.692180+0000","last_fullsized":"2024-09-13T08:45:33.692180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:14.833887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.668085+0000","last_change":"2024-09-13T08:45:33.663806+0000","last_active":"2024-09-13T08:45:33.668085+0000","last_peered":"2024-09-13T08:45:33.668085+0000","last_clean":"2024-09-13T08:45:33.668085+0000","last_became_active":"2024-09-13T08:45:33.663498+0000","last_became_peered":"2024-09-13T08:45:33.663498+0000","last_unstale":"2024-09-13T08:45:33.668085+0000","last_undegraded":"2024-09-13T08:45:33.668085+0000","last_fullsized":"2024-09-13T08:45:33.668085+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:40:14.829023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683182+0000","last_change":"2024-09-13T08:45:33.674360+0000","last_active":"2024-09-13T08:45:33.683182+0000","last_peered":"2024-09-13T08:45:33.683182+0000","last_clean":"2024-09-13T08:45:33.683182+0000","last_became_active":"2024-09-13T08:45:33.673982+0000","last_became_peered":"2024-09-13T08:45:33.673982+0000","last_unstale":"2024-09-13T08:45:33.683182+0000","last_undegraded":"2024-09-13T08:45:33.683182+0000","last_fullsized":"2024-09-13T08:45:33.683182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:16:14.468554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689721+0000","last_change":"2024-09-13T08:45:33.689213+0000","last_active":"2024-09-13T08:45:33.689721+0000","last_peered":"2024-09-13T08:45:33.689721+0000","last_clean":"2024-09-13T08:45:33.689721+0000","last_became_active":"2024-09-13T08:45:33.688971+0000","last_became_peered":"2024-09-13T08:45:33.688971+0000","last_unstale":"2024-09-13T08:45:33.689721+0000","last_undegraded":"2024-09-13T08:45:33.689721+0000","last_fullsized":"2024-09-13T08:45:33.689721+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:48:29.114234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.687983+0000","last_change":"2024-09-13T08:45:33.672566+0000","last_active":"2024-09-13T08:45:33.687983+0000","last_peered":"2024-09-13T08:45:33.687983+0000","last_clean":"2024-09-13T08:45:33.687983+0000","last_became_active":"2024-09-13T08:45:33.672302+0000","last_became_peered":"2024-09-13T08:45:33.672302+0000","last_unstale":"2024-09-13T08:45:33.687983+0000","last_undegraded":"2024-09-13T08:45:33.687983+0000","last_fullsized":"2024-09-13T08:45:33.687983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:51:14.557452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684957+0000","last_change":"2024-09-13T08:45:33.675233+0000","last_active":"2024-09-13T08:45:33.684957+0000","last_peered":"2024-09-13T08:45:33.684957+0000","last_clean":"2024-09-13T08:45:33.684957+0000","last_became_active":"2024-09-13T08:45:33.674920+0000","last_became_peered":"2024-09-13T08:45:33.674920+0000","last_unstale":"2024-09-13T08:45:33.684957+0000","last_undegraded":"2024-09-13T08:45:33.684957+0000","last_fullsized":"2024-09-13T08:45:33.684957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:10.521583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683642+0000","last_change":"2024-09-13T08:45:33.679637+0000","last_active":"2024-09-13T08:45:33.683642+0000","last_peered":"2024-09-13T08:45:33.683642+0000","last_clean":"2024-09-13T08:45:33.683642+0000","last_became_active":"2024-09-13T08:45:33.679391+0000","last_became_peered":"2024-09-13T08:45:33.679391+0000","last_unstale":"2024-09-13T08:45:33.683642+0000","last_undegraded":"2024-09-13T08:45:33.683642+0000","last_fullsized":"2024-09-13T08:45:33.683642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:56.161329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691281+0000","last_change":"2024-09-13T08:45:33.688757+0000","last_active":"2024-09-13T08:45:33.691281+0000","last_peered":"2024-09-13T08:45:33.691281+0000","last_clean":"2024-09-13T08:45:33.691281+0000","last_became_active":"2024-09-13T08:45:33.687386+0000","last_became_peered":"2024-09-13T08:45:33.687386+0000","last_unstale":"2024-09-13T08:45:33.691281+0000","last_undegraded":"2024-09-13T08:45:33.691281+0000","last_fullsized":"2024-09-13T08:45:33.691281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:27.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"74'4","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689227+0000","last_change":"2024-09-13T08:45:32.657493+0000","last_active":"2024-09-13T08:45:33.689227+0000","last_peered":"2024-09-13T08:45:33.689227+0000","last_clean":"2024-09-13T08:45:33.689227+0000","last_became_active":"2024-09-13T08:45:32.656217+0000","last_became_peered":"2024-09-13T08:45:32.656217+0000","last_unstale":"2024-09-13T08:45:33.689227+0000","last_undegraded":"2024-09-13T08:45:33.689227+0000","last_fullsized":"2024-09-13T08:45:33.689227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:13.023500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683491+0000","last_change":"2024-09-13T08:45:33.678452+0000","last_active":"2024-09-13T08:45:33.683491+0000","last_peered":"2024-09-13T08:45:33.683491+0000","last_clean":"2024-09-13T08:45:33.683491+0000","last_became_active":"2024-09-13T08:45:33.678224+0000","last_became_peered":"2024-09-13T08:45:33.678224+0000","last_unstale":"2024-09-13T08:45:33.683491+0000","last_undegraded":"2024-09-13T08:45:33.683491+0000","last_fullsized":"2024-09-13T08:45:33.683491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:52:00.198742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684555+0000","last_change":"2024-09-13T08:45:32.654382+0000","last_active":"2024-09-13T08:45:33.684555+0000","last_peered":"2024-09-13T08:45:33.684555+0000","last_clean":"2024-09-13T08:45:33.684555+0000","last_became_active":"2024-09-13T08:45:32.654090+0000","last_became_peered":"2024-09-13T08:45:32.654090+0000","last_unstale":"2024-09-13T08:45:33.684555+0000","last_undegraded":"2024-09-13T08:45:33.684555+0000","last_fullsized":"2024-09-13T08:45:33.684555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:42.438703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.690499+0000","last_change":"2024-09-13T08:45:33.686795+0000","last_active":"2024-09-13T08:45:33.690499+0000","last_peered":"2024-09-13T08:45:33.690499+0000","last_clean":"2024-09-13T08:45:33.690499+0000","last_became_active":"2024-09-13T08:45:33.686497+0000","last_became_peered":"2024-09-13T08:45:33.686497+0000","last_unstale":"2024-09-13T08:45:33.690499+0000","last_undegraded":"2024-09-13T08:45:33.690499+0000","last_fullsized":"2024-09-13T08:45:33.690499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:27.050847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694008+0000","last_change":"2024-09-13T08:45:32.656780+0000","last_active":"2024-09-13T08:45:33.694008+0000","last_peered":"2024-09-13T08:45:33.694008+0000","last_clean":"2024-09-13T08:45:33.694008+0000","last_became_active":"2024-09-13T08:45:32.653941+0000","last_became_peered":"2024-09-13T08:45:32.653941+0000","last_unstale":"2024-09-13T08:45:33.694008+0000","last_undegraded":"2024-09-13T08:45:33.694008+0000","last_fullsized":"2024-09-13T08:45:33.694008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:12.824731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667910+0000","last_change":"2024-09-13T08:45:33.663007+0000","last_active":"2024-09-13T08:45:33.667910+0000","last_peered":"2024-09-13T08:45:33.667910+0000","last_clean":"2024-09-13T08:45:33.667910+0000","last_became_active":"2024-09-13T08:45:33.662591+0000","last_became_peered":"2024-09-13T08:45:33.662591+0000","last_unstale":"2024-09-13T08:45:33.667910+0000","last_undegraded":"2024-09-13T08:45:33.667910+0000","last_fullsized":"2024-09-13T08:45:33.667910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:59.290026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.690456+0000","last_change":"2024-09-13T08:45:32.655814+0000","last_active":"2024-09-13T08:45:33.690456+0000","last_peered":"2024-09-13T08:45:33.690456+0000","last_clean":"2024-09-13T08:45:33.690456+0000","last_became_active":"2024-09-13T08:45:32.655509+0000","last_became_peered":"2024-09-13T08:45:32.655509+0000","last_unstale":"2024-09-13T08:45:33.690456+0000","last_undegraded":"2024-09-13T08:45:33.690456+0000","last_fullsized":"2024-09-13T08:45:33.690456+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:56:39.873532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688008+0000","last_change":"2024-09-13T08:45:33.676002+0000","last_active":"2024-09-13T08:45:33.688008+0000","last_peered":"2024-09-13T08:45:33.688008+0000","last_clean":"2024-09-13T08:45:33.688008+0000","last_became_active":"2024-09-13T08:45:33.675711+0000","last_became_peered":"2024-09-13T08:45:33.675711+0000","last_unstale":"2024-09-13T08:45:33.688008+0000","last_undegraded":"2024-09-13T08:45:33.688008+0000","last_fullsized":"2024-09-13T08:45:33.688008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:34.875650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685399+0000","last_change":"2024-09-13T08:45:32.650357+0000","last_active":"2024-09-13T08:45:33.685399+0000","last_peered":"2024-09-13T08:45:33.685399+0000","last_clean":"2024-09-13T08:45:33.685399+0000","last_became_active":"2024-09-13T08:45:32.650092+0000","last_became_peered":"2024-09-13T08:45:32.650092+0000","last_unstale":"2024-09-13T08:45:33.685399+0000","last_undegraded":"2024-09-13T08:45:33.685399+0000","last_fullsized":"2024-09-13T08:45:33.685399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:11.131195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689533+0000","last_change":"2024-09-13T08:45:33.686668+0000","last_active":"2024-09-13T08:45:33.689533+0000","last_peered":"2024-09-13T08:45:33.689533+0000","last_clean":"2024-09-13T08:45:33.689533+0000","last_became_active":"2024-09-13T08:45:33.686340+0000","last_became_peered":"2024-09-13T08:45:33.686340+0000","last_unstale":"2024-09-13T08:45:33.689533+0000","last_undegraded":"2024-09-13T08:45:33.689533+0000","last_fullsized":"2024-09-13T08:45:33.689533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:45:11.714466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685672+0000","last_change":"2024-09-13T08:45:32.652970+0000","last_active":"2024-09-13T08:45:33.685672+0000","last_peered":"2024-09-13T08:45:33.685672+0000","last_clean":"2024-09-13T08:45:33.685672+0000","last_became_active":"2024-09-13T08:45:32.652708+0000","last_became_peered":"2024-09-13T08:45:32.652708+0000","last_unstale":"2024-09-13T08:45:33.685672+0000","last_undegraded":"2024-09-13T08:45:33.685672+0000","last_fullsized":"2024-09-13T08:45:33.685672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:26.448081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.644840+0000","last_change":"2024-09-13T08:45:32.644840+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.644840+0000","last_undegraded":"2024-09-13T08:45:32.644840+0000","last_fullsized":"2024-09-13T08:45:32.644840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T08:45:32.660378+0000","last_change":"2024-09-13T08:45:32.659746+0000","last_active":"2024-09-13T08:45:32.660378+0000","last_peered":"2024-09-13T08:45:32.660378+0000","last_clean":"2024-09-13T08:45:32.660378+0000","last_became_active":"2024-09-13T08:45:32.659314+0000","last_became_peered":"2024-09-13T08:45:32.659314+0000","last_unstale":"2024-09-13T08:45:32.660378+0000","last_undegraded":"2024-09-13T08:45:32.660378+0000","last_fullsized":"2024-09-13T08:45:32.660378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:08:43.439003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671291+0000","last_change":"2024-09-13T08:45:33.668090+0000","last_active":"2024-09-13T08:45:33.671291+0000","last_peered":"2024-09-13T08:45:33.671291+0000","last_clean":"2024-09-13T08:45:33.671291+0000","last_became_active":"2024-09-13T08:45:33.667802+0000","last_became_peered":"2024-09-13T08:45:33.667802+0000","last_unstale":"2024-09-13T08:45:33.671291+0000","last_undegraded":"2024-09-13T08:45:33.671291+0000","last_fullsized":"2024-09-13T08:45:33.671291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:02:31.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660673+0000","last_change":"2024-09-13T08:45:32.654860+0000","last_active":"2024-09-13T08:45:33.660673+0000","last_peered":"2024-09-13T08:45:33.660673+0000","last_clean":"2024-09-13T08:45:33.660673+0000","last_became_active":"2024-09-13T08:45:32.652510+0000","last_became_peered":"2024-09-13T08:45:32.652510+0000","last_unstale":"2024-09-13T08:45:33.660673+0000","last_undegraded":"2024-09-13T08:45:33.660673+0000","last_fullsized":"2024-09-13T08:45:33.660673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:16.016602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681459+0000","last_change":"2024-09-13T08:45:33.679316+0000","last_active":"2024-09-13T08:45:33.681459+0000","last_peered":"2024-09-13T08:45:33.681459+0000","last_clean":"2024-09-13T08:45:33.681459+0000","last_became_active":"2024-09-13T08:45:33.678605+0000","last_became_peered":"2024-09-13T08:45:33.678605+0000","last_unstale":"2024-09-13T08:45:33.681459+0000","last_undegraded":"2024-09-13T08:45:33.681459+0000","last_fullsized":"2024-09-13T08:45:33.681459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:01:38.403717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685240+0000","last_change":"2024-09-13T08:45:32.655976+0000","last_active":"2024-09-13T08:45:33.685240+0000","last_peered":"2024-09-13T08:45:33.685240+0000","last_clean":"2024-09-13T08:45:33.685240+0000","last_became_active":"2024-09-13T08:45:32.654678+0000","last_became_peered":"2024-09-13T08:45:32.654678+0000","last_unstale":"2024-09-13T08:45:33.685240+0000","last_undegraded":"2024-09-13T08:45:33.685240+0000","last_fullsized":"2024-09-13T08:45:33.685240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:07.445463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684924+0000","last_change":"2024-09-13T08:45:33.682397+0000","last_active":"2024-09-13T08:45:33.684924+0000","last_peered":"2024-09-13T08:45:33.684924+0000","last_clean":"2024-09-13T08:45:33.684924+0000","last_became_active":"2024-09-13T08:45:33.682010+0000","last_became_peered":"2024-09-13T08:45:33.682010+0000","last_unstale":"2024-09-13T08:45:33.684924+0000","last_undegraded":"2024-09-13T08:45:33.684924+0000","last_fullsized":"2024-09-13T08:45:33.684924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:39.373555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.670069+0000","last_change":"2024-09-13T08:45:32.654977+0000","last_active":"2024-09-13T08:45:33.670069+0000","last_peered":"2024-09-13T08:45:33.670069+0000","last_clean":"2024-09-13T08:45:33.670069+0000","last_became_active":"2024-09-13T08:45:32.654656+0000","last_became_peered":"2024-09-13T08:45:32.654656+0000","last_unstale":"2024-09-13T08:45:33.670069+0000","last_undegraded":"2024-09-13T08:45:33.670069+0000","last_fullsized":"2024-09-13T08:45:33.670069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:09.832815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689499+0000","last_change":"2024-09-13T08:45:33.682432+0000","last_active":"2024-09-13T08:45:33.689499+0000","last_peered":"2024-09-13T08:45:33.689499+0000","last_clean":"2024-09-13T08:45:33.689499+0000","last_became_active":"2024-09-13T08:45:33.681551+0000","last_became_peered":"2024-09-13T08:45:33.681551+0000","last_unstale":"2024-09-13T08:45:33.689499+0000","last_undegraded":"2024-09-13T08:45:33.689499+0000","last_fullsized":"2024-09-13T08:45:33.689499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:06.280244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"74'3","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682838+0000","last_change":"2024-09-13T08:45:32.644983+0000","last_active":"2024-09-13T08:45:33.682838+0000","last_peered":"2024-09-13T08:45:33.682838+0000","last_clean":"2024-09-13T08:45:33.682838+0000","last_became_active":"2024-09-13T08:45:32.644649+0000","last_became_peered":"2024-09-13T08:45:32.644649+0000","last_unstale":"2024-09-13T08:45:33.682838+0000","last_undegraded":"2024-09-13T08:45:33.682838+0000","last_fullsized":"2024-09-13T08:45:33.682838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:16:03.736952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.669128+0000","last_change":"2024-09-13T08:45:33.663848+0000","last_active":"2024-09-13T08:45:33.669128+0000","last_peered":"2024-09-13T08:45:33.669128+0000","last_clean":"2024-09-13T08:45:33.669128+0000","last_became_active":"2024-09-13T08:45:33.663559+0000","last_became_peered":"2024-09-13T08:45:33.663559+0000","last_unstale":"2024-09-13T08:45:33.669128+0000","last_undegraded":"2024-09-13T08:45:33.669128+0000","last_fullsized":"2024-09-13T08:45:33.669128+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:51.161859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T08:45:32.657443+0000","last_change":"2024-09-13T08:45:32.656020+0000","last_active":"2024-09-13T08:45:32.657443+0000","last_peered":"2024-09-13T08:45:32.657443+0000","last_clean":"2024-09-13T08:45:32.657443+0000","last_became_active":"2024-09-13T08:45:32.655215+0000","last_became_peered":"2024-09-13T08:45:32.655215+0000","last_unstale":"2024-09-13T08:45:32.657443+0000","last_undegraded":"2024-09-13T08:45:32.657443+0000","last_fullsized":"2024-09-13T08:45:32.657443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:44.310136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.677142+0000","last_change":"2024-09-13T08:45:33.671033+0000","last_active":"2024-09-13T08:45:33.677142+0000","last_peered":"2024-09-13T08:45:33.677142+0000","last_clean":"2024-09-13T08:45:33.677142+0000","last_became_active":"2024-09-13T08:45:33.670779+0000","last_became_peered":"2024-09-13T08:45:33.670779+0000","last_unstale":"2024-09-13T08:45:33.677142+0000","last_undegraded":"2024-09-13T08:45:33.677142+0000","last_fullsized":"2024-09-13T08:45:33.677142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:47:52.333905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T08:45:32.657464+0000","last_change":"2024-09-13T08:45:32.656334+0000","last_active":"2024-09-13T08:45:32.657464+0000","last_peered":"2024-09-13T08:45:32.657464+0000","last_clean":"2024-09-13T08:45:32.657464+0000","last_became_active":"2024-09-13T08:45:32.656007+0000","last_became_peered":"2024-09-13T08:45:32.656007+0000","last_unstale":"2024-09-13T08:45:32.657464+0000","last_undegraded":"2024-09-13T08:45:32.657464+0000","last_fullsized":"2024-09-13T08:45:32.657464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:45.983114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682359+0000","last_change":"2024-09-13T08:45:33.681389+0000","last_active":"2024-09-13T08:45:33.682359+0000","last_peered":"2024-09-13T08:45:33.682359+0000","last_clean":"2024-09-13T08:45:33.682359+0000","last_became_active":"2024-09-13T08:45:33.681085+0000","last_became_peered":"2024-09-13T08:45:33.681085+0000","last_unstale":"2024-09-13T08:45:33.682359+0000","last_undegraded":"2024-09-13T08:45:33.682359+0000","last_fullsized":"2024-09-13T08:45:33.682359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:38.549474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.652883+0000","last_change":"2024-09-13T08:45:32.641547+0000","last_active":"2024-09-13T08:45:33.652883+0000","last_peered":"2024-09-13T08:45:33.652883+0000","last_clean":"2024-09-13T08:45:33.652883+0000","last_became_active":"2024-09-13T08:45:32.641221+0000","last_became_peered":"2024-09-13T08:45:32.641221+0000","last_unstale":"2024-09-13T08:45:33.652883+0000","last_undegraded":"2024-09-13T08:45:33.652883+0000","last_fullsized":"2024-09-13T08:45:33.652883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:37.612881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671771+0000","last_change":"2024-09-13T08:45:33.668342+0000","last_active":"2024-09-13T08:45:33.671771+0000","last_peered":"2024-09-13T08:45:33.671771+0000","last_clean":"2024-09-13T08:45:33.671771+0000","last_became_active":"2024-09-13T08:45:33.668068+0000","last_became_peered":"2024-09-13T08:45:33.668068+0000","last_unstale":"2024-09-13T08:45:33.671771+0000","last_undegraded":"2024-09-13T08:45:33.671771+0000","last_fullsized":"2024-09-13T08:45:33.671771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:51:14.557452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.653649+0000","last_change":"2024-09-13T08:45:32.645856+0000","last_active":"2024-09-13T08:45:33.653649+0000","last_peered":"2024-09-13T08:45:33.653649+0000","last_clean":"2024-09-13T08:45:33.653649+0000","last_became_active":"2024-09-13T08:45:32.645551+0000","last_became_peered":"2024-09-13T08:45:32.645551+0000","last_unstale":"2024-09-13T08:45:33.653649+0000","last_undegraded":"2024-09-13T08:45:33.653649+0000","last_fullsized":"2024-09-13T08:45:33.653649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:13.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682542+0000","last_change":"2024-09-13T08:45:33.679240+0000","last_active":"2024-09-13T08:45:33.682542+0000","last_peered":"2024-09-13T08:45:33.682542+0000","last_clean":"2024-09-13T08:45:33.682542+0000","last_became_active":"2024-09-13T08:45:33.678945+0000","last_became_peered":"2024-09-13T08:45:33.678945+0000","last_unstale":"2024-09-13T08:45:33.682542+0000","last_undegraded":"2024-09-13T08:45:33.682542+0000","last_fullsized":"2024-09-13T08:45:33.682542+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:18.936719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"74'8","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691071+0000","last_change":"2024-09-13T08:45:32.644799+0000","last_active":"2024-09-13T08:45:33.691071+0000","last_peered":"2024-09-13T08:45:33.691071+0000","last_clean":"2024-09-13T08:45:33.691071+0000","last_became_active":"2024-09-13T08:45:32.644559+0000","last_became_peered":"2024-09-13T08:45:32.644559+0000","last_unstale":"2024-09-13T08:45:33.691071+0000","last_undegraded":"2024-09-13T08:45:33.691071+0000","last_fullsized":"2024-09-13T08:45:33.691071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:23.392065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680801+0000","last_change":"2024-09-13T08:45:33.676706+0000","last_active":"2024-09-13T08:45:33.680801+0000","last_peered":"2024-09-13T08:45:33.680801+0000","last_clean":"2024-09-13T08:45:33.680801+0000","last_became_active":"2024-09-13T08:45:33.676442+0000","last_became_peered":"2024-09-13T08:45:33.676442+0000","last_unstale":"2024-09-13T08:45:33.680801+0000","last_undegraded":"2024-09-13T08:45:33.680801+0000","last_fullsized":"2024-09-13T08:45:33.680801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:00.210640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694996+0000","last_change":"2024-09-13T08:45:32.655505+0000","last_active":"2024-09-13T08:45:33.694996+0000","last_peered":"2024-09-13T08:45:33.694996+0000","last_clean":"2024-09-13T08:45:33.694996+0000","last_became_active":"2024-09-13T08:45:32.655218+0000","last_became_peered":"2024-09-13T08:45:32.655218+0000","last_unstale":"2024-09-13T08:45:33.694996+0000","last_undegraded":"2024-09-13T08:45:33.694996+0000","last_fullsized":"2024-09-13T08:45:33.694996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:04.883497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680842+0000","last_change":"2024-09-13T08:45:33.675626+0000","last_active":"2024-09-13T08:45:33.680842+0000","last_peered":"2024-09-13T08:45:33.680842+0000","last_clean":"2024-09-13T08:45:33.680842+0000","last_became_active":"2024-09-13T08:45:33.675346+0000","last_became_peered":"2024-09-13T08:45:33.675346+0000","last_unstale":"2024-09-13T08:45:33.680842+0000","last_undegraded":"2024-09-13T08:45:33.680842+0000","last_fullsized":"2024-09-13T08:45:33.680842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:22.946990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685223+0000","last_change":"2024-09-13T08:45:32.658779+0000","last_active":"2024-09-13T08:45:33.685223+0000","last_peered":"2024-09-13T08:45:33.685223+0000","last_clean":"2024-09-13T08:45:33.685223+0000","last_became_active":"2024-09-13T08:45:32.658420+0000","last_became_peered":"2024-09-13T08:45:32.658420+0000","last_unstale":"2024-09-13T08:45:33.685223+0000","last_undegraded":"2024-09-13T08:45:33.685223+0000","last_fullsized":"2024-09-13T08:45:33.685223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:18.130462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680052+0000","last_change":"2024-09-13T08:45:33.675553+0000","last_active":"2024-09-13T08:45:33.680052+0000","last_peered":"2024-09-13T08:45:33.680052+0000","last_clean":"2024-09-13T08:45:33.680052+0000","last_became_active":"2024-09-13T08:45:33.675233+0000","last_became_peered":"2024-09-13T08:45:33.675233+0000","last_unstale":"2024-09-13T08:45:33.680052+0000","last_undegraded":"2024-09-13T08:45:33.680052+0000","last_fullsized":"2024-09-13T08:45:33.680052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:41.676077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.692093+0000","last_change":"2024-09-13T08:45:32.659506+0000","last_active":"2024-09-13T08:45:33.692093+0000","last_peered":"2024-09-13T08:45:33.692093+0000","last_clean":"2024-09-13T08:45:33.692093+0000","last_became_active":"2024-09-13T08:45:32.659220+0000","last_became_peered":"2024-09-13T08:45:32.659220+0000","last_unstale":"2024-09-13T08:45:33.692093+0000","last_undegraded":"2024-09-13T08:45:33.692093+0000","last_fullsized":"2024-09-13T08:45:33.692093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:30.318592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689473+0000","last_change":"2024-09-13T08:45:33.688805+0000","last_active":"2024-09-13T08:45:33.689473+0000","last_peered":"2024-09-13T08:45:33.689473+0000","last_clean":"2024-09-13T08:45:33.689473+0000","last_became_active":"2024-09-13T08:45:33.688534+0000","last_became_peered":"2024-09-13T08:45:33.688534+0000","last_unstale":"2024-09-13T08:45:33.689473+0000","last_undegraded":"2024-09-13T08:45:33.689473+0000","last_fullsized":"2024-09-13T08:45:33.689473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:11.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660382+0000","last_change":"2024-09-13T08:45:32.655084+0000","last_active":"2024-09-13T08:45:33.660382+0000","last_peered":"2024-09-13T08:45:33.660382+0000","last_clean":"2024-09-13T08:45:33.660382+0000","last_became_active":"2024-09-13T08:45:32.654795+0000","last_became_peered":"2024-09-13T08:45:32.654795+0000","last_unstale":"2024-09-13T08:45:33.660382+0000","last_undegraded":"2024-09-13T08:45:33.660382+0000","last_fullsized":"2024-09-13T08:45:33.660382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:23:00.745264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.641979+0000","last_change":"2024-09-13T08:45:32.641979+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.641979+0000","last_undegraded":"2024-09-13T08:45:32.641979+0000","last_fullsized":"2024-09-13T08:45:32.641979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.661857+0000","last_change":"2024-09-13T08:45:32.648855+0000","last_active":"2024-09-13T08:45:33.661857+0000","last_peered":"2024-09-13T08:45:33.661857+0000","last_clean":"2024-09-13T08:45:33.661857+0000","last_became_active":"2024-09-13T08:45:32.648572+0000","last_became_peered":"2024-09-13T08:45:32.648572+0000","last_unstale":"2024-09-13T08:45:33.661857+0000","last_undegraded":"2024-09-13T08:45:33.661857+0000","last_fullsized":"2024-09-13T08:45:33.661857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:11.383879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.644401+0000","last_change":"2024-09-13T08:45:32.644401+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.644401+0000","last_undegraded":"2024-09-13T08:45:32.644401+0000","last_fullsized":"2024-09-13T08:45:32.644401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.664419+0000","last_change":"2024-09-13T08:45:32.651171+0000","last_active":"2024-09-13T08:45:33.664419+0000","last_peered":"2024-09-13T08:45:33.664419+0000","last_clean":"2024-09-13T08:45:33.664419+0000","last_became_active":"2024-09-13T08:45:32.650437+0000","last_became_peered":"2024-09-13T08:45:32.650437+0000","last_unstale":"2024-09-13T08:45:33.664419+0000","last_undegraded":"2024-09-13T08:45:33.664419+0000","last_fullsized":"2024-09-13T08:45:33.664419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:43.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671316+0000","last_change":"2024-09-13T08:45:33.670268+0000","last_active":"2024-09-13T08:45:33.671316+0000","last_peered":"2024-09-13T08:45:33.671316+0000","last_clean":"2024-09-13T08:45:33.671316+0000","last_became_active":"2024-09-13T08:45:33.669974+0000","last_became_peered":"2024-09-13T08:45:33.669974+0000","last_unstale":"2024-09-13T08:45:33.671316+0000","last_undegraded":"2024-09-13T08:45:33.671316+0000","last_fullsized":"2024-09-13T08:45:33.671316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:14.031957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.657855+0000","last_change":"2024-09-13T08:45:32.652196+0000","last_active":"2024-09-13T08:45:33.657855+0000","last_peered":"2024-09-13T08:45:33.657855+0000","last_clean":"2024-09-13T08:45:33.657855+0000","last_became_active":"2024-09-13T08:45:32.651851+0000","last_became_peered":"2024-09-13T08:45:32.651851+0000","last_unstale":"2024-09-13T08:45:33.657855+0000","last_undegraded":"2024-09-13T08:45:33.657855+0000","last_fullsized":"2024-09-13T08:45:33.657855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:48:29.415637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.668002+0000","last_change":"2024-09-13T08:45:33.664149+0000","last_active":"2024-09-13T08:45:33.668002+0000","last_peered":"2024-09-13T08:45:33.668002+0000","last_clean":"2024-09-13T08:45:33.668002+0000","last_became_active":"2024-09-13T08:45:33.663531+0000","last_became_peered":"2024-09-13T08:45:33.663531+0000","last_unstale":"2024-09-13T08:45:33.668002+0000","last_undegraded":"2024-09-13T08:45:33.668002+0000","last_fullsized":"2024-09-13T08:45:33.668002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:18:36.348488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694733+0000","last_change":"2024-09-13T08:45:32.660012+0000","last_active":"2024-09-13T08:45:33.694733+0000","last_peered":"2024-09-13T08:45:33.694733+0000","last_clean":"2024-09-13T08:45:33.694733+0000","last_became_active":"2024-09-13T08:45:32.659398+0000","last_became_peered":"2024-09-13T08:45:32.659398+0000","last_unstale":"2024-09-13T08:45:33.694733+0000","last_undegraded":"2024-09-13T08:45:33.694733+0000","last_fullsized":"2024-09-13T08:45:33.694733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:12.195364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667819+0000","last_change":"2024-09-13T08:45:33.660680+0000","last_active":"2024-09-13T08:45:33.667819+0000","last_peered":"2024-09-13T08:45:33.667819+0000","last_clean":"2024-09-13T08:45:33.667819+0000","last_became_active":"2024-09-13T08:45:33.660407+0000","last_became_peered":"2024-09-13T08:45:33.660407+0000","last_unstale":"2024-09-13T08:45:33.667819+0000","last_undegraded":"2024-09-13T08:45:33.667819+0000","last_fullsized":"2024-09-13T08:45:33.667819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:21:26.352682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684746+0000","last_change":"2024-09-13T08:45:32.661277+0000","last_active":"2024-09-13T08:45:33.684746+0000","last_peered":"2024-09-13T08:45:33.684746+0000","last_clean":"2024-09-13T08:45:33.684746+0000","last_became_active":"2024-09-13T08:45:32.660211+0000","last_became_peered":"2024-09-13T08:45:32.660211+0000","last_unstale":"2024-09-13T08:45:33.684746+0000","last_undegraded":"2024-09-13T08:45:33.684746+0000","last_fullsized":"2024-09-13T08:45:33.684746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:01.561248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684943+0000","last_change":"2024-09-13T08:45:33.683345+0000","last_active":"2024-09-13T08:45:33.684943+0000","last_peered":"2024-09-13T08:45:33.684943+0000","last_clean":"2024-09-13T08:45:33.684943+0000","last_became_active":"2024-09-13T08:45:33.682992+0000","last_became_peered":"2024-09-13T08:45:33.682992+0000","last_unstale":"2024-09-13T08:45:33.684943+0000","last_undegraded":"2024-09-13T08:45:33.684943+0000","last_fullsized":"2024-09-13T08:45:33.684943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:47.803484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660890+0000","last_change":"2024-09-13T08:45:32.646650+0000","last_active":"2024-09-13T08:45:33.660890+0000","last_peered":"2024-09-13T08:45:33.660890+0000","last_clean":"2024-09-13T08:45:33.660890+0000","last_became_active":"2024-09-13T08:45:32.646362+0000","last_became_peered":"2024-09-13T08:45:32.646362+0000","last_unstale":"2024-09-13T08:45:33.660890+0000","last_undegraded":"2024-09-13T08:45:33.660890+0000","last_fullsized":"2024-09-13T08:45:33.660890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:48:47.580467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680628+0000","last_change":"2024-09-13T08:45:33.678066+0000","last_active":"2024-09-13T08:45:33.680628+0000","last_peered":"2024-09-13T08:45:33.680628+0000","last_clean":"2024-09-13T08:45:33.680628+0000","last_became_active":"2024-09-13T08:45:33.674778+0000","last_became_peered":"2024-09-13T08:45:33.674778+0000","last_unstale":"2024-09-13T08:45:33.680628+0000","last_undegraded":"2024-09-13T08:45:33.680628+0000","last_fullsized":"2024-09-13T08:45:33.680628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:56.987435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.661122+0000","last_change":"2024-09-13T08:45:32.646318+0000","last_active":"2024-09-13T08:45:33.661122+0000","last_peered":"2024-09-13T08:45:33.661122+0000","last_clean":"2024-09-13T08:45:33.661122+0000","last_became_active":"2024-09-13T08:45:32.645801+0000","last_became_peered":"2024-09-13T08:45:32.645801+0000","last_unstale":"2024-09-13T08:45:33.661122+0000","last_undegraded":"2024-09-13T08:45:33.661122+0000","last_fullsized":"2024-09-13T08:45:33.661122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:40.323104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.672925+0000","last_change":"2024-09-13T08:45:33.670611+0000","last_active":"2024-09-13T08:45:33.672925+0000","last_peered":"2024-09-13T08:45:33.672925+0000","last_clean":"2024-09-13T08:45:33.672925+0000","last_became_active":"2024-09-13T08:45:33.670334+0000","last_became_peered":"2024-09-13T08:45:33.670334+0000","last_unstale":"2024-09-13T08:45:33.672925+0000","last_undegraded":"2024-09-13T08:45:33.672925+0000","last_fullsized":"2024-09-13T08:45:33.672925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:08.841526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.662228+0000","last_change":"2024-09-13T08:45:32.650061+0000","last_active":"2024-09-13T08:45:33.662228+0000","last_peered":"2024-09-13T08:45:33.662228+0000","last_clean":"2024-09-13T08:45:33.662228+0000","last_became_active":"2024-09-13T08:45:32.649743+0000","last_became_peered":"2024-09-13T08:45:32.649743+0000","last_unstale":"2024-09-13T08:45:33.662228+0000","last_undegraded":"2024-09-13T08:45:33.662228+0000","last_fullsized":"2024-09-13T08:45:33.662228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:16.817675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691205+0000","last_change":"2024-09-13T08:45:33.682926+0000","last_active":"2024-09-13T08:45:33.691205+0000","last_peered":"2024-09-13T08:45:33.691205+0000","last_clean":"2024-09-13T08:45:33.691205+0000","last_became_active":"2024-09-13T08:45:33.681036+0000","last_became_peered":"2024-09-13T08:45:33.681036+0000","last_unstale":"2024-09-13T08:45:33.691205+0000","last_undegraded":"2024-09-13T08:45:33.691205+0000","last_fullsized":"2024-09-13T08:45:33.691205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:31.247460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.655978+0000","last_change":"2024-09-13T08:45:32.651908+0000","last_active":"2024-09-13T08:45:33.655978+0000","last_peered":"2024-09-13T08:45:33.655978+0000","last_clean":"2024-09-13T08:45:33.655978+0000","last_became_active":"2024-09-13T08:45:32.651622+0000","last_became_peered":"2024-09-13T08:45:32.651622+0000","last_unstale":"2024-09-13T08:45:33.655978+0000","last_undegraded":"2024-09-13T08:45:33.655978+0000","last_fullsized":"2024-09-13T08:45:33.655978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:32.504688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-13T08:45:32.642635+0000","last_change":"2024-09-13T08:45:32.642635+0000","last_active":"2024-09-13T08:45:32.627543+0000","last_peered":"2024-09-13T08:45:32.627543+0000","last_clean":"2024-09-13T08:45:32.627543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:45:32.642635+0000","last_undegraded":"2024-09-13T08:45:32.642635+0000","last_fullsized":"2024-09-13T08:45:32.642635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.664624+0000","last_change":"2024-09-13T08:45:32.656890+0000","last_active":"2024-09-13T08:45:33.664624+0000","last_peered":"2024-09-13T08:45:33.664624+0000","last_clean":"2024-09-13T08:45:33.664624+0000","last_became_active":"2024-09-13T08:45:32.656592+0000","last_became_peered":"2024-09-13T08:45:32.656592+0000","last_unstale":"2024-09-13T08:45:33.664624+0000","last_undegraded":"2024-09-13T08:45:33.664624+0000","last_fullsized":"2024-09-13T08:45:33.664624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:38.811210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675107+0000","last_change":"2024-09-13T08:45:33.669976+0000","last_active":"2024-09-13T08:45:33.675107+0000","last_peered":"2024-09-13T08:45:33.675107+0000","last_clean":"2024-09-13T08:45:33.675107+0000","last_became_active":"2024-09-13T08:45:33.669693+0000","last_became_peered":"2024-09-13T08:45:33.669693+0000","last_unstale":"2024-09-13T08:45:33.675107+0000","last_undegraded":"2024-09-13T08:45:33.675107+0000","last_fullsized":"2024-09-13T08:45:33.675107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:27:36.696332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.658907+0000","last_change":"2024-09-13T08:45:32.654218+0000","last_active":"2024-09-13T08:45:33.658907+0000","last_peered":"2024-09-13T08:45:33.658907+0000","last_clean":"2024-09-13T08:45:33.658907+0000","last_became_active":"2024-09-13T08:45:32.653519+0000","last_became_peered":"2024-09-13T08:45:32.653519+0000","last_unstale":"2024-09-13T08:45:33.658907+0000","last_undegraded":"2024-09-13T08:45:33.658907+0000","last_fullsized":"2024-09-13T08:45:33.658907+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:33.012849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681797+0000","last_change":"2024-09-13T08:45:33.679432+0000","last_active":"2024-09-13T08:45:33.681797+0000","last_peered":"2024-09-13T08:45:33.681797+0000","last_clean":"2024-09-13T08:45:33.681797+0000","last_became_active":"2024-09-13T08:45:33.679129+0000","last_became_peered":"2024-09-13T08:45:33.679129+0000","last_unstale":"2024-09-13T08:45:33.681797+0000","last_undegraded":"2024-09-13T08:45:33.681797+0000","last_fullsized":"2024-09-13T08:45:33.681797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:31.247460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681998+0000","last_change":"2024-09-13T08:45:32.647002+0000","last_active":"2024-09-13T08:45:33.681998+0000","last_peered":"2024-09-13T08:45:33.681998+0000","last_clean":"2024-09-13T08:45:33.681998+0000","last_became_active":"2024-09-13T08:45:32.645456+0000","last_became_peered":"2024-09-13T08:45:32.645456+0000","last_unstale":"2024-09-13T08:45:33.681998+0000","last_undegraded":"2024-09-13T08:45:33.681998+0000","last_fullsized":"2024-09-13T08:45:33.681998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:41.793310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":212,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.652555+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:45:33.652555+0000","last_peered":"2024-09-13T08:45:33.652555+0000","last_clean":"2024-09-13T08:45:33.652555+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:45:33.652555+0000","last_undegraded":"2024-09-13T08:45:33.652555+0000","last_fullsized":"2024-09-13T08:45:33.652555+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509540,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50120,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321720,"statfs":{"total":96636764160,"available":96585441280,"internally_reserved":0,"allocated":905216,"data_stored":380942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50696,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321144,"statfs":{"total":96636764160,"available":96584851456,"internally_reserved":0,"allocated":1490944,"data_stored":969788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509541,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54288,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317552,"statfs":{"total":96636764160,"available":96581173248,"internally_reserved":0,"allocated":905216,"data_stored":380024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":880640,"data_stored":372499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":897024,"data_stored":379912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509542,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46532,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325308,"statfs":{"total":96636764160,"available":96589115392,"internally_reserved":0,"allocated":1486848,"data_stored":969698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":897024,"data_stored":379400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542246,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":880640,"data_stored":372499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:45:38.631 INFO:tasks.ceph.ceph_manager.ceph:PG 11.18 is not active+clean 2024-09-13T08:45:38.631 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:45:32.644840+0000', 'last_change': '2024-09-13T08:45:32.644840+0000', 'last_active': '2024-09-13T08:45:32.627543+0000', 'last_peered': '2024-09-13T08:45:32.627543+0000', 'last_clean': '2024-09-13T08:45:32.627543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:45:32.644840+0000', 'last_undegraded': '2024-09-13T08:45:32.644840+0000', 'last_fullsized': '2024-09-13T08:45:32.644840+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'last_clean_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-13T08:45:38.631 INFO:tasks.ceph.ceph_manager.ceph:PG 11.b is not active+clean 2024-09-13T08:45:38.631 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:45:32.641979+0000', 'last_change': '2024-09-13T08:45:32.641979+0000', 'last_active': '2024-09-13T08:45:32.627543+0000', 'last_peered': '2024-09-13T08:45:32.627543+0000', 'last_clean': '2024-09-13T08:45:32.627543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:45:32.641979+0000', 'last_undegraded': '2024-09-13T08:45:32.641979+0000', 'last_fullsized': '2024-09-13T08:45:32.641979+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'last_clean_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-13T08:45:38.631 INFO:tasks.ceph.ceph_manager.ceph:PG 11.8 is not active+clean 2024-09-13T08:45:38.631 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:45:32.644401+0000', 'last_change': '2024-09-13T08:45:32.644401+0000', 'last_active': '2024-09-13T08:45:32.627543+0000', 'last_peered': '2024-09-13T08:45:32.627543+0000', 'last_clean': '2024-09-13T08:45:32.627543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:45:32.644401+0000', 'last_undegraded': '2024-09-13T08:45:32.644401+0000', 'last_fullsized': '2024-09-13T08:45:32.644401+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'last_clean_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-13T08:45:38.631 INFO:tasks.ceph.ceph_manager.ceph:PG 11.0 is not active+clean 2024-09-13T08:45:38.631 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:45:32.642635+0000', 'last_change': '2024-09-13T08:45:32.642635+0000', 'last_active': '2024-09-13T08:45:32.627543+0000', 'last_peered': '2024-09-13T08:45:32.627543+0000', 'last_clean': '2024-09-13T08:45:32.627543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:45:32.642635+0000', 'last_undegraded': '2024-09-13T08:45:32.642635+0000', 'last_fullsized': '2024-09-13T08:45:32.642635+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'last_clean_scrub_stamp': '2024-09-13T08:45:32.627543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-13T08:45:38.632 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T08:45:38.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.813+0000 7fd673800640 1 -- 172.21.15.44:0/1765528835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd66c0ccfa0 msgr2=0x7fd66c0cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.813+0000 7fd673800640 1 --2- 172.21.15.44:0/1765528835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd66c0ccfa0 0x7fd66c0cd3a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0af510 tx=0x7fd66401ccc0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.813+0000 7fd673800640 1 -- 172.21.15.44:0/1765528835 shutdown_connections 2024-09-13T08:45:38.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.813+0000 7fd673800640 1 --2- 172.21.15.44:0/1765528835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd66c0ccfa0 0x7fd66c0cd3a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.813+0000 7fd673800640 1 --2- 172.21.15.44:0/1765528835 >> v2:172.21.15.129:3300/0 conn(0x7fd66c15e270 0x7fd66c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.813+0000 7fd673800640 1 -- 172.21.15.44:0/1765528835 >> 172.21.15.44:0/1765528835 conn(0x7fd66c06d220 msgr2=0x7fd66c0b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:38.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.813+0000 7fd673800640 1 -- 172.21.15.44:0/1765528835 shutdown_connections 2024-09-13T08:45:38.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.813+0000 7fd673800640 1 -- 172.21.15.44:0/1765528835 wait complete. 2024-09-13T08:45:38.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.813+0000 7fd673800640 1 Processor -- start 2024-09-13T08:45:38.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd673800640 1 -- start start 2024-09-13T08:45:38.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd673800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd66c0ccfa0 0x7fd66c151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd673800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd66c15de90 0x7fd66c1fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd673800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd66c15e270 0x7fd66c155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd673800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c16ae70 con 0x7fd66c0ccfa0 2024-09-13T08:45:38.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd673800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c16acf0 con 0x7fd66c15e270 2024-09-13T08:45:38.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd673800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd66c16aff0 con 0x7fd66c15de90 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd670e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd66c0ccfa0 0x7fd66c151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd670e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd66c0ccfa0 0x7fd66c151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38052/0 (socket says 172.21.15.44:38052) 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd670e00640 1 -- 172.21.15.44:0/1510939457 learned_addr learned my addr 172.21.15.44:0/1510939457 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd671800640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd66c15e270 0x7fd66c155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd66be00640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd66c15de90 0x7fd66c1fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd670e00640 1 -- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd66c15de90 msgr2=0x7fd66c1fafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd670e00640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd66c15de90 0x7fd66c1fafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd670e00640 1 -- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd66c15e270 msgr2=0x7fd66c155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd670e00640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd66c15e270 0x7fd66c155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd670e00640 1 -- 172.21.15.44:0/1510939457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c0cf7f0 con 0x7fd66c0ccfa0 2024-09-13T08:45:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd66be00640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd66c15de90 0x7fd66c1fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:38.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd671800640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd66c15e270 0x7fd66c155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:38.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd670e00640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd66c0ccfa0 0x7fd66c151080 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c450 tx=0x7fd66000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:38.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd668c00640 1 -- 172.21.15.44:0/1510939457 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660012830 con 0x7fd66c0ccfa0 2024-09-13T08:45:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd668c00640 1 -- 172.21.15.44:0/1510939457 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd660017070 con 0x7fd66c0ccfa0 2024-09-13T08:45:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd668c00640 1 -- 172.21.15.44:0/1510939457 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660012d80 con 0x7fd66c0ccfa0 2024-09-13T08:45:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd673800640 1 -- 172.21.15.44:0/1510939457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66c1fb950 con 0x7fd66c0ccfa0 2024-09-13T08:45:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd673800640 1 -- 172.21.15.44:0/1510939457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66c0cfa90 con 0x7fd66c0ccfa0 2024-09-13T08:45:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.817+0000 7fd673800640 1 -- 172.21.15.44:0/1510939457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66c06a4a0 con 0x7fd66c0ccfa0 2024-09-13T08:45:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.821+0000 7fd668c00640 1 -- 172.21.15.44:0/1510939457 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fd660007020 con 0x7fd66c0ccfa0 2024-09-13T08:45:38.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.821+0000 7fd668c00640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd64c073b50 0x7fd64c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:38.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.821+0000 7fd668c00640 1 -- 172.21.15.44:0/1510939457 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7fd660093100 con 0x7fd66c0ccfa0 2024-09-13T08:45:38.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.821+0000 7fd66be00640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd64c073b50 0x7fd64c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.821+0000 7fd66be00640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd64c073b50 0x7fd64c076010 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd658005fd0 tx=0x7fd65800a540 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:38.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:38.825+0000 7fd668c00640 1 -- 172.21.15.44:0/1510939457 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd66005fda0 con 0x7fd66c0ccfa0 2024-09-13T08:45:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.061+0000 7fd673800640 1 -- 172.21.15.44:0/1510939457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd66c071000 con 0x7fd66c0ccfa0 2024-09-13T08:45:39.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd668c00640 1 -- 172.21.15.44:0/1510939457 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1056 (secure 0 0 0) 0x7fd66005f6a0 con 0x7fd66c0ccfa0 2024-09-13T08:45:39.065 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:39.065 INFO:teuthology.orchestra.run.smithi044.stdout:{"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":509,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":75,"num_osds":8,"num_up_osds":8,"osd_up_since":1726216643,"num_in_osds":8,"osd_in_since":1726216632,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":411750400,"bytes_avail":772682362880,"bytes_total":773094113280,"write_bytes_sec":2386,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":46,"btime":"2024-09-13T08:45:36:672438+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6333}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-13T08:44:24.622159+0000","services":{}},"progress_events":{}} 2024-09-13T08:45:39.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 -- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd64c073b50 msgr2=0x7fd64c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:39.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd64c073b50 0x7fd64c076010 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd658005fd0 tx=0x7fd65800a540 comp rx=0 tx=0).stop 2024-09-13T08:45:39.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 -- 172.21.15.44:0/1510939457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd66c0ccfa0 msgr2=0x7fd66c151080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:39.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd66c0ccfa0 0x7fd66c151080 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c450 tx=0x7fd66000c910 comp rx=0 tx=0).stop 2024-09-13T08:45:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 -- 172.21.15.44:0/1510939457 shutdown_connections 2024-09-13T08:45:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd64c073b50 0x7fd64c076010 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd66c15e270 0x7fd66c155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd66c15de90 0x7fd66c1fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 --2- 172.21.15.44:0/1510939457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd66c0ccfa0 0x7fd66c151080 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 -- 172.21.15.44:0/1510939457 >> 172.21.15.44:0/1510939457 conn(0x7fd66c06d220 msgr2=0x7fd66c160140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 -- 172.21.15.44:0/1510939457 shutdown_connections 2024-09-13T08:45:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.065+0000 7fd673800640 1 -- 172.21.15.44:0/1510939457 wait complete. 2024-09-13T08:45:39.086 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-13T08:45:39.087 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:45:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 -- 172.21.15.44:0/461304031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15bea0 msgr2=0x7f2b6c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 --2- 172.21.15.44:0/461304031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15bea0 0x7f2b6c0ce670 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009900 tx=0x7f2b5c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 -- 172.21.15.44:0/461304031 shutdown_connections 2024-09-13T08:45:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 --2- 172.21.15.44:0/461304031 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b6c0d28e0 0x7f2b6c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 --2- 172.21.15.44:0/461304031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15bea0 0x7f2b6c0ce670 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 -- 172.21.15.44:0/461304031 >> 172.21.15.44:0/461304031 conn(0x7f2b6c0d1dc0 msgr2=0x7f2b6c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 -- 172.21.15.44:0/461304031 shutdown_connections 2024-09-13T08:45:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 -- 172.21.15.44:0/461304031 wait complete. 2024-09-13T08:45:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 Processor -- start 2024-09-13T08:45:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 -- start start 2024-09-13T08:45:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b6c0cdb30 0x7f2b6c1593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c0d28e0 0x7f2b6c0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b6c15bea0 0x7f2b6c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c168fc0 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c168e40 con 0x7f2b6c0cdb30 2024-09-13T08:45:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b71800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c169140 con 0x7f2b6c15bea0 2024-09-13T08:45:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c0d28e0 0x7f2b6c0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c0d28e0 0x7f2b6c0c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38068/0 (socket says 172.21.15.44:38068) 2024-09-13T08:45:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b63e00640 1 -- 172.21.15.44:0/2442619188 learned_addr learned my addr 172.21.15.44:0/2442619188 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b6aa00640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b6c0cdb30 0x7f2b6c1593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b6b400640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b6c15bea0 0x7f2b6c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b63e00640 1 -- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b6c15bea0 msgr2=0x7f2b6c157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b63e00640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b6c15bea0 0x7f2b6c157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b63e00640 1 -- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b6c0cdb30 msgr2=0x7f2b6c1593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b63e00640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b6c0cdb30 0x7f2b6c1593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b63e00640 1 -- 172.21.15.44:0/2442619188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c157cc0 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b6aa00640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b6c0cdb30 0x7f2b6c1593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.233+0000 7f2b6b400640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b6c15bea0 0x7f2b6c157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b63e00640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c0d28e0 0x7f2b6c0c7c30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c450 tx=0x7f2b5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b62a00640 1 -- 172.21.15.44:0/2442619188 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b540052d0 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b62a00640 1 -- 172.21.15.44:0/2442619188 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b54005470 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b62a00640 1 -- 172.21.15.44:0/2442619188 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54012890 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b71800640 1 -- 172.21.15.44:0/2442619188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c0b3040 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b71800640 1 -- 172.21.15.44:0/2442619188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c1f8ee0 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b71800640 1 -- 172.21.15.44:0/2442619188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6c1f89f0 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b62a00640 1 -- 172.21.15.44:0/2442619188 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f2b54003620 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b62a00640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b48073b20 0x7f2b48075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b62a00640 1 -- 172.21.15.44:0/2442619188 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f2b54092390 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.237+0000 7f2b6aa00640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b48073b20 0x7f2b48075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.241+0000 7f2b6aa00640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b48073b20 0x7f2b48075fe0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0af000 tx=0x7f2b5c01b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.241+0000 7f2b62a00640 1 -- 172.21.15.44:0/2442619188 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2b6c157cc0 con 0x7f2b6c0d28e0 2024-09-13T08:45:39.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.413+0000 7f2b71800640 1 -- 172.21.15.44:0/2442619188 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b6c0cc9d0 con 0x7f2b48073b20 2024-09-13T08:45:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.421+0000 7f2b62a00640 1 -- 172.21.15.44:0/2442619188 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252057 (secure 0 0 0) 0x7f2b6c0cc9d0 con 0x7f2b48073b20 2024-09-13T08:45:39.424 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:45:39.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 -- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b48073b20 msgr2=0x7f2b48075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:39.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b48073b20 0x7f2b48075fe0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0af000 tx=0x7f2b5c01b120 comp rx=0 tx=0).stop 2024-09-13T08:45:39.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 -- 172.21.15.44:0/2442619188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c0d28e0 msgr2=0x7f2b6c0c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:39.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c0d28e0 0x7f2b6c0c7c30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c450 tx=0x7f2b5400c910 comp rx=0 tx=0).stop 2024-09-13T08:45:39.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 -- 172.21.15.44:0/2442619188 shutdown_connections 2024-09-13T08:45:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b48073b20 0x7f2b48075fe0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b6c15bea0 0x7f2b6c157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c0d28e0 0x7f2b6c0c7c30 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 --2- 172.21.15.44:0/2442619188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b6c0cdb30 0x7f2b6c1593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 -- 172.21.15.44:0/2442619188 >> 172.21.15.44:0/2442619188 conn(0x7f2b6c0d1dc0 msgr2=0x7f2b6c0d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 -- 172.21.15.44:0/2442619188 shutdown_connections 2024-09-13T08:45:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:39.425+0000 7f2b71800640 1 -- 172.21.15.44:0/2442619188 wait complete. 2024-09-13T08:45:39.444 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-13T08:45:38.641353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402100,"kb_used_data":8188,"kb_used_omap":12,"kb_used_meta":393715,"kb_avail":754572620,"statfs":{"total":773094113280,"available":772682362880,"internally_reserved":0,"allocated":8384512,"data_stored":4219595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":403164721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.007629"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689535+0000","last_change":"2024-09-13T08:45:33.684143+0000","last_active":"2024-09-13T08:45:33.689535+0000","last_peered":"2024-09-13T08:45:33.689535+0000","last_clean":"2024-09-13T08:45:33.689535+0000","last_became_active":"2024-09-13T08:45:33.683889+0000","last_became_peered":"2024-09-13T08:45:33.683889+0000","last_unstale":"2024-09-13T08:45:33.689535+0000","last_undegraded":"2024-09-13T08:45:33.689535+0000","last_fullsized":"2024-09-13T08:45:33.689535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:07.001116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688670+0000","last_change":"2024-09-13T08:45:33.685799+0000","last_active":"2024-09-13T08:45:33.688670+0000","last_peered":"2024-09-13T08:45:33.688670+0000","last_clean":"2024-09-13T08:45:33.688670+0000","last_became_active":"2024-09-13T08:45:33.685475+0000","last_became_peered":"2024-09-13T08:45:33.685475+0000","last_unstale":"2024-09-13T08:45:33.688670+0000","last_undegraded":"2024-09-13T08:45:33.688670+0000","last_fullsized":"2024-09-13T08:45:33.688670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:35.321800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684692+0000","last_change":"2024-09-13T08:45:33.683135+0000","last_active":"2024-09-13T08:45:33.684692+0000","last_peered":"2024-09-13T08:45:33.684692+0000","last_clean":"2024-09-13T08:45:33.684692+0000","last_became_active":"2024-09-13T08:45:33.682789+0000","last_became_peered":"2024-09-13T08:45:33.682789+0000","last_unstale":"2024-09-13T08:45:33.684692+0000","last_undegraded":"2024-09-13T08:45:33.684692+0000","last_fullsized":"2024-09-13T08:45:33.684692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:47:16.394704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684656+0000","last_change":"2024-09-13T08:45:33.682881+0000","last_active":"2024-09-13T08:45:33.684656+0000","last_peered":"2024-09-13T08:45:33.684656+0000","last_clean":"2024-09-13T08:45:33.684656+0000","last_became_active":"2024-09-13T08:45:33.681260+0000","last_became_peered":"2024-09-13T08:45:33.681260+0000","last_unstale":"2024-09-13T08:45:33.684656+0000","last_undegraded":"2024-09-13T08:45:33.684656+0000","last_fullsized":"2024-09-13T08:45:33.684656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:34.085379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675098+0000","last_change":"2024-09-13T08:45:33.673961+0000","last_active":"2024-09-13T08:45:33.675098+0000","last_peered":"2024-09-13T08:45:33.675098+0000","last_clean":"2024-09-13T08:45:33.675098+0000","last_became_active":"2024-09-13T08:45:33.673661+0000","last_became_peered":"2024-09-13T08:45:33.673661+0000","last_unstale":"2024-09-13T08:45:33.675098+0000","last_undegraded":"2024-09-13T08:45:33.675098+0000","last_fullsized":"2024-09-13T08:45:33.675098+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:05.467156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689608+0000","last_change":"2024-09-13T08:45:33.686731+0000","last_active":"2024-09-13T08:45:33.689608+0000","last_peered":"2024-09-13T08:45:33.689608+0000","last_clean":"2024-09-13T08:45:33.689608+0000","last_became_active":"2024-09-13T08:45:33.686352+0000","last_became_peered":"2024-09-13T08:45:33.686352+0000","last_unstale":"2024-09-13T08:45:33.689608+0000","last_undegraded":"2024-09-13T08:45:33.689608+0000","last_fullsized":"2024-09-13T08:45:33.689608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:43:22.048135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.666172+0000","last_change":"2024-09-13T08:45:33.660192+0000","last_active":"2024-09-13T08:45:33.666172+0000","last_peered":"2024-09-13T08:45:33.666172+0000","last_clean":"2024-09-13T08:45:33.666172+0000","last_became_active":"2024-09-13T08:45:33.659425+0000","last_became_peered":"2024-09-13T08:45:33.659425+0000","last_unstale":"2024-09-13T08:45:33.666172+0000","last_undegraded":"2024-09-13T08:45:33.666172+0000","last_fullsized":"2024-09-13T08:45:33.666172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:27.961233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688123+0000","last_change":"2024-09-13T08:45:33.671839+0000","last_active":"2024-09-13T08:45:33.688123+0000","last_peered":"2024-09-13T08:45:33.688123+0000","last_clean":"2024-09-13T08:45:33.688123+0000","last_became_active":"2024-09-13T08:45:33.671538+0000","last_became_peered":"2024-09-13T08:45:33.671538+0000","last_unstale":"2024-09-13T08:45:33.688123+0000","last_undegraded":"2024-09-13T08:45:33.688123+0000","last_fullsized":"2024-09-13T08:45:33.688123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:51.161859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684700+0000","last_change":"2024-09-13T08:45:33.683295+0000","last_active":"2024-09-13T08:45:33.684700+0000","last_peered":"2024-09-13T08:45:33.684700+0000","last_clean":"2024-09-13T08:45:33.684700+0000","last_became_active":"2024-09-13T08:45:33.681988+0000","last_became_peered":"2024-09-13T08:45:33.681988+0000","last_unstale":"2024-09-13T08:45:33.684700+0000","last_undegraded":"2024-09-13T08:45:33.684700+0000","last_fullsized":"2024-09-13T08:45:33.684700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:34:29.504228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681352+0000","last_change":"2024-09-13T08:45:33.667868+0000","last_active":"2024-09-13T08:45:33.681352+0000","last_peered":"2024-09-13T08:45:33.681352+0000","last_clean":"2024-09-13T08:45:33.681352+0000","last_became_active":"2024-09-13T08:45:33.667543+0000","last_became_peered":"2024-09-13T08:45:33.667543+0000","last_unstale":"2024-09-13T08:45:33.681352+0000","last_undegraded":"2024-09-13T08:45:33.681352+0000","last_fullsized":"2024-09-13T08:45:33.681352+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:10.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691398+0000","last_change":"2024-09-13T08:45:33.689087+0000","last_active":"2024-09-13T08:45:33.691398+0000","last_peered":"2024-09-13T08:45:33.691398+0000","last_clean":"2024-09-13T08:45:33.691398+0000","last_became_active":"2024-09-13T08:45:33.688839+0000","last_became_peered":"2024-09-13T08:45:33.688839+0000","last_unstale":"2024-09-13T08:45:33.691398+0000","last_undegraded":"2024-09-13T08:45:33.691398+0000","last_fullsized":"2024-09-13T08:45:33.691398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:20:46.695925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.666207+0000","last_change":"2024-09-13T08:45:33.664987+0000","last_active":"2024-09-13T08:45:33.666207+0000","last_peered":"2024-09-13T08:45:33.666207+0000","last_clean":"2024-09-13T08:45:33.666207+0000","last_became_active":"2024-09-13T08:45:33.664707+0000","last_became_peered":"2024-09-13T08:45:33.664707+0000","last_unstale":"2024-09-13T08:45:33.666207+0000","last_undegraded":"2024-09-13T08:45:33.666207+0000","last_fullsized":"2024-09-13T08:45:33.666207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:03.021478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683740+0000","last_change":"2024-09-13T08:45:33.681174+0000","last_active":"2024-09-13T08:45:33.683740+0000","last_peered":"2024-09-13T08:45:33.683740+0000","last_clean":"2024-09-13T08:45:33.683740+0000","last_became_active":"2024-09-13T08:45:33.680680+0000","last_became_peered":"2024-09-13T08:45:33.680680+0000","last_unstale":"2024-09-13T08:45:33.683740+0000","last_undegraded":"2024-09-13T08:45:33.683740+0000","last_fullsized":"2024-09-13T08:45:33.683740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:46.928710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671724+0000","last_change":"2024-09-13T08:45:33.668789+0000","last_active":"2024-09-13T08:45:33.671724+0000","last_peered":"2024-09-13T08:45:33.671724+0000","last_clean":"2024-09-13T08:45:33.671724+0000","last_became_active":"2024-09-13T08:45:33.668462+0000","last_became_peered":"2024-09-13T08:45:33.668462+0000","last_unstale":"2024-09-13T08:45:33.671724+0000","last_undegraded":"2024-09-13T08:45:33.671724+0000","last_fullsized":"2024-09-13T08:45:33.671724+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:32:11.095202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.673865+0000","last_change":"2024-09-13T08:45:33.671105+0000","last_active":"2024-09-13T08:45:33.673865+0000","last_peered":"2024-09-13T08:45:33.673865+0000","last_clean":"2024-09-13T08:45:33.673865+0000","last_became_active":"2024-09-13T08:45:33.670718+0000","last_became_peered":"2024-09-13T08:45:33.670718+0000","last_unstale":"2024-09-13T08:45:33.673865+0000","last_undegraded":"2024-09-13T08:45:33.673865+0000","last_fullsized":"2024-09-13T08:45:33.673865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:34.021306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691685+0000","last_change":"2024-09-13T08:45:33.686654+0000","last_active":"2024-09-13T08:45:33.691685+0000","last_peered":"2024-09-13T08:45:33.691685+0000","last_clean":"2024-09-13T08:45:33.691685+0000","last_became_active":"2024-09-13T08:45:33.686375+0000","last_became_peered":"2024-09-13T08:45:33.686375+0000","last_unstale":"2024-09-13T08:45:33.691685+0000","last_undegraded":"2024-09-13T08:45:33.691685+0000","last_fullsized":"2024-09-13T08:45:33.691685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:03.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.672216+0000","last_change":"2024-09-13T08:45:33.670204+0000","last_active":"2024-09-13T08:45:33.672216+0000","last_peered":"2024-09-13T08:45:33.672216+0000","last_clean":"2024-09-13T08:45:33.672216+0000","last_became_active":"2024-09-13T08:45:33.669936+0000","last_became_peered":"2024-09-13T08:45:33.669936+0000","last_unstale":"2024-09-13T08:45:33.672216+0000","last_undegraded":"2024-09-13T08:45:33.672216+0000","last_fullsized":"2024-09-13T08:45:33.672216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:34.875650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667097+0000","last_change":"2024-09-13T08:45:33.663820+0000","last_active":"2024-09-13T08:45:33.667097+0000","last_peered":"2024-09-13T08:45:33.667097+0000","last_clean":"2024-09-13T08:45:33.667097+0000","last_became_active":"2024-09-13T08:45:33.663147+0000","last_became_peered":"2024-09-13T08:45:33.663147+0000","last_unstale":"2024-09-13T08:45:33.667097+0000","last_undegraded":"2024-09-13T08:45:33.667097+0000","last_fullsized":"2024-09-13T08:45:33.667097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:16:16.642146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.687335+0000","last_change":"2024-09-13T08:45:33.680218+0000","last_active":"2024-09-13T08:45:33.687335+0000","last_peered":"2024-09-13T08:45:33.687335+0000","last_clean":"2024-09-13T08:45:33.687335+0000","last_became_active":"2024-09-13T08:45:33.679938+0000","last_became_peered":"2024-09-13T08:45:33.679938+0000","last_unstale":"2024-09-13T08:45:33.687335+0000","last_undegraded":"2024-09-13T08:45:33.687335+0000","last_fullsized":"2024-09-13T08:45:33.687335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:05.467156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675520+0000","last_change":"2024-09-13T08:45:33.674349+0000","last_active":"2024-09-13T08:45:33.675520+0000","last_peered":"2024-09-13T08:45:33.675520+0000","last_clean":"2024-09-13T08:45:33.675520+0000","last_became_active":"2024-09-13T08:45:33.674004+0000","last_became_peered":"2024-09-13T08:45:33.674004+0000","last_unstale":"2024-09-13T08:45:33.675520+0000","last_undegraded":"2024-09-13T08:45:33.675520+0000","last_fullsized":"2024-09-13T08:45:33.675520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:06.280244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683524+0000","last_change":"2024-09-13T08:45:33.679260+0000","last_active":"2024-09-13T08:45:33.683524+0000","last_peered":"2024-09-13T08:45:33.683524+0000","last_clean":"2024-09-13T08:45:33.683524+0000","last_became_active":"2024-09-13T08:45:33.678976+0000","last_became_peered":"2024-09-13T08:45:33.678976+0000","last_unstale":"2024-09-13T08:45:33.683524+0000","last_undegraded":"2024-09-13T08:45:33.683524+0000","last_fullsized":"2024-09-13T08:45:33.683524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:41:27.288006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.673433+0000","last_change":"2024-09-13T08:45:33.666891+0000","last_active":"2024-09-13T08:45:33.673433+0000","last_peered":"2024-09-13T08:45:33.673433+0000","last_clean":"2024-09-13T08:45:33.673433+0000","last_became_active":"2024-09-13T08:45:33.666585+0000","last_became_peered":"2024-09-13T08:45:33.666585+0000","last_unstale":"2024-09-13T08:45:33.673433+0000","last_undegraded":"2024-09-13T08:45:33.673433+0000","last_fullsized":"2024-09-13T08:45:33.673433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:26:12.036228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.678998+0000","last_change":"2024-09-13T08:45:33.670178+0000","last_active":"2024-09-13T08:45:33.678998+0000","last_peered":"2024-09-13T08:45:33.678998+0000","last_clean":"2024-09-13T08:45:33.678998+0000","last_became_active":"2024-09-13T08:45:33.669687+0000","last_became_peered":"2024-09-13T08:45:33.669687+0000","last_unstale":"2024-09-13T08:45:33.678998+0000","last_undegraded":"2024-09-13T08:45:33.678998+0000","last_fullsized":"2024-09-13T08:45:33.678998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:11:12.735593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688146+0000","last_change":"2024-09-13T08:45:33.677052+0000","last_active":"2024-09-13T08:45:33.688146+0000","last_peered":"2024-09-13T08:45:33.688146+0000","last_clean":"2024-09-13T08:45:33.688146+0000","last_became_active":"2024-09-13T08:45:33.676767+0000","last_became_peered":"2024-09-13T08:45:33.676767+0000","last_unstale":"2024-09-13T08:45:33.688146+0000","last_undegraded":"2024-09-13T08:45:33.688146+0000","last_fullsized":"2024-09-13T08:45:33.688146+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:08.841526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.679899+0000","last_change":"2024-09-13T08:45:33.670240+0000","last_active":"2024-09-13T08:45:33.679899+0000","last_peered":"2024-09-13T08:45:33.679899+0000","last_clean":"2024-09-13T08:45:33.679899+0000","last_became_active":"2024-09-13T08:45:33.669952+0000","last_became_peered":"2024-09-13T08:45:33.669952+0000","last_unstale":"2024-09-13T08:45:33.679899+0000","last_undegraded":"2024-09-13T08:45:33.679899+0000","last_fullsized":"2024-09-13T08:45:33.679899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:15.070109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.692180+0000","last_change":"2024-09-13T08:45:33.690462+0000","last_active":"2024-09-13T08:45:33.692180+0000","last_peered":"2024-09-13T08:45:33.692180+0000","last_clean":"2024-09-13T08:45:33.692180+0000","last_became_active":"2024-09-13T08:45:33.690197+0000","last_became_peered":"2024-09-13T08:45:33.690197+0000","last_unstale":"2024-09-13T08:45:33.692180+0000","last_undegraded":"2024-09-13T08:45:33.692180+0000","last_fullsized":"2024-09-13T08:45:33.692180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:14.833887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.668085+0000","last_change":"2024-09-13T08:45:33.663806+0000","last_active":"2024-09-13T08:45:33.668085+0000","last_peered":"2024-09-13T08:45:33.668085+0000","last_clean":"2024-09-13T08:45:33.668085+0000","last_became_active":"2024-09-13T08:45:33.663498+0000","last_became_peered":"2024-09-13T08:45:33.663498+0000","last_unstale":"2024-09-13T08:45:33.668085+0000","last_undegraded":"2024-09-13T08:45:33.668085+0000","last_fullsized":"2024-09-13T08:45:33.668085+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:40:14.829023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683182+0000","last_change":"2024-09-13T08:45:33.674360+0000","last_active":"2024-09-13T08:45:33.683182+0000","last_peered":"2024-09-13T08:45:33.683182+0000","last_clean":"2024-09-13T08:45:33.683182+0000","last_became_active":"2024-09-13T08:45:33.673982+0000","last_became_peered":"2024-09-13T08:45:33.673982+0000","last_unstale":"2024-09-13T08:45:33.683182+0000","last_undegraded":"2024-09-13T08:45:33.683182+0000","last_fullsized":"2024-09-13T08:45:33.683182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:16:14.468554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689721+0000","last_change":"2024-09-13T08:45:33.689213+0000","last_active":"2024-09-13T08:45:33.689721+0000","last_peered":"2024-09-13T08:45:33.689721+0000","last_clean":"2024-09-13T08:45:33.689721+0000","last_became_active":"2024-09-13T08:45:33.688971+0000","last_became_peered":"2024-09-13T08:45:33.688971+0000","last_unstale":"2024-09-13T08:45:33.689721+0000","last_undegraded":"2024-09-13T08:45:33.689721+0000","last_fullsized":"2024-09-13T08:45:33.689721+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:48:29.114234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.687983+0000","last_change":"2024-09-13T08:45:33.672566+0000","last_active":"2024-09-13T08:45:33.687983+0000","last_peered":"2024-09-13T08:45:33.687983+0000","last_clean":"2024-09-13T08:45:33.687983+0000","last_became_active":"2024-09-13T08:45:33.672302+0000","last_became_peered":"2024-09-13T08:45:33.672302+0000","last_unstale":"2024-09-13T08:45:33.687983+0000","last_undegraded":"2024-09-13T08:45:33.687983+0000","last_fullsized":"2024-09-13T08:45:33.687983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:51:14.557452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684957+0000","last_change":"2024-09-13T08:45:33.675233+0000","last_active":"2024-09-13T08:45:33.684957+0000","last_peered":"2024-09-13T08:45:33.684957+0000","last_clean":"2024-09-13T08:45:33.684957+0000","last_became_active":"2024-09-13T08:45:33.674920+0000","last_became_peered":"2024-09-13T08:45:33.674920+0000","last_unstale":"2024-09-13T08:45:33.684957+0000","last_undegraded":"2024-09-13T08:45:33.684957+0000","last_fullsized":"2024-09-13T08:45:33.684957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:10.521583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683642+0000","last_change":"2024-09-13T08:45:33.679637+0000","last_active":"2024-09-13T08:45:33.683642+0000","last_peered":"2024-09-13T08:45:33.683642+0000","last_clean":"2024-09-13T08:45:33.683642+0000","last_became_active":"2024-09-13T08:45:33.679391+0000","last_became_peered":"2024-09-13T08:45:33.679391+0000","last_unstale":"2024-09-13T08:45:33.683642+0000","last_undegraded":"2024-09-13T08:45:33.683642+0000","last_fullsized":"2024-09-13T08:45:33.683642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:56.161329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691281+0000","last_change":"2024-09-13T08:45:33.688757+0000","last_active":"2024-09-13T08:45:33.691281+0000","last_peered":"2024-09-13T08:45:33.691281+0000","last_clean":"2024-09-13T08:45:33.691281+0000","last_became_active":"2024-09-13T08:45:33.687386+0000","last_became_peered":"2024-09-13T08:45:33.687386+0000","last_unstale":"2024-09-13T08:45:33.691281+0000","last_undegraded":"2024-09-13T08:45:33.691281+0000","last_fullsized":"2024-09-13T08:45:33.691281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:27.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"74'4","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689227+0000","last_change":"2024-09-13T08:45:32.657493+0000","last_active":"2024-09-13T08:45:33.689227+0000","last_peered":"2024-09-13T08:45:33.689227+0000","last_clean":"2024-09-13T08:45:33.689227+0000","last_became_active":"2024-09-13T08:45:32.656217+0000","last_became_peered":"2024-09-13T08:45:32.656217+0000","last_unstale":"2024-09-13T08:45:33.689227+0000","last_undegraded":"2024-09-13T08:45:33.689227+0000","last_fullsized":"2024-09-13T08:45:33.689227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:13.023500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.683491+0000","last_change":"2024-09-13T08:45:33.678452+0000","last_active":"2024-09-13T08:45:33.683491+0000","last_peered":"2024-09-13T08:45:33.683491+0000","last_clean":"2024-09-13T08:45:33.683491+0000","last_became_active":"2024-09-13T08:45:33.678224+0000","last_became_peered":"2024-09-13T08:45:33.678224+0000","last_unstale":"2024-09-13T08:45:33.683491+0000","last_undegraded":"2024-09-13T08:45:33.683491+0000","last_fullsized":"2024-09-13T08:45:33.683491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:52:00.198742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684555+0000","last_change":"2024-09-13T08:45:32.654382+0000","last_active":"2024-09-13T08:45:33.684555+0000","last_peered":"2024-09-13T08:45:33.684555+0000","last_clean":"2024-09-13T08:45:33.684555+0000","last_became_active":"2024-09-13T08:45:32.654090+0000","last_became_peered":"2024-09-13T08:45:32.654090+0000","last_unstale":"2024-09-13T08:45:33.684555+0000","last_undegraded":"2024-09-13T08:45:33.684555+0000","last_fullsized":"2024-09-13T08:45:33.684555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:42.438703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.690499+0000","last_change":"2024-09-13T08:45:33.686795+0000","last_active":"2024-09-13T08:45:33.690499+0000","last_peered":"2024-09-13T08:45:33.690499+0000","last_clean":"2024-09-13T08:45:33.690499+0000","last_became_active":"2024-09-13T08:45:33.686497+0000","last_became_peered":"2024-09-13T08:45:33.686497+0000","last_unstale":"2024-09-13T08:45:33.690499+0000","last_undegraded":"2024-09-13T08:45:33.690499+0000","last_fullsized":"2024-09-13T08:45:33.690499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:27.050847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694008+0000","last_change":"2024-09-13T08:45:32.656780+0000","last_active":"2024-09-13T08:45:33.694008+0000","last_peered":"2024-09-13T08:45:33.694008+0000","last_clean":"2024-09-13T08:45:33.694008+0000","last_became_active":"2024-09-13T08:45:32.653941+0000","last_became_peered":"2024-09-13T08:45:32.653941+0000","last_unstale":"2024-09-13T08:45:33.694008+0000","last_undegraded":"2024-09-13T08:45:33.694008+0000","last_fullsized":"2024-09-13T08:45:33.694008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:12.824731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667910+0000","last_change":"2024-09-13T08:45:33.663007+0000","last_active":"2024-09-13T08:45:33.667910+0000","last_peered":"2024-09-13T08:45:33.667910+0000","last_clean":"2024-09-13T08:45:33.667910+0000","last_became_active":"2024-09-13T08:45:33.662591+0000","last_became_peered":"2024-09-13T08:45:33.662591+0000","last_unstale":"2024-09-13T08:45:33.667910+0000","last_undegraded":"2024-09-13T08:45:33.667910+0000","last_fullsized":"2024-09-13T08:45:33.667910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:59.290026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.690456+0000","last_change":"2024-09-13T08:45:32.655814+0000","last_active":"2024-09-13T08:45:33.690456+0000","last_peered":"2024-09-13T08:45:33.690456+0000","last_clean":"2024-09-13T08:45:33.690456+0000","last_became_active":"2024-09-13T08:45:32.655509+0000","last_became_peered":"2024-09-13T08:45:32.655509+0000","last_unstale":"2024-09-13T08:45:33.690456+0000","last_undegraded":"2024-09-13T08:45:33.690456+0000","last_fullsized":"2024-09-13T08:45:33.690456+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:56:39.873532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.688008+0000","last_change":"2024-09-13T08:45:33.676002+0000","last_active":"2024-09-13T08:45:33.688008+0000","last_peered":"2024-09-13T08:45:33.688008+0000","last_clean":"2024-09-13T08:45:33.688008+0000","last_became_active":"2024-09-13T08:45:33.675711+0000","last_became_peered":"2024-09-13T08:45:33.675711+0000","last_unstale":"2024-09-13T08:45:33.688008+0000","last_undegraded":"2024-09-13T08:45:33.688008+0000","last_fullsized":"2024-09-13T08:45:33.688008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:34.875650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685399+0000","last_change":"2024-09-13T08:45:32.650357+0000","last_active":"2024-09-13T08:45:33.685399+0000","last_peered":"2024-09-13T08:45:33.685399+0000","last_clean":"2024-09-13T08:45:33.685399+0000","last_became_active":"2024-09-13T08:45:32.650092+0000","last_became_peered":"2024-09-13T08:45:32.650092+0000","last_unstale":"2024-09-13T08:45:33.685399+0000","last_undegraded":"2024-09-13T08:45:33.685399+0000","last_fullsized":"2024-09-13T08:45:33.685399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:11.131195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689533+0000","last_change":"2024-09-13T08:45:33.686668+0000","last_active":"2024-09-13T08:45:33.689533+0000","last_peered":"2024-09-13T08:45:33.689533+0000","last_clean":"2024-09-13T08:45:33.689533+0000","last_became_active":"2024-09-13T08:45:33.686340+0000","last_became_peered":"2024-09-13T08:45:33.686340+0000","last_unstale":"2024-09-13T08:45:33.689533+0000","last_undegraded":"2024-09-13T08:45:33.689533+0000","last_fullsized":"2024-09-13T08:45:33.689533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:45:11.714466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685672+0000","last_change":"2024-09-13T08:45:32.652970+0000","last_active":"2024-09-13T08:45:33.685672+0000","last_peered":"2024-09-13T08:45:33.685672+0000","last_clean":"2024-09-13T08:45:33.685672+0000","last_became_active":"2024-09-13T08:45:32.652708+0000","last_became_peered":"2024-09-13T08:45:32.652708+0000","last_unstale":"2024-09-13T08:45:33.685672+0000","last_undegraded":"2024-09-13T08:45:33.685672+0000","last_fullsized":"2024-09-13T08:45:33.685672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:26.448081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.888913+0000","last_change":"2024-09-13T08:45:33.660387+0000","last_active":"2024-09-13T08:45:34.888913+0000","last_peered":"2024-09-13T08:45:34.888913+0000","last_clean":"2024-09-13T08:45:34.888913+0000","last_became_active":"2024-09-13T08:45:33.660031+0000","last_became_peered":"2024-09-13T08:45:33.660031+0000","last_unstale":"2024-09-13T08:45:34.888913+0000","last_undegraded":"2024-09-13T08:45:34.888913+0000","last_fullsized":"2024-09-13T08:45:34.888913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:39:43.561306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.889030+0000","last_change":"2024-09-13T08:45:32.659746+0000","last_active":"2024-09-13T08:45:34.889030+0000","last_peered":"2024-09-13T08:45:34.889030+0000","last_clean":"2024-09-13T08:45:34.889030+0000","last_became_active":"2024-09-13T08:45:32.659314+0000","last_became_peered":"2024-09-13T08:45:32.659314+0000","last_unstale":"2024-09-13T08:45:34.889030+0000","last_undegraded":"2024-09-13T08:45:34.889030+0000","last_fullsized":"2024-09-13T08:45:34.889030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:02.826589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671291+0000","last_change":"2024-09-13T08:45:33.668090+0000","last_active":"2024-09-13T08:45:33.671291+0000","last_peered":"2024-09-13T08:45:33.671291+0000","last_clean":"2024-09-13T08:45:33.671291+0000","last_became_active":"2024-09-13T08:45:33.667802+0000","last_became_peered":"2024-09-13T08:45:33.667802+0000","last_unstale":"2024-09-13T08:45:33.671291+0000","last_undegraded":"2024-09-13T08:45:33.671291+0000","last_fullsized":"2024-09-13T08:45:33.671291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:02:31.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660673+0000","last_change":"2024-09-13T08:45:32.654860+0000","last_active":"2024-09-13T08:45:33.660673+0000","last_peered":"2024-09-13T08:45:33.660673+0000","last_clean":"2024-09-13T08:45:33.660673+0000","last_became_active":"2024-09-13T08:45:32.652510+0000","last_became_peered":"2024-09-13T08:45:32.652510+0000","last_unstale":"2024-09-13T08:45:33.660673+0000","last_undegraded":"2024-09-13T08:45:33.660673+0000","last_fullsized":"2024-09-13T08:45:33.660673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:16.016602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681459+0000","last_change":"2024-09-13T08:45:33.679316+0000","last_active":"2024-09-13T08:45:33.681459+0000","last_peered":"2024-09-13T08:45:33.681459+0000","last_clean":"2024-09-13T08:45:33.681459+0000","last_became_active":"2024-09-13T08:45:33.678605+0000","last_became_peered":"2024-09-13T08:45:33.678605+0000","last_unstale":"2024-09-13T08:45:33.681459+0000","last_undegraded":"2024-09-13T08:45:33.681459+0000","last_fullsized":"2024-09-13T08:45:33.681459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:01:38.403717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685240+0000","last_change":"2024-09-13T08:45:32.655976+0000","last_active":"2024-09-13T08:45:33.685240+0000","last_peered":"2024-09-13T08:45:33.685240+0000","last_clean":"2024-09-13T08:45:33.685240+0000","last_became_active":"2024-09-13T08:45:32.654678+0000","last_became_peered":"2024-09-13T08:45:32.654678+0000","last_unstale":"2024-09-13T08:45:33.685240+0000","last_undegraded":"2024-09-13T08:45:33.685240+0000","last_fullsized":"2024-09-13T08:45:33.685240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:07.445463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684924+0000","last_change":"2024-09-13T08:45:33.682397+0000","last_active":"2024-09-13T08:45:33.684924+0000","last_peered":"2024-09-13T08:45:33.684924+0000","last_clean":"2024-09-13T08:45:33.684924+0000","last_became_active":"2024-09-13T08:45:33.682010+0000","last_became_peered":"2024-09-13T08:45:33.682010+0000","last_unstale":"2024-09-13T08:45:33.684924+0000","last_undegraded":"2024-09-13T08:45:33.684924+0000","last_fullsized":"2024-09-13T08:45:33.684924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:39.373555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.670069+0000","last_change":"2024-09-13T08:45:32.654977+0000","last_active":"2024-09-13T08:45:33.670069+0000","last_peered":"2024-09-13T08:45:33.670069+0000","last_clean":"2024-09-13T08:45:33.670069+0000","last_became_active":"2024-09-13T08:45:32.654656+0000","last_became_peered":"2024-09-13T08:45:32.654656+0000","last_unstale":"2024-09-13T08:45:33.670069+0000","last_undegraded":"2024-09-13T08:45:33.670069+0000","last_fullsized":"2024-09-13T08:45:33.670069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:09.832815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689499+0000","last_change":"2024-09-13T08:45:33.682432+0000","last_active":"2024-09-13T08:45:33.689499+0000","last_peered":"2024-09-13T08:45:33.689499+0000","last_clean":"2024-09-13T08:45:33.689499+0000","last_became_active":"2024-09-13T08:45:33.681551+0000","last_became_peered":"2024-09-13T08:45:33.681551+0000","last_unstale":"2024-09-13T08:45:33.689499+0000","last_undegraded":"2024-09-13T08:45:33.689499+0000","last_fullsized":"2024-09-13T08:45:33.689499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:06.280244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"74'3","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682838+0000","last_change":"2024-09-13T08:45:32.644983+0000","last_active":"2024-09-13T08:45:33.682838+0000","last_peered":"2024-09-13T08:45:33.682838+0000","last_clean":"2024-09-13T08:45:33.682838+0000","last_became_active":"2024-09-13T08:45:32.644649+0000","last_became_peered":"2024-09-13T08:45:32.644649+0000","last_unstale":"2024-09-13T08:45:33.682838+0000","last_undegraded":"2024-09-13T08:45:33.682838+0000","last_fullsized":"2024-09-13T08:45:33.682838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:16:03.736952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.669128+0000","last_change":"2024-09-13T08:45:33.663848+0000","last_active":"2024-09-13T08:45:33.669128+0000","last_peered":"2024-09-13T08:45:33.669128+0000","last_clean":"2024-09-13T08:45:33.669128+0000","last_became_active":"2024-09-13T08:45:33.663559+0000","last_became_peered":"2024-09-13T08:45:33.663559+0000","last_unstale":"2024-09-13T08:45:33.669128+0000","last_undegraded":"2024-09-13T08:45:33.669128+0000","last_fullsized":"2024-09-13T08:45:33.669128+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:51.161859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"74'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.889209+0000","last_change":"2024-09-13T08:45:32.656020+0000","last_active":"2024-09-13T08:45:34.889209+0000","last_peered":"2024-09-13T08:45:34.889209+0000","last_clean":"2024-09-13T08:45:34.889209+0000","last_became_active":"2024-09-13T08:45:32.655215+0000","last_became_peered":"2024-09-13T08:45:32.655215+0000","last_unstale":"2024-09-13T08:45:34.889209+0000","last_undegraded":"2024-09-13T08:45:34.889209+0000","last_fullsized":"2024-09-13T08:45:34.889209+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:55.743757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.677142+0000","last_change":"2024-09-13T08:45:33.671033+0000","last_active":"2024-09-13T08:45:33.677142+0000","last_peered":"2024-09-13T08:45:33.677142+0000","last_clean":"2024-09-13T08:45:33.677142+0000","last_became_active":"2024-09-13T08:45:33.670779+0000","last_became_peered":"2024-09-13T08:45:33.670779+0000","last_unstale":"2024-09-13T08:45:33.677142+0000","last_undegraded":"2024-09-13T08:45:33.677142+0000","last_fullsized":"2024-09-13T08:45:33.677142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:47:52.333905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.888969+0000","last_change":"2024-09-13T08:45:32.656334+0000","last_active":"2024-09-13T08:45:34.888969+0000","last_peered":"2024-09-13T08:45:34.888969+0000","last_clean":"2024-09-13T08:45:34.888969+0000","last_became_active":"2024-09-13T08:45:32.656007+0000","last_became_peered":"2024-09-13T08:45:32.656007+0000","last_unstale":"2024-09-13T08:45:34.888969+0000","last_undegraded":"2024-09-13T08:45:34.888969+0000","last_fullsized":"2024-09-13T08:45:34.888969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:13.439219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682359+0000","last_change":"2024-09-13T08:45:33.681389+0000","last_active":"2024-09-13T08:45:33.682359+0000","last_peered":"2024-09-13T08:45:33.682359+0000","last_clean":"2024-09-13T08:45:33.682359+0000","last_became_active":"2024-09-13T08:45:33.681085+0000","last_became_peered":"2024-09-13T08:45:33.681085+0000","last_unstale":"2024-09-13T08:45:33.682359+0000","last_undegraded":"2024-09-13T08:45:33.682359+0000","last_fullsized":"2024-09-13T08:45:33.682359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:38.549474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.652883+0000","last_change":"2024-09-13T08:45:32.641547+0000","last_active":"2024-09-13T08:45:33.652883+0000","last_peered":"2024-09-13T08:45:33.652883+0000","last_clean":"2024-09-13T08:45:33.652883+0000","last_became_active":"2024-09-13T08:45:32.641221+0000","last_became_peered":"2024-09-13T08:45:32.641221+0000","last_unstale":"2024-09-13T08:45:33.652883+0000","last_undegraded":"2024-09-13T08:45:33.652883+0000","last_fullsized":"2024-09-13T08:45:33.652883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:37.612881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671771+0000","last_change":"2024-09-13T08:45:33.668342+0000","last_active":"2024-09-13T08:45:33.671771+0000","last_peered":"2024-09-13T08:45:33.671771+0000","last_clean":"2024-09-13T08:45:33.671771+0000","last_became_active":"2024-09-13T08:45:33.668068+0000","last_became_peered":"2024-09-13T08:45:33.668068+0000","last_unstale":"2024-09-13T08:45:33.671771+0000","last_undegraded":"2024-09-13T08:45:33.671771+0000","last_fullsized":"2024-09-13T08:45:33.671771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:51:14.557452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.653649+0000","last_change":"2024-09-13T08:45:32.645856+0000","last_active":"2024-09-13T08:45:33.653649+0000","last_peered":"2024-09-13T08:45:33.653649+0000","last_clean":"2024-09-13T08:45:33.653649+0000","last_became_active":"2024-09-13T08:45:32.645551+0000","last_became_peered":"2024-09-13T08:45:32.645551+0000","last_unstale":"2024-09-13T08:45:33.653649+0000","last_undegraded":"2024-09-13T08:45:33.653649+0000","last_fullsized":"2024-09-13T08:45:33.653649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:13.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.682542+0000","last_change":"2024-09-13T08:45:33.679240+0000","last_active":"2024-09-13T08:45:33.682542+0000","last_peered":"2024-09-13T08:45:33.682542+0000","last_clean":"2024-09-13T08:45:33.682542+0000","last_became_active":"2024-09-13T08:45:33.678945+0000","last_became_peered":"2024-09-13T08:45:33.678945+0000","last_unstale":"2024-09-13T08:45:33.682542+0000","last_undegraded":"2024-09-13T08:45:33.682542+0000","last_fullsized":"2024-09-13T08:45:33.682542+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:18.936719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"74'8","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691071+0000","last_change":"2024-09-13T08:45:32.644799+0000","last_active":"2024-09-13T08:45:33.691071+0000","last_peered":"2024-09-13T08:45:33.691071+0000","last_clean":"2024-09-13T08:45:33.691071+0000","last_became_active":"2024-09-13T08:45:32.644559+0000","last_became_peered":"2024-09-13T08:45:32.644559+0000","last_unstale":"2024-09-13T08:45:33.691071+0000","last_undegraded":"2024-09-13T08:45:33.691071+0000","last_fullsized":"2024-09-13T08:45:33.691071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:23.392065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680801+0000","last_change":"2024-09-13T08:45:33.676706+0000","last_active":"2024-09-13T08:45:33.680801+0000","last_peered":"2024-09-13T08:45:33.680801+0000","last_clean":"2024-09-13T08:45:33.680801+0000","last_became_active":"2024-09-13T08:45:33.676442+0000","last_became_peered":"2024-09-13T08:45:33.676442+0000","last_unstale":"2024-09-13T08:45:33.680801+0000","last_undegraded":"2024-09-13T08:45:33.680801+0000","last_fullsized":"2024-09-13T08:45:33.680801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:00.210640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694996+0000","last_change":"2024-09-13T08:45:32.655505+0000","last_active":"2024-09-13T08:45:33.694996+0000","last_peered":"2024-09-13T08:45:33.694996+0000","last_clean":"2024-09-13T08:45:33.694996+0000","last_became_active":"2024-09-13T08:45:32.655218+0000","last_became_peered":"2024-09-13T08:45:32.655218+0000","last_unstale":"2024-09-13T08:45:33.694996+0000","last_undegraded":"2024-09-13T08:45:33.694996+0000","last_fullsized":"2024-09-13T08:45:33.694996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:04.883497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680842+0000","last_change":"2024-09-13T08:45:33.675626+0000","last_active":"2024-09-13T08:45:33.680842+0000","last_peered":"2024-09-13T08:45:33.680842+0000","last_clean":"2024-09-13T08:45:33.680842+0000","last_became_active":"2024-09-13T08:45:33.675346+0000","last_became_peered":"2024-09-13T08:45:33.675346+0000","last_unstale":"2024-09-13T08:45:33.680842+0000","last_undegraded":"2024-09-13T08:45:33.680842+0000","last_fullsized":"2024-09-13T08:45:33.680842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:22.946990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.685223+0000","last_change":"2024-09-13T08:45:32.658779+0000","last_active":"2024-09-13T08:45:33.685223+0000","last_peered":"2024-09-13T08:45:33.685223+0000","last_clean":"2024-09-13T08:45:33.685223+0000","last_became_active":"2024-09-13T08:45:32.658420+0000","last_became_peered":"2024-09-13T08:45:32.658420+0000","last_unstale":"2024-09-13T08:45:33.685223+0000","last_undegraded":"2024-09-13T08:45:33.685223+0000","last_fullsized":"2024-09-13T08:45:33.685223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:18.130462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680052+0000","last_change":"2024-09-13T08:45:33.675553+0000","last_active":"2024-09-13T08:45:33.680052+0000","last_peered":"2024-09-13T08:45:33.680052+0000","last_clean":"2024-09-13T08:45:33.680052+0000","last_became_active":"2024-09-13T08:45:33.675233+0000","last_became_peered":"2024-09-13T08:45:33.675233+0000","last_unstale":"2024-09-13T08:45:33.680052+0000","last_undegraded":"2024-09-13T08:45:33.680052+0000","last_fullsized":"2024-09-13T08:45:33.680052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:41.676077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.692093+0000","last_change":"2024-09-13T08:45:32.659506+0000","last_active":"2024-09-13T08:45:33.692093+0000","last_peered":"2024-09-13T08:45:33.692093+0000","last_clean":"2024-09-13T08:45:33.692093+0000","last_became_active":"2024-09-13T08:45:32.659220+0000","last_became_peered":"2024-09-13T08:45:32.659220+0000","last_unstale":"2024-09-13T08:45:33.692093+0000","last_undegraded":"2024-09-13T08:45:33.692093+0000","last_fullsized":"2024-09-13T08:45:33.692093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:30.318592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.689473+0000","last_change":"2024-09-13T08:45:33.688805+0000","last_active":"2024-09-13T08:45:33.689473+0000","last_peered":"2024-09-13T08:45:33.689473+0000","last_clean":"2024-09-13T08:45:33.689473+0000","last_became_active":"2024-09-13T08:45:33.688534+0000","last_became_peered":"2024-09-13T08:45:33.688534+0000","last_unstale":"2024-09-13T08:45:33.689473+0000","last_undegraded":"2024-09-13T08:45:33.689473+0000","last_fullsized":"2024-09-13T08:45:33.689473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:11.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660382+0000","last_change":"2024-09-13T08:45:32.655084+0000","last_active":"2024-09-13T08:45:33.660382+0000","last_peered":"2024-09-13T08:45:33.660382+0000","last_clean":"2024-09-13T08:45:33.660382+0000","last_became_active":"2024-09-13T08:45:32.654795+0000","last_became_peered":"2024-09-13T08:45:32.654795+0000","last_unstale":"2024-09-13T08:45:33.660382+0000","last_undegraded":"2024-09-13T08:45:33.660382+0000","last_fullsized":"2024-09-13T08:45:33.660382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:23:00.745264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.888994+0000","last_change":"2024-09-13T08:45:33.672926+0000","last_active":"2024-09-13T08:45:34.888994+0000","last_peered":"2024-09-13T08:45:34.888994+0000","last_clean":"2024-09-13T08:45:34.888994+0000","last_became_active":"2024-09-13T08:45:33.672641+0000","last_became_peered":"2024-09-13T08:45:33.672641+0000","last_unstale":"2024-09-13T08:45:34.888994+0000","last_undegraded":"2024-09-13T08:45:34.888994+0000","last_fullsized":"2024-09-13T08:45:34.888994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:24.386769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.661857+0000","last_change":"2024-09-13T08:45:32.648855+0000","last_active":"2024-09-13T08:45:33.661857+0000","last_peered":"2024-09-13T08:45:33.661857+0000","last_clean":"2024-09-13T08:45:33.661857+0000","last_became_active":"2024-09-13T08:45:32.648572+0000","last_became_peered":"2024-09-13T08:45:32.648572+0000","last_unstale":"2024-09-13T08:45:33.661857+0000","last_undegraded":"2024-09-13T08:45:33.661857+0000","last_fullsized":"2024-09-13T08:45:33.661857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:11.383879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.888982+0000","last_change":"2024-09-13T08:45:33.663742+0000","last_active":"2024-09-13T08:45:34.888982+0000","last_peered":"2024-09-13T08:45:34.888982+0000","last_clean":"2024-09-13T08:45:34.888982+0000","last_became_active":"2024-09-13T08:45:33.663370+0000","last_became_peered":"2024-09-13T08:45:33.663370+0000","last_unstale":"2024-09-13T08:45:34.888982+0000","last_undegraded":"2024-09-13T08:45:34.888982+0000","last_fullsized":"2024-09-13T08:45:34.888982+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:19.194461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.664419+0000","last_change":"2024-09-13T08:45:32.651171+0000","last_active":"2024-09-13T08:45:33.664419+0000","last_peered":"2024-09-13T08:45:33.664419+0000","last_clean":"2024-09-13T08:45:33.664419+0000","last_became_active":"2024-09-13T08:45:32.650437+0000","last_became_peered":"2024-09-13T08:45:32.650437+0000","last_unstale":"2024-09-13T08:45:33.664419+0000","last_undegraded":"2024-09-13T08:45:33.664419+0000","last_fullsized":"2024-09-13T08:45:33.664419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:43.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.671316+0000","last_change":"2024-09-13T08:45:33.670268+0000","last_active":"2024-09-13T08:45:33.671316+0000","last_peered":"2024-09-13T08:45:33.671316+0000","last_clean":"2024-09-13T08:45:33.671316+0000","last_became_active":"2024-09-13T08:45:33.669974+0000","last_became_peered":"2024-09-13T08:45:33.669974+0000","last_unstale":"2024-09-13T08:45:33.671316+0000","last_undegraded":"2024-09-13T08:45:33.671316+0000","last_fullsized":"2024-09-13T08:45:33.671316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:14.031957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.657855+0000","last_change":"2024-09-13T08:45:32.652196+0000","last_active":"2024-09-13T08:45:33.657855+0000","last_peered":"2024-09-13T08:45:33.657855+0000","last_clean":"2024-09-13T08:45:33.657855+0000","last_became_active":"2024-09-13T08:45:32.651851+0000","last_became_peered":"2024-09-13T08:45:32.651851+0000","last_unstale":"2024-09-13T08:45:33.657855+0000","last_undegraded":"2024-09-13T08:45:33.657855+0000","last_fullsized":"2024-09-13T08:45:33.657855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:48:29.415637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.668002+0000","last_change":"2024-09-13T08:45:33.664149+0000","last_active":"2024-09-13T08:45:33.668002+0000","last_peered":"2024-09-13T08:45:33.668002+0000","last_clean":"2024-09-13T08:45:33.668002+0000","last_became_active":"2024-09-13T08:45:33.663531+0000","last_became_peered":"2024-09-13T08:45:33.663531+0000","last_unstale":"2024-09-13T08:45:33.668002+0000","last_undegraded":"2024-09-13T08:45:33.668002+0000","last_fullsized":"2024-09-13T08:45:33.668002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:18:36.348488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.694733+0000","last_change":"2024-09-13T08:45:32.660012+0000","last_active":"2024-09-13T08:45:33.694733+0000","last_peered":"2024-09-13T08:45:33.694733+0000","last_clean":"2024-09-13T08:45:33.694733+0000","last_became_active":"2024-09-13T08:45:32.659398+0000","last_became_peered":"2024-09-13T08:45:32.659398+0000","last_unstale":"2024-09-13T08:45:33.694733+0000","last_undegraded":"2024-09-13T08:45:33.694733+0000","last_fullsized":"2024-09-13T08:45:33.694733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:12.195364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.667819+0000","last_change":"2024-09-13T08:45:33.660680+0000","last_active":"2024-09-13T08:45:33.667819+0000","last_peered":"2024-09-13T08:45:33.667819+0000","last_clean":"2024-09-13T08:45:33.667819+0000","last_became_active":"2024-09-13T08:45:33.660407+0000","last_became_peered":"2024-09-13T08:45:33.660407+0000","last_unstale":"2024-09-13T08:45:33.667819+0000","last_undegraded":"2024-09-13T08:45:33.667819+0000","last_fullsized":"2024-09-13T08:45:33.667819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:21:26.352682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684746+0000","last_change":"2024-09-13T08:45:32.661277+0000","last_active":"2024-09-13T08:45:33.684746+0000","last_peered":"2024-09-13T08:45:33.684746+0000","last_clean":"2024-09-13T08:45:33.684746+0000","last_became_active":"2024-09-13T08:45:32.660211+0000","last_became_peered":"2024-09-13T08:45:32.660211+0000","last_unstale":"2024-09-13T08:45:33.684746+0000","last_undegraded":"2024-09-13T08:45:33.684746+0000","last_fullsized":"2024-09-13T08:45:33.684746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:01.561248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.684943+0000","last_change":"2024-09-13T08:45:33.683345+0000","last_active":"2024-09-13T08:45:33.684943+0000","last_peered":"2024-09-13T08:45:33.684943+0000","last_clean":"2024-09-13T08:45:33.684943+0000","last_became_active":"2024-09-13T08:45:33.682992+0000","last_became_peered":"2024-09-13T08:45:33.682992+0000","last_unstale":"2024-09-13T08:45:33.684943+0000","last_undegraded":"2024-09-13T08:45:33.684943+0000","last_fullsized":"2024-09-13T08:45:33.684943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:47.803484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.660890+0000","last_change":"2024-09-13T08:45:32.646650+0000","last_active":"2024-09-13T08:45:33.660890+0000","last_peered":"2024-09-13T08:45:33.660890+0000","last_clean":"2024-09-13T08:45:33.660890+0000","last_became_active":"2024-09-13T08:45:32.646362+0000","last_became_peered":"2024-09-13T08:45:32.646362+0000","last_unstale":"2024-09-13T08:45:33.660890+0000","last_undegraded":"2024-09-13T08:45:33.660890+0000","last_fullsized":"2024-09-13T08:45:33.660890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:48:47.580467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.680628+0000","last_change":"2024-09-13T08:45:33.678066+0000","last_active":"2024-09-13T08:45:33.680628+0000","last_peered":"2024-09-13T08:45:33.680628+0000","last_clean":"2024-09-13T08:45:33.680628+0000","last_became_active":"2024-09-13T08:45:33.674778+0000","last_became_peered":"2024-09-13T08:45:33.674778+0000","last_unstale":"2024-09-13T08:45:33.680628+0000","last_undegraded":"2024-09-13T08:45:33.680628+0000","last_fullsized":"2024-09-13T08:45:33.680628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:56.987435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.661122+0000","last_change":"2024-09-13T08:45:32.646318+0000","last_active":"2024-09-13T08:45:33.661122+0000","last_peered":"2024-09-13T08:45:33.661122+0000","last_clean":"2024-09-13T08:45:33.661122+0000","last_became_active":"2024-09-13T08:45:32.645801+0000","last_became_peered":"2024-09-13T08:45:32.645801+0000","last_unstale":"2024-09-13T08:45:33.661122+0000","last_undegraded":"2024-09-13T08:45:33.661122+0000","last_fullsized":"2024-09-13T08:45:33.661122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:40.323104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.672925+0000","last_change":"2024-09-13T08:45:33.670611+0000","last_active":"2024-09-13T08:45:33.672925+0000","last_peered":"2024-09-13T08:45:33.672925+0000","last_clean":"2024-09-13T08:45:33.672925+0000","last_became_active":"2024-09-13T08:45:33.670334+0000","last_became_peered":"2024-09-13T08:45:33.670334+0000","last_unstale":"2024-09-13T08:45:33.672925+0000","last_undegraded":"2024-09-13T08:45:33.672925+0000","last_fullsized":"2024-09-13T08:45:33.672925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:08.841526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.662228+0000","last_change":"2024-09-13T08:45:32.650061+0000","last_active":"2024-09-13T08:45:33.662228+0000","last_peered":"2024-09-13T08:45:33.662228+0000","last_clean":"2024-09-13T08:45:33.662228+0000","last_became_active":"2024-09-13T08:45:32.649743+0000","last_became_peered":"2024-09-13T08:45:32.649743+0000","last_unstale":"2024-09-13T08:45:33.662228+0000","last_undegraded":"2024-09-13T08:45:33.662228+0000","last_fullsized":"2024-09-13T08:45:33.662228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:16.817675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.691205+0000","last_change":"2024-09-13T08:45:33.682926+0000","last_active":"2024-09-13T08:45:33.691205+0000","last_peered":"2024-09-13T08:45:33.691205+0000","last_clean":"2024-09-13T08:45:33.691205+0000","last_became_active":"2024-09-13T08:45:33.681036+0000","last_became_peered":"2024-09-13T08:45:33.681036+0000","last_unstale":"2024-09-13T08:45:33.691205+0000","last_undegraded":"2024-09-13T08:45:33.691205+0000","last_fullsized":"2024-09-13T08:45:33.691205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:31.247460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.655978+0000","last_change":"2024-09-13T08:45:32.651908+0000","last_active":"2024-09-13T08:45:33.655978+0000","last_peered":"2024-09-13T08:45:33.655978+0000","last_clean":"2024-09-13T08:45:33.655978+0000","last_became_active":"2024-09-13T08:45:32.651622+0000","last_became_peered":"2024-09-13T08:45:32.651622+0000","last_unstale":"2024-09-13T08:45:33.655978+0000","last_undegraded":"2024-09-13T08:45:33.655978+0000","last_fullsized":"2024-09-13T08:45:33.655978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:32.504688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.889297+0000","last_change":"2024-09-13T08:45:33.673105+0000","last_active":"2024-09-13T08:45:34.889297+0000","last_peered":"2024-09-13T08:45:34.889297+0000","last_clean":"2024-09-13T08:45:34.889297+0000","last_became_active":"2024-09-13T08:45:33.672786+0000","last_became_peered":"2024-09-13T08:45:33.672786+0000","last_unstale":"2024-09-13T08:45:34.889297+0000","last_undegraded":"2024-09-13T08:45:34.889297+0000","last_fullsized":"2024-09-13T08:45:34.889297+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:54.590873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.664624+0000","last_change":"2024-09-13T08:45:32.656890+0000","last_active":"2024-09-13T08:45:33.664624+0000","last_peered":"2024-09-13T08:45:33.664624+0000","last_clean":"2024-09-13T08:45:33.664624+0000","last_became_active":"2024-09-13T08:45:32.656592+0000","last_became_peered":"2024-09-13T08:45:32.656592+0000","last_unstale":"2024-09-13T08:45:33.664624+0000","last_undegraded":"2024-09-13T08:45:33.664624+0000","last_fullsized":"2024-09-13T08:45:33.664624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:38.811210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.675107+0000","last_change":"2024-09-13T08:45:33.669976+0000","last_active":"2024-09-13T08:45:33.675107+0000","last_peered":"2024-09-13T08:45:33.675107+0000","last_clean":"2024-09-13T08:45:33.675107+0000","last_became_active":"2024-09-13T08:45:33.669693+0000","last_became_peered":"2024-09-13T08:45:33.669693+0000","last_unstale":"2024-09-13T08:45:33.675107+0000","last_undegraded":"2024-09-13T08:45:33.675107+0000","last_fullsized":"2024-09-13T08:45:33.675107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:27:36.696332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.658907+0000","last_change":"2024-09-13T08:45:32.654218+0000","last_active":"2024-09-13T08:45:33.658907+0000","last_peered":"2024-09-13T08:45:33.658907+0000","last_clean":"2024-09-13T08:45:33.658907+0000","last_became_active":"2024-09-13T08:45:32.653519+0000","last_became_peered":"2024-09-13T08:45:32.653519+0000","last_unstale":"2024-09-13T08:45:33.658907+0000","last_undegraded":"2024-09-13T08:45:33.658907+0000","last_fullsized":"2024-09-13T08:45:33.658907+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:33.012849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681797+0000","last_change":"2024-09-13T08:45:33.679432+0000","last_active":"2024-09-13T08:45:33.681797+0000","last_peered":"2024-09-13T08:45:33.681797+0000","last_clean":"2024-09-13T08:45:33.681797+0000","last_became_active":"2024-09-13T08:45:33.679129+0000","last_became_peered":"2024-09-13T08:45:33.679129+0000","last_unstale":"2024-09-13T08:45:33.681797+0000","last_undegraded":"2024-09-13T08:45:33.681797+0000","last_fullsized":"2024-09-13T08:45:33.681797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:31.247460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.681998+0000","last_change":"2024-09-13T08:45:32.647002+0000","last_active":"2024-09-13T08:45:33.681998+0000","last_peered":"2024-09-13T08:45:33.681998+0000","last_clean":"2024-09-13T08:45:33.681998+0000","last_became_active":"2024-09-13T08:45:32.645456+0000","last_became_peered":"2024-09-13T08:45:32.645456+0000","last_unstale":"2024-09-13T08:45:33.681998+0000","last_undegraded":"2024-09-13T08:45:33.681998+0000","last_fullsized":"2024-09-13T08:45:33.681998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:41.793310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":212,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T08:45:33.652555+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:45:33.652555+0000","last_peered":"2024-09-13T08:45:33.652555+0000","last_clean":"2024-09-13T08:45:33.652555+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:45:33.652555+0000","last_undegraded":"2024-09-13T08:45:33.652555+0000","last_fullsized":"2024-09-13T08:45:33.652555+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509540,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50120,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321720,"statfs":{"total":96636764160,"available":96585441280,"internally_reserved":0,"allocated":905216,"data_stored":380942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50696,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321144,"statfs":{"total":96636764160,"available":96584851456,"internally_reserved":0,"allocated":1490944,"data_stored":969788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509541,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54288,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317552,"statfs":{"total":96636764160,"available":96581173248,"internally_reserved":0,"allocated":905216,"data_stored":380024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":880640,"data_stored":372499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":897024,"data_stored":379912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509542,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46532,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325308,"statfs":{"total":96636764160,"available":96589115392,"internally_reserved":0,"allocated":1486848,"data_stored":969698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":897024,"data_stored":379400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50140,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321700,"statfs":{"total":96636764160,"available":96585420800,"internally_reserved":0,"allocated":921600,"data_stored":387332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:45:42.450 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:45:42.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.605+0000 7f1635c00640 1 -- 172.21.15.44:0/1440916501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16300cdc20 msgr2=0x7f16300cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:42.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.605+0000 7f1635c00640 1 --2- 172.21.15.44:0/1440916501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16300cdc20 0x7f16300cb360 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f16300b3730 tx=0x7f161c01c8a0 comp rx=0 tx=0).stop 2024-09-13T08:45:42.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.605+0000 7f1635c00640 1 -- 172.21.15.44:0/1440916501 shutdown_connections 2024-09-13T08:45:42.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.605+0000 7f1635c00640 1 --2- 172.21.15.44:0/1440916501 >> v2:172.21.15.129:3300/0 conn(0x7f16300cb8a0 0x7f16300cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:42.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.605+0000 7f1635c00640 1 --2- 172.21.15.44:0/1440916501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16300cdc20 0x7f16300cb360 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:42.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.605+0000 7f1635c00640 1 -- 172.21.15.44:0/1440916501 >> 172.21.15.44:0/1440916501 conn(0x7f163006dd30 msgr2=0x7f163006e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:42.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.605+0000 7f1635c00640 1 -- 172.21.15.44:0/1440916501 shutdown_connections 2024-09-13T08:45:42.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.605+0000 7f1635c00640 1 -- 172.21.15.44:0/1440916501 wait complete. 2024-09-13T08:45:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635c00640 1 Processor -- start 2024-09-13T08:45:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635c00640 1 -- start start 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16300cb8a0 0x7f1630165340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16300cdc20 0x7f16300b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1630155080 0x7f1630163b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1630167b70 con 0x7f16300cdc20 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f16300cf9e0 con 0x7f1630155080 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1630167cf0 con 0x7f16300cb8a0 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f162f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16300cdc20 0x7f16300b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f162f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16300cdc20 0x7f16300b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42288/0 (socket says 172.21.15.44:42288) 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f162f400640 1 -- 172.21.15.44:0/1417693730 learned_addr learned my addr 172.21.15.44:0/1417693730 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f162fe00640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16300cb8a0 0x7f1630165340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635200640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1630155080 0x7f1630163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f162f400640 1 -- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16300cb8a0 msgr2=0x7f1630165340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f162f400640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16300cb8a0 0x7f1630165340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f162f400640 1 -- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1630155080 msgr2=0x7f1630163b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f162f400640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1630155080 0x7f1630163b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f162f400640 1 -- 172.21.15.44:0/1417693730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16300b7070 con 0x7f16300cdc20 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635200640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1630155080 0x7f1630163b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f162f400640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16300cdc20 0x7f16300b1f30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f161c000c10 tx=0x7f161c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1627e00640 1 -- 172.21.15.44:0/1417693730 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0bb070 con 0x7f16300cdc20 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1627e00640 1 -- 172.21.15.44:0/1417693730 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f161c0053e0 con 0x7f16300cdc20 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1627e00640 1 -- 172.21.15.44:0/1417693730 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0b2040 con 0x7f16300cdc20 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635c00640 1 -- 172.21.15.44:0/1417693730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16300b7350 con 0x7f16300cdc20 2024-09-13T08:45:42.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.609+0000 7f1635c00640 1 -- 172.21.15.44:0/1417693730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16300b7910 con 0x7f16300cdc20 2024-09-13T08:45:42.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.613+0000 7f1635c00640 1 -- 172.21.15.44:0/1417693730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16300b7c40 con 0x7f16300cdc20 2024-09-13T08:45:42.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.613+0000 7f1627e00640 1 -- 172.21.15.44:0/1417693730 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f161c0be070 con 0x7f16300cdc20 2024-09-13T08:45:42.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.613+0000 7f1627e00640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1610073b20 0x7f1610075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:42.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.613+0000 7f1627e00640 1 -- 172.21.15.44:0/1417693730 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f161c12c260 con 0x7f16300cdc20 2024-09-13T08:45:42.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.613+0000 7f162fe00640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1610073b20 0x7f1610075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:42.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.613+0000 7f162fe00640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1610073b20 0x7f1610075fe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f16200012f0 tx=0x7f1620009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:42.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.617+0000 7f1627e00640 1 -- 172.21.15.44:0/1417693730 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f161c016490 con 0x7f16300cdc20 2024-09-13T08:45:42.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.793+0000 7f1635c00640 1 -- 172.21.15.44:0/1417693730 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f163006d510 con 0x7f1610073b20 2024-09-13T08:45:42.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.797+0000 7f1627e00640 1 -- 172.21.15.44:0/1417693730 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252009 (secure 0 0 0) 0x7f163006d510 con 0x7f1610073b20 2024-09-13T08:45:42.803 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:42.804 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:45:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 -- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1610073b20 msgr2=0x7f1610075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1610073b20 0x7f1610075fe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f16200012f0 tx=0x7f1620009040 comp rx=0 tx=0).stop 2024-09-13T08:45:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 -- 172.21.15.44:0/1417693730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16300cdc20 msgr2=0x7f16300b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16300cdc20 0x7f16300b1f30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f161c000c10 tx=0x7f161c004600 comp rx=0 tx=0).stop 2024-09-13T08:45:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 -- 172.21.15.44:0/1417693730 shutdown_connections 2024-09-13T08:45:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1610073b20 0x7f1610075fe0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1630155080 0x7f1630163b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16300cdc20 0x7f16300b1f30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 --2- 172.21.15.44:0/1417693730 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16300cb8a0 0x7f1630165340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 -- 172.21.15.44:0/1417693730 >> 172.21.15.44:0/1417693730 conn(0x7f163006dd30 msgr2=0x7f1630154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 -- 172.21.15.44:0/1417693730 shutdown_connections 2024-09-13T08:45:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:42.801+0000 7f1635c00640 1 -- 172.21.15.44:0/1417693730 wait complete. 2024-09-13T08:45:42.825 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-13T08:45:42.642622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402216,"kb_used_data":8296,"kb_used_omap":12,"kb_used_meta":393715,"kb_avail":754572504,"statfs":{"total":773094113280,"available":772682244096,"internally_reserved":0,"allocated":8495104,"data_stored":4271044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":403164721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.990133"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654225+0000","last_change":"2024-09-13T08:45:33.684143+0000","last_active":"2024-09-13T08:45:34.654225+0000","last_peered":"2024-09-13T08:45:34.654225+0000","last_clean":"2024-09-13T08:45:34.654225+0000","last_became_active":"2024-09-13T08:45:33.683889+0000","last_became_peered":"2024-09-13T08:45:33.683889+0000","last_unstale":"2024-09-13T08:45:34.654225+0000","last_undegraded":"2024-09-13T08:45:34.654225+0000","last_fullsized":"2024-09-13T08:45:34.654225+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:24:46.868667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.651175+0000","last_change":"2024-09-13T08:45:33.685799+0000","last_active":"2024-09-13T08:45:34.651175+0000","last_peered":"2024-09-13T08:45:34.651175+0000","last_clean":"2024-09-13T08:45:34.651175+0000","last_became_active":"2024-09-13T08:45:33.685475+0000","last_became_peered":"2024-09-13T08:45:33.685475+0000","last_unstale":"2024-09-13T08:45:34.651175+0000","last_undegraded":"2024-09-13T08:45:34.651175+0000","last_fullsized":"2024-09-13T08:45:34.651175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:44.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.955480+0000","last_change":"2024-09-13T08:45:33.683135+0000","last_active":"2024-09-13T08:45:34.955480+0000","last_peered":"2024-09-13T08:45:34.955480+0000","last_clean":"2024-09-13T08:45:34.955480+0000","last_became_active":"2024-09-13T08:45:33.682789+0000","last_became_peered":"2024-09-13T08:45:33.682789+0000","last_unstale":"2024-09-13T08:45:34.955480+0000","last_undegraded":"2024-09-13T08:45:34.955480+0000","last_fullsized":"2024-09-13T08:45:34.955480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:08.422179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.955475+0000","last_change":"2024-09-13T08:45:33.682881+0000","last_active":"2024-09-13T08:45:34.955475+0000","last_peered":"2024-09-13T08:45:34.955475+0000","last_clean":"2024-09-13T08:45:34.955475+0000","last_became_active":"2024-09-13T08:45:33.681260+0000","last_became_peered":"2024-09-13T08:45:33.681260+0000","last_unstale":"2024-09-13T08:45:34.955475+0000","last_undegraded":"2024-09-13T08:45:34.955475+0000","last_fullsized":"2024-09-13T08:45:34.955475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:18:06.747912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654757+0000","last_change":"2024-09-13T08:45:33.673961+0000","last_active":"2024-09-13T08:45:34.654757+0000","last_peered":"2024-09-13T08:45:34.654757+0000","last_clean":"2024-09-13T08:45:34.654757+0000","last_became_active":"2024-09-13T08:45:33.673661+0000","last_became_peered":"2024-09-13T08:45:33.673661+0000","last_unstale":"2024-09-13T08:45:34.654757+0000","last_undegraded":"2024-09-13T08:45:34.654757+0000","last_fullsized":"2024-09-13T08:45:34.654757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:08.453920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.954238+0000","last_change":"2024-09-13T08:45:33.686731+0000","last_active":"2024-09-13T08:45:34.954238+0000","last_peered":"2024-09-13T08:45:34.954238+0000","last_clean":"2024-09-13T08:45:34.954238+0000","last_became_active":"2024-09-13T08:45:33.686352+0000","last_became_peered":"2024-09-13T08:45:33.686352+0000","last_unstale":"2024-09-13T08:45:34.954238+0000","last_undegraded":"2024-09-13T08:45:34.954238+0000","last_fullsized":"2024-09-13T08:45:34.954238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:40.457494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656209+0000","last_change":"2024-09-13T08:45:33.660192+0000","last_active":"2024-09-13T08:45:34.656209+0000","last_peered":"2024-09-13T08:45:34.656209+0000","last_clean":"2024-09-13T08:45:34.656209+0000","last_became_active":"2024-09-13T08:45:33.659425+0000","last_became_peered":"2024-09-13T08:45:33.659425+0000","last_unstale":"2024-09-13T08:45:34.656209+0000","last_undegraded":"2024-09-13T08:45:34.656209+0000","last_fullsized":"2024-09-13T08:45:34.656209+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:39:21.820225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653491+0000","last_change":"2024-09-13T08:45:33.671839+0000","last_active":"2024-09-13T08:45:34.653491+0000","last_peered":"2024-09-13T08:45:34.653491+0000","last_clean":"2024-09-13T08:45:34.653491+0000","last_became_active":"2024-09-13T08:45:33.671538+0000","last_became_peered":"2024-09-13T08:45:33.671538+0000","last_unstale":"2024-09-13T08:45:34.653491+0000","last_undegraded":"2024-09-13T08:45:34.653491+0000","last_fullsized":"2024-09-13T08:45:34.653491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:59.657764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.955688+0000","last_change":"2024-09-13T08:45:33.683295+0000","last_active":"2024-09-13T08:45:34.955688+0000","last_peered":"2024-09-13T08:45:34.955688+0000","last_clean":"2024-09-13T08:45:34.955688+0000","last_became_active":"2024-09-13T08:45:33.681988+0000","last_became_peered":"2024-09-13T08:45:33.681988+0000","last_unstale":"2024-09-13T08:45:34.955688+0000","last_undegraded":"2024-09-13T08:45:34.955688+0000","last_fullsized":"2024-09-13T08:45:34.955688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:10:17.409549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656780+0000","last_change":"2024-09-13T08:45:33.667868+0000","last_active":"2024-09-13T08:45:34.656780+0000","last_peered":"2024-09-13T08:45:34.656780+0000","last_clean":"2024-09-13T08:45:34.656780+0000","last_became_active":"2024-09-13T08:45:33.667543+0000","last_became_peered":"2024-09-13T08:45:33.667543+0000","last_unstale":"2024-09-13T08:45:34.656780+0000","last_undegraded":"2024-09-13T08:45:34.656780+0000","last_fullsized":"2024-09-13T08:45:34.656780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:09:50.658366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653995+0000","last_change":"2024-09-13T08:45:33.689087+0000","last_active":"2024-09-13T08:45:34.653995+0000","last_peered":"2024-09-13T08:45:34.653995+0000","last_clean":"2024-09-13T08:45:34.653995+0000","last_became_active":"2024-09-13T08:45:33.688839+0000","last_became_peered":"2024-09-13T08:45:33.688839+0000","last_unstale":"2024-09-13T08:45:34.653995+0000","last_undegraded":"2024-09-13T08:45:34.653995+0000","last_fullsized":"2024-09-13T08:45:34.653995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:06.045101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653642+0000","last_change":"2024-09-13T08:45:33.664987+0000","last_active":"2024-09-13T08:45:34.653642+0000","last_peered":"2024-09-13T08:45:34.653642+0000","last_clean":"2024-09-13T08:45:34.653642+0000","last_became_active":"2024-09-13T08:45:33.664707+0000","last_became_peered":"2024-09-13T08:45:33.664707+0000","last_unstale":"2024-09-13T08:45:34.653642+0000","last_undegraded":"2024-09-13T08:45:34.653642+0000","last_fullsized":"2024-09-13T08:45:34.653642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:44.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.955583+0000","last_change":"2024-09-13T08:45:33.681174+0000","last_active":"2024-09-13T08:45:34.955583+0000","last_peered":"2024-09-13T08:45:34.955583+0000","last_clean":"2024-09-13T08:45:34.955583+0000","last_became_active":"2024-09-13T08:45:33.680680+0000","last_became_peered":"2024-09-13T08:45:33.680680+0000","last_unstale":"2024-09-13T08:45:34.955583+0000","last_undegraded":"2024-09-13T08:45:34.955583+0000","last_fullsized":"2024-09-13T08:45:34.955583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:38.549474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653193+0000","last_change":"2024-09-13T08:45:33.668789+0000","last_active":"2024-09-13T08:45:34.653193+0000","last_peered":"2024-09-13T08:45:34.653193+0000","last_clean":"2024-09-13T08:45:34.653193+0000","last_became_active":"2024-09-13T08:45:33.668462+0000","last_became_peered":"2024-09-13T08:45:33.668462+0000","last_unstale":"2024-09-13T08:45:34.653193+0000","last_undegraded":"2024-09-13T08:45:34.653193+0000","last_fullsized":"2024-09-13T08:45:34.653193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:05.822529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.652977+0000","last_change":"2024-09-13T08:45:33.671105+0000","last_active":"2024-09-13T08:45:34.652977+0000","last_peered":"2024-09-13T08:45:34.652977+0000","last_clean":"2024-09-13T08:45:34.652977+0000","last_became_active":"2024-09-13T08:45:33.670718+0000","last_became_peered":"2024-09-13T08:45:33.670718+0000","last_unstale":"2024-09-13T08:45:34.652977+0000","last_undegraded":"2024-09-13T08:45:34.652977+0000","last_fullsized":"2024-09-13T08:45:34.652977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:34.178628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653629+0000","last_change":"2024-09-13T08:45:33.686654+0000","last_active":"2024-09-13T08:45:34.653629+0000","last_peered":"2024-09-13T08:45:34.653629+0000","last_clean":"2024-09-13T08:45:34.653629+0000","last_became_active":"2024-09-13T08:45:33.686375+0000","last_became_peered":"2024-09-13T08:45:33.686375+0000","last_unstale":"2024-09-13T08:45:34.653629+0000","last_undegraded":"2024-09-13T08:45:34.653629+0000","last_fullsized":"2024-09-13T08:45:34.653629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:08.680395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.655037+0000","last_change":"2024-09-13T08:45:33.670204+0000","last_active":"2024-09-13T08:45:34.655037+0000","last_peered":"2024-09-13T08:45:34.655037+0000","last_clean":"2024-09-13T08:45:34.655037+0000","last_became_active":"2024-09-13T08:45:33.669936+0000","last_became_peered":"2024-09-13T08:45:33.669936+0000","last_unstale":"2024-09-13T08:45:34.655037+0000","last_undegraded":"2024-09-13T08:45:34.655037+0000","last_fullsized":"2024-09-13T08:45:34.655037+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:27.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.655081+0000","last_change":"2024-09-13T08:45:33.663820+0000","last_active":"2024-09-13T08:45:34.655081+0000","last_peered":"2024-09-13T08:45:34.655081+0000","last_clean":"2024-09-13T08:45:34.655081+0000","last_became_active":"2024-09-13T08:45:33.663147+0000","last_became_peered":"2024-09-13T08:45:33.663147+0000","last_unstale":"2024-09-13T08:45:34.655081+0000","last_undegraded":"2024-09-13T08:45:34.655081+0000","last_fullsized":"2024-09-13T08:45:34.655081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:20:46.695925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654019+0000","last_change":"2024-09-13T08:45:33.680218+0000","last_active":"2024-09-13T08:45:34.654019+0000","last_peered":"2024-09-13T08:45:34.654019+0000","last_clean":"2024-09-13T08:45:34.654019+0000","last_became_active":"2024-09-13T08:45:33.679938+0000","last_became_peered":"2024-09-13T08:45:33.679938+0000","last_unstale":"2024-09-13T08:45:34.654019+0000","last_undegraded":"2024-09-13T08:45:34.654019+0000","last_fullsized":"2024-09-13T08:45:34.654019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:10:58.138365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654837+0000","last_change":"2024-09-13T08:45:33.674349+0000","last_active":"2024-09-13T08:45:34.654837+0000","last_peered":"2024-09-13T08:45:34.654837+0000","last_clean":"2024-09-13T08:45:34.654837+0000","last_became_active":"2024-09-13T08:45:33.674004+0000","last_became_peered":"2024-09-13T08:45:33.674004+0000","last_unstale":"2024-09-13T08:45:34.654837+0000","last_undegraded":"2024-09-13T08:45:34.654837+0000","last_fullsized":"2024-09-13T08:45:34.654837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:05.891953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.650942+0000","last_change":"2024-09-13T08:45:33.679260+0000","last_active":"2024-09-13T08:45:34.650942+0000","last_peered":"2024-09-13T08:45:34.650942+0000","last_clean":"2024-09-13T08:45:34.650942+0000","last_became_active":"2024-09-13T08:45:33.678976+0000","last_became_peered":"2024-09-13T08:45:33.678976+0000","last_unstale":"2024-09-13T08:45:34.650942+0000","last_undegraded":"2024-09-13T08:45:34.650942+0000","last_fullsized":"2024-09-13T08:45:34.650942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:29:54.561617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.954293+0000","last_change":"2024-09-13T08:45:33.666891+0000","last_active":"2024-09-13T08:45:34.954293+0000","last_peered":"2024-09-13T08:45:34.954293+0000","last_clean":"2024-09-13T08:45:34.954293+0000","last_became_active":"2024-09-13T08:45:33.666585+0000","last_became_peered":"2024-09-13T08:45:33.666585+0000","last_unstale":"2024-09-13T08:45:34.954293+0000","last_undegraded":"2024-09-13T08:45:34.954293+0000","last_fullsized":"2024-09-13T08:45:34.954293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:06.562957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656235+0000","last_change":"2024-09-13T08:45:33.670178+0000","last_active":"2024-09-13T08:45:34.656235+0000","last_peered":"2024-09-13T08:45:34.656235+0000","last_clean":"2024-09-13T08:45:34.656235+0000","last_became_active":"2024-09-13T08:45:33.669687+0000","last_became_peered":"2024-09-13T08:45:33.669687+0000","last_unstale":"2024-09-13T08:45:34.656235+0000","last_undegraded":"2024-09-13T08:45:34.656235+0000","last_fullsized":"2024-09-13T08:45:34.656235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:50.243054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653718+0000","last_change":"2024-09-13T08:45:33.677052+0000","last_active":"2024-09-13T08:45:34.653718+0000","last_peered":"2024-09-13T08:45:34.653718+0000","last_clean":"2024-09-13T08:45:34.653718+0000","last_became_active":"2024-09-13T08:45:33.676767+0000","last_became_peered":"2024-09-13T08:45:33.676767+0000","last_unstale":"2024-09-13T08:45:34.653718+0000","last_undegraded":"2024-09-13T08:45:34.653718+0000","last_fullsized":"2024-09-13T08:45:34.653718+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:37.538590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656819+0000","last_change":"2024-09-13T08:45:33.670240+0000","last_active":"2024-09-13T08:45:34.656819+0000","last_peered":"2024-09-13T08:45:34.656819+0000","last_clean":"2024-09-13T08:45:34.656819+0000","last_became_active":"2024-09-13T08:45:33.669952+0000","last_became_peered":"2024-09-13T08:45:33.669952+0000","last_unstale":"2024-09-13T08:45:34.656819+0000","last_undegraded":"2024-09-13T08:45:34.656819+0000","last_fullsized":"2024-09-13T08:45:34.656819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:51:58.856035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653944+0000","last_change":"2024-09-13T08:45:33.690462+0000","last_active":"2024-09-13T08:45:34.653944+0000","last_peered":"2024-09-13T08:45:34.653944+0000","last_clean":"2024-09-13T08:45:34.653944+0000","last_became_active":"2024-09-13T08:45:33.690197+0000","last_became_peered":"2024-09-13T08:45:33.690197+0000","last_unstale":"2024-09-13T08:45:34.653944+0000","last_undegraded":"2024-09-13T08:45:34.653944+0000","last_fullsized":"2024-09-13T08:45:34.653944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:57.368479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654865+0000","last_change":"2024-09-13T08:45:33.663806+0000","last_active":"2024-09-13T08:45:34.654865+0000","last_peered":"2024-09-13T08:45:34.654865+0000","last_clean":"2024-09-13T08:45:34.654865+0000","last_became_active":"2024-09-13T08:45:33.663498+0000","last_became_peered":"2024-09-13T08:45:33.663498+0000","last_unstale":"2024-09-13T08:45:34.654865+0000","last_undegraded":"2024-09-13T08:45:34.654865+0000","last_fullsized":"2024-09-13T08:45:34.654865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:11:19.273536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.650869+0000","last_change":"2024-09-13T08:45:33.674360+0000","last_active":"2024-09-13T08:45:34.650869+0000","last_peered":"2024-09-13T08:45:34.650869+0000","last_clean":"2024-09-13T08:45:34.650869+0000","last_became_active":"2024-09-13T08:45:33.673982+0000","last_became_peered":"2024-09-13T08:45:33.673982+0000","last_unstale":"2024-09-13T08:45:34.650869+0000","last_undegraded":"2024-09-13T08:45:34.650869+0000","last_fullsized":"2024-09-13T08:45:34.650869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:29.126046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.955599+0000","last_change":"2024-09-13T08:45:33.689213+0000","last_active":"2024-09-13T08:45:34.955599+0000","last_peered":"2024-09-13T08:45:34.955599+0000","last_clean":"2024-09-13T08:45:34.955599+0000","last_became_active":"2024-09-13T08:45:33.688971+0000","last_became_peered":"2024-09-13T08:45:33.688971+0000","last_unstale":"2024-09-13T08:45:34.955599+0000","last_undegraded":"2024-09-13T08:45:34.955599+0000","last_fullsized":"2024-09-13T08:45:34.955599+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:03.437540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653681+0000","last_change":"2024-09-13T08:45:33.672566+0000","last_active":"2024-09-13T08:45:34.653681+0000","last_peered":"2024-09-13T08:45:34.653681+0000","last_clean":"2024-09-13T08:45:34.653681+0000","last_became_active":"2024-09-13T08:45:33.672302+0000","last_became_peered":"2024-09-13T08:45:33.672302+0000","last_unstale":"2024-09-13T08:45:34.653681+0000","last_undegraded":"2024-09-13T08:45:34.653681+0000","last_fullsized":"2024-09-13T08:45:34.653681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:53.265496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.650985+0000","last_change":"2024-09-13T08:45:33.675233+0000","last_active":"2024-09-13T08:45:34.650985+0000","last_peered":"2024-09-13T08:45:34.650985+0000","last_clean":"2024-09-13T08:45:34.650985+0000","last_became_active":"2024-09-13T08:45:33.674920+0000","last_became_peered":"2024-09-13T08:45:33.674920+0000","last_unstale":"2024-09-13T08:45:34.650985+0000","last_undegraded":"2024-09-13T08:45:34.650985+0000","last_fullsized":"2024-09-13T08:45:34.650985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:34.178628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.651307+0000","last_change":"2024-09-13T08:45:33.679637+0000","last_active":"2024-09-13T08:45:34.651307+0000","last_peered":"2024-09-13T08:45:34.651307+0000","last_clean":"2024-09-13T08:45:34.651307+0000","last_became_active":"2024-09-13T08:45:33.679391+0000","last_became_peered":"2024-09-13T08:45:33.679391+0000","last_unstale":"2024-09-13T08:45:34.651307+0000","last_undegraded":"2024-09-13T08:45:34.651307+0000","last_fullsized":"2024-09-13T08:45:34.651307+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:33:36.731420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654171+0000","last_change":"2024-09-13T08:45:33.688757+0000","last_active":"2024-09-13T08:45:34.654171+0000","last_peered":"2024-09-13T08:45:34.654171+0000","last_clean":"2024-09-13T08:45:34.654171+0000","last_became_active":"2024-09-13T08:45:33.687386+0000","last_became_peered":"2024-09-13T08:45:33.687386+0000","last_unstale":"2024-09-13T08:45:34.654171+0000","last_undegraded":"2024-09-13T08:45:34.654171+0000","last_fullsized":"2024-09-13T08:45:34.654171+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:28.517472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"74'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.954377+0000","last_change":"2024-09-13T08:45:32.657493+0000","last_active":"2024-09-13T08:45:34.954377+0000","last_peered":"2024-09-13T08:45:34.954377+0000","last_clean":"2024-09-13T08:45:34.954377+0000","last_became_active":"2024-09-13T08:45:32.656217+0000","last_became_peered":"2024-09-13T08:45:32.656217+0000","last_unstale":"2024-09-13T08:45:34.954377+0000","last_undegraded":"2024-09-13T08:45:34.954377+0000","last_fullsized":"2024-09-13T08:45:34.954377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:13.023500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.955753+0000","last_change":"2024-09-13T08:45:33.678452+0000","last_active":"2024-09-13T08:45:34.955753+0000","last_peered":"2024-09-13T08:45:34.955753+0000","last_clean":"2024-09-13T08:45:34.955753+0000","last_became_active":"2024-09-13T08:45:33.678224+0000","last_became_peered":"2024-09-13T08:45:33.678224+0000","last_unstale":"2024-09-13T08:45:34.955753+0000","last_undegraded":"2024-09-13T08:45:34.955753+0000","last_fullsized":"2024-09-13T08:45:34.955753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:35.993378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.655196+0000","last_change":"2024-09-13T08:45:32.654382+0000","last_active":"2024-09-13T08:45:34.655196+0000","last_peered":"2024-09-13T08:45:34.655196+0000","last_clean":"2024-09-13T08:45:34.655196+0000","last_became_active":"2024-09-13T08:45:32.654090+0000","last_became_peered":"2024-09-13T08:45:32.654090+0000","last_unstale":"2024-09-13T08:45:34.655196+0000","last_undegraded":"2024-09-13T08:45:34.655196+0000","last_fullsized":"2024-09-13T08:45:34.655196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:42.438703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.954356+0000","last_change":"2024-09-13T08:45:33.686795+0000","last_active":"2024-09-13T08:45:34.954356+0000","last_peered":"2024-09-13T08:45:34.954356+0000","last_clean":"2024-09-13T08:45:34.954356+0000","last_became_active":"2024-09-13T08:45:33.686497+0000","last_became_peered":"2024-09-13T08:45:33.686497+0000","last_unstale":"2024-09-13T08:45:34.954356+0000","last_undegraded":"2024-09-13T08:45:34.954356+0000","last_fullsized":"2024-09-13T08:45:34.954356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:41:55.806552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654106+0000","last_change":"2024-09-13T08:45:32.656780+0000","last_active":"2024-09-13T08:45:34.654106+0000","last_peered":"2024-09-13T08:45:34.654106+0000","last_clean":"2024-09-13T08:45:34.654106+0000","last_became_active":"2024-09-13T08:45:32.653941+0000","last_became_peered":"2024-09-13T08:45:32.653941+0000","last_unstale":"2024-09-13T08:45:34.654106+0000","last_undegraded":"2024-09-13T08:45:34.654106+0000","last_fullsized":"2024-09-13T08:45:34.654106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:12.824731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656576+0000","last_change":"2024-09-13T08:45:33.663007+0000","last_active":"2024-09-13T08:45:34.656576+0000","last_peered":"2024-09-13T08:45:34.656576+0000","last_clean":"2024-09-13T08:45:34.656576+0000","last_became_active":"2024-09-13T08:45:33.662591+0000","last_became_peered":"2024-09-13T08:45:33.662591+0000","last_unstale":"2024-09-13T08:45:34.656576+0000","last_undegraded":"2024-09-13T08:45:34.656576+0000","last_fullsized":"2024-09-13T08:45:34.656576+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:08.422179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.954571+0000","last_change":"2024-09-13T08:45:32.655814+0000","last_active":"2024-09-13T08:45:34.954571+0000","last_peered":"2024-09-13T08:45:34.954571+0000","last_clean":"2024-09-13T08:45:34.954571+0000","last_became_active":"2024-09-13T08:45:32.655509+0000","last_became_peered":"2024-09-13T08:45:32.655509+0000","last_unstale":"2024-09-13T08:45:34.954571+0000","last_undegraded":"2024-09-13T08:45:34.954571+0000","last_fullsized":"2024-09-13T08:45:34.954571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:56:39.873532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653700+0000","last_change":"2024-09-13T08:45:33.676002+0000","last_active":"2024-09-13T08:45:34.653700+0000","last_peered":"2024-09-13T08:45:34.653700+0000","last_clean":"2024-09-13T08:45:34.653700+0000","last_became_active":"2024-09-13T08:45:33.675711+0000","last_became_peered":"2024-09-13T08:45:33.675711+0000","last_unstale":"2024-09-13T08:45:34.653700+0000","last_undegraded":"2024-09-13T08:45:34.653700+0000","last_fullsized":"2024-09-13T08:45:34.653700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:19.915681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653075+0000","last_change":"2024-09-13T08:45:32.650357+0000","last_active":"2024-09-13T08:45:34.653075+0000","last_peered":"2024-09-13T08:45:34.653075+0000","last_clean":"2024-09-13T08:45:34.653075+0000","last_became_active":"2024-09-13T08:45:32.650092+0000","last_became_peered":"2024-09-13T08:45:32.650092+0000","last_unstale":"2024-09-13T08:45:34.653075+0000","last_undegraded":"2024-09-13T08:45:34.653075+0000","last_fullsized":"2024-09-13T08:45:34.653075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:11.131195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.954449+0000","last_change":"2024-09-13T08:45:33.686668+0000","last_active":"2024-09-13T08:45:34.954449+0000","last_peered":"2024-09-13T08:45:34.954449+0000","last_clean":"2024-09-13T08:45:34.954449+0000","last_became_active":"2024-09-13T08:45:33.686340+0000","last_became_peered":"2024-09-13T08:45:33.686340+0000","last_unstale":"2024-09-13T08:45:34.954449+0000","last_undegraded":"2024-09-13T08:45:34.954449+0000","last_fullsized":"2024-09-13T08:45:34.954449+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:29.425144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656137+0000","last_change":"2024-09-13T08:45:32.652970+0000","last_active":"2024-09-13T08:45:34.656137+0000","last_peered":"2024-09-13T08:45:34.656137+0000","last_clean":"2024-09-13T08:45:34.656137+0000","last_became_active":"2024-09-13T08:45:32.652708+0000","last_became_peered":"2024-09-13T08:45:32.652708+0000","last_unstale":"2024-09-13T08:45:34.656137+0000","last_undegraded":"2024-09-13T08:45:34.656137+0000","last_fullsized":"2024-09-13T08:45:34.656137+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:26.448081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.888913+0000","last_change":"2024-09-13T08:45:33.660387+0000","last_active":"2024-09-13T08:45:34.888913+0000","last_peered":"2024-09-13T08:45:34.888913+0000","last_clean":"2024-09-13T08:45:34.888913+0000","last_became_active":"2024-09-13T08:45:33.660031+0000","last_became_peered":"2024-09-13T08:45:33.660031+0000","last_unstale":"2024-09-13T08:45:34.888913+0000","last_undegraded":"2024-09-13T08:45:34.888913+0000","last_fullsized":"2024-09-13T08:45:34.888913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:39:43.561306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.889030+0000","last_change":"2024-09-13T08:45:32.659746+0000","last_active":"2024-09-13T08:45:34.889030+0000","last_peered":"2024-09-13T08:45:34.889030+0000","last_clean":"2024-09-13T08:45:34.889030+0000","last_became_active":"2024-09-13T08:45:32.659314+0000","last_became_peered":"2024-09-13T08:45:32.659314+0000","last_unstale":"2024-09-13T08:45:34.889030+0000","last_undegraded":"2024-09-13T08:45:34.889030+0000","last_fullsized":"2024-09-13T08:45:34.889030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:02.826589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653027+0000","last_change":"2024-09-13T08:45:33.668090+0000","last_active":"2024-09-13T08:45:34.653027+0000","last_peered":"2024-09-13T08:45:34.653027+0000","last_clean":"2024-09-13T08:45:34.653027+0000","last_became_active":"2024-09-13T08:45:33.667802+0000","last_became_peered":"2024-09-13T08:45:33.667802+0000","last_unstale":"2024-09-13T08:45:34.653027+0000","last_undegraded":"2024-09-13T08:45:34.653027+0000","last_fullsized":"2024-09-13T08:45:34.653027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:58:36.405093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.652866+0000","last_change":"2024-09-13T08:45:32.654860+0000","last_active":"2024-09-13T08:45:34.652866+0000","last_peered":"2024-09-13T08:45:34.652866+0000","last_clean":"2024-09-13T08:45:34.652866+0000","last_became_active":"2024-09-13T08:45:32.652510+0000","last_became_peered":"2024-09-13T08:45:32.652510+0000","last_unstale":"2024-09-13T08:45:34.652866+0000","last_undegraded":"2024-09-13T08:45:34.652866+0000","last_fullsized":"2024-09-13T08:45:34.652866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:16.016602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656844+0000","last_change":"2024-09-13T08:45:33.679316+0000","last_active":"2024-09-13T08:45:34.656844+0000","last_peered":"2024-09-13T08:45:34.656844+0000","last_clean":"2024-09-13T08:45:34.656844+0000","last_became_active":"2024-09-13T08:45:33.678605+0000","last_became_peered":"2024-09-13T08:45:33.678605+0000","last_unstale":"2024-09-13T08:45:34.656844+0000","last_undegraded":"2024-09-13T08:45:34.656844+0000","last_fullsized":"2024-09-13T08:45:34.656844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:39.819666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653869+0000","last_change":"2024-09-13T08:45:32.655976+0000","last_active":"2024-09-13T08:45:34.653869+0000","last_peered":"2024-09-13T08:45:34.653869+0000","last_clean":"2024-09-13T08:45:34.653869+0000","last_became_active":"2024-09-13T08:45:32.654678+0000","last_became_peered":"2024-09-13T08:45:32.654678+0000","last_unstale":"2024-09-13T08:45:34.653869+0000","last_undegraded":"2024-09-13T08:45:34.653869+0000","last_fullsized":"2024-09-13T08:45:34.653869+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:07.445463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.651066+0000","last_change":"2024-09-13T08:45:33.682397+0000","last_active":"2024-09-13T08:45:34.651066+0000","last_peered":"2024-09-13T08:45:34.651066+0000","last_clean":"2024-09-13T08:45:34.651066+0000","last_became_active":"2024-09-13T08:45:33.682010+0000","last_became_peered":"2024-09-13T08:45:33.682010+0000","last_unstale":"2024-09-13T08:45:34.651066+0000","last_undegraded":"2024-09-13T08:45:34.651066+0000","last_fullsized":"2024-09-13T08:45:34.651066+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:12.392336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.651382+0000","last_change":"2024-09-13T08:45:32.654977+0000","last_active":"2024-09-13T08:45:34.651382+0000","last_peered":"2024-09-13T08:45:34.651382+0000","last_clean":"2024-09-13T08:45:34.651382+0000","last_became_active":"2024-09-13T08:45:32.654656+0000","last_became_peered":"2024-09-13T08:45:32.654656+0000","last_unstale":"2024-09-13T08:45:34.651382+0000","last_undegraded":"2024-09-13T08:45:34.651382+0000","last_fullsized":"2024-09-13T08:45:34.651382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:09.832815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654149+0000","last_change":"2024-09-13T08:45:33.682432+0000","last_active":"2024-09-13T08:45:34.654149+0000","last_peered":"2024-09-13T08:45:34.654149+0000","last_clean":"2024-09-13T08:45:34.654149+0000","last_became_active":"2024-09-13T08:45:33.681551+0000","last_became_peered":"2024-09-13T08:45:33.681551+0000","last_unstale":"2024-09-13T08:45:34.654149+0000","last_undegraded":"2024-09-13T08:45:34.654149+0000","last_fullsized":"2024-09-13T08:45:34.654149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:30.406938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"74'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653764+0000","last_change":"2024-09-13T08:45:32.644983+0000","last_active":"2024-09-13T08:45:34.653764+0000","last_peered":"2024-09-13T08:45:34.653764+0000","last_clean":"2024-09-13T08:45:34.653764+0000","last_became_active":"2024-09-13T08:45:32.644649+0000","last_became_peered":"2024-09-13T08:45:32.644649+0000","last_unstale":"2024-09-13T08:45:34.653764+0000","last_undegraded":"2024-09-13T08:45:34.653764+0000","last_fullsized":"2024-09-13T08:45:34.653764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:16:03.736952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654893+0000","last_change":"2024-09-13T08:45:33.663848+0000","last_active":"2024-09-13T08:45:34.654893+0000","last_peered":"2024-09-13T08:45:34.654893+0000","last_clean":"2024-09-13T08:45:34.654893+0000","last_became_active":"2024-09-13T08:45:33.663559+0000","last_became_peered":"2024-09-13T08:45:33.663559+0000","last_unstale":"2024-09-13T08:45:34.654893+0000","last_undegraded":"2024-09-13T08:45:34.654893+0000","last_fullsized":"2024-09-13T08:45:34.654893+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:03.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"74'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.889209+0000","last_change":"2024-09-13T08:45:32.656020+0000","last_active":"2024-09-13T08:45:34.889209+0000","last_peered":"2024-09-13T08:45:34.889209+0000","last_clean":"2024-09-13T08:45:34.889209+0000","last_became_active":"2024-09-13T08:45:32.655215+0000","last_became_peered":"2024-09-13T08:45:32.655215+0000","last_unstale":"2024-09-13T08:45:34.889209+0000","last_undegraded":"2024-09-13T08:45:34.889209+0000","last_fullsized":"2024-09-13T08:45:34.889209+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:55.743757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.954525+0000","last_change":"2024-09-13T08:45:33.671033+0000","last_active":"2024-09-13T08:45:34.954525+0000","last_peered":"2024-09-13T08:45:34.954525+0000","last_clean":"2024-09-13T08:45:34.954525+0000","last_became_active":"2024-09-13T08:45:33.670779+0000","last_became_peered":"2024-09-13T08:45:33.670779+0000","last_unstale":"2024-09-13T08:45:34.954525+0000","last_undegraded":"2024-09-13T08:45:34.954525+0000","last_fullsized":"2024-09-13T08:45:34.954525+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:25.671165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.888969+0000","last_change":"2024-09-13T08:45:32.656334+0000","last_active":"2024-09-13T08:45:34.888969+0000","last_peered":"2024-09-13T08:45:34.888969+0000","last_clean":"2024-09-13T08:45:34.888969+0000","last_became_active":"2024-09-13T08:45:32.656007+0000","last_became_peered":"2024-09-13T08:45:32.656007+0000","last_unstale":"2024-09-13T08:45:34.888969+0000","last_undegraded":"2024-09-13T08:45:34.888969+0000","last_fullsized":"2024-09-13T08:45:34.888969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:13.439219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656473+0000","last_change":"2024-09-13T08:45:33.681389+0000","last_active":"2024-09-13T08:45:34.656473+0000","last_peered":"2024-09-13T08:45:34.656473+0000","last_clean":"2024-09-13T08:45:34.656473+0000","last_became_active":"2024-09-13T08:45:33.681085+0000","last_became_peered":"2024-09-13T08:45:33.681085+0000","last_unstale":"2024-09-13T08:45:34.656473+0000","last_undegraded":"2024-09-13T08:45:34.656473+0000","last_fullsized":"2024-09-13T08:45:34.656473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:08.361035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656235+0000","last_change":"2024-09-13T08:45:32.641547+0000","last_active":"2024-09-13T08:45:34.656235+0000","last_peered":"2024-09-13T08:45:34.656235+0000","last_clean":"2024-09-13T08:45:34.656235+0000","last_became_active":"2024-09-13T08:45:32.641221+0000","last_became_peered":"2024-09-13T08:45:32.641221+0000","last_unstale":"2024-09-13T08:45:34.656235+0000","last_undegraded":"2024-09-13T08:45:34.656235+0000","last_fullsized":"2024-09-13T08:45:34.656235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:37.612881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654034+0000","last_change":"2024-09-13T08:45:33.668342+0000","last_active":"2024-09-13T08:45:34.654034+0000","last_peered":"2024-09-13T08:45:34.654034+0000","last_clean":"2024-09-13T08:45:34.654034+0000","last_became_active":"2024-09-13T08:45:33.668068+0000","last_became_peered":"2024-09-13T08:45:33.668068+0000","last_unstale":"2024-09-13T08:45:34.654034+0000","last_undegraded":"2024-09-13T08:45:34.654034+0000","last_fullsized":"2024-09-13T08:45:34.654034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:07.001116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656316+0000","last_change":"2024-09-13T08:45:32.645856+0000","last_active":"2024-09-13T08:45:34.656316+0000","last_peered":"2024-09-13T08:45:34.656316+0000","last_clean":"2024-09-13T08:45:34.656316+0000","last_became_active":"2024-09-13T08:45:32.645551+0000","last_became_peered":"2024-09-13T08:45:32.645551+0000","last_unstale":"2024-09-13T08:45:34.656316+0000","last_undegraded":"2024-09-13T08:45:34.656316+0000","last_fullsized":"2024-09-13T08:45:34.656316+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:13.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.955317+0000","last_change":"2024-09-13T08:45:33.679240+0000","last_active":"2024-09-13T08:45:34.955317+0000","last_peered":"2024-09-13T08:45:34.955317+0000","last_clean":"2024-09-13T08:45:34.955317+0000","last_became_active":"2024-09-13T08:45:33.678945+0000","last_became_peered":"2024-09-13T08:45:33.678945+0000","last_unstale":"2024-09-13T08:45:34.955317+0000","last_undegraded":"2024-09-13T08:45:34.955317+0000","last_fullsized":"2024-09-13T08:45:34.955317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:48:30.513683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"74'8","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656040+0000","last_change":"2024-09-13T08:45:32.644799+0000","last_active":"2024-09-13T08:45:34.656040+0000","last_peered":"2024-09-13T08:45:34.656040+0000","last_clean":"2024-09-13T08:45:34.656040+0000","last_became_active":"2024-09-13T08:45:32.644559+0000","last_became_peered":"2024-09-13T08:45:32.644559+0000","last_unstale":"2024-09-13T08:45:34.656040+0000","last_undegraded":"2024-09-13T08:45:34.656040+0000","last_fullsized":"2024-09-13T08:45:34.656040+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:23.392065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.651410+0000","last_change":"2024-09-13T08:45:33.676706+0000","last_active":"2024-09-13T08:45:34.651410+0000","last_peered":"2024-09-13T08:45:34.651410+0000","last_clean":"2024-09-13T08:45:34.651410+0000","last_became_active":"2024-09-13T08:45:33.676442+0000","last_became_peered":"2024-09-13T08:45:33.676442+0000","last_unstale":"2024-09-13T08:45:34.651410+0000","last_undegraded":"2024-09-13T08:45:34.651410+0000","last_fullsized":"2024-09-13T08:45:34.651410+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:30.752153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654190+0000","last_change":"2024-09-13T08:45:32.655505+0000","last_active":"2024-09-13T08:45:34.654190+0000","last_peered":"2024-09-13T08:45:34.654190+0000","last_clean":"2024-09-13T08:45:34.654190+0000","last_became_active":"2024-09-13T08:45:32.655218+0000","last_became_peered":"2024-09-13T08:45:32.655218+0000","last_unstale":"2024-09-13T08:45:34.654190+0000","last_undegraded":"2024-09-13T08:45:34.654190+0000","last_fullsized":"2024-09-13T08:45:34.654190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:04.883497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.651283+0000","last_change":"2024-09-13T08:45:33.675626+0000","last_active":"2024-09-13T08:45:34.651283+0000","last_peered":"2024-09-13T08:45:34.651283+0000","last_clean":"2024-09-13T08:45:34.651283+0000","last_became_active":"2024-09-13T08:45:33.675346+0000","last_became_peered":"2024-09-13T08:45:33.675346+0000","last_unstale":"2024-09-13T08:45:34.651283+0000","last_undegraded":"2024-09-13T08:45:34.651283+0000","last_fullsized":"2024-09-13T08:45:34.651283+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:43:22.048135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.651459+0000","last_change":"2024-09-13T08:45:32.658779+0000","last_active":"2024-09-13T08:45:34.651459+0000","last_peered":"2024-09-13T08:45:34.651459+0000","last_clean":"2024-09-13T08:45:34.651459+0000","last_became_active":"2024-09-13T08:45:32.658420+0000","last_became_peered":"2024-09-13T08:45:32.658420+0000","last_unstale":"2024-09-13T08:45:34.651459+0000","last_undegraded":"2024-09-13T08:45:34.651459+0000","last_fullsized":"2024-09-13T08:45:34.651459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:18.130462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.955610+0000","last_change":"2024-09-13T08:45:33.675553+0000","last_active":"2024-09-13T08:45:34.955610+0000","last_peered":"2024-09-13T08:45:34.955610+0000","last_clean":"2024-09-13T08:45:34.955610+0000","last_became_active":"2024-09-13T08:45:33.675233+0000","last_became_peered":"2024-09-13T08:45:33.675233+0000","last_unstale":"2024-09-13T08:45:34.955610+0000","last_undegraded":"2024-09-13T08:45:34.955610+0000","last_fullsized":"2024-09-13T08:45:34.955610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:07:18.583988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.650845+0000","last_change":"2024-09-13T08:45:32.659506+0000","last_active":"2024-09-13T08:45:34.650845+0000","last_peered":"2024-09-13T08:45:34.650845+0000","last_clean":"2024-09-13T08:45:34.650845+0000","last_became_active":"2024-09-13T08:45:32.659220+0000","last_became_peered":"2024-09-13T08:45:32.659220+0000","last_unstale":"2024-09-13T08:45:34.650845+0000","last_undegraded":"2024-09-13T08:45:34.650845+0000","last_fullsized":"2024-09-13T08:45:34.650845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:30.318592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.955710+0000","last_change":"2024-09-13T08:45:33.688805+0000","last_active":"2024-09-13T08:45:34.955710+0000","last_peered":"2024-09-13T08:45:34.955710+0000","last_clean":"2024-09-13T08:45:34.955710+0000","last_became_active":"2024-09-13T08:45:33.688534+0000","last_became_peered":"2024-09-13T08:45:33.688534+0000","last_unstale":"2024-09-13T08:45:34.955710+0000","last_undegraded":"2024-09-13T08:45:34.955710+0000","last_fullsized":"2024-09-13T08:45:34.955710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:16.649199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.954581+0000","last_change":"2024-09-13T08:45:32.655084+0000","last_active":"2024-09-13T08:45:34.954581+0000","last_peered":"2024-09-13T08:45:34.954581+0000","last_clean":"2024-09-13T08:45:34.954581+0000","last_became_active":"2024-09-13T08:45:32.654795+0000","last_became_peered":"2024-09-13T08:45:32.654795+0000","last_unstale":"2024-09-13T08:45:34.954581+0000","last_undegraded":"2024-09-13T08:45:34.954581+0000","last_fullsized":"2024-09-13T08:45:34.954581+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:23:00.745264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.888994+0000","last_change":"2024-09-13T08:45:33.672926+0000","last_active":"2024-09-13T08:45:34.888994+0000","last_peered":"2024-09-13T08:45:34.888994+0000","last_clean":"2024-09-13T08:45:34.888994+0000","last_became_active":"2024-09-13T08:45:33.672641+0000","last_became_peered":"2024-09-13T08:45:33.672641+0000","last_unstale":"2024-09-13T08:45:34.888994+0000","last_undegraded":"2024-09-13T08:45:34.888994+0000","last_fullsized":"2024-09-13T08:45:34.888994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:24.386769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.954545+0000","last_change":"2024-09-13T08:45:32.648855+0000","last_active":"2024-09-13T08:45:34.954545+0000","last_peered":"2024-09-13T08:45:34.954545+0000","last_clean":"2024-09-13T08:45:34.954545+0000","last_became_active":"2024-09-13T08:45:32.648572+0000","last_became_peered":"2024-09-13T08:45:32.648572+0000","last_unstale":"2024-09-13T08:45:34.954545+0000","last_undegraded":"2024-09-13T08:45:34.954545+0000","last_fullsized":"2024-09-13T08:45:34.954545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:11.383879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.888982+0000","last_change":"2024-09-13T08:45:33.663742+0000","last_active":"2024-09-13T08:45:34.888982+0000","last_peered":"2024-09-13T08:45:34.888982+0000","last_clean":"2024-09-13T08:45:34.888982+0000","last_became_active":"2024-09-13T08:45:33.663370+0000","last_became_peered":"2024-09-13T08:45:33.663370+0000","last_unstale":"2024-09-13T08:45:34.888982+0000","last_undegraded":"2024-09-13T08:45:34.888982+0000","last_fullsized":"2024-09-13T08:45:34.888982+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:19.194461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.651145+0000","last_change":"2024-09-13T08:45:32.651171+0000","last_active":"2024-09-13T08:45:34.651145+0000","last_peered":"2024-09-13T08:45:34.651145+0000","last_clean":"2024-09-13T08:45:34.651145+0000","last_became_active":"2024-09-13T08:45:32.650437+0000","last_became_peered":"2024-09-13T08:45:32.650437+0000","last_unstale":"2024-09-13T08:45:34.651145+0000","last_undegraded":"2024-09-13T08:45:34.651145+0000","last_fullsized":"2024-09-13T08:45:34.651145+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:43.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653277+0000","last_change":"2024-09-13T08:45:33.670268+0000","last_active":"2024-09-13T08:45:34.653277+0000","last_peered":"2024-09-13T08:45:34.653277+0000","last_clean":"2024-09-13T08:45:34.653277+0000","last_became_active":"2024-09-13T08:45:33.669974+0000","last_became_peered":"2024-09-13T08:45:33.669974+0000","last_unstale":"2024-09-13T08:45:34.653277+0000","last_undegraded":"2024-09-13T08:45:34.653277+0000","last_fullsized":"2024-09-13T08:45:34.653277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:29.873455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653450+0000","last_change":"2024-09-13T08:45:32.652196+0000","last_active":"2024-09-13T08:45:34.653450+0000","last_peered":"2024-09-13T08:45:34.653450+0000","last_clean":"2024-09-13T08:45:34.653450+0000","last_became_active":"2024-09-13T08:45:32.651851+0000","last_became_peered":"2024-09-13T08:45:32.651851+0000","last_unstale":"2024-09-13T08:45:34.653450+0000","last_undegraded":"2024-09-13T08:45:34.653450+0000","last_fullsized":"2024-09-13T08:45:34.653450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:48:29.415637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.655112+0000","last_change":"2024-09-13T08:45:33.664149+0000","last_active":"2024-09-13T08:45:34.655112+0000","last_peered":"2024-09-13T08:45:34.655112+0000","last_clean":"2024-09-13T08:45:34.655112+0000","last_became_active":"2024-09-13T08:45:33.663531+0000","last_became_peered":"2024-09-13T08:45:33.663531+0000","last_unstale":"2024-09-13T08:45:34.655112+0000","last_undegraded":"2024-09-13T08:45:34.655112+0000","last_fullsized":"2024-09-13T08:45:34.655112+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:14.833887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654301+0000","last_change":"2024-09-13T08:45:32.660012+0000","last_active":"2024-09-13T08:45:34.654301+0000","last_peered":"2024-09-13T08:45:34.654301+0000","last_clean":"2024-09-13T08:45:34.654301+0000","last_became_active":"2024-09-13T08:45:32.659398+0000","last_became_peered":"2024-09-13T08:45:32.659398+0000","last_unstale":"2024-09-13T08:45:34.654301+0000","last_undegraded":"2024-09-13T08:45:34.654301+0000","last_fullsized":"2024-09-13T08:45:34.654301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:12.195364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656836+0000","last_change":"2024-09-13T08:45:33.660680+0000","last_active":"2024-09-13T08:45:34.656836+0000","last_peered":"2024-09-13T08:45:34.656836+0000","last_clean":"2024-09-13T08:45:34.656836+0000","last_became_active":"2024-09-13T08:45:33.660407+0000","last_became_peered":"2024-09-13T08:45:33.660407+0000","last_unstale":"2024-09-13T08:45:34.656836+0000","last_undegraded":"2024-09-13T08:45:34.656836+0000","last_fullsized":"2024-09-13T08:45:34.656836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:44.383506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654363+0000","last_change":"2024-09-13T08:45:32.661277+0000","last_active":"2024-09-13T08:45:34.654363+0000","last_peered":"2024-09-13T08:45:34.654363+0000","last_clean":"2024-09-13T08:45:34.654363+0000","last_became_active":"2024-09-13T08:45:32.660211+0000","last_became_peered":"2024-09-13T08:45:32.660211+0000","last_unstale":"2024-09-13T08:45:34.654363+0000","last_undegraded":"2024-09-13T08:45:34.654363+0000","last_fullsized":"2024-09-13T08:45:34.654363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:01.561248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.651105+0000","last_change":"2024-09-13T08:45:33.683345+0000","last_active":"2024-09-13T08:45:34.651105+0000","last_peered":"2024-09-13T08:45:34.651105+0000","last_clean":"2024-09-13T08:45:34.651105+0000","last_became_active":"2024-09-13T08:45:33.682992+0000","last_became_peered":"2024-09-13T08:45:33.682992+0000","last_unstale":"2024-09-13T08:45:34.651105+0000","last_undegraded":"2024-09-13T08:45:34.651105+0000","last_fullsized":"2024-09-13T08:45:34.651105+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:29.873455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653814+0000","last_change":"2024-09-13T08:45:32.646650+0000","last_active":"2024-09-13T08:45:34.653814+0000","last_peered":"2024-09-13T08:45:34.653814+0000","last_clean":"2024-09-13T08:45:34.653814+0000","last_became_active":"2024-09-13T08:45:32.646362+0000","last_became_peered":"2024-09-13T08:45:32.646362+0000","last_unstale":"2024-09-13T08:45:34.653814+0000","last_undegraded":"2024-09-13T08:45:34.653814+0000","last_fullsized":"2024-09-13T08:45:34.653814+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:48:47.580467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656679+0000","last_change":"2024-09-13T08:45:33.678066+0000","last_active":"2024-09-13T08:45:34.656679+0000","last_peered":"2024-09-13T08:45:34.656679+0000","last_clean":"2024-09-13T08:45:34.656679+0000","last_became_active":"2024-09-13T08:45:33.674778+0000","last_became_peered":"2024-09-13T08:45:33.674778+0000","last_unstale":"2024-09-13T08:45:34.656679+0000","last_undegraded":"2024-09-13T08:45:34.656679+0000","last_fullsized":"2024-09-13T08:45:34.656679+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:24:04.349210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653723+0000","last_change":"2024-09-13T08:45:32.646318+0000","last_active":"2024-09-13T08:45:34.653723+0000","last_peered":"2024-09-13T08:45:34.653723+0000","last_clean":"2024-09-13T08:45:34.653723+0000","last_became_active":"2024-09-13T08:45:32.645801+0000","last_became_peered":"2024-09-13T08:45:32.645801+0000","last_unstale":"2024-09-13T08:45:34.653723+0000","last_undegraded":"2024-09-13T08:45:34.653723+0000","last_fullsized":"2024-09-13T08:45:34.653723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:40.323104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654682+0000","last_change":"2024-09-13T08:45:33.670611+0000","last_active":"2024-09-13T08:45:34.654682+0000","last_peered":"2024-09-13T08:45:34.654682+0000","last_clean":"2024-09-13T08:45:34.654682+0000","last_became_active":"2024-09-13T08:45:33.670334+0000","last_became_peered":"2024-09-13T08:45:33.670334+0000","last_unstale":"2024-09-13T08:45:34.654682+0000","last_undegraded":"2024-09-13T08:45:34.654682+0000","last_fullsized":"2024-09-13T08:45:34.654682+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:13.833188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653450+0000","last_change":"2024-09-13T08:45:32.650061+0000","last_active":"2024-09-13T08:45:34.653450+0000","last_peered":"2024-09-13T08:45:34.653450+0000","last_clean":"2024-09-13T08:45:34.653450+0000","last_became_active":"2024-09-13T08:45:32.649743+0000","last_became_peered":"2024-09-13T08:45:32.649743+0000","last_unstale":"2024-09-13T08:45:34.653450+0000","last_undegraded":"2024-09-13T08:45:34.653450+0000","last_fullsized":"2024-09-13T08:45:34.653450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:16.817675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.653947+0000","last_change":"2024-09-13T08:45:33.682926+0000","last_active":"2024-09-13T08:45:34.653947+0000","last_peered":"2024-09-13T08:45:34.653947+0000","last_clean":"2024-09-13T08:45:34.653947+0000","last_became_active":"2024-09-13T08:45:33.681036+0000","last_became_peered":"2024-09-13T08:45:33.681036+0000","last_unstale":"2024-09-13T08:45:34.653947+0000","last_undegraded":"2024-09-13T08:45:34.653947+0000","last_fullsized":"2024-09-13T08:45:34.653947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:03:11.912163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654758+0000","last_change":"2024-09-13T08:45:32.651908+0000","last_active":"2024-09-13T08:45:34.654758+0000","last_peered":"2024-09-13T08:45:34.654758+0000","last_clean":"2024-09-13T08:45:34.654758+0000","last_became_active":"2024-09-13T08:45:32.651622+0000","last_became_peered":"2024-09-13T08:45:32.651622+0000","last_unstale":"2024-09-13T08:45:34.654758+0000","last_undegraded":"2024-09-13T08:45:34.654758+0000","last_fullsized":"2024-09-13T08:45:34.654758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:32.504688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.889297+0000","last_change":"2024-09-13T08:45:33.673105+0000","last_active":"2024-09-13T08:45:34.889297+0000","last_peered":"2024-09-13T08:45:34.889297+0000","last_clean":"2024-09-13T08:45:34.889297+0000","last_became_active":"2024-09-13T08:45:33.672786+0000","last_became_peered":"2024-09-13T08:45:33.672786+0000","last_unstale":"2024-09-13T08:45:34.889297+0000","last_undegraded":"2024-09-13T08:45:34.889297+0000","last_fullsized":"2024-09-13T08:45:34.889297+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:54.590873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.651215+0000","last_change":"2024-09-13T08:45:32.656890+0000","last_active":"2024-09-13T08:45:34.651215+0000","last_peered":"2024-09-13T08:45:34.651215+0000","last_clean":"2024-09-13T08:45:34.651215+0000","last_became_active":"2024-09-13T08:45:32.656592+0000","last_became_peered":"2024-09-13T08:45:32.656592+0000","last_unstale":"2024-09-13T08:45:34.651215+0000","last_undegraded":"2024-09-13T08:45:34.651215+0000","last_fullsized":"2024-09-13T08:45:34.651215+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:38.811210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.650786+0000","last_change":"2024-09-13T08:45:33.669976+0000","last_active":"2024-09-13T08:45:34.650786+0000","last_peered":"2024-09-13T08:45:34.650786+0000","last_clean":"2024-09-13T08:45:34.650786+0000","last_became_active":"2024-09-13T08:45:33.669693+0000","last_became_peered":"2024-09-13T08:45:33.669693+0000","last_unstale":"2024-09-13T08:45:34.650786+0000","last_undegraded":"2024-09-13T08:45:34.650786+0000","last_fullsized":"2024-09-13T08:45:34.650786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:58:36.405093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.954250+0000","last_change":"2024-09-13T08:45:32.654218+0000","last_active":"2024-09-13T08:45:34.954250+0000","last_peered":"2024-09-13T08:45:34.954250+0000","last_clean":"2024-09-13T08:45:34.954250+0000","last_became_active":"2024-09-13T08:45:32.653519+0000","last_became_peered":"2024-09-13T08:45:32.653519+0000","last_unstale":"2024-09-13T08:45:34.954250+0000","last_undegraded":"2024-09-13T08:45:34.954250+0000","last_fullsized":"2024-09-13T08:45:34.954250+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:33.012849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.654274+0000","last_change":"2024-09-13T08:45:33.679432+0000","last_active":"2024-09-13T08:45:34.654274+0000","last_peered":"2024-09-13T08:45:34.654274+0000","last_clean":"2024-09-13T08:45:34.654274+0000","last_became_active":"2024-09-13T08:45:33.679129+0000","last_became_peered":"2024-09-13T08:45:33.679129+0000","last_unstale":"2024-09-13T08:45:34.654274+0000","last_undegraded":"2024-09-13T08:45:34.654274+0000","last_fullsized":"2024-09-13T08:45:34.654274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:32.627543+0000","last_clean_scrub_stamp":"2024-09-13T08:45:32.627543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:02.569704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656416+0000","last_change":"2024-09-13T08:45:32.647002+0000","last_active":"2024-09-13T08:45:34.656416+0000","last_peered":"2024-09-13T08:45:34.656416+0000","last_clean":"2024-09-13T08:45:34.656416+0000","last_became_active":"2024-09-13T08:45:32.645456+0000","last_became_peered":"2024-09-13T08:45:32.645456+0000","last_unstale":"2024-09-13T08:45:34.656416+0000","last_undegraded":"2024-09-13T08:45:34.656416+0000","last_fullsized":"2024-09-13T08:45:34.656416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:45:31.619086+0000","last_clean_scrub_stamp":"2024-09-13T08:45:31.619086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:41.793310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":214,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T08:45:34.656118+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:45:34.656118+0000","last_peered":"2024-09-13T08:45:34.656118+0000","last_clean":"2024-09-13T08:45:34.656118+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:45:34.656118+0000","last_undegraded":"2024-09-13T08:45:34.656118+0000","last_fullsized":"2024-09-13T08:45:34.656118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509541,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50136,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321704,"statfs":{"total":96636764160,"available":96585424896,"internally_reserved":0,"allocated":917504,"data_stored":387298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50712,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321128,"statfs":{"total":96636764160,"available":96584835072,"internally_reserved":0,"allocated":1503232,"data_stored":976144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54292,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317548,"statfs":{"total":96636764160,"available":96581169152,"internally_reserved":0,"allocated":917504,"data_stored":386380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509542,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50132,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321708,"statfs":{"total":96636764160,"available":96585428992,"internally_reserved":0,"allocated":917504,"data_stored":385812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50128,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321712,"statfs":{"total":96636764160,"available":96585433088,"internally_reserved":0,"allocated":909312,"data_stored":386268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509543,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46548,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325292,"statfs":{"total":96636764160,"available":96589099008,"internally_reserved":0,"allocated":1499136,"data_stored":976054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50128,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321712,"statfs":{"total":96636764160,"available":96585433088,"internally_reserved":0,"allocated":909312,"data_stored":385756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50140,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321700,"statfs":{"total":96636764160,"available":96585420800,"internally_reserved":0,"allocated":921600,"data_stored":387332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:45:42.829 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:45:42.829 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T08:45:43.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.021+0000 7ff826400640 1 -- 172.21.15.44:0/479752378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff804004060 msgr2=0x7ff804004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.021+0000 7ff826400640 1 --2- 172.21.15.44:0/479752378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff804004060 0x7ff804004810 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff8080016f0 tx=0x7ff80801c760 comp rx=0 tx=0).stop 2024-09-13T08:45:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.021+0000 7ff81fe00640 1 -- 172.21.15.44:0/479752378 reap_dead start 2024-09-13T08:45:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.021+0000 7ff826400640 1 -- 172.21.15.44:0/479752378 shutdown_connections 2024-09-13T08:45:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.021+0000 7ff826400640 1 -- 172.21.15.44:0/479752378 >> 172.21.15.44:0/479752378 conn(0x7ff8200d1e00 msgr2=0x7ff8200d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.021+0000 7ff826400640 1 -- 172.21.15.44:0/479752378 shutdown_connections 2024-09-13T08:45:43.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.021+0000 7ff826400640 1 -- 172.21.15.44:0/479752378 wait complete. 2024-09-13T08:45:43.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.021+0000 7ff826400640 1 Processor -- start 2024-09-13T08:45:43.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.021+0000 7ff826400640 1 -- start start 2024-09-13T08:45:43.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff826400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff804004060 0x7ff820155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff826400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8201f6290 0x7ff820071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff826400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff820072080 0x7ff8200b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff826400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8200b1a30 con 0x7ff8201f6290 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff826400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8200b2c60 con 0x7ff804004060 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff826400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff8200b1bb0 con 0x7ff820072080 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8201f6290 0x7ff820071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8201f6290 0x7ff820071b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42300/0 (socket says 172.21.15.44:42300) 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817e00640 1 -- 172.21.15.44:0/707926978 learned_addr learned my addr 172.21.15.44:0/707926978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff81f400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff804004060 0x7ff820155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff81fe00640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff820072080 0x7ff8200b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:43.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817e00640 1 -- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff820072080 msgr2=0x7ff8200b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817e00640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff820072080 0x7ff8200b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817e00640 1 -- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff804004060 msgr2=0x7ff820155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817e00640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff804004060 0x7ff820155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817e00640 1 -- 172.21.15.44:0/707926978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8201f26a0 con 0x7ff8201f6290 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff81f400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff804004060 0x7ff820155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817e00640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8201f6290 0x7ff820071b40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff80801cc40 tx=0x7ff808006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817400640 1 -- 172.21.15.44:0/707926978 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8080a5680 con 0x7ff8201f6290 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817400640 1 -- 172.21.15.44:0/707926978 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8080a5910 con 0x7ff8201f6290 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff817400640 1 -- 172.21.15.44:0/707926978 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8080a5d20 con 0x7ff8201f6290 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff826400640 1 -- 172.21.15.44:0/707926978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8201f2980 con 0x7ff8201f6290 2024-09-13T08:45:43.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff826400640 1 -- 172.21.15.44:0/707926978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8201f2f40 con 0x7ff8201f6290 2024-09-13T08:45:43.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.025+0000 7ff826400640 1 -- 172.21.15.44:0/707926978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8200b1320 con 0x7ff8201f6290 2024-09-13T08:45:43.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.029+0000 7ff817400640 1 -- 172.21.15.44:0/707926978 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7ff808003600 con 0x7ff8201f6290 2024-09-13T08:45:43.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.029+0000 7ff817400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7fc073b50 0x7ff7fc076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:43.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.029+0000 7ff817400640 1 -- 172.21.15.44:0/707926978 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7ff80812be80 con 0x7ff8201f6290 2024-09-13T08:45:43.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.029+0000 7ff81f400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7fc073b50 0x7ff7fc076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:43.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.029+0000 7ff81f400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7fc073b50 0x7ff7fc076010 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff810001420 tx=0x7ff8100068b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:43.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.033+0000 7ff817400640 1 -- 172.21.15.44:0/707926978 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff8080b4070 con 0x7ff8201f6290 2024-09-13T08:45:43.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.269+0000 7ff826400640 1 -- 172.21.15.44:0/707926978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff820072460 con 0x7ff8201f6290 2024-09-13T08:45:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.273+0000 7ff817400640 1 -- 172.21.15.44:0/707926978 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7ff808016510 con 0x7ff8201f6290 2024-09-13T08:45:43.274 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T08:45:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:45:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-09-13T08:45:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-09-13T08:45:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-13T08:45:43.275 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.273+0000 7ff826400640 1 -- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7fc073b50 msgr2=0x7ff7fc076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.273+0000 7ff826400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7fc073b50 0x7ff7fc076010 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff810001420 tx=0x7ff8100068b0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.273+0000 7ff826400640 1 -- 172.21.15.44:0/707926978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8201f6290 msgr2=0x7ff820071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.273+0000 7ff826400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8201f6290 0x7ff820071b40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff80801cc40 tx=0x7ff808006950 comp rx=0 tx=0).stop 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.277+0000 7ff826400640 1 -- 172.21.15.44:0/707926978 shutdown_connections 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.277+0000 7ff826400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7fc073b50 0x7ff7fc076010 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.277+0000 7ff826400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff820072080 0x7ff8200b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.277+0000 7ff826400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8201f6290 0x7ff820071b40 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.277+0000 7ff826400640 1 --2- 172.21.15.44:0/707926978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff804004060 0x7ff820155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.277+0000 7ff826400640 1 -- 172.21.15.44:0/707926978 >> 172.21.15.44:0/707926978 conn(0x7ff8200d1e00 msgr2=0x7ff8200ae920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:43.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.277+0000 7ff826400640 1 -- 172.21.15.44:0/707926978 shutdown_connections 2024-09-13T08:45:43.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.277+0000 7ff826400640 1 -- 172.21.15.44:0/707926978 wait complete. 2024-09-13T08:45:43.297 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:45:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.441+0000 7f2e90600640 1 -- 172.21.15.44:0/3826986894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e8815bea0 msgr2=0x7f2e880ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.441+0000 7f2e90600640 1 --2- 172.21.15.44:0/3826986894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e8815bea0 0x7f2e880ce670 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009900 tx=0x7f2e7c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.441+0000 7f2e90600640 1 -- 172.21.15.44:0/3826986894 shutdown_connections 2024-09-13T08:45:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.441+0000 7f2e90600640 1 --2- 172.21.15.44:0/3826986894 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2e880d28e0 0x7f2e880cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.441+0000 7f2e90600640 1 --2- 172.21.15.44:0/3826986894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e8815bea0 0x7f2e880ce670 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.441+0000 7f2e90600640 1 -- 172.21.15.44:0/3826986894 >> 172.21.15.44:0/3826986894 conn(0x7f2e880d1dc0 msgr2=0x7f2e880d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.441+0000 7f2e90600640 1 -- 172.21.15.44:0/3826986894 shutdown_connections 2024-09-13T08:45:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.441+0000 7f2e90600640 1 -- 172.21.15.44:0/3826986894 wait complete. 2024-09-13T08:45:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e90600640 1 Processor -- start 2024-09-13T08:45:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e90600640 1 -- start start 2024-09-13T08:45:43.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e90600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e880cdb30 0x7f2e881f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e90600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e880d28e0 0x7f2e881512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e90600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2e8815bea0 0x7f2e880b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e90600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88168fc0 con 0x7f2e880cdb30 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e90600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88168e40 con 0x7f2e880d28e0 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e90600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2e88169140 con 0x7f2e8815bea0 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e8dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e880cdb30 0x7f2e881f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e8dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e880cdb30 0x7f2e881f8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42320/0 (socket says 172.21.15.44:42320) 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e8dc00640 1 -- 172.21.15.44:0/3839276028 learned_addr learned my addr 172.21.15.44:0/3839276028 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e8dc00640 1 -- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2e8815bea0 msgr2=0x7f2e880b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e8dc00640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2e8815bea0 0x7f2e880b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e8dc00640 1 -- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e880d28e0 msgr2=0x7f2e881512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e8dc00640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e880d28e0 0x7f2e881512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e8dc00640 1 -- 172.21.15.44:0/3839276028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e881f9180 con 0x7f2e880cdb30 2024-09-13T08:45:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e8dc00640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e880cdb30 0x7f2e881f8c40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009850 tx=0x7f2e7c0a5810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e85600640 1 -- 172.21.15.44:0/3839276028 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0b2040 con 0x7f2e880cdb30 2024-09-13T08:45:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e85600640 1 -- 172.21.15.44:0/3839276028 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e7c0044b0 con 0x7f2e880cdb30 2024-09-13T08:45:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e85600640 1 -- 172.21.15.44:0/3839276028 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c005290 con 0x7f2e880cdb30 2024-09-13T08:45:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e90600640 1 -- 172.21.15.44:0/3839276028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e881517f0 con 0x7f2e880cdb30 2024-09-13T08:45:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.445+0000 7f2e90600640 1 -- 172.21.15.44:0/3839276028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e880b2dd0 con 0x7f2e880cdb30 2024-09-13T08:45:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.449+0000 7f2e90600640 1 -- 172.21.15.44:0/3839276028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e880cc9d0 con 0x7f2e880cdb30 2024-09-13T08:45:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.449+0000 7f2e85600640 1 -- 172.21.15.44:0/3839276028 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f2e7c005430 con 0x7f2e880cdb30 2024-09-13T08:45:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.449+0000 7f2e85600640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e68073b20 0x7f2e68075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.449+0000 7f2e85600640 1 -- 172.21.15.44:0/3839276028 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f2e7c12c340 con 0x7f2e880cdb30 2024-09-13T08:45:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.449+0000 7f2e8d200640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e68073b20 0x7f2e68075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:43.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.449+0000 7f2e8d200640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e68073b20 0x7f2e68075fe0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2e74000f70 tx=0x7f2e740079e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:43.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.453+0000 7f2e85600640 1 -- 172.21.15.44:0/3839276028 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2e7c0164d0 con 0x7f2e880cdb30 2024-09-13T08:45:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.661+0000 7f2e90600640 1 -- 172.21.15.44:0/3839276028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e88158430 con 0x7f2e880cdb30 2024-09-13T08:45:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e85600640 1 -- 172.21.15.44:0/3839276028 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5228 (secure 0 0 0) 0x7f2e7c0f8fe0 con 0x7f2e880cdb30 2024-09-13T08:45:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:45:43.665 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":46,"btime":"2024-09-13T08:45:36:672438+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3854096181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3854096181},{"type":"v1","addr":"172.21.15.129:6836","nonce":3854096181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2183755853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":2183755853},{"type":"v1","addr":"172.21.15.129:6837","nonce":2183755853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6330,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1061181525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1061181525},{"type":"v1","addr":"172.21.15.44:6834","nonce":1061181525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:45:33.055598+0000","modified":"2024-09-13T08:45:36.075582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/651900305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":651900305},{"type":"v1","addr":"172.21.15.44:6835","nonce":651900305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6333,"qdb_cluster":[6333]},"id":5}]} 2024-09-13T08:45:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 46 2024-09-13T08:45:43.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 -- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e68073b20 msgr2=0x7f2e68075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:43.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e68073b20 0x7f2e68075fe0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2e74000f70 tx=0x7f2e740079e0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 -- 172.21.15.44:0/3839276028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e880cdb30 msgr2=0x7f2e881f8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:43.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e880cdb30 0x7f2e881f8c40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009850 tx=0x7f2e7c0a5810 comp rx=0 tx=0).stop 2024-09-13T08:45:43.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 -- 172.21.15.44:0/3839276028 shutdown_connections 2024-09-13T08:45:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e68073b20 0x7f2e68075fe0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2e8815bea0 0x7f2e880b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e880d28e0 0x7f2e881512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 --2- 172.21.15.44:0/3839276028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e880cdb30 0x7f2e881f8c40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 -- 172.21.15.44:0/3839276028 >> 172.21.15.44:0/3839276028 conn(0x7f2e880d1dc0 msgr2=0x7f2e8806f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 -- 172.21.15.44:0/3839276028 shutdown_connections 2024-09-13T08:45:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:43.665+0000 7f2e90600640 1 -- 172.21.15.44:0/3839276028 wait complete. 2024-09-13T08:45:44.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T08:45:33.055598+0000', 'modified': '2024-09-13T08:45:36.075582+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6333}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6333': {'gid': 6333, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/651900305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 651900305}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 651900305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6333, 'qdb_cluster': [6333]} 2024-09-13T08:45:44.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T08:45:44.091 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T08:45:44.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.241+0000 7fcb08c00640 1 -- 172.21.15.44:0/3751110912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406f1d0 msgr2=0x7fcb040d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.241+0000 7fcb08c00640 1 --2- 172.21.15.44:0/3751110912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406f1d0 0x7fcb040d0960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009bc0 tx=0x7fcafc01cba0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.241+0000 7fcb08c00640 1 -- 172.21.15.44:0/3751110912 shutdown_connections 2024-09-13T08:45:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.241+0000 7fcb08c00640 1 --2- 172.21.15.44:0/3751110912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406f1d0 0x7fcb040d0960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.241+0000 7fcb08c00640 1 --2- 172.21.15.44:0/3751110912 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcb04159990 0x7fcb040d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.241+0000 7fcb08c00640 1 -- 172.21.15.44:0/3751110912 >> 172.21.15.44:0/3751110912 conn(0x7fcb040b2240 msgr2=0x7fcb040b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.241+0000 7fcb08c00640 1 -- 172.21.15.44:0/3751110912 shutdown_connections 2024-09-13T08:45:44.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.241+0000 7fcb08c00640 1 -- 172.21.15.44:0/3751110912 wait complete. 2024-09-13T08:45:44.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.241+0000 7fcb08c00640 1 Processor -- start 2024-09-13T08:45:44.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.241+0000 7fcb08c00640 1 -- start start 2024-09-13T08:45:44.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb08c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406f1d0 0x7fcb04153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb08c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcb040afac0 0x7fcb041fada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb08c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb04159990 0x7fcb040ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb08c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb041691e0 con 0x7fcb04159990 2024-09-13T08:45:44.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb08c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb04169060 con 0x7fcb0406f1d0 2024-09-13T08:45:44.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb08c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcb04169360 con 0x7fcb040afac0 2024-09-13T08:45:44.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb02800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb04159990 0x7fcb040ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb02800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb04159990 0x7fcb040ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42326/0 (socket says 172.21.15.44:42326) 2024-09-13T08:45:44.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb02800640 1 -- 172.21.15.44:0/3330198903 learned_addr learned my addr 172.21.15.44:0/3330198903 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:44.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb01e00640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406f1d0 0x7fcb04153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcafbe00640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcb040afac0 0x7fcb041fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb02800640 1 -- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcb040afac0 msgr2=0x7fcb041fada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb02800640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcb040afac0 0x7fcb041fada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb02800640 1 -- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406f1d0 msgr2=0x7fcb04153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb02800640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406f1d0 0x7fcb04153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb02800640 1 -- 172.21.15.44:0/3330198903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb04157360 con 0x7fcb04159990 2024-09-13T08:45:44.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb02800640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb04159990 0x7fcb040ade00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009b10 tx=0x7fcafc0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:44.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcafa000640 1 -- 172.21.15.44:0/3330198903 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0bb070 con 0x7fcb04159990 2024-09-13T08:45:44.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcafa000640 1 -- 172.21.15.44:0/3330198903 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcafc004e00 con 0x7fcb04159990 2024-09-13T08:45:44.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb08c00640 1 -- 172.21.15.44:0/3330198903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb04157640 con 0x7fcb04159990 2024-09-13T08:45:44.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcafa000640 1 -- 172.21.15.44:0/3330198903 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0b2040 con 0x7fcb04159990 2024-09-13T08:45:44.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.245+0000 7fcb08c00640 1 -- 172.21.15.44:0/3330198903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb040b1f70 con 0x7fcb04159990 2024-09-13T08:45:44.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcafa000640 1 -- 172.21.15.44:0/3330198903 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fcafc0be070 con 0x7fcb04159990 2024-09-13T08:45:44.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcafa000640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae0073b20 0x7fcae0075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcafa000640 1 -- 172.21.15.44:0/3330198903 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7fcafc12c390 con 0x7fcb04159990 2024-09-13T08:45:44.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb0c07c000 1 Processor -- start 2024-09-13T08:45:44.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb01e00640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae0073b20 0x7fcae0075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb0c07c000 1 -- start start 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb01e00640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae0073b20 0x7fcae0075fe0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40096c0 tx=0x7fcaf4009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4d99d0 0x55e21e4d9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e48d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e48de10 0x55e21e48e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e21e3ae3f0 con 0x55e21e4da2f0 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e21e2ebb20 con 0x55e21e48de10 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55e21e2daf50 con 0x55e21e4d99d0 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcafbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e48d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcafbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e48d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42336/0 (socket says 172.21.15.44:42336) 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcafbe00640 1 -- 172.21.15.44:0/1720472800 learned_addr learned my addr 172.21.15.44:0/1720472800 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:44.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb01e00640 1 --2- 172.21.15.44:0/1720472800 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4d99d0 0x55e21e4d9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.249+0000 7fcb02800640 1 --2- 172.21.15.44:0/1720472800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e48de10 0x55e21e48e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcafbe00640 1 -- 172.21.15.44:0/1720472800 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4d99d0 msgr2=0x55e21e4d9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcafbe00640 1 --2- 172.21.15.44:0/1720472800 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4d99d0 0x55e21e4d9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcafbe00640 1 -- 172.21.15.44:0/1720472800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e48de10 msgr2=0x55e21e48e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcafbe00640 1 --2- 172.21.15.44:0/1720472800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e48de10 0x55e21e48e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcafbe00640 1 -- 172.21.15.44:0/1720472800 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21e2a2590 con 0x55e21e4da2f0 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcafbe00640 1 --2- 172.21.15.44:0/1720472800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e48d8d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0017e0 tx=0x7fcaec007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7732dfc7fd39ca53 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcaf2a00640 1 -- 172.21.15.44:0/1720472800 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0052e0 con 0x55e21e4da2f0 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcaf2a00640 1 -- 172.21.15.44:0/1720472800 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaec018070 con 0x55e21e4da2f0 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcaf2a00640 1 -- 172.21.15.44:0/1720472800 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00f9d0 con 0x55e21e4da2f0 2024-09-13T08:45:44.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1720472800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 msgr2=0x55e21e48d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/1720472800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e48d8d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0017e0 tx=0x7fcaec007e40 comp rx=0 tx=0).stop 2024-09-13T08:45:44.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1720472800 shutdown_connections 2024-09-13T08:45:44.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/1720472800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e48de10 0x55e21e48e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/1720472800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e48d8d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/1720472800 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4d99d0 0x55e21e4d9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1720472800 >> 172.21.15.44:0/1720472800 conn(0x55e21e377130 msgr2=0x55e21e4bce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:44.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1720472800 shutdown_connections 2024-09-13T08:45:44.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1720472800 wait complete. 2024-09-13T08:45:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 -- start start 2024-09-13T08:45:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 Processor -- start 2024-09-13T08:45:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e4ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e4bae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4da2f0 0x55e21e4bb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e21e2755d0 con 0x55e21e48de10 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e21e2ded60 con 0x55e21e4d99d0 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55e21e281500 con 0x55e21e4da2f0 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb0c07c000 10 client.0 _ll_register_callbacks cb 0x55e21e285810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb01e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e4ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb01e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e4ba6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42348/0 (socket says 172.21.15.44:42348) 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb01e00640 1 -- 172.21.15.44:0/1799932811 learned_addr learned my addr 172.21.15.44:0/1799932811 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb02800640 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4da2f0 0x55e21e4bb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcafbe00640 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e4bae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb01e00640 1 -- 172.21.15.44:0/1799932811 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4da2f0 msgr2=0x55e21e4bb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb01e00640 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4da2f0 0x55e21e4bb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb01e00640 1 -- 172.21.15.44:0/1799932811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 msgr2=0x55e21e4bae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb01e00640 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e4bae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcb01e00640 1 -- 172.21.15.44:0/1799932811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21e279170 con 0x55e21e48de10 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.253+0000 7fcafbe00640 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e4bae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb01e00640 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e4ba6a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4004600 tx=0x7fcaf4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0aa19b57c56c5e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcae7400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcae7400640 1 -- 172.21.15.44:0/1799932811 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf401f980 con 0x55e21e48de10 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcae7400640 1 -- 172.21.15.44:0/1799932811 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7fcaf4029b30 con 0x55e21e48de10 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1799932811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e21e2730f0 con 0x55e21e48de10 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcae7400640 1 -- 172.21.15.44:0/1799932811 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fcaf4029d70 con 0x55e21e48de10 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 10 client.6486 fetch_fsmap learned FSMap version 46 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1799932811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55e21e35fef0 con 0x55e21e48de10 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcae7400640 1 -- 172.21.15.44:0/1799932811 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55e21e2730f0 con 0x55e21e48de10 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcae7400640 1 -- 172.21.15.44:0/1799932811 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7fcaf4030050 con 0x55e21e48de10 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 10 client.6486 fetch_fsmap finished waiting for FSMap version 46 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 10 client.6486 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T08:45:44.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 20 client.6486 populate_metadata read hostname 'smithi044' 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.44:6833/651900305,v1:172.21.15.44:6835/651900305] conn(0x55e21e4bc0f0 0x55e21e4ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 4 client.6486 mds_command: new command op to 6333 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1799932811 --> [v2:172.21.15.44:6833/651900305,v1:172.21.15.44:6835/651900305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e21e2eda70 con 0x55e21e4bc0f0 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcafbe00640 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.44:6833/651900305,v1:172.21.15.44:6835/651900305] conn(0x55e21e4bc0f0 0x55e21e4ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcafbe00640 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.44:6833/651900305,v1:172.21.15.44:6835/651900305] conn(0x55e21e4bc0f0 0x55e21e4ebc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4e742f9422e709 server_cookie=a544fe42142dbc in_seq=0 out_seq=0 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcae7400640 10 client.6486 ms_handle_connect on v2:172.21.15.44:6833/651900305 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcae7400640 1 -- 172.21.15.44:0/1799932811 <== mds.0 v2:172.21.15.44:6833/651900305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e21e2eda70 con 0x55e21e4bc0f0 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcae7400640 10 client.6486 handle_command_reply: tid=0 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 client.6486 shutdown 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1799932811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 msgr2=0x55e21e4ba6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e4ba6a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4004600 tx=0x7fcaf4004740 comp rx=0 tx=0).stop 2024-09-13T08:45:44.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1799932811 shutdown_connections 2024-09-13T08:45:44.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.44:6833/651900305,v1:172.21.15.44:6835/651900305] conn(0x55e21e4bc0f0 0x55e21e4ebc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4da2f0 0x55e21e4bb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e4bae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/1799932811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e4ba6a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1799932811 >> 172.21.15.44:0/1799932811 conn(0x55e21e377130 msgr2=0x55e21e48cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:44.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcae7400640 0 client.6486 ms_handle_reset on v2:172.21.15.44:6833/651900305 2024-09-13T08:45:44.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1799932811 shutdown_connections 2024-09-13T08:45:44.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 1 -- 172.21.15.44:0/1799932811 wait complete. 2024-09-13T08:45:44.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.257+0000 7fcb0c07c000 20 client.6486 trim_cache size 0 max 16384 2024-09-13T08:45:44.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb0c07c000 1 Processor -- start 2024-09-13T08:45:44.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb0c07c000 1 -- start start 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e48de10 0x55e21e54bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e54c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e54ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb04203ee0 con 0x55e21e4da2f0 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040b0fb0 con 0x55e21e4d99d0 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcb040b0310 con 0x55e21e48de10 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb02800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e54ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb02800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e54ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42356/0 (socket says 172.21.15.44:42356) 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.277+0000 7fcb02800640 1 -- 172.21.15.44:0/2306088637 learned_addr learned my addr 172.21.15.44:0/2306088637 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb02800640 1 -- 172.21.15.44:0/2306088637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e48de10 msgr2=0x55e21e54bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:45:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb01e00640 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e48de10 0x55e21e54bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcafbe00640 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e54c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb02800640 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e48de10 0x55e21e54bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb02800640 1 -- 172.21.15.44:0/2306088637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 msgr2=0x55e21e54c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb02800640 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e54c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb02800640 1 -- 172.21.15.44:0/2306088637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21e336d90 con 0x55e21e4da2f0 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcafbe00640 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e54c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb01e00640 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e48de10 0x55e21e54bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb02800640 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e54ca60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0f8970 tx=0x7fcafc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cb7a97b48fb1a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcaf2000640 1 -- 172.21.15.44:0/2306088637 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0f5030 con 0x55e21e4da2f0 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcaf2000640 1 -- 172.21.15.44:0/2306088637 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcafc112030 con 0x55e21e4da2f0 2024-09-13T08:45:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcaf2000640 1 -- 172.21.15.44:0/2306088637 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0f84f0 con 0x55e21e4da2f0 2024-09-13T08:45:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2306088637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 msgr2=0x55e21e54ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e54ca60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0f8970 tx=0x7fcafc009500 comp rx=0 tx=0).stop 2024-09-13T08:45:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2306088637 shutdown_connections 2024-09-13T08:45:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e4da2f0 0x55e21e54ca60 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e54c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/2306088637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e48de10 0x55e21e54bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2306088637 >> 172.21.15.44:0/2306088637 conn(0x55e21e377130 msgr2=0x55e21e495a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2306088637 shutdown_connections 2024-09-13T08:45:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2306088637 wait complete. 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 -- start start 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 Processor -- start 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e550230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e550770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4da2f0 0x55e21e550cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e21e2755d0 con 0x55e21e48de10 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e21e4f0ce0 con 0x55e21e4d99d0 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55e21e39a730 con 0x55e21e4da2f0 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb0c07c000 10 client.0 _ll_register_callbacks cb 0x7fcaf4011be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb01e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e550230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb01e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e550230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42372/0 (socket says 172.21.15.44:42372) 2024-09-13T08:45:44.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb01e00640 1 -- 172.21.15.44:0/2800048226 learned_addr learned my addr 172.21.15.44:0/2800048226 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb01e00640 1 -- 172.21.15.44:0/2800048226 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4da2f0 msgr2=0x55e21e550cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb01e00640 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4da2f0 0x55e21e550cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb01e00640 1 -- 172.21.15.44:0/2800048226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 msgr2=0x55e21e550770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb01e00640 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e550770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.281+0000 7fcb01e00640 1 -- 172.21.15.44:0/2800048226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21e2a1a90 con 0x55e21e48de10 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb01e00640 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e550230 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcaf401fda0 tx=0x7fcaf40043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905d284a668e24a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcae7400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcae7400640 1 -- 172.21.15.44:0/2800048226 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf401f310 con 0x55e21e48de10 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcae7400640 1 -- 172.21.15.44:0/2800048226 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7fcaf4032670 con 0x55e21e48de10 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2800048226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcafc112030 con 0x55e21e48de10 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcae7400640 1 -- 172.21.15.44:0/2800048226 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fcaf403e050 con 0x55e21e48de10 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 10 client.6492 fetch_fsmap learned FSMap version 46 2024-09-13T08:45:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2800048226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55e21e280050 con 0x55e21e48de10 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcae7400640 1 -- 172.21.15.44:0/2800048226 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcafc112030 con 0x55e21e48de10 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcae7400640 1 -- 172.21.15.44:0/2800048226 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7fcaf4041020 con 0x55e21e48de10 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 10 client.6492 fetch_fsmap finished waiting for FSMap version 46 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 10 client.6492 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 20 client.6492 populate_metadata read hostname 'smithi044' 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.44:6833/651900305,v1:172.21.15.44:6835/651900305] conn(0x55e21e4edfb0 0x55e21e4ee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 4 client.6492 mds_command: new command op to 6333 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2800048226 --> [v2:172.21.15.44:6833/651900305,v1:172.21.15.44:6835/651900305] -- command(tid 0: {"prefix": "status"}) -- 0x55e21e2e9520 con 0x55e21e4edfb0 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcafbe00640 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.44:6833/651900305,v1:172.21.15.44:6835/651900305] conn(0x55e21e4edfb0 0x55e21e4ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcafbe00640 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.44:6833/651900305,v1:172.21.15.44:6835/651900305] conn(0x55e21e4edfb0 0x55e21e4ee390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3129ca469897419b server_cookie=98dcda9c33b7af15 in_seq=0 out_seq=0 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcae7400640 10 client.6492 ms_handle_connect on v2:172.21.15.44:6833/651900305 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcae7400640 1 -- 172.21.15.44:0/2800048226 <== mds.0 v2:172.21.15.44:6833/651900305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55e21e2e9520 con 0x55e21e4edfb0 2024-09-13T08:45:44.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcae7400640 10 client.6492 handle_command_reply: tid=0 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 client.6492 shutdown 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2800048226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 msgr2=0x55e21e550230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e550230 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcaf401fda0 tx=0x7fcaf40043e0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2800048226 shutdown_connections 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.44:6833/651900305,v1:172.21.15.44:6835/651900305] conn(0x55e21e4edfb0 0x55e21e4ee390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55e21e4da2f0 0x55e21e550cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e21e4d99d0 0x55e21e550770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 --2- 172.21.15.44:0/2800048226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e21e48de10 0x55e21e550230 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcae7400640 0 client.6492 ms_handle_reset on v2:172.21.15.44:6833/651900305 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2800048226 >> 172.21.15.44:0/2800048226 conn(0x55e21e377130 msgr2=0x55e21e4d8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2800048226 shutdown_connections 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 1 -- 172.21.15.44:0/2800048226 wait complete. 2024-09-13T08:45:44.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb0c07c000 20 client.6492 trim_cache size 0 max 16384 2024-09-13T08:45:44.292 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:45:44.292 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:45:44.292 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T08:45:44.292 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 6333, 2024-09-13T08:45:44.292 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T08:45:44.292 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T08:45:44.293 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T08:45:44.293 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 10.640892317, 2024-09-13T08:45:44.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 46, 2024-09-13T08:45:44.293 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 75, 2024-09-13T08:45:44.293 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-09-13T08:45:44.293 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 15.083494907 2024-09-13T08:45:44.293 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:45:44.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 -- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae0073b20 msgr2=0x7fcae0075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae0073b20 0x7fcae0075fe0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40096c0 tx=0x7fcaf4009260 comp rx=0 tx=0).stop 2024-09-13T08:45:44.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 -- 172.21.15.44:0/3330198903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb04159990 msgr2=0x7fcb040ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb04159990 0x7fcb040ade00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009b10 tx=0x7fcafc0092c0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 -- 172.21.15.44:0/3330198903 shutdown_connections 2024-09-13T08:45:44.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae0073b20 0x7fcae0075fe0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb04159990 0x7fcb040ade00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcb040afac0 0x7fcb041fada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 --2- 172.21.15.44:0/3330198903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406f1d0 0x7fcb04153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 -- 172.21.15.44:0/3330198903 >> 172.21.15.44:0/3330198903 conn(0x7fcb040b2240 msgr2=0x7fcb040b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:44.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 -- 172.21.15.44:0/3330198903 shutdown_connections 2024-09-13T08:45:44.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.285+0000 7fcb08c00640 1 -- 172.21.15.44:0/3330198903 wait complete. 2024-09-13T08:45:44.307 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:45:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.453+0000 7f4424e00640 1 -- 172.21.15.44:0/3118691889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44200cecf0 msgr2=0x7f44200cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.453+0000 7f4424e00640 1 --2- 172.21.15.44:0/3118691889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44200cecf0 0x7f44200cc940 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f44200d0780 tx=0x7f440801c700 comp rx=0 tx=0).stop 2024-09-13T08:45:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.453+0000 7f4424e00640 1 -- 172.21.15.44:0/3118691889 shutdown_connections 2024-09-13T08:45:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.453+0000 7f4424e00640 1 --2- 172.21.15.44:0/3118691889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44200cecf0 0x7f44200cc940 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.453+0000 7f4424e00640 1 -- 172.21.15.44:0/3118691889 >> 172.21.15.44:0/3118691889 conn(0x7f44200d1ea0 msgr2=0x7f44200d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.453+0000 7f4424e00640 1 -- 172.21.15.44:0/3118691889 shutdown_connections 2024-09-13T08:45:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.453+0000 7f4424e00640 1 -- 172.21.15.44:0/3118691889 wait complete. 2024-09-13T08:45:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.453+0000 7f4424e00640 1 Processor -- start 2024-09-13T08:45:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.453+0000 7f4424e00640 1 -- start start 2024-09-13T08:45:44.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4424e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44200cecf0 0x7f442014fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4424e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f442015bd50 0x7f44200b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4424e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442015c130 0x7f4420155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4424e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f442016a5b0 con 0x7f44200cecf0 2024-09-13T08:45:44.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4424e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f442016a430 con 0x7f442015c130 2024-09-13T08:45:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4424e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f442016a730 con 0x7f442015bd50 2024-09-13T08:45:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44200cecf0 0x7f442014fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44200cecf0 0x7f442014fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42388/0 (socket says 172.21.15.44:42388) 2024-09-13T08:45:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441de00640 1 -- 172.21.15.44:0/747465704 learned_addr learned my addr 172.21.15.44:0/747465704 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4417e00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f442015bd50 0x7f44200b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441e800640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442015c130 0x7f4420155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441de00640 1 -- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f442015bd50 msgr2=0x7f44200b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441de00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f442015bd50 0x7f44200b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441de00640 1 -- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442015c130 msgr2=0x7f4420155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441de00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442015c130 0x7f4420155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441de00640 1 -- 172.21.15.44:0/747465704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44200b5fc0 con 0x7f44200cecf0 2024-09-13T08:45:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441e800640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442015c130 0x7f4420155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f441de00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44200cecf0 0x7f442014fdd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f441000c4c0 tx=0x7f441000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4416000640 1 -- 172.21.15.44:0/747465704 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44100052d0 con 0x7f44200cecf0 2024-09-13T08:45:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4416000640 1 -- 172.21.15.44:0/747465704 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4410005490 con 0x7f44200cecf0 2024-09-13T08:45:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4424e00640 1 -- 172.21.15.44:0/747465704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44200b62a0 con 0x7f44200cecf0 2024-09-13T08:45:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4416000640 1 -- 172.21.15.44:0/747465704 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410012870 con 0x7f44200cecf0 2024-09-13T08:45:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4424e00640 1 -- 172.21.15.44:0/747465704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44200b6860 con 0x7f44200cecf0 2024-09-13T08:45:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.457+0000 7f4424e00640 1 -- 172.21.15.44:0/747465704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44200ce5e0 con 0x7f44200cecf0 2024-09-13T08:45:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.461+0000 7f4416000640 1 -- 172.21.15.44:0/747465704 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f4410012a10 con 0x7f44200cecf0 2024-09-13T08:45:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.461+0000 7f4416000640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43fc073b20 0x7f43fc075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.461+0000 7f4416000640 1 -- 172.21.15.44:0/747465704 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f4410092300 con 0x7f44200cecf0 2024-09-13T08:45:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.461+0000 7f4417e00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43fc073b20 0x7f43fc075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.461+0000 7f4417e00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43fc073b20 0x7f43fc075fe0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f44080af000 tx=0x7f440801af40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.465+0000 7f4416000640 1 -- 172.21.15.44:0/747465704 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f441005f000 con 0x7f44200cecf0 2024-09-13T08:45:44.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.677+0000 7f4424e00640 1 -- 172.21.15.44:0/747465704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4420071570 con 0x7f44200cecf0 2024-09-13T08:45:44.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4416000640 1 -- 172.21.15.44:0/747465704 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f441005e900 con 0x7f44200cecf0 2024-09-13T08:45:44.681 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:45:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 -- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43fc073b20 msgr2=0x7f43fc075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43fc073b20 0x7f43fc075fe0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f44080af000 tx=0x7f440801af40 comp rx=0 tx=0).stop 2024-09-13T08:45:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 -- 172.21.15.44:0/747465704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44200cecf0 msgr2=0x7f442014fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44200cecf0 0x7f442014fdd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f441000c4c0 tx=0x7f441000c980 comp rx=0 tx=0).stop 2024-09-13T08:45:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 -- 172.21.15.44:0/747465704 shutdown_connections 2024-09-13T08:45:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43fc073b20 0x7f43fc075fe0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442015c130 0x7f4420155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f442015bd50 0x7f44200b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 --2- 172.21.15.44:0/747465704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44200cecf0 0x7f442014fdd0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:44.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 -- 172.21.15.44:0/747465704 >> 172.21.15.44:0/747465704 conn(0x7f44200d1ea0 msgr2=0x7f442015b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:44.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 -- 172.21.15.44:0/747465704 shutdown_connections 2024-09-13T08:45:44.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44.681+0000 7f4424e00640 1 -- 172.21.15.44:0/747465704 wait complete. 2024-09-13T08:45:44.703 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T08:45:44.703 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:45:44.703 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:45:44.703 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T08:45:44.703 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:45:44.703 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T08:45:44.703 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T08:45:44.703 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:45:44.703 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:45:44.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:44.704 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:45:44.714 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41938sec preferred_lft 41938sec 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:45:44.715 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:45:44.716 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:45:44.717 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:45:44.717 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T08:45:44.717 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:45:44.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:44.718 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:45:44.778 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:45:44.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:44.780 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:45:44.833 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:45:44.833 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:45:44.834 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:45:44.834 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:45:44.834 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:45:44.834 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:45:44.834 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:45:44.834 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:45:44.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:45:44.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:45:44.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:45:44.834 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:45:44.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:45:44.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:45:44.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:44.973 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:45:44.983 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:45:44.983 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T08:45:44.984 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:45:44.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:44.984 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T08:45:45.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:45:45.044 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T08:45:45.100 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:45:45.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:45:45.100 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:45:45.151 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:45:45.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:45.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:45:45.241 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:45:45.241 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:45:45.241 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:45:45.256 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:45:45.256 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:45:45.257 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:45:45.257 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:45:45.257 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:45:45.265 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:45:45.265 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:45:45.266 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:45:45.266 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:45:45.266 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:45:45.266 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:45:45.266 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:45:45.266 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:45:45.270 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:45:45.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:45.270 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:45:45.290 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:45:45.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:45:45.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:45:45.395 INFO:teuthology.orchestra.run.smithi044.stdout:client6352 2024-09-13T08:45:45.396 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:45:45.396 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:45.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-13T08:45:45.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.597+0000 7ff81fa00640 1 -- 172.21.15.44:0/4257287232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8180d28e0 msgr2=0x7ff8180d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:45.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.597+0000 7ff81fa00640 1 --2- 172.21.15.44:0/4257287232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8180d28e0 0x7ff8180d0340 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff804009840 tx=0x7ff80401c7f0 comp rx=0 tx=0).stop 2024-09-13T08:45:45.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.597+0000 7ff81fa00640 1 -- 172.21.15.44:0/4257287232 shutdown_connections 2024-09-13T08:45:45.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.597+0000 7ff81fa00640 1 --2- 172.21.15.44:0/4257287232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8180d28e0 0x7ff8180d0340 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:45.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.597+0000 7ff81fa00640 1 -- 172.21.15.44:0/4257287232 >> 172.21.15.44:0/4257287232 conn(0x7ff8180d1dc0 msgr2=0x7ff8180d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:45.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.597+0000 7ff81fa00640 1 -- 172.21.15.44:0/4257287232 shutdown_connections 2024-09-13T08:45:45.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.597+0000 7ff81fa00640 1 -- 172.21.15.44:0/4257287232 wait complete. 2024-09-13T08:45:45.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 Processor -- start 2024-09-13T08:45:45.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 -- start start 2024-09-13T08:45:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8180af850 0x7ff8180cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8180d28e0 0x7ff8180ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff81815f390 0x7ff818151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff818169eb0 con 0x7ff8180d28e0 2024-09-13T08:45:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff818169d30 con 0x7ff8180af850 2024-09-13T08:45:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff81816a030 con 0x7ff81815f390 2024-09-13T08:45:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff817e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8180d28e0 0x7ff8180ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff817e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8180d28e0 0x7ff8180ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42404/0 (socket says 172.21.15.44:42404) 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff817e00640 1 -- 172.21.15.44:0/2744658782 learned_addr learned my addr 172.21.15.44:0/2744658782 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81d000640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8180af850 0x7ff8180cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81da00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff81815f390 0x7ff818151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff817e00640 1 -- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff81815f390 msgr2=0x7ff818151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff817e00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff81815f390 0x7ff818151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff817e00640 1 -- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8180af850 msgr2=0x7ff8180cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff817e00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8180af850 0x7ff8180cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff817e00640 1 -- 172.21.15.44:0/2744658782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8180ce7d0 con 0x7ff8180d28e0 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81da00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff81815f390 0x7ff818151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff817e00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8180d28e0 0x7ff8180ce290 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff80401a420 tx=0x7ff804004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff814c00640 1 -- 172.21.15.44:0/2744658782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040bb070 con 0x7ff8180d28e0 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff814c00640 1 -- 172.21.15.44:0/2744658782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff804005320 con 0x7ff8180d28e0 2024-09-13T08:45:45.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 -- 172.21.15.44:0/2744658782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8180cea70 con 0x7ff8180d28e0 2024-09-13T08:45:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff814c00640 1 -- 172.21.15.44:0/2744658782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040b2040 con 0x7ff8180d28e0 2024-09-13T08:45:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 -- 172.21.15.44:0/2744658782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8180cf030 con 0x7ff8180d28e0 2024-09-13T08:45:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.601+0000 7ff81fa00640 1 -- 172.21.15.44:0/2744658782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8180afc30 con 0x7ff8180d28e0 2024-09-13T08:45:45.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.605+0000 7ff814c00640 1 -- 172.21.15.44:0/2744658782 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7ff8040be070 con 0x7ff8180d28e0 2024-09-13T08:45:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.605+0000 7ff814c00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7f8073b20 0x7ff7f8075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.605+0000 7ff814c00640 1 -- 172.21.15.44:0/2744658782 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7ff80412c1d0 con 0x7ff8180d28e0 2024-09-13T08:45:45.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.605+0000 7ff81d000640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7f8073b20 0x7ff7f8075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:45.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.605+0000 7ff81d000640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7f8073b20 0x7ff7f8075fe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff80c001650 tx=0x7ff80c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:45.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.609+0000 7ff814c00640 1 -- 172.21.15.44:0/2744658782 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff804016410 con 0x7ff8180d28e0 2024-09-13T08:45:45.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:45.825+0000 7ff81fa00640 1 -- 172.21.15.44:0/2744658782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7ff81815e1b0 con 0x7ff8180d28e0 2024-09-13T08:45:46.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.725+0000 7ff814c00640 1 -- 172.21.15.44:0/2744658782 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7ff8040f8e70 con 0x7ff8180d28e0 2024-09-13T08:45:46.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 -- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7f8073b20 msgr2=0x7ff7f8075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:46.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7f8073b20 0x7ff7f8075fe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff80c001650 tx=0x7ff80c009040 comp rx=0 tx=0).stop 2024-09-13T08:45:46.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 -- 172.21.15.44:0/2744658782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8180d28e0 msgr2=0x7ff8180ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:46.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8180d28e0 0x7ff8180ce290 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff80401a420 tx=0x7ff804004220 comp rx=0 tx=0).stop 2024-09-13T08:45:46.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 -- 172.21.15.44:0/2744658782 shutdown_connections 2024-09-13T08:45:46.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff7f8073b20 0x7ff7f8075fe0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:46.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff81815f390 0x7ff818151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:46.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8180d28e0 0x7ff8180ce290 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:46.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 --2- 172.21.15.44:0/2744658782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8180af850 0x7ff8180cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:46.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 -- 172.21.15.44:0/2744658782 >> 172.21.15.44:0/2744658782 conn(0x7ff8180d1dc0 msgr2=0x7ff8180711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:46.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 -- 172.21.15.44:0/2744658782 shutdown_connections 2024-09-13T08:45:46.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.729+0000 7ff81fa00640 1 -- 172.21.15.44:0/2744658782 wait complete. 2024-09-13T08:45:46.752 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:45:46.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.905+0000 7f9fa7600640 1 -- 172.21.15.44:0/3382393433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f88005240 msgr2=0x7f9f8800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:46.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.905+0000 7f9fa7600640 1 --2- 172.21.15.44:0/3382393433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f88005240 0x7f9f8800b9d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9f98001dc0 tx=0x7f9f9801ca70 comp rx=0 tx=0).stop 2024-09-13T08:45:46.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.905+0000 7f9fa7600640 1 -- 172.21.15.44:0/3382393433 shutdown_connections 2024-09-13T08:45:46.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.905+0000 7f9fa7600640 1 --2- 172.21.15.44:0/3382393433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f88005240 0x7f9f8800b9d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:46.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.905+0000 7f9fa7600640 1 -- 172.21.15.44:0/3382393433 >> 172.21.15.44:0/3382393433 conn(0x7f9fa00b1c20 msgr2=0x7f9fa00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:46.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.905+0000 7f9fa7600640 1 -- 172.21.15.44:0/3382393433 shutdown_connections 2024-09-13T08:45:46.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.905+0000 7f9fa7600640 1 -- 172.21.15.44:0/3382393433 wait complete. 2024-09-13T08:45:46.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.905+0000 7f9fa7600640 1 Processor -- start 2024-09-13T08:45:46.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.905+0000 7f9fa7600640 1 -- start start 2024-09-13T08:45:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa7600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9f88005240 0x7f9fa014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa7600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fa0147b50 0x7f9fa0159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa7600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa0149120 0x7f9fa0149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa7600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa006ee60 con 0x7f9fa0147b50 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa7600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0073230 con 0x7f9fa0149120 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa7600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa006efe0 con 0x7f9f88005240 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fa0147b50 0x7f9fa0159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fa0147b50 0x7f9fa0159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42432/0 (socket says 172.21.15.44:42432) 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9fe00640 1 -- 172.21.15.44:0/1380068295 learned_addr learned my addr 172.21.15.44:0/1380068295 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa4c00640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9f88005240 0x7f9fa014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa5600640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa0149120 0x7f9fa0149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9fe00640 1 -- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9f88005240 msgr2=0x7f9fa014a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9fe00640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9f88005240 0x7f9fa014a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:46.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9fe00640 1 -- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa0149120 msgr2=0x7f9fa0149580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:46.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9fe00640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa0149120 0x7f9fa0149580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:46.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9fe00640 1 -- 172.21.15.44:0/1380068295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa014ab30 con 0x7f9fa0147b50 2024-09-13T08:45:46.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa4c00640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9f88005240 0x7f9fa014a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:46.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa5600640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa0149120 0x7f9fa0149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:46.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9fe00640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fa0147b50 0x7f9fa0159200 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000fc0 tx=0x7f9f8c00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:46.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9cc00640 1 -- 172.21.15.44:0/1380068295 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c015020 con 0x7f9fa0147b50 2024-09-13T08:45:46.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9cc00640 1 -- 172.21.15.44:0/1380068295 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f8c0109a0 con 0x7f9fa0147b50 2024-09-13T08:45:46.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa7600640 1 -- 172.21.15.44:0/1380068295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa0148080 con 0x7f9fa0147b50 2024-09-13T08:45:46.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9f9cc00640 1 -- 172.21.15.44:0/1380068295 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c010df0 con 0x7f9fa0147b50 2024-09-13T08:45:46.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa7600640 1 -- 172.21.15.44:0/1380068295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa0148640 con 0x7f9fa0147b50 2024-09-13T08:45:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.909+0000 7f9fa7600640 1 -- 172.21.15.44:0/1380068295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa0074d10 con 0x7f9fa0147b50 2024-09-13T08:45:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.913+0000 7f9f9cc00640 1 -- 172.21.15.44:0/1380068295 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f9f8c018020 con 0x7f9fa0147b50 2024-09-13T08:45:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.913+0000 7f9f9cc00640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f80073b50 0x7f9f80076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.913+0000 7f9f9cc00640 1 -- 172.21.15.44:0/1380068295 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f9f8c092f00 con 0x7f9fa0147b50 2024-09-13T08:45:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.913+0000 7f9fa4c00640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f80073b50 0x7f9f80076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.913+0000 7f9fa4c00640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f80073b50 0x7f9f80076010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009be0 tx=0x7f9f940091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:46.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:46.917+0000 7f9f9cc00640 1 -- 172.21.15.44:0/1380068295 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9f8c01f020 con 0x7f9fa0147b50 2024-09-13T08:45:47.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.117+0000 7f9fa7600640 1 -- 172.21.15.44:0/1380068295 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9fa006a330 con 0x7f9f80073b50 2024-09-13T08:45:47.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.117+0000 7f9f9cc00640 1 -- 172.21.15.44:0/1380068295 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9fa006a330 con 0x7f9f80073b50 2024-09-13T08:45:47.120 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:45:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:45:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:45:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:45:47.121 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:45:47.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 -- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f80073b50 msgr2=0x7f9f80076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f80073b50 0x7f9f80076010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009be0 tx=0x7f9f940091e0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 -- 172.21.15.44:0/1380068295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fa0147b50 msgr2=0x7f9fa0159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fa0147b50 0x7f9fa0159200 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000fc0 tx=0x7f9f8c00ea20 comp rx=0 tx=0).stop 2024-09-13T08:45:47.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 -- 172.21.15.44:0/1380068295 shutdown_connections 2024-09-13T08:45:47.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f80073b50 0x7f9f80076010 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa0149120 0x7f9fa0149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fa0147b50 0x7f9fa0159200 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 --2- 172.21.15.44:0/1380068295 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9f88005240 0x7f9fa014a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 -- 172.21.15.44:0/1380068295 >> 172.21.15.44:0/1380068295 conn(0x7f9fa00b1c20 msgr2=0x7f9fa00b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:47.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 -- 172.21.15.44:0/1380068295 shutdown_connections 2024-09-13T08:45:47.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.121+0000 7f9fa7600640 1 -- 172.21.15.44:0/1380068295 wait complete. 2024-09-13T08:45:47.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:45:47.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.289+0000 7f08d5c00640 1 -- 172.21.15.44:0/2449593605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d00745a0 msgr2=0x7f08d015fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.289+0000 7f08d5c00640 1 --2- 172.21.15.44:0/2449593605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d00745a0 0x7f08d015fe90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009c80 tx=0x7f08c801cc50 comp rx=0 tx=0).stop 2024-09-13T08:45:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.289+0000 7f08d5c00640 1 -- 172.21.15.44:0/2449593605 shutdown_connections 2024-09-13T08:45:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.289+0000 7f08d5c00640 1 --2- 172.21.15.44:0/2449593605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d00745a0 0x7f08d015fe90 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.289+0000 7f08d5c00640 1 --2- 172.21.15.44:0/2449593605 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08d015bdf0 0x7f08d015f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.289+0000 7f08d5c00640 1 -- 172.21.15.44:0/2449593605 >> 172.21.15.44:0/2449593605 conn(0x7f08d00b18f0 msgr2=0x7f08d00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.289+0000 7f08d5c00640 1 -- 172.21.15.44:0/2449593605 shutdown_connections 2024-09-13T08:45:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.289+0000 7f08d5c00640 1 -- 172.21.15.44:0/2449593605 wait complete. 2024-09-13T08:45:47.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 Processor -- start 2024-09-13T08:45:47.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 -- start start 2024-09-13T08:45:47.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08d00745a0 0x7f08d0152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:47.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08d00cd500 0x7f08d01525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:47.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d015bdf0 0x7f08d00acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:47.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f08d0168f30 con 0x7f08d015bdf0 2024-09-13T08:45:47.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08d015d030 con 0x7f08d00745a0 2024-09-13T08:45:47.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f08d01690b0 con 0x7f08d00cd500 2024-09-13T08:45:47.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d015bdf0 0x7f08d00acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:47.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d015bdf0 0x7f08d00acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42448/0 (socket says 172.21.15.44:42448) 2024-09-13T08:45:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08cf400640 1 -- 172.21.15.44:0/1758441490 learned_addr learned my addr 172.21.15.44:0/1758441490 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08cea00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08d00745a0 0x7f08d0152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08c7e00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08d00cd500 0x7f08d01525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08cf400640 1 -- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08d00cd500 msgr2=0x7f08d01525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08cf400640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08d00cd500 0x7f08d01525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08cf400640 1 -- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08d00745a0 msgr2=0x7f08d0152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08cf400640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08d00745a0 0x7f08d0152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08cf400640 1 -- 172.21.15.44:0/1758441490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d00b4140 con 0x7f08d015bdf0 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08cf400640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d015bdf0 0x7f08d00acd40 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f08c801c6c0 tx=0x7f08c80a5ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08c6a00640 1 -- 172.21.15.44:0/1758441490 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c80bb070 con 0x7f08d015bdf0 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08c6a00640 1 -- 172.21.15.44:0/1758441490 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08c8004d40 con 0x7f08d015bdf0 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08c6a00640 1 -- 172.21.15.44:0/1758441490 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c80b2040 con 0x7f08d015bdf0 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 -- 172.21.15.44:0/1758441490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d00b4420 con 0x7f08d015bdf0 2024-09-13T08:45:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 -- 172.21.15.44:0/1758441490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d00ae940 con 0x7f08d015bdf0 2024-09-13T08:45:47.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.293+0000 7f08d5c00640 1 -- 172.21.15.44:0/1758441490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d0073e90 con 0x7f08d015bdf0 2024-09-13T08:45:47.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.297+0000 7f08c6a00640 1 -- 172.21.15.44:0/1758441490 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f08c80be070 con 0x7f08d015bdf0 2024-09-13T08:45:47.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.297+0000 7f08c6a00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f08ac073b50 0x7f08ac076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:47.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.297+0000 7f08c6a00640 1 -- 172.21.15.44:0/1758441490 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f08c812c6d0 con 0x7f08d015bdf0 2024-09-13T08:45:47.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.297+0000 7f08cea00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f08ac073b50 0x7f08ac076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:47.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.297+0000 7f08cea00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f08ac073b50 0x7f08ac076010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f08d015ddd0 tx=0x7f08c000a250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:47.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.301+0000 7f08c6a00640 1 -- 172.21.15.44:0/1758441490 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f08c8016990 con 0x7f08d015bdf0 2024-09-13T08:45:47.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.477+0000 7f08d5c00640 1 -- 172.21.15.44:0/1758441490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f08d0070aa0 con 0x7f08d015bdf0 2024-09-13T08:45:47.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.477+0000 7f08c6a00640 1 -- 172.21.15.44:0/1758441490 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7f08c80f8d20 con 0x7f08d015bdf0 2024-09-13T08:45:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 -- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f08ac073b50 msgr2=0x7f08ac076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f08ac073b50 0x7f08ac076010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f08d015ddd0 tx=0x7f08c000a250 comp rx=0 tx=0).stop 2024-09-13T08:45:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 -- 172.21.15.44:0/1758441490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d015bdf0 msgr2=0x7f08d00acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d015bdf0 0x7f08d00acd40 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f08c801c6c0 tx=0x7f08c80a5ac0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 -- 172.21.15.44:0/1758441490 shutdown_connections 2024-09-13T08:45:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f08ac073b50 0x7f08ac076010 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d015bdf0 0x7f08d00acd40 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08d00cd500 0x7f08d01525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 --2- 172.21.15.44:0/1758441490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08d00745a0 0x7f08d0152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 -- 172.21.15.44:0/1758441490 >> 172.21.15.44:0/1758441490 conn(0x7f08d00b18f0 msgr2=0x7f08d00b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:47.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 -- 172.21.15.44:0/1758441490 shutdown_connections 2024-09-13T08:45:47.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.481+0000 7f08d5c00640 1 -- 172.21.15.44:0/1758441490 wait complete. 2024-09-13T08:45:47.501 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-13T08:45:47.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.649+0000 7fb3ca600640 1 -- 172.21.15.44:0/4118911217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c40ce7b0 msgr2=0x7fb3c40cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.649+0000 7fb3ca600640 1 --2- 172.21.15.44:0/4118911217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c40ce7b0 0x7fb3c40cf9c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40b7d20 tx=0x7fb3bc01cb50 comp rx=0 tx=0).stop 2024-09-13T08:45:47.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.649+0000 7fb3ca600640 1 -- 172.21.15.44:0/4118911217 shutdown_connections 2024-09-13T08:45:47.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.649+0000 7fb3ca600640 1 --2- 172.21.15.44:0/4118911217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c40ce7b0 0x7fb3c40cf9c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.649+0000 7fb3ca600640 1 --2- 172.21.15.44:0/4118911217 >> v2:172.21.15.129:3300/0 conn(0x7fb3c40b1190 0x7fb3c40ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.649+0000 7fb3ca600640 1 -- 172.21.15.44:0/4118911217 >> 172.21.15.44:0/4118911217 conn(0x7fb3c40d1df0 msgr2=0x7fb3c40d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:47.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.649+0000 7fb3ca600640 1 -- 172.21.15.44:0/4118911217 shutdown_connections 2024-09-13T08:45:47.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.649+0000 7fb3ca600640 1 -- 172.21.15.44:0/4118911217 wait complete. 2024-09-13T08:45:47.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.649+0000 7fb3ca600640 1 Processor -- start 2024-09-13T08:45:47.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.649+0000 7fb3ca600640 1 -- start start 2024-09-13T08:45:47.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3ca600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb3c40b1190 0x7fb3c41533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3ca600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3c40ce7b0 0x7fb3c4152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3ca600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c41603f0 0x7fb3c40b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3ca600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c40b3290 con 0x7fb3c41603f0 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3ca600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c40b2b80 con 0x7fb3c40ce7b0 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3ca600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c40b3410 con 0x7fb3c40b1190 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c41603f0 0x7fb3c40b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c41603f0 0x7fb3c40b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42470/0 (socket says 172.21.15.44:42470) 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3c3e00640 1 -- 172.21.15.44:0/2558941801 learned_addr learned my addr 172.21.15.44:0/2558941801 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3bbe00640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3c40ce7b0 0x7fb3c4152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3c3400640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb3c40b1190 0x7fb3c41533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3c3e00640 1 -- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb3c40b1190 msgr2=0x7fb3c41533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3c3e00640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb3c40b1190 0x7fb3c41533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3c3e00640 1 -- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3c40ce7b0 msgr2=0x7fb3c4152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3c3e00640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3c40ce7b0 0x7fb3c4152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3c3e00640 1 -- 172.21.15.44:0/2558941801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c406ddd0 con 0x7fb3c41603f0 2024-09-13T08:45:47.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3c3e00640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c41603f0 0x7fb3c40b0590 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009b10 tx=0x7fb3bc0a5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:47.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3bb400640 1 -- 172.21.15.44:0/2558941801 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0bb070 con 0x7fb3c41603f0 2024-09-13T08:45:47.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3bb400640 1 -- 172.21.15.44:0/2558941801 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3bc004d40 con 0x7fb3c41603f0 2024-09-13T08:45:47.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3bb400640 1 -- 172.21.15.44:0/2558941801 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0b2040 con 0x7fb3c41603f0 2024-09-13T08:45:47.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3ca600640 1 -- 172.21.15.44:0/2558941801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c406e0b0 con 0x7fb3c41603f0 2024-09-13T08:45:47.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3ca600640 1 -- 172.21.15.44:0/2558941801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c4166140 con 0x7fb3c41603f0 2024-09-13T08:45:47.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3ca600640 1 -- 172.21.15.44:0/2558941801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c40ce270 con 0x7fb3c41603f0 2024-09-13T08:45:47.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.653+0000 7fb3bb400640 1 -- 172.21.15.44:0/2558941801 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fb3bc0a5bb0 con 0x7fb3c41603f0 2024-09-13T08:45:47.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.657+0000 7fb3bb400640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb3a0073b20 0x7fb3a0075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:47.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.657+0000 7fb3bb400640 1 -- 172.21.15.44:0/2558941801 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7fb3bc12c570 con 0x7fb3c41603f0 2024-09-13T08:45:47.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.657+0000 7fb3c3400640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb3a0073b20 0x7fb3a0075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:47.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.657+0000 7fb3c3400640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb3a0073b20 0x7fb3a0075fe0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4007fd0 tx=0x7fb3b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:47.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.661+0000 7fb3bb400640 1 -- 172.21.15.44:0/2558941801 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb3bc016770 con 0x7fb3c41603f0 2024-09-13T08:45:47.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.837+0000 7fb3ca600640 1 -- 172.21.15.44:0/2558941801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fb3c40b7d70 con 0x7fb3c41603f0 2024-09-13T08:45:47.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.837+0000 7fb3bb400640 1 -- 172.21.15.44:0/2558941801 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v37) ==== 116+0+2 (secure 0 0 0) 0x7fb3bc0f8bc0 con 0x7fb3c41603f0 2024-09-13T08:45:47.839 INFO:teuthology.orchestra.run.smithi044.stdout:4 2024-09-13T08:45:47.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 -- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb3a0073b20 msgr2=0x7fb3a0075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb3a0073b20 0x7fb3a0075fe0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4007fd0 tx=0x7fb3b4009040 comp rx=0 tx=0).stop 2024-09-13T08:45:47.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 -- 172.21.15.44:0/2558941801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c41603f0 msgr2=0x7fb3c40b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:47.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c41603f0 0x7fb3c40b0590 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009b10 tx=0x7fb3bc0a5a40 comp rx=0 tx=0).stop 2024-09-13T08:45:47.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 -- 172.21.15.44:0/2558941801 shutdown_connections 2024-09-13T08:45:47.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb3a0073b20 0x7fb3a0075fe0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c41603f0 0x7fb3c40b0590 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3c40ce7b0 0x7fb3c4152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 --2- 172.21.15.44:0/2558941801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb3c40b1190 0x7fb3c41533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:47.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 -- 172.21.15.44:0/2558941801 >> 172.21.15.44:0/2558941801 conn(0x7fb3c40d1df0 msgr2=0x7fb3c40b7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:47.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 -- 172.21.15.44:0/2558941801 shutdown_connections 2024-09-13T08:45:47.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:47.841+0000 7fb3ca600640 1 -- 172.21.15.44:0/2558941801 wait complete. 2024-09-13T08:45:47.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:45:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.009+0000 7fa280800640 1 -- 172.21.15.44:0/2184164453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27815db50 msgr2=0x7fa2780b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.009+0000 7fa280800640 1 --2- 172.21.15.44:0/2184164453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27815db50 0x7fa2780b4340 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa274009c80 tx=0x7fa27401cc80 comp rx=0 tx=0).stop 2024-09-13T08:45:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.009+0000 7fa280800640 1 -- 172.21.15.44:0/2184164453 shutdown_connections 2024-09-13T08:45:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.009+0000 7fa280800640 1 --2- 172.21.15.44:0/2184164453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27815db50 0x7fa2780b4340 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.009+0000 7fa280800640 1 --2- 172.21.15.44:0/2184164453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa27806f8b0 0x7fa2780acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.009+0000 7fa280800640 1 -- 172.21.15.44:0/2184164453 >> 172.21.15.44:0/2184164453 conn(0x7fa2780b18f0 msgr2=0x7fa2780b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.009+0000 7fa280800640 1 -- 172.21.15.44:0/2184164453 shutdown_connections 2024-09-13T08:45:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.009+0000 7fa280800640 1 -- 172.21.15.44:0/2184164453 wait complete. 2024-09-13T08:45:48.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa280800640 1 Processor -- start 2024-09-13T08:45:48.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa280800640 1 -- start start 2024-09-13T08:45:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa280800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27806f8b0 0x7fa278156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa280800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2780cd100 0x7fa278164ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa280800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa27815db50 0x7fa2781650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa280800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2780cf5a0 con 0x7fa27806f8b0 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa280800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2780cf420 con 0x7fa2780cd100 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa280800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa278168130 con 0x7fa27815db50 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27806f8b0 0x7fa278156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27806f8b0 0x7fa278156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42492/0 (socket says 172.21.15.44:42492) 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27de00640 1 -- 172.21.15.44:0/3655815121 learned_addr learned my addr 172.21.15.44:0/3655815121 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27d400640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2780cd100 0x7fa278164ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27e800640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa27815db50 0x7fa2781650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27de00640 1 -- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa27815db50 msgr2=0x7fa2781650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27de00640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa27815db50 0x7fa2781650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27de00640 1 -- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2780cd100 msgr2=0x7fa278164ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27de00640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2780cd100 0x7fa278164ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27de00640 1 -- 172.21.15.44:0/3655815121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27809bac0 con 0x7fa27806f8b0 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27d400640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2780cd100 0x7fa278164ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa27de00640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27806f8b0 0x7fa278156da0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00d950 tx=0x7fa26c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa272000640 1 -- 172.21.15.44:0/3655815121 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c017070 con 0x7fa27806f8b0 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa272000640 1 -- 172.21.15.44:0/3655815121 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa26c009d20 con 0x7fa27806f8b0 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa272000640 1 -- 172.21.15.44:0/3655815121 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0106c0 con 0x7fa27806f8b0 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa280800640 1 -- 172.21.15.44:0/3655815121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27809bd60 con 0x7fa27806f8b0 2024-09-13T08:45:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.013+0000 7fa280800640 1 -- 172.21.15.44:0/3655815121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2781572e0 con 0x7fa27806f8b0 2024-09-13T08:45:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.017+0000 7fa280800640 1 -- 172.21.15.44:0/3655815121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2780acd40 con 0x7fa27806f8b0 2024-09-13T08:45:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.017+0000 7fa272000640 1 -- 172.21.15.44:0/3655815121 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7fa26c010860 con 0x7fa27806f8b0 2024-09-13T08:45:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.017+0000 7fa272000640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa25c073b20 0x7fa25c075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.017+0000 7fa272000640 1 -- 172.21.15.44:0/3655815121 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7fa26c092c00 con 0x7fa27806f8b0 2024-09-13T08:45:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.017+0000 7fa27d400640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa25c073b20 0x7fa25c075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.021+0000 7fa27d400640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa25c073b20 0x7fa25c075fe0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa268007fd0 tx=0x7fa268009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:48.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.021+0000 7fa272000640 1 -- 172.21.15.44:0/3655815121 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa26c05f8a0 con 0x7fa27806f8b0 2024-09-13T08:45:48.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.193+0000 7fa280800640 1 -- 172.21.15.44:0/3655815121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fa2780008e0 con 0x7fa27806f8b0 2024-09-13T08:45:48.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa272000640 1 -- 172.21.15.44:0/3655815121 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v38)=0 v38) ==== 131+0+0 (secure 0 0 0) 0x7fa26c05f1a0 con 0x7fa27806f8b0 2024-09-13T08:45:48.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 -- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa25c073b20 msgr2=0x7fa25c075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa25c073b20 0x7fa25c075fe0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa268007fd0 tx=0x7fa268009040 comp rx=0 tx=0).stop 2024-09-13T08:45:48.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 -- 172.21.15.44:0/3655815121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27806f8b0 msgr2=0x7fa278156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27806f8b0 0x7fa278156da0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00d950 tx=0x7fa26c00de10 comp rx=0 tx=0).stop 2024-09-13T08:45:48.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 -- 172.21.15.44:0/3655815121 shutdown_connections 2024-09-13T08:45:48.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa25c073b20 0x7fa25c075fe0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa27815db50 0x7fa2781650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2780cd100 0x7fa278164ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 --2- 172.21.15.44:0/3655815121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27806f8b0 0x7fa278156da0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 -- 172.21.15.44:0/3655815121 >> 172.21.15.44:0/3655815121 conn(0x7fa2780b18f0 msgr2=0x7fa278072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:48.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 -- 172.21.15.44:0/3655815121 shutdown_connections 2024-09-13T08:45:48.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.201+0000 7fa280800640 1 -- 172.21.15.44:0/3655815121 wait complete. 2024-09-13T08:45:48.219 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:48.216+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:48.219 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:48.216+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:48.220 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:48.216+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:48.220 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:48.216+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:48.220 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:48.216+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:48.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-13T08:45:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 -- 172.21.15.44:0/475095757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a2c15dae0 msgr2=0x7f0a2c0acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 --2- 172.21.15.44:0/475095757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a2c15dae0 0x7f0a2c0acd40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0af800 tx=0x7f0a1c01c6e0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 -- 172.21.15.44:0/475095757 shutdown_connections 2024-09-13T08:45:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 --2- 172.21.15.44:0/475095757 >> v2:172.21.15.129:3300/0 conn(0x7f0a2c0cd100 0x7f0a2c0b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 --2- 172.21.15.44:0/475095757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a2c15dae0 0x7f0a2c0acd40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 -- 172.21.15.44:0/475095757 >> 172.21.15.44:0/475095757 conn(0x7f0a2c0b18f0 msgr2=0x7f0a2c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 -- 172.21.15.44:0/475095757 shutdown_connections 2024-09-13T08:45:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 -- 172.21.15.44:0/475095757 wait complete. 2024-09-13T08:45:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 Processor -- start 2024-09-13T08:45:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 -- start start 2024-09-13T08:45:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a2c0b4810 0x7f0a2c15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a2c0cd100 0x7f0a2c09c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a2c15dae0 0x7f0a2c09ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c167330 con 0x7f0a2c0cd100 2024-09-13T08:45:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c0cf520 con 0x7f0a2c15dae0 2024-09-13T08:45:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a32000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2c1674b0 con 0x7f0a2c0b4810 2024-09-13T08:45:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a22a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a2c0cd100 0x7f0a2c09c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a22a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a2c0cd100 0x7f0a2c09c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42504/0 (socket says 172.21.15.44:42504) 2024-09-13T08:45:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a22a00640 1 -- 172.21.15.44:0/589832697 learned_addr learned my addr 172.21.15.44:0/589832697 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a2b400640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a2c0b4810 0x7f0a2c15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a2be00640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a2c15dae0 0x7f0a2c09ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a22a00640 1 -- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a2c0b4810 msgr2=0x7f0a2c15a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a22a00640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a2c0b4810 0x7f0a2c15a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a22a00640 1 -- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a2c15dae0 msgr2=0x7f0a2c09ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a22a00640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a2c15dae0 0x7f0a2c09ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a22a00640 1 -- 172.21.15.44:0/589832697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c158a90 con 0x7f0a2c0cd100 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.373+0000 7f0a2be00640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a2c15dae0 0x7f0a2c09ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.377+0000 7f0a22a00640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a2c0cd100 0x7f0a2c09c550 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400ddc0 tx=0x7f0a1400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.377+0000 7f0a23e00640 1 -- 172.21.15.44:0/589832697 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1401b070 con 0x7f0a2c0cd100 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.377+0000 7f0a23e00640 1 -- 172.21.15.44:0/589832697 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a1400e740 con 0x7f0a2c0cd100 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.377+0000 7f0a32000640 1 -- 172.21.15.44:0/589832697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c1fb260 con 0x7f0a2c0cd100 2024-09-13T08:45:48.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.377+0000 7f0a23e00640 1 -- 172.21.15.44:0/589832697 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400eb90 con 0x7f0a2c0cd100 2024-09-13T08:45:48.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.377+0000 7f0a32000640 1 -- 172.21.15.44:0/589832697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c1fb520 con 0x7f0a2c0cd100 2024-09-13T08:45:48.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.377+0000 7f0a32000640 1 -- 172.21.15.44:0/589832697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c069410 con 0x7f0a2c0cd100 2024-09-13T08:45:48.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.377+0000 7f0a23e00640 1 -- 172.21.15.44:0/589832697 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f0a1401e070 con 0x7f0a2c0cd100 2024-09-13T08:45:48.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.377+0000 7f0a23e00640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a08073b20 0x7f0a08075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.377+0000 7f0a23e00640 1 -- 172.21.15.44:0/589832697 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f0a14092ea0 con 0x7f0a2c0cd100 2024-09-13T08:45:48.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.381+0000 7f0a2b400640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a08073b20 0x7f0a08075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.381+0000 7f0a2b400640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a08073b20 0x7f0a08075fe0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0af000 tx=0x7f0a1c0b2040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:48.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.381+0000 7f0a23e00640 1 -- 172.21.15.44:0/589832697 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0a1405f270 con 0x7f0a2c0cd100 2024-09-13T08:45:48.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.557+0000 7f0a32000640 1 -- 172.21.15.44:0/589832697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f0a2c0b1ee0 con 0x7f0a2c0cd100 2024-09-13T08:45:48.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.561+0000 7f0a23e00640 1 -- 172.21.15.44:0/589832697 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v39)=0 v39) ==== 136+0+0 (secure 0 0 0) 0x7f0a14063ca0 con 0x7f0a2c0cd100 2024-09-13T08:45:48.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 -- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a08073b20 msgr2=0x7f0a08075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a08073b20 0x7f0a08075fe0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0af000 tx=0x7f0a1c0b2040 comp rx=0 tx=0).stop 2024-09-13T08:45:48.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 -- 172.21.15.44:0/589832697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a2c0cd100 msgr2=0x7f0a2c09c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a2c0cd100 0x7f0a2c09c550 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400ddc0 tx=0x7f0a1400a620 comp rx=0 tx=0).stop 2024-09-13T08:45:48.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 -- 172.21.15.44:0/589832697 shutdown_connections 2024-09-13T08:45:48.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a08073b20 0x7f0a08075fe0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a2c15dae0 0x7f0a2c09ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a2c0cd100 0x7f0a2c09c550 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 --2- 172.21.15.44:0/589832697 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a2c0b4810 0x7f0a2c15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 -- 172.21.15.44:0/589832697 >> 172.21.15.44:0/589832697 conn(0x7f0a2c0b18f0 msgr2=0x7f0a2c1597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:48.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 -- 172.21.15.44:0/589832697 shutdown_connections 2024-09-13T08:45:48.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.565+0000 7f0a32000640 1 -- 172.21.15.44:0/589832697 wait complete. 2024-09-13T08:45:48.586 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-13T08:45:48.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.737+0000 7f669b200640 1 -- 172.21.15.44:0/2370261543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66940b2a20 msgr2=0x7f66940d0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.737+0000 7f669b200640 1 --2- 172.21.15.44:0/2370261543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66940b2a20 0x7f66940d0080 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6680009900 tx=0x7f668001c8e0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.737+0000 7f669b200640 1 -- 172.21.15.44:0/2370261543 shutdown_connections 2024-09-13T08:45:48.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.737+0000 7f669b200640 1 --2- 172.21.15.44:0/2370261543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66940b2a20 0x7f66940d0080 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.737+0000 7f669b200640 1 -- 172.21.15.44:0/2370261543 >> 172.21.15.44:0/2370261543 conn(0x7f66940d1c60 msgr2=0x7f66940d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:48.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.737+0000 7f669b200640 1 -- 172.21.15.44:0/2370261543 shutdown_connections 2024-09-13T08:45:48.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.737+0000 7f669b200640 1 -- 172.21.15.44:0/2370261543 wait complete. 2024-09-13T08:45:48.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 Processor -- start 2024-09-13T08:45:48.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 -- start start 2024-09-13T08:45:48.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66940b2a20 0x7f6694156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66940cfca0 0x7f669415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6694160150 0x7f669415bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6694169ee0 con 0x7f66940cfca0 2024-09-13T08:45:48.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6694169d60 con 0x7f6694160150 2024-09-13T08:45:48.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f669416a060 con 0x7f66940b2a20 2024-09-13T08:45:48.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f6699400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66940cfca0 0x7f669415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f6699400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66940cfca0 0x7f669415b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42522/0 (socket says 172.21.15.44:42522) 2024-09-13T08:45:48.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f6699400640 1 -- 172.21.15.44:0/3426899330 learned_addr learned my addr 172.21.15.44:0/3426899330 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:48.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f6699e00640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66940b2a20 0x7f6694156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669a800640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6694160150 0x7f669415bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f6699400640 1 -- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66940b2a20 msgr2=0x7f6694156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f6699400640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66940b2a20 0x7f6694156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f6699400640 1 -- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6694160150 msgr2=0x7f669415bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f6699400640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6694160150 0x7f669415bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f6699400640 1 -- 172.21.15.44:0/3426899330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6694164f90 con 0x7f66940cfca0 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f6699400640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66940cfca0 0x7f669415b850 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6680009850 tx=0x7f6680004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f668e000640 1 -- 172.21.15.44:0/3426899330 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66800b2040 con 0x7f66940cfca0 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f668e000640 1 -- 172.21.15.44:0/3426899330 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6680005020 con 0x7f66940cfca0 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 -- 172.21.15.44:0/3426899330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6694165260 con 0x7f66940cfca0 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f668e000640 1 -- 172.21.15.44:0/3426899330 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680005430 con 0x7f66940cfca0 2024-09-13T08:45:48.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 -- 172.21.15.44:0/3426899330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6694165aa0 con 0x7f66940cfca0 2024-09-13T08:45:48.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.741+0000 7f669b200640 1 -- 172.21.15.44:0/3426899330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66940d0080 con 0x7f66940cfca0 2024-09-13T08:45:48.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.745+0000 7f668e000640 1 -- 172.21.15.44:0/3426899330 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f66800046d0 con 0x7f66940cfca0 2024-09-13T08:45:48.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.745+0000 7f668e000640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6674073b20 0x7f6674075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:48.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.745+0000 7f668e000640 1 -- 172.21.15.44:0/3426899330 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f668012cd80 con 0x7f66940cfca0 2024-09-13T08:45:48.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.745+0000 7f6699e00640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6674073b20 0x7f6674075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:48.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.745+0000 7f6699e00640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6674073b20 0x7f6674075fe0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6688000f70 tx=0x7f6688009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:48.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.749+0000 7f668e000640 1 -- 172.21.15.44:0/3426899330 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6694165260 con 0x7f66940cfca0 2024-09-13T08:45:48.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.949+0000 7f669b200640 1 -- 172.21.15.44:0/3426899330 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f66940cc140 con 0x7f6674073b20 2024-09-13T08:45:48.959 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:48.956+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:48.959 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:48.956+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:48.959 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:48.956+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:48.959 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:48.956+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:48.959 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:45:48.956+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:45:48.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.993+0000 7f668e000640 1 -- 172.21.15.44:0/3426899330 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f66940cc140 con 0x7f6674073b20 2024-09-13T08:45:48.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 -- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6674073b20 msgr2=0x7f6674075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6674073b20 0x7f6674075fe0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6688000f70 tx=0x7f6688009040 comp rx=0 tx=0).stop 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 -- 172.21.15.44:0/3426899330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66940cfca0 msgr2=0x7f669415b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66940cfca0 0x7f669415b850 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6680009850 tx=0x7f6680004040 comp rx=0 tx=0).stop 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 -- 172.21.15.44:0/3426899330 shutdown_connections 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6674073b20 0x7f6674075fe0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6694160150 0x7f669415bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66940cfca0 0x7f669415b850 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 --2- 172.21.15.44:0/3426899330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66940b2a20 0x7f6694156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 -- 172.21.15.44:0/3426899330 >> 172.21.15.44:0/3426899330 conn(0x7f66940d1c60 msgr2=0x7f66940cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 -- 172.21.15.44:0/3426899330 shutdown_connections 2024-09-13T08:45:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:48.997+0000 7f669b200640 1 -- 172.21.15.44:0/3426899330 wait complete. 2024-09-13T08:45:49.018 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:45:49.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 -- 172.21.15.44:0/1585661981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6598003ab0 msgr2=0x7f6598003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:49.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 --2- 172.21.15.44:0/1585661981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6598003ab0 0x7f6598003ed0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f65a40016f0 tx=0x7f65a401c750 comp rx=0 tx=0).stop 2024-09-13T08:45:49.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 -- 172.21.15.44:0/1585661981 shutdown_connections 2024-09-13T08:45:49.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 --2- 172.21.15.44:0/1585661981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6598003ab0 0x7f6598003ed0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:49.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 -- 172.21.15.44:0/1585661981 >> 172.21.15.44:0/1585661981 conn(0x7f65b40b18f0 msgr2=0x7f65b40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:49.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 -- 172.21.15.44:0/1585661981 shutdown_connections 2024-09-13T08:45:49.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 -- 172.21.15.44:0/1585661981 wait complete. 2024-09-13T08:45:49.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 Processor -- start 2024-09-13T08:45:49.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 -- start start 2024-09-13T08:45:49.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6598003ab0 0x7f65b414de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b409c6b0 0x7f65b409cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65b415c0f0 0x7f65b415c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40cec80 con 0x7f65b409c6b0 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40ceb00 con 0x7f6598003ab0 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65baa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f65b40cee00 con 0x7f65b415c0f0 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b409c6b0 0x7f65b409cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b409c6b0 0x7f65b409cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42534/0 (socket says 172.21.15.44:42534) 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65abe00640 1 -- 172.21.15.44:0/1673112438 learned_addr learned my addr 172.21.15.44:0/1673112438 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65abe00640 1 -- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65b415c0f0 msgr2=0x7f65b415c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65abe00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65b415c0f0 0x7f65b415c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65abe00640 1 -- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6598003ab0 msgr2=0x7f65b414de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:49.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65b3e00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6598003ab0 0x7f65b414de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65abe00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6598003ab0 0x7f65b414de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65abe00640 1 -- 172.21.15.44:0/1673112438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b415bf00 con 0x7f65b409c6b0 2024-09-13T08:45:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.169+0000 7f65b3e00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6598003ab0 0x7f65b414de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:45:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65abe00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b409c6b0 0x7f65b409cb30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f659c004fd0 tx=0x7f659c00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65b0c00640 1 -- 172.21.15.44:0/1673112438 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c015070 con 0x7f65b409c6b0 2024-09-13T08:45:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65b0c00640 1 -- 172.21.15.44:0/1673112438 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f659c010980 con 0x7f65b409c6b0 2024-09-13T08:45:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65b0c00640 1 -- 172.21.15.44:0/1673112438 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c010db0 con 0x7f65b409c6b0 2024-09-13T08:45:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65baa00640 1 -- 172.21.15.44:0/1673112438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b40ca330 con 0x7f65b409c6b0 2024-09-13T08:45:49.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65baa00640 1 -- 172.21.15.44:0/1673112438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b40ca8f0 con 0x7f65b409c6b0 2024-09-13T08:45:49.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65baa00640 1 -- 172.21.15.44:0/1673112438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b4069410 con 0x7f65b409c6b0 2024-09-13T08:45:49.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65b0c00640 1 -- 172.21.15.44:0/1673112438 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98202+0+0 (secure 0 0 0) 0x7f659c018070 con 0x7f65b409c6b0 2024-09-13T08:45:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65b0c00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6590073b50 0x7f6590076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:45:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65b0c00640 1 -- 172.21.15.44:0/1673112438 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f659c092da0 con 0x7f65b409c6b0 2024-09-13T08:45:49.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.173+0000 7f65b3e00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6590073b50 0x7f6590076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:45:49.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.177+0000 7f65b3e00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6590073b50 0x7f6590076010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f65a40016c0 tx=0x7f65a4019540 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:45:49.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.177+0000 7f65b0c00640 1 -- 172.21.15.44:0/1673112438 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f659c097e20 con 0x7f65b409c6b0 2024-09-13T08:45:49.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.381+0000 7f65baa00640 1 -- 172.21.15.44:0/1673112438 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f65b40cf2f0 con 0x7f6590073b50 2024-09-13T08:45:49.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.385+0000 7f65b0c00640 1 -- 172.21.15.44:0/1673112438 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f65b40cf2f0 con 0x7f6590073b50 2024-09-13T08:45:49.388 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/2aa71638-d6c5-4b2f-97e9-9c31d677599f 2024-09-13T08:45:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 -- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6590073b50 msgr2=0x7f6590076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6590073b50 0x7f6590076010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f65a40016c0 tx=0x7f65a4019540 comp rx=0 tx=0).stop 2024-09-13T08:45:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 -- 172.21.15.44:0/1673112438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b409c6b0 msgr2=0x7f65b409cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:45:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b409c6b0 0x7f65b409cb30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f659c004fd0 tx=0x7f659c00ea20 comp rx=0 tx=0).stop 2024-09-13T08:45:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 -- 172.21.15.44:0/1673112438 shutdown_connections 2024-09-13T08:45:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6590073b50 0x7f6590076010 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65b415c0f0 0x7f65b415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b409c6b0 0x7f65b409cb30 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 --2- 172.21.15.44:0/1673112438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6598003ab0 0x7f65b414de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:45:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 -- 172.21.15.44:0/1673112438 >> 172.21.15.44:0/1673112438 conn(0x7f65b40b18f0 msgr2=0x7f65b40add00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:45:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 -- 172.21.15.44:0/1673112438 shutdown_connections 2024-09-13T08:45:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:45:49.389+0000 7f65baa00640 1 -- 172.21.15.44:0/1673112438 wait complete. 2024-09-13T08:45:49.410 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/2aa71638-d6c5-4b2f-97e9-9c31d677599f 2024-09-13T08:45:49.410 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:45:49.410 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:49.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:49.419 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/2aa71638-d6c5-4b2f-97e9-9c31d677599f/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:45:58.671 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:45:58.671 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:45:58.671 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.19622 s, 117 MB/s 2024-09-13T08:45:58.672 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:45:58.672 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:45:58.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:45:58.681 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/2aa71638-d6c5-4b2f-97e9-9c31d677599f/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:46:02.075 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:46:02.079 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:46:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:46:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:46:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.40783 s, 114 MB/s 2024-09-13T08:46:08.142 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:46:08.143 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:46:08.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:46:08.151 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/2aa71638-d6c5-4b2f-97e9-9c31d677599f/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:46:17.802 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:46:17.802 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:46:17.802 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.59961 s, 112 MB/s 2024-09-13T08:46:17.803 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-13T08:46:17.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.953+0000 7f9508c00640 1 -- 172.21.15.44:0/2779720051 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f95040734f0 msgr2=0x7f950415a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:17.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.953+0000 7f9508c00640 1 --2- 172.21.15.44:0/2779720051 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f95040734f0 0x7f950415a9d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009840 tx=0x7f94f401c820 comp rx=0 tx=0).stop 2024-09-13T08:46:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.953+0000 7f9508c00640 1 -- 172.21.15.44:0/2779720051 shutdown_connections 2024-09-13T08:46:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.953+0000 7f9508c00640 1 --2- 172.21.15.44:0/2779720051 >> v2:172.21.15.129:3300/0 conn(0x7f95040cf440 0x7f95040ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.953+0000 7f9508c00640 1 --2- 172.21.15.44:0/2779720051 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f95040734f0 0x7f950415a9d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.953+0000 7f9508c00640 1 -- 172.21.15.44:0/2779720051 >> 172.21.15.44:0/2779720051 conn(0x7f95040cfbc0 msgr2=0x7f95040cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.953+0000 7f9508c00640 1 -- 172.21.15.44:0/2779720051 shutdown_connections 2024-09-13T08:46:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.953+0000 7f9508c00640 1 -- 172.21.15.44:0/2779720051 wait complete. 2024-09-13T08:46:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 Processor -- start 2024-09-13T08:46:17.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 -- start start 2024-09-13T08:46:17.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95040734f0 0x7f9504150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:17.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f95040ce890 0x7f950415d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:17.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95040cf440 0x7f950415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:17.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9504167360 con 0x7f95040734f0 2024-09-13T08:46:17.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f950415ed80 con 0x7f95040cf440 2024-09-13T08:46:17.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f95041674e0 con 0x7f95040ce890 2024-09-13T08:46:17.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9501e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95040734f0 0x7f9504150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:17.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9501e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95040734f0 0x7f9504150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48714/0 (socket says 172.21.15.44:48714) 2024-09-13T08:46:17.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9501e00640 1 -- 172.21.15.44:0/1509695220 learned_addr learned my addr 172.21.15.44:0/1509695220 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:17.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9502800640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95040cf440 0x7f950415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:17.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9501e00640 1 -- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f95040ce890 msgr2=0x7f950415d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:17.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9501e00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f95040ce890 0x7f950415d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:17.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9501e00640 1 -- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95040cf440 msgr2=0x7f950415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:17.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9501e00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95040cf440 0x7f950415dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:17.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9501e00640 1 -- 172.21.15.44:0/1509695220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9504150b50 con 0x7f95040734f0 2024-09-13T08:46:17.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9502800640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95040cf440 0x7f950415dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:17.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9501e00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95040734f0 0x7f9504150610 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009790 tx=0x7f94f4005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:17.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f94fa000640 1 -- 172.21.15.44:0/1509695220 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f40b2040 con 0x7f95040734f0 2024-09-13T08:46:17.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f94fa000640 1 -- 172.21.15.44:0/1509695220 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94f4004240 con 0x7f95040734f0 2024-09-13T08:46:17.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f94fa000640 1 -- 172.21.15.44:0/1509695220 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4005020 con 0x7f95040734f0 2024-09-13T08:46:17.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 -- 172.21.15.44:0/1509695220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950409adf0 con 0x7f95040734f0 2024-09-13T08:46:17.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 -- 172.21.15.44:0/1509695220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950409b3b0 con 0x7f95040734f0 2024-09-13T08:46:17.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.957+0000 7f9508c00640 1 -- 172.21.15.44:0/1509695220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9504069410 con 0x7f95040734f0 2024-09-13T08:46:17.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.961+0000 7f94fa000640 1 -- 172.21.15.44:0/1509695220 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 31) ==== 98304+0+0 (secure 0 0 0) 0x7f94f40a55f0 con 0x7f95040734f0 2024-09-13T08:46:17.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.961+0000 7f94fa000640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0073c40 0x7f94e0076100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:17.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.961+0000 7f94fa000640 1 -- 172.21.15.44:0/1509695220 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f94f412c630 con 0x7f95040734f0 2024-09-13T08:46:17.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.961+0000 7f94fbe00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0073c40 0x7f94e0076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:17.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.961+0000 7f94fbe00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0073c40 0x7f94e0076100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f950415a3e0 tx=0x7f94ec00a610 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:17.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:17.965+0000 7f94fa000640 1 -- 172.21.15.44:0/1509695220 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f94f4016410 con 0x7f95040734f0 2024-09-13T08:46:18.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.169+0000 7f9508c00640 1 -- 172.21.15.44:0/1509695220 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f95040d0850 con 0x7f94e0073c40 2024-09-13T08:46:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.181+0000 7f94fa000640 1 -- 172.21.15.44:0/1509695220 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f95040d0850 con 0x7f94e0073c40 2024-09-13T08:46:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 -- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0073c40 msgr2=0x7f94e0076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0073c40 0x7f94e0076100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f950415a3e0 tx=0x7f94ec00a610 comp rx=0 tx=0).stop 2024-09-13T08:46:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 -- 172.21.15.44:0/1509695220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95040734f0 msgr2=0x7f9504150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95040734f0 0x7f9504150610 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009790 tx=0x7f94f4005ca0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 -- 172.21.15.44:0/1509695220 shutdown_connections 2024-09-13T08:46:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0073c40 0x7f94e0076100 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95040cf440 0x7f950415dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f95040ce890 0x7f950415d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 --2- 172.21.15.44:0/1509695220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95040734f0 0x7f9504150610 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 -- 172.21.15.44:0/1509695220 >> 172.21.15.44:0/1509695220 conn(0x7f95040cfbc0 msgr2=0x7f95040b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 -- 172.21.15.44:0/1509695220 shutdown_connections 2024-09-13T08:46:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.185+0000 7f9508c00640 1 -- 172.21.15.44:0/1509695220 wait complete. 2024-09-13T08:46:18.206 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:46:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 -- 172.21.15.44:0/221542271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2fc003c20 msgr2=0x7fd2fc004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 --2- 172.21.15.44:0/221542271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2fc003c20 0x7fd2fc004020 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd308001860 tx=0x7fd30801c8e0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31a800640 1 -- 172.21.15.44:0/221542271 reap_dead start 2024-09-13T08:46:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 -- 172.21.15.44:0/221542271 shutdown_connections 2024-09-13T08:46:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 -- 172.21.15.44:0/221542271 >> 172.21.15.44:0/221542271 conn(0x7fd3140d1dd0 msgr2=0x7fd3140d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 -- 172.21.15.44:0/221542271 shutdown_connections 2024-09-13T08:46:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 -- 172.21.15.44:0/221542271 wait complete. 2024-09-13T08:46:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 Processor -- start 2024-09-13T08:46:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 -- start start 2024-09-13T08:46:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd2fc003c20 0x7fd3140b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3140b75e0 0x7fd31406f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd314161c40 0x7fd3141620f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140b1220 con 0x7fd3140b75e0 2024-09-13T08:46:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140b0ee0 con 0x7fd314161c40 2024-09-13T08:46:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31c800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd3140b13a0 con 0x7fd2fc003c20 2024-09-13T08:46:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd319400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3140b75e0 0x7fd31406f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd319400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3140b75e0 0x7fd31406f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48732/0 (socket says 172.21.15.44:48732) 2024-09-13T08:46:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd319400640 1 -- 172.21.15.44:0/2711866096 learned_addr learned my addr 172.21.15.44:0/2711866096 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd31a800640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd314161c40 0x7fd3141620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd319e00640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd2fc003c20 0x7fd3140b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd319400640 1 -- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd2fc003c20 msgr2=0x7fd3140b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.357+0000 7fd319400640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd2fc003c20 0x7fd3140b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd319400640 1 -- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd314161c40 msgr2=0x7fd3141620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd319400640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd314161c40 0x7fd3141620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd319400640 1 -- 172.21.15.44:0/2711866096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3141fbd70 con 0x7fd3140b75e0 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd319400640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3140b75e0 0x7fd31406f630 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd304011a40 tx=0x7fd304011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd30e000640 1 -- 172.21.15.44:0/2711866096 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304004030 con 0x7fd3140b75e0 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd30e000640 1 -- 172.21.15.44:0/2711866096 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3040041d0 con 0x7fd3140b75e0 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd31c800640 1 -- 172.21.15.44:0/2711866096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3141fc050 con 0x7fd3140b75e0 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd30e000640 1 -- 172.21.15.44:0/2711866096 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304003600 con 0x7fd3140b75e0 2024-09-13T08:46:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd31c800640 1 -- 172.21.15.44:0/2711866096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3141fc610 con 0x7fd3140b75e0 2024-09-13T08:46:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd31c800640 1 -- 172.21.15.44:0/2711866096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3140affc0 con 0x7fd3140b75e0 2024-09-13T08:46:18.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd30e000640 1 -- 172.21.15.44:0/2711866096 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 31) ==== 98304+0+0 (secure 0 0 0) 0x7fd3040037a0 con 0x7fd3140b75e0 2024-09-13T08:46:18.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.361+0000 7fd30e000640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd2f8073c70 0x7fd2f8076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:18.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.365+0000 7fd30e000640 1 -- 172.21.15.44:0/2711866096 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fd304093d80 con 0x7fd3140b75e0 2024-09-13T08:46:18.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.365+0000 7fd319e00640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd2f8073c70 0x7fd2f8076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:18.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.365+0000 7fd319e00640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd2f8073c70 0x7fd2f8076130 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd3080a9000 tx=0x7fd308019720 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:18.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.365+0000 7fd30e000640 1 -- 172.21.15.44:0/2711866096 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd3040606f0 con 0x7fd3140b75e0 2024-09-13T08:46:18.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.569+0000 7fd31c800640 1 -- 172.21.15.44:0/2711866096 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd3140cc290 con 0x7fd2f8073c70 2024-09-13T08:46:18.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.573+0000 7fd30e000640 1 -- 172.21.15.44:0/2711866096 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fd3140cc290 con 0x7fd2f8073c70 2024-09-13T08:46:18.575 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/2aa71638-d6c5-4b2f-97e9-9c31d677599f 2024-09-13T08:46:18.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 -- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd2f8073c70 msgr2=0x7fd2f8076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:18.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd2f8073c70 0x7fd2f8076130 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd3080a9000 tx=0x7fd308019720 comp rx=0 tx=0).stop 2024-09-13T08:46:18.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 -- 172.21.15.44:0/2711866096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3140b75e0 msgr2=0x7fd31406f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:18.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3140b75e0 0x7fd31406f630 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd304011a40 tx=0x7fd304011f00 comp rx=0 tx=0).stop 2024-09-13T08:46:18.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 -- 172.21.15.44:0/2711866096 shutdown_connections 2024-09-13T08:46:18.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd2f8073c70 0x7fd2f8076130 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd314161c40 0x7fd3141620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3140b75e0 0x7fd31406f630 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 --2- 172.21.15.44:0/2711866096 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd2fc003c20 0x7fd3140b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:18.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 -- 172.21.15.44:0/2711866096 >> 172.21.15.44:0/2711866096 conn(0x7fd3140d1dd0 msgr2=0x7fd3140ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:18.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 -- 172.21.15.44:0/2711866096 shutdown_connections 2024-09-13T08:46:18.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:18.577+0000 7fd31c800640 1 -- 172.21.15.44:0/2711866096 wait complete. 2024-09-13T08:46:18.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:46:18.596 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/2aa71638-d6c5-4b2f-97e9-9c31d677599f 2024-09-13T08:46:18.596 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:46:18.605 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/2aa71638-d6c5-4b2f-97e9-9c31d677599f 2024-09-13T08:46:18.605 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3055190016" 2024-09-13T08:46:18.605 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:18.606 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-13T08:46:20.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:46:20.608 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/2aa71638-d6c5-4b2f-97e9-9c31d677599f 2024-09-13T08:46:20.608 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:46:20.618 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/2aa71638-d6c5-4b2f-97e9-9c31d677599f 2024-09-13T08:46:20.618 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3221225472" 2024-09-13T08:46:20.619 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:20.620 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-13T08:46:20.620 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone54740 2024-09-13T08:46:20.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.829+0000 7f7b8ac00640 1 -- 172.21.15.44:0/3144544055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b840d2590 msgr2=0x7f7b840ce980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:20.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.829+0000 7f7b8ac00640 1 --2- 172.21.15.44:0/3144544055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b840d2590 0x7f7b840ce980 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7b8408c240 tx=0x7f7b7c01cb50 comp rx=0 tx=0).stop 2024-09-13T08:46:20.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.829+0000 7f7b8ac00640 1 -- 172.21.15.44:0/3144544055 shutdown_connections 2024-09-13T08:46:20.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.829+0000 7f7b8ac00640 1 --2- 172.21.15.44:0/3144544055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b840d2590 0x7f7b840ce980 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:20.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.829+0000 7f7b8ac00640 1 --2- 172.21.15.44:0/3144544055 >> v2:172.21.15.129:3300/0 conn(0x7f7b8415df50 0x7f7b840d4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:20.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.829+0000 7f7b8ac00640 1 -- 172.21.15.44:0/3144544055 >> 172.21.15.44:0/3144544055 conn(0x7f7b840d3e70 msgr2=0x7f7b840d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:20.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.829+0000 7f7b8ac00640 1 -- 172.21.15.44:0/3144544055 shutdown_connections 2024-09-13T08:46:20.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.829+0000 7f7b8ac00640 1 -- 172.21.15.44:0/3144544055 wait complete. 2024-09-13T08:46:20.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.829+0000 7f7b8ac00640 1 Processor -- start 2024-09-13T08:46:20.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8ac00640 1 -- start start 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b840d2590 0x7f7b841599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8ac00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b8415df50 0x7f7b8415b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8ac00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b8415ef20 0x7f7b84152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8416ae40 con 0x7f7b840d2590 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8ac00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b840cfd00 con 0x7f7b8415df50 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8ac00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8416afc0 con 0x7f7b8415ef20 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b89800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b840d2590 0x7f7b841599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b89800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b840d2590 0x7f7b841599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48748/0 (socket says 172.21.15.44:48748) 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b89800640 1 -- 172.21.15.44:0/4197796362 learned_addr learned my addr 172.21.15.44:0/4197796362 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b88e00640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b8415df50 0x7f7b8415b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8a200640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b8415ef20 0x7f7b84152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b89800640 1 -- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b8415ef20 msgr2=0x7f7b84152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b89800640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b8415ef20 0x7f7b84152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b89800640 1 -- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b8415df50 msgr2=0x7f7b8415b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b89800640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b8415df50 0x7f7b8415b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b89800640 1 -- 172.21.15.44:0/4197796362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b84159fa0 con 0x7f7b840d2590 2024-09-13T08:46:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b89800640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b840d2590 0x7f7b841599c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800d950 tx=0x7f7b7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b81600640 1 -- 172.21.15.44:0/4197796362 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78017070 con 0x7f7b840d2590 2024-09-13T08:46:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b81600640 1 -- 172.21.15.44:0/4197796362 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b78009d20 con 0x7f7b840d2590 2024-09-13T08:46:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8ac00640 1 -- 172.21.15.44:0/4197796362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8415a280 con 0x7f7b840d2590 2024-09-13T08:46:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b81600640 1 -- 172.21.15.44:0/4197796362 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780106c0 con 0x7f7b840d2590 2024-09-13T08:46:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8ac00640 1 -- 172.21.15.44:0/4197796362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b8415a840 con 0x7f7b840d2590 2024-09-13T08:46:20.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.833+0000 7f7b8ac00640 1 -- 172.21.15.44:0/4197796362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b840ce980 con 0x7f7b840d2590 2024-09-13T08:46:20.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.837+0000 7f7b81600640 1 -- 172.21.15.44:0/4197796362 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 31) ==== 98304+0+0 (secure 0 0 0) 0x7f7b78010860 con 0x7f7b840d2590 2024-09-13T08:46:20.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.837+0000 7f7b81600640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b64073c70 0x7f7b64076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:20.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.837+0000 7f7b81600640 1 -- 172.21.15.44:0/4197796362 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f7b78092ec0 con 0x7f7b840d2590 2024-09-13T08:46:20.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.837+0000 7f7b88e00640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b64073c70 0x7f7b64076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:20.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.837+0000 7f7b88e00640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b64073c70 0x7f7b64076130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7b840b9410 tx=0x7f7b70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:20.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:20.841+0000 7f7b81600640 1 -- 172.21.15.44:0/4197796362 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7b7809b020 con 0x7f7b840d2590 2024-09-13T08:46:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.053+0000 7f7b8ac00640 1 -- 172.21.15.44:0/4197796362 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone54740", "target": ["mon-mgr", ""]}) -- 0x7f7b84161740 con 0x7f7b64073c70 2024-09-13T08:46:21.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.121+0000 7f7b81600640 1 -- 172.21.15.44:0/4197796362 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7b84161740 con 0x7f7b64073c70 2024-09-13T08:46:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 -- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b64073c70 msgr2=0x7f7b64076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b64073c70 0x7f7b64076130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7b840b9410 tx=0x7f7b70009040 comp rx=0 tx=0).stop 2024-09-13T08:46:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 -- 172.21.15.44:0/4197796362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b840d2590 msgr2=0x7f7b841599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b840d2590 0x7f7b841599c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800d950 tx=0x7f7b7800de10 comp rx=0 tx=0).stop 2024-09-13T08:46:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 -- 172.21.15.44:0/4197796362 shutdown_connections 2024-09-13T08:46:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b64073c70 0x7f7b64076130 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b8415ef20 0x7f7b84152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b8415df50 0x7f7b8415b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 --2- 172.21.15.44:0/4197796362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b840d2590 0x7f7b841599c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 -- 172.21.15.44:0/4197796362 >> 172.21.15.44:0/4197796362 conn(0x7f7b840d3e70 msgr2=0x7f7b8415f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 -- 172.21.15.44:0/4197796362 shutdown_connections 2024-09-13T08:46:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.125+0000 7f7b8ac00640 1 -- 172.21.15.44:0/4197796362 wait complete. 2024-09-13T08:46:21.128 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:21.124+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:21.128 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:21.124+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:21.128 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:21.124+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:21.128 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:21.124+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:21.128 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:21.124+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:21.145 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:21.144+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:21.146 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:21.144+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:21.146 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:21.144+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:21.146 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:21.144+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:21.146 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:21.144+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:21.148 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone3811 2024-09-13T08:46:21.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.301+0000 7fe8b0200640 1 -- 172.21.15.44:0/2212283612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8a81605d0 msgr2=0x7fe8a80d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.301+0000 7fe8b0200640 1 --2- 172.21.15.44:0/2212283612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8a81605d0 0x7fe8a80d0690 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009840 tx=0x7fe89c01c820 comp rx=0 tx=0).stop 2024-09-13T08:46:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.301+0000 7fe8b0200640 1 -- 172.21.15.44:0/2212283612 shutdown_connections 2024-09-13T08:46:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.301+0000 7fe8b0200640 1 --2- 172.21.15.44:0/2212283612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8a81605d0 0x7fe8a80d0690 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.301+0000 7fe8b0200640 1 -- 172.21.15.44:0/2212283612 >> 172.21.15.44:0/2212283612 conn(0x7fe8a80d1dd0 msgr2=0x7fe8a80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.301+0000 7fe8b0200640 1 -- 172.21.15.44:0/2212283612 shutdown_connections 2024-09-13T08:46:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.301+0000 7fe8b0200640 1 -- 172.21.15.44:0/2212283612 wait complete. 2024-09-13T08:46:21.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8b0200640 1 Processor -- start 2024-09-13T08:46:21.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8b0200640 1 -- start start 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8b0200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a80afb10 0x7fe8a8164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8b0200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8a80b1650 0x7fe8a81522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8b0200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8a81605d0 0x7fe8a814e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8b0200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a80b1220 con 0x7fe8a80afb10 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8b0200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a80b0ee0 con 0x7fe8a80b1650 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8b0200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a80b13a0 con 0x7fe8a81605d0 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ad800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a80afb10 0x7fe8a8164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ad800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a80afb10 0x7fe8a8164ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48762/0 (socket says 172.21.15.44:48762) 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ad800640 1 -- 172.21.15.44:0/2454175720 learned_addr learned my addr 172.21.15.44:0/2454175720 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ace00640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8a80b1650 0x7fe8a81522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ae200640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8a81605d0 0x7fe8a814e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ad800640 1 -- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8a81605d0 msgr2=0x7fe8a814e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ad800640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8a81605d0 0x7fe8a814e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ad800640 1 -- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8a80b1650 msgr2=0x7fe8a81522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ad800640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8a80b1650 0x7fe8a81522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ad800640 1 -- 172.21.15.44:0/2454175720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a814eff0 con 0x7fe8a80afb10 2024-09-13T08:46:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ace00640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8a80b1650 0x7fe8a81522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:46:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8ad800640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a80afb10 0x7fe8a8164ed0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009790 tx=0x7fe89c0a6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8a5600640 1 -- 172.21.15.44:0/2454175720 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c004040 con 0x7fe8a80afb10 2024-09-13T08:46:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8a5600640 1 -- 172.21.15.44:0/2454175720 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe89c0042d0 con 0x7fe8a80afb10 2024-09-13T08:46:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8a5600640 1 -- 172.21.15.44:0/2454175720 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0044c0 con 0x7fe8a80afb10 2024-09-13T08:46:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8b0200640 1 -- 172.21.15.44:0/2454175720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a80ce550 con 0x7fe8a80afb10 2024-09-13T08:46:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.305+0000 7fe8b0200640 1 -- 172.21.15.44:0/2454175720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a8165410 con 0x7fe8a80afb10 2024-09-13T08:46:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.309+0000 7fe8b0200640 1 -- 172.21.15.44:0/2454175720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a80affc0 con 0x7fe8a80afb10 2024-09-13T08:46:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.309+0000 7fe8a5600640 1 -- 172.21.15.44:0/2454175720 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 31) ==== 98304+0+0 (secure 0 0 0) 0x7fe89c0c1030 con 0x7fe8a80afb10 2024-09-13T08:46:21.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.309+0000 7fe8a5600640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe888073c40 0x7fe888076100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:21.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.309+0000 7fe8a5600640 1 -- 172.21.15.44:0/2454175720 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fe89c12d330 con 0x7fe8a80afb10 2024-09-13T08:46:21.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.309+0000 7fe8ace00640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe888073c40 0x7fe888076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:21.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.309+0000 7fe8ace00640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe888073c40 0x7fe888076100 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8166ff0 tx=0x7fe894009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:21.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.313+0000 7fe8a5600640 1 -- 172.21.15.44:0/2454175720 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe89c0c8020 con 0x7fe8a80afb10 2024-09-13T08:46:21.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:21.521+0000 7fe8b0200640 1 -- 172.21.15.44:0/2454175720 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone3811", "target": ["mon-mgr", ""]}) -- 0x7fe8a814eda0 con 0x7fe888073c40 2024-09-13T08:46:22.130 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:22.128+0000 7f8ab3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:22.130 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:22.128+0000 7f8ab3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:22.130 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:22.128+0000 7f8ab3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:22.130 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:22.128+0000 7f8ab3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:22.130 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:22.128+0000 7f8ab3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:22.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8a5600640 1 -- 172.21.15.44:0/2454175720 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe8a814eda0 con 0x7fe888073c40 2024-09-13T08:46:22.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 -- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe888073c40 msgr2=0x7fe888076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:22.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe888073c40 0x7fe888076100 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8166ff0 tx=0x7fe894009040 comp rx=0 tx=0).stop 2024-09-13T08:46:22.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 -- 172.21.15.44:0/2454175720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a80afb10 msgr2=0x7fe8a8164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:22.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a80afb10 0x7fe8a8164ed0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009790 tx=0x7fe89c0a6600 comp rx=0 tx=0).stop 2024-09-13T08:46:22.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 -- 172.21.15.44:0/2454175720 shutdown_connections 2024-09-13T08:46:22.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe888073c40 0x7fe888076100 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:22.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8a81605d0 0x7fe8a814e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:22.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8a80b1650 0x7fe8a81522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:22.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 --2- 172.21.15.44:0/2454175720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a80afb10 0x7fe8a8164ed0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:22.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 -- 172.21.15.44:0/2454175720 >> 172.21.15.44:0/2454175720 conn(0x7fe8a80d1dd0 msgr2=0x7fe8a80b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:22.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.237+0000 7fe8b0200640 1 -- 172.21.15.44:0/2454175720 shutdown_connections 2024-09-13T08:46:22.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.241+0000 7fe8b0200640 1 -- 172.21.15.44:0/2454175720 wait complete. 2024-09-13T08:46:22.269 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone75132 2024-09-13T08:46:22.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 -- 172.21.15.44:0/4254948796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397806d720 msgr2=0x7f3978073820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:22.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 --2- 172.21.15.44:0/4254948796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397806d720 0x7f3978073820 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f39780cbe70 tx=0x7f396801c800 comp rx=0 tx=0).stop 2024-09-13T08:46:22.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 -- 172.21.15.44:0/4254948796 shutdown_connections 2024-09-13T08:46:22.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 --2- 172.21.15.44:0/4254948796 >> v2:172.21.15.129:3300/0 conn(0x7f397806cc20 0x7f39780cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:22.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 --2- 172.21.15.44:0/4254948796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397806d720 0x7f3978073820 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:22.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 -- 172.21.15.44:0/4254948796 >> 172.21.15.44:0/4254948796 conn(0x7f39780d0c90 msgr2=0x7f39780d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:22.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 -- 172.21.15.44:0/4254948796 shutdown_connections 2024-09-13T08:46:22.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 -- 172.21.15.44:0/4254948796 wait complete. 2024-09-13T08:46:22.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 Processor -- start 2024-09-13T08:46:22.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 -- start start 2024-09-13T08:46:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.521+0000 7f397d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397806cc20 0x7f39780c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f397d000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f397806d720 0x7f397815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f397d000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39780ccce0 0x7f397815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f397d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f397816b4f0 con 0x7f397806cc20 2024-09-13T08:46:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f397d000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f397816b370 con 0x7f397806d720 2024-09-13T08:46:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f397d000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f397816b670 con 0x7f39780ccce0 2024-09-13T08:46:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f3975e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397806cc20 0x7f39780c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f3975e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397806cc20 0x7f39780c2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52212/0 (socket says 172.21.15.44:52212) 2024-09-13T08:46:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f3975e00640 1 -- 172.21.15.44:0/3464514254 learned_addr learned my addr 172.21.15.44:0/3464514254 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f3975e00640 1 -- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39780ccce0 msgr2=0x7f397815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f3975e00640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39780ccce0 0x7f397815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f3975e00640 1 -- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f397806d720 msgr2=0x7f397815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f3975e00640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f397806d720 0x7f397815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f3975e00640 1 -- 172.21.15.44:0/3464514254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39780c7f00 con 0x7f397806cc20 2024-09-13T08:46:22.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f3975e00640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397806cc20 0x7f39780c2830 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3968009790 tx=0x7f39680a66d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:22.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f396e000640 1 -- 172.21.15.44:0/3464514254 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39680be020 con 0x7f397806cc20 2024-09-13T08:46:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f397d000640 1 -- 172.21.15.44:0/3464514254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39780c81e0 con 0x7f397806cc20 2024-09-13T08:46:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f396e000640 1 -- 172.21.15.44:0/3464514254 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39680043b0 con 0x7f397806cc20 2024-09-13T08:46:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f396e000640 1 -- 172.21.15.44:0/3464514254 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968003840 con 0x7f397806cc20 2024-09-13T08:46:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f397d000640 1 -- 172.21.15.44:0/3464514254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39780c2db0 con 0x7f397806cc20 2024-09-13T08:46:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.525+0000 7f397d000640 1 -- 172.21.15.44:0/3464514254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39780cc7a0 con 0x7f397806cc20 2024-09-13T08:46:22.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.529+0000 7f396e000640 1 -- 172.21.15.44:0/3464514254 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 31) ==== 98304+0+0 (secure 0 0 0) 0x7f39680c1020 con 0x7f397806cc20 2024-09-13T08:46:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.529+0000 7f396e000640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3958073c70 0x7f3958076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.529+0000 7f396e000640 1 -- 172.21.15.44:0/3464514254 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f396812dd10 con 0x7f397806cc20 2024-09-13T08:46:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.533+0000 7f396e000640 1 -- 172.21.15.44:0/3464514254 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f39680c8030 con 0x7f397806cc20 2024-09-13T08:46:22.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.537+0000 7f3975400640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3958073c70 0x7f3958076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:22.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.541+0000 7f3975400640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3958073c70 0x7f3958076130 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f39640058d0 tx=0x7f3964005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:22.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:22.889+0000 7f397d000640 1 -- 172.21.15.44:0/3464514254 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone75132", "target": ["mon-mgr", ""]}) -- 0x7f39780c4560 con 0x7f3958073c70 2024-09-13T08:46:23.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.265+0000 7f396e000640 1 -- 172.21.15.44:0/3464514254 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 32) ==== 98457+0+0 (secure 0 0 0) 0x7f39680f62d0 con 0x7f397806cc20 2024-09-13T08:46:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.541+0000 7f396e000640 1 -- 172.21.15.44:0/3464514254 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f39780c4560 con 0x7f3958073c70 2024-09-13T08:46:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 -- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3958073c70 msgr2=0x7f3958076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3958073c70 0x7f3958076130 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f39640058d0 tx=0x7f3964005860 comp rx=0 tx=0).stop 2024-09-13T08:46:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 -- 172.21.15.44:0/3464514254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397806cc20 msgr2=0x7f39780c2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397806cc20 0x7f39780c2830 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3968009790 tx=0x7f39680a66d0 comp rx=0 tx=0).stop 2024-09-13T08:46:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 -- 172.21.15.44:0/3464514254 shutdown_connections 2024-09-13T08:46:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3958073c70 0x7f3958076130 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39780ccce0 0x7f397815dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f397806d720 0x7f397815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 --2- 172.21.15.44:0/3464514254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397806cc20 0x7f39780c2830 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 -- 172.21.15.44:0/3464514254 >> 172.21.15.44:0/3464514254 conn(0x7f39780d0c90 msgr2=0x7f397806fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 -- 172.21.15.44:0/3464514254 shutdown_connections 2024-09-13T08:46:23.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.545+0000 7f397d000640 1 -- 172.21.15.44:0/3464514254 wait complete. 2024-09-13T08:46:23.573 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:23.568+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:23.573 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:23.568+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:23.573 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:23.568+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:23.573 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:23.568+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:23.574 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:23.568+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:23.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone53093 2024-09-13T08:46:23.681 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:23.676+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:23.682 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:23.676+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:23.682 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:23.676+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:23.682 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:23.676+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:23.682 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:23.676+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:23.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.889+0000 7f9012e00640 1 -- 172.21.15.44:0/4180432395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff0003a60 msgr2=0x7f8ff0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.889+0000 7f9012e00640 1 --2- 172.21.15.44:0/4180432395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff0003a60 0x7f8ff0003e60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc006d90 tx=0x7f8ffc01f170 comp rx=0 tx=0).stop 2024-09-13T08:46:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.889+0000 7f9012e00640 1 -- 172.21.15.44:0/4180432395 shutdown_connections 2024-09-13T08:46:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.889+0000 7f9012e00640 1 --2- 172.21.15.44:0/4180432395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff0003a60 0x7f8ff0003e60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.889+0000 7f9012e00640 1 -- 172.21.15.44:0/4180432395 >> 172.21.15.44:0/4180432395 conn(0x7f900c0cd740 msgr2=0x7f900c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.889+0000 7f9012e00640 1 -- 172.21.15.44:0/4180432395 shutdown_connections 2024-09-13T08:46:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.889+0000 7f9012e00640 1 -- 172.21.15.44:0/4180432395 wait complete. 2024-09-13T08:46:23.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9012e00640 1 Processor -- start 2024-09-13T08:46:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9012e00640 1 -- start start 2024-09-13T08:46:23.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9012e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0003a60 0x7f900c0b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:23.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9012e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f900c0b3d40 0x7f900c0b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:23.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9012e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f900c1546a0 0x7f900c0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9012e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f900c0743e0 con 0x7f900c1546a0 2024-09-13T08:46:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9012e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f900c074260 con 0x7f8ff0003a60 2024-09-13T08:46:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9012e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f900c074560 con 0x7f900c0b3d40 2024-09-13T08:46:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9010e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f900c1546a0 0x7f900c0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9010e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f900c1546a0 0x7f900c0c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52226/0 (socket says 172.21.15.44:52226) 2024-09-13T08:46:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.893+0000 7f9010e00640 1 -- 172.21.15.44:0/1447804410 learned_addr learned my addr 172.21.15.44:0/1447804410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.897+0000 7f9010e00640 1 -- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f900c0b3d40 msgr2=0x7f900c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.897+0000 7f9010e00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f900c0b3d40 0x7f900c0b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.897+0000 7f9010e00640 1 -- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0003a60 msgr2=0x7f900c0b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.897+0000 7f9010e00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0003a60 0x7f900c0b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.897+0000 7f9010e00640 1 -- 172.21.15.44:0/1447804410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c155120 con 0x7f900c1546a0 2024-09-13T08:46:23.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.897+0000 7f9010e00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f900c1546a0 0x7f900c0c8f20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f900c0cc4f0 tx=0x7f900400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:23.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.897+0000 7f9003e00640 1 -- 172.21.15.44:0/1447804410 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004017070 con 0x7f900c1546a0 2024-09-13T08:46:23.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.897+0000 7f9003e00640 1 -- 172.21.15.44:0/1447804410 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90040050d0 con 0x7f900c1546a0 2024-09-13T08:46:23.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.897+0000 7f9003e00640 1 -- 172.21.15.44:0/1447804410 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004005520 con 0x7f900c1546a0 2024-09-13T08:46:23.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.897+0000 7f9012e00640 1 -- 172.21.15.44:0/1447804410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c15c360 con 0x7f900c1546a0 2024-09-13T08:46:23.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.901+0000 7f9012e00640 1 -- 172.21.15.44:0/1447804410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c15c920 con 0x7f900c1546a0 2024-09-13T08:46:23.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.905+0000 7f9012e00640 1 -- 172.21.15.44:0/1447804410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c06cfa0 con 0x7f900c1546a0 2024-09-13T08:46:23.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.917+0000 7f9003e00640 1 -- 172.21.15.44:0/1447804410 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 32) ==== 98457+0+0 (secure 0 0 0) 0x7f9004012400 con 0x7f900c1546a0 2024-09-13T08:46:23.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.917+0000 7f9003e00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fec073e20 0x7f8fec0762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.917+0000 7f9003e00640 1 -- 172.21.15.44:0/1447804410 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f9004091cc0 con 0x7f900c1546a0 2024-09-13T08:46:23.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.921+0000 7f900be00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fec073e20 0x7f8fec0762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:23.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.921+0000 7f9003e00640 1 -- 172.21.15.44:0/1447804410 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f900c155120 con 0x7f900c1546a0 2024-09-13T08:46:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:23.929+0000 7f900be00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fec073e20 0x7f8fec0762e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c470 tx=0x7f8ff0005840 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:24.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.305+0000 7f9012e00640 1 -- 172.21.15.44:0/1447804410 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone53093", "target": ["mon-mgr", ""]}) -- 0x7f900c0c4590 con 0x7f8fec073e20 2024-09-13T08:46:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.953+0000 7f9003e00640 1 -- 172.21.15.44:0/1447804410 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f900c0c4590 con 0x7f8fec073e20 2024-09-13T08:46:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 -- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fec073e20 msgr2=0x7f8fec0762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fec073e20 0x7f8fec0762e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c470 tx=0x7f8ff0005840 comp rx=0 tx=0).stop 2024-09-13T08:46:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 -- 172.21.15.44:0/1447804410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f900c1546a0 msgr2=0x7f900c0c8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f900c1546a0 0x7f900c0c8f20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f900c0cc4f0 tx=0x7f900400c910 comp rx=0 tx=0).stop 2024-09-13T08:46:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 -- 172.21.15.44:0/1447804410 shutdown_connections 2024-09-13T08:46:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fec073e20 0x7f8fec0762e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f900c1546a0 0x7f900c0c8f20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f900c0b3d40 0x7f900c0b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 --2- 172.21.15.44:0/1447804410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0003a60 0x7f900c0b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 -- 172.21.15.44:0/1447804410 >> 172.21.15.44:0/1447804410 conn(0x7f900c0cd740 msgr2=0x7f900c0a1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 -- 172.21.15.44:0/1447804410 shutdown_connections 2024-09-13T08:46:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:24.957+0000 7f9012e00640 1 -- 172.21.15.44:0/1447804410 wait complete. 2024-09-13T08:46:24.995 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone57364 2024-09-13T08:46:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.241+0000 7fe722e00640 1 -- 172.21.15.44:0/1138860914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0cfd30 msgr2=0x7fe71c15fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.241+0000 7fe722e00640 1 --2- 172.21.15.44:0/1138860914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0cfd30 0x7fe71c15fc40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe710009900 tx=0x7fe71001c8b0 comp rx=0 tx=0).stop 2024-09-13T08:46:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.241+0000 7fe722e00640 1 -- 172.21.15.44:0/1138860914 shutdown_connections 2024-09-13T08:46:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.241+0000 7fe722e00640 1 --2- 172.21.15.44:0/1138860914 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe71c0b25e0 0x7fe71c0af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.241+0000 7fe722e00640 1 --2- 172.21.15.44:0/1138860914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0cfd30 0x7fe71c15fc40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.241+0000 7fe722e00640 1 -- 172.21.15.44:0/1138860914 >> 172.21.15.44:0/1138860914 conn(0x7fe71c0d1cf0 msgr2=0x7fe71c0d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.241+0000 7fe722e00640 1 -- 172.21.15.44:0/1138860914 shutdown_connections 2024-09-13T08:46:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.241+0000 7fe722e00640 1 -- 172.21.15.44:0/1138860914 wait complete. 2024-09-13T08:46:25.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe722e00640 1 Processor -- start 2024-09-13T08:46:25.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe722e00640 1 -- start start 2024-09-13T08:46:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe722e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c0b25e0 0x7fe71c157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe722e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0cfd30 0x7fe71c14c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe722e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe71c0d2810 0x7fe71c1fb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe722e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c168f10 con 0x7fe71c0cfd30 2024-09-13T08:46:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe722e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c0b07b0 con 0x7fe71c0b25e0 2024-09-13T08:46:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe722e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe71c169090 con 0x7fe71c0d2810 2024-09-13T08:46:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe721000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0cfd30 0x7fe71c14c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe721000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0cfd30 0x7fe71c14c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52244/0 (socket says 172.21.15.44:52244) 2024-09-13T08:46:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe721000640 1 -- 172.21.15.44:0/1125754296 learned_addr learned my addr 172.21.15.44:0/1125754296 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe721000640 1 -- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe71c0d2810 msgr2=0x7fe71c1fb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe721000640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe71c0d2810 0x7fe71c1fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe721000640 1 -- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c0b25e0 msgr2=0x7fe71c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe721000640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c0b25e0 0x7fe71c157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe721000640 1 -- 172.21.15.44:0/1125754296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c158260 con 0x7fe71c0cfd30 2024-09-13T08:46:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.245+0000 7fe721000640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0cfd30 0x7fe71c14c200 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c400 tx=0x7fe70800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:25.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.249+0000 7fe719600640 1 -- 172.21.15.44:0/1125754296 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708017070 con 0x7fe71c0cfd30 2024-09-13T08:46:25.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.249+0000 7fe719600640 1 -- 172.21.15.44:0/1125754296 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe708003600 con 0x7fe71c0cfd30 2024-09-13T08:46:25.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.249+0000 7fe719600640 1 -- 172.21.15.44:0/1125754296 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708003a50 con 0x7fe71c0cfd30 2024-09-13T08:46:25.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.249+0000 7fe722e00640 1 -- 172.21.15.44:0/1125754296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c158540 con 0x7fe71c0cfd30 2024-09-13T08:46:25.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.249+0000 7fe722e00640 1 -- 172.21.15.44:0/1125754296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c158b00 con 0x7fe71c0cfd30 2024-09-13T08:46:25.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.249+0000 7fe722e00640 1 -- 172.21.15.44:0/1125754296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c0af540 con 0x7fe71c0cfd30 2024-09-13T08:46:25.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.253+0000 7fe719600640 1 -- 172.21.15.44:0/1125754296 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 32) ==== 98457+0+0 (secure 0 0 0) 0x7fe70801a070 con 0x7fe71c0cfd30 2024-09-13T08:46:25.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.253+0000 7fe719600640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc073df0 0x7fe6fc0762b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:25.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.253+0000 7fe719600640 1 -- 172.21.15.44:0/1125754296 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fe708092310 con 0x7fe71c0cfd30 2024-09-13T08:46:25.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.257+0000 7fe719600640 1 -- 172.21.15.44:0/1125754296 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe708021020 con 0x7fe71c0cfd30 2024-09-13T08:46:25.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.265+0000 7fe721a00640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc073df0 0x7fe6fc0762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:25.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.265+0000 7fe721a00640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc073df0 0x7fe6fc0762b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe7100af000 tx=0x7fe710019650 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.333+0000 7fe719600640 1 -- 172.21.15.44:0/1125754296 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 33) ==== 98559+0+0 (secure 0 0 0) 0x7fe708092d10 con 0x7fe71c0cfd30 2024-09-13T08:46:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:25.577+0000 7fe722e00640 1 -- 172.21.15.44:0/1125754296 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone57364", "target": ["mon-mgr", ""]}) -- 0x7fe71c14c8b0 con 0x7fe6fc073df0 2024-09-13T08:46:26.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.321+0000 7fe719600640 1 -- 172.21.15.44:0/1125754296 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe71c14c8b0 con 0x7fe6fc073df0 2024-09-13T08:46:26.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 -- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc073df0 msgr2=0x7fe6fc0762b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc073df0 0x7fe6fc0762b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe7100af000 tx=0x7fe710019650 comp rx=0 tx=0).stop 2024-09-13T08:46:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 -- 172.21.15.44:0/1125754296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0cfd30 msgr2=0x7fe71c14c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0cfd30 0x7fe71c14c200 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c400 tx=0x7fe70800c8c0 comp rx=0 tx=0).stop 2024-09-13T08:46:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 -- 172.21.15.44:0/1125754296 shutdown_connections 2024-09-13T08:46:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc073df0 0x7fe6fc0762b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:26.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe71c0d2810 0x7fe71c1fb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:26.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0cfd30 0x7fe71c14c200 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:26.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 --2- 172.21.15.44:0/1125754296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c0b25e0 0x7fe71c157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:26.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 -- 172.21.15.44:0/1125754296 >> 172.21.15.44:0/1125754296 conn(0x7fe71c0d1cf0 msgr2=0x7fe71c155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:26.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 -- 172.21.15.44:0/1125754296 shutdown_connections 2024-09-13T08:46:26.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.329+0000 7fe722e00640 1 -- 172.21.15.44:0/1125754296 wait complete. 2024-09-13T08:46:26.374 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone21695 2024-09-13T08:46:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.649+0000 7fa47fe00640 1 -- 172.21.15.44:0/4009558303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4800cc8f0 msgr2=0x7fa48015c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.649+0000 7fa47fe00640 1 --2- 172.21.15.44:0/4009558303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4800cc8f0 0x7fa48015c0b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009840 tx=0x7fa46c01c820 comp rx=0 tx=0).stop 2024-09-13T08:46:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.649+0000 7fa47fe00640 1 -- 172.21.15.44:0/4009558303 shutdown_connections 2024-09-13T08:46:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.649+0000 7fa47fe00640 1 --2- 172.21.15.44:0/4009558303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4800cc8f0 0x7fa48015c0b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.649+0000 7fa47fe00640 1 -- 172.21.15.44:0/4009558303 >> 172.21.15.44:0/4009558303 conn(0x7fa4800b1c70 msgr2=0x7fa4800b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.649+0000 7fa47fe00640 1 -- 172.21.15.44:0/4009558303 shutdown_connections 2024-09-13T08:46:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.649+0000 7fa47fe00640 1 -- 172.21.15.44:0/4009558303 wait complete. 2024-09-13T08:46:26.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47fe00640 1 Processor -- start 2024-09-13T08:46:26.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47fe00640 1 -- start start 2024-09-13T08:46:26.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa480071210 0x7fa4801fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:26.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4800cc8f0 0x7fa48013f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:26.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa48015bcd0 0x7fa480158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:26.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa48016a130 con 0x7fa4800cc8f0 2024-09-13T08:46:26.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa480169fb0 con 0x7fa48015bcd0 2024-09-13T08:46:26.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa48016a2b0 con 0x7fa480071210 2024-09-13T08:46:26.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4800cc8f0 0x7fa48013f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:26.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4800cc8f0 0x7fa48013f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52268/0 (socket says 172.21.15.44:52268) 2024-09-13T08:46:26.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47e000640 1 -- 172.21.15.44:0/2954216754 learned_addr learned my addr 172.21.15.44:0/2954216754 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:26.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47e000640 1 -- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa480071210 msgr2=0x7fa4801fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:26.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47e000640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa480071210 0x7fa4801fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:26.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47e000640 1 -- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa48015bcd0 msgr2=0x7fa480158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:26.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47e000640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa48015bcd0 0x7fa480158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:26.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47e000640 1 -- 172.21.15.44:0/2954216754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa480151220 con 0x7fa4800cc8f0 2024-09-13T08:46:26.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.653+0000 7fa47e000640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4800cc8f0 0x7fa48013f200 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa47000c490 tx=0x7fa470001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:26.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.657+0000 7fa476a00640 1 -- 172.21.15.44:0/2954216754 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470017070 con 0x7fa4800cc8f0 2024-09-13T08:46:26.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.657+0000 7fa476a00640 1 -- 172.21.15.44:0/2954216754 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa470003600 con 0x7fa4800cc8f0 2024-09-13T08:46:26.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.657+0000 7fa476a00640 1 -- 172.21.15.44:0/2954216754 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470003a90 con 0x7fa4800cc8f0 2024-09-13T08:46:26.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.657+0000 7fa47fe00640 1 -- 172.21.15.44:0/2954216754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4801fc2d0 con 0x7fa4800cc8f0 2024-09-13T08:46:26.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.657+0000 7fa47fe00640 1 -- 172.21.15.44:0/2954216754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa480152e70 con 0x7fa4800cc8f0 2024-09-13T08:46:26.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.657+0000 7fa47fe00640 1 -- 172.21.15.44:0/2954216754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4580052a0 con 0x7fa4800cc8f0 2024-09-13T08:46:26.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.657+0000 7fa476a00640 1 -- 172.21.15.44:0/2954216754 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 33) ==== 98559+0+0 (secure 0 0 0) 0x7fa47001a070 con 0x7fa4800cc8f0 2024-09-13T08:46:26.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.661+0000 7fa476a00640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa460073f90 0x7fa460076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:26.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.661+0000 7fa476a00640 1 -- 172.21.15.44:0/2954216754 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fa470092060 con 0x7fa4800cc8f0 2024-09-13T08:46:26.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.665+0000 7fa476a00640 1 -- 172.21.15.44:0/2954216754 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa4801fc2d0 con 0x7fa4800cc8f0 2024-09-13T08:46:26.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.673+0000 7fa47ea00640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa460073f90 0x7fa460076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:26.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:26.681+0000 7fa47ea00640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa460073f90 0x7fa460076450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa478001ff0 tx=0x7fa478001600 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:27.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:27.081+0000 7fa47fe00640 1 -- 172.21.15.44:0/2954216754 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone21695", "target": ["mon-mgr", ""]}) -- 0x7fa458000c90 con 0x7fa460073f90 2024-09-13T08:46:28.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.848+0000 7fa476a00640 1 -- 172.21.15.44:0/2954216754 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa458000c90 con 0x7fa460073f90 2024-09-13T08:46:28.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 -- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa460073f90 msgr2=0x7fa460076450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:28.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa460073f90 0x7fa460076450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa478001ff0 tx=0x7fa478001600 comp rx=0 tx=0).stop 2024-09-13T08:46:28.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 -- 172.21.15.44:0/2954216754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4800cc8f0 msgr2=0x7fa48013f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:28.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4800cc8f0 0x7fa48013f200 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa47000c490 tx=0x7fa470001e80 comp rx=0 tx=0).stop 2024-09-13T08:46:28.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 -- 172.21.15.44:0/2954216754 shutdown_connections 2024-09-13T08:46:28.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa460073f90 0x7fa460076450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:28.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa48015bcd0 0x7fa480158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:28.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4800cc8f0 0x7fa48013f200 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:28.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 --2- 172.21.15.44:0/2954216754 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa480071210 0x7fa4801fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:28.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 -- 172.21.15.44:0/2954216754 >> 172.21.15.44:0/2954216754 conn(0x7fa4800b1c70 msgr2=0x7fa4800d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:28.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 -- 172.21.15.44:0/2954216754 shutdown_connections 2024-09-13T08:46:28.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:28.852+0000 7fa47fe00640 1 -- 172.21.15.44:0/2954216754 wait complete. 2024-09-13T08:46:28.917 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41056 2024-09-13T08:46:29.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.200+0000 7fd6c5a00640 1 -- 172.21.15.44:0/2933609053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6c00cfee0 msgr2=0x7fd6c00d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.200+0000 7fd6c5a00640 1 --2- 172.21.15.44:0/2933609053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6c00cfee0 0x7fd6c00d0330 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd6c006ac60 tx=0x7fd6a801cba0 comp rx=0 tx=0).stop 2024-09-13T08:46:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.200+0000 7fd6c5a00640 1 -- 172.21.15.44:0/2933609053 shutdown_connections 2024-09-13T08:46:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.200+0000 7fd6c5a00640 1 --2- 172.21.15.44:0/2933609053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6c00cfee0 0x7fd6c00d0330 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.200+0000 7fd6c5a00640 1 --2- 172.21.15.44:0/2933609053 >> v2:172.21.15.129:3300/0 conn(0x7fd6c015ac00 0x7fd6c00b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.200+0000 7fd6c5a00640 1 -- 172.21.15.44:0/2933609053 >> 172.21.15.44:0/2933609053 conn(0x7fd6c006ecd0 msgr2=0x7fd6c006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.200+0000 7fd6c5a00640 1 -- 172.21.15.44:0/2933609053 shutdown_connections 2024-09-13T08:46:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.200+0000 7fd6c5a00640 1 -- 172.21.15.44:0/2933609053 wait complete. 2024-09-13T08:46:29.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.204+0000 7fd6c5a00640 1 Processor -- start 2024-09-13T08:46:29.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.204+0000 7fd6c5a00640 1 -- start start 2024-09-13T08:46:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.204+0000 7fd6c5a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd6c00cfee0 0x7fd6c0148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.204+0000 7fd6c5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6c015ac00 0x7fd6c0149480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.204+0000 7fd6c5a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6c015afe0 0x7fd6c01499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.204+0000 7fd6c5a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c00702c0 con 0x7fd6c015ac00 2024-09-13T08:46:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.204+0000 7fd6c5a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0070140 con 0x7fd6c015afe0 2024-09-13T08:46:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.204+0000 7fd6c5a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c0070440 con 0x7fd6c00cfee0 2024-09-13T08:46:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6c015ac00 0x7fd6c0149480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6c015ac00 0x7fd6c0149480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52294/0 (socket says 172.21.15.44:52294) 2024-09-13T08:46:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b7e00640 1 -- 172.21.15.44:0/1377586565 learned_addr learned my addr 172.21.15.44:0/1377586565 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b7e00640 1 -- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd6c00cfee0 msgr2=0x7fd6c0148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b7e00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd6c00cfee0 0x7fd6c0148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b7e00640 1 -- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6c015afe0 msgr2=0x7fd6c01499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b7e00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6c015afe0 0x7fd6c01499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b7e00640 1 -- 172.21.15.44:0/1377586565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c014a7e0 con 0x7fd6c015ac00 2024-09-13T08:46:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b7e00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6c015ac00 0x7fd6c0149480 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8000f90 tx=0x7fd6a80051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b6a00640 1 -- 172.21.15.44:0/1377586565 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80bc020 con 0x7fd6c015ac00 2024-09-13T08:46:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b6a00640 1 -- 172.21.15.44:0/1377586565 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6a8005740 con 0x7fd6c015ac00 2024-09-13T08:46:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6b6a00640 1 -- 172.21.15.44:0/1377586565 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80b2040 con 0x7fd6c015ac00 2024-09-13T08:46:29.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6c5a00640 1 -- 172.21.15.44:0/1377586565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c0149240 con 0x7fd6c015ac00 2024-09-13T08:46:29.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.208+0000 7fd6c5a00640 1 -- 172.21.15.44:0/1377586565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c00b24a0 con 0x7fd6c015ac00 2024-09-13T08:46:29.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.212+0000 7fd6c5a00640 1 -- 172.21.15.44:0/1377586565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c00d0330 con 0x7fd6c015ac00 2024-09-13T08:46:29.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.212+0000 7fd6b6a00640 1 -- 172.21.15.44:0/1377586565 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 33) ==== 98559+0+0 (secure 0 0 0) 0x7fd6a80c1440 con 0x7fd6c015ac00 2024-09-13T08:46:29.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.216+0000 7fd6b6a00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd69c073f40 0x7fd69c076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:29.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.216+0000 7fd6b6a00640 1 -- 172.21.15.44:0/1377586565 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fd6a8002020 con 0x7fd6c015ac00 2024-09-13T08:46:29.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.224+0000 7fd6b6a00640 1 -- 172.21.15.44:0/1377586565 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd6c014a7e0 con 0x7fd6c015ac00 2024-09-13T08:46:29.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.228+0000 7fd6bea00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd69c073f40 0x7fd69c076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.564+0000 7fd6bea00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd69c073f40 0x7fd69c076400 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6c014d830 tx=0x7fd6b0006c50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:29.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:29.600+0000 7fd6c5a00640 1 -- 172.21.15.44:0/1377586565 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41056", "target": ["mon-mgr", ""]}) -- 0x7fd6c00af920 con 0x7fd69c073f40 2024-09-13T08:46:30.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.928+0000 7fd6b6a00640 1 -- 172.21.15.44:0/1377586565 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd6c00af920 con 0x7fd69c073f40 2024-09-13T08:46:30.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.928+0000 7fd6c5a00640 1 -- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd69c073f40 msgr2=0x7fd69c076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:30.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.928+0000 7fd6c5a00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd69c073f40 0x7fd69c076400 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6c014d830 tx=0x7fd6b0006c50 comp rx=0 tx=0).stop 2024-09-13T08:46:30.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.928+0000 7fd6c5a00640 1 -- 172.21.15.44:0/1377586565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6c015ac00 msgr2=0x7fd6c0149480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:30.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.928+0000 7fd6c5a00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6c015ac00 0x7fd6c0149480 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8000f90 tx=0x7fd6a80051a0 comp rx=0 tx=0).stop 2024-09-13T08:46:30.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.932+0000 7fd6c5a00640 1 -- 172.21.15.44:0/1377586565 shutdown_connections 2024-09-13T08:46:30.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.932+0000 7fd6c5a00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd69c073f40 0x7fd69c076400 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:30.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.932+0000 7fd6c5a00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6c015afe0 0x7fd6c01499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:30.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.932+0000 7fd6c5a00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6c015ac00 0x7fd6c0149480 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:30.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.932+0000 7fd6c5a00640 1 --2- 172.21.15.44:0/1377586565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd6c00cfee0 0x7fd6c0148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:30.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.932+0000 7fd6c5a00640 1 -- 172.21.15.44:0/1377586565 >> 172.21.15.44:0/1377586565 conn(0x7fd6c006ecd0 msgr2=0x7fd6c00ce470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:30.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.932+0000 7fd6c5a00640 1 -- 172.21.15.44:0/1377586565 shutdown_connections 2024-09-13T08:46:30.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:30.932+0000 7fd6c5a00640 1 -- 172.21.15.44:0/1377586565 wait complete. 2024-09-13T08:46:30.971 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:46:31.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.252+0000 7fe38e000640 1 -- 172.21.15.44:0/2824499282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe388073860 msgr2=0x7fe388073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:31.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.252+0000 7fe38e000640 1 --2- 172.21.15.44:0/2824499282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe388073860 0x7fe388073ce0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe380009bc0 tx=0x7fe38001cb10 comp rx=0 tx=0).stop 2024-09-13T08:46:31.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.256+0000 7fe38e000640 1 -- 172.21.15.44:0/2824499282 shutdown_connections 2024-09-13T08:46:31.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.256+0000 7fe38e000640 1 --2- 172.21.15.44:0/2824499282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe388073860 0x7fe388073ce0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:31.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.256+0000 7fe38e000640 1 --2- 172.21.15.44:0/2824499282 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3880d01a0 0x7fe38815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:31.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.256+0000 7fe38e000640 1 -- 172.21.15.44:0/2824499282 >> 172.21.15.44:0/2824499282 conn(0x7fe3880d0cd0 msgr2=0x7fe3880d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:31.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.256+0000 7fe38e000640 1 -- 172.21.15.44:0/2824499282 shutdown_connections 2024-09-13T08:46:31.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.256+0000 7fe38e000640 1 -- 172.21.15.44:0/2824499282 wait complete. 2024-09-13T08:46:31.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.256+0000 7fe38e000640 1 Processor -- start 2024-09-13T08:46:31.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.256+0000 7fe38e000640 1 -- start start 2024-09-13T08:46:31.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe38e000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe388073860 0x7fe3880c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:31.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe38e000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3880d01a0 0x7fe38813ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:31.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe38e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38815ab80 0x7fe388140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:31.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe38e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe388167a40 con 0x7fe38815ab80 2024-09-13T08:46:31.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe38e000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe3881678c0 con 0x7fe388073860 2024-09-13T08:46:31.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe38e000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe388167bc0 con 0x7fe3880d01a0 2024-09-13T08:46:31.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe387e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38815ab80 0x7fe388140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:31.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe387e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38815ab80 0x7fe388140450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52314/0 (socket says 172.21.15.44:52314) 2024-09-13T08:46:31.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe387e00640 1 -- 172.21.15.44:0/3614979249 learned_addr learned my addr 172.21.15.44:0/3614979249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:31.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe387e00640 1 -- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3880d01a0 msgr2=0x7fe38813ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:31.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe387e00640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3880d01a0 0x7fe38813ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:31.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe387e00640 1 -- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe388073860 msgr2=0x7fe3880c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:31.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe387e00640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe388073860 0x7fe3880c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:31.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe387e00640 1 -- 172.21.15.44:0/3614979249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3880c5830 con 0x7fe38815ab80 2024-09-13T08:46:31.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.260+0000 7fe387e00640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38815ab80 0x7fe388140450 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe38001a740 tx=0x7fe3800a5a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:31.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.264+0000 7fe37f400640 1 -- 172.21.15.44:0/3614979249 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3800bb070 con 0x7fe38815ab80 2024-09-13T08:46:31.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.264+0000 7fe37f400640 1 -- 172.21.15.44:0/3614979249 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe380004e00 con 0x7fe38815ab80 2024-09-13T08:46:31.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.264+0000 7fe37f400640 1 -- 172.21.15.44:0/3614979249 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3800b2040 con 0x7fe38815ab80 2024-09-13T08:46:31.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.264+0000 7fe38e000640 1 -- 172.21.15.44:0/3614979249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3880c5ad0 con 0x7fe38815ab80 2024-09-13T08:46:31.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.264+0000 7fe38e000640 1 -- 172.21.15.44:0/3614979249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3880c6090 con 0x7fe38815ab80 2024-09-13T08:46:31.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.264+0000 7fe38e000640 1 -- 172.21.15.44:0/3614979249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe388073ce0 con 0x7fe38815ab80 2024-09-13T08:46:31.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.264+0000 7fe37f400640 1 -- 172.21.15.44:0/3614979249 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 33) ==== 98559+0+0 (secure 0 0 0) 0x7fe3800be070 con 0x7fe38815ab80 2024-09-13T08:46:31.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.268+0000 7fe37f400640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe368073f60 0x7fe368076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:31.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.268+0000 7fe37f400640 1 -- 172.21.15.44:0/3614979249 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fe38012cf20 con 0x7fe38815ab80 2024-09-13T08:46:31.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.272+0000 7fe37f400640 1 -- 172.21.15.44:0/3614979249 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe3800c5020 con 0x7fe38815ab80 2024-09-13T08:46:31.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.300+0000 7fe387400640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe368073f60 0x7fe368076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:31.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.324+0000 7fe387400640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe368073f60 0x7fe368076420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe374001530 tx=0x7fe374007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:31.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.720+0000 7fe38e000640 1 -- 172.21.15.44:0/3614979249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe388075ae0 con 0x7fe38815ab80 2024-09-13T08:46:31.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.720+0000 7fe37f400640 1 -- 172.21.15.44:0/3614979249 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2871 (secure 0 0 0) 0x7fe380016790 con 0x7fe38815ab80 2024-09-13T08:46:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:31.722 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:46:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:46:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:46:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:46:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:46:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:46:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 562, 2024-09-13T08:46:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 77, 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:46:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 1938, 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 7978820508, 2024-09-13T08:46:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 15391526912, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 757702586368, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_objects_per_sec": 1, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_bytes_per_sec": 8030444, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_keys_per_sec": 0, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects_recovered": 23, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "num_bytes_recovered": 96468992, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "num_keys_recovered": 0, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 410252867, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 465114816, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 99, 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 125 2024-09-13T08:46:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 47, 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:45:46:720592+0000", 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 5, 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 5, 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:46:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 6333 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:46:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:46:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:46:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:46:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:46:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 14, 2024-09-13T08:46:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:46:30.664832+0000", 2024-09-13T08:46:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: "services": { 2024-09-13T08:46:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "daemons": { 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "summary": "", 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "a": { 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "start_epoch": 0, 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "start_stamp": "0.000000", 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 0, 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": {}, 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "task_status": {} 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": { 2024-09-13T08:46:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-ongoing-clones": { 2024-09-13T08:46:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "3 ongoing clones - average progress is 16.58% (3s)\n [====........................] (remaining: 15s)", 2024-09-13T08:46:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.16580000519752502, 2024-09-13T08:46:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:46:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:31.731 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:46:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 -- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe368073f60 msgr2=0x7fe368076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe368073f60 0x7fe368076420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe374001530 tx=0x7fe374007a40 comp rx=0 tx=0).stop 2024-09-13T08:46:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 -- 172.21.15.44:0/3614979249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38815ab80 msgr2=0x7fe388140450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38815ab80 0x7fe388140450 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe38001a740 tx=0x7fe3800a5a10 comp rx=0 tx=0).stop 2024-09-13T08:46:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 -- 172.21.15.44:0/3614979249 shutdown_connections 2024-09-13T08:46:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe368073f60 0x7fe368076420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:31.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38815ab80 0x7fe388140450 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:31.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3880d01a0 0x7fe38813ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:31.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 --2- 172.21.15.44:0/3614979249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe388073860 0x7fe3880c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:31.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 -- 172.21.15.44:0/3614979249 >> 172.21.15.44:0/3614979249 conn(0x7fe3880d0cd0 msgr2=0x7fe388154a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:31.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 -- 172.21.15.44:0/3614979249 shutdown_connections 2024-09-13T08:46:31.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:31.724+0000 7fe38e000640 1 -- 172.21.15.44:0/3614979249 wait complete. 2024-09-13T08:46:32.132 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:46:32.138 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:46:32.762 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:46:33.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.060+0000 7f4660c00640 1 -- 172.21.15.44:0/4051208909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f465c0b2db0 msgr2=0x7f465c15ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:33.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.060+0000 7f4660c00640 1 --2- 172.21.15.44:0/4051208909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f465c0b2db0 0x7f465c15ba10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4648009840 tx=0x7f464801c680 comp rx=0 tx=0).stop 2024-09-13T08:46:33.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.060+0000 7f4660c00640 1 -- 172.21.15.44:0/4051208909 shutdown_connections 2024-09-13T08:46:33.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.060+0000 7f4660c00640 1 --2- 172.21.15.44:0/4051208909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f465c0b2db0 0x7f465c15ba10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.060+0000 7f4660c00640 1 -- 172.21.15.44:0/4051208909 >> 172.21.15.44:0/4051208909 conn(0x7f465c06cdf0 msgr2=0x7f465c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:33.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.060+0000 7f4660c00640 1 -- 172.21.15.44:0/4051208909 shutdown_connections 2024-09-13T08:46:33.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.060+0000 7f4660c00640 1 -- 172.21.15.44:0/4051208909 wait complete. 2024-09-13T08:46:33.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.064+0000 7f4660c00640 1 Processor -- start 2024-09-13T08:46:33.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.064+0000 7f4660c00640 1 -- start start 2024-09-13T08:46:33.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.064+0000 7f4660c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f465c06fdc0 0x7f465c0b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:33.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.064+0000 7f4660c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f465c072f00 0x7f465c201d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:33.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.064+0000 7f4660c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f465c0b2db0 0x7f465c0cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:33.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.064+0000 7f4660c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f465c169010 con 0x7f465c06fdc0 2024-09-13T08:46:33.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.064+0000 7f4660c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f465c168e90 con 0x7f465c0b2db0 2024-09-13T08:46:33.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.064+0000 7f4660c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f465c169190 con 0x7f465c072f00 2024-09-13T08:46:33.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4659e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f465c06fdc0 0x7f465c0b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:33.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4659e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f465c06fdc0 0x7f465c0b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34922/0 (socket says 172.21.15.44:34922) 2024-09-13T08:46:33.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4659e00640 1 -- 172.21.15.44:0/1926443551 learned_addr learned my addr 172.21.15.44:0/1926443551 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:33.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4659e00640 1 -- 172.21.15.44:0/1926443551 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f465c072f00 msgr2=0x7f465c201d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:33.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4659e00640 1 --2- 172.21.15.44:0/1926443551 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f465c072f00 0x7f465c201d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4659e00640 1 -- 172.21.15.44:0/1926443551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f465c0b2db0 msgr2=0x7f465c0cec30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:46:33.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4659e00640 1 --2- 172.21.15.44:0/1926443551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f465c0b2db0 0x7f465c0cec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4659e00640 1 -- 172.21.15.44:0/1926443551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c0cf560 con 0x7f465c06fdc0 2024-09-13T08:46:33.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4659e00640 1 --2- 172.21.15.44:0/1926443551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f465c06fdc0 0x7f465c0b0400 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f464c00c7e0 tx=0x7f464c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:33.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4652000640 1 -- 172.21.15.44:0/1926443551 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c017070 con 0x7f465c06fdc0 2024-09-13T08:46:33.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4652000640 1 -- 172.21.15.44:0/1926443551 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f464c005360 con 0x7f465c06fdc0 2024-09-13T08:46:33.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4652000640 1 -- 172.21.15.44:0/1926443551 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c00a8a0 con 0x7f465c06fdc0 2024-09-13T08:46:33.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.068+0000 7f4660c00640 1 -- 172.21.15.44:0/1926443551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465c0cf840 con 0x7f465c06fdc0 2024-09-13T08:46:33.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.072+0000 7f4660c00640 1 -- 172.21.15.44:0/1926443551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465c0afee0 con 0x7f465c06fdc0 2024-09-13T08:46:33.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.072+0000 7f4660c00640 1 -- 172.21.15.44:0/1926443551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465c15ba10 con 0x7f465c06fdc0 2024-09-13T08:46:33.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.076+0000 7f4652000640 1 -- 172.21.15.44:0/1926443551 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 33) ==== 98559+0+0 (secure 0 0 0) 0x7f464c01a070 con 0x7f465c06fdc0 2024-09-13T08:46:33.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.076+0000 7f4652000640 1 --2- 172.21.15.44:0/1926443551 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f463c073f40 0x7f463c076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:33.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.076+0000 7f4652000640 1 -- 172.21.15.44:0/1926443551 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f464c0921a0 con 0x7f465c06fdc0 2024-09-13T08:46:33.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.084+0000 7f4652000640 1 -- 172.21.15.44:0/1926443551 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f464c05eb80 con 0x7f465c06fdc0 2024-09-13T08:46:33.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.468+0000 7f4660c00640 1 -- 172.21.15.44:0/1926443551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f465c0b4170 con 0x7f465c06fdc0 2024-09-13T08:46:33.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.468+0000 7f4652000640 1 -- 172.21.15.44:0/1926443551 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2876 (secure 0 0 0) 0x7f464c05e480 con 0x7f465c06fdc0 2024-09-13T08:46:33.472 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:46:33.473 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 564, 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:33.474 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 77, 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:46:33.475 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 1938, 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 7978820508, 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 15391526912, 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 757702586368, 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 410175919, 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 395908888, 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 99, 2024-09-13T08:46:33.476 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 108 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 47, 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:45:46:720592+0000", 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 5, 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 5, 2024-09-13T08:46:33.477 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 6333 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:46:33.478 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:46:33.479 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:46:33.479 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:46:33.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:33.479 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:46:33.479 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 14, 2024-09-13T08:46:33.479 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:46:30.664832+0000", 2024-09-13T08:46:33.479 INFO:teuthology.orchestra.run.smithi044.stdout: "services": { 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: "daemons": { 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: "summary": "", 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: "a": { 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: "start_epoch": 0, 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: "start_stamp": "0.000000", 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 0, 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": {}, 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: "task_status": {} 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:33.480 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": { 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-ongoing-clones": { 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "4 ongoing clones - average progress is 30.989% (8s)\n [========....................] (remaining: 19s)", 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.30989000201225281, 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-total-clones": { 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "Total 5 clones - average progress is 38.333% (0s)\n [==========..................] ", 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.38332998752593994, 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:33.482 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:46:33.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 -- 172.21.15.44:0/1926443551 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f463c073f40 msgr2=0x7f463c076400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 --2- 172.21.15.44:0/1926443551 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f463c073f40 0x7f463c076400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 -- 172.21.15.44:0/1926443551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f465c06fdc0 msgr2=0x7f465c0b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 --2- 172.21.15.44:0/1926443551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f465c06fdc0 0x7f465c0b0400 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f464c00c7e0 tx=0x7f464c00cca0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 -- 172.21.15.44:0/1926443551 shutdown_connections 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 --2- 172.21.15.44:0/1926443551 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f463c073f40 0x7f463c076400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 --2- 172.21.15.44:0/1926443551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f465c0b2db0 0x7f465c0cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 --2- 172.21.15.44:0/1926443551 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f465c072f00 0x7f465c201d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 --2- 172.21.15.44:0/1926443551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f465c06fdc0 0x7f465c0b0400 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 -- 172.21.15.44:0/1926443551 >> 172.21.15.44:0/1926443551 conn(0x7f465c06cdf0 msgr2=0x7f465c0cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 -- 172.21.15.44:0/1926443551 shutdown_connections 2024-09-13T08:46:33.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.472+0000 7f4660c00640 1 -- 172.21.15.44:0/1926443551 wait complete. 2024-09-13T08:46:33.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone54740 2024-09-13T08:46:33.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.800+0000 7fbe79800640 1 -- 172.21.15.44:0/2261228654 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe7406d7c0 msgr2=0x7fbe7406cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:33.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.800+0000 7fbe79800640 1 --2- 172.21.15.44:0/2261228654 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe7406d7c0 0x7fbe7406cc20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009900 tx=0x7fbe6001c850 comp rx=0 tx=0).stop 2024-09-13T08:46:33.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.800+0000 7fbe79800640 1 -- 172.21.15.44:0/2261228654 shutdown_connections 2024-09-13T08:46:33.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.800+0000 7fbe79800640 1 --2- 172.21.15.44:0/2261228654 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe7406d7c0 0x7fbe7406cc20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.800+0000 7fbe79800640 1 --2- 172.21.15.44:0/2261228654 >> v2:172.21.15.129:3300/0 conn(0x7fbe74156280 0x7fbe7415c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.800+0000 7fbe79800640 1 -- 172.21.15.44:0/2261228654 >> 172.21.15.44:0/2261228654 conn(0x7fbe740d0a90 msgr2=0x7fbe740d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:33.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.800+0000 7fbe79800640 1 -- 172.21.15.44:0/2261228654 shutdown_connections 2024-09-13T08:46:33.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.800+0000 7fbe79800640 1 -- 172.21.15.44:0/2261228654 wait complete. 2024-09-13T08:46:33.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe79800640 1 Processor -- start 2024-09-13T08:46:33.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe79800640 1 -- start start 2024-09-13T08:46:33.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe79800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe7406d160 0x7fbe740d1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:33.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe79800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe7406d7c0 0x7fbe7415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:33.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe79800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe74156280 0x7fbe7415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:33.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe79800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe74168c60 con 0x7fbe7406d160 2024-09-13T08:46:33.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe79800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe74168ae0 con 0x7fbe74156280 2024-09-13T08:46:33.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe79800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fbe74168de0 con 0x7fbe7406d7c0 2024-09-13T08:46:33.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe73e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe7406d160 0x7fbe740d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:33.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe73e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe7406d160 0x7fbe740d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34936/0 (socket says 172.21.15.44:34936) 2024-09-13T08:46:33.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe73e00640 1 -- 172.21.15.44:0/2375671168 learned_addr learned my addr 172.21.15.44:0/2375671168 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:33.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.804+0000 7fbe78e00640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe74156280 0x7fbe7415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:33.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.808+0000 7fbe73e00640 1 -- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe7406d7c0 msgr2=0x7fbe7415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:33.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.808+0000 7fbe73e00640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe7406d7c0 0x7fbe7415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.808+0000 7fbe73e00640 1 -- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe74156280 msgr2=0x7fbe7415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:33.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.808+0000 7fbe73e00640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe74156280 0x7fbe7415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:33.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.808+0000 7fbe73e00640 1 -- 172.21.15.44:0/2375671168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe740b4190 con 0x7fbe7406d160 2024-09-13T08:46:33.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.808+0000 7fbe73e00640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe7406d160 0x7fbe740d1420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400d750 tx=0x7fbe6400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.808+0000 7fbe6be00640 1 -- 172.21.15.44:0/2375671168 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64017070 con 0x7fbe7406d160 2024-09-13T08:46:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.808+0000 7fbe6be00640 1 -- 172.21.15.44:0/2375671168 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe64009d20 con 0x7fbe7406d160 2024-09-13T08:46:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.812+0000 7fbe79800640 1 -- 172.21.15.44:0/2375671168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe740b3bd0 con 0x7fbe7406d160 2024-09-13T08:46:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.812+0000 7fbe6be00640 1 -- 172.21.15.44:0/2375671168 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64005280 con 0x7fbe7406d160 2024-09-13T08:46:33.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.812+0000 7fbe79800640 1 -- 172.21.15.44:0/2375671168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe740b62f0 con 0x7fbe7406d160 2024-09-13T08:46:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.812+0000 7fbe79800640 1 -- 172.21.15.44:0/2375671168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe740b6100 con 0x7fbe7406d160 2024-09-13T08:46:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.816+0000 7fbe6be00640 1 -- 172.21.15.44:0/2375671168 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 33) ==== 98559+0+0 (secure 0 0 0) 0x7fbe64005420 con 0x7fbe7406d160 2024-09-13T08:46:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.816+0000 7fbe6be00640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe54073f90 0x7fbe54076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.816+0000 7fbe6be00640 1 -- 172.21.15.44:0/2375671168 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fbe64092d00 con 0x7fbe7406d160 2024-09-13T08:46:33.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.824+0000 7fbe6be00640 1 -- 172.21.15.44:0/2375671168 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fbe6400de60 con 0x7fbe7406d160 2024-09-13T08:46:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.840+0000 7fbe73400640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe54073f90 0x7fbe54076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:33.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:33.884+0000 7fbe73400640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe54073f90 0x7fbe54076450 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbe600af000 tx=0x7fbe600195f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:34.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:34.240+0000 7fbe79800640 1 -- 172.21.15.44:0/2375671168 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone54740", "target": ["mon-mgr", ""]}) -- 0x7fbe740b3980 con 0x7fbe54073f90 2024-09-13T08:46:34.365 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:34.360+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:34.366 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:34.360+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:34.366 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:34.360+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:34.366 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:34.360+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:34.366 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:34.360+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:34.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:34.880+0000 7fbe6be00640 1 -- 172.21.15.44:0/2375671168 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7fbe6405b230 con 0x7fbe7406d160 2024-09-13T08:46:35.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.420+0000 7fbe6be00640 1 -- 172.21.15.44:0/2375671168 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbe740b3980 con 0x7fbe54073f90 2024-09-13T08:46:35.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 -- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe54073f90 msgr2=0x7fbe54076450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:35.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe54073f90 0x7fbe54076450 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbe600af000 tx=0x7fbe600195f0 comp rx=0 tx=0).stop 2024-09-13T08:46:35.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 -- 172.21.15.44:0/2375671168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe7406d160 msgr2=0x7fbe740d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:35.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe7406d160 0x7fbe740d1420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400d750 tx=0x7fbe6400dc10 comp rx=0 tx=0).stop 2024-09-13T08:46:35.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 -- 172.21.15.44:0/2375671168 shutdown_connections 2024-09-13T08:46:35.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe54073f90 0x7fbe54076450 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:35.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe74156280 0x7fbe7415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:35.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe7406d7c0 0x7fbe7415eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:35.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 --2- 172.21.15.44:0/2375671168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe7406d160 0x7fbe740d1420 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:35.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 -- 172.21.15.44:0/2375671168 >> 172.21.15.44:0/2375671168 conn(0x7fbe740d0a90 msgr2=0x7fbe741657b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:35.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 -- 172.21.15.44:0/2375671168 shutdown_connections 2024-09-13T08:46:35.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.424+0000 7fbe79800640 1 -- 172.21.15.44:0/2375671168 wait complete. 2024-09-13T08:46:35.459 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone3811 2024-09-13T08:46:35.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.716+0000 7fb5c6000640 1 -- 172.21.15.44:0/1872565420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c00aea00 msgr2=0x7fb5c00cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:35.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.716+0000 7fb5c6000640 1 --2- 172.21.15.44:0/1872565420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c00aea00 0x7fb5c00cdce0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800be10 tx=0x7fb5b801e9f0 comp rx=0 tx=0).stop 2024-09-13T08:46:35.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.716+0000 7fb5c6000640 1 -- 172.21.15.44:0/1872565420 shutdown_connections 2024-09-13T08:46:35.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.716+0000 7fb5c6000640 1 --2- 172.21.15.44:0/1872565420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c00aea00 0x7fb5c00cdce0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:35.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.716+0000 7fb5c6000640 1 -- 172.21.15.44:0/1872565420 >> 172.21.15.44:0/1872565420 conn(0x7fb5c00cfa00 msgr2=0x7fb5c00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:35.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.716+0000 7fb5c6000640 1 -- 172.21.15.44:0/1872565420 shutdown_connections 2024-09-13T08:46:35.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.716+0000 7fb5c6000640 1 -- 172.21.15.44:0/1872565420 wait complete. 2024-09-13T08:46:35.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.720+0000 7fb5c6000640 1 Processor -- start 2024-09-13T08:46:35.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.720+0000 7fb5c6000640 1 -- start start 2024-09-13T08:46:35.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5c6000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb5c00aea00 0x7fb5c00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:35.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5c6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c00ca760 0x7fb5c00cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:35.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5c6000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c015e450 0x7fb5c0158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:35.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5c6000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c006f6f0 con 0x7fb5c00ca760 2024-09-13T08:46:35.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5c6000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00af180 con 0x7fb5c015e450 2024-09-13T08:46:35.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5c6000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c006f870 con 0x7fb5c00aea00 2024-09-13T08:46:35.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c00ca760 0x7fb5c00cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:35.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c00ca760 0x7fb5c00cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34948/0 (socket says 172.21.15.44:34948) 2024-09-13T08:46:35.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5bfe00640 1 -- 172.21.15.44:0/490092389 learned_addr learned my addr 172.21.15.44:0/490092389 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:35.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5c5600640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c015e450 0x7fb5c0158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:35.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5bfe00640 1 -- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb5c00aea00 msgr2=0x7fb5c00cd570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:46:35.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5bfe00640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb5c00aea00 0x7fb5c00cd570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:35.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5bfe00640 1 -- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c015e450 msgr2=0x7fb5c0158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:35.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5bfe00640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c015e450 0x7fb5c0158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:35.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.724+0000 7fb5bfe00640 1 -- 172.21.15.44:0/490092389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c014cde0 con 0x7fb5c00ca760 2024-09-13T08:46:35.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.728+0000 7fb5bfe00640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c00ca760 0x7fb5c00cdab0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00c490 tx=0x7fb5ac001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:35.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.728+0000 7fb5bcc00640 1 -- 172.21.15.44:0/490092389 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac017070 con 0x7fb5c00ca760 2024-09-13T08:46:35.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.728+0000 7fb5bcc00640 1 -- 172.21.15.44:0/490092389 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5ac003600 con 0x7fb5c00ca760 2024-09-13T08:46:35.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.728+0000 7fb5bcc00640 1 -- 172.21.15.44:0/490092389 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac003a90 con 0x7fb5c00ca760 2024-09-13T08:46:35.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.728+0000 7fb5c6000640 1 -- 172.21.15.44:0/490092389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c0156ee0 con 0x7fb5c00ca760 2024-09-13T08:46:35.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.728+0000 7fb5c6000640 1 -- 172.21.15.44:0/490092389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c00c5cf0 con 0x7fb5c00ca760 2024-09-13T08:46:35.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.728+0000 7fb5c6000640 1 -- 172.21.15.44:0/490092389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c00ca050 con 0x7fb5c00ca760 2024-09-13T08:46:35.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.732+0000 7fb5bcc00640 1 -- 172.21.15.44:0/490092389 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7fb5ac003c30 con 0x7fb5c00ca760 2024-09-13T08:46:35.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.732+0000 7fb5bcc00640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0073fa0 0x7fb5a0076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:35.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.732+0000 7fb5bcc00640 1 -- 172.21.15.44:0/490092389 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fb5ac091b80 con 0x7fb5c00ca760 2024-09-13T08:46:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.740+0000 7fb5c4c00640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0073fa0 0x7fb5a0076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.744+0000 7fb5bcc00640 1 -- 172.21.15.44:0/490092389 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb5ac05e880 con 0x7fb5c00ca760 2024-09-13T08:46:35.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:35.744+0000 7fb5c4c00640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0073fa0 0x7fb5a0076460 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4006fd0 tx=0x7fb5b4006b40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:36.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.140+0000 7fb5c6000640 1 -- 172.21.15.44:0/490092389 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone3811", "target": ["mon-mgr", ""]}) -- 0x7fb5c00d0780 con 0x7fb5a0073fa0 2024-09-13T08:46:36.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.764+0000 7fb5bcc00640 1 -- 172.21.15.44:0/490092389 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb5c00d0780 con 0x7fb5a0073fa0 2024-09-13T08:46:36.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.768+0000 7fb5c6000640 1 -- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0073fa0 msgr2=0x7fb5a0076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:36.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.768+0000 7fb5c6000640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0073fa0 0x7fb5a0076460 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4006fd0 tx=0x7fb5b4006b40 comp rx=0 tx=0).stop 2024-09-13T08:46:36.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.768+0000 7fb5c6000640 1 -- 172.21.15.44:0/490092389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c00ca760 msgr2=0x7fb5c00cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:36.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.768+0000 7fb5c6000640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c00ca760 0x7fb5c00cdab0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00c490 tx=0x7fb5ac001e80 comp rx=0 tx=0).stop 2024-09-13T08:46:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.772+0000 7fb5c6000640 1 -- 172.21.15.44:0/490092389 shutdown_connections 2024-09-13T08:46:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.772+0000 7fb5c6000640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0073fa0 0x7fb5a0076460 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.772+0000 7fb5c6000640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c015e450 0x7fb5c0158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.772+0000 7fb5c6000640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c00ca760 0x7fb5c00cdab0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.772+0000 7fb5c6000640 1 --2- 172.21.15.44:0/490092389 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb5c00aea00 0x7fb5c00cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.772+0000 7fb5c6000640 1 -- 172.21.15.44:0/490092389 >> 172.21.15.44:0/490092389 conn(0x7fb5c00cfa00 msgr2=0x7fb5c00ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.772+0000 7fb5c6000640 1 -- 172.21.15.44:0/490092389 shutdown_connections 2024-09-13T08:46:36.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:36.772+0000 7fb5c6000640 1 -- 172.21.15.44:0/490092389 wait complete. 2024-09-13T08:46:36.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone75132 2024-09-13T08:46:37.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.068+0000 7fbf57800640 1 -- 172.21.15.44:0/3335196943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf500d0780 msgr2=0x7fbf5006cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.068+0000 7fbf57800640 1 --2- 172.21.15.44:0/3335196943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf500d0780 0x7fbf5006cc20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbf500b03d0 tx=0x7fbf3c01c8c0 comp rx=0 tx=0).stop 2024-09-13T08:46:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.068+0000 7fbf57800640 1 -- 172.21.15.44:0/3335196943 shutdown_connections 2024-09-13T08:46:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.068+0000 7fbf57800640 1 --2- 172.21.15.44:0/3335196943 >> v2:172.21.15.129:3300/0 conn(0x7fbf500748a0 0x7fbf5006d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.068+0000 7fbf57800640 1 --2- 172.21.15.44:0/3335196943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf500d0780 0x7fbf5006cc20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.068+0000 7fbf57800640 1 -- 172.21.15.44:0/3335196943 >> 172.21.15.44:0/3335196943 conn(0x7fbf500d0c90 msgr2=0x7fbf500d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.068+0000 7fbf57800640 1 -- 172.21.15.44:0/3335196943 shutdown_connections 2024-09-13T08:46:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.068+0000 7fbf57800640 1 -- 172.21.15.44:0/3335196943 wait complete. 2024-09-13T08:46:37.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.068+0000 7fbf57800640 1 Processor -- start 2024-09-13T08:46:37.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf57800640 1 -- start start 2024-09-13T08:46:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf57800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf500748a0 0x7fbf500c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf57800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf500cd280 0x7fbf500b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf57800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf500d0780 0x7fbf50159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf57800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5016ace0 con 0x7fbf500748a0 2024-09-13T08:46:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf57800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5016ab60 con 0x7fbf500cd280 2024-09-13T08:46:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf57800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5016ae60 con 0x7fbf500d0780 2024-09-13T08:46:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf54e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf500748a0 0x7fbf500c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf54e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf500748a0 0x7fbf500c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34972/0 (socket says 172.21.15.44:34972) 2024-09-13T08:46:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf54e00640 1 -- 172.21.15.44:0/3456846774 learned_addr learned my addr 172.21.15.44:0/3456846774 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf54e00640 1 -- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf500d0780 msgr2=0x7fbf50159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf54e00640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf500d0780 0x7fbf50159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf54e00640 1 -- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf500cd280 msgr2=0x7fbf500b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf54e00640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf500cd280 0x7fbf500b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf54e00640 1 -- 172.21.15.44:0/3456846774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf500c7b70 con 0x7fbf500748a0 2024-09-13T08:46:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf54e00640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf500748a0 0x7fbf500c7140 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400c4c0 tx=0x7fbf44001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf4cc00640 1 -- 172.21.15.44:0/3456846774 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44017070 con 0x7fbf500748a0 2024-09-13T08:46:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf4cc00640 1 -- 172.21.15.44:0/3456846774 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf44003600 con 0x7fbf500748a0 2024-09-13T08:46:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf4cc00640 1 -- 172.21.15.44:0/3456846774 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44003a50 con 0x7fbf500748a0 2024-09-13T08:46:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf57800640 1 -- 172.21.15.44:0/3456846774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf50155c80 con 0x7fbf500748a0 2024-09-13T08:46:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf57800640 1 -- 172.21.15.44:0/3456846774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf50156240 con 0x7fbf500748a0 2024-09-13T08:46:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.072+0000 7fbf57800640 1 -- 172.21.15.44:0/3456846774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf500d0070 con 0x7fbf500748a0 2024-09-13T08:46:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.076+0000 7fbf4cc00640 1 -- 172.21.15.44:0/3456846774 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7fbf44003bf0 con 0x7fbf500748a0 2024-09-13T08:46:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.076+0000 7fbf4cc00640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf30073fa0 0x7fbf30076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.076+0000 7fbf4cc00640 1 -- 172.21.15.44:0/3456846774 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fbf44091bb0 con 0x7fbf500748a0 2024-09-13T08:46:37.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.084+0000 7fbf4cc00640 1 -- 172.21.15.44:0/3456846774 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fbf4405e8b0 con 0x7fbf500748a0 2024-09-13T08:46:37.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.100+0000 7fbf4fe00640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf30073fa0 0x7fbf30076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:37.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.116+0000 7fbf4fe00640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf30073fa0 0x7fbf30076460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0af000 tx=0x7fbf3c019660 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:37.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:37.496+0000 7fbf57800640 1 -- 172.21.15.44:0/3456846774 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone75132", "target": ["mon-mgr", ""]}) -- 0x7fbf501589d0 con 0x7fbf30073fa0 2024-09-13T08:46:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.180+0000 7fbf4cc00640 1 -- 172.21.15.44:0/3456846774 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbf501589d0 con 0x7fbf30073fa0 2024-09-13T08:46:38.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 -- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf30073fa0 msgr2=0x7fbf30076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf30073fa0 0x7fbf30076460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0af000 tx=0x7fbf3c019660 comp rx=0 tx=0).stop 2024-09-13T08:46:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 -- 172.21.15.44:0/3456846774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf500748a0 msgr2=0x7fbf500c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf500748a0 0x7fbf500c7140 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400c4c0 tx=0x7fbf44001e80 comp rx=0 tx=0).stop 2024-09-13T08:46:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 -- 172.21.15.44:0/3456846774 shutdown_connections 2024-09-13T08:46:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbf30073fa0 0x7fbf30076460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbf500d0780 0x7fbf50159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf500cd280 0x7fbf500b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 --2- 172.21.15.44:0/3456846774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf500748a0 0x7fbf500c7140 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 -- 172.21.15.44:0/3456846774 >> 172.21.15.44:0/3456846774 conn(0x7fbf500d0c90 msgr2=0x7fbf50157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:38.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.188+0000 7fbf57800640 1 -- 172.21.15.44:0/3456846774 shutdown_connections 2024-09-13T08:46:38.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.192+0000 7fbf57800640 1 -- 172.21.15.44:0/3456846774 wait complete. 2024-09-13T08:46:38.225 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone53093 2024-09-13T08:46:38.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.492+0000 7fb474c00640 1 -- 172.21.15.44:0/1418532029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4700b3300 msgr2=0x7fb470165950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:38.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.492+0000 7fb474c00640 1 --2- 172.21.15.44:0/1418532029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4700b3300 0x7fb470165950 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009900 tx=0x7fb45c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:46:38.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.492+0000 7fb474c00640 1 -- 172.21.15.44:0/1418532029 shutdown_connections 2024-09-13T08:46:38.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.492+0000 7fb474c00640 1 --2- 172.21.15.44:0/1418532029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4700b3300 0x7fb470165950 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:38.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.492+0000 7fb474c00640 1 -- 172.21.15.44:0/1418532029 >> 172.21.15.44:0/1418532029 conn(0x7fb4700b18f0 msgr2=0x7fb4700b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:38.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.492+0000 7fb474c00640 1 -- 172.21.15.44:0/1418532029 shutdown_connections 2024-09-13T08:46:38.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.492+0000 7fb474c00640 1 -- 172.21.15.44:0/1418532029 wait complete. 2024-09-13T08:46:38.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb474c00640 1 Processor -- start 2024-09-13T08:46:38.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb474c00640 1 -- start start 2024-09-13T08:46:38.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb474c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4700b3300 0x7fb47014d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:38.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb474c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4700b4bd0 0x7fb4700aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:38.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb474c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb47015b580 0x7fb4701548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:38.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb474c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb470168c00 con 0x7fb4700b4bd0 2024-09-13T08:46:38.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb474c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb470168a80 con 0x7fb47015b580 2024-09-13T08:46:38.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb474c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb470168d80 con 0x7fb4700b3300 2024-09-13T08:46:38.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4700b4bd0 0x7fb4700aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:38.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4700b4bd0 0x7fb4700aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34990/0 (socket says 172.21.15.44:34990) 2024-09-13T08:46:38.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46d400640 1 -- 172.21.15.44:0/3327962396 learned_addr learned my addr 172.21.15.44:0/3327962396 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:38.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46de00640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4700b3300 0x7fb47014d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:38.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46e800640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb47015b580 0x7fb4701548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:38.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46d400640 1 -- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4700b3300 msgr2=0x7fb47014d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:38.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46d400640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4700b3300 0x7fb47014d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:38.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46d400640 1 -- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb47015b580 msgr2=0x7fb4701548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:38.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46d400640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb47015b580 0x7fb4701548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:38.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46d400640 1 -- 172.21.15.44:0/3327962396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb470152870 con 0x7fb4700b4bd0 2024-09-13T08:46:38.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb46d400640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4700b4bd0 0x7fb4700aeb20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c490 tx=0x7fb460001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:38.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb466000640 1 -- 172.21.15.44:0/3327962396 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460017070 con 0x7fb4700b4bd0 2024-09-13T08:46:38.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb466000640 1 -- 172.21.15.44:0/3327962396 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb460003600 con 0x7fb4700b4bd0 2024-09-13T08:46:38.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb466000640 1 -- 172.21.15.44:0/3327962396 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460003a90 con 0x7fb4700b4bd0 2024-09-13T08:46:38.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb474c00640 1 -- 172.21.15.44:0/3327962396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb470152b50 con 0x7fb4700b4bd0 2024-09-13T08:46:38.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.496+0000 7fb474c00640 1 -- 172.21.15.44:0/3327962396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb470153110 con 0x7fb4700b4bd0 2024-09-13T08:46:38.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.500+0000 7fb474c00640 1 -- 172.21.15.44:0/3327962396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4700b2bf0 con 0x7fb4700b4bd0 2024-09-13T08:46:38.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.500+0000 7fb466000640 1 -- 172.21.15.44:0/3327962396 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7fb460003c30 con 0x7fb4700b4bd0 2024-09-13T08:46:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.504+0000 7fb466000640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb450073fa0 0x7fb450076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.504+0000 7fb466000640 1 -- 172.21.15.44:0/3327962396 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fb460091b60 con 0x7fb4700b4bd0 2024-09-13T08:46:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.508+0000 7fb466000640 1 -- 172.21.15.44:0/3327962396 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb46005e860 con 0x7fb4700b4bd0 2024-09-13T08:46:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.512+0000 7fb46de00640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb450073fa0 0x7fb450076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:38.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.524+0000 7fb46de00640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb450073fa0 0x7fb450076460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb4680012f0 tx=0x7fb468009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:38.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:38.980+0000 7fb474c00640 1 -- 172.21.15.44:0/3327962396 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone53093", "target": ["mon-mgr", ""]}) -- 0x7fb4700ce690 con 0x7fb450073fa0 2024-09-13T08:46:40.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.396+0000 7fb466000640 1 -- 172.21.15.44:0/3327962396 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb4700ce690 con 0x7fb450073fa0 2024-09-13T08:46:40.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 -- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb450073fa0 msgr2=0x7fb450076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb450073fa0 0x7fb450076460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb4680012f0 tx=0x7fb468009040 comp rx=0 tx=0).stop 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 -- 172.21.15.44:0/3327962396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4700b4bd0 msgr2=0x7fb4700aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4700b4bd0 0x7fb4700aeb20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c490 tx=0x7fb460001e80 comp rx=0 tx=0).stop 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 -- 172.21.15.44:0/3327962396 shutdown_connections 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb450073fa0 0x7fb450076460 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb47015b580 0x7fb4701548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4700b4bd0 0x7fb4700aeb20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 --2- 172.21.15.44:0/3327962396 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4700b3300 0x7fb47014d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 -- 172.21.15.44:0/3327962396 >> 172.21.15.44:0/3327962396 conn(0x7fb4700b18f0 msgr2=0x7fb470150740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 -- 172.21.15.44:0/3327962396 shutdown_connections 2024-09-13T08:46:40.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.400+0000 7fb474c00640 1 -- 172.21.15.44:0/3327962396 wait complete. 2024-09-13T08:46:40.447 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone57364 2024-09-13T08:46:40.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.696+0000 7f386dc00640 1 -- 172.21.15.44:0/1105798702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3860036700 msgr2=0x7f38600060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:40.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.696+0000 7f386dc00640 1 --2- 172.21.15.44:0/1105798702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3860036700 0x7f38600060c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f386806cb90 tx=0x7f3868098480 comp rx=0 tx=0).stop 2024-09-13T08:46:40.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.696+0000 7f386dc00640 1 -- 172.21.15.44:0/1105798702 shutdown_connections 2024-09-13T08:46:40.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.696+0000 7f386dc00640 1 --2- 172.21.15.44:0/1105798702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3860036700 0x7f38600060c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:40.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.696+0000 7f386dc00640 1 --2- 172.21.15.44:0/1105798702 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f386000a480 0x7f386000a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:40.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.696+0000 7f386dc00640 1 -- 172.21.15.44:0/1105798702 >> 172.21.15.44:0/1105798702 conn(0x7f3860054570 msgr2=0x7f3860054980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:40.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.700+0000 7f386dc00640 1 -- 172.21.15.44:0/1105798702 shutdown_connections 2024-09-13T08:46:40.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.700+0000 7f386dc00640 1 -- 172.21.15.44:0/1105798702 wait complete. 2024-09-13T08:46:40.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.708+0000 7f386dc00640 1 Processor -- start 2024-09-13T08:46:40.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.708+0000 7f386dc00640 1 -- start start 2024-09-13T08:46:40.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.708+0000 7f386dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386000a480 0x7f3860047820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:40.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.708+0000 7f386dc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f386001c370 0x7f38600d9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:40.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f386dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3860036700 0x7f3860020910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f386dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3860038380 con 0x7f386000a480 2024-09-13T08:46:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f386dc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3860038200 con 0x7f3860036700 2024-09-13T08:46:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f386dc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3860038500 con 0x7f386001c370 2024-09-13T08:46:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f3867e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386000a480 0x7f3860047820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:40.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f3867e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386000a480 0x7f3860047820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35014/0 (socket says 172.21.15.44:35014) 2024-09-13T08:46:40.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f3867e00640 1 -- 172.21.15.44:0/863071831 learned_addr learned my addr 172.21.15.44:0/863071831 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f3867e00640 1 -- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f386001c370 msgr2=0x7f38600d9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f3867e00640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f386001c370 0x7f38600d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f3867e00640 1 -- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3860036700 msgr2=0x7f3860020910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f3867e00640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3860036700 0x7f3860020910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f3867e00640 1 -- 172.21.15.44:0/863071831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386004b100 con 0x7f386000a480 2024-09-13T08:46:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f3867e00640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386000a480 0x7f3860047820 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f385c00b4a0 tx=0x7f385c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f385be00640 1 -- 172.21.15.44:0/863071831 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c014070 con 0x7f386000a480 2024-09-13T08:46:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f385be00640 1 -- 172.21.15.44:0/863071831 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f385c005b10 con 0x7f386000a480 2024-09-13T08:46:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f385be00640 1 -- 172.21.15.44:0/863071831 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c010610 con 0x7f386000a480 2024-09-13T08:46:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f386dc00640 1 -- 172.21.15.44:0/863071831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386004ae20 con 0x7f386000a480 2024-09-13T08:46:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.712+0000 7f386dc00640 1 -- 172.21.15.44:0/863071831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3860020110 con 0x7f386000a480 2024-09-13T08:46:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.716+0000 7f386dc00640 1 -- 172.21.15.44:0/863071831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386000a860 con 0x7f386000a480 2024-09-13T08:46:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.720+0000 7f385be00640 1 -- 172.21.15.44:0/863071831 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7f385c006020 con 0x7f386000a480 2024-09-13T08:46:40.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.724+0000 7f385be00640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3848074070 0x7f3848076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:40.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.724+0000 7f385be00640 1 -- 172.21.15.44:0/863071831 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f385c092870 con 0x7f386000a480 2024-09-13T08:46:40.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.724+0000 7f3867400640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3848074070 0x7f3848076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:40.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.736+0000 7f385be00640 1 -- 172.21.15.44:0/863071831 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f385c05f510 con 0x7f386000a480 2024-09-13T08:46:40.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:40.736+0000 7f3867400640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3848074070 0x7f3848076530 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3860047d60 tx=0x7f3854007970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:41.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.224+0000 7f386dc00640 1 -- 172.21.15.44:0/863071831 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone57364", "target": ["mon-mgr", ""]}) -- 0x7f386001d990 con 0x7f3848074070 2024-09-13T08:46:41.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.992+0000 7f385be00640 1 -- 172.21.15.44:0/863071831 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f386001d990 con 0x7f3848074070 2024-09-13T08:46:41.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.992+0000 7f386dc00640 1 -- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3848074070 msgr2=0x7f3848076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.992+0000 7f386dc00640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3848074070 0x7f3848076530 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3860047d60 tx=0x7f3854007970 comp rx=0 tx=0).stop 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.992+0000 7f386dc00640 1 -- 172.21.15.44:0/863071831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386000a480 msgr2=0x7f3860047820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.992+0000 7f386dc00640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386000a480 0x7f3860047820 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f385c00b4a0 tx=0x7f385c00b960 comp rx=0 tx=0).stop 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.996+0000 7f386dc00640 1 -- 172.21.15.44:0/863071831 shutdown_connections 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.996+0000 7f386dc00640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3848074070 0x7f3848076530 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.996+0000 7f386dc00640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3860036700 0x7f3860020910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.996+0000 7f386dc00640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f386001c370 0x7f38600d9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.996+0000 7f386dc00640 1 --2- 172.21.15.44:0/863071831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386000a480 0x7f3860047820 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.996+0000 7f386dc00640 1 -- 172.21.15.44:0/863071831 >> 172.21.15.44:0/863071831 conn(0x7f3860054570 msgr2=0x7f38600dd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.996+0000 7f386dc00640 1 -- 172.21.15.44:0/863071831 shutdown_connections 2024-09-13T08:46:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:41.996+0000 7f386dc00640 1 -- 172.21.15.44:0/863071831 wait complete. 2024-09-13T08:46:42.023 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone21695 2024-09-13T08:46:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.256+0000 7f9fc5e00640 1 -- 172.21.15.44:0/3374531574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fa4004810 msgr2=0x7f9fa4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.256+0000 7f9fc5e00640 1 --2- 172.21.15.44:0/3374531574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fa4004810 0x7f9fa4004c90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800bb10 tx=0x7f9fa801cda0 comp rx=0 tx=0).stop 2024-09-13T08:46:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.256+0000 7f9fc5e00640 1 -- 172.21.15.44:0/3374531574 shutdown_connections 2024-09-13T08:46:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.256+0000 7f9fc5e00640 1 --2- 172.21.15.44:0/3374531574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fa4004810 0x7f9fa4004c90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.256+0000 7f9fc5e00640 1 -- 172.21.15.44:0/3374531574 >> 172.21.15.44:0/3374531574 conn(0x7f9fc00b18f0 msgr2=0x7f9fc00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.256+0000 7f9fc5e00640 1 -- 172.21.15.44:0/3374531574 shutdown_connections 2024-09-13T08:46:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.256+0000 7f9fc5e00640 1 -- 172.21.15.44:0/3374531574 wait complete. 2024-09-13T08:46:42.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 Processor -- start 2024-09-13T08:46:42.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 -- start start 2024-09-13T08:46:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9fa4004810 0x7f9fc015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fc01f6300 0x7f9fc01efba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc01f6780 0x7f9fc01f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00b2f80 con 0x7f9fc01f6780 2024-09-13T08:46:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00b2e00 con 0x7f9fc01f6300 2024-09-13T08:46:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc00cd000 con 0x7f9fa4004810 2024-09-13T08:46:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc01f6780 0x7f9fc01f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc01f6780 0x7f9fc01f00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57000/0 (socket says 172.21.15.44:57000) 2024-09-13T08:46:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fbf400640 1 -- 172.21.15.44:0/4070561227 learned_addr learned my addr 172.21.15.44:0/4070561227 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:42.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fbf400640 1 -- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9fa4004810 msgr2=0x7f9fc015bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:42.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fbf400640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9fa4004810 0x7f9fc015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:42.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fbf400640 1 -- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fc01f6300 msgr2=0x7f9fc01efba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:42.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fbf400640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fc01f6300 0x7f9fc01efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:42.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fbf400640 1 -- 172.21.15.44:0/4070561227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc01f3a10 con 0x7f9fc01f6780 2024-09-13T08:46:42.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fbf400640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc01f6780 0x7f9fc01f00e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80036b0 tx=0x7f9fb800f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:42.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fb7e00640 1 -- 172.21.15.44:0/4070561227 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8008270 con 0x7f9fc01f6780 2024-09-13T08:46:42.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fb7e00640 1 -- 172.21.15.44:0/4070561227 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fb8008500 con 0x7f9fc01f6780 2024-09-13T08:46:42.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fb7e00640 1 -- 172.21.15.44:0/4070561227 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8006980 con 0x7f9fc01f6780 2024-09-13T08:46:42.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 -- 172.21.15.44:0/4070561227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc01536c0 con 0x7f9fc01f6780 2024-09-13T08:46:42.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 -- 172.21.15.44:0/4070561227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc0153c80 con 0x7f9fc01f6780 2024-09-13T08:46:42.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.260+0000 7f9fc5e00640 1 -- 172.21.15.44:0/4070561227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc015e990 con 0x7f9fc01f6780 2024-09-13T08:46:42.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.264+0000 7f9fb7e00640 1 -- 172.21.15.44:0/4070561227 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7f9fb8014070 con 0x7f9fc01f6780 2024-09-13T08:46:42.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.264+0000 7f9fb7e00640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f9c073fa0 0x7f9f9c076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:42.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.264+0000 7f9fb7e00640 1 -- 172.21.15.44:0/4070561227 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f9fb8094210 con 0x7f9fc01f6780 2024-09-13T08:46:42.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.264+0000 7f9fbea00640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f9c073fa0 0x7f9f9c076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:42.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.268+0000 7f9fb7e00640 1 -- 172.21.15.44:0/4070561227 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9fb801f020 con 0x7f9fc01f6780 2024-09-13T08:46:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.280+0000 7f9fbea00640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f9c073fa0 0x7f9f9c076460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0001cb0 tx=0x7f9fb0007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.608+0000 7f9fc5e00640 1 -- 172.21.15.44:0/4070561227 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone21695", "target": ["mon-mgr", ""]}) -- 0x7f9fc006f2a0 con 0x7f9f9c073fa0 2024-09-13T08:46:42.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fb7e00640 1 -- 172.21.15.44:0/4070561227 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9fc006f2a0 con 0x7f9f9c073fa0 2024-09-13T08:46:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 -- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f9c073fa0 msgr2=0x7f9f9c076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f9c073fa0 0x7f9f9c076460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0001cb0 tx=0x7f9fb0007d40 comp rx=0 tx=0).stop 2024-09-13T08:46:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 -- 172.21.15.44:0/4070561227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc01f6780 msgr2=0x7f9fc01f00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc01f6780 0x7f9fc01f00e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80036b0 tx=0x7f9fb800f820 comp rx=0 tx=0).stop 2024-09-13T08:46:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 -- 172.21.15.44:0/4070561227 shutdown_connections 2024-09-13T08:46:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9f9c073fa0 0x7f9f9c076460 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc01f6780 0x7f9fc01f00e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fc01f6300 0x7f9fc01efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:42.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 --2- 172.21.15.44:0/4070561227 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9fa4004810 0x7f9fc015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:42.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 -- 172.21.15.44:0/4070561227 >> 172.21.15.44:0/4070561227 conn(0x7f9fc00b18f0 msgr2=0x7f9fc00ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:42.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 -- 172.21.15.44:0/4070561227 shutdown_connections 2024-09-13T08:46:42.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:42.984+0000 7f9fc5e00640 1 -- 172.21.15.44:0/4070561227 wait complete. 2024-09-13T08:46:43.030 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41056 2024-09-13T08:46:43.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.260+0000 7ff1c8800640 1 -- 172.21.15.44:0/3006059635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1a8006400 msgr2=0x7ff1a800cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:43.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.260+0000 7ff1c8800640 1 --2- 172.21.15.44:0/3006059635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1a8006400 0x7ff1a800cae0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00c640 tx=0x7ff1bc01d260 comp rx=0 tx=0).stop 2024-09-13T08:46:43.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.260+0000 7ff1c6800640 1 -- 172.21.15.44:0/3006059635 reap_dead start 2024-09-13T08:46:43.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.260+0000 7ff1c8800640 1 -- 172.21.15.44:0/3006059635 shutdown_connections 2024-09-13T08:46:43.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.260+0000 7ff1c8800640 1 -- 172.21.15.44:0/3006059635 >> 172.21.15.44:0/3006059635 conn(0x7ff1c00b18f0 msgr2=0x7ff1c00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:43.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.260+0000 7ff1c8800640 1 -- 172.21.15.44:0/3006059635 shutdown_connections 2024-09-13T08:46:43.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.260+0000 7ff1c8800640 1 -- 172.21.15.44:0/3006059635 wait complete. 2024-09-13T08:46:43.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.260+0000 7ff1c8800640 1 Processor -- start 2024-09-13T08:46:43.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.260+0000 7ff1c8800640 1 -- start start 2024-09-13T08:46:43.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.264+0000 7ff1c8800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1a8006400 0x7ff1c01f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:43.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.264+0000 7ff1c8800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c015ce80 0x7ff1c00adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:43.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.264+0000 7ff1c8800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1c00ae090 0x7ff1c009ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:43.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.264+0000 7ff1c8800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0166420 con 0x7ff1c00ae090 2024-09-13T08:46:43.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.264+0000 7ff1c8800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01662a0 con 0x7ff1c015ce80 2024-09-13T08:46:43.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.264+0000 7ff1c8800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c01665a0 con 0x7ff1a8006400 2024-09-13T08:46:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.264+0000 7ff1c6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1c00ae090 0x7ff1c009ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.264+0000 7ff1c5e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1a8006400 0x7ff1c01f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.264+0000 7ff1c5e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1a8006400 0x7ff1c01f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:44850/0 (socket says 172.21.15.44:44850) 2024-09-13T08:46:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.264+0000 7ff1c5e00640 1 -- 172.21.15.44:0/1615813368 learned_addr learned my addr 172.21.15.44:0/1615813368 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c5400640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c015ce80 0x7ff1c00adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c6800640 1 -- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1a8006400 msgr2=0x7ff1c01f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c6800640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1a8006400 0x7ff1c01f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c6800640 1 -- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c015ce80 msgr2=0x7ff1c00adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:43.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c6800640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c015ce80 0x7ff1c00adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:43.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c6800640 1 -- 172.21.15.44:0/1615813368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c015b890 con 0x7ff1c00ae090 2024-09-13T08:46:43.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c5400640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c015ce80 0x7ff1c00adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:43.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c6800640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1c00ae090 0x7ff1c009ce40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0a9000 tx=0x7ff1bc006640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:43.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1ba000640 1 -- 172.21.15.44:0/1615813368 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc004e60 con 0x7ff1c00ae090 2024-09-13T08:46:43.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c8800640 1 -- 172.21.15.44:0/1615813368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c015bbd0 con 0x7ff1c00ae090 2024-09-13T08:46:43.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1ba000640 1 -- 172.21.15.44:0/1615813368 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1bc006730 con 0x7ff1c00ae090 2024-09-13T08:46:43.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c8800640 1 -- 172.21.15.44:0/1615813368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c015c130 con 0x7ff1c00ae090 2024-09-13T08:46:43.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1ba000640 1 -- 172.21.15.44:0/1615813368 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc001fb0 con 0x7ff1c00ae090 2024-09-13T08:46:43.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.268+0000 7ff1c8800640 1 -- 172.21.15.44:0/1615813368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c00cc740 con 0x7ff1c00ae090 2024-09-13T08:46:43.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.276+0000 7ff1ba000640 1 -- 172.21.15.44:0/1615813368 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7ff1bc0c0060 con 0x7ff1c00ae090 2024-09-13T08:46:43.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.280+0000 7ff1ba000640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a4073ff0 0x7ff1a40764b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:43.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.280+0000 7ff1ba000640 1 -- 172.21.15.44:0/1615813368 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7ff1bc12d610 con 0x7ff1c00ae090 2024-09-13T08:46:43.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.280+0000 7ff1c5e00640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a4073ff0 0x7ff1a40764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:43.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.284+0000 7ff1ba000640 1 -- 172.21.15.44:0/1615813368 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff1bc0c8020 con 0x7ff1c00ae090 2024-09-13T08:46:43.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.284+0000 7ff1c5e00640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a4073ff0 0x7ff1a40764b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006800 tx=0x7ff1b4006b20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:43.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.592+0000 7ff1c8800640 1 -- 172.21.15.44:0/1615813368 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41056", "target": ["mon-mgr", ""]}) -- 0x7ff1c00cd3d0 con 0x7ff1a4073ff0 2024-09-13T08:46:43.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1ba000640 1 -- 172.21.15.44:0/1615813368 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff1c00cd3d0 con 0x7ff1a4073ff0 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 -- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a4073ff0 msgr2=0x7ff1a40764b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a4073ff0 0x7ff1a40764b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006800 tx=0x7ff1b4006b20 comp rx=0 tx=0).stop 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 -- 172.21.15.44:0/1615813368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1c00ae090 msgr2=0x7ff1c009ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1c00ae090 0x7ff1c009ce40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0a9000 tx=0x7ff1bc006640 comp rx=0 tx=0).stop 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 -- 172.21.15.44:0/1615813368 shutdown_connections 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a4073ff0 0x7ff1a40764b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1c00ae090 0x7ff1c009ce40 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c015ce80 0x7ff1c00adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 --2- 172.21.15.44:0/1615813368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1a8006400 0x7ff1c01f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 -- 172.21.15.44:0/1615813368 >> 172.21.15.44:0/1615813368 conn(0x7ff1c00b18f0 msgr2=0x7ff1c00b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:43.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 -- 172.21.15.44:0/1615813368 shutdown_connections 2024-09-13T08:46:43.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.768+0000 7ff1c8800640 1 -- 172.21.15.44:0/1615813368 wait complete. 2024-09-13T08:46:43.791 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:46:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.936+0000 7efd84c00640 1 -- 172.21.15.44:0/3822903804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd800cdb30 msgr2=0x7efd800af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.936+0000 7efd84c00640 1 --2- 172.21.15.44:0/3822903804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd800cdb30 0x7efd800af0c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7efd78009bc0 tx=0x7efd7801cba0 comp rx=0 tx=0).stop 2024-09-13T08:46:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.936+0000 7efd84c00640 1 -- 172.21.15.44:0/3822903804 shutdown_connections 2024-09-13T08:46:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.936+0000 7efd84c00640 1 --2- 172.21.15.44:0/3822903804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd800cdb30 0x7efd800af0c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.936+0000 7efd84c00640 1 -- 172.21.15.44:0/3822903804 >> 172.21.15.44:0/3822903804 conn(0x7efd800d1dc0 msgr2=0x7efd800d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.936+0000 7efd84c00640 1 -- 172.21.15.44:0/3822903804 shutdown_connections 2024-09-13T08:46:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.936+0000 7efd84c00640 1 -- 172.21.15.44:0/3822903804 wait complete. 2024-09-13T08:46:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.936+0000 7efd84c00640 1 Processor -- start 2024-09-13T08:46:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.936+0000 7efd84c00640 1 -- start start 2024-09-13T08:46:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd84c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd800cdb30 0x7efd8014f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd84c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efd800ced80 0x7efd800b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd84c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8015bea0 0x7efd800c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd84c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd8016a8a0 con 0x7efd800cdb30 2024-09-13T08:46:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd84c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd8016a720 con 0x7efd8015bea0 2024-09-13T08:46:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd84c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7efd8016aa20 con 0x7efd800ced80 2024-09-13T08:46:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd800cdb30 0x7efd8014f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd800cdb30 0x7efd8014f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57052/0 (socket says 172.21.15.44:57052) 2024-09-13T08:46:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7de00640 1 -- 172.21.15.44:0/898687294 learned_addr learned my addr 172.21.15.44:0/898687294 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7d400640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efd800ced80 0x7efd800b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7e800640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8015bea0 0x7efd800c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7de00640 1 -- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efd800ced80 msgr2=0x7efd800b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7de00640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efd800ced80 0x7efd800b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7de00640 1 -- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8015bea0 msgr2=0x7efd800c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7de00640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8015bea0 0x7efd800c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7de00640 1 -- 172.21.15.44:0/898687294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd80159db0 con 0x7efd800cdb30 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd7de00640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd800cdb30 0x7efd8014f3b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c420 tx=0x7efd7000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd76000640 1 -- 172.21.15.44:0/898687294 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70017070 con 0x7efd800cdb30 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd76000640 1 -- 172.21.15.44:0/898687294 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd700053c0 con 0x7efd800cdb30 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd76000640 1 -- 172.21.15.44:0/898687294 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70003860 con 0x7efd800cdb30 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd84c00640 1 -- 172.21.15.44:0/898687294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8015a090 con 0x7efd800cdb30 2024-09-13T08:46:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd84c00640 1 -- 172.21.15.44:0/898687294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8014f8f0 con 0x7efd800cdb30 2024-09-13T08:46:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd84c00640 1 -- 172.21.15.44:0/898687294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd800ce670 con 0x7efd800cdb30 2024-09-13T08:46:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.940+0000 7efd76000640 1 -- 172.21.15.44:0/898687294 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7efd7001a070 con 0x7efd800cdb30 2024-09-13T08:46:43.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.944+0000 7efd76000640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efd60073fa0 0x7efd60076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:43.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.944+0000 7efd76000640 1 -- 172.21.15.44:0/898687294 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7efd70091fe0 con 0x7efd800cdb30 2024-09-13T08:46:43.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.944+0000 7efd7d400640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efd60073fa0 0x7efd60076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.944+0000 7efd7d400640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efd60073fa0 0x7efd60076460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efd800b4870 tx=0x7efd6c007cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:43.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:43.948+0000 7efd76000640 1 -- 172.21.15.44:0/898687294 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7efd7005e2f0 con 0x7efd800cdb30 2024-09-13T08:46:44.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.148+0000 7efd84c00640 1 -- 172.21.15.44:0/898687294 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7efd800724d0 con 0x7efd60073fa0 2024-09-13T08:46:44.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd76000640 1 -- 172.21.15.44:0/898687294 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7efd800724d0 con 0x7efd60073fa0 2024-09-13T08:46:44.152 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:46:44.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:44.153 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:46:44.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:44.153 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 -- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efd60073fa0 msgr2=0x7efd60076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efd60073fa0 0x7efd60076460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efd800b4870 tx=0x7efd6c007cd0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 -- 172.21.15.44:0/898687294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd800cdb30 msgr2=0x7efd8014f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd800cdb30 0x7efd8014f3b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c420 tx=0x7efd7000c8e0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 -- 172.21.15.44:0/898687294 shutdown_connections 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efd60073fa0 0x7efd60076460 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8015bea0 0x7efd800c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efd800ced80 0x7efd800b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 --2- 172.21.15.44:0/898687294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd800cdb30 0x7efd8014f3b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 -- 172.21.15.44:0/898687294 >> 172.21.15.44:0/898687294 conn(0x7efd800d1dc0 msgr2=0x7efd800d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:44.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 -- 172.21.15.44:0/898687294 shutdown_connections 2024-09-13T08:46:44.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.152+0000 7efd84c00640 1 -- 172.21.15.44:0/898687294 wait complete. 2024-09-13T08:46:44.174 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-13T08:46:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.320+0000 7f11e2c00640 1 -- 172.21.15.44:0/2667964281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11dc0ce980 msgr2=0x7f11dc0cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.320+0000 7f11e2c00640 1 --2- 172.21.15.44:0/2667964281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11dc0ce980 0x7f11dc0cf9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009bc0 tx=0x7f11d401cb70 comp rx=0 tx=0).stop 2024-09-13T08:46:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.320+0000 7f11e2c00640 1 -- 172.21.15.44:0/2667964281 shutdown_connections 2024-09-13T08:46:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.320+0000 7f11e2c00640 1 --2- 172.21.15.44:0/2667964281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11dc0ce980 0x7f11dc0cf9c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.320+0000 7f11e2c00640 1 -- 172.21.15.44:0/2667964281 >> 172.21.15.44:0/2667964281 conn(0x7f11dc0d1df0 msgr2=0x7f11dc0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.320+0000 7f11e2c00640 1 -- 172.21.15.44:0/2667964281 shutdown_connections 2024-09-13T08:46:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.320+0000 7f11e2c00640 1 -- 172.21.15.44:0/2667964281 wait complete. 2024-09-13T08:46:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11e2c00640 1 Processor -- start 2024-09-13T08:46:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11e2c00640 1 -- start start 2024-09-13T08:46:44.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11e2c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11dc0b1190 0x7f11dc0af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:44.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11e2c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11dc0ce980 0x7f11dc0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:44.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11e2c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11dc1603f0 0x7f11dc166950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:44.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11e2c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc0b3410 con 0x7f11dc0b1190 2024-09-13T08:46:44.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11e2c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc0b3290 con 0x7f11dc1603f0 2024-09-13T08:46:44.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11e2c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f11dc16a500 con 0x7f11dc0ce980 2024-09-13T08:46:44.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11dc0b1190 0x7f11dc0af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11dc0b1190 0x7f11dc0af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57058/0 (socket says 172.21.15.44:57058) 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11dbe00640 1 -- 172.21.15.44:0/1848141421 learned_addr learned my addr 172.21.15.44:0/1848141421 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11e0c00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11dc1603f0 0x7f11dc166950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11d3e00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11dc0ce980 0x7f11dc0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11dbe00640 1 -- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11dc0ce980 msgr2=0x7f11dc0afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11dbe00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11dc0ce980 0x7f11dc0afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11dbe00640 1 -- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11dc1603f0 msgr2=0x7f11dc166950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11dbe00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11dc1603f0 0x7f11dc166950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11dbe00640 1 -- 172.21.15.44:0/1848141421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11dc157820 con 0x7f11dc0b1190 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11d3e00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11dc0ce980 0x7f11dc0afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11dbe00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11dc0b1190 0x7f11dc0af8f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f11cc001ec0 tx=0x7f11cc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11d8c00640 1 -- 172.21.15.44:0/1848141421 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc017070 con 0x7f11dc0b1190 2024-09-13T08:46:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11d8c00640 1 -- 172.21.15.44:0/1848141421 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11cc003600 con 0x7f11dc0b1190 2024-09-13T08:46:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.324+0000 7f11e2c00640 1 -- 172.21.15.44:0/1848141421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11dc156080 con 0x7f11dc0b1190 2024-09-13T08:46:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.328+0000 7f11d8c00640 1 -- 172.21.15.44:0/1848141421 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc003a50 con 0x7f11dc0b1190 2024-09-13T08:46:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.328+0000 7f11e2c00640 1 -- 172.21.15.44:0/1848141421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11dc157da0 con 0x7f11dc0b1190 2024-09-13T08:46:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.328+0000 7f11e2c00640 1 -- 172.21.15.44:0/1848141421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11dc0cf9c0 con 0x7f11dc0b1190 2024-09-13T08:46:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.328+0000 7f11d8c00640 1 -- 172.21.15.44:0/1848141421 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7f11cc01a070 con 0x7f11dc0b1190 2024-09-13T08:46:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.328+0000 7f11d8c00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b8073fa0 0x7f11b8076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.328+0000 7f11d8c00640 1 -- 172.21.15.44:0/1848141421 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f11cc091ee0 con 0x7f11dc0b1190 2024-09-13T08:46:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.328+0000 7f11d3e00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b8073fa0 0x7f11b8076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:44.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.332+0000 7f11d3e00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b8073fa0 0x7f11b8076460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f11c4007a60 tx=0x7f11c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:44.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.332+0000 7f11d8c00640 1 -- 172.21.15.44:0/1848141421 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f11cc05e250 con 0x7f11dc0b1190 2024-09-13T08:46:44.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.532+0000 7f11e2c00640 1 -- 172.21.15.44:0/1848141421 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11dc155e30 con 0x7f11b8073fa0 2024-09-13T08:46:44.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11d8c00640 1 -- 172.21.15.44:0/1848141421 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+380 (secure 0 0 0) 0x7f11dc155e30 con 0x7f11b8073fa0 2024-09-13T08:46:44.541 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:44.541 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:46:44.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:44.541 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone54740" 2024-09-13T08:46:44.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:44.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:44.541 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sv1" 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone75132" 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone41056" 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone3811" 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone57364" 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:44.543 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone53093" 2024-09-13T08:46:44.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:46:44.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:44.543 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone21695" 2024-09-13T08:46:44.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:44.543 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:46:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 -- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b8073fa0 msgr2=0x7f11b8076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b8073fa0 0x7f11b8076460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f11c4007a60 tx=0x7f11c4009040 comp rx=0 tx=0).stop 2024-09-13T08:46:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 -- 172.21.15.44:0/1848141421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11dc0b1190 msgr2=0x7f11dc0af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11dc0b1190 0x7f11dc0af8f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f11cc001ec0 tx=0x7f11cc00c890 comp rx=0 tx=0).stop 2024-09-13T08:46:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 -- 172.21.15.44:0/1848141421 shutdown_connections 2024-09-13T08:46:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b8073fa0 0x7f11b8076460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11dc1603f0 0x7f11dc166950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11dc0ce980 0x7f11dc0afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 --2- 172.21.15.44:0/1848141421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11dc0b1190 0x7f11dc0af8f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 -- 172.21.15.44:0/1848141421 >> 172.21.15.44:0/1848141421 conn(0x7f11dc0d1df0 msgr2=0x7f11dc0b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:44.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 -- 172.21.15.44:0/1848141421 shutdown_connections 2024-09-13T08:46:44.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.540+0000 7f11e2c00640 1 -- 172.21.15.44:0/1848141421 wait complete. 2024-09-13T08:46:44.564 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone54740 2024-09-13T08:46:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.712+0000 7f299c200640 1 -- 172.21.15.44:0/4194413504 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f29941601a0 msgr2=0x7f299415c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.712+0000 7f299c200640 1 --2- 172.21.15.44:0/4194413504 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f29941601a0 0x7f299415c070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2980009900 tx=0x7f298001c790 comp rx=0 tx=0).stop 2024-09-13T08:46:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.712+0000 7f299c200640 1 -- 172.21.15.44:0/4194413504 shutdown_connections 2024-09-13T08:46:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.712+0000 7f299c200640 1 --2- 172.21.15.44:0/4194413504 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f29941601a0 0x7f299415c070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.712+0000 7f299c200640 1 --2- 172.21.15.44:0/4194413504 >> v2:172.21.15.129:3300/0 conn(0x7f299415bc90 0x7f2994074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.712+0000 7f299c200640 1 -- 172.21.15.44:0/4194413504 >> 172.21.15.44:0/4194413504 conn(0x7f299406dd30 msgr2=0x7f299406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.712+0000 7f299c200640 1 -- 172.21.15.44:0/4194413504 shutdown_connections 2024-09-13T08:46:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.712+0000 7f299c200640 1 -- 172.21.15.44:0/4194413504 wait complete. 2024-09-13T08:46:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299c200640 1 Processor -- start 2024-09-13T08:46:44.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299c200640 1 -- start start 2024-09-13T08:46:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299c200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f299415bc90 0x7f299414c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299c200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f299415c5b0 0x7f2994070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299c200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29941601a0 0x7f2994164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299c200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2994168cb0 con 0x7f299415c5b0 2024-09-13T08:46:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299c200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2994168b30 con 0x7f29941601a0 2024-09-13T08:46:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299c200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2994168e30 con 0x7f299415bc90 2024-09-13T08:46:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2998e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f299415c5b0 0x7f2994070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2998e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f299415c5b0 0x7f2994070a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57066/0 (socket says 172.21.15.44:57066) 2024-09-13T08:46:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2998e00640 1 -- 172.21.15.44:0/2189215997 learned_addr learned my addr 172.21.15.44:0/2189215997 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:44.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2999800640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f299415bc90 0x7f299414c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:44.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299a200640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29941601a0 0x7f2994164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:44.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2998e00640 1 -- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f299415bc90 msgr2=0x7f299414c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2998e00640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f299415bc90 0x7f299414c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2998e00640 1 -- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29941601a0 msgr2=0x7f2994164cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2998e00640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29941601a0 0x7f2994164cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2998e00640 1 -- 172.21.15.44:0/2189215997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29940714a0 con 0x7f299415c5b0 2024-09-13T08:46:44.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2998e00640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f299415c5b0 0x7f2994070a00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2980009850 tx=0x7f29800a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:44.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2991600640 1 -- 172.21.15.44:0/2189215997 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29800b2040 con 0x7f299415c5b0 2024-09-13T08:46:44.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2991600640 1 -- 172.21.15.44:0/2189215997 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29800042c0 con 0x7f299415c5b0 2024-09-13T08:46:44.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299c200640 1 -- 172.21.15.44:0/2189215997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299406f850 con 0x7f299415c5b0 2024-09-13T08:46:44.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f2991600640 1 -- 172.21.15.44:0/2189215997 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980005030 con 0x7f299415c5b0 2024-09-13T08:46:44.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.716+0000 7f299c200640 1 -- 172.21.15.44:0/2189215997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299414c890 con 0x7f299415c5b0 2024-09-13T08:46:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.720+0000 7f299c200640 1 -- 172.21.15.44:0/2189215997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2994074580 con 0x7f299415c5b0 2024-09-13T08:46:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.720+0000 7f2991600640 1 -- 172.21.15.44:0/2189215997 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7f29800051d0 con 0x7f299415c5b0 2024-09-13T08:46:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.720+0000 7f2991600640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2974073fa0 0x7f2974076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.720+0000 7f2991600640 1 -- 172.21.15.44:0/2189215997 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f298012c7b0 con 0x7f299415c5b0 2024-09-13T08:46:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.720+0000 7f2999800640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2974073fa0 0x7f2974076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.724+0000 7f2999800640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2974073fa0 0x7f2974076460 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2988009600 tx=0x7f29880091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.724+0000 7f2991600640 1 -- 172.21.15.44:0/2189215997 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f29800164d0 con 0x7f299415c5b0 2024-09-13T08:46:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.932+0000 7f299c200640 1 -- 172.21.15.44:0/2189215997 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone54740", "target": ["mon-mgr", ""]}) -- 0x7f29941570d0 con 0x7f2974073fa0 2024-09-13T08:46:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.948+0000 7f2991600640 1 -- 172.21.15.44:0/2189215997 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f29941570d0 con 0x7f2974073fa0 2024-09-13T08:46:44.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 -- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2974073fa0 msgr2=0x7f2974076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2974073fa0 0x7f2974076460 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2988009600 tx=0x7f29880091e0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 -- 172.21.15.44:0/2189215997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f299415c5b0 msgr2=0x7f2994070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f299415c5b0 0x7f2994070a00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2980009850 tx=0x7f29800a56f0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 -- 172.21.15.44:0/2189215997 shutdown_connections 2024-09-13T08:46:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2974073fa0 0x7f2974076460 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29941601a0 0x7f2994164cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f299415c5b0 0x7f2994070a00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 --2- 172.21.15.44:0/2189215997 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f299415bc90 0x7f299414c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 -- 172.21.15.44:0/2189215997 >> 172.21.15.44:0/2189215997 conn(0x7f299406dd30 msgr2=0x7f2994156350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 -- 172.21.15.44:0/2189215997 shutdown_connections 2024-09-13T08:46:44.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:44.952+0000 7f299c200640 1 -- 172.21.15.44:0/2189215997 wait complete. 2024-09-13T08:46:44.964 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:44.963+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:44.964 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:44.963+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:44.964 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:44.963+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:44.964 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:44.963+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:44.964 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:46:44.963+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:46:44.977 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-13T08:46:45.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 -- 172.21.15.44:0/57023641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21a415be10 msgr2=0x7f21a40ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 --2- 172.21.15.44:0/57023641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21a415be10 0x7f21a40ce700 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2198009840 tx=0x7f219801f010 comp rx=0 tx=0).stop 2024-09-13T08:46:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 -- 172.21.15.44:0/57023641 shutdown_connections 2024-09-13T08:46:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 --2- 172.21.15.44:0/57023641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21a415be10 0x7f21a40ce700 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 -- 172.21.15.44:0/57023641 >> 172.21.15.44:0/57023641 conn(0x7f21a40d1da0 msgr2=0x7f21a40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 -- 172.21.15.44:0/57023641 shutdown_connections 2024-09-13T08:46:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 -- 172.21.15.44:0/57023641 wait complete. 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 Processor -- start 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 -- start start 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a40b0b60 0x7f21a4149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21a415be10 0x7f21a40c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21a415c1f0 0x7f21a40cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f21a40cd670 con 0x7f21a40b0b60 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f21a40cd4f0 con 0x7f21a415c1f0 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21ac400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f21a40cd7f0 con 0x7f21a415be10 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21a9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a40b0b60 0x7f21a4149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21a9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a40b0b60 0x7f21a4149330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57078/0 (socket says 172.21.15.44:57078) 2024-09-13T08:46:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21a9a00640 1 -- 172.21.15.44:0/781089073 learned_addr learned my addr 172.21.15.44:0/781089073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21aa400640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21a415c1f0 0x7f21a40cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.128+0000 7f21a9000640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21a415be10 0x7f21a40c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a9a00640 1 -- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21a415be10 msgr2=0x7f21a40c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a9a00640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21a415be10 0x7f21a40c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a9a00640 1 -- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21a415c1f0 msgr2=0x7f21a40cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a9a00640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21a415c1f0 0x7f21a40cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a9a00640 1 -- 172.21.15.44:0/781089073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a41f5850 con 0x7f21a40b0b60 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a9000640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21a415be10 0x7f21a40c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a9a00640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a40b0b60 0x7f21a4149330 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f219801bc20 tx=0x7f2198004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a1600640 1 -- 172.21.15.44:0/781089073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21980b5040 con 0x7f21a40b0b60 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a1600640 1 -- 172.21.15.44:0/781089073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2198005220 con 0x7f21a40b0b60 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21ac400640 1 -- 172.21.15.44:0/781089073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a41f4c40 con 0x7f21a40b0b60 2024-09-13T08:46:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a1600640 1 -- 172.21.15.44:0/781089073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21980046d0 con 0x7f21a40b0b60 2024-09-13T08:46:45.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21ac400640 1 -- 172.21.15.44:0/781089073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a41f5200 con 0x7f21a40b0b60 2024-09-13T08:46:45.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21ac400640 1 -- 172.21.15.44:0/781089073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a4159c30 con 0x7f21a40b0b60 2024-09-13T08:46:45.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a1600640 1 -- 172.21.15.44:0/781089073 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7f21980c0060 con 0x7f21a40b0b60 2024-09-13T08:46:45.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a1600640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2184073fa0 0x7f2184076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.132+0000 7f21a1600640 1 -- 172.21.15.44:0/781089073 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f219812f900 con 0x7f21a40b0b60 2024-09-13T08:46:45.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.136+0000 7f21a9000640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2184073fa0 0x7f2184076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.136+0000 7f21a9000640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2184073fa0 0x7f2184076460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2190001130 tx=0x7f2190009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:45.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.136+0000 7f21a1600640 1 -- 172.21.15.44:0/781089073 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2198018b70 con 0x7f21a40b0b60 2024-09-13T08:46:45.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.340+0000 7f21ac400640 1 -- 172.21.15.44:0/781089073 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21a40cdca0 con 0x7f2184073fa0 2024-09-13T08:46:45.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.356+0000 7f21a1600640 1 -- 172.21.15.44:0/781089073 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f21a40cdca0 con 0x7f2184073fa0 2024-09-13T08:46:45.359 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:45.359 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:46:45.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:46:45.359 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1" 2024-09-13T08:46:45.360 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:46:45.360 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:46:45.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 -- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2184073fa0 msgr2=0x7f2184076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2184073fa0 0x7f2184076460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2190001130 tx=0x7f2190009040 comp rx=0 tx=0).stop 2024-09-13T08:46:45.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 -- 172.21.15.44:0/781089073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a40b0b60 msgr2=0x7f21a4149330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a40b0b60 0x7f21a4149330 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f219801bc20 tx=0x7f2198004620 comp rx=0 tx=0).stop 2024-09-13T08:46:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 -- 172.21.15.44:0/781089073 shutdown_connections 2024-09-13T08:46:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2184073fa0 0x7f2184076460 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21a415c1f0 0x7f21a40cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f21a415be10 0x7f21a40c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 --2- 172.21.15.44:0/781089073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a40b0b60 0x7f21a4149330 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 -- 172.21.15.44:0/781089073 >> 172.21.15.44:0/781089073 conn(0x7f21a40d1da0 msgr2=0x7f21a4072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 -- 172.21.15.44:0/781089073 shutdown_connections 2024-09-13T08:46:45.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.360+0000 7f21ac400640 1 -- 172.21.15.44:0/781089073 wait complete. 2024-09-13T08:46:45.382 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-13T08:46:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.536+0000 7f029aa00640 1 -- 172.21.15.44:0/1758568361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c004810 msgr2=0x7f027c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.536+0000 7f029aa00640 1 --2- 172.21.15.44:0/1758568361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c004810 0x7f027c004c90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0280009d10 tx=0x7f028001d3f0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.536+0000 7f029aa00640 1 -- 172.21.15.44:0/1758568361 shutdown_connections 2024-09-13T08:46:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.536+0000 7f029aa00640 1 --2- 172.21.15.44:0/1758568361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c004810 0x7f027c004c90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.536+0000 7f029aa00640 1 -- 172.21.15.44:0/1758568361 >> 172.21.15.44:0/1758568361 conn(0x7f02940d1cf0 msgr2=0x7f02940d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.536+0000 7f029aa00640 1 -- 172.21.15.44:0/1758568361 shutdown_connections 2024-09-13T08:46:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.536+0000 7f029aa00640 1 -- 172.21.15.44:0/1758568361 wait complete. 2024-09-13T08:46:45.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.536+0000 7f029aa00640 1 Processor -- start 2024-09-13T08:46:45.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.536+0000 7f029aa00640 1 -- start start 2024-09-13T08:46:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f029aa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f027c004810 0x7f029415f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f029aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f029415d9d0 0x7f0294151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f029aa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294159b60 0x7f0294159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f029aa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f029415fdc0 con 0x7f029415d9d0 2024-09-13T08:46:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f029aa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f029415fc40 con 0x7f0294159b60 2024-09-13T08:46:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f029aa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f029415ff40 con 0x7f027c004810 2024-09-13T08:46:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0298c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f029415d9d0 0x7f0294151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0298c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f029415d9d0 0x7f0294151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57104/0 (socket says 172.21.15.44:57104) 2024-09-13T08:46:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0298c00640 1 -- 172.21.15.44:0/986432194 learned_addr learned my addr 172.21.15.44:0/986432194 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0299600640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f027c004810 0x7f029415f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f029a000640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294159b60 0x7f0294159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0298c00640 1 -- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f027c004810 msgr2=0x7f029415f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0298c00640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f027c004810 0x7f029415f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0298c00640 1 -- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294159b60 msgr2=0x7f0294159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0298c00640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294159b60 0x7f0294159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0298c00640 1 -- 172.21.15.44:0/986432194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02941f8200 con 0x7f029415d9d0 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0298c00640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f029415d9d0 0x7f0294151200 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f028001d220 tx=0x7f0280009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0291600640 1 -- 172.21.15.44:0/986432194 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280003800 con 0x7f029415d9d0 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0291600640 1 -- 172.21.15.44:0/986432194 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02800039a0 con 0x7f029415d9d0 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0291600640 1 -- 172.21.15.44:0/986432194 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280003dd0 con 0x7f029415d9d0 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f029aa00640 1 -- 172.21.15.44:0/986432194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02941f84e0 con 0x7f029415d9d0 2024-09-13T08:46:45.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f029aa00640 1 -- 172.21.15.44:0/986432194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02941f8aa0 con 0x7f029415d9d0 2024-09-13T08:46:45.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f029aa00640 1 -- 172.21.15.44:0/986432194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02941604a0 con 0x7f029415d9d0 2024-09-13T08:46:45.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.540+0000 7f0291600640 1 -- 172.21.15.44:0/986432194 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7f02800c9440 con 0x7f029415d9d0 2024-09-13T08:46:45.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.544+0000 7f0291600640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0274074020 0x7f02740764e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.544+0000 7f0291600640 1 -- 172.21.15.44:0/986432194 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f028012e260 con 0x7f029415d9d0 2024-09-13T08:46:45.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.544+0000 7f0299600640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0274074020 0x7f02740764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.544+0000 7f0299600640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0274074020 0x7f02740764e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0288000c10 tx=0x7f0288006b40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.548+0000 7f0291600640 1 -- 172.21.15.44:0/986432194 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0280017100 con 0x7f029415d9d0 2024-09-13T08:46:45.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.760+0000 7f029aa00640 1 -- 172.21.15.44:0/986432194 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02941f7fb0 con 0x7f0274074020 2024-09-13T08:46:45.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f0291600640 1 -- 172.21.15.44:0/986432194 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f02941f7fb0 con 0x7f0274074020 2024-09-13T08:46:45.793 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 -- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0274074020 msgr2=0x7f02740764e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0274074020 0x7f02740764e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0288000c10 tx=0x7f0288006b40 comp rx=0 tx=0).stop 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 -- 172.21.15.44:0/986432194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f029415d9d0 msgr2=0x7f0294151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f029415d9d0 0x7f0294151200 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f028001d220 tx=0x7f0280009980 comp rx=0 tx=0).stop 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 -- 172.21.15.44:0/986432194 shutdown_connections 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0274074020 0x7f02740764e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294159b60 0x7f0294159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f029415d9d0 0x7f0294151200 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 --2- 172.21.15.44:0/986432194 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f027c004810 0x7f029415f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 -- 172.21.15.44:0/986432194 >> 172.21.15.44:0/986432194 conn(0x7f02940d1cf0 msgr2=0x7f02940b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 -- 172.21.15.44:0/986432194 shutdown_connections 2024-09-13T08:46:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.792+0000 7f029aa00640 1 -- 172.21.15.44:0/986432194 wait complete. 2024-09-13T08:46:45.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-13T08:46:45.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.964+0000 7fe3ccc00640 1 -- 172.21.15.44:0/594047335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c8161230 msgr2=0x7fe3c80d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.964+0000 7fe3ccc00640 1 --2- 172.21.15.44:0/594047335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c8161230 0x7fe3c80d4510 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8162a70 tx=0x7fe3b801c800 comp rx=0 tx=0).stop 2024-09-13T08:46:45.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.964+0000 7fe3ccc00640 1 -- 172.21.15.44:0/594047335 shutdown_connections 2024-09-13T08:46:45.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.964+0000 7fe3ccc00640 1 --2- 172.21.15.44:0/594047335 >> v2:172.21.15.129:3300/0 conn(0x7fe3c80d2950 0x7fe3c80d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.964+0000 7fe3ccc00640 1 --2- 172.21.15.44:0/594047335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c8161230 0x7fe3c80d4510 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.964+0000 7fe3ccc00640 1 -- 172.21.15.44:0/594047335 >> 172.21.15.44:0/594047335 conn(0x7fe3c809d0c0 msgr2=0x7fe3c80d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:45.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.964+0000 7fe3ccc00640 1 -- 172.21.15.44:0/594047335 shutdown_connections 2024-09-13T08:46:45.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.964+0000 7fe3ccc00640 1 -- 172.21.15.44:0/594047335 wait complete. 2024-09-13T08:46:45.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.964+0000 7fe3ccc00640 1 Processor -- start 2024-09-13T08:46:45.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.964+0000 7fe3ccc00640 1 -- start start 2024-09-13T08:46:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3ccc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c80b0a90 0x7fe3c8159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3ccc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3c80d2950 0x7fe3c814dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3ccc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c8161230 0x7fe3c80b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3ccc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c816af60 con 0x7fe3c8161230 2024-09-13T08:46:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3ccc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c80d4c40 con 0x7fe3c80b0a90 2024-09-13T08:46:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3ccc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c816b0e0 con 0x7fe3c80d2950 2024-09-13T08:46:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c8161230 0x7fe3c80b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c8161230 0x7fe3c80b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57108/0 (socket says 172.21.15.44:57108) 2024-09-13T08:46:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3c7e00640 1 -- 172.21.15.44:0/823133344 learned_addr learned my addr 172.21.15.44:0/823133344 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3bfe00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3c80d2950 0x7fe3c814dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3c7400640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c80b0a90 0x7fe3c8159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3c7e00640 1 -- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3c80d2950 msgr2=0x7fe3c814dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3c7e00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3c80d2950 0x7fe3c814dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3c7e00640 1 -- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c80b0a90 msgr2=0x7fe3c8159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3c7e00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c80b0a90 0x7fe3c8159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3c7e00640 1 -- 172.21.15.44:0/823133344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c8158540 con 0x7fe3c8161230 2024-09-13T08:46:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3bfe00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3c80d2950 0x7fe3c814dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:46:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3c7e00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c8161230 0x7fe3c80b3e50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000eb50 tx=0x7fe3c000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:45.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3bf400640 1 -- 172.21.15.44:0/823133344 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0004d30 con 0x7fe3c8161230 2024-09-13T08:46:45.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3bf400640 1 -- 172.21.15.44:0/823133344 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3c000ee50 con 0x7fe3c8161230 2024-09-13T08:46:45.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3ccc00640 1 -- 172.21.15.44:0/823133344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c8158820 con 0x7fe3c8161230 2024-09-13T08:46:45.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3bf400640 1 -- 172.21.15.44:0/823133344 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0005280 con 0x7fe3c8161230 2024-09-13T08:46:45.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3ccc00640 1 -- 172.21.15.44:0/823133344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c8158de0 con 0x7fe3c8161230 2024-09-13T08:46:45.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3ccc00640 1 -- 172.21.15.44:0/823133344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c808d540 con 0x7fe3c8161230 2024-09-13T08:46:45.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.968+0000 7fe3bf400640 1 -- 172.21.15.44:0/823133344 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7fe3c0005420 con 0x7fe3c8161230 2024-09-13T08:46:45.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.972+0000 7fe3bf400640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe3a4073fa0 0x7fe3a4076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:45.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.972+0000 7fe3bf400640 1 -- 172.21.15.44:0/823133344 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fe3c00939a0 con 0x7fe3c8161230 2024-09-13T08:46:45.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.972+0000 7fe3c7400640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe3a4073fa0 0x7fe3a4076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:45.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.972+0000 7fe3c7400640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe3a4073fa0 0x7fe3a4076460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80af000 tx=0x7fe3b80a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:45.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:45.976+0000 7fe3bf400640 1 -- 172.21.15.44:0/823133344 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe3c0060300 con 0x7fe3c8161230 2024-09-13T08:46:46.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.176+0000 7fe3ccc00640 1 -- 172.21.15.44:0/823133344 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe3c8069e30 con 0x7fe3a4073fa0 2024-09-13T08:46:46.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.196+0000 7fe3bf400640 1 -- 172.21.15.44:0/823133344 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe3b80b3040 con 0x7fe3a4073fa0 2024-09-13T08:46:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 -- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe3a4073fa0 msgr2=0x7fe3a4076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe3a4073fa0 0x7fe3a4076460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80af000 tx=0x7fe3b80a7040 comp rx=0 tx=0).stop 2024-09-13T08:46:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 -- 172.21.15.44:0/823133344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c8161230 msgr2=0x7fe3c80b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c8161230 0x7fe3c80b3e50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000eb50 tx=0x7fe3c000c6a0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 -- 172.21.15.44:0/823133344 shutdown_connections 2024-09-13T08:46:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe3a4073fa0 0x7fe3a4076460 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c8161230 0x7fe3c80b3e50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe3c80d2950 0x7fe3c814dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 --2- 172.21.15.44:0/823133344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c80b0a90 0x7fe3c8159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 -- 172.21.15.44:0/823133344 >> 172.21.15.44:0/823133344 conn(0x7fe3c809d0c0 msgr2=0x7fe3c80b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 -- 172.21.15.44:0/823133344 shutdown_connections 2024-09-13T08:46:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.200+0000 7fe3ccc00640 1 -- 172.21.15.44:0/823133344 wait complete. 2024-09-13T08:46:46.224 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone75132 2024-09-13T08:46:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.376+0000 7f2449200640 1 -- 172.21.15.44:0/2828311028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f244415ba80 msgr2=0x7f24440742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.376+0000 7f2449200640 1 --2- 172.21.15.44:0/2828311028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f244415ba80 0x7f24440742d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2434009840 tx=0x7f243401c740 comp rx=0 tx=0).stop 2024-09-13T08:46:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.376+0000 7f2449200640 1 -- 172.21.15.44:0/2828311028 shutdown_connections 2024-09-13T08:46:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.376+0000 7f2449200640 1 --2- 172.21.15.44:0/2828311028 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24440aff80 0x7f24440c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.376+0000 7f2449200640 1 --2- 172.21.15.44:0/2828311028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f244415ba80 0x7f24440742d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.376+0000 7f2449200640 1 -- 172.21.15.44:0/2828311028 >> 172.21.15.44:0/2828311028 conn(0x7f24440b2900 msgr2=0x7f24440b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.376+0000 7f2449200640 1 -- 172.21.15.44:0/2828311028 shutdown_connections 2024-09-13T08:46:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.376+0000 7f2449200640 1 -- 172.21.15.44:0/2828311028 wait complete. 2024-09-13T08:46:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.376+0000 7f2449200640 1 Processor -- start 2024-09-13T08:46:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.376+0000 7f2449200640 1 -- start start 2024-09-13T08:46:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2449200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24440aff80 0x7f2444151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2449200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f244415ba80 0x7f24441f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2449200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f244415e570 0x7f24441f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2449200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24441691a0 con 0x7f244415e570 2024-09-13T08:46:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2449200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2444169020 con 0x7f244415ba80 2024-09-13T08:46:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2449200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2444169320 con 0x7f24440aff80 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2442800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f244415e570 0x7f24441f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2442800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f244415e570 0x7f24441f6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57126/0 (socket says 172.21.15.44:57126) 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2442800640 1 -- 172.21.15.44:0/4162733485 learned_addr learned my addr 172.21.15.44:0/4162733485 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2441400640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f244415ba80 0x7f24441f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2441e00640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24440aff80 0x7f2444151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2442800640 1 -- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24440aff80 msgr2=0x7f2444151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2442800640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24440aff80 0x7f2444151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2442800640 1 -- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f244415ba80 msgr2=0x7f24441f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2442800640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f244415ba80 0x7f24441f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2442800640 1 -- 172.21.15.44:0/4162733485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24441f9bc0 con 0x7f244415e570 2024-09-13T08:46:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2442800640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f244415e570 0x7f24441f6690 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f243c00eb20 tx=0x7f243c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f243a000640 1 -- 172.21.15.44:0/4162733485 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c004b30 con 0x7f244415e570 2024-09-13T08:46:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f243a000640 1 -- 172.21.15.44:0/4162733485 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f243c004cf0 con 0x7f244415e570 2024-09-13T08:46:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f243a000640 1 -- 172.21.15.44:0/4162733485 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c005280 con 0x7f244415e570 2024-09-13T08:46:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2449200640 1 -- 172.21.15.44:0/4162733485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24441f9ea0 con 0x7f244415e570 2024-09-13T08:46:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2449200640 1 -- 172.21.15.44:0/4162733485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24441f5e90 con 0x7f244415e570 2024-09-13T08:46:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.380+0000 7f2449200640 1 -- 172.21.15.44:0/4162733485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24440c5b70 con 0x7f244415e570 2024-09-13T08:46:46.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.384+0000 7f243a000640 1 -- 172.21.15.44:0/4162733485 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7f243c005420 con 0x7f244415e570 2024-09-13T08:46:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.384+0000 7f243a000640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2424073fa0 0x7f2424076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.384+0000 7f243a000640 1 -- 172.21.15.44:0/4162733485 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f243c093f00 con 0x7f244415e570 2024-09-13T08:46:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.384+0000 7f2441e00640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2424073fa0 0x7f2424076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:46.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.384+0000 7f2441e00640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2424073fa0 0x7f2424076460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f24340ae000 tx=0x7f243401af80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:46.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.384+0000 7f243a000640 1 -- 172.21.15.44:0/4162733485 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f243c0608a0 con 0x7f244415e570 2024-09-13T08:46:46.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.588+0000 7f2449200640 1 -- 172.21.15.44:0/4162733485 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone75132", "target": ["mon-mgr", ""]}) -- 0x7f244409ee80 con 0x7f2424073fa0 2024-09-13T08:46:46.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.612+0000 7f243a000640 1 -- 172.21.15.44:0/4162733485 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f244409ee80 con 0x7f2424073fa0 2024-09-13T08:46:46.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 -- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2424073fa0 msgr2=0x7f2424076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:46.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2424073fa0 0x7f2424076460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f24340ae000 tx=0x7f243401af80 comp rx=0 tx=0).stop 2024-09-13T08:46:46.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 -- 172.21.15.44:0/4162733485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f244415e570 msgr2=0x7f24441f6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:46.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f244415e570 0x7f24441f6690 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f243c00eb20 tx=0x7f243c00c2d0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 -- 172.21.15.44:0/4162733485 shutdown_connections 2024-09-13T08:46:46.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2424073fa0 0x7f2424076460 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f244415e570 0x7f24441f6690 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f244415ba80 0x7f24441f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 --2- 172.21.15.44:0/4162733485 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f24440aff80 0x7f2444151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 -- 172.21.15.44:0/4162733485 >> 172.21.15.44:0/4162733485 conn(0x7f24440b2900 msgr2=0x7f24441540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:46.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 -- 172.21.15.44:0/4162733485 shutdown_connections 2024-09-13T08:46:46.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.616+0000 7f2449200640 1 -- 172.21.15.44:0/4162733485 wait complete. 2024-09-13T08:46:46.637 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41056 2024-09-13T08:46:46.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.836+0000 7feb4e600640 1 -- 172.21.15.44:0/2726748762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb4815de40 msgr2=0x7feb480b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:46.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.836+0000 7feb4e600640 1 --2- 172.21.15.44:0/2726748762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb4815de40 0x7feb480b35a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feb4808d8f0 tx=0x7feb3801c890 comp rx=0 tx=0).stop 2024-09-13T08:46:46.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.836+0000 7feb4e600640 1 -- 172.21.15.44:0/2726748762 shutdown_connections 2024-09-13T08:46:46.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.836+0000 7feb4e600640 1 --2- 172.21.15.44:0/2726748762 >> v2:172.21.15.44:3300/0 conn(0x7feb480d2540 0x7feb480d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.836+0000 7feb4e600640 1 --2- 172.21.15.44:0/2726748762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb4815de40 0x7feb480b35a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.836+0000 7feb4e600640 1 -- 172.21.15.44:0/2726748762 >> 172.21.15.44:0/2726748762 conn(0x7feb480d3f90 msgr2=0x7feb480d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:46.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.836+0000 7feb4e600640 1 -- 172.21.15.44:0/2726748762 shutdown_connections 2024-09-13T08:46:46.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.836+0000 7feb4e600640 1 -- 172.21.15.44:0/2726748762 wait complete. 2024-09-13T08:46:46.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb4e600640 1 Processor -- start 2024-09-13T08:46:46.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb4e600640 1 -- start start 2024-09-13T08:46:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb4e600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feb480d2540 0x7feb482088d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb4e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4815de40 0x7feb482042a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb4e600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb4815e220 0x7feb480b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb4e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feb4816b0e0 con 0x7feb4815de40 2024-09-13T08:46:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb4e600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feb4816af60 con 0x7feb4815e220 2024-09-13T08:46:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb4e600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7feb4816b260 con 0x7feb480d2540 2024-09-13T08:46:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4815de40 0x7feb482042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4815de40 0x7feb482042a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57146/0 (socket says 172.21.15.44:57146) 2024-09-13T08:46:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb3fe00640 1 -- 172.21.15.44:0/2887737016 learned_addr learned my addr 172.21.15.44:0/2887737016 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb47400640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feb480d2540 0x7feb482088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb47e00640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb4815e220 0x7feb480b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb3fe00640 1 -- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feb480d2540 msgr2=0x7feb482088d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb3fe00640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feb480d2540 0x7feb482088d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb3fe00640 1 -- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb4815e220 msgr2=0x7feb480b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb3fe00640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb4815e220 0x7feb480b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.840+0000 7feb3fe00640 1 -- 172.21.15.44:0/2887737016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb480b8970 con 0x7feb4815de40 2024-09-13T08:46:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.844+0000 7feb3fe00640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4815de40 0x7feb482042a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feb3000c480 tx=0x7feb3000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.844+0000 7feb3f400640 1 -- 172.21.15.44:0/2887737016 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300052d0 con 0x7feb4815de40 2024-09-13T08:46:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.844+0000 7feb3f400640 1 -- 172.21.15.44:0/2887737016 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb30005470 con 0x7feb4815de40 2024-09-13T08:46:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.844+0000 7feb3f400640 1 -- 172.21.15.44:0/2887737016 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30012890 con 0x7feb4815de40 2024-09-13T08:46:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.844+0000 7feb4e600640 1 -- 172.21.15.44:0/2887737016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb480b8c50 con 0x7feb4815de40 2024-09-13T08:46:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.844+0000 7feb4e600640 1 -- 172.21.15.44:0/2887737016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb480b9210 con 0x7feb4815de40 2024-09-13T08:46:46.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.844+0000 7feb4e600640 1 -- 172.21.15.44:0/2887737016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb480d29b0 con 0x7feb4815de40 2024-09-13T08:46:46.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.848+0000 7feb3f400640 1 -- 172.21.15.44:0/2887737016 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 34) ==== 98610+0+0 (secure 0 0 0) 0x7feb30003600 con 0x7feb4815de40 2024-09-13T08:46:46.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.848+0000 7feb3f400640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feb24074070 0x7feb24076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:46.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.848+0000 7feb3f400640 1 -- 172.21.15.44:0/2887737016 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7feb300928d0 con 0x7feb4815de40 2024-09-13T08:46:46.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.852+0000 7feb47400640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feb24074070 0x7feb24076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:46.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.852+0000 7feb47400640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feb24074070 0x7feb24076530 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feb380af000 tx=0x7feb3801b0f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:46.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:46.860+0000 7feb3f400640 1 -- 172.21.15.44:0/2887737016 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7feb3005f230 con 0x7feb4815de40 2024-09-13T08:46:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.124+0000 7feb3f400640 1 -- 172.21.15.44:0/2887737016 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7feb3005ada0 con 0x7feb4815de40 2024-09-13T08:46:47.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.148+0000 7feb4e600640 1 -- 172.21.15.44:0/2887737016 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41056", "target": ["mon-mgr", ""]}) -- 0x7feb480d0f70 con 0x7feb24074070 2024-09-13T08:46:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb3f400640 1 -- 172.21.15.44:0/2887737016 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feb38004020 con 0x7feb24074070 2024-09-13T08:46:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 -- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feb24074070 msgr2=0x7feb24076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feb24074070 0x7feb24076530 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feb380af000 tx=0x7feb3801b0f0 comp rx=0 tx=0).stop 2024-09-13T08:46:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 -- 172.21.15.44:0/2887737016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4815de40 msgr2=0x7feb482042a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4815de40 0x7feb482042a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feb3000c480 tx=0x7feb3000c940 comp rx=0 tx=0).stop 2024-09-13T08:46:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 -- 172.21.15.44:0/2887737016 shutdown_connections 2024-09-13T08:46:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feb24074070 0x7feb24076530 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb4815e220 0x7feb480b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4815de40 0x7feb482042a0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 --2- 172.21.15.44:0/2887737016 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feb480d2540 0x7feb482088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 -- 172.21.15.44:0/2887737016 >> 172.21.15.44:0/2887737016 conn(0x7feb480d3f90 msgr2=0x7feb480b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 -- 172.21.15.44:0/2887737016 shutdown_connections 2024-09-13T08:46:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.504+0000 7feb4e600640 1 -- 172.21.15.44:0/2887737016 wait complete. 2024-09-13T08:46:47.530 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone3811 2024-09-13T08:46:47.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.684+0000 7f92b4c00640 1 -- 172.21.15.44:0/2156030066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b015eff0 msgr2=0x7f92b015f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:47.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.684+0000 7f92b4c00640 1 --2- 172.21.15.44:0/2156030066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b015eff0 0x7f92b015f410 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f92b006dc00 tx=0x7f92a801cd50 comp rx=0 tx=0).stop 2024-09-13T08:46:47.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.684+0000 7f92b4c00640 1 -- 172.21.15.44:0/2156030066 shutdown_connections 2024-09-13T08:46:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.684+0000 7f92b4c00640 1 --2- 172.21.15.44:0/2156030066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b015eff0 0x7f92b015f410 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.684+0000 7f92b4c00640 1 --2- 172.21.15.44:0/2156030066 >> v2:172.21.15.129:3300/0 conn(0x7f92b00d09f0 0x7f92b00d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.684+0000 7f92b4c00640 1 -- 172.21.15.44:0/2156030066 >> 172.21.15.44:0/2156030066 conn(0x7f92b00d1da0 msgr2=0x7f92b00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.684+0000 7f92b4c00640 1 -- 172.21.15.44:0/2156030066 shutdown_connections 2024-09-13T08:46:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.684+0000 7f92b4c00640 1 -- 172.21.15.44:0/2156030066 wait complete. 2024-09-13T08:46:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.684+0000 7f92b4c00640 1 Processor -- start 2024-09-13T08:46:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.684+0000 7f92b4c00640 1 -- start start 2024-09-13T08:46:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92b4c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92b00cff00 0x7f92b014fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92b4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b015eff0 0x7f92b0159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92b4c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b01f6250 0x7f92b0153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92b4c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f92b00b0bd0 con 0x7f92b015eff0 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92b4c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f92b00b0a50 con 0x7f92b01f6250 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92b4c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f92b0150300 con 0x7f92b00cff00 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b015eff0 0x7f92b0159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b015eff0 0x7f92b0159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57166/0 (socket says 172.21.15.44:57166) 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7e00640 1 -- 172.21.15.44:0/1812032644 learned_addr learned my addr 172.21.15.44:0/1812032644 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92afe00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b01f6250 0x7f92b0153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92af400640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92b00cff00 0x7f92b014fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7e00640 1 -- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92b00cff00 msgr2=0x7f92b014fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7e00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92b00cff00 0x7f92b014fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7e00640 1 -- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b01f6250 msgr2=0x7f92b0153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:47.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7e00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b01f6250 0x7f92b0153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7e00640 1 -- 172.21.15.44:0/1812032644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b0157740 con 0x7f92b015eff0 2024-09-13T08:46:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92af400640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92b00cff00 0x7f92b014fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:46:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92afe00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b01f6250 0x7f92b0153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:46:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7e00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b015eff0 0x7f92b0159650 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f929800e9f0 tx=0x7f929800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7400640 1 -- 172.21.15.44:0/1812032644 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929800cd80 con 0x7f92b015eff0 2024-09-13T08:46:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7400640 1 -- 172.21.15.44:0/1812032644 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9298010430 con 0x7f92b015eff0 2024-09-13T08:46:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92a7400640 1 -- 172.21.15.44:0/1812032644 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298010860 con 0x7f92b015eff0 2024-09-13T08:46:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92b4c00640 1 -- 172.21.15.44:0/1812032644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b0157a10 con 0x7f92b015eff0 2024-09-13T08:46:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92b4c00640 1 -- 172.21.15.44:0/1812032644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b01505a0 con 0x7f92b015eff0 2024-09-13T08:46:47.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.688+0000 7f92b4c00640 1 -- 172.21.15.44:0/1812032644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b00d28c0 con 0x7f92b015eff0 2024-09-13T08:46:47.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.692+0000 7f92a7400640 1 -- 172.21.15.44:0/1812032644 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f9298010a00 con 0x7f92b015eff0 2024-09-13T08:46:47.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.692+0000 7f92a7400640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f928c074060 0x7f928c076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.692+0000 7f92a7400640 1 -- 172.21.15.44:0/1812032644 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f9298093770 con 0x7f92b015eff0 2024-09-13T08:46:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.692+0000 7f92af400640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f928c074060 0x7f928c076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.692+0000 7f92af400640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f928c074060 0x7f928c076520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f92b00cef90 tx=0x7f92a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:47.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:47.696+0000 7f92a7400640 1 -- 172.21.15.44:0/1812032644 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f929805fb30 con 0x7f92b015eff0 2024-09-13T08:46:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:48.084+0000 7f92b4c00640 1 -- 172.21.15.44:0/1812032644 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone3811", "target": ["mon-mgr", ""]}) -- 0x7f92b0069e60 con 0x7f928c074060 2024-09-13T08:46:49.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92a7400640 1 -- 172.21.15.44:0/1812032644 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f92a0001ca0 con 0x7f928c074060 2024-09-13T08:46:49.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 -- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f928c074060 msgr2=0x7f928c076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:49.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f928c074060 0x7f928c076520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f92b00cef90 tx=0x7f92a0009040 comp rx=0 tx=0).stop 2024-09-13T08:46:49.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 -- 172.21.15.44:0/1812032644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b015eff0 msgr2=0x7f92b0159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:49.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b015eff0 0x7f92b0159650 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f929800e9f0 tx=0x7f929800eeb0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 -- 172.21.15.44:0/1812032644 shutdown_connections 2024-09-13T08:46:49.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f928c074060 0x7f928c076520 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b01f6250 0x7f92b0153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b015eff0 0x7f92b0159650 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 --2- 172.21.15.44:0/1812032644 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92b00cff00 0x7f92b014fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 -- 172.21.15.44:0/1812032644 >> 172.21.15.44:0/1812032644 conn(0x7f92b00d1da0 msgr2=0x7f92b0070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:49.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 -- 172.21.15.44:0/1812032644 shutdown_connections 2024-09-13T08:46:49.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.224+0000 7f92b4c00640 1 -- 172.21.15.44:0/1812032644 wait complete. 2024-09-13T08:46:49.249 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone57364 2024-09-13T08:46:49.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.400+0000 7f2db1400640 1 -- 172.21.15.44:0/1754307517 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2dac0b0b60 msgr2=0x7f2dac0ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.400+0000 7f2db1400640 1 --2- 172.21.15.44:0/1754307517 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2dac0b0b60 0x7f2dac0ccbd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009bc0 tx=0x7f2da401cba0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.400+0000 7f2db1400640 1 -- 172.21.15.44:0/1754307517 shutdown_connections 2024-09-13T08:46:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.400+0000 7f2db1400640 1 --2- 172.21.15.44:0/1754307517 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2dac0b0b60 0x7f2dac0ccbd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.400+0000 7f2db1400640 1 --2- 172.21.15.44:0/1754307517 >> v2:172.21.15.129:3300/0 conn(0x7f2dac15c1f0 0x7f2dac0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.400+0000 7f2db1400640 1 -- 172.21.15.44:0/1754307517 >> 172.21.15.44:0/1754307517 conn(0x7f2dac0d1da0 msgr2=0x7f2dac0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.400+0000 7f2db1400640 1 -- 172.21.15.44:0/1754307517 shutdown_connections 2024-09-13T08:46:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.400+0000 7f2db1400640 1 -- 172.21.15.44:0/1754307517 wait complete. 2024-09-13T08:46:49.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.400+0000 7f2db1400640 1 Processor -- start 2024-09-13T08:46:49.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.400+0000 7f2db1400640 1 -- start start 2024-09-13T08:46:49.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2db1400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2dac0b0b60 0x7f2dac14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:49.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2db1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dac15be10 0x7f2dac149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:49.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2db1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dac15c1f0 0x7f2dac0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:49.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2db1400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac169040 con 0x7f2dac15be10 2024-09-13T08:46:49.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2db1400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac168ec0 con 0x7f2dac15c1f0 2024-09-13T08:46:49.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2db1400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2dac1691c0 con 0x7f2dac0b0b60 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dac15be10 0x7f2dac149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dac15be10 0x7f2dac149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57168/0 (socket says 172.21.15.44:57168) 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da3e00640 1 -- 172.21.15.44:0/2840512006 learned_addr learned my addr 172.21.15.44:0/2840512006 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2daa800640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2dac0b0b60 0x7f2dac14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2dab200640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dac15c1f0 0x7f2dac0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da3e00640 1 -- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2dac0b0b60 msgr2=0x7f2dac14acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da3e00640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2dac0b0b60 0x7f2dac14acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da3e00640 1 -- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dac15c1f0 msgr2=0x7f2dac0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da3e00640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dac15c1f0 0x7f2dac0b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da3e00640 1 -- 172.21.15.44:0/2840512006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac0af470 con 0x7f2dac15be10 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2dab200640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dac15c1f0 0x7f2dac0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2daa800640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2dac0b0b60 0x7f2dac14acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da3e00640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dac15be10 0x7f2dac149450 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400e970 tx=0x7f2d9400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:49.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da2a00640 1 -- 172.21.15.44:0/2840512006 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400cd80 con 0x7f2dac15be10 2024-09-13T08:46:49.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da2a00640 1 -- 172.21.15.44:0/2840512006 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d94004020 con 0x7f2dac15be10 2024-09-13T08:46:49.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2da2a00640 1 -- 172.21.15.44:0/2840512006 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94004470 con 0x7f2dac15be10 2024-09-13T08:46:49.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2db1400640 1 -- 172.21.15.44:0/2840512006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dac0c6940 con 0x7f2dac15be10 2024-09-13T08:46:49.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2db1400640 1 -- 172.21.15.44:0/2840512006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dac0c6380 con 0x7f2dac15be10 2024-09-13T08:46:49.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.404+0000 7f2db1400640 1 -- 172.21.15.44:0/2840512006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dac0ccbd0 con 0x7f2dac15be10 2024-09-13T08:46:49.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.408+0000 7f2da2a00640 1 -- 172.21.15.44:0/2840512006 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f2d94003600 con 0x7f2dac15be10 2024-09-13T08:46:49.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.408+0000 7f2da2a00640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2d88074060 0x7f2d88076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:49.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.408+0000 7f2da2a00640 1 -- 172.21.15.44:0/2840512006 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7f2d94093700 con 0x7f2dac15be10 2024-09-13T08:46:49.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.408+0000 7f2daa800640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2d88074060 0x7f2d88076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:49.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.408+0000 7f2daa800640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2d88074060 0x7f2d88076520 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0af6b0 tx=0x7f2d9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:49.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.412+0000 7f2da2a00640 1 -- 172.21.15.44:0/2840512006 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2d9409d080 con 0x7f2dac15be10 2024-09-13T08:46:49.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.616+0000 7f2db1400640 1 -- 172.21.15.44:0/2840512006 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone57364", "target": ["mon-mgr", ""]}) -- 0x7f2dac070350 con 0x7f2d88074060 2024-09-13T08:46:49.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.636+0000 7f2da2a00640 1 -- 172.21.15.44:0/2840512006 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2dac070350 con 0x7f2d88074060 2024-09-13T08:46:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 -- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2d88074060 msgr2=0x7f2d88076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2d88074060 0x7f2d88076520 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0af6b0 tx=0x7f2d9c009040 comp rx=0 tx=0).stop 2024-09-13T08:46:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 -- 172.21.15.44:0/2840512006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dac15be10 msgr2=0x7f2dac149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dac15be10 0x7f2dac149450 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400e970 tx=0x7f2d9400ee30 comp rx=0 tx=0).stop 2024-09-13T08:46:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 -- 172.21.15.44:0/2840512006 shutdown_connections 2024-09-13T08:46:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2d88074060 0x7f2d88076520 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dac15c1f0 0x7f2dac0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dac15be10 0x7f2dac149450 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 --2- 172.21.15.44:0/2840512006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2dac0b0b60 0x7f2dac14acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 -- 172.21.15.44:0/2840512006 >> 172.21.15.44:0/2840512006 conn(0x7f2dac0d1da0 msgr2=0x7f2dac0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:49.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 -- 172.21.15.44:0/2840512006 shutdown_connections 2024-09-13T08:46:49.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.640+0000 7f2db1400640 1 -- 172.21.15.44:0/2840512006 wait complete. 2024-09-13T08:46:49.667 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone53093 2024-09-13T08:46:49.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.876+0000 7fb4bec00640 1 -- 172.21.15.44:0/4082029196 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4b80b2fe0 msgr2=0x7fb4b8159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:49.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.876+0000 7fb4bec00640 1 --2- 172.21.15.44:0/4082029196 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4b80b2fe0 0x7fb4b8159d30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009870 tx=0x7fb4ac01c6f0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.876+0000 7fb4bec00640 1 -- 172.21.15.44:0/4082029196 shutdown_connections 2024-09-13T08:46:49.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.876+0000 7fb4bec00640 1 --2- 172.21.15.44:0/4082029196 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4b80b2fe0 0x7fb4b8159d30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.876+0000 7fb4bec00640 1 -- 172.21.15.44:0/4082029196 >> 172.21.15.44:0/4082029196 conn(0x7fb4b806ec90 msgr2=0x7fb4b806f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:49.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.876+0000 7fb4bec00640 1 -- 172.21.15.44:0/4082029196 shutdown_connections 2024-09-13T08:46:49.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.876+0000 7fb4bec00640 1 -- 172.21.15.44:0/4082029196 wait complete. 2024-09-13T08:46:49.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.876+0000 7fb4bec00640 1 Processor -- start 2024-09-13T08:46:49.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bec00640 1 -- start start 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bec00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4b806aba0 0x7fb4b80cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4b80b2fe0 0x7fb4b8132960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bec00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4b80cc8e0 0x7fb4b8151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b80742d0 con 0x7fb4b80b2fe0 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bec00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b80ccfa0 con 0x7fb4b80cc8e0 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bec00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b80cfd40 con 0x7fb4b806aba0 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4b80b2fe0 0x7fb4b8132960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4b80b2fe0 0x7fb4b8132960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57184/0 (socket says 172.21.15.44:57184) 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bce00640 1 -- 172.21.15.44:0/1210912750 learned_addr learned my addr 172.21.15.44:0/1210912750 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bd800640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4b806aba0 0x7fb4b80cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4be200640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4b80cc8e0 0x7fb4b8151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bce00640 1 -- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4b806aba0 msgr2=0x7fb4b80cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:49.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bce00640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4b806aba0 0x7fb4b80cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bce00640 1 -- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4b80cc8e0 msgr2=0x7fb4b8151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:49.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bce00640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4b80cc8e0 0x7fb4b8151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:49.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bce00640 1 -- 172.21.15.44:0/1210912750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b8153be0 con 0x7fb4b80b2fe0 2024-09-13T08:46:49.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bd800640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4b806aba0 0x7fb4b80cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:46:49.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bce00640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4b80b2fe0 0x7fb4b8132960 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400faa0 tx=0x7fb4a400fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:49.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4b5600640 1 -- 172.21.15.44:0/1210912750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a401a070 con 0x7fb4b80b2fe0 2024-09-13T08:46:49.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4b5600640 1 -- 172.21.15.44:0/1210912750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4a4008570 con 0x7fb4b80b2fe0 2024-09-13T08:46:49.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bec00640 1 -- 172.21.15.44:0/1210912750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b8153ec0 con 0x7fb4b80b2fe0 2024-09-13T08:46:49.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4b5600640 1 -- 172.21.15.44:0/1210912750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400d970 con 0x7fb4b80b2fe0 2024-09-13T08:46:49.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bec00640 1 -- 172.21.15.44:0/1210912750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b8154480 con 0x7fb4b80b2fe0 2024-09-13T08:46:49.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.880+0000 7fb4bec00640 1 -- 172.21.15.44:0/1210912750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b8135930 con 0x7fb4b80b2fe0 2024-09-13T08:46:49.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.884+0000 7fb4b5600640 1 -- 172.21.15.44:0/1210912750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7fb4a401d070 con 0x7fb4b80b2fe0 2024-09-13T08:46:49.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.884+0000 7fb4b5600640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb498074030 0x7fb4980764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:49.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.884+0000 7fb4b5600640 1 -- 172.21.15.44:0/1210912750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5852+0+0 (secure 0 0 0) 0x7fb4a4095580 con 0x7fb4b80b2fe0 2024-09-13T08:46:49.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.884+0000 7fb4bd800640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb498074030 0x7fb4980764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.884+0000 7fb4bd800640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb498074030 0x7fb4980764f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0af000 tx=0x7fb4ac01af30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:49.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:49.888+0000 7fb4b5600640 1 -- 172.21.15.44:0/1210912750 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb4a4061880 con 0x7fb4b80b2fe0 2024-09-13T08:46:50.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.088+0000 7fb4bec00640 1 -- 172.21.15.44:0/1210912750 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone53093", "target": ["mon-mgr", ""]}) -- 0x7fb4b8159d30 con 0x7fb498074030 2024-09-13T08:46:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.108+0000 7fb4b5600640 1 -- 172.21.15.44:0/1210912750 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb4b8159d30 con 0x7fb498074030 2024-09-13T08:46:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 -- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb498074030 msgr2=0x7fb4980764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb498074030 0x7fb4980764f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0af000 tx=0x7fb4ac01af30 comp rx=0 tx=0).stop 2024-09-13T08:46:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 -- 172.21.15.44:0/1210912750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4b80b2fe0 msgr2=0x7fb4b8132960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4b80b2fe0 0x7fb4b8132960 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400faa0 tx=0x7fb4a400fea0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 -- 172.21.15.44:0/1210912750 shutdown_connections 2024-09-13T08:46:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb498074030 0x7fb4980764f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4b80cc8e0 0x7fb4b8151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4b80b2fe0 0x7fb4b8132960 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 --2- 172.21.15.44:0/1210912750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4b806aba0 0x7fb4b80cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 -- 172.21.15.44:0/1210912750 >> 172.21.15.44:0/1210912750 conn(0x7fb4b806ec90 msgr2=0x7fb4b81559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:50.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 -- 172.21.15.44:0/1210912750 shutdown_connections 2024-09-13T08:46:50.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.112+0000 7fb4bec00640 1 -- 172.21.15.44:0/1210912750 wait complete. 2024-09-13T08:46:50.147 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone21695 2024-09-13T08:46:50.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.304+0000 7fb99fe00640 1 -- 172.21.15.44:0/2279407987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9a00aff60 msgr2=0x7fb9a00b03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:50.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.304+0000 7fb99fe00640 1 --2- 172.21.15.44:0/2279407987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9a00aff60 0x7fb9a00b03b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009840 tx=0x7fb98c01c680 comp rx=0 tx=0).stop 2024-09-13T08:46:50.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.304+0000 7fb99fe00640 1 -- 172.21.15.44:0/2279407987 shutdown_connections 2024-09-13T08:46:50.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.304+0000 7fb99fe00640 1 --2- 172.21.15.44:0/2279407987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9a00aff60 0x7fb9a00b03b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.304+0000 7fb99fe00640 1 -- 172.21.15.44:0/2279407987 >> 172.21.15.44:0/2279407987 conn(0x7fb9a00d1da0 msgr2=0x7fb9a00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:50.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.304+0000 7fb99fe00640 1 -- 172.21.15.44:0/2279407987 shutdown_connections 2024-09-13T08:46:50.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.304+0000 7fb99fe00640 1 -- 172.21.15.44:0/2279407987 wait complete. 2024-09-13T08:46:50.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 Processor -- start 2024-09-13T08:46:50.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 -- start start 2024-09-13T08:46:50.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9a006df90 0x7fb9a01521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:50.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9a006e370 0x7fb9a00cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:50.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9a00aff60 0x7fb9a00cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a016bd20 con 0x7fb9a00aff60 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a016bba0 con 0x7fb9a006e370 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a016bea0 con 0x7fb9a006df90 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9a00aff60 0x7fb9a00cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9a00aff60 0x7fb9a00cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57190/0 (socket says 172.21.15.44:57190) 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99f400640 1 -- 172.21.15.44:0/1975376723 learned_addr learned my addr 172.21.15.44:0/1975376723 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99e000640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9a006e370 0x7fb9a00cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99ea00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9a006df90 0x7fb9a01521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99f400640 1 -- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9a006df90 msgr2=0x7fb9a01521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99f400640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9a006df90 0x7fb9a01521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99f400640 1 -- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9a006e370 msgr2=0x7fb9a00cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99f400640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9a006e370 0x7fb9a00cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99f400640 1 -- 172.21.15.44:0/1975376723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a00b59b0 con 0x7fb9a00aff60 2024-09-13T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99e000640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9a006e370 0x7fb9a00cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99ea00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9a006df90 0x7fb9a01521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99f400640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9a00aff60 0x7fb9a00cc910 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb998001c60 tx=0x7fb998001d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb996a00640 1 -- 172.21.15.44:0/1975376723 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998016020 con 0x7fb9a00aff60 2024-09-13T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb996a00640 1 -- 172.21.15.44:0/1975376723 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb99800acf0 con 0x7fb9a00aff60 2024-09-13T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb996a00640 1 -- 172.21.15.44:0/1975376723 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998012680 con 0x7fb9a00aff60 2024-09-13T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 -- 172.21.15.44:0/1975376723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a00ceff0 con 0x7fb9a00aff60 2024-09-13T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 -- 172.21.15.44:0/1975376723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a00cf5b0 con 0x7fb9a00aff60 2024-09-13T08:46:50.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.308+0000 7fb99fe00640 1 -- 172.21.15.44:0/1975376723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a00b03b0 con 0x7fb9a00aff60 2024-09-13T08:46:50.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.312+0000 7fb996a00640 1 -- 172.21.15.44:0/1975376723 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7fb998019020 con 0x7fb9a00aff60 2024-09-13T08:46:50.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.312+0000 7fb996a00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb980074030 0x7fb9800764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:50.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.312+0000 7fb996a00640 1 -- 172.21.15.44:0/1975376723 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5908+0+0 (secure 0 0 0) 0x7fb998093340 con 0x7fb9a00aff60 2024-09-13T08:46:50.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.312+0000 7fb99ea00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb980074030 0x7fb9800764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:50.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.312+0000 7fb99ea00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb980074030 0x7fb9800764f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb9a015bd30 tx=0x7fb990009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:50.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.316+0000 7fb996a00640 1 -- 172.21.15.44:0/1975376723 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb998020020 con 0x7fb9a00aff60 2024-09-13T08:46:50.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.516+0000 7fb99fe00640 1 -- 172.21.15.44:0/1975376723 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone21695", "target": ["mon-mgr", ""]}) -- 0x7fb9a01526e0 con 0x7fb980074030 2024-09-13T08:46:50.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb996a00640 1 -- 172.21.15.44:0/1975376723 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb990001cd0 con 0x7fb980074030 2024-09-13T08:46:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 -- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb980074030 msgr2=0x7fb9800764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb980074030 0x7fb9800764f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb9a015bd30 tx=0x7fb990009040 comp rx=0 tx=0).stop 2024-09-13T08:46:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 -- 172.21.15.44:0/1975376723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9a00aff60 msgr2=0x7fb9a00cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9a00aff60 0x7fb9a00cc910 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb998001c60 tx=0x7fb998001d50 comp rx=0 tx=0).stop 2024-09-13T08:46:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 -- 172.21.15.44:0/1975376723 shutdown_connections 2024-09-13T08:46:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb980074030 0x7fb9800764f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9a00aff60 0x7fb9a00cc910 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9a006e370 0x7fb9a00cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 --2- 172.21.15.44:0/1975376723 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9a006df90 0x7fb9a01521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 -- 172.21.15.44:0/1975376723 >> 172.21.15.44:0/1975376723 conn(0x7fb9a00d1da0 msgr2=0x7fb9a00ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:50.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 -- 172.21.15.44:0/1975376723 shutdown_connections 2024-09-13T08:46:50.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.536+0000 7fb99fe00640 1 -- 172.21.15.44:0/1975376723 wait complete. 2024-09-13T08:46:50.562 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-13T08:46:50.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:46:50.617 INFO:teuthology.orchestra.run.smithi044.stdout:2 2024-09-13T08:46:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:46:50.623 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:46:50.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 -- 172.21.15.44:0/2946664830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c0ccf10 msgr2=0x7f1d9c0cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:50.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 --2- 172.21.15.44:0/2946664830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c0ccf10 0x7f1d9c0cd3a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0738e0 tx=0x7f1d9401cd50 comp rx=0 tx=0).stop 2024-09-13T08:46:50.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 -- 172.21.15.44:0/2946664830 shutdown_connections 2024-09-13T08:46:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 --2- 172.21.15.44:0/2946664830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c0ccf10 0x7f1d9c0cd3a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 --2- 172.21.15.44:0/2946664830 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d9c0b2a00 0x7f1d9c0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 --2- 172.21.15.44:0/2946664830 >> v2:172.21.15.129:3300/0 conn(0x7f1d9c15f340 0x7f1d9c0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 -- 172.21.15.44:0/2946664830 >> 172.21.15.44:0/2946664830 conn(0x7f1d9c0d1e00 msgr2=0x7f1d9c0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 -- 172.21.15.44:0/2946664830 shutdown_connections 2024-09-13T08:46:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 -- 172.21.15.44:0/2946664830 wait complete. 2024-09-13T08:46:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 Processor -- start 2024-09-13T08:46:50.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 -- start start 2024-09-13T08:46:50.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d9c0b2a00 0x7f1d9c157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:50.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c0ccf10 0x7f1d9c0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:50.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d9c15f340 0x7f1d9c071510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:50.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c0b1ad0 con 0x7f1d9c0ccf10 2024-09-13T08:46:50.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c0b1950 con 0x7f1d9c0b2a00 2024-09-13T08:46:50.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1da0c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9c0b1c50 con 0x7f1d9c15f340 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1d91400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c0ccf10 0x7f1d9c0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1d91400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c0ccf10 0x7f1d9c0c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57208/0 (socket says 172.21.15.44:57208) 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1d91400640 1 -- 172.21.15.44:0/2016149953 learned_addr learned my addr 172.21.15.44:0/2016149953 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1d99e00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d9c0b2a00 0x7f1d9c157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.812+0000 7f1d9a800640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d9c15f340 0x7f1d9c071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d91400640 1 -- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d9c15f340 msgr2=0x7f1d9c071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d91400640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d9c15f340 0x7f1d9c071510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d91400640 1 -- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d9c0b2a00 msgr2=0x7f1d9c157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d91400640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d9c0b2a00 0x7f1d9c157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d91400640 1 -- 172.21.15.44:0/2016149953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c0c7850 con 0x7f1d9c0ccf10 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d99e00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d9c0b2a00 0x7f1d9c157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:46:50.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d91400640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c0ccf10 0x7f1d9c0c96e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400e9c0 tx=0x7f1d8400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:50.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d92a00640 1 -- 172.21.15.44:0/2016149953 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400ccc0 con 0x7f1d9c0ccf10 2024-09-13T08:46:50.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d92a00640 1 -- 172.21.15.44:0/2016149953 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d84004020 con 0x7f1d9c0ccf10 2024-09-13T08:46:50.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d92a00640 1 -- 172.21.15.44:0/2016149953 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84004470 con 0x7f1d9c0ccf10 2024-09-13T08:46:50.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1da0c00640 1 -- 172.21.15.44:0/2016149953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c0c8240 con 0x7f1d9c0ccf10 2024-09-13T08:46:50.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1da0c00640 1 -- 172.21.15.44:0/2016149953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c0c8800 con 0x7f1d9c0ccf10 2024-09-13T08:46:50.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1da0c00640 1 -- 172.21.15.44:0/2016149953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9c15b8b0 con 0x7f1d9c0ccf10 2024-09-13T08:46:50.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d92a00640 1 -- 172.21.15.44:0/2016149953 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f1d84003600 con 0x7f1d9c0ccf10 2024-09-13T08:46:50.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d92a00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1d78074060 0x7f1d78076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:50.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.816+0000 7f1d92a00640 1 -- 172.21.15.44:0/2016149953 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5908+0+0 (secure 0 0 0) 0x7f1d84093610 con 0x7f1d9c0ccf10 2024-09-13T08:46:50.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.820+0000 7f1d99e00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1d78074060 0x7f1d78076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:50.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.820+0000 7f1d99e00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1d78074060 0x7f1d78076520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009f30 tx=0x7f1d8c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:50.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.820+0000 7f1d92a00640 1 -- 172.21.15.44:0/2016149953 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1d8405f910 con 0x7f1d9c0ccf10 2024-09-13T08:46:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:50.996+0000 7f1da0c00640 1 -- 172.21.15.44:0/2016149953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f1d9c0b0ce0 con 0x7f1d9c0ccf10 2024-09-13T08:46:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1d92a00640 1 -- 172.21.15.44:0/2016149953 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v39)=0 v39) ==== 131+0+0 (secure 0 0 0) 0x7f1d84064340 con 0x7f1d9c0ccf10 2024-09-13T08:46:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 -- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1d78074060 msgr2=0x7f1d78076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1d78074060 0x7f1d78076520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009f30 tx=0x7f1d8c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 -- 172.21.15.44:0/2016149953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c0ccf10 msgr2=0x7f1d9c0c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c0ccf10 0x7f1d9c0c96e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400e9c0 tx=0x7f1d8400ee80 comp rx=0 tx=0).stop 2024-09-13T08:46:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 -- 172.21.15.44:0/2016149953 shutdown_connections 2024-09-13T08:46:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1d78074060 0x7f1d78076520 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d9c15f340 0x7f1d9c071510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c0ccf10 0x7f1d9c0c96e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 --2- 172.21.15.44:0/2016149953 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d9c0b2a00 0x7f1d9c157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 -- 172.21.15.44:0/2016149953 >> 172.21.15.44:0/2016149953 conn(0x7f1d9c0d1e00 msgr2=0x7f1d9c0af260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 -- 172.21.15.44:0/2016149953 shutdown_connections 2024-09-13T08:46:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.000+0000 7f1da0c00640 1 -- 172.21.15.44:0/2016149953 wait complete. 2024-09-13T08:46:51.022 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:46:51.022 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:46:51.126 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:46:51.211 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:46:51.211 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:46:51.211 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:46:51.212 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:46:51.221 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:46:51.221 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:46:51.272 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:46:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 -- 172.21.15.44:0/2003466287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84880cecf0 msgr2=0x7f84880cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 --2- 172.21.15.44:0/2003466287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84880cecf0 0x7f84880cc940 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f84880d0780 tx=0x7f847401c700 comp rx=0 tx=0).stop 2024-09-13T08:46:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 -- 172.21.15.44:0/2003466287 shutdown_connections 2024-09-13T08:46:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 --2- 172.21.15.44:0/2003466287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84880cecf0 0x7f84880cc940 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 -- 172.21.15.44:0/2003466287 >> 172.21.15.44:0/2003466287 conn(0x7f84880d1ea0 msgr2=0x7f84880d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 -- 172.21.15.44:0/2003466287 shutdown_connections 2024-09-13T08:46:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 -- 172.21.15.44:0/2003466287 wait complete. 2024-09-13T08:46:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 Processor -- start 2024-09-13T08:46:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 -- start start 2024-09-13T08:46:51.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84880cecf0 0x7f84880c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:51.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f848815bd50 0x7f84881509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:51.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f848815c130 0x7f84881554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:51.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f848816a5b0 con 0x7f848815c130 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f848816a430 con 0x7f84880cecf0 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f8490400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f848816a730 con 0x7f848815bd50 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f848e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f848815c130 0x7f84881554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f848e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f848815c130 0x7f84881554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57224/0 (socket says 172.21.15.44:57224) 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f848e400640 1 -- 172.21.15.44:0/4260314142 learned_addr learned my addr 172.21.15.44:0/4260314142 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f848da00640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84880cecf0 0x7f84880c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.468+0000 7f848d000640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f848815bd50 0x7f84881509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f848e400640 1 -- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f848815bd50 msgr2=0x7f84881509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f848e400640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f848815bd50 0x7f84881509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f848e400640 1 -- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84880cecf0 msgr2=0x7f84880c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f848e400640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84880cecf0 0x7f84880c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f848e400640 1 -- 172.21.15.44:0/4260314142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84880cb350 con 0x7f848815c130 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f848e400640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f848815c130 0x7f84881554b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f848000c760 tx=0x7f848000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:51.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f8485600640 1 -- 172.21.15.44:0/4260314142 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480003db0 con 0x7f848815c130 2024-09-13T08:46:51.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f8485600640 1 -- 172.21.15.44:0/4260314142 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f848000ce10 con 0x7f848815c130 2024-09-13T08:46:51.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f8490400640 1 -- 172.21.15.44:0/4260314142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84880cb630 con 0x7f848815c130 2024-09-13T08:46:51.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f8485600640 1 -- 172.21.15.44:0/4260314142 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800052d0 con 0x7f848815c130 2024-09-13T08:46:51.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f8490400640 1 -- 172.21.15.44:0/4260314142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84880cbbf0 con 0x7f848815c130 2024-09-13T08:46:51.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f8490400640 1 -- 172.21.15.44:0/4260314142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84880ce5e0 con 0x7f848815c130 2024-09-13T08:46:51.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.472+0000 7f8485600640 1 -- 172.21.15.44:0/4260314142 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f8480005470 con 0x7f848815c130 2024-09-13T08:46:51.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.476+0000 7f8485600640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8468074030 0x7f84680764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:51.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.476+0000 7f8485600640 1 -- 172.21.15.44:0/4260314142 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5908+0+0 (secure 0 0 0) 0x7f8480092950 con 0x7f848815c130 2024-09-13T08:46:51.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.476+0000 7f848da00640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8468074030 0x7f84680764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:51.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.476+0000 7f848da00640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8468074030 0x7f84680764f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f847c007fd0 tx=0x7f847c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:51.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.476+0000 7f8485600640 1 -- 172.21.15.44:0/4260314142 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f848005ec50 con 0x7f848815c130 2024-09-13T08:46:51.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.668+0000 7f8490400640 1 -- 172.21.15.44:0/4260314142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8488076360 con 0x7f848815c130 2024-09-13T08:46:51.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.668+0000 7f8485600640 1 -- 172.21.15.44:0/4260314142 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) ==== 80+0+0 (secure 0 0 0) 0x7f8480063680 con 0x7f848815c130 2024-09-13T08:46:51.670 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:46:51.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 -- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8468074030 msgr2=0x7f84680764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:51.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8468074030 0x7f84680764f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f847c007fd0 tx=0x7f847c009040 comp rx=0 tx=0).stop 2024-09-13T08:46:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 -- 172.21.15.44:0/4260314142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f848815c130 msgr2=0x7f84881554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f848815c130 0x7f84881554b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f848000c760 tx=0x7f848000cc20 comp rx=0 tx=0).stop 2024-09-13T08:46:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 -- 172.21.15.44:0/4260314142 shutdown_connections 2024-09-13T08:46:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8468074030 0x7f84680764f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f848815c130 0x7f84881554b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f848815bd50 0x7f84881509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 --2- 172.21.15.44:0/4260314142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84880cecf0 0x7f84880c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 -- 172.21.15.44:0/4260314142 >> 172.21.15.44:0/4260314142 conn(0x7f84880d1ea0 msgr2=0x7f84880d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 -- 172.21.15.44:0/4260314142 shutdown_connections 2024-09-13T08:46:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:51.672+0000 7f8490400640 1 -- 172.21.15.44:0/4260314142 wait complete. 2024-09-13T08:46:51.693 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T08:46:51.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:46:51.694 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T08:46:51.738 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:46:51.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:46:51.738 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T08:46:51.794 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:46:51.794 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:46:51.794 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:46:51.807 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:46:51.807 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:46:51.808 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:46:51.808 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:46:51.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:46:51.849 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:46:51.849 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:46:51.849 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:46:51.865 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:46:51.866 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:46:52.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.156+0000 7f1ce2000640 1 -- 172.21.15.44:0/3220125147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cdc15f000 msgr2=0x7f1cdc0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:52.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.156+0000 7f1ce2000640 1 --2- 172.21.15.44:0/3220125147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cdc15f000 0x7f1cdc0d01b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009840 tx=0x7f1ccc01c790 comp rx=0 tx=0).stop 2024-09-13T08:46:52.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.156+0000 7f1ce2000640 1 -- 172.21.15.44:0/3220125147 shutdown_connections 2024-09-13T08:46:52.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.156+0000 7f1ce2000640 1 --2- 172.21.15.44:0/3220125147 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cdc0b2590 0x7f1cdc0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.156+0000 7f1ce2000640 1 --2- 172.21.15.44:0/3220125147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cdc15f000 0x7f1cdc0d01b0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.156+0000 7f1ce2000640 1 -- 172.21.15.44:0/3220125147 >> 172.21.15.44:0/3220125147 conn(0x7f1cdc09aa10 msgr2=0x7f1cdc09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.156+0000 7f1ce2000640 1 -- 172.21.15.44:0/3220125147 shutdown_connections 2024-09-13T08:46:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.156+0000 7f1ce2000640 1 -- 172.21.15.44:0/3220125147 wait complete. 2024-09-13T08:46:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.160+0000 7f1ce2000640 1 Processor -- start 2024-09-13T08:46:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.160+0000 7f1ce2000640 1 -- start start 2024-09-13T08:46:52.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.160+0000 7f1ce2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cdc0b2590 0x7f1cdc071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:52.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.160+0000 7f1ce2000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cdc0cefb0 0x7f1cdc1fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:52.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.160+0000 7f1ce2000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cdc15f000 0x7f1cdc1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:52.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc168f00 con 0x7f1cdc0b2590 2024-09-13T08:46:52.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce2000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc168d80 con 0x7f1cdc15f000 2024-09-13T08:46:52.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce2000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1cdc169080 con 0x7f1cdc0cefb0 2024-09-13T08:46:52.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cdc0b2590 0x7f1cdc071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:52.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cdc0b2590 0x7f1cdc071630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43822/0 (socket says 172.21.15.44:43822) 2024-09-13T08:46:52.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce0c00640 1 -- 172.21.15.44:0/1338120346 learned_addr learned my addr 172.21.15.44:0/1338120346 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:52.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce1600640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cdc15f000 0x7f1cdc1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:52.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1cd3e00640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cdc0cefb0 0x7f1cdc1fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:52.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce0c00640 1 -- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cdc0cefb0 msgr2=0x7f1cdc1fb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:52.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce0c00640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cdc0cefb0 0x7f1cdc1fb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce0c00640 1 -- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cdc15f000 msgr2=0x7f1cdc1519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:52.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce0c00640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cdc15f000 0x7f1cdc1519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce0c00640 1 -- 172.21.15.44:0/1338120346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc071b70 con 0x7f1cdc0b2590 2024-09-13T08:46:52.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce1600640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cdc15f000 0x7f1cdc1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:52.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1cd3e00640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cdc0cefb0 0x7f1cdc1fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:52.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.164+0000 7f1ce0c00640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cdc0b2590 0x7f1cdc071630 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009790 tx=0x7f1ccc0a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:52.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.168+0000 7f1cd9600640 1 -- 172.21.15.44:0/1338120346 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0ac030 con 0x7f1cdc0b2590 2024-09-13T08:46:52.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.168+0000 7f1cd9600640 1 -- 172.21.15.44:0/1338120346 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ccc005430 con 0x7f1cdc0b2590 2024-09-13T08:46:52.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.168+0000 7f1cd9600640 1 -- 172.21.15.44:0/1338120346 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0046d0 con 0x7f1cdc0b2590 2024-09-13T08:46:52.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.168+0000 7f1ce2000640 1 -- 172.21.15.44:0/1338120346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cdc1fbf20 con 0x7f1cdc0b2590 2024-09-13T08:46:52.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.168+0000 7f1ce2000640 1 -- 172.21.15.44:0/1338120346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cdc157640 con 0x7f1cdc0b2590 2024-09-13T08:46:52.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.168+0000 7f1ce2000640 1 -- 172.21.15.44:0/1338120346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cdc157840 con 0x7f1cdc0b2590 2024-09-13T08:46:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.172+0000 7f1cd9600640 1 -- 172.21.15.44:0/1338120346 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f1ccc0af070 con 0x7f1cdc0b2590 2024-09-13T08:46:52.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.176+0000 7f1cd9600640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb8074080 0x7f1cb8076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:52.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.176+0000 7f1cd9600640 1 -- 172.21.15.44:0/1338120346 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5908+0+0 (secure 0 0 0) 0x7f1ccc12cda0 con 0x7f1cdc0b2590 2024-09-13T08:46:52.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.176+0000 7f1cd3e00640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb8074080 0x7f1cb8076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:52.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.176+0000 7f1cd3e00640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb8074080 0x7f1cb8076540 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40046c0 tx=0x7f1cc4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:52.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.184+0000 7f1cd9600640 1 -- 172.21.15.44:0/1338120346 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1ccc016410 con 0x7f1cdc0b2590 2024-09-13T08:46:52.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 -- 172.21.15.44:0/1338120346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cdc0b1300 con 0x7f1cdc0b2590 2024-09-13T08:46:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1cd9600640 1 -- 172.21.15.44:0/1338120346 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7f1ccc0f9730 con 0x7f1cdc0b2590 2024-09-13T08:46:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:52.450 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":47,"btime":"2024-09-13T08:45:46:720592+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3854096181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3854096181},{"type":"v1","addr":"172.21.15.129:6836","nonce":3854096181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2183755853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":2183755853},{"type":"v1","addr":"172.21.15.129:6837","nonce":2183755853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6330,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1061181525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1061181525},{"type":"v1","addr":"172.21.15.44:6834","nonce":1061181525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:45:33.055598+0000","modified":"2024-09-13T08:45:36.075582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/651900305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":651900305},{"type":"v1","addr":"172.21.15.44:6835","nonce":651900305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6333,"qdb_cluster":[6333]},"id":5}]} 2024-09-13T08:46:52.451 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 47 2024-09-13T08:46:52.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 -- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb8074080 msgr2=0x7f1cb8076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:52.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb8074080 0x7f1cb8076540 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40046c0 tx=0x7f1cc4009040 comp rx=0 tx=0).stop 2024-09-13T08:46:52.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 -- 172.21.15.44:0/1338120346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cdc0b2590 msgr2=0x7f1cdc071630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:52.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cdc0b2590 0x7f1cdc071630 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009790 tx=0x7f1ccc0a6570 comp rx=0 tx=0).stop 2024-09-13T08:46:52.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 -- 172.21.15.44:0/1338120346 shutdown_connections 2024-09-13T08:46:52.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb8074080 0x7f1cb8076540 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cdc15f000 0x7f1cdc1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cdc0cefb0 0x7f1cdc1fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 --2- 172.21.15.44:0/1338120346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cdc0b2590 0x7f1cdc071630 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 -- 172.21.15.44:0/1338120346 >> 172.21.15.44:0/1338120346 conn(0x7f1cdc09aa10 msgr2=0x7f1cdc071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:52.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 -- 172.21.15.44:0/1338120346 shutdown_connections 2024-09-13T08:46:52.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.448+0000 7f1ce2000640 1 -- 172.21.15.44:0/1338120346 wait complete. 2024-09-13T08:46:52.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:46:52.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.616+0000 7f41fb000640 1 -- 172.21.15.44:0/3370823703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41d8003f80 msgr2=0x7f41d8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:52.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.616+0000 7f41fb000640 1 --2- 172.21.15.44:0/3370823703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41d8003f80 0x7f41d8004810 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f41dc001770 tx=0x7f41dc01c860 comp rx=0 tx=0).stop 2024-09-13T08:46:52.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.616+0000 7f41f9000640 1 -- 172.21.15.44:0/3370823703 reap_dead start 2024-09-13T08:46:52.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.616+0000 7f41fb000640 1 -- 172.21.15.44:0/3370823703 shutdown_connections 2024-09-13T08:46:52.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.616+0000 7f41fb000640 1 -- 172.21.15.44:0/3370823703 >> 172.21.15.44:0/3370823703 conn(0x7f41f40b18f0 msgr2=0x7f41f40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:52.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41fb000640 1 -- 172.21.15.44:0/3370823703 shutdown_connections 2024-09-13T08:46:52.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41fb000640 1 -- 172.21.15.44:0/3370823703 wait complete. 2024-09-13T08:46:52.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41fb000640 1 Processor -- start 2024-09-13T08:46:52.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41fb000640 1 -- start start 2024-09-13T08:46:52.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41fb000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41d8003f80 0x7f41f41f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:52.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41fb000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41f41427b0 0x7f41f4141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:52.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41fb000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f41f4162360 0x7f41f41627e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:52.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41fb000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41f4166060 con 0x7f41d8003f80 2024-09-13T08:46:52.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41fb000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f41f4165ee0 con 0x7f41f41427b0 2024-09-13T08:46:52.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41fb000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f41f41661e0 con 0x7f41f4162360 2024-09-13T08:46:52.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41d8003f80 0x7f41f41f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41d8003f80 0x7f41f41f7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43836/0 (socket says 172.21.15.44:43836) 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f3e00640 1 -- 172.21.15.44:0/1838914844 learned_addr learned my addr 172.21.15.44:0/1838914844 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41ebe00640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41f41427b0 0x7f41f4141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f9000640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f41f4162360 0x7f41f41627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f3e00640 1 -- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f41f4162360 msgr2=0x7f41f41627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f3e00640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f41f4162360 0x7f41f41627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f3e00640 1 -- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41f41427b0 msgr2=0x7f41f4141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f3e00640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41f41427b0 0x7f41f4141f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f3e00640 1 -- 172.21.15.44:0/1838914844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f41f81b0 con 0x7f41d8003f80 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f3e00640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41d8003f80 0x7f41f41f7c70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f41e4006480 tx=0x7f41e40039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.620+0000 7f41f0c00640 1 -- 172.21.15.44:0/1838914844 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4015070 con 0x7f41d8003f80 2024-09-13T08:46:52.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.624+0000 7f41f0c00640 1 -- 172.21.15.44:0/1838914844 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41e4011920 con 0x7f41d8003f80 2024-09-13T08:46:52.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.624+0000 7f41f0c00640 1 -- 172.21.15.44:0/1838914844 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4011d70 con 0x7f41d8003f80 2024-09-13T08:46:52.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.624+0000 7f41fb000640 1 -- 172.21.15.44:0/1838914844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f41f8450 con 0x7f41d8003f80 2024-09-13T08:46:52.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.624+0000 7f41fb000640 1 -- 172.21.15.44:0/1838914844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f41f8a10 con 0x7f41d8003f80 2024-09-13T08:46:52.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.624+0000 7f41fb000640 1 -- 172.21.15.44:0/1838914844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f4069410 con 0x7f41d8003f80 2024-09-13T08:46:52.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.624+0000 7f41f0c00640 1 -- 172.21.15.44:0/1838914844 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f41e4018070 con 0x7f41d8003f80 2024-09-13T08:46:52.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.624+0000 7f41f0c00640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41d0074030 0x7f41d00764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:52.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.624+0000 7f41f0c00640 1 -- 172.21.15.44:0/1838914844 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5908+0+0 (secure 0 0 0) 0x7f41e40935c0 con 0x7f41d8003f80 2024-09-13T08:46:52.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.624+0000 7f41ebe00640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41d0074030 0x7f41d00764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:52.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.628+0000 7f41ebe00640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41d0074030 0x7f41d00764f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0a8000 tx=0x7f41dc019600 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:52.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.628+0000 7f41f0c00640 1 -- 172.21.15.44:0/1838914844 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f41e405f840 con 0x7f41d8003f80 2024-09-13T08:46:52.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.956+0000 7f41fb000640 1 -- 172.21.15.44:0/1838914844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41f406fca0 con 0x7f41d8003f80 2024-09-13T08:46:52.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.956+0000 7f41f0c00640 1 -- 172.21.15.44:0/1838914844 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7f41e4064270 con 0x7f41d8003f80 2024-09-13T08:46:52.958 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:52.958 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":47,"btime":"2024-09-13T08:45:46:720592+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3854096181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3854096181},{"type":"v1","addr":"172.21.15.129:6836","nonce":3854096181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2183755853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":2183755853},{"type":"v1","addr":"172.21.15.129:6837","nonce":2183755853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6330,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1061181525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1061181525},{"type":"v1","addr":"172.21.15.44:6834","nonce":1061181525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:45:33.055598+0000","modified":"2024-09-13T08:45:36.075582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/651900305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":651900305},{"type":"v1","addr":"172.21.15.44:6835","nonce":651900305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6333,"qdb_cluster":[6333]},"id":5}]} 2024-09-13T08:46:52.958 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 47 2024-09-13T08:46:52.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 -- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41d0074030 msgr2=0x7f41d00764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:52.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41d0074030 0x7f41d00764f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0a8000 tx=0x7f41dc019600 comp rx=0 tx=0).stop 2024-09-13T08:46:52.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 -- 172.21.15.44:0/1838914844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41d8003f80 msgr2=0x7f41f41f7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:52.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41d8003f80 0x7f41f41f7c70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f41e4006480 tx=0x7f41e40039d0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 -- 172.21.15.44:0/1838914844 shutdown_connections 2024-09-13T08:46:52.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41d0074030 0x7f41d00764f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f41f4162360 0x7f41f41627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41f41427b0 0x7f41f4141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 --2- 172.21.15.44:0/1838914844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41d8003f80 0x7f41f41f7c70 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:52.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 -- 172.21.15.44:0/1838914844 >> 172.21.15.44:0/1838914844 conn(0x7f41f40b18f0 msgr2=0x7f41f4143520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:52.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 -- 172.21.15.44:0/1838914844 shutdown_connections 2024-09-13T08:46:52.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:52.960+0000 7f41fb000640 1 -- 172.21.15.44:0/1838914844 wait complete. 2024-09-13T08:46:52.999 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:46:53.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.248+0000 7fa2ada00640 1 -- 172.21.15.44:0/3600499494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa28c004810 msgr2=0x7fa28c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:53.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.248+0000 7fa2ada00640 1 --2- 172.21.15.44:0/3600499494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa28c004810 0x7fa28c004c90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa2940016f0 tx=0x7fa29401c750 comp rx=0 tx=0).stop 2024-09-13T08:46:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.248+0000 7fa2ada00640 1 -- 172.21.15.44:0/3600499494 shutdown_connections 2024-09-13T08:46:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.248+0000 7fa2ada00640 1 --2- 172.21.15.44:0/3600499494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa28c004810 0x7fa28c004c90 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.248+0000 7fa2ada00640 1 -- 172.21.15.44:0/3600499494 >> 172.21.15.44:0/3600499494 conn(0x7fa2a80b2860 msgr2=0x7fa2a80b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.248+0000 7fa2ada00640 1 -- 172.21.15.44:0/3600499494 shutdown_connections 2024-09-13T08:46:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.248+0000 7fa2ada00640 1 -- 172.21.15.44:0/3600499494 wait complete. 2024-09-13T08:46:53.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2ada00640 1 Processor -- start 2024-09-13T08:46:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2ada00640 1 -- start start 2024-09-13T08:46:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2ada00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa28c004810 0x7fa2a80afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2ada00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2a815b990 0x7fa2a813e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2ada00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa2a815be10 0x7fa2a813e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2ada00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a815ecb0 con 0x7fa28c004810 2024-09-13T08:46:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2ada00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a815ee30 con 0x7fa2a815b990 2024-09-13T08:46:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2ada00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a815efb0 con 0x7fa2a815be10 2024-09-13T08:46:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa28c004810 0x7fa2a80afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa28c004810 0x7fa2a80afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43852/0 (socket says 172.21.15.44:43852) 2024-09-13T08:46:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6a00640 1 -- 172.21.15.44:0/1392534903 learned_addr learned my addr 172.21.15.44:0/1392534903 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a7400640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa2a815be10 0x7fa2a813e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6a00640 1 -- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa2a815be10 msgr2=0x7fa2a813e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6000640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2a815b990 0x7fa2a813e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6a00640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa2a815be10 0x7fa2a813e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6a00640 1 -- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2a815b990 msgr2=0x7fa2a813e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:53.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6a00640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2a815b990 0x7fa2a813e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6a00640 1 -- 172.21.15.44:0/1392534903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a81f5370 con 0x7fa28c004810 2024-09-13T08:46:53.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a7400640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa2a815be10 0x7fa2a813e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:53.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6000640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2a815b990 0x7fa2a813e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2a6a00640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa28c004810 0x7fa2a80afd30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa298001870 tx=0x7fa298012870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa29ea00640 1 -- 172.21.15.44:0/1392534903 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298013920 con 0x7fa28c004810 2024-09-13T08:46:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa29ea00640 1 -- 172.21.15.44:0/1392534903 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa298013ac0 con 0x7fa28c004810 2024-09-13T08:46:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa29ea00640 1 -- 172.21.15.44:0/1392534903 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298010480 con 0x7fa28c004810 2024-09-13T08:46:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2ada00640 1 -- 172.21.15.44:0/1392534903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a81f5650 con 0x7fa28c004810 2024-09-13T08:46:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.252+0000 7fa2ada00640 1 -- 172.21.15.44:0/1392534903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81f5c10 con 0x7fa28c004810 2024-09-13T08:46:53.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.256+0000 7fa2ada00640 1 -- 172.21.15.44:0/1392534903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a80716d0 con 0x7fa28c004810 2024-09-13T08:46:53.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.256+0000 7fa29ea00640 1 -- 172.21.15.44:0/1392534903 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7fa298010620 con 0x7fa28c004810 2024-09-13T08:46:53.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.256+0000 7fa29ea00640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa288074030 0x7fa2880764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:53.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.256+0000 7fa29ea00640 1 -- 172.21.15.44:0/1392534903 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5908+0+0 (secure 0 0 0) 0x7fa298096ba0 con 0x7fa28c004810 2024-09-13T08:46:53.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.256+0000 7fa2a6000640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa288074030 0x7fa2880764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:53.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.256+0000 7fa2a6000640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa288074030 0x7fa2880764f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa2940016c0 tx=0x7fa294019540 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:53.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.260+0000 7fa29ea00640 1 -- 172.21.15.44:0/1392534903 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa298063550 con 0x7fa28c004810 2024-09-13T08:46:53.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.436+0000 7fa2ada00640 1 -- 172.21.15.44:0/1392534903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2a8070970 con 0x7fa28c004810 2024-09-13T08:46:53.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa29ea00640 1 -- 172.21.15.44:0/1392534903 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+17022 (secure 0 0 0) 0x7fa298062e50 con 0x7fa28c004810 2024-09-13T08:46:53.441 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:53.441 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":78,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:46:50.129011+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:45:30.771920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T08:45:32.014594+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":78,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.c","mappings":[{"from":2,"to":7}]},{"pgid":"11.1","mappings":[{"from":1,"to":4}]},{"pgid":"11.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:46:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 -- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa288074030 msgr2=0x7fa2880764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa288074030 0x7fa2880764f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa2940016c0 tx=0x7fa294019540 comp rx=0 tx=0).stop 2024-09-13T08:46:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 -- 172.21.15.44:0/1392534903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa28c004810 msgr2=0x7fa2a80afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa28c004810 0x7fa2a80afd30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa298001870 tx=0x7fa298012870 comp rx=0 tx=0).stop 2024-09-13T08:46:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 -- 172.21.15.44:0/1392534903 shutdown_connections 2024-09-13T08:46:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa288074030 0x7fa2880764f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa2a815be10 0x7fa2a813e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2a815b990 0x7fa2a813e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 --2- 172.21.15.44:0/1392534903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa28c004810 0x7fa2a80afd30 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 -- 172.21.15.44:0/1392534903 >> 172.21.15.44:0/1392534903 conn(0x7fa2a80b2860 msgr2=0x7fa2a813fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 -- 172.21.15.44:0/1392534903 shutdown_connections 2024-09-13T08:46:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.440+0000 7fa2ada00640 1 -- 172.21.15.44:0/1392534903 wait complete. 2024-09-13T08:46:53.463 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:46:53.463 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:46:53.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.608+0000 7f581e200640 1 -- 172.21.15.44:0/2394472274 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f581815af60 msgr2=0x7f58180cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:53.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.608+0000 7f581e200640 1 --2- 172.21.15.44:0/2394472274 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f581815af60 0x7f58180cffc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5808009870 tx=0x7f580801c830 comp rx=0 tx=0).stop 2024-09-13T08:46:53.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.608+0000 7f581e200640 1 -- 172.21.15.44:0/2394472274 shutdown_connections 2024-09-13T08:46:53.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.608+0000 7f581e200640 1 --2- 172.21.15.44:0/2394472274 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f581815af60 0x7f58180cffc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.608+0000 7f581e200640 1 -- 172.21.15.44:0/2394472274 >> 172.21.15.44:0/2394472274 conn(0x7f58180d0c90 msgr2=0x7f58180d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:53.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.608+0000 7f581e200640 1 -- 172.21.15.44:0/2394472274 shutdown_connections 2024-09-13T08:46:53.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.608+0000 7f581e200640 1 -- 172.21.15.44:0/2394472274 wait complete. 2024-09-13T08:46:53.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.608+0000 7f581e200640 1 Processor -- start 2024-09-13T08:46:53.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.608+0000 7f581e200640 1 -- start start 2024-09-13T08:46:53.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f581e200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f58180737c0 0x7f58180c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:53.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f581e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5818074330 0x7f58180a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:53.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f581e200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f581815af60 0x7f58180a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:53.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f581e200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5818168cd0 con 0x7f5818074330 2024-09-13T08:46:53.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f581e200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5818168b50 con 0x7f581815af60 2024-09-13T08:46:53.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f581e200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5818168e50 con 0x7f58180737c0 2024-09-13T08:46:53.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5818074330 0x7f58180a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:53.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5818074330 0x7f58180a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43868/0 (socket says 172.21.15.44:43868) 2024-09-13T08:46:53.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580ea00640 1 -- 172.21.15.44:0/1131274337 learned_addr learned my addr 172.21.15.44:0/1131274337 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:53.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580ea00640 1 -- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f58180737c0 msgr2=0x7f58180c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:53.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580ea00640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f58180737c0 0x7f58180c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f5817e00640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f581815af60 0x7f58180a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:53.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580ea00640 1 -- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f581815af60 msgr2=0x7f58180a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:53.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580ea00640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f581815af60 0x7f58180a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580ea00640 1 -- 172.21.15.44:0/1131274337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581809ee20 con 0x7f5818074330 2024-09-13T08:46:53.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f5817e00640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f581815af60 0x7f58180a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:53.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580ea00640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5818074330 0x7f58180a1490 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f580000c500 tx=0x7f580000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:53.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580fe00640 1 -- 172.21.15.44:0/1131274337 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000052d0 con 0x7f5818074330 2024-09-13T08:46:53.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580fe00640 1 -- 172.21.15.44:0/1131274337 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5800005490 con 0x7f5818074330 2024-09-13T08:46:53.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580fe00640 1 -- 172.21.15.44:0/1131274337 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800012850 con 0x7f5818074330 2024-09-13T08:46:53.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f581e200640 1 -- 172.21.15.44:0/1131274337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58181567c0 con 0x7f5818074330 2024-09-13T08:46:53.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f581e200640 1 -- 172.21.15.44:0/1131274337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5818156d80 con 0x7f5818074330 2024-09-13T08:46:53.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f581e200640 1 -- 172.21.15.44:0/1131274337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58180cffc0 con 0x7f5818074330 2024-09-13T08:46:53.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.612+0000 7f580fe00640 1 -- 172.21.15.44:0/1131274337 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f58000129f0 con 0x7f5818074330 2024-09-13T08:46:53.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.616+0000 7f580fe00640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57f4074030 0x7f57f40764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:53.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.616+0000 7f580fe00640 1 -- 172.21.15.44:0/1131274337 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5908+0+0 (secure 0 0 0) 0x7f5800092980 con 0x7f5818074330 2024-09-13T08:46:53.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.616+0000 7f5817400640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57f4074030 0x7f57f40764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:53.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.616+0000 7f5817400640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57f4074030 0x7f57f40764f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f58080af000 tx=0x7f58080195d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:53.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.620+0000 7f580fe00640 1 -- 172.21.15.44:0/1131274337 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f580005f2d0 con 0x7f5818074330 2024-09-13T08:46:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.832+0000 7f581e200640 1 -- 172.21.15.44:0/1131274337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58181586c0 con 0x7f5818074330 2024-09-13T08:46:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.832+0000 7f580fe00640 1 -- 172.21.15.44:0/1131274337 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7f580005ebd0 con 0x7f5818074330 2024-09-13T08:46:53.836 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:53.836 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":47,"btime":"2024-09-13T08:45:46:720592+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3854096181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3854096181},{"type":"v1","addr":"172.21.15.129:6836","nonce":3854096181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2183755853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":2183755853},{"type":"v1","addr":"172.21.15.129:6837","nonce":2183755853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6330,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1061181525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1061181525},{"type":"v1","addr":"172.21.15.44:6834","nonce":1061181525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:45:33.055598+0000","modified":"2024-09-13T08:45:36.075582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/651900305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":651900305},{"type":"v1","addr":"172.21.15.44:6835","nonce":651900305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6333,"qdb_cluster":[6333]},"id":5}]} 2024-09-13T08:46:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 47 2024-09-13T08:46:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 -- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57f4074030 msgr2=0x7f57f40764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57f4074030 0x7f57f40764f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f58080af000 tx=0x7f58080195d0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 -- 172.21.15.44:0/1131274337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5818074330 msgr2=0x7f58180a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5818074330 0x7f58180a1490 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f580000c500 tx=0x7f580000c9c0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 -- 172.21.15.44:0/1131274337 shutdown_connections 2024-09-13T08:46:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57f4074030 0x7f57f40764f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f581815af60 0x7f58180a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5818074330 0x7f58180a1490 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 --2- 172.21.15.44:0/1131274337 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f58180737c0 0x7f58180c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 -- 172.21.15.44:0/1131274337 >> 172.21.15.44:0/1131274337 conn(0x7f58180d0c90 msgr2=0x7f5818157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 -- 172.21.15.44:0/1131274337 shutdown_connections 2024-09-13T08:46:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:53.836+0000 7f581e200640 1 -- 172.21.15.44:0/1131274337 wait complete. 2024-09-13T08:46:53.858 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:46:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.004+0000 7f421b600640 1 -- 172.21.15.44:0/635593787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f421415bea0 msgr2=0x7f42140ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.004+0000 7f421b600640 1 --2- 172.21.15.44:0/635593787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f421415bea0 0x7f42140ce670 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4204009bc0 tx=0x7f420401ca30 comp rx=0 tx=0).stop 2024-09-13T08:46:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.004+0000 7f421b600640 1 -- 172.21.15.44:0/635593787 shutdown_connections 2024-09-13T08:46:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.004+0000 7f421b600640 1 --2- 172.21.15.44:0/635593787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f421415bea0 0x7f42140ce670 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.004+0000 7f421b600640 1 -- 172.21.15.44:0/635593787 >> 172.21.15.44:0/635593787 conn(0x7f42140d1dc0 msgr2=0x7f42140d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.004+0000 7f421b600640 1 -- 172.21.15.44:0/635593787 shutdown_connections 2024-09-13T08:46:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.004+0000 7f421b600640 1 -- 172.21.15.44:0/635593787 wait complete. 2024-09-13T08:46:54.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 Processor -- start 2024-09-13T08:46:54.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 -- start start 2024-09-13T08:46:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140cdb30 0x7f4214167520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42140d28e0 0x7f42140b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f421415bea0 0x7f42141567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f421416a930 con 0x7f42140cdb30 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f421416a7b0 con 0x7f421415bea0 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f421416aab0 con 0x7f42140d28e0 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4218c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140cdb30 0x7f4214167520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4218c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140cdb30 0x7f4214167520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43878/0 (socket says 172.21.15.44:43878) 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4218c00640 1 -- 172.21.15.44:0/1358974492 learned_addr learned my addr 172.21.15.44:0/1358974492 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f420be00640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42140d28e0 0x7f42140b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4219600640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f421415bea0 0x7f42141567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4218c00640 1 -- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42140d28e0 msgr2=0x7f42140b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4218c00640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42140d28e0 0x7f42140b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4218c00640 1 -- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f421415bea0 msgr2=0x7f42141567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4218c00640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f421415bea0 0x7f42141567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4218c00640 1 -- 172.21.15.44:0/1358974492 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4214155470 con 0x7f42140cdb30 2024-09-13T08:46:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f420be00640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42140d28e0 0x7f42140b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:46:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4218c00640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140cdb30 0x7f4214167520 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4204009b10 tx=0x7f4204009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4211600640 1 -- 172.21.15.44:0/1358974492 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42040bb070 con 0x7f42140cdb30 2024-09-13T08:46:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4211600640 1 -- 172.21.15.44:0/1358974492 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4204004e00 con 0x7f42140cdb30 2024-09-13T08:46:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f4211600640 1 -- 172.21.15.44:0/1358974492 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42040b2040 con 0x7f42140cdb30 2024-09-13T08:46:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 -- 172.21.15.44:0/1358974492 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4214155730 con 0x7f42140cdb30 2024-09-13T08:46:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 -- 172.21.15.44:0/1358974492 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4214155cf0 con 0x7f42140cdb30 2024-09-13T08:46:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.008+0000 7f421b600640 1 -- 172.21.15.44:0/1358974492 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42140ce670 con 0x7f42140cdb30 2024-09-13T08:46:54.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.012+0000 7f4211600640 1 -- 172.21.15.44:0/1358974492 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f42040be070 con 0x7f42140cdb30 2024-09-13T08:46:54.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.012+0000 7f4211600640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41f0074060 0x7f41f0076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.012+0000 7f4211600640 1 -- 172.21.15.44:0/1358974492 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5908+0+0 (secure 0 0 0) 0x7f420412c650 con 0x7f42140cdb30 2024-09-13T08:46:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.012+0000 7f420be00640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41f0074060 0x7f41f0076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.012+0000 7f420be00640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41f0074060 0x7f41f0076520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4214154840 tx=0x7f41fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:54.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.016+0000 7f4211600640 1 -- 172.21.15.44:0/1358974492 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f42040166b0 con 0x7f42140cdb30 2024-09-13T08:46:54.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.228+0000 7f421b600640 1 -- 172.21.15.44:0/1358974492 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4214154590 con 0x7f42140cdb30 2024-09-13T08:46:54.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f4211600640 1 -- 172.21.15.44:0/1358974492 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7f42040f8fa0 con 0x7f42140cdb30 2024-09-13T08:46:54.233 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:54.233 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":47,"btime":"2024-09-13T08:45:46:720592+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3854096181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3854096181},{"type":"v1","addr":"172.21.15.129:6836","nonce":3854096181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2183755853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":2183755853},{"type":"v1","addr":"172.21.15.129:6837","nonce":2183755853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6330,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1061181525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1061181525},{"type":"v1","addr":"172.21.15.44:6834","nonce":1061181525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:45:33.055598+0000","modified":"2024-09-13T08:45:36.075582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/651900305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":651900305},{"type":"v1","addr":"172.21.15.44:6835","nonce":651900305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6333,"qdb_cluster":[6333]},"id":5}]} 2024-09-13T08:46:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 47 2024-09-13T08:46:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 -- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41f0074060 msgr2=0x7f41f0076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41f0074060 0x7f41f0076520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4214154840 tx=0x7f41fc009040 comp rx=0 tx=0).stop 2024-09-13T08:46:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 -- 172.21.15.44:0/1358974492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140cdb30 msgr2=0x7f4214167520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140cdb30 0x7f4214167520 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4204009b10 tx=0x7f4204009200 comp rx=0 tx=0).stop 2024-09-13T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 -- 172.21.15.44:0/1358974492 shutdown_connections 2024-09-13T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41f0074060 0x7f41f0076520 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f421415bea0 0x7f42141567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42140d28e0 0x7f42140b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 --2- 172.21.15.44:0/1358974492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140cdb30 0x7f4214167520 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 -- 172.21.15.44:0/1358974492 >> 172.21.15.44:0/1358974492 conn(0x7f42140d1dc0 msgr2=0x7f4214075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 -- 172.21.15.44:0/1358974492 shutdown_connections 2024-09-13T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.232+0000 7f421b600640 1 -- 172.21.15.44:0/1358974492 wait complete. 2024-09-13T08:46:54.255 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:46:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 -- 172.21.15.44:0/2406107224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe00b56a0 msgr2=0x7f2fe0070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 --2- 172.21.15.44:0/2406107224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe00b56a0 0x7f2fe0070950 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0069da0 tx=0x7f2fd801cd80 comp rx=0 tx=0).stop 2024-09-13T08:46:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 -- 172.21.15.44:0/2406107224 shutdown_connections 2024-09-13T08:46:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 --2- 172.21.15.44:0/2406107224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe00b56a0 0x7f2fe0070950 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 --2- 172.21.15.44:0/2406107224 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fe00b0ce0 0x7f2fe00b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 --2- 172.21.15.44:0/2406107224 >> v2:172.21.15.129:3300/0 conn(0x7f2fe00b4d80 0x7f2fe00b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 -- 172.21.15.44:0/2406107224 >> 172.21.15.44:0/2406107224 conn(0x7f2fe009aa10 msgr2=0x7f2fe009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 -- 172.21.15.44:0/2406107224 shutdown_connections 2024-09-13T08:46:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 -- 172.21.15.44:0/2406107224 wait complete. 2024-09-13T08:46:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 Processor -- start 2024-09-13T08:46:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.400+0000 7f2fdfe00640 1 -- start start 2024-09-13T08:46:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe00b0ce0 0x7f2fe014f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdfe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fe00b4d80 0x7f2fe013c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdfe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fe00b56a0 0x7f2fe015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe0167800 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdfe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe0167680 con 0x7f2fe00b4d80 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdfe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe0167980 con 0x7f2fe00b56a0 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe00b0ce0 0x7f2fe014f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe00b0ce0 0x7f2fe014f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43906/0 (socket says 172.21.15.44:43906) 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdea00640 1 -- 172.21.15.44:0/1391231134 learned_addr learned my addr 172.21.15.44:0/1391231134 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fde000640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fe00b4d80 0x7f2fe013c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdf400640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fe00b56a0 0x7f2fe015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdea00640 1 -- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fe00b56a0 msgr2=0x7f2fe015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdea00640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fe00b56a0 0x7f2fe015a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdea00640 1 -- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fe00b4d80 msgr2=0x7f2fe013c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdea00640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fe00b4d80 0x7f2fe013c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdea00640 1 -- 172.21.15.44:0/1391231134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe015a220 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdea00640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe00b0ce0 0x7f2fe014f020 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000d980 tx=0x7f2fd000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:54.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fd6a00640 1 -- 172.21.15.44:0/1391231134 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0017030 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fd6a00640 1 -- 172.21.15.44:0/1391231134 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fd0009b20 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fd6a00640 1 -- 172.21.15.44:0/1391231134 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0004020 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdfe00640 1 -- 172.21.15.44:0/1391231134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe01523d0 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdfe00640 1 -- 172.21.15.44:0/1391231134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe0152990 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.404+0000 7f2fdfe00640 1 -- 172.21.15.44:0/1391231134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe00b11a0 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.408+0000 7f2fd6a00640 1 -- 172.21.15.44:0/1391231134 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f2fd001a070 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.408+0000 7f2fd6a00640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2fc0074060 0x7f2fc0076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.408+0000 7f2fd6a00640 1 -- 172.21.15.44:0/1391231134 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5908+0+0 (secure 0 0 0) 0x7f2fd0092d70 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.408+0000 7f2fde000640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2fc0074060 0x7f2fc0076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.408+0000 7f2fde000640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2fc0074060 0x7f2fc0076520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2fe01401b0 tx=0x7f2fcc00b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.412+0000 7f2fd6a00640 1 -- 172.21.15.44:0/1391231134 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2fd005f0d0 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.588+0000 7f2fdfe00640 1 -- 172.21.15.44:0/1391231134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fe00ce0c0 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.588+0000 7f2fd6a00640 1 -- 172.21.15.44:0/1391231134 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+17022 (secure 0 0 0) 0x7f2fd0063b00 con 0x7f2fe00b0ce0 2024-09-13T08:46:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:54.592 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":78,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:46:50.129011+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:45:30.771920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T08:45:32.014594+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":78,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.c","mappings":[{"from":2,"to":7}]},{"pgid":"11.1","mappings":[{"from":1,"to":4}]},{"pgid":"11.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:46:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 -- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2fc0074060 msgr2=0x7f2fc0076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2fc0074060 0x7f2fc0076520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2fe01401b0 tx=0x7f2fcc00b390 comp rx=0 tx=0).stop 2024-09-13T08:46:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 -- 172.21.15.44:0/1391231134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe00b0ce0 msgr2=0x7f2fe014f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe00b0ce0 0x7f2fe014f020 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000d980 tx=0x7f2fd000de40 comp rx=0 tx=0).stop 2024-09-13T08:46:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 -- 172.21.15.44:0/1391231134 shutdown_connections 2024-09-13T08:46:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2fc0074060 0x7f2fc0076520 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2fe00b56a0 0x7f2fe015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fe00b4d80 0x7f2fe013c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 --2- 172.21.15.44:0/1391231134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe00b0ce0 0x7f2fe014f020 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 -- 172.21.15.44:0/1391231134 >> 172.21.15.44:0/1391231134 conn(0x7f2fe009aa10 msgr2=0x7f2fe00b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:54.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 -- 172.21.15.44:0/1391231134 shutdown_connections 2024-09-13T08:46:54.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.592+0000 7f2fdfe00640 1 -- 172.21.15.44:0/1391231134 wait complete. 2024-09-13T08:46:54.615 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T08:46:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 -- 172.21.15.44:0/3842757745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc8003a40 msgr2=0x7f0fc8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 --2- 172.21.15.44:0/3842757745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc8003a40 0x7f0fc8003e60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400bb10 tx=0x7f0fd401cda0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0fe8c00640 1 -- 172.21.15.44:0/3842757745 reap_dead start 2024-09-13T08:46:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 -- 172.21.15.44:0/3842757745 shutdown_connections 2024-09-13T08:46:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 -- 172.21.15.44:0/3842757745 >> 172.21.15.44:0/3842757745 conn(0x7f0fe40b18f0 msgr2=0x7f0fe40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 -- 172.21.15.44:0/3842757745 shutdown_connections 2024-09-13T08:46:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 -- 172.21.15.44:0/3842757745 wait complete. 2024-09-13T08:46:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 Processor -- start 2024-09-13T08:46:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 -- start start 2024-09-13T08:46:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fc8003a40 0x7f0fe40addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe415cf00 0x7f0fe4147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fe4148500 0x7f0fe41fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe4166120 con 0x7f0fe4148500 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe40cf4f0 con 0x7f0fe415cf00 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0feac00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe41662a0 con 0x7f0fc8003a40 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.764+0000 7f0fe8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fe4148500 0x7f0fe41fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fe4148500 0x7f0fe41fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43930/0 (socket says 172.21.15.44:43930) 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe8c00640 1 -- 172.21.15.44:0/1635026122 learned_addr learned my addr 172.21.15.44:0/1635026122 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fdbe00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe415cf00 0x7f0fe4147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe3e00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fc8003a40 0x7f0fe40addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe8c00640 1 -- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fc8003a40 msgr2=0x7f0fe40addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe8c00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fc8003a40 0x7f0fe40addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe8c00640 1 -- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe415cf00 msgr2=0x7f0fe4147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe8c00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe415cf00 0x7f0fe4147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe8c00640 1 -- 172.21.15.44:0/1635026122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe415c140 con 0x7f0fe4148500 2024-09-13T08:46:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fdbe00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe415cf00 0x7f0fe4147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe8c00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fe4148500 0x7f0fe41fba10 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0fe406aab0 tx=0x7f0fdc00eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe0c00640 1 -- 172.21.15.44:0/1635026122 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc015020 con 0x7f0fe4148500 2024-09-13T08:46:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe0c00640 1 -- 172.21.15.44:0/1635026122 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fdc00c9e0 con 0x7f0fe4148500 2024-09-13T08:46:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe0c00640 1 -- 172.21.15.44:0/1635026122 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00ce30 con 0x7f0fe4148500 2024-09-13T08:46:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0feac00640 1 -- 172.21.15.44:0/1635026122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe415c420 con 0x7f0fe4148500 2024-09-13T08:46:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0feac00640 1 -- 172.21.15.44:0/1635026122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe4148970 con 0x7f0fe4148500 2024-09-13T08:46:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0feac00640 1 -- 172.21.15.44:0/1635026122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe4069410 con 0x7f0fe4148500 2024-09-13T08:46:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.768+0000 7f0fe0c00640 1 -- 172.21.15.44:0/1635026122 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f0fdc018020 con 0x7f0fe4148500 2024-09-13T08:46:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.772+0000 7f0fe0c00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fc0074030 0x7f0fc00764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.772+0000 7f0fe0c00640 1 -- 172.21.15.44:0/1635026122 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5908+0+0 (secure 0 0 0) 0x7f0fdc093270 con 0x7f0fe4148500 2024-09-13T08:46:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.772+0000 7f0fe3e00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fc0074030 0x7f0fc00764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.772+0000 7f0fe3e00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fc0074030 0x7f0fc00764f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40b0000 tx=0x7f0fd40a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:54.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.772+0000 7f0fe0c00640 1 -- 172.21.15.44:0/1635026122 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0fdc01f020 con 0x7f0fe4148500 2024-09-13T08:46:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.984+0000 7f0feac00640 1 -- 172.21.15.44:0/1635026122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0fe40ce810 con 0x7f0fe4148500 2024-09-13T08:46:54.986 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T08:46:54.984+0000 7f03b3c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T08:46:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.984+0000 7f0fe0c00640 1 -- 172.21.15.44:0/1635026122 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f0fdc05fbc0 con 0x7f0fe4148500 2024-09-13T08:46:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T08:46:54.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 -- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fc0074030 msgr2=0x7f0fc00764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fc0074030 0x7f0fc00764f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40b0000 tx=0x7f0fd40a7040 comp rx=0 tx=0).stop 2024-09-13T08:46:54.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 -- 172.21.15.44:0/1635026122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fe4148500 msgr2=0x7f0fe41fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fe4148500 0x7f0fe41fba10 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0fe406aab0 tx=0x7f0fdc00eae0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 -- 172.21.15.44:0/1635026122 shutdown_connections 2024-09-13T08:46:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fc0074030 0x7f0fc00764f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fe4148500 0x7f0fe41fba10 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe415cf00 0x7f0fe4147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 --2- 172.21.15.44:0/1635026122 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fc8003a40 0x7f0fe40addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 -- 172.21.15.44:0/1635026122 >> 172.21.15.44:0/1635026122 conn(0x7f0fe40b18f0 msgr2=0x7f0fe40b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 -- 172.21.15.44:0/1635026122 shutdown_connections 2024-09-13T08:46:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:54.988+0000 7f0feac00640 1 -- 172.21.15.44:0/1635026122 wait complete. 2024-09-13T08:46:55.014 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T08:46:55.056 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:46:55.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 -- 172.21.15.44:0/1544609116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f93f80d04d0 msgr2=0x7f93f80af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 --2- 172.21.15.44:0/1544609116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f93f80d04d0 0x7f93f80af200 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009c80 tx=0x7f93f001f5a0 comp rx=0 tx=0).stop 2024-09-13T08:46:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 -- 172.21.15.44:0/1544609116 shutdown_connections 2024-09-13T08:46:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 --2- 172.21.15.44:0/1544609116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f93f80d04d0 0x7f93f80af200 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 --2- 172.21.15.44:0/1544609116 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f93f80d00f0 0x7f93f80b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 -- 172.21.15.44:0/1544609116 >> 172.21.15.44:0/1544609116 conn(0x7f93f80d1e90 msgr2=0x7f93f80d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 -- 172.21.15.44:0/1544609116 shutdown_connections 2024-09-13T08:46:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 -- 172.21.15.44:0/1544609116 wait complete. 2024-09-13T08:46:55.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 Processor -- start 2024-09-13T08:46:55.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 -- start start 2024-09-13T08:46:55.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93f80d00f0 0x7f93f814eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:55.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f93f80d04d0 0x7f93f80b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:55.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f93f80d29b0 0x7f93f80b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:55.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f93f80b0ed0 con 0x7f93f80d00f0 2024-09-13T08:46:55.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f93f80b0d50 con 0x7f93f80d04d0 2024-09-13T08:46:55.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93ff800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f93f80af740 con 0x7f93f80d29b0 2024-09-13T08:46:55.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93fce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93f80d00f0 0x7f93f814eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:55.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93fce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93f80d00f0 0x7f93f814eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43954/0 (socket says 172.21.15.44:43954) 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93fce00640 1 -- 172.21.15.44:0/915870907 learned_addr learned my addr 172.21.15.44:0/915870907 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93f7e00640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f93f80d04d0 0x7f93f80b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93fd800640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f93f80d29b0 0x7f93f80b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93fce00640 1 -- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f93f80d29b0 msgr2=0x7f93f80b73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93fce00640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f93f80d29b0 0x7f93f80b73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93fce00640 1 -- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f93f80d04d0 msgr2=0x7f93f80b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93fce00640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f93f80d04d0 0x7f93f80b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93fce00640 1 -- 172.21.15.44:0/915870907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f815aba0 con 0x7f93f80d00f0 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.168+0000 7f93fce00640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93f80d00f0 0x7f93f814eea0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00e9f0 tx=0x7f93ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.172+0000 7f93f4c00640 1 -- 172.21.15.44:0/915870907 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00cd80 con 0x7f93f80d00f0 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.172+0000 7f93f4c00640 1 -- 172.21.15.44:0/915870907 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93ec010430 con 0x7f93f80d00f0 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.172+0000 7f93f4c00640 1 -- 172.21.15.44:0/915870907 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec010860 con 0x7f93f80d00f0 2024-09-13T08:46:55.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.172+0000 7f93ff800640 1 -- 172.21.15.44:0/915870907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f815ae70 con 0x7f93f80d00f0 2024-09-13T08:46:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.172+0000 7f93ff800640 1 -- 172.21.15.44:0/915870907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f815b710 con 0x7f93f80d00f0 2024-09-13T08:46:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.172+0000 7f93ff800640 1 -- 172.21.15.44:0/915870907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f80af200 con 0x7f93f80d00f0 2024-09-13T08:46:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.172+0000 7f93f4c00640 1 -- 172.21.15.44:0/915870907 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f93ec010a00 con 0x7f93f80d00f0 2024-09-13T08:46:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.172+0000 7f93f4c00640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f93d8074060 0x7f93d8076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.172+0000 7f93f4c00640 1 -- 172.21.15.44:0/915870907 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5966+0+0 (secure 0 0 0) 0x7f93ec093820 con 0x7f93f80d00f0 2024-09-13T08:46:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.172+0000 7f93f7e00640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f93d8074060 0x7f93d8076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.176+0000 7f93f7e00640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f93d8074060 0x7f93d8076520 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f93f81f81a0 tx=0x7f93e4009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:55.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.176+0000 7f93f4c00640 1 -- 172.21.15.44:0/915870907 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f93ec05fbe0 con 0x7f93f80d00f0 2024-09-13T08:46:55.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.392+0000 7f93ff800640 1 -- 172.21.15.44:0/915870907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f93f806f470 con 0x7f93f80d00f0 2024-09-13T08:46:55.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.988+0000 7f93f4c00640 1 -- 172.21.15.44:0/915870907 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7f93ec064610 con 0x7f93f80d00f0 2024-09-13T08:46:55.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 -- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f93d8074060 msgr2=0x7f93d8076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:55.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f93d8074060 0x7f93d8076520 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f93f81f81a0 tx=0x7f93e4009060 comp rx=0 tx=0).stop 2024-09-13T08:46:55.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 -- 172.21.15.44:0/915870907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93f80d00f0 msgr2=0x7f93f814eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:55.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93f80d00f0 0x7f93f814eea0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00e9f0 tx=0x7f93ec00eeb0 comp rx=0 tx=0).stop 2024-09-13T08:46:55.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 -- 172.21.15.44:0/915870907 shutdown_connections 2024-09-13T08:46:55.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f93d8074060 0x7f93d8076520 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:55.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f93f80d29b0 0x7f93f80b73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:55.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f93f80d04d0 0x7f93f80b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:55.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 --2- 172.21.15.44:0/915870907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93f80d00f0 0x7f93f814eea0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:55.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 -- 172.21.15.44:0/915870907 >> 172.21.15.44:0/915870907 conn(0x7f93f80d1e90 msgr2=0x7f93f80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:55.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 -- 172.21.15.44:0/915870907 shutdown_connections 2024-09-13T08:46:55.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:55.992+0000 7f93ff800640 1 -- 172.21.15.44:0/915870907 wait complete. 2024-09-13T08:46:56.018 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T08:46:56.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.216+0000 7f8b83a00640 1 -- 172.21.15.44:0/2626110161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b7c15f390 msgr2=0x7f8b7c0cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:56.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.216+0000 7f8b83a00640 1 --2- 172.21.15.44:0/2626110161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b7c15f390 0x7f8b7c0cfc30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0cc4a0 tx=0x7f8b7001c890 comp rx=0 tx=0).stop 2024-09-13T08:46:56.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.216+0000 7f8b83a00640 1 -- 172.21.15.44:0/2626110161 shutdown_connections 2024-09-13T08:46:56.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.216+0000 7f8b83a00640 1 --2- 172.21.15.44:0/2626110161 >> v2:172.21.15.129:3300/0 conn(0x7f8b7c0d28e0 0x7f8b7c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:56.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.216+0000 7f8b83a00640 1 --2- 172.21.15.44:0/2626110161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b7c15f390 0x7f8b7c0cfc30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:56.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.216+0000 7f8b83a00640 1 -- 172.21.15.44:0/2626110161 >> 172.21.15.44:0/2626110161 conn(0x7f8b7c0d1dc0 msgr2=0x7f8b7c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:56.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.216+0000 7f8b83a00640 1 -- 172.21.15.44:0/2626110161 shutdown_connections 2024-09-13T08:46:56.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.216+0000 7f8b83a00640 1 -- 172.21.15.44:0/2626110161 wait complete. 2024-09-13T08:46:56.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.216+0000 7f8b83a00640 1 Processor -- start 2024-09-13T08:46:56.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.216+0000 7f8b83a00640 1 -- start start 2024-09-13T08:46:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b83a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b7c0d06b0 0x7f8b7c153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b83a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b7c0d28e0 0x7f8b7c14ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b83a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b7c15f390 0x7f8b7c1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b83a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c168f30 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b83a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c168db0 con 0x7f8b7c15f390 2024-09-13T08:46:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b83a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c1690b0 con 0x7f8b7c0d06b0 2024-09-13T08:46:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b7be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b7c0d28e0 0x7f8b7c14ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b7be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b7c0d28e0 0x7f8b7c14ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43964/0 (socket says 172.21.15.44:43964) 2024-09-13T08:46:56.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b7be00640 1 -- 172.21.15.44:0/643459749 learned_addr learned my addr 172.21.15.44:0/643459749 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:56.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b81000640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b7c0d06b0 0x7f8b7c153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:56.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b81a00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b7c15f390 0x7f8b7c1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:56.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b7be00640 1 -- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b7c0d06b0 msgr2=0x7f8b7c153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:56.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b7be00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b7c0d06b0 0x7f8b7c153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:56.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b7be00640 1 -- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b7c15f390 msgr2=0x7f8b7c1554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:56.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b7be00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b7c15f390 0x7f8b7c1554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:56.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b7be00640 1 -- 172.21.15.44:0/643459749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c0b4e60 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b7be00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b7c0d28e0 0x7f8b7c14ff00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800b580 tx=0x7f8b6800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:56.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b78c00640 1 -- 172.21.15.44:0/643459749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68014070 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b78c00640 1 -- 172.21.15.44:0/643459749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b68010980 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b83a00640 1 -- 172.21.15.44:0/643459749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7c0b5140 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b78c00640 1 -- 172.21.15.44:0/643459749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68010dd0 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b83a00640 1 -- 172.21.15.44:0/643459749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7c0b5700 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.220+0000 7f8b83a00640 1 -- 172.21.15.44:0/643459749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b7c0b1350 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.224+0000 7f8b78c00640 1 -- 172.21.15.44:0/643459749 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f8b68017070 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.224+0000 7f8b78c00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b5c074060 0x7f8b5c076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:56.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.224+0000 7f8b78c00640 1 -- 172.21.15.44:0/643459749 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5966+0+0 (secure 0 0 0) 0x7f8b68092b50 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.224+0000 7f8b81000640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b5c074060 0x7f8b5c076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:56.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.224+0000 7f8b81000640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b5c074060 0x7f8b5c076520 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8b700af000 tx=0x7f8b7001b0d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:56.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.228+0000 7f8b78c00640 1 -- 172.21.15.44:0/643459749 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8b6805edd0 con 0x7f8b7c0d28e0 2024-09-13T08:46:56.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:56.404+0000 7f8b83a00640 1 -- 172.21.15.44:0/643459749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8b7c1fb300 con 0x7f8b7c0d28e0 2024-09-13T08:46:57.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.172+0000 7f8b78c00640 1 -- 172.21.15.44:0/643459749 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) ==== 181+0+0 (secure 0 0 0) 0x7f8b68063800 con 0x7f8b7c0d28e0 2024-09-13T08:46:57.173 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T08:46:57.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 -- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b5c074060 msgr2=0x7f8b5c076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b5c074060 0x7f8b5c076520 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8b700af000 tx=0x7f8b7001b0d0 comp rx=0 tx=0).stop 2024-09-13T08:46:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 -- 172.21.15.44:0/643459749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b7c0d28e0 msgr2=0x7f8b7c14ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b7c0d28e0 0x7f8b7c14ff00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800b580 tx=0x7f8b6800ba40 comp rx=0 tx=0).stop 2024-09-13T08:46:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 -- 172.21.15.44:0/643459749 shutdown_connections 2024-09-13T08:46:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b5c074060 0x7f8b5c076520 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b7c15f390 0x7f8b7c1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b7c0d28e0 0x7f8b7c14ff00 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 --2- 172.21.15.44:0/643459749 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b7c0d06b0 0x7f8b7c153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 -- 172.21.15.44:0/643459749 >> 172.21.15.44:0/643459749 conn(0x7f8b7c0d1dc0 msgr2=0x7f8b7c072310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 -- 172.21.15.44:0/643459749 shutdown_connections 2024-09-13T08:46:57.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.176+0000 7f8b83a00640 1 -- 172.21.15.44:0/643459749 wait complete. 2024-09-13T08:46:57.204 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T08:46:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.356+0000 7f9e4d800640 1 -- 172.21.15.44:0/84377550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e480747a0 msgr2=0x7f9e4815e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.356+0000 7f9e4d800640 1 --2- 172.21.15.44:0/84377550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e480747a0 0x7f9e4815e410 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9e480b0880 tx=0x7f9e3001c6e0 comp rx=0 tx=0).stop 2024-09-13T08:46:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.356+0000 7f9e4d800640 1 -- 172.21.15.44:0/84377550 shutdown_connections 2024-09-13T08:46:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.356+0000 7f9e4d800640 1 --2- 172.21.15.44:0/84377550 >> v2:172.21.15.129:3300/0 conn(0x7f9e480d1110 0x7f9e480d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.356+0000 7f9e4d800640 1 --2- 172.21.15.44:0/84377550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e480747a0 0x7f9e4815e410 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.356+0000 7f9e4d800640 1 -- 172.21.15.44:0/84377550 >> 172.21.15.44:0/84377550 conn(0x7f9e4806dd30 msgr2=0x7f9e4806e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.356+0000 7f9e4d800640 1 -- 172.21.15.44:0/84377550 shutdown_connections 2024-09-13T08:46:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.356+0000 7f9e4d800640 1 -- 172.21.15.44:0/84377550 wait complete. 2024-09-13T08:46:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e4d800640 1 Processor -- start 2024-09-13T08:46:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e4d800640 1 -- start start 2024-09-13T08:46:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e4d800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e4806d200 0x7f9e4813adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e4d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e480747a0 0x7f9e48158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e4d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e480d1110 0x7f9e48159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e4d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4816ada0 con 0x7f9e480d1110 2024-09-13T08:46:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e4d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9e480b5f70 con 0x7f9e480747a0 2024-09-13T08:46:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e4d800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4816af20 con 0x7f9e4806d200 2024-09-13T08:46:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e47400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e480d1110 0x7f9e48159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e47400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e480d1110 0x7f9e48159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43978/0 (socket says 172.21.15.44:43978) 2024-09-13T08:46:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e47400640 1 -- 172.21.15.44:0/735656782 learned_addr learned my addr 172.21.15.44:0/735656782 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e3fe00640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e480747a0 0x7f9e48158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e46a00640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e4806d200 0x7f9e4813adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e47400640 1 -- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e4806d200 msgr2=0x7f9e4813adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e47400640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e4806d200 0x7f9e4813adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e47400640 1 -- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e480747a0 msgr2=0x7f9e48158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e47400640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e480747a0 0x7f9e48158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e47400640 1 -- 172.21.15.44:0/735656782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e481fad20 con 0x7f9e480d1110 2024-09-13T08:46:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e47400640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e480d1110 0x7f9e48159010 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000e990 tx=0x7f9e4000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e3ea00640 1 -- 172.21.15.44:0/735656782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40004d30 con 0x7f9e480d1110 2024-09-13T08:46:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e4d800640 1 -- 172.21.15.44:0/735656782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e481fb000 con 0x7f9e480d1110 2024-09-13T08:46:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e3ea00640 1 -- 172.21.15.44:0/735656782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e4000cb00 con 0x7f9e480d1110 2024-09-13T08:46:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.360+0000 7f9e3ea00640 1 -- 172.21.15.44:0/735656782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40005020 con 0x7f9e480d1110 2024-09-13T08:46:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.364+0000 7f9e4d800640 1 -- 172.21.15.44:0/735656782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e481fb5c0 con 0x7f9e480d1110 2024-09-13T08:46:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.364+0000 7f9e4d800640 1 -- 172.21.15.44:0/735656782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e4806f810 con 0x7f9e480d1110 2024-09-13T08:46:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.364+0000 7f9e3ea00640 1 -- 172.21.15.44:0/735656782 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f9e400051c0 con 0x7f9e480d1110 2024-09-13T08:46:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.364+0000 7f9e3ea00640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e24074030 0x7f9e240764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.364+0000 7f9e3ea00640 1 -- 172.21.15.44:0/735656782 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5499+0+0 (secure 0 0 0) 0x7f9e400935b0 con 0x7f9e480d1110 2024-09-13T08:46:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.364+0000 7f9e46a00640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e24074030 0x7f9e240764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.368+0000 7f9e46a00640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e24074030 0x7f9e240764f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9e380098e0 tx=0x7f9e38009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.368+0000 7f9e3ea00640 1 -- 172.21.15.44:0/735656782 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9e40016030 con 0x7f9e480d1110 2024-09-13T08:46:57.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:57.552+0000 7f9e4d800640 1 -- 172.21.15.44:0/735656782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9e4813b2f0 con 0x7f9e480d1110 2024-09-13T08:46:58.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.172+0000 7f9e3ea00640 1 -- 172.21.15.44:0/735656782 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) ==== 169+0+0 (secure 0 0 0) 0x7f9e4005f890 con 0x7f9e480d1110 2024-09-13T08:46:58.175 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 -- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e24074030 msgr2=0x7f9e240764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e24074030 0x7f9e240764f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9e380098e0 tx=0x7f9e38009060 comp rx=0 tx=0).stop 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 -- 172.21.15.44:0/735656782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e480d1110 msgr2=0x7f9e48159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e480d1110 0x7f9e48159010 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000e990 tx=0x7f9e4000ee50 comp rx=0 tx=0).stop 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 -- 172.21.15.44:0/735656782 shutdown_connections 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e24074030 0x7f9e240764f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e480d1110 0x7f9e48159010 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e480747a0 0x7f9e48158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 --2- 172.21.15.44:0/735656782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e4806d200 0x7f9e4813adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 -- 172.21.15.44:0/735656782 >> 172.21.15.44:0/735656782 conn(0x7f9e4806dd30 msgr2=0x7f9e480add50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 -- 172.21.15.44:0/735656782 shutdown_connections 2024-09-13T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.180+0000 7f9e4d800640 1 -- 172.21.15.44:0/735656782 wait complete. 2024-09-13T08:46:58.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-13T08:46:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 -- 172.21.15.44:0/671815705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6f4004810 msgr2=0x7fe6f4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 --2- 172.21.15.44:0/671815705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6f4004810 0x7fe6f4004c90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009b30 tx=0x7fe6f801d1e0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe70fe00640 1 -- 172.21.15.44:0/671815705 reap_dead start 2024-09-13T08:46:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 -- 172.21.15.44:0/671815705 shutdown_connections 2024-09-13T08:46:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 -- 172.21.15.44:0/671815705 >> 172.21.15.44:0/671815705 conn(0x7fe7100d1e30 msgr2=0x7fe7100d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:58.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 -- 172.21.15.44:0/671815705 shutdown_connections 2024-09-13T08:46:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 -- 172.21.15.44:0/671815705 wait complete. 2024-09-13T08:46:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 Processor -- start 2024-09-13T08:46:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 -- start start 2024-09-13T08:46:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f4004810 0x7fe7100c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7101f8a30 0x7fe7101f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7100cead0 0x7fe7100cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7100b1f60 con 0x7fe7100cead0 2024-09-13T08:46:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7100b1de0 con 0x7fe6f4004810 2024-09-13T08:46:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe716800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe7100b20e0 con 0x7fe7101f8a30 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe70fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7100cead0 0x7fe7100cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe70fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7100cead0 0x7fe7100cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44006/0 (socket says 172.21.15.44:44006) 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe70fe00640 1 -- 172.21.15.44:0/3074682264 learned_addr learned my addr 172.21.15.44:0/3074682264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe70f400640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f4004810 0x7fe7100c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe706a00640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7101f8a30 0x7fe7101f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe70fe00640 1 -- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7101f8a30 msgr2=0x7fe7101f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe70fe00640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7101f8a30 0x7fe7101f93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.360+0000 7fe70fe00640 1 -- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f4004810 msgr2=0x7fe7100c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe70fe00640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f4004810 0x7fe7100c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe70fe00640 1 -- 172.21.15.44:0/3074682264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7101f98e0 con 0x7fe7100cead0 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe70f400640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f4004810 0x7fe7100c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe706a00640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7101f8a30 0x7fe7101f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:46:58.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe70fe00640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7100cead0 0x7fe7100cef50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe7100af470 tx=0x7fe708010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:58.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe707e00640 1 -- 172.21.15.44:0/3074682264 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708003980 con 0x7fe7100cead0 2024-09-13T08:46:58.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe707e00640 1 -- 172.21.15.44:0/3074682264 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe708015070 con 0x7fe7100cead0 2024-09-13T08:46:58.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe716800640 1 -- 172.21.15.44:0/3074682264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7100c9a30 con 0x7fe7100cead0 2024-09-13T08:46:58.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe707e00640 1 -- 172.21.15.44:0/3074682264 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708004030 con 0x7fe7100cead0 2024-09-13T08:46:58.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe716800640 1 -- 172.21.15.44:0/3074682264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7100c9ff0 con 0x7fe7100cead0 2024-09-13T08:46:58.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe716800640 1 -- 172.21.15.44:0/3074682264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7100b0130 con 0x7fe7100cead0 2024-09-13T08:46:58.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.364+0000 7fe707e00640 1 -- 172.21.15.44:0/3074682264 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7fe7080041d0 con 0x7fe7100cead0 2024-09-13T08:46:58.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.368+0000 7fe707e00640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6ec074060 0x7fe6ec076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:58.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.368+0000 7fe707e00640 1 -- 172.21.15.44:0/3074682264 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5027+0+0 (secure 0 0 0) 0x7fe7080937d0 con 0x7fe7100cead0 2024-09-13T08:46:58.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.368+0000 7fe70f400640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6ec074060 0x7fe6ec076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:58.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.368+0000 7fe70f400640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6ec074060 0x7fe6ec076520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe700001870 tx=0x7fe700001ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:58.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.372+0000 7fe707e00640 1 -- 172.21.15.44:0/3074682264 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe708060120 con 0x7fe7100cead0 2024-09-13T08:46:58.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.552+0000 7fe716800640 1 -- 172.21.15.44:0/3074682264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fe71015aa90 con 0x7fe7100cead0 2024-09-13T08:46:58.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.556+0000 7fe707e00640 1 -- 172.21.15.44:0/3074682264 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v40) ==== 71+0+0 (secure 0 0 0) 0x7fe70805fa20 con 0x7fe7100cead0 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 -- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6ec074060 msgr2=0x7fe6ec076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6ec074060 0x7fe6ec076520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe700001870 tx=0x7fe700001ce0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 -- 172.21.15.44:0/3074682264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7100cead0 msgr2=0x7fe7100cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7100cead0 0x7fe7100cef50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe7100af470 tx=0x7fe708010f40 comp rx=0 tx=0).stop 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 -- 172.21.15.44:0/3074682264 shutdown_connections 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6ec074060 0x7fe6ec076520 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7100cead0 0x7fe7100cef50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7101f8a30 0x7fe7101f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 --2- 172.21.15.44:0/3074682264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f4004810 0x7fe7100c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 -- 172.21.15.44:0/3074682264 >> 172.21.15.44:0/3074682264 conn(0x7fe7100d1e30 msgr2=0x7fe71015b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 -- 172.21.15.44:0/3074682264 shutdown_connections 2024-09-13T08:46:58.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.560+0000 7fe716800640 1 -- 172.21.15.44:0/3074682264 wait complete. 2024-09-13T08:46:58.583 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-13T08:46:58.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 -- 172.21.15.44:0/3960531347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b540cf4e0 msgr2=0x7f4b540c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:58.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 --2- 172.21.15.44:0/3960531347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b540cf4e0 0x7f4b540c5dd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4b540d0a60 tx=0x7f4b4c01cc40 comp rx=0 tx=0).stop 2024-09-13T08:46:58.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 -- 172.21.15.44:0/3960531347 shutdown_connections 2024-09-13T08:46:58.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 --2- 172.21.15.44:0/3960531347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b540cf4e0 0x7f4b540c5dd0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 --2- 172.21.15.44:0/3960531347 >> v2:172.21.15.129:3300/0 conn(0x7f4b5415c130 0x7f4b540ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 --2- 172.21.15.44:0/3960531347 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b5415bd50 0x7f4b540ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 -- 172.21.15.44:0/3960531347 >> 172.21.15.44:0/3960531347 conn(0x7f4b540d1ea0 msgr2=0x7f4b540d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:58.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 -- 172.21.15.44:0/3960531347 shutdown_connections 2024-09-13T08:46:58.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 -- 172.21.15.44:0/3960531347 wait complete. 2024-09-13T08:46:58.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 Processor -- start 2024-09-13T08:46:58.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 -- start start 2024-09-13T08:46:58.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b540cf4e0 0x7f4b541fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:58.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5415bd50 0x7f4b541f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b5415c130 0x7f4b54158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540cccd0 con 0x7f4b5415bd50 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540ccb50 con 0x7f4b540cf4e0 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.728+0000 7f4b59e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4b540cce50 con 0x7f4b5415c130 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5415bd50 0x7f4b541f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5415bd50 0x7f4b541f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44024/0 (socket says 172.21.15.44:44024) 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4be00640 1 -- 172.21.15.44:0/931962374 learned_addr learned my addr 172.21.15.44:0/931962374 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b52a00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b540cf4e0 0x7f4b541fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b53400640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b5415c130 0x7f4b54158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4be00640 1 -- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b5415c130 msgr2=0x7f4b54158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4be00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b5415c130 0x7f4b54158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4be00640 1 -- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b540cf4e0 msgr2=0x7f4b541fb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:58.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4be00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b540cf4e0 0x7f4b541fb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:58.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4be00640 1 -- 172.21.15.44:0/931962374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b54166110 con 0x7f4b5415bd50 2024-09-13T08:46:58.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4be00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5415bd50 0x7f4b541f6cf0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00e7a0 tx=0x7f4b3c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:58.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4b400640 1 -- 172.21.15.44:0/931962374 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00ccc0 con 0x7f4b5415bd50 2024-09-13T08:46:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4b400640 1 -- 172.21.15.44:0/931962374 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b3c010430 con 0x7f4b5415bd50 2024-09-13T08:46:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4b400640 1 -- 172.21.15.44:0/931962374 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c010860 con 0x7f4b5415bd50 2024-09-13T08:46:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b59e00640 1 -- 172.21.15.44:0/931962374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5406e870 con 0x7f4b5415bd50 2024-09-13T08:46:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b59e00640 1 -- 172.21.15.44:0/931962374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b541f7990 con 0x7f4b5415bd50 2024-09-13T08:46:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b59e00640 1 -- 172.21.15.44:0/931962374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b540c5dd0 con 0x7f4b5415bd50 2024-09-13T08:46:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.732+0000 7f4b4b400640 1 -- 172.21.15.44:0/931962374 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f4b3c010a00 con 0x7f4b5415bd50 2024-09-13T08:46:58.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.736+0000 7f4b4b400640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b30074030 0x7f4b300764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:58.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.736+0000 7f4b4b400640 1 -- 172.21.15.44:0/931962374 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5027+0+0 (secure 0 0 0) 0x7f4b3c0936c0 con 0x7f4b5415bd50 2024-09-13T08:46:58.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.736+0000 7f4b52a00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b30074030 0x7f4b300764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.736+0000 7f4b52a00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b30074030 0x7f4b300764f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009b60 tx=0x7f4b440091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:58.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.740+0000 7f4b4b400640 1 -- 172.21.15.44:0/931962374 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4b3c05f9c0 con 0x7f4b5415bd50 2024-09-13T08:46:58.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:58.976+0000 7f4b59e00640 1 -- 172.21.15.44:0/931962374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7f4b54070c10 con 0x7f4b5415bd50 2024-09-13T08:46:59.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.184+0000 7f4b4b400640 1 -- 172.21.15.44:0/931962374 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v416) ==== 167+0+0 (secure 0 0 0) 0x7f4b3c0643f0 con 0x7f4b5415bd50 2024-09-13T08:46:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 -- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b30074030 msgr2=0x7f4b300764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b30074030 0x7f4b300764f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009b60 tx=0x7f4b440091e0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 -- 172.21.15.44:0/931962374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5415bd50 msgr2=0x7f4b541f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5415bd50 0x7f4b541f6cf0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00e7a0 tx=0x7f4b3c00ec60 comp rx=0 tx=0).stop 2024-09-13T08:46:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 -- 172.21.15.44:0/931962374 shutdown_connections 2024-09-13T08:46:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b30074030 0x7f4b300764f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b5415c130 0x7f4b54158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5415bd50 0x7f4b541f6cf0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 --2- 172.21.15.44:0/931962374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b540cf4e0 0x7f4b541fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 -- 172.21.15.44:0/931962374 >> 172.21.15.44:0/931962374 conn(0x7f4b540d1ea0 msgr2=0x7f4b540d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 -- 172.21.15.44:0/931962374 shutdown_connections 2024-09-13T08:46:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.188+0000 7f4b59e00640 1 -- 172.21.15.44:0/931962374 wait complete. 2024-09-13T08:46:59.210 INFO:tasks.cephfs_test_runner:Test that 2 progress bars are printed in output of "ceph status" ... ok 2024-09-13T08:46:59.211 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress indicated by progress bar for ongoing+pending clones 2024-09-13T08:46:59.211 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-13T08:46:59.211 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:46:59.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.360+0000 7fe2c1400640 1 -- 172.21.15.44:0/269971871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2bc15bf70 msgr2=0x7fe2bc0b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.360+0000 7fe2c1400640 1 --2- 172.21.15.44:0/269971871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2bc15bf70 0x7fe2bc0b53e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009c80 tx=0x7fe2b401cd10 comp rx=0 tx=0).stop 2024-09-13T08:46:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.360+0000 7fe2c1400640 1 -- 172.21.15.44:0/269971871 shutdown_connections 2024-09-13T08:46:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.360+0000 7fe2c1400640 1 --2- 172.21.15.44:0/269971871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2bc15bf70 0x7fe2bc0b53e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.360+0000 7fe2c1400640 1 --2- 172.21.15.44:0/269971871 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2bc15bb90 0x7fe2bc15f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.360+0000 7fe2c1400640 1 -- 172.21.15.44:0/269971871 >> 172.21.15.44:0/269971871 conn(0x7fe2bc0d1d90 msgr2=0x7fe2bc0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.360+0000 7fe2c1400640 1 -- 172.21.15.44:0/269971871 shutdown_connections 2024-09-13T08:46:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.360+0000 7fe2c1400640 1 -- 172.21.15.44:0/269971871 wait complete. 2024-09-13T08:46:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.360+0000 7fe2c1400640 1 Processor -- start 2024-09-13T08:46:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.360+0000 7fe2c1400640 1 -- start start 2024-09-13T08:46:59.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2c1400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2bc0d2940 0x7fe2bc14f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2c1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2bc15bb90 0x7fe2bc155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2c1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2bc15bf70 0x7fe2bc06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2c1400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc168ca0 con 0x7fe2bc15bf70 2024-09-13T08:46:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2c1400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc0b6640 con 0x7fe2bc15bb90 2024-09-13T08:46:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2c1400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe2bc168e20 con 0x7fe2bc0d2940 2024-09-13T08:46:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2bb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2bc15bf70 0x7fe2bc06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:59.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2bb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2bc15bf70 0x7fe2bc06e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44036/0 (socket says 172.21.15.44:44036) 2024-09-13T08:46:59.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2bb200640 1 -- 172.21.15.44:0/3801267354 learned_addr learned my addr 172.21.15.44:0/3801267354 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:59.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2ba800640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2bc0d2940 0x7fe2bc14f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:59.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2b3e00640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2bc15bb90 0x7fe2bc155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:59.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2bb200640 1 -- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2bc0d2940 msgr2=0x7fe2bc14f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2bb200640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2bc0d2940 0x7fe2bc14f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2bb200640 1 -- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2bc15bb90 msgr2=0x7fe2bc155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2bb200640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2bc15bb90 0x7fe2bc155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2bb200640 1 -- 172.21.15.44:0/3801267354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc06f9c0 con 0x7fe2bc15bf70 2024-09-13T08:46:59.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2bb200640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2bc15bf70 0x7fe2bc06e8b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe2b401cb40 tx=0x7fe2b40093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:59.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2b2a00640 1 -- 172.21.15.44:0/3801267354 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40b2040 con 0x7fe2bc15bf70 2024-09-13T08:46:59.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2b2a00640 1 -- 172.21.15.44:0/3801267354 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2b4004200 con 0x7fe2bc15bf70 2024-09-13T08:46:59.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2c1400640 1 -- 172.21.15.44:0/3801267354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc06fca0 con 0x7fe2bc15bf70 2024-09-13T08:46:59.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2b2a00640 1 -- 172.21.15.44:0/3801267354 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4004610 con 0x7fe2bc15bf70 2024-09-13T08:46:59.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2c1400640 1 -- 172.21.15.44:0/3801267354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc06e0b0 con 0x7fe2bc15bf70 2024-09-13T08:46:59.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.364+0000 7fe2c1400640 1 -- 172.21.15.44:0/3801267354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2bc0b53e0 con 0x7fe2bc15bf70 2024-09-13T08:46:59.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.368+0000 7fe2b2a00640 1 -- 172.21.15.44:0/3801267354 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7fe2b40a5bb0 con 0x7fe2bc15bf70 2024-09-13T08:46:59.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.368+0000 7fe2b2a00640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe298074060 0x7fe298076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:59.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.368+0000 7fe2b2a00640 1 -- 172.21.15.44:0/3801267354 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5027+0+0 (secure 0 0 0) 0x7fe2b412d5f0 con 0x7fe2bc15bf70 2024-09-13T08:46:59.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.368+0000 7fe2ba800640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe298074060 0x7fe298076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:59.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.368+0000 7fe2ba800640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe298074060 0x7fe298076520 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006fd0 tx=0x7fe2ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:59.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.372+0000 7fe2b2a00640 1 -- 172.21.15.44:0/3801267354 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe2b4016990 con 0x7fe2bc15bf70 2024-09-13T08:46:59.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 -- 172.21.15.44:0/3801267354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2bc153500 con 0x7fe2bc15bf70 2024-09-13T08:46:59.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2b2a00640 1 -- 172.21.15.44:0/3801267354 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+13175 (secure 0 0 0) 0x7fe2b40f9ec0 con 0x7fe2bc15bf70 2024-09-13T08:46:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:46:59.566 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":81,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:46:58.175096+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/651900305":"2024-09-14T08:46:54.987725+0000","172.21.15.44:6833/651900305":"2024-09-14T08:46:54.987725+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:46:59.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 -- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe298074060 msgr2=0x7fe298076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe298074060 0x7fe298076520 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006fd0 tx=0x7fe2ac008040 comp rx=0 tx=0).stop 2024-09-13T08:46:59.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 -- 172.21.15.44:0/3801267354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2bc15bf70 msgr2=0x7fe2bc06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2bc15bf70 0x7fe2bc06e8b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe2b401cb40 tx=0x7fe2b40093f0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 -- 172.21.15.44:0/3801267354 shutdown_connections 2024-09-13T08:46:59.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe298074060 0x7fe298076520 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2bc15bf70 0x7fe2bc06e8b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2bc15bb90 0x7fe2bc155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 --2- 172.21.15.44:0/3801267354 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2bc0d2940 0x7fe2bc14f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 -- 172.21.15.44:0/3801267354 >> 172.21.15.44:0/3801267354 conn(0x7fe2bc0d1d90 msgr2=0x7fe2bc070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:59.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 -- 172.21.15.44:0/3801267354 shutdown_connections 2024-09-13T08:46:59.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.564+0000 7fe2c1400640 1 -- 172.21.15.44:0/3801267354 wait complete. 2024-09-13T08:46:59.589 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T08:37:24.563344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T08:46:59.589 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T08:46:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.736+0000 7f9c09e00640 1 -- 172.21.15.44:0/292660671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0415f340 msgr2=0x7f9c040b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.736+0000 7f9c09e00640 1 --2- 172.21.15.44:0/292660671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0415f340 0x7f9c040b34a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9c04073ef0 tx=0x7f9bf401c6b0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.736+0000 7f9c09e00640 1 -- 172.21.15.44:0/292660671 shutdown_connections 2024-09-13T08:46:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.736+0000 7f9c09e00640 1 --2- 172.21.15.44:0/292660671 >> v2:172.21.15.129:3300/0 conn(0x7f9c040b2a00 0x7f9c040b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.736+0000 7f9c09e00640 1 --2- 172.21.15.44:0/292660671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0415f340 0x7f9c040b34a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.736+0000 7f9c09e00640 1 -- 172.21.15.44:0/292660671 >> 172.21.15.44:0/292660671 conn(0x7f9c040d1e00 msgr2=0x7f9c040d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.736+0000 7f9c09e00640 1 -- 172.21.15.44:0/292660671 shutdown_connections 2024-09-13T08:46:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.736+0000 7f9c09e00640 1 -- 172.21.15.44:0/292660671 wait complete. 2024-09-13T08:46:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 Processor -- start 2024-09-13T08:46:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 -- start start 2024-09-13T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c040b2a00 0x7f9c04164970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c040ccf10 0x7f9c04159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c0415f340 0x7f9c04159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04169050 con 0x7f9c040ccf10 2024-09-13T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04168ed0 con 0x7f9c0415f340 2024-09-13T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9c041691d0 con 0x7f9c040b2a00 2024-09-13T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c040ccf10 0x7f9c04159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c040ccf10 0x7f9c04159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44048/0 (socket says 172.21.15.44:44048) 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfbe00640 1 -- 172.21.15.44:0/1727267750 learned_addr learned my addr 172.21.15.44:0/1727267750 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c02a00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c040b2a00 0x7f9c04164970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfbe00640 1 -- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c040b2a00 msgr2=0x7f9c04164970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c03400640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c0415f340 0x7f9c04159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfbe00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c040b2a00 0x7f9c04164970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfbe00640 1 -- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c0415f340 msgr2=0x7f9c04159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfbe00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c0415f340 0x7f9c04159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfbe00640 1 -- 172.21.15.44:0/1727267750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c04075b50 con 0x7f9c040ccf10 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c02a00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c040b2a00 0x7f9c04164970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c03400640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c0415f340 0x7f9c04159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfbe00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c040ccf10 0x7f9c04159500 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00b580 tx=0x7f9bec00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfb400640 1 -- 172.21.15.44:0/1727267750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec014070 con 0x7f9c040ccf10 2024-09-13T08:46:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfb400640 1 -- 172.21.15.44:0/1727267750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bec010980 con 0x7f9c040ccf10 2024-09-13T08:46:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9bfb400640 1 -- 172.21.15.44:0/1727267750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec010dd0 con 0x7f9c040ccf10 2024-09-13T08:46:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 -- 172.21.15.44:0/1727267750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c041f8050 con 0x7f9c040ccf10 2024-09-13T08:46:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 -- 172.21.15.44:0/1727267750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c041f85d0 con 0x7f9c040ccf10 2024-09-13T08:46:59.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.740+0000 7f9c09e00640 1 -- 172.21.15.44:0/1727267750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c040cd340 con 0x7f9c040ccf10 2024-09-13T08:46:59.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.744+0000 7f9bfb400640 1 -- 172.21.15.44:0/1727267750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f9bec017070 con 0x7f9c040ccf10 2024-09-13T08:46:59.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.744+0000 7f9bfb400640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be0074030 0x7f9be00764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:46:59.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.744+0000 7f9bfb400640 1 -- 172.21.15.44:0/1727267750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5027+0+0 (secure 0 0 0) 0x7f9bec092c80 con 0x7f9c040ccf10 2024-09-13T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.744+0000 7f9c02a00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be0074030 0x7f9be00764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.744+0000 7f9c02a00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be0074030 0x7f9be00764f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40af000 tx=0x7f9bf401aef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:46:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.748+0000 7f9bfb400640 1 -- 172.21.15.44:0/1727267750 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9bec05f550 con 0x7f9c040ccf10 2024-09-13T08:46:59.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.920+0000 7f9c09e00640 1 -- 172.21.15.44:0/1727267750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9c0415aff0 con 0x7f9c040ccf10 2024-09-13T08:46:59.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9bfb400640 1 -- 172.21.15.44:0/1727267750 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f9bec05ee50 con 0x7f9c040ccf10 2024-09-13T08:46:59.925 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T08:46:59.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 -- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be0074030 msgr2=0x7f9be00764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be0074030 0x7f9be00764f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40af000 tx=0x7f9bf401aef0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 -- 172.21.15.44:0/1727267750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c040ccf10 msgr2=0x7f9c04159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c040ccf10 0x7f9c04159500 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00b580 tx=0x7f9bec00ba40 comp rx=0 tx=0).stop 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 -- 172.21.15.44:0/1727267750 shutdown_connections 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9be0074030 0x7f9be00764f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c0415f340 0x7f9c04159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c040ccf10 0x7f9c04159500 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 --2- 172.21.15.44:0/1727267750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9c040b2a00 0x7f9c04164970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 -- 172.21.15.44:0/1727267750 >> 172.21.15.44:0/1727267750 conn(0x7f9c040d1e00 msgr2=0x7f9c0415dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 -- 172.21.15.44:0/1727267750 shutdown_connections 2024-09-13T08:46:59.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:46:59.924+0000 7f9c09e00640 1 -- 172.21.15.44:0/1727267750 wait complete. 2024-09-13T08:46:59.946 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-13T08:47:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.092+0000 7f9910600640 1 -- 172.21.15.44:0/1064584796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99080d28e0 msgr2=0x7f99080cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.092+0000 7f9910600640 1 --2- 172.21.15.44:0/1064584796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99080d28e0 0x7f99080cc9d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f99080745d0 tx=0x7f990001cd80 comp rx=0 tx=0).stop 2024-09-13T08:47:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.092+0000 7f9910600640 1 -- 172.21.15.44:0/1064584796 shutdown_connections 2024-09-13T08:47:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.092+0000 7f9910600640 1 --2- 172.21.15.44:0/1064584796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99080d28e0 0x7f99080cc9d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.092+0000 7f9910600640 1 --2- 172.21.15.44:0/1064584796 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f99080cdb30 0x7f99080cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.092+0000 7f9910600640 1 --2- 172.21.15.44:0/1064584796 >> v2:172.21.15.129:3300/0 conn(0x7f990815bea0 0x7f99080ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.092+0000 7f9910600640 1 -- 172.21.15.44:0/1064584796 >> 172.21.15.44:0/1064584796 conn(0x7f99080d1dc0 msgr2=0x7f99080d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.092+0000 7f9910600640 1 -- 172.21.15.44:0/1064584796 shutdown_connections 2024-09-13T08:47:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.092+0000 7f9910600640 1 -- 172.21.15.44:0/1064584796 wait complete. 2024-09-13T08:47:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 Processor -- start 2024-09-13T08:47:00.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 -- start start 2024-09-13T08:47:00.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99080cdb30 0x7f99081524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99080d28e0 0x7f99080ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f990815bea0 0x7f9908157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99080af120 con 0x7f99080cdb30 2024-09-13T08:47:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99080aefa0 con 0x7f99080d28e0 2024-09-13T08:47:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f99080af2a0 con 0x7f990815bea0 2024-09-13T08:47:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f990dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99080cdb30 0x7f99081524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f990dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99080cdb30 0x7f99081524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44072/0 (socket says 172.21.15.44:44072) 2024-09-13T08:47:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f990dc00640 1 -- 172.21.15.44:0/969446893 learned_addr learned my addr 172.21.15.44:0/969446893 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:00.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f990dc00640 1 -- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f990815bea0 msgr2=0x7f9908157f80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:47:00.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f990dc00640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f990815bea0 0x7f9908157f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:00.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f990dc00640 1 -- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99080d28e0 msgr2=0x7f99080ca6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:47:00.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f990dc00640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99080d28e0 0x7f99080ca6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:00.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f990dc00640 1 -- 172.21.15.44:0/969446893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99080cac20 con 0x7f99080cdb30 2024-09-13T08:47:00.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f990dc00640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99080cdb30 0x7f99081524e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00e7a0 tx=0x7f98fc00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:00.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9905600640 1 -- 172.21.15.44:0/969446893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc00cc60 con 0x7f99080cdb30 2024-09-13T08:47:00.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9905600640 1 -- 172.21.15.44:0/969446893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98fc010430 con 0x7f99080cdb30 2024-09-13T08:47:00.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9905600640 1 -- 172.21.15.44:0/969446893 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc010860 con 0x7f99080cdb30 2024-09-13T08:47:00.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 -- 172.21.15.44:0/969446893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99081558c0 con 0x7f99080cdb30 2024-09-13T08:47:00.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 -- 172.21.15.44:0/969446893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9908155300 con 0x7f99080cdb30 2024-09-13T08:47:00.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.096+0000 7f9910600640 1 -- 172.21.15.44:0/969446893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99080af830 con 0x7f99080cdb30 2024-09-13T08:47:00.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.100+0000 7f9905600640 1 -- 172.21.15.44:0/969446893 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f98fc010a00 con 0x7f99080cdb30 2024-09-13T08:47:00.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.100+0000 7f9905600640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98e8074030 0x7f98e80764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:00.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.100+0000 7f9905600640 1 -- 172.21.15.44:0/969446893 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5027+0+0 (secure 0 0 0) 0x7f98fc093770 con 0x7f99080cdb30 2024-09-13T08:47:00.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.100+0000 7f990d200640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98e8074030 0x7f98e80764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:00.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.100+0000 7f990d200640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98e8074030 0x7f98e80764f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9908154c70 tx=0x7f98f40091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:00.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.104+0000 7f9905600640 1 -- 172.21.15.44:0/969446893 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f98fc09a030 con 0x7f99080cdb30 2024-09-13T08:47:00.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:00.340+0000 7f9910600640 1 -- 172.21.15.44:0/969446893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f99081566a0 con 0x7f99080cdb30 2024-09-13T08:47:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.192+0000 7f9905600640 1 -- 172.21.15.44:0/969446893 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v418) ==== 177+0+0 (secure 0 0 0) 0x7f98fc0600c0 con 0x7f99080cdb30 2024-09-13T08:47:01.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 -- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98e8074030 msgr2=0x7f98e80764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98e8074030 0x7f98e80764f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9908154c70 tx=0x7f98f40091e0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 -- 172.21.15.44:0/969446893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99080cdb30 msgr2=0x7f99081524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99080cdb30 0x7f99081524e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00e7a0 tx=0x7f98fc00ec60 comp rx=0 tx=0).stop 2024-09-13T08:47:01.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 -- 172.21.15.44:0/969446893 shutdown_connections 2024-09-13T08:47:01.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98e8074030 0x7f98e80764f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f990815bea0 0x7f9908157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99080d28e0 0x7f99080ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 --2- 172.21.15.44:0/969446893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99080cdb30 0x7f99081524e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 -- 172.21.15.44:0/969446893 >> 172.21.15.44:0/969446893 conn(0x7f99080d1dc0 msgr2=0x7f9908159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:01.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 -- 172.21.15.44:0/969446893 shutdown_connections 2024-09-13T08:47:01.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.196+0000 7f9910600640 1 -- 172.21.15.44:0/969446893 wait complete. 2024-09-13T08:47:01.223 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-13T08:47:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.372+0000 7f7f43800640 1 -- 172.21.15.44:0/1936015665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f3c0ced80 msgr2=0x7f7f3c0ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.372+0000 7f7f43800640 1 --2- 172.21.15.44:0/1936015665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f3c0ced80 0x7f7f3c0ae9b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009840 tx=0x7f7f2801c740 comp rx=0 tx=0).stop 2024-09-13T08:47:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.372+0000 7f7f43800640 1 -- 172.21.15.44:0/1936015665 shutdown_connections 2024-09-13T08:47:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.372+0000 7f7f43800640 1 --2- 172.21.15.44:0/1936015665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f3c0ced80 0x7f7f3c0ae9b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.372+0000 7f7f43800640 1 -- 172.21.15.44:0/1936015665 >> 172.21.15.44:0/1936015665 conn(0x7f7f3c0d1dc0 msgr2=0x7f7f3c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.372+0000 7f7f43800640 1 -- 172.21.15.44:0/1936015665 shutdown_connections 2024-09-13T08:47:01.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.372+0000 7f7f43800640 1 -- 172.21.15.44:0/1936015665 wait complete. 2024-09-13T08:47:01.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f43800640 1 Processor -- start 2024-09-13T08:47:01.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f43800640 1 -- start start 2024-09-13T08:47:01.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f43800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f3c0cdb30 0x7f7f3c14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:01.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f43800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0ced80 0x7f7f3c14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f43800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f3c15bea0 0x7f7f3c0cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f43800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c16a8a0 con 0x7f7f3c0ced80 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f43800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c16a720 con 0x7f7f3c0cdb30 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f43800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3c16aa20 con 0x7f7f3c15bea0 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0ced80 0x7f7f3c14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0ced80 0x7f7f3c14f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44074/0 (socket says 172.21.15.44:44074) 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f3be00640 1 -- 172.21.15.44:0/3899284563 learned_addr learned my addr 172.21.15.44:0/3899284563 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f41800640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f3c15bea0 0x7f7f3c0cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f40e00640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f3c0cdb30 0x7f7f3c14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f3be00640 1 -- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f3c15bea0 msgr2=0x7f7f3c0cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f3be00640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f3c15bea0 0x7f7f3c0cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f3be00640 1 -- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f3c0cdb30 msgr2=0x7f7f3c14fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f3be00640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f3c0cdb30 0x7f7f3c14fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f3be00640 1 -- 172.21.15.44:0/3899284563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c150430 con 0x7f7f3c0ced80 2024-09-13T08:47:01.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f41800640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f3c15bea0 0x7f7f3c0cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:47:01.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f3be00640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0ced80 0x7f7f3c14f6e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009790 tx=0x7f7f28005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:01.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f38c00640 1 -- 172.21.15.44:0/3899284563 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f280be070 con 0x7f7f3c0ced80 2024-09-13T08:47:01.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f38c00640 1 -- 172.21.15.44:0/3899284563 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f280040f0 con 0x7f7f3c0ced80 2024-09-13T08:47:01.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f38c00640 1 -- 172.21.15.44:0/3899284563 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f280046d0 con 0x7f7f3c0ced80 2024-09-13T08:47:01.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f43800640 1 -- 172.21.15.44:0/3899284563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c0c9bf0 con 0x7f7f3c0ced80 2024-09-13T08:47:01.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.376+0000 7f7f43800640 1 -- 172.21.15.44:0/3899284563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c153d40 con 0x7f7f3c0ced80 2024-09-13T08:47:01.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.380+0000 7f7f43800640 1 -- 172.21.15.44:0/3899284563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3c0ce670 con 0x7f7f3c0ced80 2024-09-13T08:47:01.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.380+0000 7f7f38c00640 1 -- 172.21.15.44:0/3899284563 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f7f28004870 con 0x7f7f3c0ced80 2024-09-13T08:47:01.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.380+0000 7f7f38c00640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f1c074030 0x7f7f1c0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:01.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.380+0000 7f7f38c00640 1 -- 172.21.15.44:0/3899284563 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5027+0+0 (secure 0 0 0) 0x7f7f2812cfe0 con 0x7f7f3c0ced80 2024-09-13T08:47:01.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.380+0000 7f7f40e00640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f1c074030 0x7f7f1c0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:01.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.384+0000 7f7f40e00640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f1c074030 0x7f7f1c0764f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7f30000f70 tx=0x7f7f30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:01.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.384+0000 7f7f38c00640 1 -- 172.21.15.44:0/3899284563 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7f280ac030 con 0x7f7f3c0ced80 2024-09-13T08:47:01.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.560+0000 7f7f43800640 1 -- 172.21.15.44:0/3899284563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f7f3c0b79a0 con 0x7f7f3c0ced80 2024-09-13T08:47:01.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.560+0000 7f7f38c00640 1 -- 172.21.15.44:0/3899284563 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v40) ==== 59+0+1998 (secure 0 0 0) 0x7f7f28016330 con 0x7f7f3c0ced80 2024-09-13T08:47:01.562 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-13T08:47:01.562 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4094.620852 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3964.014083 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3978.263337 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4002.298162 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4667.000623 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4664.046790 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6403.016405 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4696.908546 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-09-13T08:47:01.563 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 2/2 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-09-13T08:47:01.564 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-09-13T08:47:01.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 -- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f1c074030 msgr2=0x7f7f1c0764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f1c074030 0x7f7f1c0764f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7f30000f70 tx=0x7f7f30009040 comp rx=0 tx=0).stop 2024-09-13T08:47:01.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 -- 172.21.15.44:0/3899284563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0ced80 msgr2=0x7f7f3c14f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0ced80 0x7f7f3c14f6e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009790 tx=0x7f7f28005760 comp rx=0 tx=0).stop 2024-09-13T08:47:01.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 -- 172.21.15.44:0/3899284563 shutdown_connections 2024-09-13T08:47:01.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f1c074030 0x7f7f1c0764f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f3c15bea0 0x7f7f3c0cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0ced80 0x7f7f3c14f6e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 --2- 172.21.15.44:0/3899284563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f3c0cdb30 0x7f7f3c14fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 -- 172.21.15.44:0/3899284563 >> 172.21.15.44:0/3899284563 conn(0x7f7f3c0d1dc0 msgr2=0x7f7f3c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:01.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 -- 172.21.15.44:0/3899284563 shutdown_connections 2024-09-13T08:47:01.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.564+0000 7f7f43800640 1 -- 172.21.15.44:0/3899284563 wait complete. 2024-09-13T08:47:01.585 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:47:01.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 -- 172.21.15.44:0/1089147919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffad4004810 msgr2=0x7ffad4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 --2- 172.21.15.44:0/1089147919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffad4004810 0x7ffad4004c90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009b90 tx=0x7ffadc01d2b0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaef400640 1 -- 172.21.15.44:0/1089147919 reap_dead start 2024-09-13T08:47:01.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 -- 172.21.15.44:0/1089147919 shutdown_connections 2024-09-13T08:47:01.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 -- 172.21.15.44:0/1089147919 >> 172.21.15.44:0/1089147919 conn(0x7ffaf00cd740 msgr2=0x7ffaf00cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:01.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 -- 172.21.15.44:0/1089147919 shutdown_connections 2024-09-13T08:47:01.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 -- 172.21.15.44:0/1089147919 wait complete. 2024-09-13T08:47:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 Processor -- start 2024-09-13T08:47:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 -- start start 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffad4004810 0x7ffaf01f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaf00a19b0 0x7ffaf009f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf009f890 0x7ffaf009fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf00cc3b0 con 0x7ffaf009f890 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf006d590 con 0x7ffaf00a19b0 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaf5a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf00cc530 con 0x7ffad4004810 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf009f890 0x7ffaf009fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf009f890 0x7ffaf009fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44086/0 (socket says 172.21.15.44:44086) 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaef400640 1 -- 172.21.15.44:0/4171243049 learned_addr learned my addr 172.21.15.44:0/4171243049 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaef400640 1 -- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffad4004810 msgr2=0x7ffaf01f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaef400640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffad4004810 0x7ffaf01f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaef400640 1 -- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaf00a19b0 msgr2=0x7ffaf009f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaef400640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaf00a19b0 0x7ffaf009f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.736+0000 7ffaef400640 1 -- 172.21.15.44:0/4171243049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf00cf730 con 0x7ffaf009f890 2024-09-13T08:47:01.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffaef400640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf009f890 0x7ffaf009fd10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00cb2c0 tx=0x7ffae800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:01.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffae6a00640 1 -- 172.21.15.44:0/4171243049 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8004d30 con 0x7ffaf009f890 2024-09-13T08:47:01.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffae6a00640 1 -- 172.21.15.44:0/4171243049 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffae800cad0 con 0x7ffaf009f890 2024-09-13T08:47:01.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffaf5a00640 1 -- 172.21.15.44:0/4171243049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf00cfa10 con 0x7ffaf009f890 2024-09-13T08:47:01.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffae6a00640 1 -- 172.21.15.44:0/4171243049 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8005020 con 0x7ffaf009f890 2024-09-13T08:47:01.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffaf5a00640 1 -- 172.21.15.44:0/4171243049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf009f0b0 con 0x7ffaf009f890 2024-09-13T08:47:01.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffaf5a00640 1 -- 172.21.15.44:0/4171243049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf00cff30 con 0x7ffaf009f890 2024-09-13T08:47:01.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffae6a00640 1 -- 172.21.15.44:0/4171243049 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7ffae80051c0 con 0x7ffaf009f890 2024-09-13T08:47:01.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffae6a00640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 0x7ffad00764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:01.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffae6a00640 1 -- 172.21.15.44:0/4171243049 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5027+0+0 (secure 0 0 0) 0x7ffae80936f0 con 0x7ffaf009f890 2024-09-13T08:47:01.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.740+0000 7ffaeea00640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 0x7ffad00764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:01.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.744+0000 7ffaeea00640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 0x7ffad00764f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffae0001000 tx=0x7ffae00015b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:01.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.744+0000 7ffae6a00640 1 -- 172.21.15.44:0/4171243049 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ffaf00cff30 con 0x7ffaf009f890 2024-09-13T08:47:01.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.924+0000 7ffaf5a00640 1 -- 172.21.15.44:0/4171243049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ffaf01602f0 con 0x7ffaf009f890 2024-09-13T08:47:01.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffae6a00640 1 -- 172.21.15.44:0/4171243049 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7ffae8060040 con 0x7ffaf009f890 2024-09-13T08:47:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 -- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 msgr2=0x7ffad00764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 0x7ffad00764f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffae0001000 tx=0x7ffae00015b0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 -- 172.21.15.44:0/4171243049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf009f890 msgr2=0x7ffaf009fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf009f890 0x7ffaf009fd10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00cb2c0 tx=0x7ffae800ed80 comp rx=0 tx=0).stop 2024-09-13T08:47:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 -- 172.21.15.44:0/4171243049 shutdown_connections 2024-09-13T08:47:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 0x7ffad00764f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf009f890 0x7ffaf009fd10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaf00a19b0 0x7ffaf009f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 --2- 172.21.15.44:0/4171243049 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffad4004810 0x7ffaf01f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:01.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 -- 172.21.15.44:0/4171243049 >> 172.21.15.44:0/4171243049 conn(0x7ffaf00cd740 msgr2=0x7ffaf00b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:01.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 -- 172.21.15.44:0/4171243049 shutdown_connections 2024-09-13T08:47:01.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:01.928+0000 7ffaf5a00640 1 -- 172.21.15.44:0/4171243049 wait complete. 2024-09-13T08:47:01.951 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:47:01.951 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:47:02.051 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:47:02.147 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:47:02.147 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:47:02.147 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:47:02.156 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:47:02.157 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:47:02.157 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:47:02.166 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:47:02.166 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:47:02.166 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:47:02.208 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:47:02.208 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:47:02.208 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:47:02.217 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:47:02.218 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T08:47:02.218 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:47:02.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:47:02.256 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:47:02.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.396+0000 7f7e8ea00640 1 -- 172.21.15.44:0/3026239144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e880b2590 msgr2=0x7f7e880b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:02.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.396+0000 7f7e8ea00640 1 --2- 172.21.15.44:0/3026239144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e880b2590 0x7f7e880b2a00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009bc0 tx=0x7f7e8001cb30 comp rx=0 tx=0).stop 2024-09-13T08:47:02.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.396+0000 7f7e8ea00640 1 -- 172.21.15.44:0/3026239144 shutdown_connections 2024-09-13T08:47:02.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.396+0000 7f7e8ea00640 1 --2- 172.21.15.44:0/3026239144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e880b2590 0x7f7e880b2a00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.396+0000 7f7e8ea00640 1 --2- 172.21.15.44:0/3026239144 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e880d01b0 0x7f7e880cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.396+0000 7f7e8ea00640 1 -- 172.21.15.44:0/3026239144 >> 172.21.15.44:0/3026239144 conn(0x7f7e8809aa10 msgr2=0x7f7e8809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:02.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.396+0000 7f7e8ea00640 1 -- 172.21.15.44:0/3026239144 shutdown_connections 2024-09-13T08:47:02.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.396+0000 7f7e8ea00640 1 -- 172.21.15.44:0/3026239144 wait complete. 2024-09-13T08:47:02.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8ea00640 1 Processor -- start 2024-09-13T08:47:02.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8ea00640 1 -- start start 2024-09-13T08:47:02.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e880b2590 0x7f7e881f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e880d01b0 0x7f7e8815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8ea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e8815f000 0x7f7e880c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8ea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88168ea0 con 0x7f7e880b2590 2024-09-13T08:47:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8ea00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88168d20 con 0x7f7e880d01b0 2024-09-13T08:47:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8ea00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7e88169020 con 0x7f7e8815f000 2024-09-13T08:47:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e880b2590 0x7f7e881f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e880b2590 0x7f7e881f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33220/0 (socket says 172.21.15.44:33220) 2024-09-13T08:47:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8d600640 1 -- 172.21.15.44:0/4096775219 learned_addr learned my addr 172.21.15.44:0/4096775219 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:02.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8cc00640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e880d01b0 0x7f7e8815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8e000640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e8815f000 0x7f7e880c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8d600640 1 -- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e8815f000 msgr2=0x7f7e880c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8d600640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e8815f000 0x7f7e880c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8d600640 1 -- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e880d01b0 msgr2=0x7f7e8815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8d600640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e880d01b0 0x7f7e8815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8d600640 1 -- 172.21.15.44:0/4096775219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e88072050 con 0x7f7e880b2590 2024-09-13T08:47:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8e000640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e8815f000 0x7f7e880c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:47:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8cc00640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e880d01b0 0x7f7e8815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:47:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8d600640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e880b2590 0x7f7e881f8460 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00c400 tx=0x7f7e7c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e85600640 1 -- 172.21.15.44:0/4096775219 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c011790 con 0x7f7e880b2590 2024-09-13T08:47:02.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e85600640 1 -- 172.21.15.44:0/4096775219 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e7c017070 con 0x7f7e880b2590 2024-09-13T08:47:02.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8ea00640 1 -- 172.21.15.44:0/4096775219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e88072330 con 0x7f7e880b2590 2024-09-13T08:47:02.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e85600640 1 -- 172.21.15.44:0/4096775219 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c011cc0 con 0x7f7e880b2590 2024-09-13T08:47:02.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.400+0000 7f7e8ea00640 1 -- 172.21.15.44:0/4096775219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881f7c60 con 0x7f7e880b2590 2024-09-13T08:47:02.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.404+0000 7f7e8ea00640 1 -- 172.21.15.44:0/4096775219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e880b2a00 con 0x7f7e880b2590 2024-09-13T08:47:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.404+0000 7f7e85600640 1 -- 172.21.15.44:0/4096775219 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f7e7c01c460 con 0x7f7e880b2590 2024-09-13T08:47:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.404+0000 7f7e85600640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e68074030 0x7f7e680764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.404+0000 7f7e85600640 1 -- 172.21.15.44:0/4096775219 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5027+0+0 (secure 0 0 0) 0x7f7e7c092a60 con 0x7f7e880b2590 2024-09-13T08:47:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.404+0000 7f7e8cc00640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e68074030 0x7f7e680764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:02.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.408+0000 7f7e8cc00640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e68074030 0x7f7e680764f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7e74006fd0 tx=0x7f7e74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:02.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.408+0000 7f7e85600640 1 -- 172.21.15.44:0/4096775219 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7e7c05f410 con 0x7f7e880b2590 2024-09-13T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 -- 172.21.15.44:0/4096775219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e88070970 con 0x7f7e880b2590 2024-09-13T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e85600640 1 -- 172.21.15.44:0/4096775219 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3914 (secure 0 0 0) 0x7f7e7c05ed10 con 0x7f7e880b2590 2024-09-13T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":49,"btime":"2024-09-13T08:46:55:989705+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3854096181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3854096181},{"type":"v1","addr":"172.21.15.129:6836","nonce":3854096181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2183755853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":2183755853},{"type":"v1","addr":"172.21.15.129:6837","nonce":2183755853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6330,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1061181525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1061181525},{"type":"v1","addr":"172.21.15.44:6834","nonce":1061181525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/739745842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":739745842},{"type":"v1","addr":"172.21.15.44:6835","nonce":739745842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-09-13T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 49 2024-09-13T08:47:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 -- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e68074030 msgr2=0x7f7e680764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e68074030 0x7f7e680764f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7e74006fd0 tx=0x7f7e74008040 comp rx=0 tx=0).stop 2024-09-13T08:47:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 -- 172.21.15.44:0/4096775219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e880b2590 msgr2=0x7f7e881f8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e880b2590 0x7f7e881f8460 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00c400 tx=0x7f7e7c00c8c0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 -- 172.21.15.44:0/4096775219 shutdown_connections 2024-09-13T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e68074030 0x7f7e680764f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e8815f000 0x7f7e880c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e880d01b0 0x7f7e8815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 --2- 172.21.15.44:0/4096775219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e880b2590 0x7f7e881f8460 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 -- 172.21.15.44:0/4096775219 >> 172.21.15.44:0/4096775219 conn(0x7f7e8809aa10 msgr2=0x7f7e8815e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 -- 172.21.15.44:0/4096775219 shutdown_connections 2024-09-13T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.620+0000 7f7e8ea00640 1 -- 172.21.15.44:0/4096775219 wait complete. 2024-09-13T08:47:02.643 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T08:47:02.643 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T08:47:02.643 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T08:47:02.644 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:47:02.644 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T08:47:02.644 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T08:47:02.644 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T08:47:02.644 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:47:02.644 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T08:47:02.644 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T08:47:02.645 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T08:47:02.645 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:47:02.645 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T08:47:02.645 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T08:47:02.645 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T08:47:02.645 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:47:02.655 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T08:47:02.655 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T08:47:02.656 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T08:47:02.656 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T08:47:02.660 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T08:47:02.660 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T08:47:02.661 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T08:47:02.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T08:47:02.706 INFO:tasks.ceph.mds.a:Started 2024-09-13T08:47:02.707 DEBUG:teuthology.parallel:result is None 2024-09-13T08:47:02.707 INFO:tasks.ceph.mds.b:Started 2024-09-13T08:47:02.708 DEBUG:teuthology.parallel:result is None 2024-09-13T08:47:02.708 INFO:tasks.ceph.mds.c:Started 2024-09-13T08:47:02.709 DEBUG:teuthology.parallel:result is None 2024-09-13T08:47:02.709 INFO:tasks.ceph.mds.d:Started 2024-09-13T08:47:02.710 DEBUG:teuthology.parallel:result is None 2024-09-13T08:47:02.710 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T08:47:02.794 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:47:02.798 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:47:02.798 INFO:tasks.ceph.mds.b.smithi129.stdout:starting mds.b at 2024-09-13T08:47:02.799 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:47:02.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.860+0000 7f79e1600640 1 -- 172.21.15.44:0/2484926007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79dc0b56a0 msgr2=0x7f79dc070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:02.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.860+0000 7f79e1600640 1 --2- 172.21.15.44:0/2484926007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79dc0b56a0 0x7f79dc070950 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f79dc069da0 tx=0x7f79d401cd80 comp rx=0 tx=0).stop 2024-09-13T08:47:02.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.860+0000 7f79e1600640 1 -- 172.21.15.44:0/2484926007 shutdown_connections 2024-09-13T08:47:02.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.860+0000 7f79e1600640 1 --2- 172.21.15.44:0/2484926007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79dc0b56a0 0x7f79dc070950 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.860+0000 7f79e1600640 1 --2- 172.21.15.44:0/2484926007 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79dc0b0ce0 0x7f79dc0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.860+0000 7f79e1600640 1 --2- 172.21.15.44:0/2484926007 >> v2:172.21.15.129:3300/0 conn(0x7f79dc0b4d80 0x7f79dc0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.860+0000 7f79e1600640 1 -- 172.21.15.44:0/2484926007 >> 172.21.15.44:0/2484926007 conn(0x7f79dc09aa10 msgr2=0x7f79dc09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:02.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.860+0000 7f79e1600640 1 -- 172.21.15.44:0/2484926007 shutdown_connections 2024-09-13T08:47:02.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.860+0000 7f79e1600640 1 -- 172.21.15.44:0/2484926007 wait complete. 2024-09-13T08:47:02.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e1600640 1 Processor -- start 2024-09-13T08:47:02.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e1600640 1 -- start start 2024-09-13T08:47:02.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e1600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79dc0b0ce0 0x7f79dc14c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:02.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e1600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79dc0b4d80 0x7f79dc14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:02.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79dc0b56a0 0x7f79dc155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:02.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e1600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc167800 con 0x7f79dc0b56a0 2024-09-13T08:47:02.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e1600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc167680 con 0x7f79dc0b0ce0 2024-09-13T08:47:02.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e1600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f79dc167980 con 0x7f79dc0b4d80 2024-09-13T08:47:02.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79dc0b56a0 0x7f79dc155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:02.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79dc0b56a0 0x7f79dc155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33278/0 (socket says 172.21.15.44:33278) 2024-09-13T08:47:02.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e0c00640 1 -- 172.21.15.44:0/3947920729 learned_addr learned my addr 172.21.15.44:0/3947920729 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:02.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79dbe00640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79dc0b0ce0 0x7f79dc14c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:02.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79d3400640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79dc0b4d80 0x7f79dc14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:02.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e0c00640 1 -- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79dc0b4d80 msgr2=0x7f79dc14e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:02.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e0c00640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79dc0b4d80 0x7f79dc14e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e0c00640 1 -- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79dc0b0ce0 msgr2=0x7f79dc14c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:02.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e0c00640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79dc0b0ce0 0x7f79dc14c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:02.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e0c00640 1 -- 172.21.15.44:0/3947920729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79dc14fc50 con 0x7f79dc0b56a0 2024-09-13T08:47:02.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79dbe00640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79dc0b0ce0 0x7f79dc14c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:47:02.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e0c00640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79dc0b56a0 0x7f79dc155510 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009bd0 tx=0x7f79d40092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:02.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79d8c00640 1 -- 172.21.15.44:0/3947920729 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d40bb070 con 0x7f79dc0b56a0 2024-09-13T08:47:02.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79d8c00640 1 -- 172.21.15.44:0/3947920729 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79d4004d40 con 0x7f79dc0b56a0 2024-09-13T08:47:02.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79d8c00640 1 -- 172.21.15.44:0/3947920729 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d40b2040 con 0x7f79dc0b56a0 2024-09-13T08:47:02.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e1600640 1 -- 172.21.15.44:0/3947920729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79dc1f90d0 con 0x7f79dc0b56a0 2024-09-13T08:47:02.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.864+0000 7f79e1600640 1 -- 172.21.15.44:0/3947920729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79dc1f9690 con 0x7f79dc0b56a0 2024-09-13T08:47:02.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.868+0000 7f79e1600640 1 -- 172.21.15.44:0/3947920729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79dc0b11a0 con 0x7f79dc0b56a0 2024-09-13T08:47:02.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.868+0000 7f79d8c00640 1 -- 172.21.15.44:0/3947920729 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f79d40a5b80 con 0x7f79dc0b56a0 2024-09-13T08:47:02.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.868+0000 7f79d8c00640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f79b8074030 0x7f79b80764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:02.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.868+0000 7f79d8c00640 1 -- 172.21.15.44:0/3947920729 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5027+0+0 (secure 0 0 0) 0x7f79d412ca40 con 0x7f79dc0b56a0 2024-09-13T08:47:02.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.868+0000 7f79dbe00640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f79b8074030 0x7f79b80764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:02.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.868+0000 7f79dbe00640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f79b8074030 0x7f79b80764f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f79dc09fc20 tx=0x7f79cc0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:02.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:02.872+0000 7f79d8c00640 1 -- 172.21.15.44:0/3947920729 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f79d4016970 con 0x7f79dc0b56a0 2024-09-13T08:47:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.056+0000 7f79e1600640 1 -- 172.21.15.44:0/3947920729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f79dc14cbd0 con 0x7f79dc0b56a0 2024-09-13T08:47:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.212+0000 7f79d8c00640 1 -- 172.21.15.44:0/3947920729 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) ==== 97+0+0 (secure 0 0 0) 0x7f79d40f8d40 con 0x7f79dc0b56a0 2024-09-13T08:47:03.215 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-09-13T08:47:03.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 -- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f79b8074030 msgr2=0x7f79b80764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f79b8074030 0x7f79b80764f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f79dc09fc20 tx=0x7f79cc0091e0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 -- 172.21.15.44:0/3947920729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79dc0b56a0 msgr2=0x7f79dc155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79dc0b56a0 0x7f79dc155510 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009bd0 tx=0x7f79d40092c0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 -- 172.21.15.44:0/3947920729 shutdown_connections 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f79b8074030 0x7f79b80764f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79dc0b56a0 0x7f79dc155510 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79dc0b4d80 0x7f79dc14e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 --2- 172.21.15.44:0/3947920729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79dc0b0ce0 0x7f79dc14c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 -- 172.21.15.44:0/3947920729 >> 172.21.15.44:0/3947920729 conn(0x7f79dc09aa10 msgr2=0x7f79dc157650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 -- 172.21.15.44:0/3947920729 shutdown_connections 2024-09-13T08:47:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.216+0000 7f79e1600640 1 -- 172.21.15.44:0/3947920729 wait complete. 2024-09-13T08:47:03.240 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T08:47:03.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.384+0000 7fe930200640 1 -- 172.21.15.44:0/1849633941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe92815e280 msgr2=0x7fe9280af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.384+0000 7fe930200640 1 --2- 172.21.15.44:0/1849633941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe92815e280 0x7fe9280af0a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe9280d1a20 tx=0x7fe91401c8c0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.384+0000 7fe930200640 1 -- 172.21.15.44:0/1849633941 shutdown_connections 2024-09-13T08:47:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.384+0000 7fe930200640 1 --2- 172.21.15.44:0/1849633941 >> v2:172.21.15.129:3300/0 conn(0x7fe9280b3150 0x7fe9280cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.384+0000 7fe930200640 1 --2- 172.21.15.44:0/1849633941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe92815e280 0x7fe9280af0a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.384+0000 7fe930200640 1 -- 172.21.15.44:0/1849633941 >> 172.21.15.44:0/1849633941 conn(0x7fe9280d1da0 msgr2=0x7fe9280d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.384+0000 7fe930200640 1 -- 172.21.15.44:0/1849633941 shutdown_connections 2024-09-13T08:47:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.384+0000 7fe930200640 1 -- 172.21.15.44:0/1849633941 wait complete. 2024-09-13T08:47:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe930200640 1 Processor -- start 2024-09-13T08:47:03.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe930200640 1 -- start start 2024-09-13T08:47:03.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe930200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9280b3150 0x7fe928155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:03.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe930200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe92815dea0 0x7fe928157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:03.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe930200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe92815e280 0x7fe9281541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:03.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe930200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe92816ad70 con 0x7fe92815dea0 2024-09-13T08:47:03.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe930200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe92816abf0 con 0x7fe92815e280 2024-09-13T08:47:03.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe930200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe92816aef0 con 0x7fe9280b3150 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe92815dea0 0x7fe928157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe92815dea0 0x7fe928157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33306/0 (socket says 172.21.15.44:33306) 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92ce00640 1 -- 172.21.15.44:0/3480030622 learned_addr learned my addr 172.21.15.44:0/3480030622 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92d800640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9280b3150 0x7fe928155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92e200640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe92815e280 0x7fe9281541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92ce00640 1 -- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9280b3150 msgr2=0x7fe928155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92ce00640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9280b3150 0x7fe928155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92ce00640 1 -- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe92815e280 msgr2=0x7fe9281541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92ce00640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe92815e280 0x7fe9281541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92ce00640 1 -- 172.21.15.44:0/3480030622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9281583d0 con 0x7fe92815dea0 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe92ce00640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe92815dea0 0x7fe928157ac0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe91401af70 tx=0x7fe9140a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:03.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe925600640 1 -- 172.21.15.44:0/3480030622 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9140ba020 con 0x7fe92815dea0 2024-09-13T08:47:03.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe925600640 1 -- 172.21.15.44:0/3480030622 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe914004430 con 0x7fe92815dea0 2024-09-13T08:47:03.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe925600640 1 -- 172.21.15.44:0/3480030622 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9140b2040 con 0x7fe92815dea0 2024-09-13T08:47:03.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe930200640 1 -- 172.21.15.44:0/3480030622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9281586b0 con 0x7fe92815dea0 2024-09-13T08:47:03.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.388+0000 7fe930200640 1 -- 172.21.15.44:0/3480030622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9281562c0 con 0x7fe92815dea0 2024-09-13T08:47:03.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.392+0000 7fe930200640 1 -- 172.21.15.44:0/3480030622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9280cffe0 con 0x7fe92815dea0 2024-09-13T08:47:03.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.392+0000 7fe925600640 1 -- 172.21.15.44:0/3480030622 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7fe914005290 con 0x7fe92815dea0 2024-09-13T08:47:03.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.392+0000 7fe925600640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe908074030 0x7fe9080764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:03.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.392+0000 7fe925600640 1 -- 172.21.15.44:0/3480030622 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 4941+0+0 (secure 0 0 0) 0x7fe91412c770 con 0x7fe92815dea0 2024-09-13T08:47:03.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.392+0000 7fe92d800640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe908074030 0x7fe9080764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:03.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.392+0000 7fe92d800640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe908074030 0x7fe9080764f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe91c007fd0 tx=0x7fe91c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:03.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.396+0000 7fe925600640 1 -- 172.21.15.44:0/3480030622 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe9140c4060 con 0x7fe92815dea0 2024-09-13T08:47:03.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.632+0000 7fe930200640 1 -- 172.21.15.44:0/3480030622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe92815b9c0 con 0x7fe92815dea0 2024-09-13T08:47:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.632+0000 7fe925600640 1 -- 172.21.15.44:0/3480030622 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7fe9140164b0 con 0x7fe92815dea0 2024-09-13T08:47:03.636 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:03.636 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:47:03.636 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-09-13T08:47:03.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.636 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-09-13T08:47:03.636 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw==", 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ==", 2024-09-13T08:47:03.637 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA==", 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:47:03.638 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:47:03.639 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:47:03.639 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.639 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.639 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-09-13T08:47:03.639 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw==", 2024-09-13T08:47:03.640 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.640 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:47:03.640 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:47:03.640 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:47:03.640 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.640 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.640 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-09-13T08:47:03.640 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg==", 2024-09-13T08:47:03.641 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.641 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:47:03.641 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:47:03.641 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.641 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-09-13T08:47:03.641 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg==", 2024-09-13T08:47:03.641 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw==", 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:47:03.642 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig==", 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.644 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-09-13T08:47:03.644 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ==", 2024-09-13T08:47:03.644 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.644 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:47:03.644 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g==", 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.645 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg==", 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.646 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg==", 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA==", 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.647 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w==", 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:47:03.648 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA==", 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:47:03.649 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w==", 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:47:03.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmDD2LJxAA3s0jEb2EfPsLcu0w145G3Q==", 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ==", 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-09-13T08:47:03.652 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg==", 2024-09-13T08:47:03.653 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:47:03.653 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:47:03.653 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:47:03.653 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:47:03.653 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.653 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.653 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T08:47:03.653 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:47:03.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 -- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe908074030 msgr2=0x7fe9080764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe908074030 0x7fe9080764f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe91c007fd0 tx=0x7fe91c009040 comp rx=0 tx=0).stop 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 -- 172.21.15.44:0/3480030622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe92815dea0 msgr2=0x7fe928157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe92815dea0 0x7fe928157ac0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe91401af70 tx=0x7fe9140a5740 comp rx=0 tx=0).stop 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 -- 172.21.15.44:0/3480030622 shutdown_connections 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe908074030 0x7fe9080764f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe92815e280 0x7fe9281541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe92815dea0 0x7fe928157ac0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 --2- 172.21.15.44:0/3480030622 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9280b3150 0x7fe928155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 -- 172.21.15.44:0/3480030622 >> 172.21.15.44:0/3480030622 conn(0x7fe9280d1da0 msgr2=0x7fe9280b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 -- 172.21.15.44:0/3480030622 shutdown_connections 2024-09-13T08:47:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.636+0000 7fe930200640 1 -- 172.21.15.44:0/3480030622 wait complete. 2024-09-13T08:47:03.659 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T08:47:03.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.804+0000 7f9bbea00640 1 -- 172.21.15.44:0/4235294651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb80cdb30 msgr2=0x7f9bb80aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.804+0000 7f9bbea00640 1 --2- 172.21.15.44:0/4235294651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb80cdb30 0x7f9bb80aeef0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80cc3a0 tx=0x7f9bb001cd80 comp rx=0 tx=0).stop 2024-09-13T08:47:03.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.804+0000 7f9bbea00640 1 -- 172.21.15.44:0/4235294651 shutdown_connections 2024-09-13T08:47:03.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.804+0000 7f9bbea00640 1 --2- 172.21.15.44:0/4235294651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb80cdb30 0x7f9bb80aeef0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.804+0000 7f9bbea00640 1 --2- 172.21.15.44:0/4235294651 >> v2:172.21.15.129:3300/0 conn(0x7f9bb80ced80 0x7f9bb80ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.804+0000 7f9bbea00640 1 -- 172.21.15.44:0/4235294651 >> 172.21.15.44:0/4235294651 conn(0x7f9bb80d1dc0 msgr2=0x7f9bb80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:03.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.804+0000 7f9bbea00640 1 -- 172.21.15.44:0/4235294651 shutdown_connections 2024-09-13T08:47:03.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.804+0000 7f9bbea00640 1 -- 172.21.15.44:0/4235294651 wait complete. 2024-09-13T08:47:03.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.804+0000 7f9bbea00640 1 Processor -- start 2024-09-13T08:47:03.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbea00640 1 -- start start 2024-09-13T08:47:03.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb80cdb30 0x7f9bb806f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:03.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9bb80ced80 0x7f9bb809e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:03.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bb815bea0 0x7f9bb815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:03.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81692f0 con 0x7f9bb80cdb30 2024-09-13T08:47:03.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbea00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb8169170 con 0x7f9bb815bea0 2024-09-13T08:47:03.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbea00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb8169470 con 0x7f9bb80ced80 2024-09-13T08:47:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb80cdb30 0x7f9bb806f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb80cdb30 0x7f9bb806f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33330/0 (socket says 172.21.15.44:33330) 2024-09-13T08:47:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb7e00640 1 -- 172.21.15.44:0/579972313 learned_addr learned my addr 172.21.15.44:0/579972313 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbca00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bb815bea0 0x7f9bb815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb7e00640 1 -- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9bb80ced80 msgr2=0x7f9bb809e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb7e00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9bb80ced80 0x7f9bb809e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb7e00640 1 -- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bb815bea0 msgr2=0x7f9bb815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb7e00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bb815bea0 0x7f9bb815adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb7e00640 1 -- 172.21.15.44:0/579972313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb809c0d0 con 0x7f9bb80cdb30 2024-09-13T08:47:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbca00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bb815bea0 0x7f9bb815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:47:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb7e00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb80cdb30 0x7f9bb806f7b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800c3c0 tx=0x7f9ba800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb4c00640 1 -- 172.21.15.44:0/579972313 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8017070 con 0x7f9bb80cdb30 2024-09-13T08:47:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb4c00640 1 -- 172.21.15.44:0/579972313 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ba80053c0 con 0x7f9bb80cdb30 2024-09-13T08:47:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bb4c00640 1 -- 172.21.15.44:0/579972313 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8003860 con 0x7f9bb80cdb30 2024-09-13T08:47:03.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbea00640 1 -- 172.21.15.44:0/579972313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb809dab0 con 0x7f9bb80cdb30 2024-09-13T08:47:03.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbea00640 1 -- 172.21.15.44:0/579972313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb809e070 con 0x7f9bb80cdb30 2024-09-13T08:47:03.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.808+0000 7f9bbea00640 1 -- 172.21.15.44:0/579972313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb809e270 con 0x7f9bb80cdb30 2024-09-13T08:47:03.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.812+0000 7f9bb4c00640 1 -- 172.21.15.44:0/579972313 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f9ba801a070 con 0x7f9bb80cdb30 2024-09-13T08:47:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.812+0000 7f9bb4c00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b94074030 0x7f9b940764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.812+0000 7f9bb4c00640 1 -- 172.21.15.44:0/579972313 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 4941+0+0 (secure 0 0 0) 0x7f9ba8091ff0 con 0x7f9bb80cdb30 2024-09-13T08:47:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.812+0000 7f9baf400640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b94074030 0x7f9b940764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:03.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.812+0000 7f9baf400640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b94074030 0x7f9b940764f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9bb81661b0 tx=0x7f9ba000b410 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:03.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.816+0000 7f9bb4c00640 1 -- 172.21.15.44:0/579972313 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9bb809e270 con 0x7f9bb80cdb30 2024-09-13T08:47:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 -- 172.21.15.44:0/579972313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9bb80ce670 con 0x7f9bb80cdb30 2024-09-13T08:47:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bb4c00640 1 -- 172.21.15.44:0/579972313 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) ==== 84+0+65 (secure 0 0 0) 0x7f9ba805e2f0 con 0x7f9bb80cdb30 2024-09-13T08:47:03.993 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:03.994 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:47:03.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:03.994 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T08:47:03.994 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T08:47:03.994 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:03.994 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 -- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b94074030 msgr2=0x7f9b940764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b94074030 0x7f9b940764f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9bb81661b0 tx=0x7f9ba000b410 comp rx=0 tx=0).stop 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 -- 172.21.15.44:0/579972313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb80cdb30 msgr2=0x7f9bb806f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb80cdb30 0x7f9bb806f7b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800c3c0 tx=0x7f9ba800c880 comp rx=0 tx=0).stop 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 -- 172.21.15.44:0/579972313 shutdown_connections 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b94074030 0x7f9b940764f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bb815bea0 0x7f9bb815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9bb80ced80 0x7f9bb809e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 --2- 172.21.15.44:0/579972313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb80cdb30 0x7f9bb806f7b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 -- 172.21.15.44:0/579972313 >> 172.21.15.44:0/579972313 conn(0x7f9bb80d1dc0 msgr2=0x7f9bb80b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 -- 172.21.15.44:0/579972313 shutdown_connections 2024-09-13T08:47:03.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:03.992+0000 7f9bbea00640 1 -- 172.21.15.44:0/579972313 wait complete. 2024-09-13T08:47:04.016 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T08:47:04.017 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T08:47:04.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 -- 172.21.15.44:0/2240680737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8194003a10 msgr2=0x7f8194003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:04.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 --2- 172.21.15.44:0/2240680737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8194003a10 0x7f8194003e30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c310 tx=0x7f81a001f720 comp rx=0 tx=0).stop 2024-09-13T08:47:04.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ae800640 1 -- 172.21.15.44:0/2240680737 reap_dead start 2024-09-13T08:47:04.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 -- 172.21.15.44:0/2240680737 shutdown_connections 2024-09-13T08:47:04.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 -- 172.21.15.44:0/2240680737 >> 172.21.15.44:0/2240680737 conn(0x7f81b00d1da0 msgr2=0x7f81b00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:04.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 -- 172.21.15.44:0/2240680737 shutdown_connections 2024-09-13T08:47:04.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 -- 172.21.15.44:0/2240680737 wait complete. 2024-09-13T08:47:04.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 Processor -- start 2024-09-13T08:47:04.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 -- start start 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8194003a10 0x7f81b01f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81b01f5ab0 0x7f81b0158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b0158880 0x7f81b015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81b00cf300 con 0x7f81b0158880 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f81b00cf180 con 0x7f8194003a10 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81b5200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f81b00cf480 con 0x7f81b01f5ab0 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ae800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b0158880 0x7f81b015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ae800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b0158880 0x7f81b015a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33342/0 (socket says 172.21.15.44:33342) 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ae800640 1 -- 172.21.15.44:0/1343654070 learned_addr learned my addr 172.21.15.44:0/1343654070 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ade00640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8194003a10 0x7f81b01f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81a7e00640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81b01f5ab0 0x7f81b0158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ae800640 1 -- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81b01f5ab0 msgr2=0x7f81b0158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:04.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ae800640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81b01f5ab0 0x7f81b0158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:04.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ae800640 1 -- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8194003a10 msgr2=0x7f81b01f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:04.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ae800640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8194003a10 0x7f81b01f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:04.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ae800640 1 -- 172.21.15.44:0/1343654070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b015b2e0 con 0x7f81b0158880 2024-09-13T08:47:04.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.168+0000 7f81ae800640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b0158880 0x7f81b015a550 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f81b00746b0 tx=0x7f81a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.172+0000 7f81a6000640 1 -- 172.21.15.44:0/1343654070 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198009730 con 0x7f81b0158880 2024-09-13T08:47:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.172+0000 7f81a6000640 1 -- 172.21.15.44:0/1343654070 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81a8003d90 con 0x7f81b0158880 2024-09-13T08:47:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.172+0000 7f81a6000640 1 -- 172.21.15.44:0/1343654070 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8004d90 con 0x7f81b0158880 2024-09-13T08:47:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.172+0000 7f81b5200640 1 -- 172.21.15.44:0/1343654070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b00c6b70 con 0x7f81b0158880 2024-09-13T08:47:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.172+0000 7f81b5200640 1 -- 172.21.15.44:0/1343654070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b00c7130 con 0x7f81b0158880 2024-09-13T08:47:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.172+0000 7f81b5200640 1 -- 172.21.15.44:0/1343654070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b00ccbd0 con 0x7f81b0158880 2024-09-13T08:47:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.172+0000 7f81a6000640 1 -- 172.21.15.44:0/1343654070 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f81a8004030 con 0x7f81b0158880 2024-09-13T08:47:04.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.172+0000 7f81a6000640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c074030 0x7f818c0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:04.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.172+0000 7f81a6000640 1 -- 172.21.15.44:0/1343654070 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 4941+0+0 (secure 0 0 0) 0x7f81a80939f0 con 0x7f81b0158880 2024-09-13T08:47:04.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.172+0000 7f81ade00640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c074030 0x7f818c0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.176+0000 7f81ade00640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c074030 0x7f818c0764f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f81a00b0000 tx=0x7f81a001c560 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:04.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.176+0000 7f81a6000640 1 -- 172.21.15.44:0/1343654070 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f81a8060340 con 0x7f81b0158880 2024-09-13T08:47:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:04.356+0000 7f81b5200640 1 -- 172.21.15.44:0/1343654070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f81b0070350 con 0x7f81b0158880 2024-09-13T08:47:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.220+0000 7f81a6000640 1 -- 172.21.15.44:0/1343654070 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) ==== 138+0+0 (secure 0 0 0) 0x7f81a805fc40 con 0x7f81b0158880 2024-09-13T08:47:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T08:47:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 -- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c074030 msgr2=0x7f818c0764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c074030 0x7f818c0764f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f81a00b0000 tx=0x7f81a001c560 comp rx=0 tx=0).stop 2024-09-13T08:47:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 -- 172.21.15.44:0/1343654070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b0158880 msgr2=0x7f81b015a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b0158880 0x7f81b015a550 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f81b00746b0 tx=0x7f81a800f790 comp rx=0 tx=0).stop 2024-09-13T08:47:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 -- 172.21.15.44:0/1343654070 shutdown_connections 2024-09-13T08:47:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f818c074030 0x7f818c0764f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b0158880 0x7f81b015a550 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81b01f5ab0 0x7f81b0158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 --2- 172.21.15.44:0/1343654070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8194003a10 0x7f81b01f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 -- 172.21.15.44:0/1343654070 >> 172.21.15.44:0/1343654070 conn(0x7f81b00d1da0 msgr2=0x7f81b0070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 -- 172.21.15.44:0/1343654070 shutdown_connections 2024-09-13T08:47:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.224+0000 7f81b5200640 1 -- 172.21.15.44:0/1343654070 wait complete. 2024-09-13T08:47:05.256 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T08:47:05.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.412+0000 7faae7200640 1 -- 172.21.15.44:0/516956372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae00b6350 msgr2=0x7faae006fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:05.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.412+0000 7faae7200640 1 --2- 172.21.15.44:0/516956372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae00b6350 0x7faae006fff0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7faae0071910 tx=0x7faad001c890 comp rx=0 tx=0).stop 2024-09-13T08:47:05.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.412+0000 7faae7200640 1 -- 172.21.15.44:0/516956372 shutdown_connections 2024-09-13T08:47:05.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.412+0000 7faae7200640 1 --2- 172.21.15.44:0/516956372 >> v2:172.21.15.129:3300/0 conn(0x7faae006cc20 0x7faae006d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:05.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.412+0000 7faae7200640 1 --2- 172.21.15.44:0/516956372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae00b6350 0x7faae006fff0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:05.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.412+0000 7faae7200640 1 -- 172.21.15.44:0/516956372 >> 172.21.15.44:0/516956372 conn(0x7faae00d0cd0 msgr2=0x7faae00d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:05.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.412+0000 7faae7200640 1 -- 172.21.15.44:0/516956372 shutdown_connections 2024-09-13T08:47:05.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.412+0000 7faae7200640 1 -- 172.21.15.44:0/516956372 wait complete. 2024-09-13T08:47:05.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.412+0000 7faae7200640 1 Processor -- start 2024-09-13T08:47:05.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.412+0000 7faae7200640 1 -- start start 2024-09-13T08:47:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faae7200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faae006cc20 0x7faae0154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faae7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae00b6350 0x7faae006f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faae7200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faae015c2d0 0x7faae006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faae7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faae0168fb0 con 0x7faae00b6350 2024-09-13T08:47:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faae7200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faae0168e30 con 0x7faae015c2d0 2024-09-13T08:47:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faae7200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7faae0169130 con 0x7faae006cc20 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faad7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae00b6350 0x7faae006f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faad7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae00b6350 0x7faae006f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33352/0 (socket says 172.21.15.44:33352) 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faad7e00640 1 -- 172.21.15.44:0/3777465591 learned_addr learned my addr 172.21.15.44:0/3777465591 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faadfe00640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faae006cc20 0x7faae0154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faae5200640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faae015c2d0 0x7faae006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faad7e00640 1 -- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faae006cc20 msgr2=0x7faae0154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faad7e00640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faae006cc20 0x7faae0154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faad7e00640 1 -- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faae015c2d0 msgr2=0x7faae006fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faad7e00640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faae015c2d0 0x7faae006fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faad7e00640 1 -- 172.21.15.44:0/3777465591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae01f7ca0 con 0x7faae00b6350 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faadfe00640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faae006cc20 0x7faae0154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faad7e00640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae00b6350 0x7faae006f700 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7faac800b3e0 tx=0x7faac800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faadcc00640 1 -- 172.21.15.44:0/3777465591 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8015020 con 0x7faae00b6350 2024-09-13T08:47:05.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faadcc00640 1 -- 172.21.15.44:0/3777465591 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faac8011940 con 0x7faae00b6350 2024-09-13T08:47:05.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faae7200640 1 -- 172.21.15.44:0/3777465591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae01f7f80 con 0x7faae00b6350 2024-09-13T08:47:05.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faadcc00640 1 -- 172.21.15.44:0/3777465591 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8011d90 con 0x7faae00b6350 2024-09-13T08:47:05.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faae7200640 1 -- 172.21.15.44:0/3777465591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae01f8540 con 0x7faae00b6350 2024-09-13T08:47:05.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.416+0000 7faae7200640 1 -- 172.21.15.44:0/3777465591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae01f8740 con 0x7faae00b6350 2024-09-13T08:47:05.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.420+0000 7faadcc00640 1 -- 172.21.15.44:0/3777465591 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7faac8018020 con 0x7faae00b6350 2024-09-13T08:47:05.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.420+0000 7faadcc00640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faabc074030 0x7faabc0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:05.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.420+0000 7faadcc00640 1 -- 172.21.15.44:0/3777465591 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5311+0+0 (secure 0 0 0) 0x7faac8093160 con 0x7faae00b6350 2024-09-13T08:47:05.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.420+0000 7faadfe00640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faabc074030 0x7faabc0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:05.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.420+0000 7faadfe00640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faabc074030 0x7faabc0764f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faad00af000 tx=0x7faad001b0d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:05.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.424+0000 7faadcc00640 1 -- 172.21.15.44:0/3777465591 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7faae01f8740 con 0x7faae00b6350 2024-09-13T08:47:05.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:05.600+0000 7faae7200640 1 -- 172.21.15.44:0/3777465591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faae006d090 con 0x7faae00b6350 2024-09-13T08:47:06.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.224+0000 7faadcc00640 1 -- 172.21.15.44:0/3777465591 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) ==== 170+0+0 (secure 0 0 0) 0x7faac801f020 con 0x7faae00b6350 2024-09-13T08:47:06.228 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T08:47:06.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.228+0000 7faae7200640 1 -- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faabc074030 msgr2=0x7faabc0764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:06.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.228+0000 7faae7200640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faabc074030 0x7faabc0764f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faad00af000 tx=0x7faad001b0d0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.228+0000 7faae7200640 1 -- 172.21.15.44:0/3777465591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae00b6350 msgr2=0x7faae006f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:06.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.228+0000 7faae7200640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae00b6350 0x7faae006f700 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7faac800b3e0 tx=0x7faac800b8a0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.228+0000 7faae7200640 1 -- 172.21.15.44:0/3777465591 shutdown_connections 2024-09-13T08:47:06.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.228+0000 7faae7200640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faabc074030 0x7faabc0764f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.228+0000 7faae7200640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faae015c2d0 0x7faae006fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.228+0000 7faae7200640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae00b6350 0x7faae006f700 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.228+0000 7faae7200640 1 --2- 172.21.15.44:0/3777465591 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faae006cc20 0x7faae0154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.228+0000 7faae7200640 1 -- 172.21.15.44:0/3777465591 >> 172.21.15.44:0/3777465591 conn(0x7faae00d0cd0 msgr2=0x7faae0159830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:06.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.232+0000 7faae7200640 1 -- 172.21.15.44:0/3777465591 shutdown_connections 2024-09-13T08:47:06.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.232+0000 7faae7200640 1 -- 172.21.15.44:0/3777465591 wait complete. 2024-09-13T08:47:06.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T08:47:06.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 -- 172.21.15.44:0/813936111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b00cd0f0 msgr2=0x7f83b0073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:06.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 --2- 172.21.15.44:0/813936111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b00cd0f0 0x7f83b0073b70 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f83b0076240 tx=0x7f83a801cd50 comp rx=0 tx=0).stop 2024-09-13T08:47:06.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 -- 172.21.15.44:0/813936111 shutdown_connections 2024-09-13T08:47:06.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 --2- 172.21.15.44:0/813936111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b00cd0f0 0x7f83b0073b70 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 --2- 172.21.15.44:0/813936111 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83b00ccc90 0x7f83b0073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 --2- 172.21.15.44:0/813936111 >> v2:172.21.15.129:3300/0 conn(0x7f83b015acb0 0x7f83b00ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 -- 172.21.15.44:0/813936111 >> 172.21.15.44:0/813936111 conn(0x7f83b00d0ab0 msgr2=0x7f83b00d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 -- 172.21.15.44:0/813936111 shutdown_connections 2024-09-13T08:47:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 -- 172.21.15.44:0/813936111 wait complete. 2024-09-13T08:47:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 Processor -- start 2024-09-13T08:47:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.424+0000 7f83afe00640 1 -- start start 2024-09-13T08:47:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83afe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b00ccc90 0x7f83b01fb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83afe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83b015acb0 0x7f83b015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83afe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83b01fbaf0 0x7f83b015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83afe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f83b0070070 con 0x7f83b00ccc90 2024-09-13T08:47:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83afe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f83b00cef60 con 0x7f83b015acb0 2024-09-13T08:47:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83afe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f83b015e840 con 0x7f83b01fbaf0 2024-09-13T08:47:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83aea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b00ccc90 0x7f83b01fb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83ae000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83b015acb0 0x7f83b015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83aea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b00ccc90 0x7f83b01fb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33374/0 (socket says 172.21.15.44:33374) 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83aea00640 1 -- 172.21.15.44:0/1380235001 learned_addr learned my addr 172.21.15.44:0/1380235001 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83af400640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83b01fbaf0 0x7f83b015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83aea00640 1 -- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83b01fbaf0 msgr2=0x7f83b015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83aea00640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83b01fbaf0 0x7f83b015b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83aea00640 1 -- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83b015acb0 msgr2=0x7f83b015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83aea00640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83b015acb0 0x7f83b015b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83aea00640 1 -- 172.21.15.44:0/1380235001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b01f8030 con 0x7f83b00ccc90 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83aea00640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b00ccc90 0x7f83b01fb5b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f83a000e7a0 tx=0x7f83a000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83a6a00640 1 -- 172.21.15.44:0/1380235001 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a000ccc0 con 0x7f83b00ccc90 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83a6a00640 1 -- 172.21.15.44:0/1380235001 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83a0010430 con 0x7f83b00ccc90 2024-09-13T08:47:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83a6a00640 1 -- 172.21.15.44:0/1380235001 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0010860 con 0x7f83b00ccc90 2024-09-13T08:47:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83afe00640 1 -- 172.21.15.44:0/1380235001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b01f8310 con 0x7f83b00ccc90 2024-09-13T08:47:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83afe00640 1 -- 172.21.15.44:0/1380235001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b00af020 con 0x7f83b00ccc90 2024-09-13T08:47:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.428+0000 7f83afe00640 1 -- 172.21.15.44:0/1380235001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b00af310 con 0x7f83b00ccc90 2024-09-13T08:47:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.432+0000 7f83a6a00640 1 -- 172.21.15.44:0/1380235001 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f83a0010a00 con 0x7f83b00ccc90 2024-09-13T08:47:06.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.432+0000 7f83a6a00640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8390074030 0x7f83900764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:06.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.432+0000 7f83a6a00640 1 -- 172.21.15.44:0/1380235001 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5693+0+0 (secure 0 0 0) 0x7f83a0093e80 con 0x7f83b00ccc90 2024-09-13T08:47:06.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.432+0000 7f83ae000640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8390074030 0x7f83900764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:06.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.432+0000 7f83ae000640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8390074030 0x7f83900764f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f83b0073d10 tx=0x7f839c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:06.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.436+0000 7f83a6a00640 1 -- 172.21.15.44:0/1380235001 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f83a0060180 con 0x7f83b00ccc90 2024-09-13T08:47:06.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:06.644+0000 7f83afe00640 1 -- 172.21.15.44:0/1380235001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f83b00d18e0 con 0x7f83b00ccc90 2024-09-13T08:47:07.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.252+0000 7f83a6a00640 1 -- 172.21.15.44:0/1380235001 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v52) ==== 174+0+0 (secure 0 0 0) 0x7f83a0064bb0 con 0x7f83b00ccc90 2024-09-13T08:47:07.263 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 12 and data pool 13 2024-09-13T08:47:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.264+0000 7f83afe00640 1 -- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8390074030 msgr2=0x7f83900764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.264+0000 7f83afe00640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8390074030 0x7f83900764f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f83b0073d10 tx=0x7f839c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.264+0000 7f83afe00640 1 -- 172.21.15.44:0/1380235001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b00ccc90 msgr2=0x7f83b01fb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.264+0000 7f83afe00640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b00ccc90 0x7f83b01fb5b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f83a000e7a0 tx=0x7f83a000ec60 comp rx=0 tx=0).stop 2024-09-13T08:47:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.276+0000 7f83afe00640 1 -- 172.21.15.44:0/1380235001 shutdown_connections 2024-09-13T08:47:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.276+0000 7f83afe00640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8390074030 0x7f83900764f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.276+0000 7f83afe00640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83b01fbaf0 0x7f83b015b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.276+0000 7f83afe00640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83b015acb0 0x7f83b015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.276+0000 7f83afe00640 1 --2- 172.21.15.44:0/1380235001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b00ccc90 0x7f83b01fb5b0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.276+0000 7f83afe00640 1 -- 172.21.15.44:0/1380235001 >> 172.21.15.44:0/1380235001 conn(0x7f83b00d0ab0 msgr2=0x7f83b00ca550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.276+0000 7f83afe00640 1 -- 172.21.15.44:0/1380235001 shutdown_connections 2024-09-13T08:47:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.280+0000 7f83afe00640 1 -- 172.21.15.44:0/1380235001 wait complete. 2024-09-13T08:47:07.307 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:47:07.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 -- 172.21.15.44:0/2495365234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf600d03e0 msgr2=0x7fdf600d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 --2- 172.21.15.44:0/2495365234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf600d03e0 0x7fdf600d0890 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdf600cceb0 tx=0x7fdf5801cd80 comp rx=0 tx=0).stop 2024-09-13T08:47:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 -- 172.21.15.44:0/2495365234 shutdown_connections 2024-09-13T08:47:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 --2- 172.21.15.44:0/2495365234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf600d03e0 0x7fdf600d0890 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 --2- 172.21.15.44:0/2495365234 >> v2:172.21.15.129:3300/0 conn(0x7fdf600b4270 0x7fdf600af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 --2- 172.21.15.44:0/2495365234 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf6015bde0 0x7fdf60068c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 -- 172.21.15.44:0/2495365234 >> 172.21.15.44:0/2495365234 conn(0x7fdf600b2240 msgr2=0x7fdf600b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:07.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 -- 172.21.15.44:0/2495365234 shutdown_connections 2024-09-13T08:47:07.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 -- 172.21.15.44:0/2495365234 wait complete. 2024-09-13T08:47:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 Processor -- start 2024-09-13T08:47:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 -- start start 2024-09-13T08:47:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf600b4270 0x7fdf601488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf600d03e0 0x7fdf60166520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf6015bde0 0x7fdf600cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6015fbf0 con 0x7fdf600b4270 2024-09-13T08:47:07.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6015fa70 con 0x7fdf6015bde0 2024-09-13T08:47:07.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.456+0000 7fdf66800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6015fd70 con 0x7fdf600d03e0 2024-09-13T08:47:07.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf5f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf600b4270 0x7fdf601488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:07.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf5f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf600b4270 0x7fdf601488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33386/0 (socket says 172.21.15.44:33386) 2024-09-13T08:47:07.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf5f400640 1 -- 172.21.15.44:0/131088073 learned_addr learned my addr 172.21.15.44:0/131088073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:07.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf5fe00640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf6015bde0 0x7fdf600cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:07.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf56a00640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf600d03e0 0x7fdf60166520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:07.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf5f400640 1 -- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf600d03e0 msgr2=0x7fdf60166520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:07.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf5f400640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf600d03e0 0x7fdf60166520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf5f400640 1 -- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf6015bde0 msgr2=0x7fdf600cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf5f400640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf6015bde0 0x7fdf600cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf5f400640 1 -- 172.21.15.44:0/131088073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf60153aa0 con 0x7fdf600b4270 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf56a00640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf600d03e0 0x7fdf60166520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf5f400640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf600b4270 0x7fdf601488d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000e7c0 tx=0x7fdf5000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf57e00640 1 -- 172.21.15.44:0/131088073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000ccc0 con 0x7fdf600b4270 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf57e00640 1 -- 172.21.15.44:0/131088073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf5000ce80 con 0x7fdf600b4270 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf57e00640 1 -- 172.21.15.44:0/131088073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50005280 con 0x7fdf600b4270 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf66800640 1 -- 172.21.15.44:0/131088073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf601537c0 con 0x7fdf600b4270 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf66800640 1 -- 172.21.15.44:0/131088073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf600cf450 con 0x7fdf600b4270 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf66800640 1 -- 172.21.15.44:0/131088073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf600d0890 con 0x7fdf600b4270 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.460+0000 7fdf57e00640 1 -- 172.21.15.44:0/131088073 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7fdf50005420 con 0x7fdf600b4270 2024-09-13T08:47:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.464+0000 7fdf57e00640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf3c074030 0x7fdf3c0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:07.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.464+0000 7fdf57e00640 1 -- 172.21.15.44:0/131088073 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5793+0+0 (secure 0 0 0) 0x7fdf500935e0 con 0x7fdf600b4270 2024-09-13T08:47:07.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.464+0000 7fdf56a00640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf3c074030 0x7fdf3c0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:07.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.464+0000 7fdf56a00640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf3c074030 0x7fdf3c0764f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdf601533d0 tx=0x7fdf48005f40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:07.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.468+0000 7fdf57e00640 1 -- 172.21.15.44:0/131088073 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdf5005ff30 con 0x7fdf600b4270 2024-09-13T08:47:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.644+0000 7fdf66800640 1 -- 172.21.15.44:0/131088073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf6015b400 con 0x7fdf600b4270 2024-09-13T08:47:07.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.644+0000 7fdf57e00640 1 -- 172.21.15.44:0/131088073 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16889 (secure 0 0 0) 0x7fdf5005f830 con 0x7fdf600b4270 2024-09-13T08:47:07.647 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:07.647 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:47:06.650533+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:47:04.371378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T08:47:05.614930+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:47:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 -- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf3c074030 msgr2=0x7fdf3c0764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf3c074030 0x7fdf3c0764f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdf601533d0 tx=0x7fdf48005f40 comp rx=0 tx=0).stop 2024-09-13T08:47:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 -- 172.21.15.44:0/131088073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf600b4270 msgr2=0x7fdf601488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf600b4270 0x7fdf601488d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000e7c0 tx=0x7fdf5000ec80 comp rx=0 tx=0).stop 2024-09-13T08:47:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 -- 172.21.15.44:0/131088073 shutdown_connections 2024-09-13T08:47:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdf3c074030 0x7fdf3c0764f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf6015bde0 0x7fdf600cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdf600d03e0 0x7fdf60166520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 --2- 172.21.15.44:0/131088073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf600b4270 0x7fdf601488d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 -- 172.21.15.44:0/131088073 >> 172.21.15.44:0/131088073 conn(0x7fdf600b2240 msgr2=0x7fdf600b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 -- 172.21.15.44:0/131088073 shutdown_connections 2024-09-13T08:47:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.648+0000 7fdf66800640 1 -- 172.21.15.44:0/131088073 wait complete. 2024-09-13T08:47:07.669 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:47:07.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.812+0000 7f8a40400640 1 -- 172.21.15.44:0/2588632727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a380d28e0 msgr2=0x7f8a380cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:07.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.812+0000 7f8a40400640 1 --2- 172.21.15.44:0/2588632727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a380d28e0 0x7f8a380cebb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f8a380745d0 tx=0x7f8a3001ccf0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.812+0000 7f8a40400640 1 -- 172.21.15.44:0/2588632727 shutdown_connections 2024-09-13T08:47:07.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.812+0000 7f8a40400640 1 --2- 172.21.15.44:0/2588632727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a380d28e0 0x7f8a380cebb0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.812+0000 7f8a40400640 1 --2- 172.21.15.44:0/2588632727 >> v2:172.21.15.129:3300/0 conn(0x7f8a3815bea0 0x7f8a380ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.812+0000 7f8a40400640 1 -- 172.21.15.44:0/2588632727 >> 172.21.15.44:0/2588632727 conn(0x7f8a380d1dc0 msgr2=0x7f8a380d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:07.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.812+0000 7f8a40400640 1 -- 172.21.15.44:0/2588632727 shutdown_connections 2024-09-13T08:47:07.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.812+0000 7f8a40400640 1 -- 172.21.15.44:0/2588632727 wait complete. 2024-09-13T08:47:07.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.812+0000 7f8a40400640 1 Processor -- start 2024-09-13T08:47:07.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a40400640 1 -- start start 2024-09-13T08:47:07.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a40400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a380cdb30 0x7f8a381f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:07.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a40400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a380d28e0 0x7f8a38156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:07.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a40400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8a3815bea0 0x7f8a380cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a40400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38168fc0 con 0x7f8a380cdb30 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a40400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38168e40 con 0x7f8a380d28e0 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a40400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8a38169140 con 0x7f8a3815bea0 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a380cdb30 0x7f8a381f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a380cdb30 0x7f8a381f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33400/0 (socket says 172.21.15.44:33400) 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3da00640 1 -- 172.21.15.44:0/372350626 learned_addr learned my addr 172.21.15.44:0/372350626 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3e400640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8a3815bea0 0x7f8a380cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3d000640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a380d28e0 0x7f8a38156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3da00640 1 -- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8a3815bea0 msgr2=0x7f8a380cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3da00640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8a3815bea0 0x7f8a380cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3da00640 1 -- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a380d28e0 msgr2=0x7f8a38156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3da00640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a380d28e0 0x7f8a38156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:07.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3da00640 1 -- 172.21.15.44:0/372350626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a380cb970 con 0x7f8a380cdb30 2024-09-13T08:47:07.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a3da00640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a380cdb30 0x7f8a381f8460 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00b410 tx=0x7f8a2c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:07.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a35600640 1 -- 172.21.15.44:0/372350626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c015020 con 0x7f8a380cdb30 2024-09-13T08:47:07.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a35600640 1 -- 172.21.15.44:0/372350626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a2c011920 con 0x7f8a380cdb30 2024-09-13T08:47:07.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a35600640 1 -- 172.21.15.44:0/372350626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c011d70 con 0x7f8a380cdb30 2024-09-13T08:47:07.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a40400640 1 -- 172.21.15.44:0/372350626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a380cbc50 con 0x7f8a380cdb30 2024-09-13T08:47:07.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a40400640 1 -- 172.21.15.44:0/372350626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a381565b0 con 0x7f8a380cdb30 2024-09-13T08:47:07.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.816+0000 7f8a40400640 1 -- 172.21.15.44:0/372350626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a380cc9d0 con 0x7f8a380cdb30 2024-09-13T08:47:07.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.820+0000 7f8a35600640 1 -- 172.21.15.44:0/372350626 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f8a2c018020 con 0x7f8a380cdb30 2024-09-13T08:47:07.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.820+0000 7f8a35600640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a18074030 0x7f8a180764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:07.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.820+0000 7f8a35600640 1 -- 172.21.15.44:0/372350626 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5793+0+0 (secure 0 0 0) 0x7f8a2c092bb0 con 0x7f8a380cdb30 2024-09-13T08:47:07.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.820+0000 7f8a3d000640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a18074030 0x7f8a180764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:07.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.820+0000 7f8a3d000640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a18074030 0x7f8a180764f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8a3806f750 tx=0x7f8a24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:07.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:07.824+0000 7f8a35600640 1 -- 172.21.15.44:0/372350626 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8a2c01f020 con 0x7f8a380cdb30 2024-09-13T08:47:08.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.000+0000 7f8a40400640 1 -- 172.21.15.44:0/372350626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a380b6cb0 con 0x7f8a380cdb30 2024-09-13T08:47:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.000+0000 7f8a35600640 1 -- 172.21.15.44:0/372350626 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16889 (secure 0 0 0) 0x7f8a2c05eeb0 con 0x7f8a380cdb30 2024-09-13T08:47:08.004 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:08.004 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:47:06.650533+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:47:04.371378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T08:47:05.614930+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:47:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 -- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a18074030 msgr2=0x7f8a180764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a18074030 0x7f8a180764f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8a3806f750 tx=0x7f8a24009040 comp rx=0 tx=0).stop 2024-09-13T08:47:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 -- 172.21.15.44:0/372350626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a380cdb30 msgr2=0x7f8a381f8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a380cdb30 0x7f8a381f8460 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00b410 tx=0x7f8a2c00b8d0 comp rx=0 tx=0).stop 2024-09-13T08:47:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 -- 172.21.15.44:0/372350626 shutdown_connections 2024-09-13T08:47:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a18074030 0x7f8a180764f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8a3815bea0 0x7f8a380cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a380d28e0 0x7f8a38156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 --2- 172.21.15.44:0/372350626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a380cdb30 0x7f8a381f8460 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 -- 172.21.15.44:0/372350626 >> 172.21.15.44:0/372350626 conn(0x7f8a380d1dc0 msgr2=0x7f8a380b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 -- 172.21.15.44:0/372350626 shutdown_connections 2024-09-13T08:47:08.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.004+0000 7f8a40400640 1 -- 172.21.15.44:0/372350626 wait complete. 2024-09-13T08:47:08.025 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T08:47:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.172+0000 7ffafb200640 1 -- 172.21.15.44:0/2796409382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40cd0d0 msgr2=0x7ffaf40cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.172+0000 7ffafb200640 1 --2- 172.21.15.44:0/2796409382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40cd0d0 0x7ffaf40cebf0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009900 tx=0x7ffadc01c8b0 comp rx=0 tx=0).stop 2024-09-13T08:47:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.172+0000 7ffafb200640 1 -- 172.21.15.44:0/2796409382 shutdown_connections 2024-09-13T08:47:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.172+0000 7ffafb200640 1 --2- 172.21.15.44:0/2796409382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40cd0d0 0x7ffaf40cebf0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.172+0000 7ffafb200640 1 --2- 172.21.15.44:0/2796409382 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffaf415af90 0x7ffaf415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.172+0000 7ffafb200640 1 -- 172.21.15.44:0/2796409382 >> 172.21.15.44:0/2796409382 conn(0x7ffaf40b18f0 msgr2=0x7ffaf40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.172+0000 7ffafb200640 1 -- 172.21.15.44:0/2796409382 shutdown_connections 2024-09-13T08:47:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.172+0000 7ffafb200640 1 -- 172.21.15.44:0/2796409382 wait complete. 2024-09-13T08:47:08.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 Processor -- start 2024-09-13T08:47:08.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 -- start start 2024-09-13T08:47:08.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40b2bf0 0x7ffaf4141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaf40cd0d0 0x7ffaf406f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffaf415af90 0x7ffaf406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4168560 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf406f670 con 0x7ffaf40cd0d0 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf41686e0 con 0x7ffaf415af90 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40b2bf0 0x7ffaf4141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40b2bf0 0x7ffaf4141890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33416/0 (socket says 172.21.15.44:33416) 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf3e00640 1 -- 172.21.15.44:0/113607177 learned_addr learned my addr 172.21.15.44:0/113607177 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaebe00640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaf40cd0d0 0x7ffaf406f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf9200640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffaf415af90 0x7ffaf406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf3e00640 1 -- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffaf415af90 msgr2=0x7ffaf406fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf3e00640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffaf415af90 0x7ffaf406fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf3e00640 1 -- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaf40cd0d0 msgr2=0x7ffaf406f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf3e00640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaf40cd0d0 0x7ffaf406f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf3e00640 1 -- 172.21.15.44:0/113607177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf4141dd0 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaebe00640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaf40cd0d0 0x7ffaf406f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:47:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf9200640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffaf415af90 0x7ffaf406fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:47:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf3e00640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40b2bf0 0x7ffaf4141890 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ffae40058d0 tx=0x7ffae400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf0c00640 1 -- 172.21.15.44:0/113607177 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4017070 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf0c00640 1 -- 172.21.15.44:0/113607177 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffae4009d20 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffaf0c00640 1 -- 172.21.15.44:0/113607177 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae40106c0 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 -- 172.21.15.44:0/113607177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf409aab0 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 -- 172.21.15.44:0/113607177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf409b070 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.176+0000 7ffafb200640 1 -- 172.21.15.44:0/113607177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf406add0 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.180+0000 7ffaf0c00640 1 -- 172.21.15.44:0/113607177 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7ffae401a070 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.180+0000 7ffaf0c00640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 0x7ffad00764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:08.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.180+0000 7ffaf0c00640 1 -- 172.21.15.44:0/113607177 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5793+0+0 (secure 0 0 0) 0x7ffae4092f20 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.180+0000 7ffaebe00640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 0x7ffad00764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:08.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.180+0000 7ffaebe00640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 0x7ffad00764f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0af000 tx=0x7ffadc01b0f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:08.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.184+0000 7ffaf0c00640 1 -- 172.21.15.44:0/113607177 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ffae405f870 con 0x7ffaf40b2bf0 2024-09-13T08:47:08.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:08.400+0000 7ffafb200640 1 -- 172.21.15.44:0/113607177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffaf4155b10 con 0x7ffaf40b2bf0 2024-09-13T08:47:09.223 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-09-13T08:47:09.220+0000 7f7d83200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:47:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.268+0000 7ffaf0c00640 1 -- 172.21.15.44:0/113607177 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) ==== 118+0+0 (secure 0 0 0) 0x7ffae405f170 con 0x7ffaf40b2bf0 2024-09-13T08:47:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 -- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 msgr2=0x7ffad00764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 0x7ffad00764f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0af000 tx=0x7ffadc01b0f0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 -- 172.21.15.44:0/113607177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40b2bf0 msgr2=0x7ffaf4141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40b2bf0 0x7ffaf4141890 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ffae40058d0 tx=0x7ffae400dde0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 -- 172.21.15.44:0/113607177 shutdown_connections 2024-09-13T08:47:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffad0074030 0x7ffad00764f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffaf415af90 0x7ffaf406fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaf40cd0d0 0x7ffaf406f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 --2- 172.21.15.44:0/113607177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40b2bf0 0x7ffaf4141890 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 -- 172.21.15.44:0/113607177 >> 172.21.15.44:0/113607177 conn(0x7ffaf40b18f0 msgr2=0x7ffaf4151d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 -- 172.21.15.44:0/113607177 shutdown_connections 2024-09-13T08:47:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.272+0000 7ffafb200640 1 -- 172.21.15.44:0/113607177 wait complete. 2024-09-13T08:47:09.297 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T08:47:09.297 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T08:47:09.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 -- 172.21.15.44:0/2619398648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4a00cd100 msgr2=0x7fb4a00b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:09.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 --2- 172.21.15.44:0/2619398648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4a00cd100 0x7fb4a00b42d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb48800aa10 tx=0x7fb48801dba0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 -- 172.21.15.44:0/2619398648 shutdown_connections 2024-09-13T08:47:09.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 --2- 172.21.15.44:0/2619398648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4a00cd100 0x7fb4a00b42d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 --2- 172.21.15.44:0/2619398648 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4a015dae0 0x7fb4a00acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 -- 172.21.15.44:0/2619398648 >> 172.21.15.44:0/2619398648 conn(0x7fb4a00b18f0 msgr2=0x7fb4a00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:09.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 -- 172.21.15.44:0/2619398648 shutdown_connections 2024-09-13T08:47:09.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 -- 172.21.15.44:0/2619398648 wait complete. 2024-09-13T08:47:09.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 Processor -- start 2024-09-13T08:47:09.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 -- start start 2024-09-13T08:47:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4a00b4810 0x7fb4a01fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a00cd100 0x7fb4a0149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4a015dae0 0x7fb4a01f8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01674b0 con 0x7fb4a00b4810 2024-09-13T08:47:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0167330 con 0x7fb4a00cd100 2024-09-13T08:47:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb4a6600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a0167630 con 0x7fb4a015dae0 2024-09-13T08:47:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb49f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4a00b4810 0x7fb4a01fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb49f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4a00b4810 0x7fb4a01fc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33422/0 (socket says 172.21.15.44:33422) 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb49f400640 1 -- 172.21.15.44:0/1719704741 learned_addr learned my addr 172.21.15.44:0/1719704741 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb496a00640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a00cd100 0x7fb4a0149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.452+0000 7fb49fe00640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4a015dae0 0x7fb4a01f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb49f400640 1 -- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4a015dae0 msgr2=0x7fb4a01f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb49f400640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4a015dae0 0x7fb4a01f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb49f400640 1 -- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a00cd100 msgr2=0x7fb4a0149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb49f400640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a00cd100 0x7fb4a0149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb49f400640 1 -- 172.21.15.44:0/1719704741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a0149e70 con 0x7fb4a00b4810 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb49fe00640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4a015dae0 0x7fb4a01f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb49f400640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4a00b4810 0x7fb4a01fc160 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb49000d720 tx=0x7fb49000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb497e00640 1 -- 172.21.15.44:0/1719704741 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490017070 con 0x7fb4a00b4810 2024-09-13T08:47:09.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb497e00640 1 -- 172.21.15.44:0/1719704741 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb490009d20 con 0x7fb4a00b4810 2024-09-13T08:47:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb497e00640 1 -- 172.21.15.44:0/1719704741 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490005280 con 0x7fb4a00b4810 2024-09-13T08:47:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb4a6600640 1 -- 172.21.15.44:0/1719704741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a014a0e0 con 0x7fb4a00b4810 2024-09-13T08:47:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb4a6600640 1 -- 172.21.15.44:0/1719704741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a014a6a0 con 0x7fb4a00b4810 2024-09-13T08:47:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb4a6600640 1 -- 172.21.15.44:0/1719704741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a0069410 con 0x7fb4a00b4810 2024-09-13T08:47:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb497e00640 1 -- 172.21.15.44:0/1719704741 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7fb490005420 con 0x7fb4a00b4810 2024-09-13T08:47:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb497e00640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb47c074030 0x7fb47c0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.456+0000 7fb497e00640 1 -- 172.21.15.44:0/1719704741 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fb490092d40 con 0x7fb4a00b4810 2024-09-13T08:47:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.460+0000 7fb496a00640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb47c074030 0x7fb47c0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:09.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.460+0000 7fb496a00640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb47c074030 0x7fb47c0764f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb4880b0000 tx=0x7fb48801c3e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.464+0000 7fb497e00640 1 -- 172.21.15.44:0/1719704741 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb49005f040 con 0x7fb4a00b4810 2024-09-13T08:47:09.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:09.672+0000 7fb4a6600640 1 -- 172.21.15.44:0/1719704741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb4a00b42d0 con 0x7fb4a00b4810 2024-09-13T08:47:10.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.268+0000 7fb497e00640 1 -- 172.21.15.44:0/1719704741 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7fb490063a70 con 0x7fb4a00b4810 2024-09-13T08:47:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 -- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb47c074030 msgr2=0x7fb47c0764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb47c074030 0x7fb47c0764f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb4880b0000 tx=0x7fb48801c3e0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 -- 172.21.15.44:0/1719704741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4a00b4810 msgr2=0x7fb4a01fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4a00b4810 0x7fb4a01fc160 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb49000d720 tx=0x7fb49000dbe0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 -- 172.21.15.44:0/1719704741 shutdown_connections 2024-09-13T08:47:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb47c074030 0x7fb47c0764f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4a015dae0 0x7fb4a01f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a00cd100 0x7fb4a0149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 --2- 172.21.15.44:0/1719704741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4a00b4810 0x7fb4a01fc160 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 -- 172.21.15.44:0/1719704741 >> 172.21.15.44:0/1719704741 conn(0x7fb4a00b18f0 msgr2=0x7fb4a0159710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 -- 172.21.15.44:0/1719704741 shutdown_connections 2024-09-13T08:47:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.272+0000 7fb4a6600640 1 -- 172.21.15.44:0/1719704741 wait complete. 2024-09-13T08:47:10.298 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T08:47:10.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:47:10.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 -- 172.21.15.44:0/1566159268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc0b42d0 msgr2=0x7facdc0b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 --2- 172.21.15.44:0/1566159268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc0b42d0 0x7facdc0b4710 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7facc4009900 tx=0x7facc401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 -- 172.21.15.44:0/1566159268 shutdown_connections 2024-09-13T08:47:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 --2- 172.21.15.44:0/1566159268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc0b42d0 0x7facdc0b4710 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 --2- 172.21.15.44:0/1566159268 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facdc15a660 0x7facdc0cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 -- 172.21.15.44:0/1566159268 >> 172.21.15.44:0/1566159268 conn(0x7facdc0b18f0 msgr2=0x7facdc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 -- 172.21.15.44:0/1566159268 shutdown_connections 2024-09-13T08:47:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 -- 172.21.15.44:0/1566159268 wait complete. 2024-09-13T08:47:10.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 Processor -- start 2024-09-13T08:47:10.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 -- start start 2024-09-13T08:47:10.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc072d00 0x7facdc1f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facdc0b42d0 0x7facdc0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facdc15a660 0x7facdc1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7facdc167190 con 0x7facdc072d00 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7facdc073fb0 con 0x7facdc15a660 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7face0e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7facdc167310 con 0x7facdc0b42d0 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facd9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc072d00 0x7facdc1f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facd9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc072d00 0x7facdc1f8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33452/0 (socket says 172.21.15.44:33452) 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facd9e00640 1 -- 172.21.15.44:0/1706533852 learned_addr learned my addr 172.21.15.44:0/1706533852 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facd1400640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facdc0b42d0 0x7facdc0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facda800640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facdc15a660 0x7facdc1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facd9e00640 1 -- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facdc0b42d0 msgr2=0x7facdc0c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facd9e00640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facdc0b42d0 0x7facdc0c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facd9e00640 1 -- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facdc15a660 msgr2=0x7facdc1502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facd9e00640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facdc15a660 0x7facdc1502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facd9e00640 1 -- 172.21.15.44:0/1706533852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc1f90a0 con 0x7facdc072d00 2024-09-13T08:47:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.448+0000 7facd9e00640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc072d00 0x7facdc1f8b60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7faccc00dba0 tx=0x7faccc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.452+0000 7facd2a00640 1 -- 172.21.15.44:0/1706533852 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc005220 con 0x7facdc072d00 2024-09-13T08:47:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.452+0000 7facd2a00640 1 -- 172.21.15.44:0/1706533852 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faccc00ae50 con 0x7facdc072d00 2024-09-13T08:47:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.452+0000 7facd2a00640 1 -- 172.21.15.44:0/1706533852 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc0047b0 con 0x7facdc072d00 2024-09-13T08:47:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.452+0000 7face0e00640 1 -- 172.21.15.44:0/1706533852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc0cba00 con 0x7facdc072d00 2024-09-13T08:47:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.452+0000 7face0e00640 1 -- 172.21.15.44:0/1706533852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc151810 con 0x7facdc072d00 2024-09-13T08:47:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.452+0000 7face0e00640 1 -- 172.21.15.44:0/1706533852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc151b40 con 0x7facdc072d00 2024-09-13T08:47:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.452+0000 7facd2a00640 1 -- 172.21.15.44:0/1706533852 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7faccc004950 con 0x7facdc072d00 2024-09-13T08:47:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.452+0000 7facd2a00640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb8074030 0x7facb80764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.452+0000 7facd2a00640 1 -- 172.21.15.44:0/1706533852 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7faccc093350 con 0x7facdc072d00 2024-09-13T08:47:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.452+0000 7facd1400640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb8074030 0x7facb80764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.456+0000 7facd1400640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb8074030 0x7facb80764f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7facc40af000 tx=0x7facc401b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.456+0000 7facd2a00640 1 -- 172.21.15.44:0/1706533852 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7facdc1f90a0 con 0x7facdc072d00 2024-09-13T08:47:10.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 -- 172.21.15.44:0/1706533852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facdc069410 con 0x7facdc072d00 2024-09-13T08:47:10.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7facd2a00640 1 -- 172.21.15.44:0/1706533852 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5228 (secure 0 0 0) 0x7faccc05fd00 con 0x7facdc072d00 2024-09-13T08:47:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:10.674 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":56,"btime":"2024-09-13T08:47:10:272875+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/833446483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":833446483},{"type":"v1","addr":"172.21.15.129:6836","nonce":833446483}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1844578230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1844578230},{"type":"v1","addr":"172.21.15.129:6837","nonce":1844578230}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2007417535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2007417535},{"type":"v1","addr":"172.21.15.44:6833","nonce":2007417535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:47:06.650500+0000","modified":"2024-09-13T08:47:09.677516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6942":{"gid":6942,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1796790158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1796790158},{"type":"v1","addr":"172.21.15.44:6835","nonce":1796790158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6942,"qdb_cluster":[6942]},"id":6}]} 2024-09-13T08:47:10.674 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 56 2024-09-13T08:47:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 -- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb8074030 msgr2=0x7facb80764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb8074030 0x7facb80764f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7facc40af000 tx=0x7facc401b120 comp rx=0 tx=0).stop 2024-09-13T08:47:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 -- 172.21.15.44:0/1706533852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc072d00 msgr2=0x7facdc1f8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc072d00 0x7facdc1f8b60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7faccc00dba0 tx=0x7faccc00a620 comp rx=0 tx=0).stop 2024-09-13T08:47:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 -- 172.21.15.44:0/1706533852 shutdown_connections 2024-09-13T08:47:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb8074030 0x7facb80764f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facdc15a660 0x7facdc1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facdc0b42d0 0x7facdc0c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 --2- 172.21.15.44:0/1706533852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc072d00 0x7facdc1f8b60 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 -- 172.21.15.44:0/1706533852 >> 172.21.15.44:0/1706533852 conn(0x7facdc0b18f0 msgr2=0x7facdc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 -- 172.21.15.44:0/1706533852 shutdown_connections 2024-09-13T08:47:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.672+0000 7face0e00640 1 -- 172.21.15.44:0/1706533852 wait complete. 2024-09-13T08:47:10.695 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:47:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.840+0000 7f88c4a00640 1 -- 172.21.15.44:0/3232120025 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88c015bd50 msgr2=0x7f88c00ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.840+0000 7f88c4a00640 1 --2- 172.21.15.44:0/3232120025 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88c015bd50 0x7f88c00ce5e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009900 tx=0x7f88b001c7c0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.840+0000 7f88c4a00640 1 -- 172.21.15.44:0/3232120025 shutdown_connections 2024-09-13T08:47:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.840+0000 7f88c4a00640 1 --2- 172.21.15.44:0/3232120025 >> v2:172.21.15.129:3300/0 conn(0x7f88c00cf4e0 0x7f88c00cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.840+0000 7f88c4a00640 1 --2- 172.21.15.44:0/3232120025 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88c015bd50 0x7f88c00ce5e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.840+0000 7f88c4a00640 1 -- 172.21.15.44:0/3232120025 >> 172.21.15.44:0/3232120025 conn(0x7f88c00d1ea0 msgr2=0x7f88c00d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.840+0000 7f88c4a00640 1 -- 172.21.15.44:0/3232120025 shutdown_connections 2024-09-13T08:47:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.840+0000 7f88c4a00640 1 -- 172.21.15.44:0/3232120025 wait complete. 2024-09-13T08:47:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.840+0000 7f88c4a00640 1 Processor -- start 2024-09-13T08:47:10.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.840+0000 7f88c4a00640 1 -- start start 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88c4a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88c00cf4e0 0x7f88c0152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88c4a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88c015bd50 0x7f88c00711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88c4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c015c130 0x7f88c0071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88c4a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0168e50 con 0x7f88c015c130 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88c4a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0168cd0 con 0x7f88c015bd50 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88c4a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f88c0168fd0 con 0x7f88c00cf4e0 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88be800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c015c130 0x7f88c0071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88be800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c015c130 0x7f88c0071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33454/0 (socket says 172.21.15.44:33454) 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88be800640 1 -- 172.21.15.44:0/1660989903 learned_addr learned my addr 172.21.15.44:0/1660989903 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88bde00640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88c00cf4e0 0x7f88c0152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88b7e00640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88c015bd50 0x7f88c00711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88be800640 1 -- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88c00cf4e0 msgr2=0x7f88c0152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88be800640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88c00cf4e0 0x7f88c0152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88be800640 1 -- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88c015bd50 msgr2=0x7f88c00711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88be800640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88c015bd50 0x7f88c00711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88be800640 1 -- 172.21.15.44:0/1660989903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c00c9a40 con 0x7f88c015c130 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88be800640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c015c130 0x7f88c0071720 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f88b800eb20 tx=0x7f88b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88b6000640 1 -- 172.21.15.44:0/1660989903 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8004b30 con 0x7f88c015c130 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88b6000640 1 -- 172.21.15.44:0/1660989903 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88b8004cf0 con 0x7f88c015c130 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88c4a00640 1 -- 172.21.15.44:0/1660989903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c0156040 con 0x7f88c015c130 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88b6000640 1 -- 172.21.15.44:0/1660989903 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8005280 con 0x7f88c015c130 2024-09-13T08:47:10.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88c4a00640 1 -- 172.21.15.44:0/1660989903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c0152590 con 0x7f88c015c130 2024-09-13T08:47:10.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88c4a00640 1 -- 172.21.15.44:0/1660989903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c00ce5e0 con 0x7f88c015c130 2024-09-13T08:47:10.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.844+0000 7f88b6000640 1 -- 172.21.15.44:0/1660989903 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f88b8005420 con 0x7f88c015c130 2024-09-13T08:47:10.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.848+0000 7f88b6000640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f889c074030 0x7f889c0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:10.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.848+0000 7f88b6000640 1 -- 172.21.15.44:0/1660989903 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f88b8093910 con 0x7f88c015c130 2024-09-13T08:47:10.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.848+0000 7f88bde00640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f889c074030 0x7f889c0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:10.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.848+0000 7f88bde00640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f889c074030 0x7f889c0764f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f88b00af000 tx=0x7f88b001b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:10.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:10.852+0000 7f88b6000640 1 -- 172.21.15.44:0/1660989903 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f88b805fc10 con 0x7f88c015c130 2024-09-13T08:47:11.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.028+0000 7f88c4a00640 1 -- 172.21.15.44:0/1660989903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88c00cfaf0 con 0x7f88c015c130 2024-09-13T08:47:11.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.028+0000 7f88b6000640 1 -- 172.21.15.44:0/1660989903 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16876 (secure 0 0 0) 0x7f88b8064640 con 0x7f88c015c130 2024-09-13T08:47:11.031 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:11.031 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":89,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:47:08.253903+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:47:04.371378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T08:47:05.614930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:47:11.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 -- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f889c074030 msgr2=0x7f889c0764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f889c074030 0x7f889c0764f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f88b00af000 tx=0x7f88b001b000 comp rx=0 tx=0).stop 2024-09-13T08:47:11.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 -- 172.21.15.44:0/1660989903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c015c130 msgr2=0x7f88c0071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c015c130 0x7f88c0071720 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f88b800eb20 tx=0x7f88b800c2d0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 -- 172.21.15.44:0/1660989903 shutdown_connections 2024-09-13T08:47:11.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f889c074030 0x7f889c0764f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c015c130 0x7f88c0071720 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88c015bd50 0x7f88c00711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 --2- 172.21.15.44:0/1660989903 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f88c00cf4e0 0x7f88c0152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 -- 172.21.15.44:0/1660989903 >> 172.21.15.44:0/1660989903 conn(0x7f88c00d1ea0 msgr2=0x7f88c00c7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:11.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 -- 172.21.15.44:0/1660989903 shutdown_connections 2024-09-13T08:47:11.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.032+0000 7f88c4a00640 1 -- 172.21.15.44:0/1660989903 wait complete. 2024-09-13T08:47:11.052 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:47:11.052 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:47:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.196+0000 7f4b2e000640 1 -- 172.21.15.44:0/2691311534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2815c1c0 msgr2=0x7f4b2806d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.196+0000 7f4b2e000640 1 --2- 172.21.15.44:0/2691311534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2815c1c0 0x7f4b2806d000 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009840 tx=0x7f4b1001c7f0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.196+0000 7f4b2e000640 1 -- 172.21.15.44:0/2691311534 shutdown_connections 2024-09-13T08:47:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.196+0000 7f4b2e000640 1 --2- 172.21.15.44:0/2691311534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2815c1c0 0x7f4b2806d000 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.196+0000 7f4b2e000640 1 --2- 172.21.15.44:0/2691311534 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b2806cc20 0x7f4b280737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.196+0000 7f4b2e000640 1 -- 172.21.15.44:0/2691311534 >> 172.21.15.44:0/2691311534 conn(0x7f4b280d0c90 msgr2=0x7f4b280d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.196+0000 7f4b2e000640 1 -- 172.21.15.44:0/2691311534 shutdown_connections 2024-09-13T08:47:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.196+0000 7f4b2e000640 1 -- 172.21.15.44:0/2691311534 wait complete. 2024-09-13T08:47:11.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b2e000640 1 Processor -- start 2024-09-13T08:47:11.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b2e000640 1 -- start start 2024-09-13T08:47:11.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b2e000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2806cc20 0x7f4b280b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:11.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b2e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2815c1c0 0x7f4b28158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:11.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b2e000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b2815f7b0 0x7f4b2815e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:11.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b2e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28156600 con 0x7f4b2815c1c0 2024-09-13T08:47:11.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b2e000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28156480 con 0x7f4b2806cc20 2024-09-13T08:47:11.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b2e000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4b28169360 con 0x7f4b2815f7b0 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2815c1c0 0x7f4b28158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2815c1c0 0x7f4b28158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33470/0 (socket says 172.21.15.44:33470) 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1fe00640 1 -- 172.21.15.44:0/947468272 learned_addr learned my addr 172.21.15.44:0/947468272 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b27400640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2806cc20 0x7f4b280b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b27e00640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b2815f7b0 0x7f4b2815e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1fe00640 1 -- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b2815f7b0 msgr2=0x7f4b2815e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1fe00640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b2815f7b0 0x7f4b2815e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1fe00640 1 -- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2806cc20 msgr2=0x7f4b280b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1fe00640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2806cc20 0x7f4b280b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1fe00640 1 -- 172.21.15.44:0/947468272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b28160180 con 0x7f4b2815c1c0 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1fe00640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2815c1c0 0x7f4b28158db0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4b10000c10 tx=0x7f4b10005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1f400640 1 -- 172.21.15.44:0/947468272 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100b2040 con 0x7f4b2815c1c0 2024-09-13T08:47:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1f400640 1 -- 172.21.15.44:0/947468272 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b10004330 con 0x7f4b2815c1c0 2024-09-13T08:47:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b1f400640 1 -- 172.21.15.44:0/947468272 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10005220 con 0x7f4b2815c1c0 2024-09-13T08:47:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b2e000640 1 -- 172.21.15.44:0/947468272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b280b0dd0 con 0x7f4b2815c1c0 2024-09-13T08:47:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.200+0000 7f4b2e000640 1 -- 172.21.15.44:0/947468272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b280b1390 con 0x7f4b2815c1c0 2024-09-13T08:47:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.204+0000 7f4b2e000640 1 -- 172.21.15.44:0/947468272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b280737c0 con 0x7f4b2815c1c0 2024-09-13T08:47:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.204+0000 7f4b1f400640 1 -- 172.21.15.44:0/947468272 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f4b100bd060 con 0x7f4b2815c1c0 2024-09-13T08:47:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.204+0000 7f4b1f400640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b04074030 0x7f4b040764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.204+0000 7f4b1f400640 1 -- 172.21.15.44:0/947468272 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f4b1012c620 con 0x7f4b2815c1c0 2024-09-13T08:47:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.204+0000 7f4b27400640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b04074030 0x7f4b040764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.204+0000 7f4b27400640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b04074030 0x7f4b040764f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009600 tx=0x7f4b180091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.208+0000 7f4b1f400640 1 -- 172.21.15.44:0/947468272 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4b10016410 con 0x7f4b2815c1c0 2024-09-13T08:47:11.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.380+0000 7f4b2e000640 1 -- 172.21.15.44:0/947468272 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b280b6c10 con 0x7f4b04074030 2024-09-13T08:47:11.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.388+0000 7f4b1f400640 1 -- 172.21.15.44:0/947468272 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252071 (secure 0 0 0) 0x7f4b280b6c10 con 0x7f4b04074030 2024-09-13T08:47:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:11.393 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 -- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b04074030 msgr2=0x7f4b040764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b04074030 0x7f4b040764f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009600 tx=0x7f4b180091e0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 -- 172.21.15.44:0/947468272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2815c1c0 msgr2=0x7f4b28158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2815c1c0 0x7f4b28158db0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4b10000c10 tx=0x7f4b10005db0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 -- 172.21.15.44:0/947468272 shutdown_connections 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4b04074030 0x7f4b040764f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4b2815f7b0 0x7f4b2815e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b2815c1c0 0x7f4b28158db0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 --2- 172.21.15.44:0/947468272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2806cc20 0x7f4b280b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 -- 172.21.15.44:0/947468272 >> 172.21.15.44:0/947468272 conn(0x7f4b280d0c90 msgr2=0x7f4b28154a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 -- 172.21.15.44:0/947468272 shutdown_connections 2024-09-13T08:47:11.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.392+0000 7f4b2e000640 1 -- 172.21.15.44:0/947468272 wait complete. 2024-09-13T08:47:11.413 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-13T08:47:10.687869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481140,"kb_used_data":9644,"kb_used_omap":12,"kb_used_meta":471283,"kb_avail":754493580,"statfs":{"total":773094113280,"available":772601425920,"internally_reserved":0,"allocated":9875456,"data_stored":4903620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":482594353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.459018"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779207+0000","last_change":"2024-09-13T08:47:07.283483+0000","last_active":"2024-09-13T08:47:08.779207+0000","last_peered":"2024-09-13T08:47:08.779207+0000","last_clean":"2024-09-13T08:47:08.779207+0000","last_became_active":"2024-09-13T08:47:07.279152+0000","last_became_peered":"2024-09-13T08:47:07.279152+0000","last_unstale":"2024-09-13T08:47:08.779207+0000","last_undegraded":"2024-09-13T08:47:08.779207+0000","last_fullsized":"2024-09-13T08:47:08.779207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:21.692069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266710+0000","last_change":"2024-09-13T08:47:07.286912+0000","last_active":"2024-09-13T08:47:08.266710+0000","last_peered":"2024-09-13T08:47:08.266710+0000","last_clean":"2024-09-13T08:47:08.266710+0000","last_became_active":"2024-09-13T08:47:07.286580+0000","last_became_peered":"2024-09-13T08:47:07.286580+0000","last_unstale":"2024-09-13T08:47:08.266710+0000","last_undegraded":"2024-09-13T08:47:08.266710+0000","last_fullsized":"2024-09-13T08:47:08.266710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:20.294539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264608+0000","last_change":"2024-09-13T08:47:07.277078+0000","last_active":"2024-09-13T08:47:08.264608+0000","last_peered":"2024-09-13T08:47:08.264608+0000","last_clean":"2024-09-13T08:47:08.264608+0000","last_became_active":"2024-09-13T08:47:07.276464+0000","last_became_peered":"2024-09-13T08:47:07.276464+0000","last_unstale":"2024-09-13T08:47:08.264608+0000","last_undegraded":"2024-09-13T08:47:08.264608+0000","last_fullsized":"2024-09-13T08:47:08.264608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:33.682366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.268253+0000","last_change":"2024-09-13T08:47:07.283154+0000","last_active":"2024-09-13T08:47:08.268253+0000","last_peered":"2024-09-13T08:47:08.268253+0000","last_clean":"2024-09-13T08:47:08.268253+0000","last_became_active":"2024-09-13T08:47:07.282934+0000","last_became_peered":"2024-09-13T08:47:07.282934+0000","last_unstale":"2024-09-13T08:47:08.268253+0000","last_undegraded":"2024-09-13T08:47:08.268253+0000","last_fullsized":"2024-09-13T08:47:08.268253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:48.582724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777803+0000","last_change":"2024-09-13T08:47:07.256022+0000","last_active":"2024-09-13T08:47:08.777803+0000","last_peered":"2024-09-13T08:47:08.777803+0000","last_clean":"2024-09-13T08:47:08.777803+0000","last_became_active":"2024-09-13T08:47:07.255702+0000","last_became_peered":"2024-09-13T08:47:07.255702+0000","last_unstale":"2024-09-13T08:47:08.777803+0000","last_undegraded":"2024-09-13T08:47:08.777803+0000","last_fullsized":"2024-09-13T08:47:08.777803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:08.772346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264100+0000","last_change":"2024-09-13T08:47:07.267150+0000","last_active":"2024-09-13T08:47:08.264100+0000","last_peered":"2024-09-13T08:47:08.264100+0000","last_clean":"2024-09-13T08:47:08.264100+0000","last_became_active":"2024-09-13T08:47:07.266861+0000","last_became_peered":"2024-09-13T08:47:07.266861+0000","last_unstale":"2024-09-13T08:47:08.264100+0000","last_undegraded":"2024-09-13T08:47:08.264100+0000","last_fullsized":"2024-09-13T08:47:08.264100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:36:06.063894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266737+0000","last_change":"2024-09-13T08:47:07.265958+0000","last_active":"2024-09-13T08:47:08.266737+0000","last_peered":"2024-09-13T08:47:08.266737+0000","last_clean":"2024-09-13T08:47:08.266737+0000","last_became_active":"2024-09-13T08:47:07.265670+0000","last_became_peered":"2024-09-13T08:47:07.265670+0000","last_unstale":"2024-09-13T08:47:08.266737+0000","last_undegraded":"2024-09-13T08:47:08.266737+0000","last_fullsized":"2024-09-13T08:47:08.266737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:48.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777359+0000","last_change":"2024-09-13T08:47:07.272690+0000","last_active":"2024-09-13T08:47:08.777359+0000","last_peered":"2024-09-13T08:47:08.777359+0000","last_clean":"2024-09-13T08:47:08.777359+0000","last_became_active":"2024-09-13T08:47:07.272340+0000","last_became_peered":"2024-09-13T08:47:07.272340+0000","last_unstale":"2024-09-13T08:47:08.777359+0000","last_undegraded":"2024-09-13T08:47:08.777359+0000","last_fullsized":"2024-09-13T08:47:08.777359+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:46:20.209257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779400+0000","last_change":"2024-09-13T08:47:07.272294+0000","last_active":"2024-09-13T08:47:08.779400+0000","last_peered":"2024-09-13T08:47:08.779400+0000","last_clean":"2024-09-13T08:47:08.779400+0000","last_became_active":"2024-09-13T08:47:07.272007+0000","last_became_peered":"2024-09-13T08:47:07.272007+0000","last_unstale":"2024-09-13T08:47:08.779400+0000","last_undegraded":"2024-09-13T08:47:08.779400+0000","last_fullsized":"2024-09-13T08:47:08.779400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:27.785222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264312+0000","last_change":"2024-09-13T08:47:07.281399+0000","last_active":"2024-09-13T08:47:08.264312+0000","last_peered":"2024-09-13T08:47:08.264312+0000","last_clean":"2024-09-13T08:47:08.264312+0000","last_became_active":"2024-09-13T08:47:07.277660+0000","last_became_peered":"2024-09-13T08:47:07.277660+0000","last_unstale":"2024-09-13T08:47:08.264312+0000","last_undegraded":"2024-09-13T08:47:08.264312+0000","last_fullsized":"2024-09-13T08:47:08.264312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:13.488445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.282812+0000","last_change":"2024-09-13T08:47:07.280911+0000","last_active":"2024-09-13T08:47:07.282812+0000","last_peered":"2024-09-13T08:47:07.282812+0000","last_clean":"2024-09-13T08:47:07.282812+0000","last_became_active":"2024-09-13T08:47:07.280626+0000","last_became_peered":"2024-09-13T08:47:07.280626+0000","last_unstale":"2024-09-13T08:47:07.282812+0000","last_undegraded":"2024-09-13T08:47:07.282812+0000","last_fullsized":"2024-09-13T08:47:07.282812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:16:18.265753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778460+0000","last_change":"2024-09-13T08:47:07.273190+0000","last_active":"2024-09-13T08:47:08.778460+0000","last_peered":"2024-09-13T08:47:08.778460+0000","last_clean":"2024-09-13T08:47:08.778460+0000","last_became_active":"2024-09-13T08:47:07.272825+0000","last_became_peered":"2024-09-13T08:47:07.272825+0000","last_unstale":"2024-09-13T08:47:08.778460+0000","last_undegraded":"2024-09-13T08:47:08.778460+0000","last_fullsized":"2024-09-13T08:47:08.778460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:17:54.456895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777832+0000","last_change":"2024-09-13T08:47:07.277642+0000","last_active":"2024-09-13T08:47:08.777832+0000","last_peered":"2024-09-13T08:47:08.777832+0000","last_clean":"2024-09-13T08:47:08.777832+0000","last_became_active":"2024-09-13T08:47:07.277283+0000","last_became_peered":"2024-09-13T08:47:07.277283+0000","last_unstale":"2024-09-13T08:47:08.777832+0000","last_undegraded":"2024-09-13T08:47:08.777832+0000","last_fullsized":"2024-09-13T08:47:08.777832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:37.494571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778630+0000","last_change":"2024-09-13T08:47:07.259608+0000","last_active":"2024-09-13T08:47:08.778630+0000","last_peered":"2024-09-13T08:47:08.778630+0000","last_clean":"2024-09-13T08:47:08.778630+0000","last_became_active":"2024-09-13T08:47:07.257983+0000","last_became_peered":"2024-09-13T08:47:07.257983+0000","last_unstale":"2024-09-13T08:47:08.778630+0000","last_undegraded":"2024-09-13T08:47:08.778630+0000","last_fullsized":"2024-09-13T08:47:08.778630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:36.886734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266996+0000","last_change":"2024-09-13T08:47:07.265243+0000","last_active":"2024-09-13T08:47:08.266996+0000","last_peered":"2024-09-13T08:47:08.266996+0000","last_clean":"2024-09-13T08:47:08.266996+0000","last_became_active":"2024-09-13T08:47:07.264708+0000","last_became_peered":"2024-09-13T08:47:07.264708+0000","last_unstale":"2024-09-13T08:47:08.266996+0000","last_undegraded":"2024-09-13T08:47:08.266996+0000","last_fullsized":"2024-09-13T08:47:08.266996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:33.256378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264006+0000","last_change":"2024-09-13T08:47:07.268059+0000","last_active":"2024-09-13T08:47:08.264006+0000","last_peered":"2024-09-13T08:47:08.264006+0000","last_clean":"2024-09-13T08:47:08.264006+0000","last_became_active":"2024-09-13T08:47:07.267171+0000","last_became_peered":"2024-09-13T08:47:07.267171+0000","last_unstale":"2024-09-13T08:47:08.264006+0000","last_undegraded":"2024-09-13T08:47:08.264006+0000","last_fullsized":"2024-09-13T08:47:08.264006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:17:54.456895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779325+0000","last_change":"2024-09-13T08:47:07.272849+0000","last_active":"2024-09-13T08:47:08.779325+0000","last_peered":"2024-09-13T08:47:08.779325+0000","last_clean":"2024-09-13T08:47:08.779325+0000","last_became_active":"2024-09-13T08:47:07.272555+0000","last_became_peered":"2024-09-13T08:47:07.272555+0000","last_unstale":"2024-09-13T08:47:08.779325+0000","last_undegraded":"2024-09-13T08:47:08.779325+0000","last_fullsized":"2024-09-13T08:47:08.779325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:09.870318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.279947+0000","last_change":"2024-09-13T08:47:07.277413+0000","last_active":"2024-09-13T08:47:07.279947+0000","last_peered":"2024-09-13T08:47:07.279947+0000","last_clean":"2024-09-13T08:47:07.279947+0000","last_became_active":"2024-09-13T08:47:07.277111+0000","last_became_peered":"2024-09-13T08:47:07.277111+0000","last_unstale":"2024-09-13T08:47:07.279947+0000","last_undegraded":"2024-09-13T08:47:07.279947+0000","last_fullsized":"2024-09-13T08:47:07.279947+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:49:23.895427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778734+0000","last_change":"2024-09-13T08:47:07.261348+0000","last_active":"2024-09-13T08:47:08.778734+0000","last_peered":"2024-09-13T08:47:08.778734+0000","last_clean":"2024-09-13T08:47:08.778734+0000","last_became_active":"2024-09-13T08:47:07.261043+0000","last_became_peered":"2024-09-13T08:47:07.261043+0000","last_unstale":"2024-09-13T08:47:08.778734+0000","last_undegraded":"2024-09-13T08:47:08.778734+0000","last_fullsized":"2024-09-13T08:47:08.778734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:01.139461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778785+0000","last_change":"2024-09-13T08:47:07.260676+0000","last_active":"2024-09-13T08:47:08.778785+0000","last_peered":"2024-09-13T08:47:08.778785+0000","last_clean":"2024-09-13T08:47:08.778785+0000","last_became_active":"2024-09-13T08:47:07.260400+0000","last_became_peered":"2024-09-13T08:47:07.260400+0000","last_unstale":"2024-09-13T08:47:08.778785+0000","last_undegraded":"2024-09-13T08:47:08.778785+0000","last_fullsized":"2024-09-13T08:47:08.778785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:20.294539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.267317+0000","last_change":"2024-09-13T08:47:07.269103+0000","last_active":"2024-09-13T08:47:08.267317+0000","last_peered":"2024-09-13T08:47:08.267317+0000","last_clean":"2024-09-13T08:47:08.267317+0000","last_became_active":"2024-09-13T08:47:07.268832+0000","last_became_peered":"2024-09-13T08:47:07.268832+0000","last_unstale":"2024-09-13T08:47:08.267317+0000","last_undegraded":"2024-09-13T08:47:08.267317+0000","last_fullsized":"2024-09-13T08:47:08.267317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:09.622670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778570+0000","last_change":"2024-09-13T08:47:07.269690+0000","last_active":"2024-09-13T08:47:08.778570+0000","last_peered":"2024-09-13T08:47:08.778570+0000","last_clean":"2024-09-13T08:47:08.778570+0000","last_became_active":"2024-09-13T08:47:07.267738+0000","last_became_peered":"2024-09-13T08:47:07.267738+0000","last_unstale":"2024-09-13T08:47:08.778570+0000","last_undegraded":"2024-09-13T08:47:08.778570+0000","last_fullsized":"2024-09-13T08:47:08.778570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:46.802435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777440+0000","last_change":"2024-09-13T08:47:07.274583+0000","last_active":"2024-09-13T08:47:08.777440+0000","last_peered":"2024-09-13T08:47:08.777440+0000","last_clean":"2024-09-13T08:47:08.777440+0000","last_became_active":"2024-09-13T08:47:07.274283+0000","last_became_peered":"2024-09-13T08:47:07.274283+0000","last_unstale":"2024-09-13T08:47:08.777440+0000","last_undegraded":"2024-09-13T08:47:08.777440+0000","last_fullsized":"2024-09-13T08:47:08.777440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:42:49.364510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.276983+0000","last_change":"2024-09-13T08:47:07.271946+0000","last_active":"2024-09-13T08:47:07.276983+0000","last_peered":"2024-09-13T08:47:07.276983+0000","last_clean":"2024-09-13T08:47:07.276983+0000","last_became_active":"2024-09-13T08:47:07.271549+0000","last_became_peered":"2024-09-13T08:47:07.271549+0000","last_unstale":"2024-09-13T08:47:07.276983+0000","last_undegraded":"2024-09-13T08:47:07.276983+0000","last_fullsized":"2024-09-13T08:47:07.276983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:38.168348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.274385+0000","last_change":"2024-09-13T08:47:07.268903+0000","last_active":"2024-09-13T08:47:07.274385+0000","last_peered":"2024-09-13T08:47:07.274385+0000","last_clean":"2024-09-13T08:47:07.274385+0000","last_became_active":"2024-09-13T08:47:07.268596+0000","last_became_peered":"2024-09-13T08:47:07.268596+0000","last_unstale":"2024-09-13T08:47:07.274385+0000","last_undegraded":"2024-09-13T08:47:07.274385+0000","last_fullsized":"2024-09-13T08:47:07.274385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:08.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778187+0000","last_change":"2024-09-13T08:47:07.268586+0000","last_active":"2024-09-13T08:47:08.778187+0000","last_peered":"2024-09-13T08:47:08.778187+0000","last_clean":"2024-09-13T08:47:08.778187+0000","last_became_active":"2024-09-13T08:47:07.268318+0000","last_became_peered":"2024-09-13T08:47:07.268318+0000","last_unstale":"2024-09-13T08:47:08.778187+0000","last_undegraded":"2024-09-13T08:47:08.778187+0000","last_fullsized":"2024-09-13T08:47:08.778187+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:48.992201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778737+0000","last_change":"2024-09-13T08:47:07.258085+0000","last_active":"2024-09-13T08:47:08.778737+0000","last_peered":"2024-09-13T08:47:08.778737+0000","last_clean":"2024-09-13T08:47:08.778737+0000","last_became_active":"2024-09-13T08:47:07.257774+0000","last_became_peered":"2024-09-13T08:47:07.257774+0000","last_unstale":"2024-09-13T08:47:08.778737+0000","last_undegraded":"2024-09-13T08:47:08.778737+0000","last_fullsized":"2024-09-13T08:47:08.778737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:54.065712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779075+0000","last_change":"2024-09-13T08:47:07.267529+0000","last_active":"2024-09-13T08:47:08.779075+0000","last_peered":"2024-09-13T08:47:08.779075+0000","last_clean":"2024-09-13T08:47:08.779075+0000","last_became_active":"2024-09-13T08:47:07.267265+0000","last_became_peered":"2024-09-13T08:47:07.267265+0000","last_unstale":"2024-09-13T08:47:08.779075+0000","last_undegraded":"2024-09-13T08:47:08.779075+0000","last_fullsized":"2024-09-13T08:47:08.779075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:45.228354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.268102+0000","last_change":"2024-09-13T08:47:07.268860+0000","last_active":"2024-09-13T08:47:08.268102+0000","last_peered":"2024-09-13T08:47:08.268102+0000","last_clean":"2024-09-13T08:47:08.268102+0000","last_became_active":"2024-09-13T08:47:07.268584+0000","last_became_peered":"2024-09-13T08:47:07.268584+0000","last_unstale":"2024-09-13T08:47:08.268102+0000","last_undegraded":"2024-09-13T08:47:08.268102+0000","last_fullsized":"2024-09-13T08:47:08.268102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:07.201021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778870+0000","last_change":"2024-09-13T08:47:07.266408+0000","last_active":"2024-09-13T08:47:08.778870+0000","last_peered":"2024-09-13T08:47:08.778870+0000","last_clean":"2024-09-13T08:47:08.778870+0000","last_became_active":"2024-09-13T08:47:07.266119+0000","last_became_peered":"2024-09-13T08:47:07.266119+0000","last_unstale":"2024-09-13T08:47:08.778870+0000","last_undegraded":"2024-09-13T08:47:08.778870+0000","last_fullsized":"2024-09-13T08:47:08.778870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:48.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264333+0000","last_change":"2024-09-13T08:47:07.262631+0000","last_active":"2024-09-13T08:47:08.264333+0000","last_peered":"2024-09-13T08:47:08.264333+0000","last_clean":"2024-09-13T08:47:08.264333+0000","last_became_active":"2024-09-13T08:47:07.262271+0000","last_became_peered":"2024-09-13T08:47:07.262271+0000","last_unstale":"2024-09-13T08:47:08.264333+0000","last_undegraded":"2024-09-13T08:47:08.264333+0000","last_fullsized":"2024-09-13T08:47:08.264333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:09.870318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264058+0000","last_change":"2024-09-13T08:47:07.267526+0000","last_active":"2024-09-13T08:47:08.264058+0000","last_peered":"2024-09-13T08:47:08.264058+0000","last_clean":"2024-09-13T08:47:08.264058+0000","last_became_active":"2024-09-13T08:47:07.266102+0000","last_became_peered":"2024-09-13T08:47:07.266102+0000","last_unstale":"2024-09-13T08:47:08.264058+0000","last_undegraded":"2024-09-13T08:47:08.264058+0000","last_fullsized":"2024-09-13T08:47:08.264058+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:46.390729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778505+0000","last_change":"2024-09-13T08:47:07.286631+0000","last_active":"2024-09-13T08:47:08.778505+0000","last_peered":"2024-09-13T08:47:08.778505+0000","last_clean":"2024-09-13T08:47:08.778505+0000","last_became_active":"2024-09-13T08:47:07.285918+0000","last_became_peered":"2024-09-13T08:47:07.285918+0000","last_unstale":"2024-09-13T08:47:08.778505+0000","last_undegraded":"2024-09-13T08:47:08.778505+0000","last_fullsized":"2024-09-13T08:47:08.778505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:46.390729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"88'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778147+0000","last_change":"2024-09-13T08:47:06.252215+0000","last_active":"2024-09-13T08:47:08.778147+0000","last_peered":"2024-09-13T08:47:08.778147+0000","last_clean":"2024-09-13T08:47:08.778147+0000","last_became_active":"2024-09-13T08:47:06.251937+0000","last_became_peered":"2024-09-13T08:47:06.251937+0000","last_unstale":"2024-09-13T08:47:08.778147+0000","last_undegraded":"2024-09-13T08:47:08.778147+0000","last_fullsized":"2024-09-13T08:47:08.778147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:37.247845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779430+0000","last_change":"2024-09-13T08:47:07.279386+0000","last_active":"2024-09-13T08:47:08.779430+0000","last_peered":"2024-09-13T08:47:08.779430+0000","last_clean":"2024-09-13T08:47:08.779430+0000","last_became_active":"2024-09-13T08:47:07.278815+0000","last_became_peered":"2024-09-13T08:47:07.278815+0000","last_unstale":"2024-09-13T08:47:08.779430+0000","last_undegraded":"2024-09-13T08:47:08.779430+0000","last_fullsized":"2024-09-13T08:47:08.779430+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:40.607679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264383+0000","last_change":"2024-09-13T08:47:06.254635+0000","last_active":"2024-09-13T08:47:08.264383+0000","last_peered":"2024-09-13T08:47:08.264383+0000","last_clean":"2024-09-13T08:47:08.264383+0000","last_became_active":"2024-09-13T08:47:06.252778+0000","last_became_peered":"2024-09-13T08:47:06.252778+0000","last_unstale":"2024-09-13T08:47:08.264383+0000","last_undegraded":"2024-09-13T08:47:08.264383+0000","last_fullsized":"2024-09-13T08:47:08.264383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:22:03.994031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778681+0000","last_change":"2024-09-13T08:47:07.275040+0000","last_active":"2024-09-13T08:47:08.778681+0000","last_peered":"2024-09-13T08:47:08.778681+0000","last_clean":"2024-09-13T08:47:08.778681+0000","last_became_active":"2024-09-13T08:47:07.272521+0000","last_became_peered":"2024-09-13T08:47:07.272521+0000","last_unstale":"2024-09-13T08:47:08.778681+0000","last_undegraded":"2024-09-13T08:47:08.778681+0000","last_fullsized":"2024-09-13T08:47:08.778681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:36:06.063894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.283428+0000","last_change":"2024-09-13T08:47:06.255406+0000","last_active":"2024-09-13T08:47:07.283428+0000","last_peered":"2024-09-13T08:47:07.283428+0000","last_clean":"2024-09-13T08:47:07.283428+0000","last_became_active":"2024-09-13T08:47:06.255173+0000","last_became_peered":"2024-09-13T08:47:06.255173+0000","last_unstale":"2024-09-13T08:47:07.283428+0000","last_undegraded":"2024-09-13T08:47:07.283428+0000","last_fullsized":"2024-09-13T08:47:07.283428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:44.182064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779148+0000","last_change":"2024-09-13T08:47:07.270828+0000","last_active":"2024-09-13T08:47:08.779148+0000","last_peered":"2024-09-13T08:47:08.779148+0000","last_clean":"2024-09-13T08:47:08.779148+0000","last_became_active":"2024-09-13T08:47:07.270452+0000","last_became_peered":"2024-09-13T08:47:07.270452+0000","last_unstale":"2024-09-13T08:47:08.779148+0000","last_undegraded":"2024-09-13T08:47:08.779148+0000","last_fullsized":"2024-09-13T08:47:08.779148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:52.872151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.268274+0000","last_change":"2024-09-13T08:47:06.247160+0000","last_active":"2024-09-13T08:47:08.268274+0000","last_peered":"2024-09-13T08:47:08.268274+0000","last_clean":"2024-09-13T08:47:08.268274+0000","last_became_active":"2024-09-13T08:47:06.246853+0000","last_became_peered":"2024-09-13T08:47:06.246853+0000","last_unstale":"2024-09-13T08:47:08.268274+0000","last_undegraded":"2024-09-13T08:47:08.268274+0000","last_fullsized":"2024-09-13T08:47:08.268274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:07:45.691363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.267525+0000","last_change":"2024-09-13T08:47:07.270495+0000","last_active":"2024-09-13T08:47:08.267525+0000","last_peered":"2024-09-13T08:47:08.267525+0000","last_clean":"2024-09-13T08:47:08.267525+0000","last_became_active":"2024-09-13T08:47:07.270203+0000","last_became_peered":"2024-09-13T08:47:07.270203+0000","last_unstale":"2024-09-13T08:47:08.267525+0000","last_undegraded":"2024-09-13T08:47:08.267525+0000","last_fullsized":"2024-09-13T08:47:08.267525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:32.865638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778006+0000","last_change":"2024-09-13T08:47:06.259102+0000","last_active":"2024-09-13T08:47:08.778006+0000","last_peered":"2024-09-13T08:47:08.778006+0000","last_clean":"2024-09-13T08:47:08.778006+0000","last_became_active":"2024-09-13T08:47:06.258631+0000","last_became_peered":"2024-09-13T08:47:06.258631+0000","last_unstale":"2024-09-13T08:47:08.778006+0000","last_undegraded":"2024-09-13T08:47:08.778006+0000","last_fullsized":"2024-09-13T08:47:08.778006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:41.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264383+0000","last_change":"2024-09-13T08:47:07.276197+0000","last_active":"2024-09-13T08:47:08.264383+0000","last_peered":"2024-09-13T08:47:08.264383+0000","last_clean":"2024-09-13T08:47:08.264383+0000","last_became_active":"2024-09-13T08:47:07.274853+0000","last_became_peered":"2024-09-13T08:47:07.274853+0000","last_unstale":"2024-09-13T08:47:08.264383+0000","last_undegraded":"2024-09-13T08:47:08.264383+0000","last_fullsized":"2024-09-13T08:47:08.264383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:18.766053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264029+0000","last_change":"2024-09-13T08:47:06.244300+0000","last_active":"2024-09-13T08:47:08.264029+0000","last_peered":"2024-09-13T08:47:08.264029+0000","last_clean":"2024-09-13T08:47:08.264029+0000","last_became_active":"2024-09-13T08:47:06.243020+0000","last_became_peered":"2024-09-13T08:47:06.243020+0000","last_unstale":"2024-09-13T08:47:08.264029+0000","last_undegraded":"2024-09-13T08:47:08.264029+0000","last_fullsized":"2024-09-13T08:47:08.264029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:59.509452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777420+0000","last_change":"2024-09-13T08:47:07.270469+0000","last_active":"2024-09-13T08:47:08.777420+0000","last_peered":"2024-09-13T08:47:08.777420+0000","last_clean":"2024-09-13T08:47:08.777420+0000","last_became_active":"2024-09-13T08:47:07.269822+0000","last_became_peered":"2024-09-13T08:47:07.269822+0000","last_unstale":"2024-09-13T08:47:08.777420+0000","last_undegraded":"2024-09-13T08:47:08.777420+0000","last_fullsized":"2024-09-13T08:47:08.777420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:02.244759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778355+0000","last_change":"2024-09-13T08:47:06.258026+0000","last_active":"2024-09-13T08:47:08.778355+0000","last_peered":"2024-09-13T08:47:08.778355+0000","last_clean":"2024-09-13T08:47:08.778355+0000","last_became_active":"2024-09-13T08:47:06.257756+0000","last_became_peered":"2024-09-13T08:47:06.257756+0000","last_unstale":"2024-09-13T08:47:08.778355+0000","last_undegraded":"2024-09-13T08:47:08.778355+0000","last_fullsized":"2024-09-13T08:47:08.778355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:06.792879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778358+0000","last_change":"2024-09-13T08:47:07.254447+0000","last_active":"2024-09-13T08:47:08.778358+0000","last_peered":"2024-09-13T08:47:08.778358+0000","last_clean":"2024-09-13T08:47:08.778358+0000","last_became_active":"2024-09-13T08:47:07.254175+0000","last_became_peered":"2024-09-13T08:47:07.254175+0000","last_unstale":"2024-09-13T08:47:08.778358+0000","last_undegraded":"2024-09-13T08:47:08.778358+0000","last_fullsized":"2024-09-13T08:47:08.778358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:45.228354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778211+0000","last_change":"2024-09-13T08:47:06.263309+0000","last_active":"2024-09-13T08:47:08.778211+0000","last_peered":"2024-09-13T08:47:08.778211+0000","last_clean":"2024-09-13T08:47:08.778211+0000","last_became_active":"2024-09-13T08:47:06.263046+0000","last_became_peered":"2024-09-13T08:47:06.263046+0000","last_unstale":"2024-09-13T08:47:08.778211+0000","last_undegraded":"2024-09-13T08:47:08.778211+0000","last_fullsized":"2024-09-13T08:47:08.778211+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:46.912237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779046+0000","last_change":"2024-09-13T08:47:07.262230+0000","last_active":"2024-09-13T08:47:08.779046+0000","last_peered":"2024-09-13T08:47:08.779046+0000","last_clean":"2024-09-13T08:47:08.779046+0000","last_became_active":"2024-09-13T08:47:07.261919+0000","last_became_peered":"2024-09-13T08:47:07.261919+0000","last_unstale":"2024-09-13T08:47:08.779046+0000","last_undegraded":"2024-09-13T08:47:08.779046+0000","last_fullsized":"2024-09-13T08:47:08.779046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:33.256378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779537+0000","last_change":"2024-09-13T08:47:06.246754+0000","last_active":"2024-09-13T08:47:08.779537+0000","last_peered":"2024-09-13T08:47:08.779537+0000","last_clean":"2024-09-13T08:47:08.779537+0000","last_became_active":"2024-09-13T08:47:06.246181+0000","last_became_peered":"2024-09-13T08:47:06.246181+0000","last_unstale":"2024-09-13T08:47:08.779537+0000","last_undegraded":"2024-09-13T08:47:08.779537+0000","last_fullsized":"2024-09-13T08:47:08.779537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:54.760904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779519+0000","last_change":"2024-09-13T08:47:07.270274+0000","last_active":"2024-09-13T08:47:08.779519+0000","last_peered":"2024-09-13T08:47:08.779519+0000","last_clean":"2024-09-13T08:47:08.779519+0000","last_became_active":"2024-09-13T08:47:07.269966+0000","last_became_peered":"2024-09-13T08:47:07.269966+0000","last_unstale":"2024-09-13T08:47:08.779519+0000","last_undegraded":"2024-09-13T08:47:08.779519+0000","last_fullsized":"2024-09-13T08:47:08.779519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:38:19.673757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264432+0000","last_change":"2024-09-13T08:47:06.258172+0000","last_active":"2024-09-13T08:47:08.264432+0000","last_peered":"2024-09-13T08:47:08.264432+0000","last_clean":"2024-09-13T08:47:08.264432+0000","last_became_active":"2024-09-13T08:47:06.257892+0000","last_became_peered":"2024-09-13T08:47:06.257892+0000","last_unstale":"2024-09-13T08:47:08.264432+0000","last_undegraded":"2024-09-13T08:47:08.264432+0000","last_fullsized":"2024-09-13T08:47:08.264432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:46:19.201898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.274255+0000","last_change":"2024-09-13T08:47:07.267661+0000","last_active":"2024-09-13T08:47:07.274255+0000","last_peered":"2024-09-13T08:47:07.274255+0000","last_clean":"2024-09-13T08:47:07.274255+0000","last_became_active":"2024-09-13T08:47:07.267351+0000","last_became_peered":"2024-09-13T08:47:07.267351+0000","last_unstale":"2024-09-13T08:47:07.274255+0000","last_undegraded":"2024-09-13T08:47:07.274255+0000","last_fullsized":"2024-09-13T08:47:07.274255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:40:40.560343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"88'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779264+0000","last_change":"2024-09-13T08:47:06.253059+0000","last_active":"2024-09-13T08:47:08.779264+0000","last_peered":"2024-09-13T08:47:08.779264+0000","last_clean":"2024-09-13T08:47:08.779264+0000","last_became_active":"2024-09-13T08:47:06.252779+0000","last_became_peered":"2024-09-13T08:47:06.252779+0000","last_unstale":"2024-09-13T08:47:08.779264+0000","last_undegraded":"2024-09-13T08:47:08.779264+0000","last_fullsized":"2024-09-13T08:47:08.779264+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:04.710637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778507+0000","last_change":"2024-09-13T08:47:07.271662+0000","last_active":"2024-09-13T08:47:08.778507+0000","last_peered":"2024-09-13T08:47:08.778507+0000","last_clean":"2024-09-13T08:47:08.778507+0000","last_became_active":"2024-09-13T08:47:07.271378+0000","last_became_peered":"2024-09-13T08:47:07.271378+0000","last_unstale":"2024-09-13T08:47:08.778507+0000","last_undegraded":"2024-09-13T08:47:08.778507+0000","last_fullsized":"2024-09-13T08:47:08.778507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:11.179044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"88'6","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778795+0000","last_change":"2024-09-13T08:47:06.260192+0000","last_active":"2024-09-13T08:47:08.778795+0000","last_peered":"2024-09-13T08:47:08.778795+0000","last_clean":"2024-09-13T08:47:08.778795+0000","last_became_active":"2024-09-13T08:47:06.256320+0000","last_became_peered":"2024-09-13T08:47:06.256320+0000","last_unstale":"2024-09-13T08:47:08.778795+0000","last_undegraded":"2024-09-13T08:47:08.778795+0000","last_fullsized":"2024-09-13T08:47:08.778795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:25.856751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.281908+0000","last_change":"2024-09-13T08:47:07.277937+0000","last_active":"2024-09-13T08:47:07.281908+0000","last_peered":"2024-09-13T08:47:07.281908+0000","last_clean":"2024-09-13T08:47:07.281908+0000","last_became_active":"2024-09-13T08:47:07.277629+0000","last_became_peered":"2024-09-13T08:47:07.277629+0000","last_unstale":"2024-09-13T08:47:07.281908+0000","last_undegraded":"2024-09-13T08:47:07.281908+0000","last_fullsized":"2024-09-13T08:47:07.281908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:06.047568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.268082+0000","last_change":"2024-09-13T08:47:06.251892+0000","last_active":"2024-09-13T08:47:08.268082+0000","last_peered":"2024-09-13T08:47:08.268082+0000","last_clean":"2024-09-13T08:47:08.268082+0000","last_became_active":"2024-09-13T08:47:06.251610+0000","last_became_peered":"2024-09-13T08:47:06.251610+0000","last_unstale":"2024-09-13T08:47:08.268082+0000","last_undegraded":"2024-09-13T08:47:08.268082+0000","last_fullsized":"2024-09-13T08:47:08.268082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:22.892519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779033+0000","last_change":"2024-09-13T08:47:07.273863+0000","last_active":"2024-09-13T08:47:08.779033+0000","last_peered":"2024-09-13T08:47:08.779033+0000","last_clean":"2024-09-13T08:47:08.779033+0000","last_became_active":"2024-09-13T08:47:07.271632+0000","last_became_peered":"2024-09-13T08:47:07.271632+0000","last_unstale":"2024-09-13T08:47:08.779033+0000","last_undegraded":"2024-09-13T08:47:08.779033+0000","last_fullsized":"2024-09-13T08:47:08.779033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:37.053901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266465+0000","last_change":"2024-09-13T08:47:06.257123+0000","last_active":"2024-09-13T08:47:08.266465+0000","last_peered":"2024-09-13T08:47:08.266465+0000","last_clean":"2024-09-13T08:47:08.266465+0000","last_became_active":"2024-09-13T08:47:06.256816+0000","last_became_peered":"2024-09-13T08:47:06.256816+0000","last_unstale":"2024-09-13T08:47:08.266465+0000","last_undegraded":"2024-09-13T08:47:08.266465+0000","last_fullsized":"2024-09-13T08:47:08.266465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:39.592371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778353+0000","last_change":"2024-09-13T08:47:07.271443+0000","last_active":"2024-09-13T08:47:08.778353+0000","last_peered":"2024-09-13T08:47:08.778353+0000","last_clean":"2024-09-13T08:47:08.778353+0000","last_became_active":"2024-09-13T08:47:07.271105+0000","last_became_peered":"2024-09-13T08:47:07.271105+0000","last_unstale":"2024-09-13T08:47:08.778353+0000","last_undegraded":"2024-09-13T08:47:08.778353+0000","last_fullsized":"2024-09-13T08:47:08.778353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:01:42.052534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.263913+0000","last_change":"2024-09-13T08:47:06.251974+0000","last_active":"2024-09-13T08:47:08.263913+0000","last_peered":"2024-09-13T08:47:08.263913+0000","last_clean":"2024-09-13T08:47:08.263913+0000","last_became_active":"2024-09-13T08:47:06.250512+0000","last_became_peered":"2024-09-13T08:47:06.250512+0000","last_unstale":"2024-09-13T08:47:08.263913+0000","last_undegraded":"2024-09-13T08:47:08.263913+0000","last_fullsized":"2024-09-13T08:47:08.263913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:23.995997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.274418+0000","last_change":"2024-09-13T08:47:07.270465+0000","last_active":"2024-09-13T08:47:07.274418+0000","last_peered":"2024-09-13T08:47:07.274418+0000","last_clean":"2024-09-13T08:47:07.274418+0000","last_became_active":"2024-09-13T08:47:07.269856+0000","last_became_peered":"2024-09-13T08:47:07.269856+0000","last_unstale":"2024-09-13T08:47:07.274418+0000","last_undegraded":"2024-09-13T08:47:07.274418+0000","last_fullsized":"2024-09-13T08:47:07.274418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:29:18.390616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"88'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.291495+0000","last_change":"2024-09-13T08:47:06.247980+0000","last_active":"2024-09-13T08:47:07.291495+0000","last_peered":"2024-09-13T08:47:07.291495+0000","last_clean":"2024-09-13T08:47:07.291495+0000","last_became_active":"2024-09-13T08:47:06.247713+0000","last_became_peered":"2024-09-13T08:47:06.247713+0000","last_unstale":"2024-09-13T08:47:07.291495+0000","last_undegraded":"2024-09-13T08:47:07.291495+0000","last_fullsized":"2024-09-13T08:47:07.291495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:22:36.922668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778733+0000","last_change":"2024-09-13T08:47:07.284620+0000","last_active":"2024-09-13T08:47:08.778733+0000","last_peered":"2024-09-13T08:47:08.778733+0000","last_clean":"2024-09-13T08:47:08.778733+0000","last_became_active":"2024-09-13T08:47:07.281855+0000","last_became_peered":"2024-09-13T08:47:07.281855+0000","last_unstale":"2024-09-13T08:47:08.778733+0000","last_undegraded":"2024-09-13T08:47:08.778733+0000","last_fullsized":"2024-09-13T08:47:08.778733+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:13.488445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778299+0000","last_change":"2024-09-13T08:47:06.259535+0000","last_active":"2024-09-13T08:47:08.778299+0000","last_peered":"2024-09-13T08:47:08.778299+0000","last_clean":"2024-09-13T08:47:08.778299+0000","last_became_active":"2024-09-13T08:47:06.259160+0000","last_became_peered":"2024-09-13T08:47:06.259160+0000","last_unstale":"2024-09-13T08:47:08.778299+0000","last_undegraded":"2024-09-13T08:47:08.778299+0000","last_fullsized":"2024-09-13T08:47:08.778299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:39:18.943376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264681+0000","last_change":"2024-09-13T08:47:07.275396+0000","last_active":"2024-09-13T08:47:08.264681+0000","last_peered":"2024-09-13T08:47:08.264681+0000","last_clean":"2024-09-13T08:47:08.264681+0000","last_became_active":"2024-09-13T08:47:07.275088+0000","last_became_peered":"2024-09-13T08:47:07.275088+0000","last_unstale":"2024-09-13T08:47:08.264681+0000","last_undegraded":"2024-09-13T08:47:08.264681+0000","last_fullsized":"2024-09-13T08:47:08.264681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:32.457578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778990+0000","last_change":"2024-09-13T08:47:06.258507+0000","last_active":"2024-09-13T08:47:08.778990+0000","last_peered":"2024-09-13T08:47:08.778990+0000","last_clean":"2024-09-13T08:47:08.778990+0000","last_became_active":"2024-09-13T08:47:06.258215+0000","last_became_peered":"2024-09-13T08:47:06.258215+0000","last_unstale":"2024-09-13T08:47:08.778990+0000","last_undegraded":"2024-09-13T08:47:08.778990+0000","last_fullsized":"2024-09-13T08:47:08.778990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:10:23.753114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778196+0000","last_change":"2024-09-13T08:47:07.282015+0000","last_active":"2024-09-13T08:47:08.778196+0000","last_peered":"2024-09-13T08:47:08.778196+0000","last_clean":"2024-09-13T08:47:08.778196+0000","last_became_active":"2024-09-13T08:47:07.281698+0000","last_became_peered":"2024-09-13T08:47:07.281698+0000","last_unstale":"2024-09-13T08:47:08.778196+0000","last_undegraded":"2024-09-13T08:47:08.778196+0000","last_fullsized":"2024-09-13T08:47:08.778196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:45:52.260401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778997+0000","last_change":"2024-09-13T08:47:06.252320+0000","last_active":"2024-09-13T08:47:08.778997+0000","last_peered":"2024-09-13T08:47:08.778997+0000","last_clean":"2024-09-13T08:47:08.778997+0000","last_became_active":"2024-09-13T08:47:06.252061+0000","last_became_peered":"2024-09-13T08:47:06.252061+0000","last_unstale":"2024-09-13T08:47:08.778997+0000","last_undegraded":"2024-09-13T08:47:08.778997+0000","last_fullsized":"2024-09-13T08:47:08.778997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:44.666211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266367+0000","last_change":"2024-09-13T08:47:07.270015+0000","last_active":"2024-09-13T08:47:08.266367+0000","last_peered":"2024-09-13T08:47:08.266367+0000","last_clean":"2024-09-13T08:47:08.266367+0000","last_became_active":"2024-09-13T08:47:07.269730+0000","last_became_peered":"2024-09-13T08:47:07.269730+0000","last_unstale":"2024-09-13T08:47:08.266367+0000","last_undegraded":"2024-09-13T08:47:08.266367+0000","last_fullsized":"2024-09-13T08:47:08.266367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:01.139461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778702+0000","last_change":"2024-09-13T08:47:06.252851+0000","last_active":"2024-09-13T08:47:08.778702+0000","last_peered":"2024-09-13T08:47:08.778702+0000","last_clean":"2024-09-13T08:47:08.778702+0000","last_became_active":"2024-09-13T08:47:06.251591+0000","last_became_peered":"2024-09-13T08:47:06.251591+0000","last_unstale":"2024-09-13T08:47:08.778702+0000","last_undegraded":"2024-09-13T08:47:08.778702+0000","last_fullsized":"2024-09-13T08:47:08.778702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:57.728136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778013+0000","last_change":"2024-09-13T08:47:07.270875+0000","last_active":"2024-09-13T08:47:08.778013+0000","last_peered":"2024-09-13T08:47:08.778013+0000","last_clean":"2024-09-13T08:47:08.778013+0000","last_became_active":"2024-09-13T08:47:07.270447+0000","last_became_peered":"2024-09-13T08:47:07.270447+0000","last_unstale":"2024-09-13T08:47:08.778013+0000","last_undegraded":"2024-09-13T08:47:08.778013+0000","last_fullsized":"2024-09-13T08:47:08.778013+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:39.531920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266544+0000","last_change":"2024-09-13T08:47:06.245122+0000","last_active":"2024-09-13T08:47:08.266544+0000","last_peered":"2024-09-13T08:47:08.266544+0000","last_clean":"2024-09-13T08:47:08.266544+0000","last_became_active":"2024-09-13T08:47:06.244855+0000","last_became_peered":"2024-09-13T08:47:06.244855+0000","last_unstale":"2024-09-13T08:47:08.266544+0000","last_undegraded":"2024-09-13T08:47:08.266544+0000","last_fullsized":"2024-09-13T08:47:08.266544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:58:46.424443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266279+0000","last_change":"2024-09-13T08:47:07.273725+0000","last_active":"2024-09-13T08:47:08.266279+0000","last_peered":"2024-09-13T08:47:08.266279+0000","last_clean":"2024-09-13T08:47:08.266279+0000","last_became_active":"2024-09-13T08:47:07.273382+0000","last_became_peered":"2024-09-13T08:47:07.273382+0000","last_unstale":"2024-09-13T08:47:08.266279+0000","last_undegraded":"2024-09-13T08:47:08.266279+0000","last_fullsized":"2024-09-13T08:47:08.266279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:01:00.963843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.260688+0000","last_change":"2024-09-13T08:47:06.241834+0000","last_active":"2024-09-13T08:47:07.260688+0000","last_peered":"2024-09-13T08:47:07.260688+0000","last_clean":"2024-09-13T08:47:07.260688+0000","last_became_active":"2024-09-13T08:47:06.240856+0000","last_became_peered":"2024-09-13T08:47:06.240856+0000","last_unstale":"2024-09-13T08:47:07.260688+0000","last_undegraded":"2024-09-13T08:47:07.260688+0000","last_fullsized":"2024-09-13T08:47:07.260688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:34.145719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777737+0000","last_change":"2024-09-13T08:47:07.271892+0000","last_active":"2024-09-13T08:47:08.777737+0000","last_peered":"2024-09-13T08:47:08.777737+0000","last_clean":"2024-09-13T08:47:08.777737+0000","last_became_active":"2024-09-13T08:47:07.271510+0000","last_became_peered":"2024-09-13T08:47:07.271510+0000","last_unstale":"2024-09-13T08:47:08.777737+0000","last_undegraded":"2024-09-13T08:47:08.777737+0000","last_fullsized":"2024-09-13T08:47:08.777737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:55.858407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778406+0000","last_change":"2024-09-13T08:47:06.253901+0000","last_active":"2024-09-13T08:47:08.778406+0000","last_peered":"2024-09-13T08:47:08.778406+0000","last_clean":"2024-09-13T08:47:08.778406+0000","last_became_active":"2024-09-13T08:47:06.253643+0000","last_became_peered":"2024-09-13T08:47:06.253643+0000","last_unstale":"2024-09-13T08:47:08.778406+0000","last_undegraded":"2024-09-13T08:47:08.778406+0000","last_fullsized":"2024-09-13T08:47:08.778406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:07.655485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778377+0000","last_change":"2024-09-13T08:47:07.263072+0000","last_active":"2024-09-13T08:47:08.778377+0000","last_peered":"2024-09-13T08:47:08.778377+0000","last_clean":"2024-09-13T08:47:08.778377+0000","last_became_active":"2024-09-13T08:47:07.261612+0000","last_became_peered":"2024-09-13T08:47:07.261612+0000","last_unstale":"2024-09-13T08:47:08.778377+0000","last_undegraded":"2024-09-13T08:47:08.778377+0000","last_fullsized":"2024-09-13T08:47:08.778377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:48:31.030455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778591+0000","last_change":"2024-09-13T08:47:06.263053+0000","last_active":"2024-09-13T08:47:08.778591+0000","last_peered":"2024-09-13T08:47:08.778591+0000","last_clean":"2024-09-13T08:47:08.778591+0000","last_became_active":"2024-09-13T08:47:06.262793+0000","last_became_peered":"2024-09-13T08:47:06.262793+0000","last_unstale":"2024-09-13T08:47:08.778591+0000","last_undegraded":"2024-09-13T08:47:08.778591+0000","last_fullsized":"2024-09-13T08:47:08.778591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:17:49.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778191+0000","last_change":"2024-09-13T08:47:07.280641+0000","last_active":"2024-09-13T08:47:08.778191+0000","last_peered":"2024-09-13T08:47:08.778191+0000","last_clean":"2024-09-13T08:47:08.778191+0000","last_became_active":"2024-09-13T08:47:07.280359+0000","last_became_peered":"2024-09-13T08:47:07.280359+0000","last_unstale":"2024-09-13T08:47:08.778191+0000","last_undegraded":"2024-09-13T08:47:08.778191+0000","last_fullsized":"2024-09-13T08:47:08.778191+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:11:03.114015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.263689+0000","last_change":"2024-09-13T08:47:06.243414+0000","last_active":"2024-09-13T08:47:07.263689+0000","last_peered":"2024-09-13T08:47:07.263689+0000","last_clean":"2024-09-13T08:47:07.263689+0000","last_became_active":"2024-09-13T08:47:06.243110+0000","last_became_peered":"2024-09-13T08:47:06.243110+0000","last_unstale":"2024-09-13T08:47:07.263689+0000","last_undegraded":"2024-09-13T08:47:07.263689+0000","last_fullsized":"2024-09-13T08:47:07.263689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:09.129430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778092+0000","last_change":"2024-09-13T08:47:07.269933+0000","last_active":"2024-09-13T08:47:08.778092+0000","last_peered":"2024-09-13T08:47:08.778092+0000","last_clean":"2024-09-13T08:47:08.778092+0000","last_became_active":"2024-09-13T08:47:07.267901+0000","last_became_peered":"2024-09-13T08:47:07.267901+0000","last_unstale":"2024-09-13T08:47:08.778092+0000","last_undegraded":"2024-09-13T08:47:08.778092+0000","last_fullsized":"2024-09-13T08:47:08.778092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:31.084989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778133+0000","last_change":"2024-09-13T08:47:06.254333+0000","last_active":"2024-09-13T08:47:08.778133+0000","last_peered":"2024-09-13T08:47:08.778133+0000","last_clean":"2024-09-13T08:47:08.778133+0000","last_became_active":"2024-09-13T08:47:06.253935+0000","last_became_peered":"2024-09-13T08:47:06.253935+0000","last_unstale":"2024-09-13T08:47:08.778133+0000","last_undegraded":"2024-09-13T08:47:08.778133+0000","last_fullsized":"2024-09-13T08:47:08.778133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:47:52.506936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264688+0000","last_change":"2024-09-13T08:47:07.281702+0000","last_active":"2024-09-13T08:47:08.264688+0000","last_peered":"2024-09-13T08:47:08.264688+0000","last_clean":"2024-09-13T08:47:08.264688+0000","last_became_active":"2024-09-13T08:47:07.281058+0000","last_became_peered":"2024-09-13T08:47:07.281058+0000","last_unstale":"2024-09-13T08:47:08.264688+0000","last_undegraded":"2024-09-13T08:47:08.264688+0000","last_fullsized":"2024-09-13T08:47:08.264688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:36:26.426873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264289+0000","last_change":"2024-09-13T08:47:06.253019+0000","last_active":"2024-09-13T08:47:08.264289+0000","last_peered":"2024-09-13T08:47:08.264289+0000","last_clean":"2024-09-13T08:47:08.264289+0000","last_became_active":"2024-09-13T08:47:06.252752+0000","last_became_peered":"2024-09-13T08:47:06.252752+0000","last_unstale":"2024-09-13T08:47:08.264289+0000","last_undegraded":"2024-09-13T08:47:08.264289+0000","last_fullsized":"2024-09-13T08:47:08.264289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:00:14.462714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778024+0000","last_change":"2024-09-13T08:47:07.269622+0000","last_active":"2024-09-13T08:47:08.778024+0000","last_peered":"2024-09-13T08:47:08.778024+0000","last_clean":"2024-09-13T08:47:08.778024+0000","last_became_active":"2024-09-13T08:47:07.269265+0000","last_became_peered":"2024-09-13T08:47:07.269265+0000","last_unstale":"2024-09-13T08:47:08.778024+0000","last_undegraded":"2024-09-13T08:47:08.778024+0000","last_fullsized":"2024-09-13T08:47:08.778024+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:57.889677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266802+0000","last_change":"2024-09-13T08:47:06.246380+0000","last_active":"2024-09-13T08:47:08.266802+0000","last_peered":"2024-09-13T08:47:08.266802+0000","last_clean":"2024-09-13T08:47:08.266802+0000","last_became_active":"2024-09-13T08:47:06.246119+0000","last_became_peered":"2024-09-13T08:47:06.246119+0000","last_unstale":"2024-09-13T08:47:08.266802+0000","last_undegraded":"2024-09-13T08:47:08.266802+0000","last_fullsized":"2024-09-13T08:47:08.266802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:35.160960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.283829+0000","last_change":"2024-09-13T08:47:07.281687+0000","last_active":"2024-09-13T08:47:07.283829+0000","last_peered":"2024-09-13T08:47:07.283829+0000","last_clean":"2024-09-13T08:47:07.283829+0000","last_became_active":"2024-09-13T08:47:07.281402+0000","last_became_peered":"2024-09-13T08:47:07.281402+0000","last_unstale":"2024-09-13T08:47:07.283829+0000","last_undegraded":"2024-09-13T08:47:07.283829+0000","last_fullsized":"2024-09-13T08:47:07.283829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:49:10.314494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264133+0000","last_change":"2024-09-13T08:47:06.244678+0000","last_active":"2024-09-13T08:47:08.264133+0000","last_peered":"2024-09-13T08:47:08.264133+0000","last_clean":"2024-09-13T08:47:08.264133+0000","last_became_active":"2024-09-13T08:47:06.243503+0000","last_became_peered":"2024-09-13T08:47:06.243503+0000","last_unstale":"2024-09-13T08:47:08.264133+0000","last_undegraded":"2024-09-13T08:47:08.264133+0000","last_fullsized":"2024-09-13T08:47:08.264133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:26:51.126833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778449+0000","last_change":"2024-09-13T08:47:07.270837+0000","last_active":"2024-09-13T08:47:08.778449+0000","last_peered":"2024-09-13T08:47:08.778449+0000","last_clean":"2024-09-13T08:47:08.778449+0000","last_became_active":"2024-09-13T08:47:07.270174+0000","last_became_peered":"2024-09-13T08:47:07.270174+0000","last_unstale":"2024-09-13T08:47:08.778449+0000","last_undegraded":"2024-09-13T08:47:08.778449+0000","last_fullsized":"2024-09-13T08:47:08.778449+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:28:00.359095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777778+0000","last_change":"2024-09-13T08:47:06.246725+0000","last_active":"2024-09-13T08:47:08.777778+0000","last_peered":"2024-09-13T08:47:08.777778+0000","last_clean":"2024-09-13T08:47:08.777778+0000","last_became_active":"2024-09-13T08:47:06.246390+0000","last_became_peered":"2024-09-13T08:47:06.246390+0000","last_unstale":"2024-09-13T08:47:08.777778+0000","last_undegraded":"2024-09-13T08:47:08.777778+0000","last_fullsized":"2024-09-13T08:47:08.777778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:43.573769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777754+0000","last_change":"2024-09-13T08:47:07.265297+0000","last_active":"2024-09-13T08:47:08.777754+0000","last_peered":"2024-09-13T08:47:08.777754+0000","last_clean":"2024-09-13T08:47:08.777754+0000","last_became_active":"2024-09-13T08:47:07.264985+0000","last_became_peered":"2024-09-13T08:47:07.264985+0000","last_unstale":"2024-09-13T08:47:08.777754+0000","last_undegraded":"2024-09-13T08:47:08.777754+0000","last_fullsized":"2024-09-13T08:47:08.777754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:36.676460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777461+0000","last_change":"2024-09-13T08:47:06.255961+0000","last_active":"2024-09-13T08:47:08.777461+0000","last_peered":"2024-09-13T08:47:08.777461+0000","last_clean":"2024-09-13T08:47:08.777461+0000","last_became_active":"2024-09-13T08:47:06.255686+0000","last_became_peered":"2024-09-13T08:47:06.255686+0000","last_unstale":"2024-09-13T08:47:08.777461+0000","last_undegraded":"2024-09-13T08:47:08.777461+0000","last_fullsized":"2024-09-13T08:47:08.777461+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:32.249019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.267162+0000","last_change":"2024-09-13T08:47:07.264195+0000","last_active":"2024-09-13T08:47:07.267162+0000","last_peered":"2024-09-13T08:47:07.267162+0000","last_clean":"2024-09-13T08:47:07.267162+0000","last_became_active":"2024-09-13T08:47:07.263862+0000","last_became_peered":"2024-09-13T08:47:07.263862+0000","last_unstale":"2024-09-13T08:47:07.267162+0000","last_undegraded":"2024-09-13T08:47:07.267162+0000","last_fullsized":"2024-09-13T08:47:07.267162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:31:46.891964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778783+0000","last_change":"2024-09-13T08:47:06.252350+0000","last_active":"2024-09-13T08:47:08.778783+0000","last_peered":"2024-09-13T08:47:08.778783+0000","last_clean":"2024-09-13T08:47:08.778783+0000","last_became_active":"2024-09-13T08:47:06.250313+0000","last_became_peered":"2024-09-13T08:47:06.250313+0000","last_unstale":"2024-09-13T08:47:08.778783+0000","last_undegraded":"2024-09-13T08:47:08.778783+0000","last_fullsized":"2024-09-13T08:47:08.778783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:01.941502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777592+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:47:08.777592+0000","last_peered":"2024-09-13T08:47:08.777592+0000","last_clean":"2024-09-13T08:47:08.777592+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:47:08.777592+0000","last_undegraded":"2024-09-13T08:47:08.777592+0000","last_fullsized":"2024-09-13T08:47:08.777592+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509559,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58616,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313224,"statfs":{"total":96636764160,"available":96576741376,"internally_reserved":0,"allocated":1077248,"data_stored":462434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509559,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59220,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312620,"statfs":{"total":96636764160,"available":96576122880,"internally_reserved":0,"allocated":1695744,"data_stored":1060762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509560,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64124,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307716,"statfs":{"total":96636764160,"available":96571101184,"internally_reserved":0,"allocated":1089536,"data_stored":468782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509560,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58624,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313216,"statfs":{"total":96636764160,"available":96576733184,"internally_reserved":0,"allocated":1085440,"data_stored":463038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509559,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58604,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313236,"statfs":{"total":96636764160,"available":96576753664,"internally_reserved":0,"allocated":1073152,"data_stored":462364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509561,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59208,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312632,"statfs":{"total":96636764160,"available":96576135168,"internally_reserved":0,"allocated":1683456,"data_stored":1059206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58620,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313220,"statfs":{"total":96636764160,"available":96576737280,"internally_reserved":0,"allocated":1089536,"data_stored":463998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542265,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64124,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307716,"statfs":{"total":96636764160,"available":96571101184,"internally_reserved":0,"allocated":1081344,"data_stored":463036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:47:11.418 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:47:11.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.600+0000 7ff707800640 1 -- 172.21.15.44:0/1037925620 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff7000cc2a0 msgr2=0x7ff7000b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.600+0000 7ff707800640 1 --2- 172.21.15.44:0/1037925620 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff7000cc2a0 0x7ff7000b3130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009840 tx=0x7ff6ec01c790 comp rx=0 tx=0).stop 2024-09-13T08:47:11.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.600+0000 7ff707800640 1 -- 172.21.15.44:0/1037925620 shutdown_connections 2024-09-13T08:47:11.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.600+0000 7ff707800640 1 --2- 172.21.15.44:0/1037925620 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff7000cc2a0 0x7ff7000b3130 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.600+0000 7ff707800640 1 --2- 172.21.15.44:0/1037925620 >> v2:172.21.15.129:3300/0 conn(0x7ff70015e260 0x7ff7000b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.600+0000 7ff707800640 1 -- 172.21.15.44:0/1037925620 >> 172.21.15.44:0/1037925620 conn(0x7ff7000b18f0 msgr2=0x7ff7000b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:11.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.600+0000 7ff707800640 1 -- 172.21.15.44:0/1037925620 shutdown_connections 2024-09-13T08:47:11.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.600+0000 7ff707800640 1 -- 172.21.15.44:0/1037925620 wait complete. 2024-09-13T08:47:11.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.600+0000 7ff707800640 1 Processor -- start 2024-09-13T08:47:11.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff707800640 1 -- start start 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff707800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7000b4cf0 0x7ff70009db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff707800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff70015e260 0x7ff700164ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff707800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff700156090 0x7ff700156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff707800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff7001683f0 con 0x7ff700156090 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff707800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff700168270 con 0x7ff7000b4cf0 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff707800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff700168570 con 0x7ff70015e260 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff705800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff700156090 0x7ff700156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff705800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff700156090 0x7ff700156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33474/0 (socket says 172.21.15.44:33474) 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff705800640 1 -- 172.21.15.44:0/2010270435 learned_addr learned my addr 172.21.15.44:0/2010270435 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff704e00640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7000b4cf0 0x7ff70009db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:11.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff6ffe00640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff70015e260 0x7ff700164ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:11.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff705800640 1 -- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff70015e260 msgr2=0x7ff700164ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff705800640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff70015e260 0x7ff700164ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff705800640 1 -- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7000b4cf0 msgr2=0x7ff70009db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff705800640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7000b4cf0 0x7ff70009db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff705800640 1 -- 172.21.15.44:0/2010270435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70009e050 con 0x7ff700156090 2024-09-13T08:47:11.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff705800640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff700156090 0x7ff700156520 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c7c0 tx=0x7ff6f800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:11.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff6fcc00640 1 -- 172.21.15.44:0/2010270435 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8003b80 con 0x7ff700156090 2024-09-13T08:47:11.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff6fcc00640 1 -- 172.21.15.44:0/2010270435 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6f8003d20 con 0x7ff700156090 2024-09-13T08:47:11.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff707800640 1 -- 172.21.15.44:0/2010270435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70009b570 con 0x7ff700156090 2024-09-13T08:47:11.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff6fcc00640 1 -- 172.21.15.44:0/2010270435 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800aab0 con 0x7ff700156090 2024-09-13T08:47:11.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff707800640 1 -- 172.21.15.44:0/2010270435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70009ab40 con 0x7ff700156090 2024-09-13T08:47:11.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.604+0000 7ff707800640 1 -- 172.21.15.44:0/2010270435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff700069410 con 0x7ff700156090 2024-09-13T08:47:11.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.608+0000 7ff6fcc00640 1 -- 172.21.15.44:0/2010270435 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7ff6f800ac50 con 0x7ff700156090 2024-09-13T08:47:11.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.608+0000 7ff6fcc00640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff6e0074030 0x7ff6e00764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:11.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.608+0000 7ff6fcc00640 1 -- 172.21.15.44:0/2010270435 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7ff6f8092a60 con 0x7ff700156090 2024-09-13T08:47:11.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.608+0000 7ff704e00640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff6e0074030 0x7ff6e00764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:11.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.608+0000 7ff704e00640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff6e0074030 0x7ff6e00764f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40096c0 tx=0x7ff6f4009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:11.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.612+0000 7ff6fcc00640 1 -- 172.21.15.44:0/2010270435 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff6f805ed60 con 0x7ff700156090 2024-09-13T08:47:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.788+0000 7ff707800640 1 -- 172.21.15.44:0/2010270435 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff700156a60 con 0x7ff6e0074030 2024-09-13T08:47:11.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.792+0000 7ff6fcc00640 1 -- 172.21.15.44:0/2010270435 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252071 (secure 0 0 0) 0x7ff700156a60 con 0x7ff6e0074030 2024-09-13T08:47:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:11.799 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:47:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 -- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff6e0074030 msgr2=0x7ff6e00764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff6e0074030 0x7ff6e00764f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40096c0 tx=0x7ff6f4009260 comp rx=0 tx=0).stop 2024-09-13T08:47:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 -- 172.21.15.44:0/2010270435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff700156090 msgr2=0x7ff700156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff700156090 0x7ff700156520 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c7c0 tx=0x7ff6f800cc80 comp rx=0 tx=0).stop 2024-09-13T08:47:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 -- 172.21.15.44:0/2010270435 shutdown_connections 2024-09-13T08:47:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff6e0074030 0x7ff6e00764f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff700156090 0x7ff700156520 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff70015e260 0x7ff700164ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 --2- 172.21.15.44:0/2010270435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7000b4cf0 0x7ff70009db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 -- 172.21.15.44:0/2010270435 >> 172.21.15.44:0/2010270435 conn(0x7ff7000b18f0 msgr2=0x7ff700159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 -- 172.21.15.44:0/2010270435 shutdown_connections 2024-09-13T08:47:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:11.796+0000 7ff707800640 1 -- 172.21.15.44:0/2010270435 wait complete. 2024-09-13T08:47:11.817 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-13T08:47:10.687869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481140,"kb_used_data":9644,"kb_used_omap":12,"kb_used_meta":471283,"kb_avail":754493580,"statfs":{"total":773094113280,"available":772601425920,"internally_reserved":0,"allocated":9875456,"data_stored":4903620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":482594353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.459018"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779207+0000","last_change":"2024-09-13T08:47:07.283483+0000","last_active":"2024-09-13T08:47:08.779207+0000","last_peered":"2024-09-13T08:47:08.779207+0000","last_clean":"2024-09-13T08:47:08.779207+0000","last_became_active":"2024-09-13T08:47:07.279152+0000","last_became_peered":"2024-09-13T08:47:07.279152+0000","last_unstale":"2024-09-13T08:47:08.779207+0000","last_undegraded":"2024-09-13T08:47:08.779207+0000","last_fullsized":"2024-09-13T08:47:08.779207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:21.692069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266710+0000","last_change":"2024-09-13T08:47:07.286912+0000","last_active":"2024-09-13T08:47:08.266710+0000","last_peered":"2024-09-13T08:47:08.266710+0000","last_clean":"2024-09-13T08:47:08.266710+0000","last_became_active":"2024-09-13T08:47:07.286580+0000","last_became_peered":"2024-09-13T08:47:07.286580+0000","last_unstale":"2024-09-13T08:47:08.266710+0000","last_undegraded":"2024-09-13T08:47:08.266710+0000","last_fullsized":"2024-09-13T08:47:08.266710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:20.294539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264608+0000","last_change":"2024-09-13T08:47:07.277078+0000","last_active":"2024-09-13T08:47:08.264608+0000","last_peered":"2024-09-13T08:47:08.264608+0000","last_clean":"2024-09-13T08:47:08.264608+0000","last_became_active":"2024-09-13T08:47:07.276464+0000","last_became_peered":"2024-09-13T08:47:07.276464+0000","last_unstale":"2024-09-13T08:47:08.264608+0000","last_undegraded":"2024-09-13T08:47:08.264608+0000","last_fullsized":"2024-09-13T08:47:08.264608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:33.682366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.268253+0000","last_change":"2024-09-13T08:47:07.283154+0000","last_active":"2024-09-13T08:47:08.268253+0000","last_peered":"2024-09-13T08:47:08.268253+0000","last_clean":"2024-09-13T08:47:08.268253+0000","last_became_active":"2024-09-13T08:47:07.282934+0000","last_became_peered":"2024-09-13T08:47:07.282934+0000","last_unstale":"2024-09-13T08:47:08.268253+0000","last_undegraded":"2024-09-13T08:47:08.268253+0000","last_fullsized":"2024-09-13T08:47:08.268253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:48.582724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777803+0000","last_change":"2024-09-13T08:47:07.256022+0000","last_active":"2024-09-13T08:47:08.777803+0000","last_peered":"2024-09-13T08:47:08.777803+0000","last_clean":"2024-09-13T08:47:08.777803+0000","last_became_active":"2024-09-13T08:47:07.255702+0000","last_became_peered":"2024-09-13T08:47:07.255702+0000","last_unstale":"2024-09-13T08:47:08.777803+0000","last_undegraded":"2024-09-13T08:47:08.777803+0000","last_fullsized":"2024-09-13T08:47:08.777803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:08.772346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264100+0000","last_change":"2024-09-13T08:47:07.267150+0000","last_active":"2024-09-13T08:47:08.264100+0000","last_peered":"2024-09-13T08:47:08.264100+0000","last_clean":"2024-09-13T08:47:08.264100+0000","last_became_active":"2024-09-13T08:47:07.266861+0000","last_became_peered":"2024-09-13T08:47:07.266861+0000","last_unstale":"2024-09-13T08:47:08.264100+0000","last_undegraded":"2024-09-13T08:47:08.264100+0000","last_fullsized":"2024-09-13T08:47:08.264100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:36:06.063894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266737+0000","last_change":"2024-09-13T08:47:07.265958+0000","last_active":"2024-09-13T08:47:08.266737+0000","last_peered":"2024-09-13T08:47:08.266737+0000","last_clean":"2024-09-13T08:47:08.266737+0000","last_became_active":"2024-09-13T08:47:07.265670+0000","last_became_peered":"2024-09-13T08:47:07.265670+0000","last_unstale":"2024-09-13T08:47:08.266737+0000","last_undegraded":"2024-09-13T08:47:08.266737+0000","last_fullsized":"2024-09-13T08:47:08.266737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:48.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777359+0000","last_change":"2024-09-13T08:47:07.272690+0000","last_active":"2024-09-13T08:47:08.777359+0000","last_peered":"2024-09-13T08:47:08.777359+0000","last_clean":"2024-09-13T08:47:08.777359+0000","last_became_active":"2024-09-13T08:47:07.272340+0000","last_became_peered":"2024-09-13T08:47:07.272340+0000","last_unstale":"2024-09-13T08:47:08.777359+0000","last_undegraded":"2024-09-13T08:47:08.777359+0000","last_fullsized":"2024-09-13T08:47:08.777359+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:46:20.209257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779400+0000","last_change":"2024-09-13T08:47:07.272294+0000","last_active":"2024-09-13T08:47:08.779400+0000","last_peered":"2024-09-13T08:47:08.779400+0000","last_clean":"2024-09-13T08:47:08.779400+0000","last_became_active":"2024-09-13T08:47:07.272007+0000","last_became_peered":"2024-09-13T08:47:07.272007+0000","last_unstale":"2024-09-13T08:47:08.779400+0000","last_undegraded":"2024-09-13T08:47:08.779400+0000","last_fullsized":"2024-09-13T08:47:08.779400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:27.785222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264312+0000","last_change":"2024-09-13T08:47:07.281399+0000","last_active":"2024-09-13T08:47:08.264312+0000","last_peered":"2024-09-13T08:47:08.264312+0000","last_clean":"2024-09-13T08:47:08.264312+0000","last_became_active":"2024-09-13T08:47:07.277660+0000","last_became_peered":"2024-09-13T08:47:07.277660+0000","last_unstale":"2024-09-13T08:47:08.264312+0000","last_undegraded":"2024-09-13T08:47:08.264312+0000","last_fullsized":"2024-09-13T08:47:08.264312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:13.488445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.282812+0000","last_change":"2024-09-13T08:47:07.280911+0000","last_active":"2024-09-13T08:47:07.282812+0000","last_peered":"2024-09-13T08:47:07.282812+0000","last_clean":"2024-09-13T08:47:07.282812+0000","last_became_active":"2024-09-13T08:47:07.280626+0000","last_became_peered":"2024-09-13T08:47:07.280626+0000","last_unstale":"2024-09-13T08:47:07.282812+0000","last_undegraded":"2024-09-13T08:47:07.282812+0000","last_fullsized":"2024-09-13T08:47:07.282812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:16:18.265753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778460+0000","last_change":"2024-09-13T08:47:07.273190+0000","last_active":"2024-09-13T08:47:08.778460+0000","last_peered":"2024-09-13T08:47:08.778460+0000","last_clean":"2024-09-13T08:47:08.778460+0000","last_became_active":"2024-09-13T08:47:07.272825+0000","last_became_peered":"2024-09-13T08:47:07.272825+0000","last_unstale":"2024-09-13T08:47:08.778460+0000","last_undegraded":"2024-09-13T08:47:08.778460+0000","last_fullsized":"2024-09-13T08:47:08.778460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:17:54.456895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777832+0000","last_change":"2024-09-13T08:47:07.277642+0000","last_active":"2024-09-13T08:47:08.777832+0000","last_peered":"2024-09-13T08:47:08.777832+0000","last_clean":"2024-09-13T08:47:08.777832+0000","last_became_active":"2024-09-13T08:47:07.277283+0000","last_became_peered":"2024-09-13T08:47:07.277283+0000","last_unstale":"2024-09-13T08:47:08.777832+0000","last_undegraded":"2024-09-13T08:47:08.777832+0000","last_fullsized":"2024-09-13T08:47:08.777832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:37.494571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778630+0000","last_change":"2024-09-13T08:47:07.259608+0000","last_active":"2024-09-13T08:47:08.778630+0000","last_peered":"2024-09-13T08:47:08.778630+0000","last_clean":"2024-09-13T08:47:08.778630+0000","last_became_active":"2024-09-13T08:47:07.257983+0000","last_became_peered":"2024-09-13T08:47:07.257983+0000","last_unstale":"2024-09-13T08:47:08.778630+0000","last_undegraded":"2024-09-13T08:47:08.778630+0000","last_fullsized":"2024-09-13T08:47:08.778630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:36.886734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266996+0000","last_change":"2024-09-13T08:47:07.265243+0000","last_active":"2024-09-13T08:47:08.266996+0000","last_peered":"2024-09-13T08:47:08.266996+0000","last_clean":"2024-09-13T08:47:08.266996+0000","last_became_active":"2024-09-13T08:47:07.264708+0000","last_became_peered":"2024-09-13T08:47:07.264708+0000","last_unstale":"2024-09-13T08:47:08.266996+0000","last_undegraded":"2024-09-13T08:47:08.266996+0000","last_fullsized":"2024-09-13T08:47:08.266996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:33.256378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264006+0000","last_change":"2024-09-13T08:47:07.268059+0000","last_active":"2024-09-13T08:47:08.264006+0000","last_peered":"2024-09-13T08:47:08.264006+0000","last_clean":"2024-09-13T08:47:08.264006+0000","last_became_active":"2024-09-13T08:47:07.267171+0000","last_became_peered":"2024-09-13T08:47:07.267171+0000","last_unstale":"2024-09-13T08:47:08.264006+0000","last_undegraded":"2024-09-13T08:47:08.264006+0000","last_fullsized":"2024-09-13T08:47:08.264006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:17:54.456895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779325+0000","last_change":"2024-09-13T08:47:07.272849+0000","last_active":"2024-09-13T08:47:08.779325+0000","last_peered":"2024-09-13T08:47:08.779325+0000","last_clean":"2024-09-13T08:47:08.779325+0000","last_became_active":"2024-09-13T08:47:07.272555+0000","last_became_peered":"2024-09-13T08:47:07.272555+0000","last_unstale":"2024-09-13T08:47:08.779325+0000","last_undegraded":"2024-09-13T08:47:08.779325+0000","last_fullsized":"2024-09-13T08:47:08.779325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:09.870318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.279947+0000","last_change":"2024-09-13T08:47:07.277413+0000","last_active":"2024-09-13T08:47:07.279947+0000","last_peered":"2024-09-13T08:47:07.279947+0000","last_clean":"2024-09-13T08:47:07.279947+0000","last_became_active":"2024-09-13T08:47:07.277111+0000","last_became_peered":"2024-09-13T08:47:07.277111+0000","last_unstale":"2024-09-13T08:47:07.279947+0000","last_undegraded":"2024-09-13T08:47:07.279947+0000","last_fullsized":"2024-09-13T08:47:07.279947+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:49:23.895427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778734+0000","last_change":"2024-09-13T08:47:07.261348+0000","last_active":"2024-09-13T08:47:08.778734+0000","last_peered":"2024-09-13T08:47:08.778734+0000","last_clean":"2024-09-13T08:47:08.778734+0000","last_became_active":"2024-09-13T08:47:07.261043+0000","last_became_peered":"2024-09-13T08:47:07.261043+0000","last_unstale":"2024-09-13T08:47:08.778734+0000","last_undegraded":"2024-09-13T08:47:08.778734+0000","last_fullsized":"2024-09-13T08:47:08.778734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:01.139461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778785+0000","last_change":"2024-09-13T08:47:07.260676+0000","last_active":"2024-09-13T08:47:08.778785+0000","last_peered":"2024-09-13T08:47:08.778785+0000","last_clean":"2024-09-13T08:47:08.778785+0000","last_became_active":"2024-09-13T08:47:07.260400+0000","last_became_peered":"2024-09-13T08:47:07.260400+0000","last_unstale":"2024-09-13T08:47:08.778785+0000","last_undegraded":"2024-09-13T08:47:08.778785+0000","last_fullsized":"2024-09-13T08:47:08.778785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:20.294539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.267317+0000","last_change":"2024-09-13T08:47:07.269103+0000","last_active":"2024-09-13T08:47:08.267317+0000","last_peered":"2024-09-13T08:47:08.267317+0000","last_clean":"2024-09-13T08:47:08.267317+0000","last_became_active":"2024-09-13T08:47:07.268832+0000","last_became_peered":"2024-09-13T08:47:07.268832+0000","last_unstale":"2024-09-13T08:47:08.267317+0000","last_undegraded":"2024-09-13T08:47:08.267317+0000","last_fullsized":"2024-09-13T08:47:08.267317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:09.622670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778570+0000","last_change":"2024-09-13T08:47:07.269690+0000","last_active":"2024-09-13T08:47:08.778570+0000","last_peered":"2024-09-13T08:47:08.778570+0000","last_clean":"2024-09-13T08:47:08.778570+0000","last_became_active":"2024-09-13T08:47:07.267738+0000","last_became_peered":"2024-09-13T08:47:07.267738+0000","last_unstale":"2024-09-13T08:47:08.778570+0000","last_undegraded":"2024-09-13T08:47:08.778570+0000","last_fullsized":"2024-09-13T08:47:08.778570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:46.802435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777440+0000","last_change":"2024-09-13T08:47:07.274583+0000","last_active":"2024-09-13T08:47:08.777440+0000","last_peered":"2024-09-13T08:47:08.777440+0000","last_clean":"2024-09-13T08:47:08.777440+0000","last_became_active":"2024-09-13T08:47:07.274283+0000","last_became_peered":"2024-09-13T08:47:07.274283+0000","last_unstale":"2024-09-13T08:47:08.777440+0000","last_undegraded":"2024-09-13T08:47:08.777440+0000","last_fullsized":"2024-09-13T08:47:08.777440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:42:49.364510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.276983+0000","last_change":"2024-09-13T08:47:07.271946+0000","last_active":"2024-09-13T08:47:07.276983+0000","last_peered":"2024-09-13T08:47:07.276983+0000","last_clean":"2024-09-13T08:47:07.276983+0000","last_became_active":"2024-09-13T08:47:07.271549+0000","last_became_peered":"2024-09-13T08:47:07.271549+0000","last_unstale":"2024-09-13T08:47:07.276983+0000","last_undegraded":"2024-09-13T08:47:07.276983+0000","last_fullsized":"2024-09-13T08:47:07.276983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:38.168348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.274385+0000","last_change":"2024-09-13T08:47:07.268903+0000","last_active":"2024-09-13T08:47:07.274385+0000","last_peered":"2024-09-13T08:47:07.274385+0000","last_clean":"2024-09-13T08:47:07.274385+0000","last_became_active":"2024-09-13T08:47:07.268596+0000","last_became_peered":"2024-09-13T08:47:07.268596+0000","last_unstale":"2024-09-13T08:47:07.274385+0000","last_undegraded":"2024-09-13T08:47:07.274385+0000","last_fullsized":"2024-09-13T08:47:07.274385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:08.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778187+0000","last_change":"2024-09-13T08:47:07.268586+0000","last_active":"2024-09-13T08:47:08.778187+0000","last_peered":"2024-09-13T08:47:08.778187+0000","last_clean":"2024-09-13T08:47:08.778187+0000","last_became_active":"2024-09-13T08:47:07.268318+0000","last_became_peered":"2024-09-13T08:47:07.268318+0000","last_unstale":"2024-09-13T08:47:08.778187+0000","last_undegraded":"2024-09-13T08:47:08.778187+0000","last_fullsized":"2024-09-13T08:47:08.778187+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:48.992201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778737+0000","last_change":"2024-09-13T08:47:07.258085+0000","last_active":"2024-09-13T08:47:08.778737+0000","last_peered":"2024-09-13T08:47:08.778737+0000","last_clean":"2024-09-13T08:47:08.778737+0000","last_became_active":"2024-09-13T08:47:07.257774+0000","last_became_peered":"2024-09-13T08:47:07.257774+0000","last_unstale":"2024-09-13T08:47:08.778737+0000","last_undegraded":"2024-09-13T08:47:08.778737+0000","last_fullsized":"2024-09-13T08:47:08.778737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:54.065712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779075+0000","last_change":"2024-09-13T08:47:07.267529+0000","last_active":"2024-09-13T08:47:08.779075+0000","last_peered":"2024-09-13T08:47:08.779075+0000","last_clean":"2024-09-13T08:47:08.779075+0000","last_became_active":"2024-09-13T08:47:07.267265+0000","last_became_peered":"2024-09-13T08:47:07.267265+0000","last_unstale":"2024-09-13T08:47:08.779075+0000","last_undegraded":"2024-09-13T08:47:08.779075+0000","last_fullsized":"2024-09-13T08:47:08.779075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:45.228354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.268102+0000","last_change":"2024-09-13T08:47:07.268860+0000","last_active":"2024-09-13T08:47:08.268102+0000","last_peered":"2024-09-13T08:47:08.268102+0000","last_clean":"2024-09-13T08:47:08.268102+0000","last_became_active":"2024-09-13T08:47:07.268584+0000","last_became_peered":"2024-09-13T08:47:07.268584+0000","last_unstale":"2024-09-13T08:47:08.268102+0000","last_undegraded":"2024-09-13T08:47:08.268102+0000","last_fullsized":"2024-09-13T08:47:08.268102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:07.201021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778870+0000","last_change":"2024-09-13T08:47:07.266408+0000","last_active":"2024-09-13T08:47:08.778870+0000","last_peered":"2024-09-13T08:47:08.778870+0000","last_clean":"2024-09-13T08:47:08.778870+0000","last_became_active":"2024-09-13T08:47:07.266119+0000","last_became_peered":"2024-09-13T08:47:07.266119+0000","last_unstale":"2024-09-13T08:47:08.778870+0000","last_undegraded":"2024-09-13T08:47:08.778870+0000","last_fullsized":"2024-09-13T08:47:08.778870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:48.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264333+0000","last_change":"2024-09-13T08:47:07.262631+0000","last_active":"2024-09-13T08:47:08.264333+0000","last_peered":"2024-09-13T08:47:08.264333+0000","last_clean":"2024-09-13T08:47:08.264333+0000","last_became_active":"2024-09-13T08:47:07.262271+0000","last_became_peered":"2024-09-13T08:47:07.262271+0000","last_unstale":"2024-09-13T08:47:08.264333+0000","last_undegraded":"2024-09-13T08:47:08.264333+0000","last_fullsized":"2024-09-13T08:47:08.264333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:09.870318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264058+0000","last_change":"2024-09-13T08:47:07.267526+0000","last_active":"2024-09-13T08:47:08.264058+0000","last_peered":"2024-09-13T08:47:08.264058+0000","last_clean":"2024-09-13T08:47:08.264058+0000","last_became_active":"2024-09-13T08:47:07.266102+0000","last_became_peered":"2024-09-13T08:47:07.266102+0000","last_unstale":"2024-09-13T08:47:08.264058+0000","last_undegraded":"2024-09-13T08:47:08.264058+0000","last_fullsized":"2024-09-13T08:47:08.264058+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:46.390729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778505+0000","last_change":"2024-09-13T08:47:07.286631+0000","last_active":"2024-09-13T08:47:08.778505+0000","last_peered":"2024-09-13T08:47:08.778505+0000","last_clean":"2024-09-13T08:47:08.778505+0000","last_became_active":"2024-09-13T08:47:07.285918+0000","last_became_peered":"2024-09-13T08:47:07.285918+0000","last_unstale":"2024-09-13T08:47:08.778505+0000","last_undegraded":"2024-09-13T08:47:08.778505+0000","last_fullsized":"2024-09-13T08:47:08.778505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:46.390729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"88'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778147+0000","last_change":"2024-09-13T08:47:06.252215+0000","last_active":"2024-09-13T08:47:08.778147+0000","last_peered":"2024-09-13T08:47:08.778147+0000","last_clean":"2024-09-13T08:47:08.778147+0000","last_became_active":"2024-09-13T08:47:06.251937+0000","last_became_peered":"2024-09-13T08:47:06.251937+0000","last_unstale":"2024-09-13T08:47:08.778147+0000","last_undegraded":"2024-09-13T08:47:08.778147+0000","last_fullsized":"2024-09-13T08:47:08.778147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:37.247845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779430+0000","last_change":"2024-09-13T08:47:07.279386+0000","last_active":"2024-09-13T08:47:08.779430+0000","last_peered":"2024-09-13T08:47:08.779430+0000","last_clean":"2024-09-13T08:47:08.779430+0000","last_became_active":"2024-09-13T08:47:07.278815+0000","last_became_peered":"2024-09-13T08:47:07.278815+0000","last_unstale":"2024-09-13T08:47:08.779430+0000","last_undegraded":"2024-09-13T08:47:08.779430+0000","last_fullsized":"2024-09-13T08:47:08.779430+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:40.607679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264383+0000","last_change":"2024-09-13T08:47:06.254635+0000","last_active":"2024-09-13T08:47:08.264383+0000","last_peered":"2024-09-13T08:47:08.264383+0000","last_clean":"2024-09-13T08:47:08.264383+0000","last_became_active":"2024-09-13T08:47:06.252778+0000","last_became_peered":"2024-09-13T08:47:06.252778+0000","last_unstale":"2024-09-13T08:47:08.264383+0000","last_undegraded":"2024-09-13T08:47:08.264383+0000","last_fullsized":"2024-09-13T08:47:08.264383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:22:03.994031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778681+0000","last_change":"2024-09-13T08:47:07.275040+0000","last_active":"2024-09-13T08:47:08.778681+0000","last_peered":"2024-09-13T08:47:08.778681+0000","last_clean":"2024-09-13T08:47:08.778681+0000","last_became_active":"2024-09-13T08:47:07.272521+0000","last_became_peered":"2024-09-13T08:47:07.272521+0000","last_unstale":"2024-09-13T08:47:08.778681+0000","last_undegraded":"2024-09-13T08:47:08.778681+0000","last_fullsized":"2024-09-13T08:47:08.778681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:36:06.063894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.283428+0000","last_change":"2024-09-13T08:47:06.255406+0000","last_active":"2024-09-13T08:47:07.283428+0000","last_peered":"2024-09-13T08:47:07.283428+0000","last_clean":"2024-09-13T08:47:07.283428+0000","last_became_active":"2024-09-13T08:47:06.255173+0000","last_became_peered":"2024-09-13T08:47:06.255173+0000","last_unstale":"2024-09-13T08:47:07.283428+0000","last_undegraded":"2024-09-13T08:47:07.283428+0000","last_fullsized":"2024-09-13T08:47:07.283428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:44.182064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779148+0000","last_change":"2024-09-13T08:47:07.270828+0000","last_active":"2024-09-13T08:47:08.779148+0000","last_peered":"2024-09-13T08:47:08.779148+0000","last_clean":"2024-09-13T08:47:08.779148+0000","last_became_active":"2024-09-13T08:47:07.270452+0000","last_became_peered":"2024-09-13T08:47:07.270452+0000","last_unstale":"2024-09-13T08:47:08.779148+0000","last_undegraded":"2024-09-13T08:47:08.779148+0000","last_fullsized":"2024-09-13T08:47:08.779148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:52.872151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.268274+0000","last_change":"2024-09-13T08:47:06.247160+0000","last_active":"2024-09-13T08:47:08.268274+0000","last_peered":"2024-09-13T08:47:08.268274+0000","last_clean":"2024-09-13T08:47:08.268274+0000","last_became_active":"2024-09-13T08:47:06.246853+0000","last_became_peered":"2024-09-13T08:47:06.246853+0000","last_unstale":"2024-09-13T08:47:08.268274+0000","last_undegraded":"2024-09-13T08:47:08.268274+0000","last_fullsized":"2024-09-13T08:47:08.268274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:07:45.691363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.267525+0000","last_change":"2024-09-13T08:47:07.270495+0000","last_active":"2024-09-13T08:47:08.267525+0000","last_peered":"2024-09-13T08:47:08.267525+0000","last_clean":"2024-09-13T08:47:08.267525+0000","last_became_active":"2024-09-13T08:47:07.270203+0000","last_became_peered":"2024-09-13T08:47:07.270203+0000","last_unstale":"2024-09-13T08:47:08.267525+0000","last_undegraded":"2024-09-13T08:47:08.267525+0000","last_fullsized":"2024-09-13T08:47:08.267525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:32.865638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778006+0000","last_change":"2024-09-13T08:47:06.259102+0000","last_active":"2024-09-13T08:47:08.778006+0000","last_peered":"2024-09-13T08:47:08.778006+0000","last_clean":"2024-09-13T08:47:08.778006+0000","last_became_active":"2024-09-13T08:47:06.258631+0000","last_became_peered":"2024-09-13T08:47:06.258631+0000","last_unstale":"2024-09-13T08:47:08.778006+0000","last_undegraded":"2024-09-13T08:47:08.778006+0000","last_fullsized":"2024-09-13T08:47:08.778006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:41.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264383+0000","last_change":"2024-09-13T08:47:07.276197+0000","last_active":"2024-09-13T08:47:08.264383+0000","last_peered":"2024-09-13T08:47:08.264383+0000","last_clean":"2024-09-13T08:47:08.264383+0000","last_became_active":"2024-09-13T08:47:07.274853+0000","last_became_peered":"2024-09-13T08:47:07.274853+0000","last_unstale":"2024-09-13T08:47:08.264383+0000","last_undegraded":"2024-09-13T08:47:08.264383+0000","last_fullsized":"2024-09-13T08:47:08.264383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:18.766053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264029+0000","last_change":"2024-09-13T08:47:06.244300+0000","last_active":"2024-09-13T08:47:08.264029+0000","last_peered":"2024-09-13T08:47:08.264029+0000","last_clean":"2024-09-13T08:47:08.264029+0000","last_became_active":"2024-09-13T08:47:06.243020+0000","last_became_peered":"2024-09-13T08:47:06.243020+0000","last_unstale":"2024-09-13T08:47:08.264029+0000","last_undegraded":"2024-09-13T08:47:08.264029+0000","last_fullsized":"2024-09-13T08:47:08.264029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:59.509452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777420+0000","last_change":"2024-09-13T08:47:07.270469+0000","last_active":"2024-09-13T08:47:08.777420+0000","last_peered":"2024-09-13T08:47:08.777420+0000","last_clean":"2024-09-13T08:47:08.777420+0000","last_became_active":"2024-09-13T08:47:07.269822+0000","last_became_peered":"2024-09-13T08:47:07.269822+0000","last_unstale":"2024-09-13T08:47:08.777420+0000","last_undegraded":"2024-09-13T08:47:08.777420+0000","last_fullsized":"2024-09-13T08:47:08.777420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:02.244759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778355+0000","last_change":"2024-09-13T08:47:06.258026+0000","last_active":"2024-09-13T08:47:08.778355+0000","last_peered":"2024-09-13T08:47:08.778355+0000","last_clean":"2024-09-13T08:47:08.778355+0000","last_became_active":"2024-09-13T08:47:06.257756+0000","last_became_peered":"2024-09-13T08:47:06.257756+0000","last_unstale":"2024-09-13T08:47:08.778355+0000","last_undegraded":"2024-09-13T08:47:08.778355+0000","last_fullsized":"2024-09-13T08:47:08.778355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:06.792879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778358+0000","last_change":"2024-09-13T08:47:07.254447+0000","last_active":"2024-09-13T08:47:08.778358+0000","last_peered":"2024-09-13T08:47:08.778358+0000","last_clean":"2024-09-13T08:47:08.778358+0000","last_became_active":"2024-09-13T08:47:07.254175+0000","last_became_peered":"2024-09-13T08:47:07.254175+0000","last_unstale":"2024-09-13T08:47:08.778358+0000","last_undegraded":"2024-09-13T08:47:08.778358+0000","last_fullsized":"2024-09-13T08:47:08.778358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:01:45.228354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778211+0000","last_change":"2024-09-13T08:47:06.263309+0000","last_active":"2024-09-13T08:47:08.778211+0000","last_peered":"2024-09-13T08:47:08.778211+0000","last_clean":"2024-09-13T08:47:08.778211+0000","last_became_active":"2024-09-13T08:47:06.263046+0000","last_became_peered":"2024-09-13T08:47:06.263046+0000","last_unstale":"2024-09-13T08:47:08.778211+0000","last_undegraded":"2024-09-13T08:47:08.778211+0000","last_fullsized":"2024-09-13T08:47:08.778211+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:46.912237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779046+0000","last_change":"2024-09-13T08:47:07.262230+0000","last_active":"2024-09-13T08:47:08.779046+0000","last_peered":"2024-09-13T08:47:08.779046+0000","last_clean":"2024-09-13T08:47:08.779046+0000","last_became_active":"2024-09-13T08:47:07.261919+0000","last_became_peered":"2024-09-13T08:47:07.261919+0000","last_unstale":"2024-09-13T08:47:08.779046+0000","last_undegraded":"2024-09-13T08:47:08.779046+0000","last_fullsized":"2024-09-13T08:47:08.779046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:33.256378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779537+0000","last_change":"2024-09-13T08:47:06.246754+0000","last_active":"2024-09-13T08:47:08.779537+0000","last_peered":"2024-09-13T08:47:08.779537+0000","last_clean":"2024-09-13T08:47:08.779537+0000","last_became_active":"2024-09-13T08:47:06.246181+0000","last_became_peered":"2024-09-13T08:47:06.246181+0000","last_unstale":"2024-09-13T08:47:08.779537+0000","last_undegraded":"2024-09-13T08:47:08.779537+0000","last_fullsized":"2024-09-13T08:47:08.779537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:54.760904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779519+0000","last_change":"2024-09-13T08:47:07.270274+0000","last_active":"2024-09-13T08:47:08.779519+0000","last_peered":"2024-09-13T08:47:08.779519+0000","last_clean":"2024-09-13T08:47:08.779519+0000","last_became_active":"2024-09-13T08:47:07.269966+0000","last_became_peered":"2024-09-13T08:47:07.269966+0000","last_unstale":"2024-09-13T08:47:08.779519+0000","last_undegraded":"2024-09-13T08:47:08.779519+0000","last_fullsized":"2024-09-13T08:47:08.779519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:38:19.673757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264432+0000","last_change":"2024-09-13T08:47:06.258172+0000","last_active":"2024-09-13T08:47:08.264432+0000","last_peered":"2024-09-13T08:47:08.264432+0000","last_clean":"2024-09-13T08:47:08.264432+0000","last_became_active":"2024-09-13T08:47:06.257892+0000","last_became_peered":"2024-09-13T08:47:06.257892+0000","last_unstale":"2024-09-13T08:47:08.264432+0000","last_undegraded":"2024-09-13T08:47:08.264432+0000","last_fullsized":"2024-09-13T08:47:08.264432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:46:19.201898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.274255+0000","last_change":"2024-09-13T08:47:07.267661+0000","last_active":"2024-09-13T08:47:07.274255+0000","last_peered":"2024-09-13T08:47:07.274255+0000","last_clean":"2024-09-13T08:47:07.274255+0000","last_became_active":"2024-09-13T08:47:07.267351+0000","last_became_peered":"2024-09-13T08:47:07.267351+0000","last_unstale":"2024-09-13T08:47:07.274255+0000","last_undegraded":"2024-09-13T08:47:07.274255+0000","last_fullsized":"2024-09-13T08:47:07.274255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:40:40.560343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"88'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779264+0000","last_change":"2024-09-13T08:47:06.253059+0000","last_active":"2024-09-13T08:47:08.779264+0000","last_peered":"2024-09-13T08:47:08.779264+0000","last_clean":"2024-09-13T08:47:08.779264+0000","last_became_active":"2024-09-13T08:47:06.252779+0000","last_became_peered":"2024-09-13T08:47:06.252779+0000","last_unstale":"2024-09-13T08:47:08.779264+0000","last_undegraded":"2024-09-13T08:47:08.779264+0000","last_fullsized":"2024-09-13T08:47:08.779264+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:04.710637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778507+0000","last_change":"2024-09-13T08:47:07.271662+0000","last_active":"2024-09-13T08:47:08.778507+0000","last_peered":"2024-09-13T08:47:08.778507+0000","last_clean":"2024-09-13T08:47:08.778507+0000","last_became_active":"2024-09-13T08:47:07.271378+0000","last_became_peered":"2024-09-13T08:47:07.271378+0000","last_unstale":"2024-09-13T08:47:08.778507+0000","last_undegraded":"2024-09-13T08:47:08.778507+0000","last_fullsized":"2024-09-13T08:47:08.778507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:11.179044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"88'6","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778795+0000","last_change":"2024-09-13T08:47:06.260192+0000","last_active":"2024-09-13T08:47:08.778795+0000","last_peered":"2024-09-13T08:47:08.778795+0000","last_clean":"2024-09-13T08:47:08.778795+0000","last_became_active":"2024-09-13T08:47:06.256320+0000","last_became_peered":"2024-09-13T08:47:06.256320+0000","last_unstale":"2024-09-13T08:47:08.778795+0000","last_undegraded":"2024-09-13T08:47:08.778795+0000","last_fullsized":"2024-09-13T08:47:08.778795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:25.856751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.281908+0000","last_change":"2024-09-13T08:47:07.277937+0000","last_active":"2024-09-13T08:47:07.281908+0000","last_peered":"2024-09-13T08:47:07.281908+0000","last_clean":"2024-09-13T08:47:07.281908+0000","last_became_active":"2024-09-13T08:47:07.277629+0000","last_became_peered":"2024-09-13T08:47:07.277629+0000","last_unstale":"2024-09-13T08:47:07.281908+0000","last_undegraded":"2024-09-13T08:47:07.281908+0000","last_fullsized":"2024-09-13T08:47:07.281908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:06.047568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.268082+0000","last_change":"2024-09-13T08:47:06.251892+0000","last_active":"2024-09-13T08:47:08.268082+0000","last_peered":"2024-09-13T08:47:08.268082+0000","last_clean":"2024-09-13T08:47:08.268082+0000","last_became_active":"2024-09-13T08:47:06.251610+0000","last_became_peered":"2024-09-13T08:47:06.251610+0000","last_unstale":"2024-09-13T08:47:08.268082+0000","last_undegraded":"2024-09-13T08:47:08.268082+0000","last_fullsized":"2024-09-13T08:47:08.268082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:22.892519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.779033+0000","last_change":"2024-09-13T08:47:07.273863+0000","last_active":"2024-09-13T08:47:08.779033+0000","last_peered":"2024-09-13T08:47:08.779033+0000","last_clean":"2024-09-13T08:47:08.779033+0000","last_became_active":"2024-09-13T08:47:07.271632+0000","last_became_peered":"2024-09-13T08:47:07.271632+0000","last_unstale":"2024-09-13T08:47:08.779033+0000","last_undegraded":"2024-09-13T08:47:08.779033+0000","last_fullsized":"2024-09-13T08:47:08.779033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:37.053901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266465+0000","last_change":"2024-09-13T08:47:06.257123+0000","last_active":"2024-09-13T08:47:08.266465+0000","last_peered":"2024-09-13T08:47:08.266465+0000","last_clean":"2024-09-13T08:47:08.266465+0000","last_became_active":"2024-09-13T08:47:06.256816+0000","last_became_peered":"2024-09-13T08:47:06.256816+0000","last_unstale":"2024-09-13T08:47:08.266465+0000","last_undegraded":"2024-09-13T08:47:08.266465+0000","last_fullsized":"2024-09-13T08:47:08.266465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:39.592371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778353+0000","last_change":"2024-09-13T08:47:07.271443+0000","last_active":"2024-09-13T08:47:08.778353+0000","last_peered":"2024-09-13T08:47:08.778353+0000","last_clean":"2024-09-13T08:47:08.778353+0000","last_became_active":"2024-09-13T08:47:07.271105+0000","last_became_peered":"2024-09-13T08:47:07.271105+0000","last_unstale":"2024-09-13T08:47:08.778353+0000","last_undegraded":"2024-09-13T08:47:08.778353+0000","last_fullsized":"2024-09-13T08:47:08.778353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:01:42.052534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.263913+0000","last_change":"2024-09-13T08:47:06.251974+0000","last_active":"2024-09-13T08:47:08.263913+0000","last_peered":"2024-09-13T08:47:08.263913+0000","last_clean":"2024-09-13T08:47:08.263913+0000","last_became_active":"2024-09-13T08:47:06.250512+0000","last_became_peered":"2024-09-13T08:47:06.250512+0000","last_unstale":"2024-09-13T08:47:08.263913+0000","last_undegraded":"2024-09-13T08:47:08.263913+0000","last_fullsized":"2024-09-13T08:47:08.263913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:23.995997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.274418+0000","last_change":"2024-09-13T08:47:07.270465+0000","last_active":"2024-09-13T08:47:07.274418+0000","last_peered":"2024-09-13T08:47:07.274418+0000","last_clean":"2024-09-13T08:47:07.274418+0000","last_became_active":"2024-09-13T08:47:07.269856+0000","last_became_peered":"2024-09-13T08:47:07.269856+0000","last_unstale":"2024-09-13T08:47:07.274418+0000","last_undegraded":"2024-09-13T08:47:07.274418+0000","last_fullsized":"2024-09-13T08:47:07.274418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:29:18.390616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"88'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.291495+0000","last_change":"2024-09-13T08:47:06.247980+0000","last_active":"2024-09-13T08:47:07.291495+0000","last_peered":"2024-09-13T08:47:07.291495+0000","last_clean":"2024-09-13T08:47:07.291495+0000","last_became_active":"2024-09-13T08:47:06.247713+0000","last_became_peered":"2024-09-13T08:47:06.247713+0000","last_unstale":"2024-09-13T08:47:07.291495+0000","last_undegraded":"2024-09-13T08:47:07.291495+0000","last_fullsized":"2024-09-13T08:47:07.291495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:22:36.922668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778733+0000","last_change":"2024-09-13T08:47:07.284620+0000","last_active":"2024-09-13T08:47:08.778733+0000","last_peered":"2024-09-13T08:47:08.778733+0000","last_clean":"2024-09-13T08:47:08.778733+0000","last_became_active":"2024-09-13T08:47:07.281855+0000","last_became_peered":"2024-09-13T08:47:07.281855+0000","last_unstale":"2024-09-13T08:47:08.778733+0000","last_undegraded":"2024-09-13T08:47:08.778733+0000","last_fullsized":"2024-09-13T08:47:08.778733+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:13.488445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778299+0000","last_change":"2024-09-13T08:47:06.259535+0000","last_active":"2024-09-13T08:47:08.778299+0000","last_peered":"2024-09-13T08:47:08.778299+0000","last_clean":"2024-09-13T08:47:08.778299+0000","last_became_active":"2024-09-13T08:47:06.259160+0000","last_became_peered":"2024-09-13T08:47:06.259160+0000","last_unstale":"2024-09-13T08:47:08.778299+0000","last_undegraded":"2024-09-13T08:47:08.778299+0000","last_fullsized":"2024-09-13T08:47:08.778299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:39:18.943376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264681+0000","last_change":"2024-09-13T08:47:07.275396+0000","last_active":"2024-09-13T08:47:08.264681+0000","last_peered":"2024-09-13T08:47:08.264681+0000","last_clean":"2024-09-13T08:47:08.264681+0000","last_became_active":"2024-09-13T08:47:07.275088+0000","last_became_peered":"2024-09-13T08:47:07.275088+0000","last_unstale":"2024-09-13T08:47:08.264681+0000","last_undegraded":"2024-09-13T08:47:08.264681+0000","last_fullsized":"2024-09-13T08:47:08.264681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:32.457578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778990+0000","last_change":"2024-09-13T08:47:06.258507+0000","last_active":"2024-09-13T08:47:08.778990+0000","last_peered":"2024-09-13T08:47:08.778990+0000","last_clean":"2024-09-13T08:47:08.778990+0000","last_became_active":"2024-09-13T08:47:06.258215+0000","last_became_peered":"2024-09-13T08:47:06.258215+0000","last_unstale":"2024-09-13T08:47:08.778990+0000","last_undegraded":"2024-09-13T08:47:08.778990+0000","last_fullsized":"2024-09-13T08:47:08.778990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:10:23.753114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778196+0000","last_change":"2024-09-13T08:47:07.282015+0000","last_active":"2024-09-13T08:47:08.778196+0000","last_peered":"2024-09-13T08:47:08.778196+0000","last_clean":"2024-09-13T08:47:08.778196+0000","last_became_active":"2024-09-13T08:47:07.281698+0000","last_became_peered":"2024-09-13T08:47:07.281698+0000","last_unstale":"2024-09-13T08:47:08.778196+0000","last_undegraded":"2024-09-13T08:47:08.778196+0000","last_fullsized":"2024-09-13T08:47:08.778196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:45:52.260401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778997+0000","last_change":"2024-09-13T08:47:06.252320+0000","last_active":"2024-09-13T08:47:08.778997+0000","last_peered":"2024-09-13T08:47:08.778997+0000","last_clean":"2024-09-13T08:47:08.778997+0000","last_became_active":"2024-09-13T08:47:06.252061+0000","last_became_peered":"2024-09-13T08:47:06.252061+0000","last_unstale":"2024-09-13T08:47:08.778997+0000","last_undegraded":"2024-09-13T08:47:08.778997+0000","last_fullsized":"2024-09-13T08:47:08.778997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:44.666211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266367+0000","last_change":"2024-09-13T08:47:07.270015+0000","last_active":"2024-09-13T08:47:08.266367+0000","last_peered":"2024-09-13T08:47:08.266367+0000","last_clean":"2024-09-13T08:47:08.266367+0000","last_became_active":"2024-09-13T08:47:07.269730+0000","last_became_peered":"2024-09-13T08:47:07.269730+0000","last_unstale":"2024-09-13T08:47:08.266367+0000","last_undegraded":"2024-09-13T08:47:08.266367+0000","last_fullsized":"2024-09-13T08:47:08.266367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:01.139461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778702+0000","last_change":"2024-09-13T08:47:06.252851+0000","last_active":"2024-09-13T08:47:08.778702+0000","last_peered":"2024-09-13T08:47:08.778702+0000","last_clean":"2024-09-13T08:47:08.778702+0000","last_became_active":"2024-09-13T08:47:06.251591+0000","last_became_peered":"2024-09-13T08:47:06.251591+0000","last_unstale":"2024-09-13T08:47:08.778702+0000","last_undegraded":"2024-09-13T08:47:08.778702+0000","last_fullsized":"2024-09-13T08:47:08.778702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:57.728136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778013+0000","last_change":"2024-09-13T08:47:07.270875+0000","last_active":"2024-09-13T08:47:08.778013+0000","last_peered":"2024-09-13T08:47:08.778013+0000","last_clean":"2024-09-13T08:47:08.778013+0000","last_became_active":"2024-09-13T08:47:07.270447+0000","last_became_peered":"2024-09-13T08:47:07.270447+0000","last_unstale":"2024-09-13T08:47:08.778013+0000","last_undegraded":"2024-09-13T08:47:08.778013+0000","last_fullsized":"2024-09-13T08:47:08.778013+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:39.531920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266544+0000","last_change":"2024-09-13T08:47:06.245122+0000","last_active":"2024-09-13T08:47:08.266544+0000","last_peered":"2024-09-13T08:47:08.266544+0000","last_clean":"2024-09-13T08:47:08.266544+0000","last_became_active":"2024-09-13T08:47:06.244855+0000","last_became_peered":"2024-09-13T08:47:06.244855+0000","last_unstale":"2024-09-13T08:47:08.266544+0000","last_undegraded":"2024-09-13T08:47:08.266544+0000","last_fullsized":"2024-09-13T08:47:08.266544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:58:46.424443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266279+0000","last_change":"2024-09-13T08:47:07.273725+0000","last_active":"2024-09-13T08:47:08.266279+0000","last_peered":"2024-09-13T08:47:08.266279+0000","last_clean":"2024-09-13T08:47:08.266279+0000","last_became_active":"2024-09-13T08:47:07.273382+0000","last_became_peered":"2024-09-13T08:47:07.273382+0000","last_unstale":"2024-09-13T08:47:08.266279+0000","last_undegraded":"2024-09-13T08:47:08.266279+0000","last_fullsized":"2024-09-13T08:47:08.266279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:01:00.963843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.260688+0000","last_change":"2024-09-13T08:47:06.241834+0000","last_active":"2024-09-13T08:47:07.260688+0000","last_peered":"2024-09-13T08:47:07.260688+0000","last_clean":"2024-09-13T08:47:07.260688+0000","last_became_active":"2024-09-13T08:47:06.240856+0000","last_became_peered":"2024-09-13T08:47:06.240856+0000","last_unstale":"2024-09-13T08:47:07.260688+0000","last_undegraded":"2024-09-13T08:47:07.260688+0000","last_fullsized":"2024-09-13T08:47:07.260688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:34.145719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777737+0000","last_change":"2024-09-13T08:47:07.271892+0000","last_active":"2024-09-13T08:47:08.777737+0000","last_peered":"2024-09-13T08:47:08.777737+0000","last_clean":"2024-09-13T08:47:08.777737+0000","last_became_active":"2024-09-13T08:47:07.271510+0000","last_became_peered":"2024-09-13T08:47:07.271510+0000","last_unstale":"2024-09-13T08:47:08.777737+0000","last_undegraded":"2024-09-13T08:47:08.777737+0000","last_fullsized":"2024-09-13T08:47:08.777737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:55.858407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778406+0000","last_change":"2024-09-13T08:47:06.253901+0000","last_active":"2024-09-13T08:47:08.778406+0000","last_peered":"2024-09-13T08:47:08.778406+0000","last_clean":"2024-09-13T08:47:08.778406+0000","last_became_active":"2024-09-13T08:47:06.253643+0000","last_became_peered":"2024-09-13T08:47:06.253643+0000","last_unstale":"2024-09-13T08:47:08.778406+0000","last_undegraded":"2024-09-13T08:47:08.778406+0000","last_fullsized":"2024-09-13T08:47:08.778406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:07.655485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778377+0000","last_change":"2024-09-13T08:47:07.263072+0000","last_active":"2024-09-13T08:47:08.778377+0000","last_peered":"2024-09-13T08:47:08.778377+0000","last_clean":"2024-09-13T08:47:08.778377+0000","last_became_active":"2024-09-13T08:47:07.261612+0000","last_became_peered":"2024-09-13T08:47:07.261612+0000","last_unstale":"2024-09-13T08:47:08.778377+0000","last_undegraded":"2024-09-13T08:47:08.778377+0000","last_fullsized":"2024-09-13T08:47:08.778377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:48:31.030455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778591+0000","last_change":"2024-09-13T08:47:06.263053+0000","last_active":"2024-09-13T08:47:08.778591+0000","last_peered":"2024-09-13T08:47:08.778591+0000","last_clean":"2024-09-13T08:47:08.778591+0000","last_became_active":"2024-09-13T08:47:06.262793+0000","last_became_peered":"2024-09-13T08:47:06.262793+0000","last_unstale":"2024-09-13T08:47:08.778591+0000","last_undegraded":"2024-09-13T08:47:08.778591+0000","last_fullsized":"2024-09-13T08:47:08.778591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:17:49.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778191+0000","last_change":"2024-09-13T08:47:07.280641+0000","last_active":"2024-09-13T08:47:08.778191+0000","last_peered":"2024-09-13T08:47:08.778191+0000","last_clean":"2024-09-13T08:47:08.778191+0000","last_became_active":"2024-09-13T08:47:07.280359+0000","last_became_peered":"2024-09-13T08:47:07.280359+0000","last_unstale":"2024-09-13T08:47:08.778191+0000","last_undegraded":"2024-09-13T08:47:08.778191+0000","last_fullsized":"2024-09-13T08:47:08.778191+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:11:03.114015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.263689+0000","last_change":"2024-09-13T08:47:06.243414+0000","last_active":"2024-09-13T08:47:07.263689+0000","last_peered":"2024-09-13T08:47:07.263689+0000","last_clean":"2024-09-13T08:47:07.263689+0000","last_became_active":"2024-09-13T08:47:06.243110+0000","last_became_peered":"2024-09-13T08:47:06.243110+0000","last_unstale":"2024-09-13T08:47:07.263689+0000","last_undegraded":"2024-09-13T08:47:07.263689+0000","last_fullsized":"2024-09-13T08:47:07.263689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:09.129430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778092+0000","last_change":"2024-09-13T08:47:07.269933+0000","last_active":"2024-09-13T08:47:08.778092+0000","last_peered":"2024-09-13T08:47:08.778092+0000","last_clean":"2024-09-13T08:47:08.778092+0000","last_became_active":"2024-09-13T08:47:07.267901+0000","last_became_peered":"2024-09-13T08:47:07.267901+0000","last_unstale":"2024-09-13T08:47:08.778092+0000","last_undegraded":"2024-09-13T08:47:08.778092+0000","last_fullsized":"2024-09-13T08:47:08.778092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:31.084989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778133+0000","last_change":"2024-09-13T08:47:06.254333+0000","last_active":"2024-09-13T08:47:08.778133+0000","last_peered":"2024-09-13T08:47:08.778133+0000","last_clean":"2024-09-13T08:47:08.778133+0000","last_became_active":"2024-09-13T08:47:06.253935+0000","last_became_peered":"2024-09-13T08:47:06.253935+0000","last_unstale":"2024-09-13T08:47:08.778133+0000","last_undegraded":"2024-09-13T08:47:08.778133+0000","last_fullsized":"2024-09-13T08:47:08.778133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:47:52.506936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264688+0000","last_change":"2024-09-13T08:47:07.281702+0000","last_active":"2024-09-13T08:47:08.264688+0000","last_peered":"2024-09-13T08:47:08.264688+0000","last_clean":"2024-09-13T08:47:08.264688+0000","last_became_active":"2024-09-13T08:47:07.281058+0000","last_became_peered":"2024-09-13T08:47:07.281058+0000","last_unstale":"2024-09-13T08:47:08.264688+0000","last_undegraded":"2024-09-13T08:47:08.264688+0000","last_fullsized":"2024-09-13T08:47:08.264688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:36:26.426873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264289+0000","last_change":"2024-09-13T08:47:06.253019+0000","last_active":"2024-09-13T08:47:08.264289+0000","last_peered":"2024-09-13T08:47:08.264289+0000","last_clean":"2024-09-13T08:47:08.264289+0000","last_became_active":"2024-09-13T08:47:06.252752+0000","last_became_peered":"2024-09-13T08:47:06.252752+0000","last_unstale":"2024-09-13T08:47:08.264289+0000","last_undegraded":"2024-09-13T08:47:08.264289+0000","last_fullsized":"2024-09-13T08:47:08.264289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:00:14.462714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778024+0000","last_change":"2024-09-13T08:47:07.269622+0000","last_active":"2024-09-13T08:47:08.778024+0000","last_peered":"2024-09-13T08:47:08.778024+0000","last_clean":"2024-09-13T08:47:08.778024+0000","last_became_active":"2024-09-13T08:47:07.269265+0000","last_became_peered":"2024-09-13T08:47:07.269265+0000","last_unstale":"2024-09-13T08:47:08.778024+0000","last_undegraded":"2024-09-13T08:47:08.778024+0000","last_fullsized":"2024-09-13T08:47:08.778024+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:57.889677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.266802+0000","last_change":"2024-09-13T08:47:06.246380+0000","last_active":"2024-09-13T08:47:08.266802+0000","last_peered":"2024-09-13T08:47:08.266802+0000","last_clean":"2024-09-13T08:47:08.266802+0000","last_became_active":"2024-09-13T08:47:06.246119+0000","last_became_peered":"2024-09-13T08:47:06.246119+0000","last_unstale":"2024-09-13T08:47:08.266802+0000","last_undegraded":"2024-09-13T08:47:08.266802+0000","last_fullsized":"2024-09-13T08:47:08.266802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:35.160960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.283829+0000","last_change":"2024-09-13T08:47:07.281687+0000","last_active":"2024-09-13T08:47:07.283829+0000","last_peered":"2024-09-13T08:47:07.283829+0000","last_clean":"2024-09-13T08:47:07.283829+0000","last_became_active":"2024-09-13T08:47:07.281402+0000","last_became_peered":"2024-09-13T08:47:07.281402+0000","last_unstale":"2024-09-13T08:47:07.283829+0000","last_undegraded":"2024-09-13T08:47:07.283829+0000","last_fullsized":"2024-09-13T08:47:07.283829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:49:10.314494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.264133+0000","last_change":"2024-09-13T08:47:06.244678+0000","last_active":"2024-09-13T08:47:08.264133+0000","last_peered":"2024-09-13T08:47:08.264133+0000","last_clean":"2024-09-13T08:47:08.264133+0000","last_became_active":"2024-09-13T08:47:06.243503+0000","last_became_peered":"2024-09-13T08:47:06.243503+0000","last_unstale":"2024-09-13T08:47:08.264133+0000","last_undegraded":"2024-09-13T08:47:08.264133+0000","last_fullsized":"2024-09-13T08:47:08.264133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:26:51.126833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778449+0000","last_change":"2024-09-13T08:47:07.270837+0000","last_active":"2024-09-13T08:47:08.778449+0000","last_peered":"2024-09-13T08:47:08.778449+0000","last_clean":"2024-09-13T08:47:08.778449+0000","last_became_active":"2024-09-13T08:47:07.270174+0000","last_became_peered":"2024-09-13T08:47:07.270174+0000","last_unstale":"2024-09-13T08:47:08.778449+0000","last_undegraded":"2024-09-13T08:47:08.778449+0000","last_fullsized":"2024-09-13T08:47:08.778449+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:28:00.359095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777778+0000","last_change":"2024-09-13T08:47:06.246725+0000","last_active":"2024-09-13T08:47:08.777778+0000","last_peered":"2024-09-13T08:47:08.777778+0000","last_clean":"2024-09-13T08:47:08.777778+0000","last_became_active":"2024-09-13T08:47:06.246390+0000","last_became_peered":"2024-09-13T08:47:06.246390+0000","last_unstale":"2024-09-13T08:47:08.777778+0000","last_undegraded":"2024-09-13T08:47:08.777778+0000","last_fullsized":"2024-09-13T08:47:08.777778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:43.573769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777754+0000","last_change":"2024-09-13T08:47:07.265297+0000","last_active":"2024-09-13T08:47:08.777754+0000","last_peered":"2024-09-13T08:47:08.777754+0000","last_clean":"2024-09-13T08:47:08.777754+0000","last_became_active":"2024-09-13T08:47:07.264985+0000","last_became_peered":"2024-09-13T08:47:07.264985+0000","last_unstale":"2024-09-13T08:47:08.777754+0000","last_undegraded":"2024-09-13T08:47:08.777754+0000","last_fullsized":"2024-09-13T08:47:08.777754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:36.676460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777461+0000","last_change":"2024-09-13T08:47:06.255961+0000","last_active":"2024-09-13T08:47:08.777461+0000","last_peered":"2024-09-13T08:47:08.777461+0000","last_clean":"2024-09-13T08:47:08.777461+0000","last_became_active":"2024-09-13T08:47:06.255686+0000","last_became_peered":"2024-09-13T08:47:06.255686+0000","last_unstale":"2024-09-13T08:47:08.777461+0000","last_undegraded":"2024-09-13T08:47:08.777461+0000","last_fullsized":"2024-09-13T08:47:08.777461+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:32.249019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T08:47:07.267162+0000","last_change":"2024-09-13T08:47:07.264195+0000","last_active":"2024-09-13T08:47:07.267162+0000","last_peered":"2024-09-13T08:47:07.267162+0000","last_clean":"2024-09-13T08:47:07.267162+0000","last_became_active":"2024-09-13T08:47:07.263862+0000","last_became_peered":"2024-09-13T08:47:07.263862+0000","last_unstale":"2024-09-13T08:47:07.267162+0000","last_undegraded":"2024-09-13T08:47:07.267162+0000","last_fullsized":"2024-09-13T08:47:07.267162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:06.226157+0000","last_clean_scrub_stamp":"2024-09-13T08:47:06.226157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:31:46.891964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.778783+0000","last_change":"2024-09-13T08:47:06.252350+0000","last_active":"2024-09-13T08:47:08.778783+0000","last_peered":"2024-09-13T08:47:08.778783+0000","last_clean":"2024-09-13T08:47:08.778783+0000","last_became_active":"2024-09-13T08:47:06.250313+0000","last_became_peered":"2024-09-13T08:47:06.250313+0000","last_unstale":"2024-09-13T08:47:08.778783+0000","last_undegraded":"2024-09-13T08:47:08.778783+0000","last_fullsized":"2024-09-13T08:47:08.778783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:47:05.218798+0000","last_clean_scrub_stamp":"2024-09-13T08:47:05.218798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:01.941502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T08:47:08.777592+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:47:08.777592+0000","last_peered":"2024-09-13T08:47:08.777592+0000","last_clean":"2024-09-13T08:47:08.777592+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:47:08.777592+0000","last_undegraded":"2024-09-13T08:47:08.777592+0000","last_fullsized":"2024-09-13T08:47:08.777592+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509559,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58616,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313224,"statfs":{"total":96636764160,"available":96576741376,"internally_reserved":0,"allocated":1077248,"data_stored":462434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509559,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59220,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312620,"statfs":{"total":96636764160,"available":96576122880,"internally_reserved":0,"allocated":1695744,"data_stored":1060762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509560,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64124,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307716,"statfs":{"total":96636764160,"available":96571101184,"internally_reserved":0,"allocated":1089536,"data_stored":468782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509560,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58624,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313216,"statfs":{"total":96636764160,"available":96576733184,"internally_reserved":0,"allocated":1085440,"data_stored":463038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509559,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58604,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313236,"statfs":{"total":96636764160,"available":96576753664,"internally_reserved":0,"allocated":1073152,"data_stored":462364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509561,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59208,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312632,"statfs":{"total":96636764160,"available":96576135168,"internally_reserved":0,"allocated":1683456,"data_stored":1059206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58620,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313220,"statfs":{"total":96636764160,"available":96576737280,"internally_reserved":0,"allocated":1089536,"data_stored":463998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542265,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64124,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307716,"statfs":{"total":96636764160,"available":96571101184,"internally_reserved":0,"allocated":1081344,"data_stored":463036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:47:11.821 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:47:11.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T08:47:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.012+0000 7f279c400640 1 -- 172.21.15.44:0/1778301462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27941605d0 msgr2=0x7f27940d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.012+0000 7f279c400640 1 --2- 172.21.15.44:0/1778301462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27941605d0 0x7f27940d0690 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2794074840 tx=0x7f278801c8a0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.012+0000 7f279c400640 1 -- 172.21.15.44:0/1778301462 shutdown_connections 2024-09-13T08:47:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.012+0000 7f279c400640 1 --2- 172.21.15.44:0/1778301462 >> v2:172.21.15.129:3300/0 conn(0x7f27940aeb00 0x7f27940aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.012+0000 7f279c400640 1 --2- 172.21.15.44:0/1778301462 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27940b1650 0x7f27940b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.012+0000 7f279c400640 1 --2- 172.21.15.44:0/1778301462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27941605d0 0x7f27940d0690 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.012+0000 7f279c400640 1 -- 172.21.15.44:0/1778301462 >> 172.21.15.44:0/1778301462 conn(0x7f27940d1dd0 msgr2=0x7f27940d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.012+0000 7f279c400640 1 -- 172.21.15.44:0/1778301462 shutdown_connections 2024-09-13T08:47:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.012+0000 7f279c400640 1 -- 172.21.15.44:0/1778301462 wait complete. 2024-09-13T08:47:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.012+0000 7f279c400640 1 Processor -- start 2024-09-13T08:47:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279c400640 1 -- start start 2024-09-13T08:47:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279c400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27940aeb00 0x7f27940cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279c400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27940b1650 0x7f2794165df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279c400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27941605d0 0x7f279406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279c400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27940b13a0 con 0x7f27941605d0 2024-09-13T08:47:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279c400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f27940b1220 con 0x7f27940b1650 2024-09-13T08:47:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279c400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f27940afd00 con 0x7f27940aeb00 2024-09-13T08:47:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27941605d0 0x7f279406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27941605d0 0x7f279406e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33478/0 (socket says 172.21.15.44:33478) 2024-09-13T08:47:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279a400640 1 -- 172.21.15.44:0/3528740832 learned_addr learned my addr 172.21.15.44:0/3528740832 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f2799a00640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27940aeb00 0x7f27940cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f2799000640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27940b1650 0x7f2794165df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279a400640 1 -- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27940aeb00 msgr2=0x7f27940cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279a400640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27940aeb00 0x7f27940cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279a400640 1 -- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27940b1650 msgr2=0x7f2794165df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279a400640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27940b1650 0x7f2794165df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279a400640 1 -- 172.21.15.44:0/3528740832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27940cbd90 con 0x7f27941605d0 2024-09-13T08:47:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279a400640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27941605d0 0x7f279406e9f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f278c00cce0 tx=0x7f278c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f2791600640 1 -- 172.21.15.44:0/3528740832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c00f450 con 0x7f27941605d0 2024-09-13T08:47:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f2791600640 1 -- 172.21.15.44:0/3528740832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f278c00f6e0 con 0x7f27941605d0 2024-09-13T08:47:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279c400640 1 -- 172.21.15.44:0/3528740832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279406df10 con 0x7f27941605d0 2024-09-13T08:47:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f2791600640 1 -- 172.21.15.44:0/3528740832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c00fb30 con 0x7f27941605d0 2024-09-13T08:47:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279c400640 1 -- 172.21.15.44:0/3528740832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279406e4d0 con 0x7f27941605d0 2024-09-13T08:47:12.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.016+0000 7f279c400640 1 -- 172.21.15.44:0/3528740832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f279406e6d0 con 0x7f27941605d0 2024-09-13T08:47:12.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.020+0000 7f2791600640 1 -- 172.21.15.44:0/3528740832 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f278c00fcd0 con 0x7f27941605d0 2024-09-13T08:47:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.020+0000 7f2791600640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2774074060 0x7f2774076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.020+0000 7f2791600640 1 -- 172.21.15.44:0/3528740832 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f278c093800 con 0x7f27941605d0 2024-09-13T08:47:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.020+0000 7f2799a00640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2774074060 0x7f2774076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.020+0000 7f2799a00640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2774074060 0x7f2774076520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f27880af000 tx=0x7f278801b0e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:12.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.024+0000 7f2791600640 1 -- 172.21.15.44:0/3528740832 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f279406e6d0 con 0x7f27941605d0 2024-09-13T08:47:12.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.264+0000 7f279c400640 1 -- 172.21.15.44:0/3528740832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f27940b1a30 con 0x7f27941605d0 2024-09-13T08:47:12.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.264+0000 7f2791600640 1 -- 172.21.15.44:0/3528740832 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f278c014030 con 0x7f27941605d0 2024-09-13T08:47:12.267 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T08:47:12.267 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:47:12.267 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-09-13T08:47:12.267 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-09-13T08:47:12.267 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-13T08:47:12.268 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-09-13T08:47:12.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 -- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2774074060 msgr2=0x7f2774076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:12.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2774074060 0x7f2774076520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f27880af000 tx=0x7f278801b0e0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 -- 172.21.15.44:0/3528740832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27941605d0 msgr2=0x7f279406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:12.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27941605d0 0x7f279406e9f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f278c00cce0 tx=0x7f278c00a620 comp rx=0 tx=0).stop 2024-09-13T08:47:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 -- 172.21.15.44:0/3528740832 shutdown_connections 2024-09-13T08:47:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2774074060 0x7f2774076520 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27941605d0 0x7f279406e9f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27940b1650 0x7f2794165df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 --2- 172.21.15.44:0/3528740832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27940aeb00 0x7f27940cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 -- 172.21.15.44:0/3528740832 >> 172.21.15.44:0/3528740832 conn(0x7f27940d1dd0 msgr2=0x7f27940d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 -- 172.21.15.44:0/3528740832 shutdown_connections 2024-09-13T08:47:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.268+0000 7f279c400640 1 -- 172.21.15.44:0/3528740832 wait complete. 2024-09-13T08:47:12.290 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:47:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 -- 172.21.15.44:0/3997310701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f727415ed00 msgr2=0x7f727415b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 --2- 172.21.15.44:0/3997310701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f727415ed00 0x7f727415b1d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f72740753f0 tx=0x7f726c01f580 comp rx=0 tx=0).stop 2024-09-13T08:47:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 -- 172.21.15.44:0/3997310701 shutdown_connections 2024-09-13T08:47:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 --2- 172.21.15.44:0/3997310701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f727415ed00 0x7f727415b1d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 --2- 172.21.15.44:0/3997310701 >> v2:172.21.15.129:3300/0 conn(0x7f7274074490 0x7f727415e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 --2- 172.21.15.44:0/3997310701 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f72741585b0 0x7f72740cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 -- 172.21.15.44:0/3997310701 >> 172.21.15.44:0/3997310701 conn(0x7f72740d0c90 msgr2=0x7f72740d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 -- 172.21.15.44:0/3997310701 shutdown_connections 2024-09-13T08:47:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 -- 172.21.15.44:0/3997310701 wait complete. 2024-09-13T08:47:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 Processor -- start 2024-09-13T08:47:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.436+0000 7f727b800640 1 -- start start 2024-09-13T08:47:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f727b800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7274074490 0x7f72741f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f727b800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72741585b0 0x7f72741f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f727b800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f727415ed00 0x7f72740b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f727b800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f727415b890 con 0x7f72741585b0 2024-09-13T08:47:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f727b800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f727415b710 con 0x7f7274074490 2024-09-13T08:47:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f727b800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f727415ba10 con 0x7f727415ed00 2024-09-13T08:47:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f726be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72741585b0 0x7f72741f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f726be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72741585b0 0x7f72741f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38274/0 (socket says 172.21.15.44:38274) 2024-09-13T08:47:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f726be00640 1 -- 172.21.15.44:0/162124850 learned_addr learned my addr 172.21.15.44:0/162124850 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f7279800640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f727415ed00 0x7f72740b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:12.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f7278e00640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7274074490 0x7f72741f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:12.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f726be00640 1 -- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f727415ed00 msgr2=0x7f72740b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:12.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f726be00640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f727415ed00 0x7f72740b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f726be00640 1 -- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7274074490 msgr2=0x7f72741f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:12.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f726be00640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7274074490 0x7f72741f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f726be00640 1 -- 172.21.15.44:0/162124850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7274155b30 con 0x7f72741585b0 2024-09-13T08:47:12.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f7278e00640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7274074490 0x7f72741f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:47:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f726be00640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72741585b0 0x7f72741f4590 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f725c00d980 tx=0x7f725c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f7271600640 1 -- 172.21.15.44:0/162124850 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c017030 con 0x7f72741585b0 2024-09-13T08:47:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f7271600640 1 -- 172.21.15.44:0/162124850 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f725c009b20 con 0x7f72741585b0 2024-09-13T08:47:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f7271600640 1 -- 172.21.15.44:0/162124850 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c004020 con 0x7f72741585b0 2024-09-13T08:47:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f727b800640 1 -- 172.21.15.44:0/162124850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72741f4c70 con 0x7f72741585b0 2024-09-13T08:47:12.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f727b800640 1 -- 172.21.15.44:0/162124850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72741f5230 con 0x7f72741585b0 2024-09-13T08:47:12.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f727b800640 1 -- 172.21.15.44:0/162124850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727415bec0 con 0x7f72741585b0 2024-09-13T08:47:12.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.440+0000 7f7271600640 1 -- 172.21.15.44:0/162124850 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f725c01a070 con 0x7f72741585b0 2024-09-13T08:47:12.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.444+0000 7f7271600640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7250074030 0x7f72500764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:12.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.444+0000 7f7271600640 1 -- 172.21.15.44:0/162124850 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f725c093050 con 0x7f72741585b0 2024-09-13T08:47:12.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.444+0000 7f7278e00640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7250074030 0x7f72500764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:12.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.444+0000 7f7278e00640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7250074030 0x7f72500764f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7264009ae0 tx=0x7f72640091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:12.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.448+0000 7f7271600640 1 -- 172.21.15.44:0/162124850 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f725c098e20 con 0x7f72741585b0 2024-09-13T08:47:12.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.660+0000 7f727b800640 1 -- 172.21.15.44:0/162124850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72740cf600 con 0x7f72741585b0 2024-09-13T08:47:12.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f7271600640 1 -- 172.21.15.44:0/162124850 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5228 (secure 0 0 0) 0x7f7274155b30 con 0x7f72741585b0 2024-09-13T08:47:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:12.665 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":56,"btime":"2024-09-13T08:47:10:272875+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/833446483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":833446483},{"type":"v1","addr":"172.21.15.129:6836","nonce":833446483}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1844578230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1844578230},{"type":"v1","addr":"172.21.15.129:6837","nonce":1844578230}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2007417535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2007417535},{"type":"v1","addr":"172.21.15.44:6833","nonce":2007417535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:47:06.650500+0000","modified":"2024-09-13T08:47:09.677516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6942":{"gid":6942,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1796790158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1796790158},{"type":"v1","addr":"172.21.15.44:6835","nonce":1796790158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6942,"qdb_cluster":[6942]},"id":6}]} 2024-09-13T08:47:12.665 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 56 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 -- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7250074030 msgr2=0x7f72500764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7250074030 0x7f72500764f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7264009ae0 tx=0x7f72640091e0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 -- 172.21.15.44:0/162124850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72741585b0 msgr2=0x7f72741f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72741585b0 0x7f72741f4590 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f725c00d980 tx=0x7f725c00de40 comp rx=0 tx=0).stop 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 -- 172.21.15.44:0/162124850 shutdown_connections 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7250074030 0x7f72500764f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f727415ed00 0x7f72740b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72741585b0 0x7f72741f4590 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 --2- 172.21.15.44:0/162124850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7274074490 0x7f72741f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 -- 172.21.15.44:0/162124850 >> 172.21.15.44:0/162124850 conn(0x7f72740d0c90 msgr2=0x7f72740ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 -- 172.21.15.44:0/162124850 shutdown_connections 2024-09-13T08:47:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:12.664+0000 7f727b800640 1 -- 172.21.15.44:0/162124850 wait complete. 2024-09-13T08:47:13.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T08:47:06.650500+0000', 'modified': '2024-09-13T08:47:09.677516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6942': {'gid': 6942, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/1796790158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 1796790158}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 1796790158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6942, 'qdb_cluster': [6942]} 2024-09-13T08:47:13.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T08:47:13.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T08:47:13.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 -- 172.21.15.44:0/3742791006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950415c1c0 msgr2=0x7f950406d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 --2- 172.21.15.44:0/3742791006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950415c1c0 0x7f950406d000 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009840 tx=0x7f94ec01c7f0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 -- 172.21.15.44:0/3742791006 shutdown_connections 2024-09-13T08:47:13.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 --2- 172.21.15.44:0/3742791006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950415c1c0 0x7f950406d000 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 --2- 172.21.15.44:0/3742791006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950406cc20 0x7f95040737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 -- 172.21.15.44:0/3742791006 >> 172.21.15.44:0/3742791006 conn(0x7f95040d0c90 msgr2=0x7f95040d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:13.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 -- 172.21.15.44:0/3742791006 shutdown_connections 2024-09-13T08:47:13.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 -- 172.21.15.44:0/3742791006 wait complete. 2024-09-13T08:47:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 Processor -- start 2024-09-13T08:47:13.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 -- start start 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f950406cc20 0x7f95041f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950415c1c0 0x7f95040b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950415f7b0 0x7f95040b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9504156600 con 0x7f950406cc20 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9504156480 con 0x7f950415f7b0 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f950b000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9504169360 con 0x7f950415c1c0 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f9503e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f950406cc20 0x7f95041f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f9503e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f950406cc20 0x7f95041f88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38286/0 (socket says 172.21.15.44:38286) 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f9503e00640 1 -- 172.21.15.44:0/3143208700 learned_addr learned my addr 172.21.15.44:0/3143208700 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f94fbe00640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950415c1c0 0x7f95040b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f9509000640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950415f7b0 0x7f95040b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f9503e00640 1 -- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950415c1c0 msgr2=0x7f95040b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f9503e00640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950415c1c0 0x7f95040b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f9503e00640 1 -- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950415f7b0 msgr2=0x7f95040b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f9503e00640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950415f7b0 0x7f95040b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f9503e00640 1 -- 172.21.15.44:0/3143208700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95040b3c60 con 0x7f950406cc20 2024-09-13T08:47:13.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.240+0000 7f9509000640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950415f7b0 0x7f95040b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:47:13.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f9503e00640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f950406cc20 0x7f95041f88d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f94f400e9d0 tx=0x7f94f400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:13.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f9500c00640 1 -- 172.21.15.44:0/3143208700 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f400cde0 con 0x7f950406cc20 2024-09-13T08:47:13.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f9500c00640 1 -- 172.21.15.44:0/3143208700 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94f4004020 con 0x7f950406cc20 2024-09-13T08:47:13.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f950b000640 1 -- 172.21.15.44:0/3143208700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95040b3f40 con 0x7f950406cc20 2024-09-13T08:47:13.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f9500c00640 1 -- 172.21.15.44:0/3143208700 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4004450 con 0x7f950406cc20 2024-09-13T08:47:13.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f950b000640 1 -- 172.21.15.44:0/3143208700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95040b4500 con 0x7f950406cc20 2024-09-13T08:47:13.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f9500c00640 1 -- 172.21.15.44:0/3143208700 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f94f4003600 con 0x7f950406cc20 2024-09-13T08:47:13.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f9500c00640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0074030 0x7f94e00764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f9500c00640 1 -- 172.21.15.44:0/3143208700 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f94f40936c0 con 0x7f950406cc20 2024-09-13T08:47:13.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f950e482000 1 Processor -- start 2024-09-13T08:47:13.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f950e482000 1 -- start start 2024-09-13T08:47:13.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.244+0000 7f94fbe00640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0074030 0x7f94e00764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 0x55c75292cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c75292d2f0 0x55c7528e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7528e0e10 0x55c7528e11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c7528013f0 con 0x55c7528e0e10 2024-09-13T08:47:13.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c75273eb20 con 0x55c75292d2f0 2024-09-13T08:47:13.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55c75272df50 con 0x55c75292c9d0 2024-09-13T08:47:13.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f9509000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7528e0e10 0x55c7528e11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f9509000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7528e0e10 0x55c7528e11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38294/0 (socket says 172.21.15.44:38294) 2024-09-13T08:47:13.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f9509000640 1 -- 172.21.15.44:0/3629499557 learned_addr learned my addr 172.21.15.44:0/3629499557 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:13.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f94fbe00640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0074030 0x7f94e00764f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0af000 tx=0x7f94ec01b030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f9503e00640 1 --2- 172.21.15.44:0/3629499557 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 0x55c75292cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f94fbe00640 1 --2- 172.21.15.44:0/3629499557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c75292d2f0 0x55c7528e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f9509000640 1 -- 172.21.15.44:0/3629499557 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 msgr2=0x55c75292cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f9509000640 1 --2- 172.21.15.44:0/3629499557 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 0x55c75292cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f9509000640 1 -- 172.21.15.44:0/3629499557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c75292d2f0 msgr2=0x55c7528e08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f9509000640 1 --2- 172.21.15.44:0/3629499557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c75292d2f0 0x55c7528e08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f9509000640 1 -- 172.21.15.44:0/3629499557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7526f5590 con 0x55c7528e0e10 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f9509000640 1 --2- 172.21.15.44:0/3629499557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7528e0e10 0x55c7528e11f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f94fc001c70 tx=0x7f94fc002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6850ac95735f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f94f8c00640 1 -- 172.21.15.44:0/3629499557 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc019070 con 0x55c7528e0e10 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f94f8c00640 1 -- 172.21.15.44:0/3629499557 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94fc003c80 con 0x55c7528e0e10 2024-09-13T08:47:13.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f94f8c00640 1 -- 172.21.15.44:0/3629499557 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc00eac0 con 0x55c7528e0e10 2024-09-13T08:47:13.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 -- 172.21.15.44:0/3629499557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7528e0e10 msgr2=0x55c7528e11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 --2- 172.21.15.44:0/3629499557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7528e0e10 0x55c7528e11f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f94fc001c70 tx=0x7f94fc002130 comp rx=0 tx=0).stop 2024-09-13T08:47:13.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 -- 172.21.15.44:0/3629499557 shutdown_connections 2024-09-13T08:47:13.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 --2- 172.21.15.44:0/3629499557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7528e0e10 0x55c7528e11f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 --2- 172.21.15.44:0/3629499557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c75292d2f0 0x55c7528e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 --2- 172.21.15.44:0/3629499557 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 0x55c75292cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 -- 172.21.15.44:0/3629499557 >> 172.21.15.44:0/3629499557 conn(0x55c7527ca130 msgr2=0x55c75290fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:13.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 -- 172.21.15.44:0/3629499557 shutdown_connections 2024-09-13T08:47:13.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 -- 172.21.15.44:0/3629499557 wait complete. 2024-09-13T08:47:13.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 -- start start 2024-09-13T08:47:13.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.248+0000 7f950e482000 1 Processor -- start 2024-09-13T08:47:13.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 0x55c75290d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 0x55c75290de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75290e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c7526c85d0 con 0x55c75292d2f0 2024-09-13T08:47:13.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c752731d60 con 0x55c7528e0e10 2024-09-13T08:47:13.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55c7526d4500 con 0x55c75292c9d0 2024-09-13T08:47:13.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 10 client.0 _ll_register_callbacks cb 0x55c7526d8810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:47:13.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9509000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75290e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9509000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75290e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38300/0 (socket says 172.21.15.44:38300) 2024-09-13T08:47:13.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9509000640 1 -- 172.21.15.44:0/4046885928 learned_addr learned my addr 172.21.15.44:0/4046885928 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:13.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9503e00640 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 0x55c75290d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94fbe00640 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 0x55c75290de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9509000640 1 -- 172.21.15.44:0/4046885928 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 msgr2=0x55c75290de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:47:13.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9509000640 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 0x55c75290de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9509000640 1 -- 172.21.15.44:0/4046885928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 msgr2=0x55c75290d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:47:13.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9509000640 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 0x55c75290d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9509000640 1 -- 172.21.15.44:0/4046885928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7526cc170 con 0x55c75292d2f0 2024-09-13T08:47:13.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94fbe00640 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 0x55c75290de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:47:13.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9503e00640 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 0x55c75290d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:47:13.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9509000640 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75290e3b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009010 tx=0x7f94fc003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7370ce4098285ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:13.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94f1600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:47:13.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94f1600640 1 -- 172.21.15.44:0/4046885928 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc019040 con 0x55c75292d2f0 2024-09-13T08:47:13.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94f1600640 1 -- 172.21.15.44:0/4046885928 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f94fc00e920 con 0x55c75292d2f0 2024-09-13T08:47:13.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 -- 172.21.15.44:0/4046885928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c7526c60f0 con 0x55c75292d2f0 2024-09-13T08:47:13.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94f1600640 1 -- 172.21.15.44:0/4046885928 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f94fc02c050 con 0x55c75292d2f0 2024-09-13T08:47:13.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 10 client.7044 fetch_fsmap learned FSMap version 56 2024-09-13T08:47:13.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 -- 172.21.15.44:0/4046885928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55c7527b2ef0 con 0x55c75292d2f0 2024-09-13T08:47:13.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94f1600640 1 -- 172.21.15.44:0/4046885928 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c7526c60f0 con 0x55c75292d2f0 2024-09-13T08:47:13.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94f1600640 1 -- 172.21.15.44:0/4046885928 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7f94fc02f020 con 0x55c75292d2f0 2024-09-13T08:47:13.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 10 client.7044 fetch_fsmap finished waiting for FSMap version 56 2024-09-13T08:47:13.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 10 client.7044 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:47:13.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 20 client.7044 populate_metadata read hostname 'smithi044' 2024-09-13T08:47:13.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.44:6834/1796790158,v1:172.21.15.44:6835/1796790158] conn(0x55c75290f0f0 0x55c75293ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 4 client.7044 mds_command: new command op to 6942 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T08:47:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 -- 172.21.15.44:0/4046885928 --> [v2:172.21.15.44:6834/1796790158,v1:172.21.15.44:6835/1796790158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c752740a70 con 0x55c75290f0f0 2024-09-13T08:47:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9503e00640 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.44:6834/1796790158,v1:172.21.15.44:6835/1796790158] conn(0x55c75290f0f0 0x55c75293ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f9503e00640 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.44:6834/1796790158,v1:172.21.15.44:6835/1796790158] conn(0x55c75290f0f0 0x55c75293ec90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3287c166136db80 server_cookie=a5272e75ca1c450f in_seq=0 out_seq=0 2024-09-13T08:47:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94f1600640 10 client.7044 ms_handle_connect on v2:172.21.15.44:6834/1796790158 2024-09-13T08:47:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94f1600640 1 -- 172.21.15.44:0/4046885928 <== mds.0 v2:172.21.15.44:6834/1796790158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c752740a70 con 0x55c75290f0f0 2024-09-13T08:47:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f94f1600640 10 client.7044 handle_command_reply: tid=0 2024-09-13T08:47:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 client.7044 shutdown 2024-09-13T08:47:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 -- 172.21.15.44:0/4046885928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 msgr2=0x55c75290e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.252+0000 7f950e482000 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75290e3b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009010 tx=0x7f94fc003dc0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.256+0000 7f950e482000 1 -- 172.21.15.44:0/4046885928 shutdown_connections 2024-09-13T08:47:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.256+0000 7f950e482000 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.44:6834/1796790158,v1:172.21.15.44:6835/1796790158] conn(0x55c75290f0f0 0x55c75293ec90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.256+0000 7f950e482000 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75290e3b0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.256+0000 7f950e482000 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292c9d0 0x55c75290de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.256+0000 7f950e482000 1 --2- 172.21.15.44:0/4046885928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 0x55c75290d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.256+0000 7f94f1600640 0 client.7044 ms_handle_reset on v2:172.21.15.44:6834/1796790158 2024-09-13T08:47:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.256+0000 7f950e482000 1 -- 172.21.15.44:0/4046885928 >> 172.21.15.44:0/4046885928 conn(0x55c7527ca130 msgr2=0x55c7528dffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:13.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.256+0000 7f950e482000 1 -- 172.21.15.44:0/4046885928 shutdown_connections 2024-09-13T08:47:13.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.256+0000 7f950e482000 1 -- 172.21.15.44:0/4046885928 wait complete. 2024-09-13T08:47:13.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.256+0000 7f950e482000 20 client.7044 trim_cache size 0 max 16384 2024-09-13T08:47:13.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.272+0000 7f950e482000 1 Processor -- start 2024-09-13T08:47:13.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.272+0000 7f950e482000 1 -- start start 2024-09-13T08:47:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c7528e0e10 0x55c75299efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c75292c9d0 0x55c75299f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75299fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9504204270 con 0x55c75292d2f0 2024-09-13T08:47:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f95040b23e0 con 0x55c75292c9d0 2024-09-13T08:47:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f95040b1740 con 0x55c7528e0e10 2024-09-13T08:47:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f9509000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75299fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f9509000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75299fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38306/0 (socket says 172.21.15.44:38306) 2024-09-13T08:47:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f9509000640 1 -- 172.21.15.44:0/528646206 learned_addr learned my addr 172.21.15.44:0/528646206 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f9503e00640 1 --2- 172.21.15.44:0/528646206 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c7528e0e10 0x55c75299efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f94fbe00640 1 --2- 172.21.15.44:0/528646206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c75292c9d0 0x55c75299f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f9509000640 1 -- 172.21.15.44:0/528646206 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c7528e0e10 msgr2=0x55c75299efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:47:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f9509000640 1 --2- 172.21.15.44:0/528646206 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c7528e0e10 0x55c75299efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f9509000640 1 -- 172.21.15.44:0/528646206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c75292c9d0 msgr2=0x55c75299f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:47:13.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f9509000640 1 --2- 172.21.15.44:0/528646206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c75292c9d0 0x55c75299f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f9509000640 1 -- 172.21.15.44:0/528646206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c752789d90 con 0x55c75292d2f0 2024-09-13T08:47:13.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f9509000640 1 --2- 172.21.15.44:0/528646206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75299fa60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00e5b0 tx=0x7f94fc002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f0e10a62cc69c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:13.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f94f3e00640 1 -- 172.21.15.44:0/528646206 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc019070 con 0x55c75292d2f0 2024-09-13T08:47:13.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f94f3e00640 1 -- 172.21.15.44:0/528646206 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94fc024070 con 0x55c75292d2f0 2024-09-13T08:47:13.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f94f3e00640 1 -- 172.21.15.44:0/528646206 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc00e920 con 0x55c75292d2f0 2024-09-13T08:47:13.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 -- 172.21.15.44:0/528646206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 msgr2=0x55c75299fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 --2- 172.21.15.44:0/528646206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75299fa60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00e5b0 tx=0x7f94fc002130 comp rx=0 tx=0).stop 2024-09-13T08:47:13.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 -- 172.21.15.44:0/528646206 shutdown_connections 2024-09-13T08:47:13.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 --2- 172.21.15.44:0/528646206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292d2f0 0x55c75299fa60 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 --2- 172.21.15.44:0/528646206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c75292c9d0 0x55c75299f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 --2- 172.21.15.44:0/528646206 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c7528e0e10 0x55c75299efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 -- 172.21.15.44:0/528646206 >> 172.21.15.44:0/528646206 conn(0x55c7527ca130 msgr2=0x55c7528e8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 -- 172.21.15.44:0/528646206 shutdown_connections 2024-09-13T08:47:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 -- 172.21.15.44:0/528646206 wait complete. 2024-09-13T08:47:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 -- start start 2024-09-13T08:47:13.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.276+0000 7f950e482000 1 Processor -- start 2024-09-13T08:47:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 0x55c7529a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292c9d0 0x55c7529a3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292d2f0 0x55c7529a3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c7527b2a60 con 0x55c75292c9d0 2024-09-13T08:47:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c7527ed730 con 0x55c7528e0e10 2024-09-13T08:47:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55c75273d8f0 con 0x55c75292d2f0 2024-09-13T08:47:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 10 client.0 _ll_register_callbacks cb 0x7f94fc017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:47:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292c9d0 0x55c7529a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292c9d0 0x55c7529a3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38312/0 (socket says 172.21.15.44:38312) 2024-09-13T08:47:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94fbe00640 1 -- 172.21.15.44:0/645536299 learned_addr learned my addr 172.21.15.44:0/645536299 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f9503e00640 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 0x55c7529a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f9509000640 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292d2f0 0x55c7529a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94fbe00640 1 -- 172.21.15.44:0/645536299 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292d2f0 msgr2=0x55c7529a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:47:13.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94fbe00640 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292d2f0 0x55c7529a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94fbe00640 1 -- 172.21.15.44:0/645536299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 msgr2=0x55c7529a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:47:13.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94fbe00640 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 0x55c7529a2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94fbe00640 1 -- 172.21.15.44:0/645536299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7526f4a90 con 0x55c75292c9d0 2024-09-13T08:47:13.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94fbe00640 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292c9d0 0x55c7529a3140 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0057d0 tx=0x7f94ec0091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c66dc21586124f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:13.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94f1600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:47:13.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94f1600640 1 -- 172.21.15.44:0/645536299 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec0b2040 con 0x55c75292c9d0 2024-09-13T08:47:13.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94f1600640 1 -- 172.21.15.44:0/645536299 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f94ec005220 con 0x55c75292c9d0 2024-09-13T08:47:13.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 1 -- 172.21.15.44:0/645536299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94fc024070 con 0x55c75292c9d0 2024-09-13T08:47:13.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94f1600640 1 -- 172.21.15.44:0/645536299 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f94ec0d5050 con 0x55c75292c9d0 2024-09-13T08:47:13.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 10 client.7050 fetch_fsmap learned FSMap version 56 2024-09-13T08:47:13.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 1 -- 172.21.15.44:0/645536299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55c7526d3050 con 0x55c75292c9d0 2024-09-13T08:47:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94f1600640 1 -- 172.21.15.44:0/645536299 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94fc024070 con 0x55c75292c9d0 2024-09-13T08:47:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f94f1600640 1 -- 172.21.15.44:0/645536299 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7f94ec0d8020 con 0x55c75292c9d0 2024-09-13T08:47:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 10 client.7050 fetch_fsmap finished waiting for FSMap version 56 2024-09-13T08:47:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 10 client.7050 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:47:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 20 client.7050 populate_metadata read hostname 'smithi044' 2024-09-13T08:47:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.44:6834/1796790158,v1:172.21.15.44:6835/1796790158] conn(0x55c752942430 0x55c752941540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 4 client.7050 mds_command: new command op to 6942 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T08:47:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.280+0000 7f950e482000 1 -- 172.21.15.44:0/645536299 --> [v2:172.21.15.44:6834/1796790158,v1:172.21.15.44:6835/1796790158] -- command(tid 0: {"prefix": "status"}) -- 0x55c75273c520 con 0x55c752942430 2024-09-13T08:47:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f9503e00640 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.44:6834/1796790158,v1:172.21.15.44:6835/1796790158] conn(0x55c752942430 0x55c752941540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f9503e00640 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.44:6834/1796790158,v1:172.21.15.44:6835/1796790158] conn(0x55c752942430 0x55c752941540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7cacdcb07f1bb1b server_cookie=9714e6ddde30bcb0 in_seq=0 out_seq=0 2024-09-13T08:47:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f94f1600640 10 client.7050 ms_handle_connect on v2:172.21.15.44:6834/1796790158 2024-09-13T08:47:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f94f1600640 1 -- 172.21.15.44:0/645536299 <== mds.0 v2:172.21.15.44:6834/1796790158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55c75273c520 con 0x55c752942430 2024-09-13T08:47:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f94f1600640 10 client.7050 handle_command_reply: tid=0 2024-09-13T08:47:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 client.7050 shutdown 2024-09-13T08:47:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 -- 172.21.15.44:0/645536299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292c9d0 msgr2=0x55c7529a3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292c9d0 0x55c7529a3140 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0057d0 tx=0x7f94ec0091e0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 -- 172.21.15.44:0/645536299 shutdown_connections 2024-09-13T08:47:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.44:6834/1796790158,v1:172.21.15.44:6835/1796790158] conn(0x55c752942430 0x55c752941540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c75292d2f0 0x55c7529a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c75292c9d0 0x55c7529a3140 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 --2- 172.21.15.44:0/645536299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7528e0e10 0x55c7529a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 -- 172.21.15.44:0/645536299 >> 172.21.15.44:0/645536299 conn(0x55c7527ca130 msgr2=0x55c75292b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f94f1600640 0 client.7050 ms_handle_reset on v2:172.21.15.44:6834/1796790158 2024-09-13T08:47:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 -- 172.21.15.44:0/645536299 shutdown_connections 2024-09-13T08:47:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 1 -- 172.21.15.44:0/645536299 wait complete. 2024-09-13T08:47:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950e482000 20 client.7050 trim_cache size 0 max 16384 2024-09-13T08:47:13.298 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:47:13.298 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:47:13.298 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T08:47:13.298 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 6942, 2024-09-13T08:47:13.298 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T08:47:13.299 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T08:47:13.299 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T08:47:13.299 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.0318561119999998, 2024-09-13T08:47:13.299 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 56, 2024-09-13T08:47:13.299 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 89, 2024-09-13T08:47:13.300 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 89, 2024-09-13T08:47:13.300 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 10.482641299999999 2024-09-13T08:47:13.300 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:47:13.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 -- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0074030 msgr2=0x7f94e00764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0074030 0x7f94e00764f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0af000 tx=0x7f94ec01b030 comp rx=0 tx=0).stop 2024-09-13T08:47:13.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 -- 172.21.15.44:0/3143208700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f950406cc20 msgr2=0x7f95041f88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f950406cc20 0x7f95041f88d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f94f400e9d0 tx=0x7f94f400ee90 comp rx=0 tx=0).stop 2024-09-13T08:47:13.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 -- 172.21.15.44:0/3143208700 shutdown_connections 2024-09-13T08:47:13.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f94e0074030 0x7f94e00764f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f950415f7b0 0x7f95040b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f950415c1c0 0x7f95040b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 --2- 172.21.15.44:0/3143208700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f950406cc20 0x7f95041f88d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 -- 172.21.15.44:0/3143208700 >> 172.21.15.44:0/3143208700 conn(0x7f95040d0c90 msgr2=0x7f95041557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 -- 172.21.15.44:0/3143208700 shutdown_connections 2024-09-13T08:47:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.284+0000 7f950b000640 1 -- 172.21.15.44:0/3143208700 wait complete. 2024-09-13T08:47:13.306 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:47:13.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.452+0000 7ff352000640 1 -- 172.21.15.44:0/3974516924 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34c15bea0 msgr2=0x7ff34c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.452+0000 7ff352000640 1 --2- 172.21.15.44:0/3974516924 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34c15bea0 0x7ff34c0ce670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009900 tx=0x7ff33c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.452+0000 7ff352000640 1 -- 172.21.15.44:0/3974516924 shutdown_connections 2024-09-13T08:47:13.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.452+0000 7ff352000640 1 --2- 172.21.15.44:0/3974516924 >> v2:172.21.15.129:3300/0 conn(0x7ff34c0d28e0 0x7ff34c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.452+0000 7ff352000640 1 --2- 172.21.15.44:0/3974516924 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34c15bea0 0x7ff34c0ce670 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.452+0000 7ff352000640 1 -- 172.21.15.44:0/3974516924 >> 172.21.15.44:0/3974516924 conn(0x7ff34c0d1dc0 msgr2=0x7ff34c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:13.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.452+0000 7ff352000640 1 -- 172.21.15.44:0/3974516924 shutdown_connections 2024-09-13T08:47:13.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.452+0000 7ff352000640 1 -- 172.21.15.44:0/3974516924 wait complete. 2024-09-13T08:47:13.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff352000640 1 Processor -- start 2024-09-13T08:47:13.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff352000640 1 -- start start 2024-09-13T08:47:13.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff352000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34c0cdb30 0x7ff34c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff352000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34c0d28e0 0x7ff34c156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff352000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff34c15bea0 0x7ff34c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff352000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c168fc0 con 0x7ff34c0d28e0 2024-09-13T08:47:13.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff352000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c168e40 con 0x7ff34c15bea0 2024-09-13T08:47:13.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff352000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c169140 con 0x7ff34c0cdb30 2024-09-13T08:47:13.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34c0d28e0 0x7ff34c156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34c0d28e0 0x7ff34c156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38328/0 (socket says 172.21.15.44:38328) 2024-09-13T08:47:13.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343e00640 1 -- 172.21.15.44:0/3520574246 learned_addr learned my addr 172.21.15.44:0/3520574246 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:13.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff34b400640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34c0cdb30 0x7ff34c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff34be00640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff34c15bea0 0x7ff34c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343e00640 1 -- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34c0cdb30 msgr2=0x7ff34c1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343e00640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34c0cdb30 0x7ff34c1573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343e00640 1 -- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff34c15bea0 msgr2=0x7ff34c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343e00640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff34c15bea0 0x7ff34c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343e00640 1 -- 172.21.15.44:0/3520574246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c0cb340 con 0x7ff34c0d28e0 2024-09-13T08:47:13.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343e00640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34c0d28e0 0x7ff34c156c20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c500 tx=0x7ff33400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:13.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343400640 1 -- 172.21.15.44:0/3520574246 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334017070 con 0x7ff34c0d28e0 2024-09-13T08:47:13.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343400640 1 -- 172.21.15.44:0/3520574246 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3340053c0 con 0x7ff34c0d28e0 2024-09-13T08:47:13.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff343400640 1 -- 172.21.15.44:0/3520574246 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334012890 con 0x7ff34c0d28e0 2024-09-13T08:47:13.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff352000640 1 -- 172.21.15.44:0/3520574246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c0cb620 con 0x7ff34c0d28e0 2024-09-13T08:47:13.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.456+0000 7ff352000640 1 -- 172.21.15.44:0/3520574246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c157160 con 0x7ff34c0d28e0 2024-09-13T08:47:13.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.460+0000 7ff352000640 1 -- 172.21.15.44:0/3520574246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c0cc9d0 con 0x7ff34c0d28e0 2024-09-13T08:47:13.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.460+0000 7ff343400640 1 -- 172.21.15.44:0/3520574246 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7ff33401a070 con 0x7ff34c0d28e0 2024-09-13T08:47:13.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.460+0000 7ff343400640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff328074060 0x7ff328076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:13.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.460+0000 7ff343400640 1 -- 172.21.15.44:0/3520574246 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7ff334091f60 con 0x7ff34c0d28e0 2024-09-13T08:47:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.460+0000 7ff34b400640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff328074060 0x7ff328076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.460+0000 7ff34b400640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff328074060 0x7ff328076520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0af000 tx=0x7ff33c01b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:13.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.464+0000 7ff343400640 1 -- 172.21.15.44:0/3520574246 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff33405e240 con 0x7ff34c0d28e0 2024-09-13T08:47:13.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.676+0000 7ff352000640 1 -- 172.21.15.44:0/3520574246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff34c0008e0 con 0x7ff34c0d28e0 2024-09-13T08:47:13.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.676+0000 7ff343400640 1 -- 172.21.15.44:0/3520574246 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7ff334062c70 con 0x7ff34c0d28e0 2024-09-13T08:47:13.680 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 -- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff328074060 msgr2=0x7ff328076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff328074060 0x7ff328076520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0af000 tx=0x7ff33c01b120 comp rx=0 tx=0).stop 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 -- 172.21.15.44:0/3520574246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34c0d28e0 msgr2=0x7ff34c156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34c0d28e0 0x7ff34c156c20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c500 tx=0x7ff33400c9c0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 -- 172.21.15.44:0/3520574246 shutdown_connections 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff328074060 0x7ff328076520 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff34c15bea0 0x7ff34c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff34c0d28e0 0x7ff34c156c20 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 --2- 172.21.15.44:0/3520574246 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff34c0cdb30 0x7ff34c1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 -- 172.21.15.44:0/3520574246 >> 172.21.15.44:0/3520574246 conn(0x7ff34c0d1dc0 msgr2=0x7ff34c159b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:13.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 -- 172.21.15.44:0/3520574246 shutdown_connections 2024-09-13T08:47:13.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13.680+0000 7ff352000640 1 -- 172.21.15.44:0/3520574246 wait complete. 2024-09-13T08:47:13.701 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T08:47:13.701 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:47:13.701 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:47:13.701 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T08:47:13.701 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:47:13.701 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T08:47:13.701 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T08:47:13.701 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:47:13.701 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:47:13.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:13.702 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:47:13.710 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:47:13.711 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:47:13.712 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41849sec preferred_lft 41849sec 2024-09-13T08:47:13.712 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:47:13.712 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:47:13.712 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:47:13.712 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:47:13.712 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:47:13.712 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:47:13.712 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:47:13.712 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:47:13.712 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:47:13.713 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:47:13.714 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:47:13.714 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T08:47:13.714 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:47:13.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:13.715 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:47:13.774 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:47:13.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:13.776 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:47:13.829 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:47:13.830 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:47:13.830 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:47:13.830 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:47:13.830 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:47:13.830 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:47:13.831 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:47:13.831 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:47:13.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:47:13.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:47:13.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:47:13.831 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:47:13.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:47:13.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:47:13.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:13.968 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:47:13.978 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:47:13.978 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T08:47:13.978 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:47:13.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:13.978 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T08:47:14.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:47:14.040 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T08:47:14.092 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:47:14.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:47:14.092 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:47:14.147 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:47:14.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:14.148 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:47:14.242 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:47:14.242 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:47:14.242 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:47:14.258 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:47:14.258 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:47:14.259 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:47:14.259 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:47:14.259 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:47:14.267 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:47:14.267 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:47:14.267 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:47:14.267 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:47:14.268 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:47:14.268 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:47:14.268 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:47:14.268 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:47:14.271 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:47:14.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:14.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:47:14.293 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:47:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:47:14.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:47:14.400 INFO:teuthology.orchestra.run.smithi044.stdout:client7056 2024-09-13T08:47:14.401 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:47:14.401 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:47:14.410 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-13T08:47:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.600+0000 7f16f6e00640 1 -- 172.21.15.44:0/2058007947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16d4004d50 msgr2=0x7f16d40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.600+0000 7f16f6e00640 1 --2- 172.21.15.44:0/2058007947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16d4004d50 0x7f16d40051d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009ee0 tx=0x7f16e801d5f0 comp rx=0 tx=0).stop 2024-09-13T08:47:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.600+0000 7f16f4e00640 1 -- 172.21.15.44:0/2058007947 reap_dead start 2024-09-13T08:47:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.600+0000 7f16f6e00640 1 -- 172.21.15.44:0/2058007947 shutdown_connections 2024-09-13T08:47:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.600+0000 7f16f6e00640 1 -- 172.21.15.44:0/2058007947 >> 172.21.15.44:0/2058007947 conn(0x7f16f00d1da0 msgr2=0x7f16f00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.600+0000 7f16f6e00640 1 -- 172.21.15.44:0/2058007947 shutdown_connections 2024-09-13T08:47:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.600+0000 7f16f6e00640 1 -- 172.21.15.44:0/2058007947 wait complete. 2024-09-13T08:47:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 Processor -- start 2024-09-13T08:47:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 -- start start 2024-09-13T08:47:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16d4004d50 0x7f16f00c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16f00c58d0 0x7f16f0153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16f00aea30 0x7f16f00aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16f00ccd50 con 0x7f16f00aea30 2024-09-13T08:47:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f16f00ccbd0 con 0x7f16d4004d50 2024-09-13T08:47:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f16f00cced0 con 0x7f16f00c58d0 2024-09-13T08:47:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16f00aea30 0x7f16f00aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16f00aea30 0x7f16f00aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38338/0 (socket says 172.21.15.44:38338) 2024-09-13T08:47:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f4e00640 1 -- 172.21.15.44:0/1253350078 learned_addr learned my addr 172.21.15.44:0/1253350078 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16e7400640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16f00c58d0 0x7f16f0153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16efe00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16d4004d50 0x7f16f00c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f4e00640 1 -- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16f00c58d0 msgr2=0x7f16f0153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f4e00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16f00c58d0 0x7f16f0153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f4e00640 1 -- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16d4004d50 msgr2=0x7f16f00c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f4e00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16d4004d50 0x7f16f00c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f4e00640 1 -- 172.21.15.44:0/1253350078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f0154080 con 0x7f16f00aea30 2024-09-13T08:47:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f4e00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16f00aea30 0x7f16f00aeeb0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f16e801d060 tx=0x7f16e8004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16ecc00640 1 -- 172.21.15.44:0/1253350078 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e80ad050 con 0x7f16f00aea30 2024-09-13T08:47:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16ecc00640 1 -- 172.21.15.44:0/1253350078 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16e80a7a90 con 0x7f16f00aea30 2024-09-13T08:47:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16ecc00640 1 -- 172.21.15.44:0/1253350078 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8009ab0 con 0x7f16f00aea30 2024-09-13T08:47:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 -- 172.21.15.44:0/1253350078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f01f62d0 con 0x7f16f00aea30 2024-09-13T08:47:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 -- 172.21.15.44:0/1253350078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f01f6890 con 0x7f16f00aea30 2024-09-13T08:47:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.604+0000 7f16f6e00640 1 -- 172.21.15.44:0/1253350078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f00af3f0 con 0x7f16f00aea30 2024-09-13T08:47:14.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.608+0000 7f16ecc00640 1 -- 172.21.15.44:0/1253350078 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f16e80b0070 con 0x7f16f00aea30 2024-09-13T08:47:14.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.608+0000 7f16ecc00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16cc074030 0x7f16cc0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:14.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.608+0000 7f16ecc00640 1 -- 172.21.15.44:0/1253350078 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f16e812d810 con 0x7f16f00aea30 2024-09-13T08:47:14.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.608+0000 7f16efe00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16cc074030 0x7f16cc0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:14.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.608+0000 7f16efe00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16cc074030 0x7f16cc0764f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f16e0000fc0 tx=0x7f16e0006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:14.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.612+0000 7f16ecc00640 1 -- 172.21.15.44:0/1253350078 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f16e8017300 con 0x7f16f00aea30 2024-09-13T08:47:14.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:14.824+0000 7f16f6e00640 1 -- 172.21.15.44:0/1253350078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f16f00d2a70 con 0x7f16f00aea30 2024-09-13T08:47:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16ecc00640 1 -- 172.21.15.44:0/1253350078 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v57) ==== 136+0+0 (secure 0 0 0) 0x7f16e80f9b10 con 0x7f16f00aea30 2024-09-13T08:47:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 -- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16cc074030 msgr2=0x7f16cc0764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16cc074030 0x7f16cc0764f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f16e0000fc0 tx=0x7f16e0006040 comp rx=0 tx=0).stop 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 -- 172.21.15.44:0/1253350078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16f00aea30 msgr2=0x7f16f00aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16f00aea30 0x7f16f00aeeb0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f16e801d060 tx=0x7f16e8004f50 comp rx=0 tx=0).stop 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 -- 172.21.15.44:0/1253350078 shutdown_connections 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16cc074030 0x7f16cc0764f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16f00aea30 0x7f16f00aeeb0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16f00c58d0 0x7f16f0153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 --2- 172.21.15.44:0/1253350078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16d4004d50 0x7f16f00c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 -- 172.21.15.44:0/1253350078 >> 172.21.15.44:0/1253350078 conn(0x7f16f00d1da0 msgr2=0x7f16f0070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 -- 172.21.15.44:0/1253350078 shutdown_connections 2024-09-13T08:47:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.292+0000 7f16f6e00640 1 -- 172.21.15.44:0/1253350078 wait complete. 2024-09-13T08:47:15.318 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:47:15.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.468+0000 7f06fce00640 1 -- 172.21.15.44:0/2129675701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06dc004810 msgr2=0x7f06dc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:15.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.468+0000 7f06fce00640 1 --2- 172.21.15.44:0/2129675701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06dc004810 0x7f06dc004c90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009b60 tx=0x7f06e001d270 comp rx=0 tx=0).stop 2024-09-13T08:47:15.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.468+0000 7f06fce00640 1 -- 172.21.15.44:0/2129675701 shutdown_connections 2024-09-13T08:47:15.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.468+0000 7f06fce00640 1 --2- 172.21.15.44:0/2129675701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06dc004810 0x7f06dc004c90 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.468+0000 7f06fce00640 1 -- 172.21.15.44:0/2129675701 >> 172.21.15.44:0/2129675701 conn(0x7f06f80d1dc0 msgr2=0x7f06f80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:15.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.468+0000 7f06fce00640 1 -- 172.21.15.44:0/2129675701 shutdown_connections 2024-09-13T08:47:15.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.468+0000 7f06fce00640 1 -- 172.21.15.44:0/2129675701 wait complete. 2024-09-13T08:47:15.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.468+0000 7f06fce00640 1 Processor -- start 2024-09-13T08:47:15.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.468+0000 7f06fce00640 1 -- start start 2024-09-13T08:47:15.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06fce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06dc004810 0x7f06f81f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:15.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06fce00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f815ecd0 0x7f06f80719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:15.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06fce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f8071f10 0x7f06f806ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06fce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06f815c490 con 0x7f06f8071f10 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06fce00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06f80d2af0 con 0x7f06dc004810 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06fce00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f06f80ce670 con 0x7f06f815ecd0 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06f6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f8071f10 0x7f06f806ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06f6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f8071f10 0x7f06f806ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38350/0 (socket says 172.21.15.44:38350) 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06f6800640 1 -- 172.21.15.44:0/3632267415 learned_addr learned my addr 172.21.15.44:0/3632267415 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06f5e00640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06dc004810 0x7f06f81f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06efe00640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f815ecd0 0x7f06f80719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06f6800640 1 -- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f815ecd0 msgr2=0x7f06f80719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06f6800640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f815ecd0 0x7f06f80719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06f6800640 1 -- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06dc004810 msgr2=0x7f06f81f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06f6800640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06dc004810 0x7f06f81f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06f6800640 1 -- 172.21.15.44:0/3632267415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f806eb60 con 0x7f06f8071f10 2024-09-13T08:47:15.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06f6800640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f8071f10 0x7f06f806ddc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f06f80d0910 tx=0x7f06f00019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:15.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06ee000640 1 -- 172.21.15.44:0/3632267415 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0015020 con 0x7f06f8071f10 2024-09-13T08:47:15.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06ee000640 1 -- 172.21.15.44:0/3632267415 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06f0011960 con 0x7f06f8071f10 2024-09-13T08:47:15.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06ee000640 1 -- 172.21.15.44:0/3632267415 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0011db0 con 0x7f06f8071f10 2024-09-13T08:47:15.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06fce00640 1 -- 172.21.15.44:0/3632267415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f815df10 con 0x7f06f8071f10 2024-09-13T08:47:15.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06fce00640 1 -- 172.21.15.44:0/3632267415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f815e4d0 con 0x7f06f8071f10 2024-09-13T08:47:15.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.472+0000 7f06fce00640 1 -- 172.21.15.44:0/3632267415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f8072390 con 0x7f06f8071f10 2024-09-13T08:47:15.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.476+0000 7f06ee000640 1 -- 172.21.15.44:0/3632267415 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f06f0018020 con 0x7f06f8071f10 2024-09-13T08:47:15.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.476+0000 7f06ee000640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06d4074060 0x7f06d4076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:15.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.476+0000 7f06ee000640 1 -- 172.21.15.44:0/3632267415 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f06f00937b0 con 0x7f06f8071f10 2024-09-13T08:47:15.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.476+0000 7f06f5e00640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06d4074060 0x7f06d4076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:15.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.476+0000 7f06f5e00640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06d4074060 0x7f06d4076520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f06e8000fc0 tx=0x7f06e80067b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.480+0000 7f06ee000640 1 -- 172.21.15.44:0/3632267415 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f06f001f020 con 0x7f06f8071f10 2024-09-13T08:47:15.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.680+0000 7f06fce00640 1 -- 172.21.15.44:0/3632267415 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f06f80cc9d0 con 0x7f06d4074060 2024-09-13T08:47:15.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.680+0000 7f06ee000640 1 -- 172.21.15.44:0/3632267415 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f06f80cc9d0 con 0x7f06d4074060 2024-09-13T08:47:15.683 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:47:15.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:15.683 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:47:15.683 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:15.683 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 -- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06d4074060 msgr2=0x7f06d4076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06d4074060 0x7f06d4076520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f06e8000fc0 tx=0x7f06e80067b0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 -- 172.21.15.44:0/3632267415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f8071f10 msgr2=0x7f06f806ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f8071f10 0x7f06f806ddc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f06f80d0910 tx=0x7f06f00019c0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 -- 172.21.15.44:0/3632267415 shutdown_connections 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f06d4074060 0x7f06d4076520 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f8071f10 0x7f06f806ddc0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06f815ecd0 0x7f06f80719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 --2- 172.21.15.44:0/3632267415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06dc004810 0x7f06f81f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 -- 172.21.15.44:0/3632267415 >> 172.21.15.44:0/3632267415 conn(0x7f06f80d1dc0 msgr2=0x7f06f80af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:15.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 -- 172.21.15.44:0/3632267415 shutdown_connections 2024-09-13T08:47:15.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.684+0000 7f06fce00640 1 -- 172.21.15.44:0/3632267415 wait complete. 2024-09-13T08:47:15.706 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:47:15.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.852+0000 7fdbf9400640 1 -- 172.21.15.44:0/2997509874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbf41605d0 msgr2=0x7fdbf40d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:15.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.852+0000 7fdbf9400640 1 --2- 172.21.15.44:0/2997509874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbf41605d0 0x7fdbf40d0690 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009900 tx=0x7fdbe401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.852+0000 7fdbf9400640 1 -- 172.21.15.44:0/2997509874 shutdown_connections 2024-09-13T08:47:15.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.852+0000 7fdbf9400640 1 --2- 172.21.15.44:0/2997509874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbf41605d0 0x7fdbf40d0690 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.852+0000 7fdbf9400640 1 -- 172.21.15.44:0/2997509874 >> 172.21.15.44:0/2997509874 conn(0x7fdbf40d1dd0 msgr2=0x7fdbf40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:15.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.852+0000 7fdbf9400640 1 -- 172.21.15.44:0/2997509874 shutdown_connections 2024-09-13T08:47:15.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.852+0000 7fdbf9400640 1 -- 172.21.15.44:0/2997509874 wait complete. 2024-09-13T08:47:15.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf9400640 1 Processor -- start 2024-09-13T08:47:15.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf9400640 1 -- start start 2024-09-13T08:47:15.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf9400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbf40afb10 0x7fdbf415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:15.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbf40b1650 0x7fdbf415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:15.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbf41605d0 0x7fdbf406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:15.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf9400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf40b13a0 con 0x7fdbf41605d0 2024-09-13T08:47:15.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf9400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf40b1220 con 0x7fdbf40b1650 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf9400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf416a550 con 0x7fdbf40afb10 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbf41605d0 0x7fdbf406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbf41605d0 0x7fdbf406dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38362/0 (socket says 172.21.15.44:38362) 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf2800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbf40afb10 0x7fdbf415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf3200640 1 -- 172.21.15.44:0/3420794400 learned_addr learned my addr 172.21.15.44:0/3420794400 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbebe00640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbf40b1650 0x7fdbf415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf3200640 1 -- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbf40afb10 msgr2=0x7fdbf415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf3200640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbf40afb10 0x7fdbf415bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf3200640 1 -- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbf40b1650 msgr2=0x7fdbf415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf3200640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbf40b1650 0x7fdbf415c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf3200640 1 -- 172.21.15.44:0/3420794400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf4167120 con 0x7fdbf41605d0 2024-09-13T08:47:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf3200640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbf41605d0 0x7fdbf406dd30 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c7f0 tx=0x7fdbec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbeaa00640 1 -- 172.21.15.44:0/3420794400 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec003b80 con 0x7fdbf41605d0 2024-09-13T08:47:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbeaa00640 1 -- 172.21.15.44:0/3420794400 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbec003d20 con 0x7fdbf41605d0 2024-09-13T08:47:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbeaa00640 1 -- 172.21.15.44:0/3420794400 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00aab0 con 0x7fdbf41605d0 2024-09-13T08:47:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.856+0000 7fdbf9400640 1 -- 172.21.15.44:0/3420794400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf4166080 con 0x7fdbf41605d0 2024-09-13T08:47:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.860+0000 7fdbf9400640 1 -- 172.21.15.44:0/3420794400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf4166640 con 0x7fdbf41605d0 2024-09-13T08:47:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.860+0000 7fdbf9400640 1 -- 172.21.15.44:0/3420794400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf40d0690 con 0x7fdbf41605d0 2024-09-13T08:47:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.860+0000 7fdbeaa00640 1 -- 172.21.15.44:0/3420794400 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7fdbec00ac50 con 0x7fdbf41605d0 2024-09-13T08:47:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.860+0000 7fdbeaa00640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbd0074060 0x7fdbd0076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.860+0000 7fdbeaa00640 1 -- 172.21.15.44:0/3420794400 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fdbec092a00 con 0x7fdbf41605d0 2024-09-13T08:47:15.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.860+0000 7fdbf2800640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbd0074060 0x7fdbd0076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:15.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.864+0000 7fdbf2800640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbd0074060 0x7fdbd0076520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40af000 tx=0x7fdbe401b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:15.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:15.864+0000 7fdbeaa00640 1 -- 172.21.15.44:0/3420794400 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdbf4167120 con 0x7fdbf41605d0 2024-09-13T08:47:16.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 -- 172.21.15.44:0/3420794400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdbf40d1080 con 0x7fdbf41605d0 2024-09-13T08:47:16.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbeaa00640 1 -- 172.21.15.44:0/3420794400 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7fdbec05ed00 con 0x7fdbf41605d0 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 -- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbd0074060 msgr2=0x7fdbd0076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbd0074060 0x7fdbd0076520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40af000 tx=0x7fdbe401b120 comp rx=0 tx=0).stop 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 -- 172.21.15.44:0/3420794400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbf41605d0 msgr2=0x7fdbf406dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbf41605d0 0x7fdbf406dd30 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c7f0 tx=0x7fdbec00ccb0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 -- 172.21.15.44:0/3420794400 shutdown_connections 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbd0074060 0x7fdbd0076520 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbf41605d0 0x7fdbf406dd30 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbf40b1650 0x7fdbf415c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 --2- 172.21.15.44:0/3420794400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbf40afb10 0x7fdbf415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 -- 172.21.15.44:0/3420794400 >> 172.21.15.44:0/3420794400 conn(0x7fdbf40d1dd0 msgr2=0x7fdbf40721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 -- 172.21.15.44:0/3420794400 shutdown_connections 2024-09-13T08:47:16.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.040+0000 7fdbf9400640 1 -- 172.21.15.44:0/3420794400 wait complete. 2024-09-13T08:47:16.064 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-13T08:47:16.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 -- 172.21.15.44:0/1627927310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33980ce980 msgr2=0x7f33980cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 --2- 172.21.15.44:0/1627927310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33980ce980 0x7f33980cf9c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3390009bc0 tx=0x7f339001cb70 comp rx=0 tx=0).stop 2024-09-13T08:47:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 -- 172.21.15.44:0/1627927310 shutdown_connections 2024-09-13T08:47:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 --2- 172.21.15.44:0/1627927310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33980ce980 0x7f33980cf9c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 -- 172.21.15.44:0/1627927310 >> 172.21.15.44:0/1627927310 conn(0x7f33980d1df0 msgr2=0x7f33980d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 -- 172.21.15.44:0/1627927310 shutdown_connections 2024-09-13T08:47:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 -- 172.21.15.44:0/1627927310 wait complete. 2024-09-13T08:47:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 Processor -- start 2024-09-13T08:47:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 -- start start 2024-09-13T08:47:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33980b1190 0x7f339815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33980ce980 0x7f33980b6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33981603f0 0x7f33980b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33980b34a0 con 0x7f33980b1190 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3398160910 con 0x7f33981603f0 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ee00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f339816a500 con 0x7f33980ce980 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f3397e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33980b1190 0x7f339815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f3397e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33980b1190 0x7f339815ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38386/0 (socket says 172.21.15.44:38386) 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f3397e00640 1 -- 172.21.15.44:0/3923535125 learned_addr learned my addr 172.21.15.44:0/3923535125 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ce00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33981603f0 0x7f33980b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f338fe00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33980ce980 0x7f33980b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f3397e00640 1 -- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33980ce980 msgr2=0x7f33980b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f3397e00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33980ce980 0x7f33980b6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f3397e00640 1 -- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33981603f0 msgr2=0x7f33980b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f3397e00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33981603f0 0x7f33980b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f3397e00640 1 -- 172.21.15.44:0/3923535125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339815c900 con 0x7f33980b1190 2024-09-13T08:47:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f339ce00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33981603f0 0x7f33980b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:47:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.212+0000 7f3397e00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33980b1190 0x7f339815ba60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f338800c3f0 tx=0x7f338800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.216+0000 7f3394c00640 1 -- 172.21.15.44:0/3923535125 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388017070 con 0x7f33980b1190 2024-09-13T08:47:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.216+0000 7f3394c00640 1 -- 172.21.15.44:0/3923535125 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33880053c0 con 0x7f33980b1190 2024-09-13T08:47:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.216+0000 7f339ee00640 1 -- 172.21.15.44:0/3923535125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33980b6e30 con 0x7f33980b1190 2024-09-13T08:47:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.216+0000 7f3394c00640 1 -- 172.21.15.44:0/3923535125 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388003860 con 0x7f33980b1190 2024-09-13T08:47:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.216+0000 7f339ee00640 1 -- 172.21.15.44:0/3923535125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33980b4dc0 con 0x7f33980b1190 2024-09-13T08:47:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.216+0000 7f339ee00640 1 -- 172.21.15.44:0/3923535125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33980b7350 con 0x7f33980b1190 2024-09-13T08:47:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.216+0000 7f3394c00640 1 -- 172.21.15.44:0/3923535125 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f338801a070 con 0x7f33980b1190 2024-09-13T08:47:16.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.216+0000 7f3394c00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3374074030 0x7f33740764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.216+0000 7f3394c00640 1 -- 172.21.15.44:0/3923535125 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f3388092040 con 0x7f33980b1190 2024-09-13T08:47:16.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.216+0000 7f338fe00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3374074030 0x7f33740764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.220+0000 7f338fe00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3374074030 0x7f33740764f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f339815caf0 tx=0x7f3380009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:16.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.220+0000 7f3394c00640 1 -- 172.21.15.44:0/3923535125 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f33980b7350 con 0x7f33980b1190 2024-09-13T08:47:16.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.396+0000 7f339ee00640 1 -- 172.21.15.44:0/3923535125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f33980cf9c0 con 0x7f33980b1190 2024-09-13T08:47:16.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.396+0000 7f3394c00640 1 -- 172.21.15.44:0/3923535125 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v41) ==== 116+0+2 (secure 0 0 0) 0x7f338805e990 con 0x7f33980b1190 2024-09-13T08:47:16.399 INFO:teuthology.orchestra.run.smithi044.stdout:4 2024-09-13T08:47:16.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 -- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3374074030 msgr2=0x7f33740764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3374074030 0x7f33740764f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f339815caf0 tx=0x7f3380009040 comp rx=0 tx=0).stop 2024-09-13T08:47:16.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 -- 172.21.15.44:0/3923535125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33980b1190 msgr2=0x7f339815ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33980b1190 0x7f339815ba60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f338800c3f0 tx=0x7f338800c8b0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 -- 172.21.15.44:0/3923535125 shutdown_connections 2024-09-13T08:47:16.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3374074030 0x7f33740764f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33981603f0 0x7f33980b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33980ce980 0x7f33980b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 --2- 172.21.15.44:0/3923535125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33980b1190 0x7f339815ba60 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 -- 172.21.15.44:0/3923535125 >> 172.21.15.44:0/3923535125 conn(0x7f33980d1df0 msgr2=0x7f339806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:16.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 -- 172.21.15.44:0/3923535125 shutdown_connections 2024-09-13T08:47:16.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.400+0000 7f339ee00640 1 -- 172.21.15.44:0/3923535125 wait complete. 2024-09-13T08:47:16.421 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:47:16.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.568+0000 7f4550600640 1 -- 172.21.15.44:0/2919143049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45481605d0 msgr2=0x7f45480d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.568+0000 7f4550600640 1 --2- 172.21.15.44:0/2919143049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45481605d0 0x7f45480d0690 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f453c009900 tx=0x7f453c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.568+0000 7f4550600640 1 -- 172.21.15.44:0/2919143049 shutdown_connections 2024-09-13T08:47:16.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.568+0000 7f4550600640 1 --2- 172.21.15.44:0/2919143049 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f45480b1650 0x7f45480b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.568+0000 7f4550600640 1 --2- 172.21.15.44:0/2919143049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45481605d0 0x7f45480d0690 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.568+0000 7f4550600640 1 -- 172.21.15.44:0/2919143049 >> 172.21.15.44:0/2919143049 conn(0x7f45480d1dd0 msgr2=0x7f45480d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:16.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.568+0000 7f4550600640 1 -- 172.21.15.44:0/2919143049 shutdown_connections 2024-09-13T08:47:16.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.568+0000 7f4550600640 1 -- 172.21.15.44:0/2919143049 wait complete. 2024-09-13T08:47:16.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4550600640 1 Processor -- start 2024-09-13T08:47:16.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4550600640 1 -- start start 2024-09-13T08:47:16.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4550600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f45480aeb00 0x7f45481566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4550600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45480b1650 0x7f4548154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4550600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45481605d0 0x7f4548152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4550600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f45480b0810 con 0x7f45480b1650 2024-09-13T08:47:16.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4550600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45480b0ee0 con 0x7f45481605d0 2024-09-13T08:47:16.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4550600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f45480b1220 con 0x7f45480aeb00 2024-09-13T08:47:16.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45480b1650 0x7f4548154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45480b1650 0x7f4548154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38400/0 (socket says 172.21.15.44:38400) 2024-09-13T08:47:16.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454d200640 1 -- 172.21.15.44:0/3257697145 learned_addr learned my addr 172.21.15.44:0/3257697145 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:16.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454dc00640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f45480aeb00 0x7f45481566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454e600640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45481605d0 0x7f4548152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454d200640 1 -- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f45480aeb00 msgr2=0x7f45481566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454d200640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f45480aeb00 0x7f45481566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454d200640 1 -- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45481605d0 msgr2=0x7f4548152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454d200640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45481605d0 0x7f4548152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454d200640 1 -- 172.21.15.44:0/3257697145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4548155f90 con 0x7f45480b1650 2024-09-13T08:47:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f454d200640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45480b1650 0x7f4548154430 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f453400ea10 tx=0x7f453400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4545600640 1 -- 172.21.15.44:0/3257697145 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453400cde0 con 0x7f45480b1650 2024-09-13T08:47:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4545600640 1 -- 172.21.15.44:0/3257697145 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4534004020 con 0x7f45480b1650 2024-09-13T08:47:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4550600640 1 -- 172.21.15.44:0/3257697145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4548154ef0 con 0x7f45480b1650 2024-09-13T08:47:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4545600640 1 -- 172.21.15.44:0/3257697145 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534004470 con 0x7f45480b1650 2024-09-13T08:47:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.572+0000 7f4550600640 1 -- 172.21.15.44:0/3257697145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45481554b0 con 0x7f45480b1650 2024-09-13T08:47:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.576+0000 7f4550600640 1 -- 172.21.15.44:0/3257697145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45480b1a30 con 0x7f45480b1650 2024-09-13T08:47:16.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.576+0000 7f4545600640 1 -- 172.21.15.44:0/3257697145 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f4534003600 con 0x7f45480b1650 2024-09-13T08:47:16.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.576+0000 7f4545600640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4528074030 0x7f45280764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.576+0000 7f4545600640 1 -- 172.21.15.44:0/3257697145 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f4534093690 con 0x7f45480b1650 2024-09-13T08:47:16.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.576+0000 7f454dc00640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4528074030 0x7f45280764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.576+0000 7f454dc00640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4528074030 0x7f45280764f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f453c0af000 tx=0x7f453c01b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:16.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.580+0000 7f4545600640 1 -- 172.21.15.44:0/3257697145 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f453405ffe0 con 0x7f45480b1650 2024-09-13T08:47:16.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.756+0000 7f4550600640 1 -- 172.21.15.44:0/3257697145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f4548158a50 con 0x7f45480b1650 2024-09-13T08:47:16.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.760+0000 7f4545600640 1 -- 172.21.15.44:0/3257697145 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v42)=0 v42) ==== 131+0+0 (secure 0 0 0) 0x7f453405f8e0 con 0x7f45480b1650 2024-09-13T08:47:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 -- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4528074030 msgr2=0x7f45280764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4528074030 0x7f45280764f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f453c0af000 tx=0x7f453c01b120 comp rx=0 tx=0).stop 2024-09-13T08:47:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 -- 172.21.15.44:0/3257697145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45480b1650 msgr2=0x7f4548154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45480b1650 0x7f4548154430 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f453400ea10 tx=0x7f453400eed0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 -- 172.21.15.44:0/3257697145 shutdown_connections 2024-09-13T08:47:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4528074030 0x7f45280764f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45481605d0 0x7f4548152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45480b1650 0x7f4548154430 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 --2- 172.21.15.44:0/3257697145 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f45480aeb00 0x7f45481566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 -- 172.21.15.44:0/3257697145 >> 172.21.15.44:0/3257697145 conn(0x7f45480d1dd0 msgr2=0x7f45480b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:16.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 -- 172.21.15.44:0/3257697145 shutdown_connections 2024-09-13T08:47:16.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.764+0000 7f4550600640 1 -- 172.21.15.44:0/3257697145 wait complete. 2024-09-13T08:47:16.783 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:16.783+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:16.783 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:16.783+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:16.783 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:16.783+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:16.783 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:16.783+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:16.783 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:16.783+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:16.785 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-13T08:47:16.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.932+0000 7f59cdc00640 1 -- 172.21.15.44:0/3137427748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c80b3190 msgr2=0x7f59c8168f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.932+0000 7f59cdc00640 1 --2- 172.21.15.44:0/3137427748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c80b3190 0x7f59c8168f20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009c80 tx=0x7f59c001cc50 comp rx=0 tx=0).stop 2024-09-13T08:47:16.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.932+0000 7f59cdc00640 1 -- 172.21.15.44:0/3137427748 shutdown_connections 2024-09-13T08:47:16.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.932+0000 7f59cdc00640 1 --2- 172.21.15.44:0/3137427748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c80b3190 0x7f59c8168f20 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.932+0000 7f59cdc00640 1 -- 172.21.15.44:0/3137427748 >> 172.21.15.44:0/3137427748 conn(0x7f59c80d1dd0 msgr2=0x7f59c80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:16.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.932+0000 7f59cdc00640 1 -- 172.21.15.44:0/3137427748 shutdown_connections 2024-09-13T08:47:16.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.932+0000 7f59cdc00640 1 -- 172.21.15.44:0/3137427748 wait complete. 2024-09-13T08:47:16.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 Processor -- start 2024-09-13T08:47:16.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 -- start start 2024-09-13T08:47:16.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c80b3190 0x7f59c80b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59c81613c0 0x7f59c815b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c81617a0 0x7f59c80cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59c816c120 con 0x7f59c81617a0 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f59c816bfa0 con 0x7f59c80b3190 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f59c816c2a0 con 0x7f59c81613c0 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59c7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c81617a0 0x7f59c80cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59c7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c81617a0 0x7f59c80cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38416/0 (socket says 172.21.15.44:38416) 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59c7400640 1 -- 172.21.15.44:0/46823348 learned_addr learned my addr 172.21.15.44:0/46823348 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59bfe00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59c81613c0 0x7f59c815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59c6a00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c80b3190 0x7f59c80b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59c7400640 1 -- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59c81613c0 msgr2=0x7f59c815b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59c7400640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59c81613c0 0x7f59c815b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59c7400640 1 -- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c80b3190 msgr2=0x7f59c80b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59c7400640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c80b3190 0x7f59c80b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59c7400640 1 -- 172.21.15.44:0/46823348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c80c9260 con 0x7f59c81617a0 2024-09-13T08:47:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59c7400640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c81617a0 0x7f59c80cc770 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009bd0 tx=0x7f59c00a5b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59bea00640 1 -- 172.21.15.44:0/46823348 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c00bb070 con 0x7f59c81617a0 2024-09-13T08:47:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59bea00640 1 -- 172.21.15.44:0/46823348 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59c0004d40 con 0x7f59c81617a0 2024-09-13T08:47:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 -- 172.21.15.44:0/46823348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c80c9540 con 0x7f59c81617a0 2024-09-13T08:47:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59bea00640 1 -- 172.21.15.44:0/46823348 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c00b2040 con 0x7f59c81617a0 2024-09-13T08:47:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 -- 172.21.15.44:0/46823348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c80c9b00 con 0x7f59c81617a0 2024-09-13T08:47:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.936+0000 7f59cdc00640 1 -- 172.21.15.44:0/46823348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c8161bf0 con 0x7f59c81617a0 2024-09-13T08:47:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.940+0000 7f59bea00640 1 -- 172.21.15.44:0/46823348 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f59c00be070 con 0x7f59c81617a0 2024-09-13T08:47:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.940+0000 7f59bea00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f59a4074030 0x7f59a40764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.940+0000 7f59bea00640 1 -- 172.21.15.44:0/46823348 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f59c012cc50 con 0x7f59c81617a0 2024-09-13T08:47:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.940+0000 7f59c6a00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f59a4074030 0x7f59a40764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:16.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.940+0000 7f59c6a00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f59a4074030 0x7f59a40764f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f59b80016c0 tx=0x7f59b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:16.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:16.944+0000 7f59bea00640 1 -- 172.21.15.44:0/46823348 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f59c0016990 con 0x7f59c81617a0 2024-09-13T08:47:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.120+0000 7f59cdc00640 1 -- 172.21.15.44:0/46823348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f59c80d0240 con 0x7f59c81617a0 2024-09-13T08:47:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59bea00640 1 -- 172.21.15.44:0/46823348 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v43)=0 v43) ==== 136+0+0 (secure 0 0 0) 0x7f59c00f95a0 con 0x7f59c81617a0 2024-09-13T08:47:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 -- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f59a4074030 msgr2=0x7f59a40764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f59a4074030 0x7f59a40764f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f59b80016c0 tx=0x7f59b8009040 comp rx=0 tx=0).stop 2024-09-13T08:47:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 -- 172.21.15.44:0/46823348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c81617a0 msgr2=0x7f59c80cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c81617a0 0x7f59c80cc770 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009bd0 tx=0x7f59c00a5b40 comp rx=0 tx=0).stop 2024-09-13T08:47:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 -- 172.21.15.44:0/46823348 shutdown_connections 2024-09-13T08:47:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f59a4074030 0x7f59a40764f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c81617a0 0x7f59c80cc770 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f59c81613c0 0x7f59c815b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 --2- 172.21.15.44:0/46823348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c80b3190 0x7f59c80b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 -- 172.21.15.44:0/46823348 >> 172.21.15.44:0/46823348 conn(0x7f59c80d1dd0 msgr2=0x7f59c8161e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 -- 172.21.15.44:0/46823348 shutdown_connections 2024-09-13T08:47:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.124+0000 7f59cdc00640 1 -- 172.21.15.44:0/46823348 wait complete. 2024-09-13T08:47:17.148 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-13T08:47:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.296+0000 7f8dbd400640 1 -- 172.21.15.44:0/1854065362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8db815be10 msgr2=0x7f8db80ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.296+0000 7f8dbd400640 1 --2- 172.21.15.44:0/1854065362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8db815be10 0x7f8db80ce700 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009840 tx=0x7f8da801f010 comp rx=0 tx=0).stop 2024-09-13T08:47:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.296+0000 7f8dbd400640 1 -- 172.21.15.44:0/1854065362 shutdown_connections 2024-09-13T08:47:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.296+0000 7f8dbd400640 1 --2- 172.21.15.44:0/1854065362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8db815be10 0x7f8db80ce700 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.296+0000 7f8dbd400640 1 -- 172.21.15.44:0/1854065362 >> 172.21.15.44:0/1854065362 conn(0x7f8db80d1da0 msgr2=0x7f8db80d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:17.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.296+0000 7f8dbd400640 1 -- 172.21.15.44:0/1854065362 shutdown_connections 2024-09-13T08:47:17.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.296+0000 7f8dbd400640 1 -- 172.21.15.44:0/1854065362 wait complete. 2024-09-13T08:47:17.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8dbd400640 1 Processor -- start 2024-09-13T08:47:17.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8dbd400640 1 -- start start 2024-09-13T08:47:17.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8dbd400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8db80b0b60 0x7f8db8144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:17.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8dbd400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8db815be10 0x7f8db8158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:17.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8dbd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8db815c1f0 0x7f8db8158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:17.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8dbd400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8db80cd670 con 0x7f8db815c1f0 2024-09-13T08:47:17.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8dbd400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8db80cd4f0 con 0x7f8db815be10 2024-09-13T08:47:17.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8dbd400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8db80cd7f0 con 0x7f8db80b0b60 2024-09-13T08:47:17.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8db7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8db815c1f0 0x7f8db8158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:17.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8db7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8db815c1f0 0x7f8db8158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38436/0 (socket says 172.21.15.44:38436) 2024-09-13T08:47:17.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8db7200640 1 -- 172.21.15.44:0/3927315946 learned_addr learned my addr 172.21.15.44:0/3927315946 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:17.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8db6800640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8db80b0b60 0x7f8db8144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:17.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8dade00640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8db815be10 0x7f8db8158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:17.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8db7200640 1 -- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8db80b0b60 msgr2=0x7f8db8144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8db7200640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8db80b0b60 0x7f8db8144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8db7200640 1 -- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8db815be10 msgr2=0x7f8db8158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8db7200640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8db815be10 0x7f8db8158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8db7200640 1 -- 172.21.15.44:0/3927315946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db81f6320 con 0x7f8db815c1f0 2024-09-13T08:47:17.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8dade00640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8db815be10 0x7f8db8158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:47:17.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8db7200640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8db815c1f0 0x7f8db8158880 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8db000cbc0 tx=0x7f8db000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:17.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8daf400640 1 -- 172.21.15.44:0/3927315946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0016020 con 0x7f8db815c1f0 2024-09-13T08:47:17.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.300+0000 7f8daf400640 1 -- 172.21.15.44:0/3927315946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8db000ab40 con 0x7f8db815c1f0 2024-09-13T08:47:17.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.304+0000 7f8dbd400640 1 -- 172.21.15.44:0/3927315946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db81f5560 con 0x7f8db815c1f0 2024-09-13T08:47:17.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.304+0000 7f8daf400640 1 -- 172.21.15.44:0/3927315946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0012420 con 0x7f8db815c1f0 2024-09-13T08:47:17.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.304+0000 7f8dbd400640 1 -- 172.21.15.44:0/3927315946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db81f5b20 con 0x7f8db815c1f0 2024-09-13T08:47:17.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.304+0000 7f8dbd400640 1 -- 172.21.15.44:0/3927315946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db80cdca0 con 0x7f8db815c1f0 2024-09-13T08:47:17.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.304+0000 7f8daf400640 1 -- 172.21.15.44:0/3927315946 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f8db0019020 con 0x7f8db815c1f0 2024-09-13T08:47:17.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.304+0000 7f8daf400640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d94074060 0x7f8d94076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:17.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.304+0000 7f8daf400640 1 -- 172.21.15.44:0/3927315946 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f8db0092bd0 con 0x7f8db815c1f0 2024-09-13T08:47:17.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.304+0000 7f8db6800640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d94074060 0x7f8d94076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:17.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.308+0000 7f8db6800640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d94074060 0x7f8d94076520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8da80b2000 tx=0x7f8da801d850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:17.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.308+0000 7f8daf400640 1 -- 172.21.15.44:0/3927315946 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8db0020030 con 0x7f8db815c1f0 2024-09-13T08:47:17.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.508+0000 7f8dbd400640 1 -- 172.21.15.44:0/3927315946 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f8db80c5220 con 0x7f8d94074060 2024-09-13T08:47:17.522 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:17.523+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:17.522 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:17.523+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:17.522 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:17.523+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:17.522 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:17.523+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:17.522 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:17.523+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:17.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8daf400640 1 -- 172.21.15.44:0/3927315946 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8db80c5220 con 0x7f8d94074060 2024-09-13T08:47:17.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 -- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d94074060 msgr2=0x7f8d94076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d94074060 0x7f8d94076520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8da80b2000 tx=0x7f8da801d850 comp rx=0 tx=0).stop 2024-09-13T08:47:17.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 -- 172.21.15.44:0/3927315946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8db815c1f0 msgr2=0x7f8db8158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8db815c1f0 0x7f8db8158880 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8db000cbc0 tx=0x7f8db000ccb0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 -- 172.21.15.44:0/3927315946 shutdown_connections 2024-09-13T08:47:17.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d94074060 0x7f8d94076520 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8db815c1f0 0x7f8db8158880 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8db815be10 0x7f8db8158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 --2- 172.21.15.44:0/3927315946 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8db80b0b60 0x7f8db8144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 -- 172.21.15.44:0/3927315946 >> 172.21.15.44:0/3927315946 conn(0x7f8db80d1da0 msgr2=0x7f8db8071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:17.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 -- 172.21.15.44:0/3927315946 shutdown_connections 2024-09-13T08:47:17.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.564+0000 7f8dbd400640 1 -- 172.21.15.44:0/3927315946 wait complete. 2024-09-13T08:47:17.587 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:47:17.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.732+0000 7f0744c00640 1 -- 172.21.15.44:0/4199006051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f074015f640 msgr2=0x7f07400cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.732+0000 7f0744c00640 1 --2- 172.21.15.44:0/4199006051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f074015f640 0x7f07400cd510 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f07400b1250 tx=0x7f073001c6c0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 -- 172.21.15.44:0/4199006051 shutdown_connections 2024-09-13T08:47:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 --2- 172.21.15.44:0/4199006051 >> v2:172.21.15.129:3300/0 conn(0x7f07400ce7f0 0x7f07400cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 --2- 172.21.15.44:0/4199006051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f074015f640 0x7f07400cd510 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 -- 172.21.15.44:0/4199006051 >> 172.21.15.44:0/4199006051 conn(0x7f07400d1e30 msgr2=0x7f07400d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 -- 172.21.15.44:0/4199006051 shutdown_connections 2024-09-13T08:47:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 -- 172.21.15.44:0/4199006051 wait complete. 2024-09-13T08:47:17.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 Processor -- start 2024-09-13T08:47:17.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 -- start start 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07400ce7f0 0x7f07401507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074015f640 0x7f074014ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f074015b860 0x7f0740153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0740169720 con 0x7f07400ce7f0 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07401695a0 con 0x7f074015f640 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f07401698a0 con 0x7f074015b860 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f073de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07400ce7f0 0x7f07401507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f073de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07400ce7f0 0x7f07401507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38454/0 (socket says 172.21.15.44:38454) 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f073de00640 1 -- 172.21.15.44:0/4289566936 learned_addr learned my addr 172.21.15.44:0/4289566936 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f073e800640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f074015b860 0x7f0740153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0737e00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074015f640 0x7f074014ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:17.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f073de00640 1 -- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f074015b860 msgr2=0x7f0740153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f073de00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f074015b860 0x7f0740153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f073de00640 1 -- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074015f640 msgr2=0x7f074014ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f073de00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074015f640 0x7f074014ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f073de00640 1 -- 172.21.15.44:0/4289566936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0740150dc0 con 0x7f07400ce7f0 2024-09-13T08:47:17.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0737e00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074015f640 0x7f074014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:47:17.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f073de00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07400ce7f0 0x7f07401507c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f07300192d0 tx=0x7f0730003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:17.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0736000640 1 -- 172.21.15.44:0/4289566936 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730005c40 con 0x7f07400ce7f0 2024-09-13T08:47:17.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0736000640 1 -- 172.21.15.44:0/4289566936 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0730004030 con 0x7f07400ce7f0 2024-09-13T08:47:17.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0736000640 1 -- 172.21.15.44:0/4289566936 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730005290 con 0x7f07400ce7f0 2024-09-13T08:47:17.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 -- 172.21.15.44:0/4289566936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074006f850 con 0x7f07400ce7f0 2024-09-13T08:47:17.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.736+0000 7f0744c00640 1 -- 172.21.15.44:0/4289566936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f074015cb00 con 0x7f07400ce7f0 2024-09-13T08:47:17.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.740+0000 7f0744c00640 1 -- 172.21.15.44:0/4289566936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f074006ac30 con 0x7f07400ce7f0 2024-09-13T08:47:17.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.740+0000 7f0736000640 1 -- 172.21.15.44:0/4289566936 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 98661+0+0 (secure 0 0 0) 0x7f0730005430 con 0x7f07400ce7f0 2024-09-13T08:47:17.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.740+0000 7f0736000640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f071c074030 0x7f071c0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:17.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.740+0000 7f0736000640 1 -- 172.21.15.44:0/4289566936 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f073012cf20 con 0x7f07400ce7f0 2024-09-13T08:47:17.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.740+0000 7f0737e00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f071c074030 0x7f071c0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:17.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.744+0000 7f0737e00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f071c074030 0x7f071c0764f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f07400c5490 tx=0x7f0728009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:17.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.744+0000 7f0736000640 1 -- 172.21.15.44:0/4289566936 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f07300c0060 con 0x7f07400ce7f0 2024-09-13T08:47:17.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.948+0000 7f0744c00640 1 -- 172.21.15.44:0/4289566936 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f07400ce0e0 con 0x7f071c074030 2024-09-13T08:47:17.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.952+0000 7f0736000640 1 -- 172.21.15.44:0/4289566936 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f07400ce0e0 con 0x7f071c074030 2024-09-13T08:47:17.954 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/f5051d5f-df45-49a0-bdf2-11fcb706b0a6 2024-09-13T08:47:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.952+0000 7f0744c00640 1 -- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f071c074030 msgr2=0x7f071c0764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.952+0000 7f0744c00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f071c074030 0x7f071c0764f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f07400c5490 tx=0x7f0728009040 comp rx=0 tx=0).stop 2024-09-13T08:47:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.952+0000 7f0744c00640 1 -- 172.21.15.44:0/4289566936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07400ce7f0 msgr2=0x7f07401507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:17.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.952+0000 7f0744c00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07400ce7f0 0x7f07401507c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f07300192d0 tx=0x7f0730003dd0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.956+0000 7f0744c00640 1 -- 172.21.15.44:0/4289566936 shutdown_connections 2024-09-13T08:47:17.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.956+0000 7f0744c00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f071c074030 0x7f071c0764f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.956+0000 7f0744c00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f074015b860 0x7f0740153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.956+0000 7f0744c00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074015f640 0x7f074014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.956+0000 7f0744c00640 1 --2- 172.21.15.44:0/4289566936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07400ce7f0 0x7f07401507c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:17.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.956+0000 7f0744c00640 1 -- 172.21.15.44:0/4289566936 >> 172.21.15.44:0/4289566936 conn(0x7f07400d1e30 msgr2=0x7f074015a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:17.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.956+0000 7f0744c00640 1 -- 172.21.15.44:0/4289566936 shutdown_connections 2024-09-13T08:47:17.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:17.956+0000 7f0744c00640 1 -- 172.21.15.44:0/4289566936 wait complete. 2024-09-13T08:47:17.975 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/f5051d5f-df45-49a0-bdf2-11fcb706b0a6 2024-09-13T08:47:17.975 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:47:17.975 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:47:17.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:17.984 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f5051d5f-df45-49a0-bdf2-11fcb706b0a6/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:47:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:47:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:47:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.14894 s, 117 MB/s 2024-09-13T08:47:27.195 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:47:27.195 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:47:27.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:27.204 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f5051d5f-df45-49a0-bdf2-11fcb706b0a6/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:47:32.305 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:47:32.309 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:47:36.920 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:47:36.920 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:47:36.920 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.65945 s, 111 MB/s 2024-09-13T08:47:36.921 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:47:36.921 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:47:36.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:36.932 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f5051d5f-df45-49a0-bdf2-11fcb706b0a6/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:47:46.656 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:47:46.656 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:47:46.657 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.67202 s, 111 MB/s 2024-09-13T08:47:46.658 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-13T08:47:46.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.808+0000 7f1bb6200640 1 -- 172.21.15.44:0/37779723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb00d1100 msgr2=0x7f1bb00b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.808+0000 7f1bb6200640 1 --2- 172.21.15.44:0/37779723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb00d1100 0x7f1bb00b3ee0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009bc0 tx=0x7f1ba801cba0 comp rx=0 tx=0).stop 2024-09-13T08:47:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.808+0000 7f1bb6200640 1 -- 172.21.15.44:0/37779723 shutdown_connections 2024-09-13T08:47:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.808+0000 7f1bb6200640 1 --2- 172.21.15.44:0/37779723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb00d1100 0x7f1bb00b3ee0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.808+0000 7f1bb6200640 1 --2- 172.21.15.44:0/37779723 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1bb015f1d0 0x7f1bb00d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.808+0000 7f1bb6200640 1 -- 172.21.15.44:0/37779723 >> 172.21.15.44:0/37779723 conn(0x7f1bb006dd30 msgr2=0x7f1bb006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.808+0000 7f1bb6200640 1 -- 172.21.15.44:0/37779723 shutdown_connections 2024-09-13T08:47:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.808+0000 7f1bb6200640 1 -- 172.21.15.44:0/37779723 wait complete. 2024-09-13T08:47:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.808+0000 7f1bb6200640 1 Processor -- start 2024-09-13T08:47:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bb6200640 1 -- start start 2024-09-13T08:47:46.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bb6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb00d1100 0x7f1bb00d0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:46.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bb6200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1bb015df90 0x7f1bb01f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:46.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bb6200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb015f1d0 0x7f1bb01446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:46.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bb6200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0167b50 con 0x7f1bb00d1100 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bb6200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01679d0 con 0x7f1bb015f1d0 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bb6200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb0167cd0 con 0x7f1bb015df90 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1baf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb00d1100 0x7f1bb00d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1baf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb00d1100 0x7f1bb00d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56304/0 (socket says 172.21.15.44:56304) 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1baf400640 1 -- 172.21.15.44:0/410984560 learned_addr learned my addr 172.21.15.44:0/410984560 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bafe00640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb015f1d0 0x7f1bb01446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1ba7e00640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1bb015df90 0x7f1bb01f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1baf400640 1 -- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1bb015df90 msgr2=0x7f1bb01f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1baf400640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1bb015df90 0x7f1bb01f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1baf400640 1 -- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb015f1d0 msgr2=0x7f1bb01446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1baf400640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb015f1d0 0x7f1bb01446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1baf400640 1 -- 172.21.15.44:0/410984560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb00b1500 con 0x7f1bb00d1100 2024-09-13T08:47:46.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1baf400640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb00d1100 0x7f1bb00d0240 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c450 tx=0x7f1ba000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:46.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1ba7400640 1 -- 172.21.15.44:0/410984560 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0016020 con 0x7f1bb00d1100 2024-09-13T08:47:46.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1ba7400640 1 -- 172.21.15.44:0/410984560 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ba0012920 con 0x7f1bb00d1100 2024-09-13T08:47:46.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1ba7400640 1 -- 172.21.15.44:0/410984560 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0012d70 con 0x7f1bb00d1100 2024-09-13T08:47:46.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bb6200640 1 -- 172.21.15.44:0/410984560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb00b17e0 con 0x7f1bb00d1100 2024-09-13T08:47:46.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bb6200640 1 -- 172.21.15.44:0/410984560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb0144bf0 con 0x7f1bb00d1100 2024-09-13T08:47:46.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.812+0000 7f1bb6200640 1 -- 172.21.15.44:0/410984560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb00b3ee0 con 0x7f1bb00d1100 2024-09-13T08:47:46.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.816+0000 7f1ba7400640 1 -- 172.21.15.44:0/410984560 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98763+0+0 (secure 0 0 0) 0x7f1ba0019020 con 0x7f1bb00d1100 2024-09-13T08:47:46.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.816+0000 7f1ba7400640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b8c074150 0x7f1b8c076610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:46.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.816+0000 7f1ba7400640 1 -- 172.21.15.44:0/410984560 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f1ba00928d0 con 0x7f1bb00d1100 2024-09-13T08:47:46.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.816+0000 7f1ba7e00640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b8c074150 0x7f1b8c076610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:46.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.816+0000 7f1ba7e00640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b8c074150 0x7f1b8c076610 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00b3a00 tx=0x7f1b98006f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:46.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:46.820+0000 7f1ba7400640 1 -- 172.21.15.44:0/410984560 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1ba0020020 con 0x7f1bb00d1100 2024-09-13T08:47:47.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.024+0000 7f1bb6200640 1 -- 172.21.15.44:0/410984560 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f1bb015ea80 con 0x7f1b8c074150 2024-09-13T08:47:47.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1ba7400640 1 -- 172.21.15.44:0/410984560 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1bb015ea80 con 0x7f1b8c074150 2024-09-13T08:47:47.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 -- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b8c074150 msgr2=0x7f1b8c076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:47.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b8c074150 0x7f1b8c076610 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00b3a00 tx=0x7f1b98006f90 comp rx=0 tx=0).stop 2024-09-13T08:47:47.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 -- 172.21.15.44:0/410984560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb00d1100 msgr2=0x7f1bb00d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:47.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb00d1100 0x7f1bb00d0240 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c450 tx=0x7f1ba000c910 comp rx=0 tx=0).stop 2024-09-13T08:47:47.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 -- 172.21.15.44:0/410984560 shutdown_connections 2024-09-13T08:47:47.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1b8c074150 0x7f1b8c076610 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb015f1d0 0x7f1bb01446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1bb015df90 0x7f1bb01f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 --2- 172.21.15.44:0/410984560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb00d1100 0x7f1bb00d0240 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 -- 172.21.15.44:0/410984560 >> 172.21.15.44:0/410984560 conn(0x7f1bb006dd30 msgr2=0x7f1bb006cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:47.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 -- 172.21.15.44:0/410984560 shutdown_connections 2024-09-13T08:47:47.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.040+0000 7f1bb6200640 1 -- 172.21.15.44:0/410984560 wait complete. 2024-09-13T08:47:47.063 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:47:47.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 -- 172.21.15.44:0/3160492612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43640d22c0 msgr2=0x7f43640cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:47.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 --2- 172.21.15.44:0/3160492612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43640d22c0 0x7f43640cc620 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f43640cf780 tx=0x7f435c01cd80 comp rx=0 tx=0).stop 2024-09-13T08:47:47.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 -- 172.21.15.44:0/3160492612 shutdown_connections 2024-09-13T08:47:47.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 --2- 172.21.15.44:0/3160492612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43640d22c0 0x7f43640cc620 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 --2- 172.21.15.44:0/3160492612 >> v2:172.21.15.129:3300/0 conn(0x7f436415bd40 0x7f43640d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 -- 172.21.15.44:0/3160492612 >> 172.21.15.44:0/3160492612 conn(0x7f436409aa10 msgr2=0x7f436409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:47.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 -- 172.21.15.44:0/3160492612 shutdown_connections 2024-09-13T08:47:47.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 -- 172.21.15.44:0/3160492612 wait complete. 2024-09-13T08:47:47.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 Processor -- start 2024-09-13T08:47:47.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 -- start start 2024-09-13T08:47:47.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.212+0000 7f4363200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43640b0d10 0x7f4364155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:47.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f4363200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43640d22c0 0x7f436414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:47.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f4363200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f436415bd40 0x7f436414ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:47.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f4363200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4364168bf0 con 0x7f43640d22c0 2024-09-13T08:47:47.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f4363200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4364168a70 con 0x7f436415bd40 2024-09-13T08:47:47.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f4363200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4364168d70 con 0x7f43640b0d10 2024-09-13T08:47:47.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43640d22c0 0x7f436414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:47.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43640d22c0 0x7f436414c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56336/0 (socket says 172.21.15.44:56336) 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435be00640 1 -- 172.21.15.44:0/1780755538 learned_addr learned my addr 172.21.15.44:0/1780755538 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f4362800640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f436415bd40 0x7f436414ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f4361e00640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43640b0d10 0x7f4364155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435be00640 1 -- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43640b0d10 msgr2=0x7f4364155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435be00640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43640b0d10 0x7f4364155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435be00640 1 -- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f436415bd40 msgr2=0x7f436414ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435be00640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f436415bd40 0x7f436414ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435be00640 1 -- 172.21.15.44:0/1780755538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43641562a0 con 0x7f43640d22c0 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435be00640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43640d22c0 0x7f436414c1e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f434c00c490 tx=0x7f434c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435a000640 1 -- 172.21.15.44:0/1780755538 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c017070 con 0x7f43640d22c0 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435a000640 1 -- 172.21.15.44:0/1780755538 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f434c003600 con 0x7f43640d22c0 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435a000640 1 -- 172.21.15.44:0/1780755538 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c003a90 con 0x7f43640d22c0 2024-09-13T08:47:47.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f4363200640 1 -- 172.21.15.44:0/1780755538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4364155a00 con 0x7f43640d22c0 2024-09-13T08:47:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f4363200640 1 -- 172.21.15.44:0/1780755538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4364155fc0 con 0x7f43640d22c0 2024-09-13T08:47:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f4363200640 1 -- 172.21.15.44:0/1780755538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43640cc620 con 0x7f43640d22c0 2024-09-13T08:47:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.216+0000 7f435a000640 1 -- 172.21.15.44:0/1780755538 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98763+0+0 (secure 0 0 0) 0x7f434c003c30 con 0x7f43640d22c0 2024-09-13T08:47:47.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.220+0000 7f435a000640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4340074180 0x7f4340076640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:47.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.220+0000 7f435a000640 1 -- 172.21.15.44:0/1780755538 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f434c091e90 con 0x7f43640d22c0 2024-09-13T08:47:47.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.220+0000 7f4361e00640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4340074180 0x7f4340076640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:47.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.220+0000 7f4361e00640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4340074180 0x7f4340076640 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4354006fd0 tx=0x7f4354008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:47.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.224+0000 7f435a000640 1 -- 172.21.15.44:0/1780755538 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f434c05e160 con 0x7f43640d22c0 2024-09-13T08:47:47.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.428+0000 7f4363200640 1 -- 172.21.15.44:0/1780755538 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4364069e30 con 0x7f4340074180 2024-09-13T08:47:47.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f435a000640 1 -- 172.21.15.44:0/1780755538 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f435400bda0 con 0x7f4340074180 2024-09-13T08:47:47.434 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/f5051d5f-df45-49a0-bdf2-11fcb706b0a6 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 -- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4340074180 msgr2=0x7f4340076640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4340074180 0x7f4340076640 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4354006fd0 tx=0x7f4354008040 comp rx=0 tx=0).stop 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 -- 172.21.15.44:0/1780755538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43640d22c0 msgr2=0x7f436414c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43640d22c0 0x7f436414c1e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f434c00c490 tx=0x7f434c001e80 comp rx=0 tx=0).stop 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 -- 172.21.15.44:0/1780755538 shutdown_connections 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4340074180 0x7f4340076640 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f436415bd40 0x7f436414ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43640d22c0 0x7f436414c1e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 --2- 172.21.15.44:0/1780755538 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43640b0d10 0x7f4364155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 -- 172.21.15.44:0/1780755538 >> 172.21.15.44:0/1780755538 conn(0x7f436409aa10 msgr2=0x7f436406f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:47.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 -- 172.21.15.44:0/1780755538 shutdown_connections 2024-09-13T08:47:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:47.432+0000 7f4363200640 1 -- 172.21.15.44:0/1780755538 wait complete. 2024-09-13T08:47:47.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:47.455 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f5051d5f-df45-49a0-bdf2-11fcb706b0a6 2024-09-13T08:47:47.455 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:47:47.465 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/f5051d5f-df45-49a0-bdf2-11fcb706b0a6 2024-09-13T08:47:47.465 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3055190016" 2024-09-13T08:47:47.465 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:47.466 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-13T08:47:49.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:47:49.467 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f5051d5f-df45-49a0-bdf2-11fcb706b0a6 2024-09-13T08:47:49.467 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:47:49.477 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/f5051d5f-df45-49a0-bdf2-11fcb706b0a6 2024-09-13T08:47:49.477 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3221225472" 2024-09-13T08:47:49.477 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:49.478 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-13T08:47:49.478 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6700 2024-09-13T08:47:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.680+0000 7f6455800640 1 -- 172.21.15.44:0/1162130421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6450161300 msgr2=0x7f64500d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.680+0000 7f6455800640 1 --2- 172.21.15.44:0/1162130421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6450161300 0x7f64500d0960 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f64500b7da0 tx=0x7f64500b2540 comp rx=0 tx=0).stop 2024-09-13T08:47:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.680+0000 7f6455800640 1 -- 172.21.15.44:0/1162130421 shutdown_connections 2024-09-13T08:47:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.680+0000 7f6455800640 1 --2- 172.21.15.44:0/1162130421 >> v2:172.21.15.129:3300/0 conn(0x7f64500cd8f0 0x7f64500cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:49.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.680+0000 7f6455800640 1 --2- 172.21.15.44:0/1162130421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6450161300 0x7f64500d0960 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:49.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.680+0000 7f6455800640 1 -- 172.21.15.44:0/1162130421 >> 172.21.15.44:0/1162130421 conn(0x7f64500d1cf0 msgr2=0x7f64500d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:49.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6455800640 1 -- 172.21.15.44:0/1162130421 shutdown_connections 2024-09-13T08:47:49.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6455800640 1 -- 172.21.15.44:0/1162130421 wait complete. 2024-09-13T08:47:49.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6455800640 1 Processor -- start 2024-09-13T08:47:49.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6455800640 1 -- start start 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6455800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64500cd8f0 0x7f64500718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6455800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64500ce700 0x7f64500b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6455800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6450161300 0x7f6450157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6455800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f645016af50 con 0x7f64500cd8f0 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6455800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f645016add0 con 0x7f64500ce700 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6455800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f645016b0d0 con 0x7f6450161300 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f644fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64500cd8f0 0x7f64500718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f644fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64500cd8f0 0x7f64500718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56356/0 (socket says 172.21.15.44:56356) 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f644fe00640 1 -- 172.21.15.44:0/1891280399 learned_addr learned my addr 172.21.15.44:0/1891280399 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6447400640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64500ce700 0x7f64500b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6454e00640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6450161300 0x7f6450157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f644fe00640 1 -- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6450161300 msgr2=0x7f6450157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f644fe00640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6450161300 0x7f6450157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f644fe00640 1 -- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64500ce700 msgr2=0x7f64500b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f644fe00640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64500ce700 0x7f64500b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f644fe00640 1 -- 172.21.15.44:0/1891280399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64500c6860 con 0x7f64500cd8f0 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f6447400640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64500ce700 0x7f64500b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f644fe00640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64500cd8f0 0x7f64500718b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f64400b0040 tx=0x7f64400a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.684+0000 7f644cc00640 1 -- 172.21.15.44:0/1891280399 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64400b2040 con 0x7f64500cd8f0 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.688+0000 7f644cc00640 1 -- 172.21.15.44:0/1891280399 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64400043b0 con 0x7f64500cd8f0 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.688+0000 7f6455800640 1 -- 172.21.15.44:0/1891280399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64500c7a10 con 0x7f64500cd8f0 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.688+0000 7f644cc00640 1 -- 172.21.15.44:0/1891280399 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440005280 con 0x7f64500cd8f0 2024-09-13T08:47:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.688+0000 7f6455800640 1 -- 172.21.15.44:0/1891280399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6450157a80 con 0x7f64500cd8f0 2024-09-13T08:47:49.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.688+0000 7f6455800640 1 -- 172.21.15.44:0/1891280399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64500cc970 con 0x7f64500cd8f0 2024-09-13T08:47:49.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.688+0000 7f644cc00640 1 -- 172.21.15.44:0/1891280399 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98763+0+0 (secure 0 0 0) 0x7f6440005420 con 0x7f64500cd8f0 2024-09-13T08:47:49.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.688+0000 7f644cc00640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f642c074180 0x7f642c076640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:49.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.688+0000 7f644cc00640 1 -- 172.21.15.44:0/1891280399 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f644012caf0 con 0x7f64500cd8f0 2024-09-13T08:47:49.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.688+0000 7f6447400640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f642c074180 0x7f642c076640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:49.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.692+0000 7f6447400640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f642c074180 0x7f642c076640 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6450168720 tx=0x7f64380091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:49.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.692+0000 7f644cc00640 1 -- 172.21.15.44:0/1891280399 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f64400164b0 con 0x7f64500cd8f0 2024-09-13T08:47:49.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.904+0000 7f6455800640 1 -- 172.21.15.44:0/1891280399 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6700", "target": ["mon-mgr", ""]}) -- 0x7f64500b4260 con 0x7f642c074180 2024-09-13T08:47:49.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f644cc00640 1 -- 172.21.15.44:0/1891280399 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f64500b4260 con 0x7f642c074180 2024-09-13T08:47:49.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 -- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f642c074180 msgr2=0x7f642c076640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f642c074180 0x7f642c076640 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6450168720 tx=0x7f64380091e0 comp rx=0 tx=0).stop 2024-09-13T08:47:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 -- 172.21.15.44:0/1891280399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64500cd8f0 msgr2=0x7f64500718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64500cd8f0 0x7f64500718b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f64400b0040 tx=0x7f64400a5770 comp rx=0 tx=0).stop 2024-09-13T08:47:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 -- 172.21.15.44:0/1891280399 shutdown_connections 2024-09-13T08:47:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f642c074180 0x7f642c076640 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6450161300 0x7f6450157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64500ce700 0x7f64500b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 --2- 172.21.15.44:0/1891280399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64500cd8f0 0x7f64500718b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 -- 172.21.15.44:0/1891280399 >> 172.21.15.44:0/1891280399 conn(0x7f64500d1cf0 msgr2=0x7f64500af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 -- 172.21.15.44:0/1891280399 shutdown_connections 2024-09-13T08:47:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:49.972+0000 7f6455800640 1 -- 172.21.15.44:0/1891280399 wait complete. 2024-09-13T08:47:49.977 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:49.979+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:49.977 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:49.979+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:49.977 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:49.979+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:49.977 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:49.979+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:49.977 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:49.979+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:49.995 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:49.995+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:49.995 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:49.995+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:49.995 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:49.995+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:49.996 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:49.995+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:49.996 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:49.995+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:49.998 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone5221 2024-09-13T08:47:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.156+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3649082283 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6b81617a0 msgr2=0x7fc6b8162300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.156+0000 7fc6bdc00640 1 --2- 172.21.15.44:0/3649082283 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6b81617a0 0x7fc6b8162300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a001c710 comp rx=0 tx=0).stop 2024-09-13T08:47:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.156+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3649082283 shutdown_connections 2024-09-13T08:47:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.156+0000 7fc6bdc00640 1 --2- 172.21.15.44:0/3649082283 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6b81617a0 0x7fc6b8162300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.156+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3649082283 >> 172.21.15.44:0/3649082283 conn(0x7fc6b80d1dd0 msgr2=0x7fc6b80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.156+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3649082283 shutdown_connections 2024-09-13T08:47:50.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.156+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3649082283 wait complete. 2024-09-13T08:47:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.156+0000 7fc6bdc00640 1 Processor -- start 2024-09-13T08:47:50.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.156+0000 7fc6bdc00640 1 -- start start 2024-09-13T08:47:50.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6bdc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6b80b3190 0x7fc6b8157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6bdc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b81613c0 0x7fc6b81512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6bdc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6b81617a0 0x7fc6b80cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6bdc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b816c590 con 0x7fc6b81613c0 2024-09-13T08:47:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6bdc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b816c410 con 0x7fc6b80b3190 2024-09-13T08:47:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6bdc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b816c710 con 0x7fc6b81617a0 2024-09-13T08:47:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6afe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b81613c0 0x7fc6b81512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6b6a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6b80b3190 0x7fc6b8157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6afe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b81613c0 0x7fc6b81512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56368/0 (socket says 172.21.15.44:56368) 2024-09-13T08:47:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6afe00640 1 -- 172.21.15.44:0/3974442222 learned_addr learned my addr 172.21.15.44:0/3974442222 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:50.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6b7400640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6b81617a0 0x7fc6b80cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:50.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6afe00640 1 -- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6b81617a0 msgr2=0x7fc6b80cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:50.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6afe00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6b81617a0 0x7fc6b80cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:50.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6afe00640 1 -- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6b80b3190 msgr2=0x7fc6b8157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6afe00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6b80b3190 0x7fc6b8157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6afe00640 1 -- 172.21.15.44:0/3974442222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b8157bf0 con 0x7fc6b81613c0 2024-09-13T08:47:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6b7400640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6b81617a0 0x7fc6b80cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:47:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6afe00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b81613c0 0x7fc6b81512a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc6a001cbf0 tx=0x7fc6a0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6aea00640 1 -- 172.21.15.44:0/3974442222 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a00b2040 con 0x7fc6b81613c0 2024-09-13T08:47:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6aea00640 1 -- 172.21.15.44:0/3974442222 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6a00bb070 con 0x7fc6b81613c0 2024-09-13T08:47:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3974442222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b8158c60 con 0x7fc6b81613c0 2024-09-13T08:47:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6aea00640 1 -- 172.21.15.44:0/3974442222 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0003600 con 0x7fc6b81613c0 2024-09-13T08:47:50.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3974442222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b8158950 con 0x7fc6b81613c0 2024-09-13T08:47:50.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.160+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3974442222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b8159180 con 0x7fc6b81613c0 2024-09-13T08:47:50.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.164+0000 7fc6aea00640 1 -- 172.21.15.44:0/3974442222 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98763+0+0 (secure 0 0 0) 0x7fc6a00a56b0 con 0x7fc6b81613c0 2024-09-13T08:47:50.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.164+0000 7fc6aea00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc694074150 0x7fc694076610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:50.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.164+0000 7fc6aea00640 1 -- 172.21.15.44:0/3974442222 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fc6a012cf30 con 0x7fc6b81613c0 2024-09-13T08:47:50.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.164+0000 7fc6b6a00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc694074150 0x7fc694076610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:50.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.164+0000 7fc6b6a00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc694074150 0x7fc694076610 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8007fd0 tx=0x7fc6a8001460 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:50.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.168+0000 7fc6aea00640 1 -- 172.21.15.44:0/3974442222 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc6a0016450 con 0x7fc6b81613c0 2024-09-13T08:47:50.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:50.376+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3974442222 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone5221", "target": ["mon-mgr", ""]}) -- 0x7fc6b815cd70 con 0x7fc694074150 2024-09-13T08:47:50.977 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:50.979+0000 7f8a86000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:50.977 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:50.979+0000 7f8a86000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:50.977 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:50.979+0000 7f8a86000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:50.977 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:50.979+0000 7f8a86000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:50.977 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:50.979+0000 7f8a86000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:51.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.056+0000 7fc6aea00640 1 -- 172.21.15.44:0/3974442222 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc6b815cd70 con 0x7fc694074150 2024-09-13T08:47:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc694074150 msgr2=0x7fc694076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc694074150 0x7fc694076610 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8007fd0 tx=0x7fc6a8001460 comp rx=0 tx=0).stop 2024-09-13T08:47:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3974442222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b81613c0 msgr2=0x7fc6b81512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b81613c0 0x7fc6b81512a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc6a001cbf0 tx=0x7fc6a0005760 comp rx=0 tx=0).stop 2024-09-13T08:47:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3974442222 shutdown_connections 2024-09-13T08:47:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc694074150 0x7fc694076610 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6b81617a0 0x7fc6b80cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b81613c0 0x7fc6b81512a0 secure :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc6a001cbf0 tx=0x7fc6a0005760 comp rx=0 tx=0).stop 2024-09-13T08:47:51.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 --2- 172.21.15.44:0/3974442222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6b80b3190 0x7fc6b8157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:51.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3974442222 >> 172.21.15.44:0/3974442222 conn(0x7fc6b80d1dd0 msgr2=0x7fc6b80b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:51.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3974442222 shutdown_connections 2024-09-13T08:47:51.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.060+0000 7fc6bdc00640 1 -- 172.21.15.44:0/3974442222 wait complete. 2024-09-13T08:47:51.095 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone25122 2024-09-13T08:47:51.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.332+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2653888772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec160490 msgr2=0x7fcbec06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.332+0000 7fcbf2e00640 1 --2- 172.21.15.44:0/2653888772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec160490 0x7fcbec06e7a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009c80 tx=0x7fcbe401cd70 comp rx=0 tx=0).stop 2024-09-13T08:47:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.332+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2653888772 shutdown_connections 2024-09-13T08:47:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.332+0000 7fcbf2e00640 1 --2- 172.21.15.44:0/2653888772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec160490 0x7fcbec06e7a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.332+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2653888772 >> 172.21.15.44:0/2653888772 conn(0x7fcbec0b18f0 msgr2=0x7fcbec0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.336+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2653888772 shutdown_connections 2024-09-13T08:47:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.336+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2653888772 wait complete. 2024-09-13T08:47:51.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.336+0000 7fcbf2e00640 1 Processor -- start 2024-09-13T08:47:51.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.336+0000 7fcbf2e00640 1 -- start start 2024-09-13T08:47:51.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf2e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbec0cf1d0 0x7fcbec15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:51.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf2e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbec15f9a0 0x7fcbec0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:51.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec160490 0x7fcbec152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:51.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf2e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec16a7c0 con 0x7fcbec160490 2024-09-13T08:47:51.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf2e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec159dd0 con 0x7fcbec0cf1d0 2024-09-13T08:47:51.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf2e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcbec16a940 con 0x7fcbec15f9a0 2024-09-13T08:47:51.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec160490 0x7fcbec152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:51.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec160490 0x7fcbec152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56382/0 (socket says 172.21.15.44:56382) 2024-09-13T08:47:51.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf0e00640 1 -- 172.21.15.44:0/2320974245 learned_addr learned my addr 172.21.15.44:0/2320974245 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:51.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf0e00640 1 -- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbec15f9a0 msgr2=0x7fcbec0b3520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:47:51.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf0e00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbec15f9a0 0x7fcbec0b3520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:51.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf0e00640 1 -- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbec0cf1d0 msgr2=0x7fcbec15aab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:47:51.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf0e00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbec0cf1d0 0x7fcbec15aab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:51.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.340+0000 7fcbf0e00640 1 -- 172.21.15.44:0/2320974245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbec0b2d90 con 0x7fcbec160490 2024-09-13T08:47:51.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.344+0000 7fcbf0e00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec160490 0x7fcbec152f40 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009bd0 tx=0x7fcbe40ae600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:51.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.344+0000 7fcbe3e00640 1 -- 172.21.15.44:0/2320974245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40b2040 con 0x7fcbec160490 2024-09-13T08:47:51.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.344+0000 7fcbe3e00640 1 -- 172.21.15.44:0/2320974245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbe40be070 con 0x7fcbec160490 2024-09-13T08:47:51.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.344+0000 7fcbe3e00640 1 -- 172.21.15.44:0/2320974245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40ae9d0 con 0x7fcbec160490 2024-09-13T08:47:51.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.344+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2320974245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbec0b4620 con 0x7fcbec160490 2024-09-13T08:47:51.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.344+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2320974245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbec152bd0 con 0x7fcbec160490 2024-09-13T08:47:51.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.348+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2320974245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc40052a0 con 0x7fcbec160490 2024-09-13T08:47:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.352+0000 7fcbe3e00640 1 -- 172.21.15.44:0/2320974245 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98763+0+0 (secure 0 0 0) 0x7fcbe40aeb70 con 0x7fcbec160490 2024-09-13T08:47:51.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.352+0000 7fcbe3e00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbcc074150 0x7fcbcc076610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:51.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.352+0000 7fcbe3e00640 1 -- 172.21.15.44:0/2320974245 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fcbe4135210 con 0x7fcbec160490 2024-09-13T08:47:51.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.352+0000 7fcbebe00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbcc074150 0x7fcbcc076610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.356+0000 7fcbebe00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbcc074150 0x7fcbcc076610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc003c80 tx=0x7fcbdc0079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:51.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.360+0000 7fcbe3e00640 1 -- 172.21.15.44:0/2320974245 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcbe4016990 con 0x7fcbec160490 2024-09-13T08:47:51.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.412+0000 7fcbe3e00640 1 -- 172.21.15.44:0/2320974245 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 37) ==== 98865+0+0 (secure 0 0 0) 0x7fcbe40fd810 con 0x7fcbec160490 2024-09-13T08:47:51.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:51.760+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2320974245 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone25122", "target": ["mon-mgr", ""]}) -- 0x7fcbc4000c90 con 0x7fcbcc074150 2024-09-13T08:47:52.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.128+0000 7fcbe3e00640 1 -- 172.21.15.44:0/2320974245 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcbc4000c90 con 0x7fcbcc074150 2024-09-13T08:47:52.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbcc074150 msgr2=0x7fcbcc076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:52.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbcc074150 0x7fcbcc076610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc003c80 tx=0x7fcbdc0079c0 comp rx=0 tx=0).stop 2024-09-13T08:47:52.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2320974245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec160490 msgr2=0x7fcbec152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:52.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec160490 0x7fcbec152f40 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009bd0 tx=0x7fcbe40ae600 comp rx=0 tx=0).stop 2024-09-13T08:47:52.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2320974245 shutdown_connections 2024-09-13T08:47:52.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbcc074150 0x7fcbcc076610 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:52.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec160490 0x7fcbec152f40 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:52.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbec15f9a0 0x7fcbec0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:52.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 --2- 172.21.15.44:0/2320974245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbec0cf1d0 0x7fcbec15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:52.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2320974245 >> 172.21.15.44:0/2320974245 conn(0x7fcbec0b18f0 msgr2=0x7fcbec15fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:52.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2320974245 shutdown_connections 2024-09-13T08:47:52.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.136+0000 7fcbf2e00640 1 -- 172.21.15.44:0/2320974245 wait complete. 2024-09-13T08:47:52.160 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:52.159+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:52.160 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:52.159+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:52.160 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:52.159+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:52.160 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:52.159+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:52.160 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:52.159+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:52.175 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone11943 2024-09-13T08:47:52.242 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:52.243+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:52.242 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:52.243+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:52.242 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:52.243+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:52.242 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:52.243+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:52.242 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:47:52.243+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:47:52.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.460+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1709710438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f3415aa80 msgr2=0x7f5f340cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:52.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.460+0000 7f5f3ac00640 1 --2- 172.21.15.44:0/1709710438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f3415aa80 0x7f5f340cba80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009900 tx=0x7f5f2801c8b0 comp rx=0 tx=0).stop 2024-09-13T08:47:52.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.460+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1709710438 shutdown_connections 2024-09-13T08:47:52.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.460+0000 7f5f3ac00640 1 --2- 172.21.15.44:0/1709710438 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f340cdc60 0x7f5f3406f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:52.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.460+0000 7f5f3ac00640 1 --2- 172.21.15.44:0/1709710438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f3415aa80 0x7f5f340cba80 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:52.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.460+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1709710438 >> 172.21.15.44:0/1709710438 conn(0x7f5f340b26f0 msgr2=0x7f5f340b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:52.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.460+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1709710438 shutdown_connections 2024-09-13T08:47:52.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.460+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1709710438 wait complete. 2024-09-13T08:47:52.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.464+0000 7f5f3ac00640 1 Processor -- start 2024-09-13T08:47:52.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.464+0000 7f5f3ac00640 1 -- start start 2024-09-13T08:47:52.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.464+0000 7f5f3ac00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f340cdc60 0x7f5f340c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:52.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.464+0000 7f5f3ac00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f340cf120 0x7f5f3406cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:52.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.464+0000 7f5f3ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f3415aa80 0x7f5f3406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:52.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.464+0000 7f5f3ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34167770 con 0x7f5f3415aa80 2024-09-13T08:47:52.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.464+0000 7f5f3ac00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341675f0 con 0x7f5f340cdc60 2024-09-13T08:47:52.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.464+0000 7f5f3ac00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5f341678f0 con 0x7f5f340cf120 2024-09-13T08:47:52.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.468+0000 7f5f3a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f3415aa80 0x7f5f3406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:52.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.468+0000 7f5f3a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f3415aa80 0x7f5f3406d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54008/0 (socket says 172.21.15.44:54008) 2024-09-13T08:47:52.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.468+0000 7f5f3a200640 1 -- 172.21.15.44:0/1689483097 learned_addr learned my addr 172.21.15.44:0/1689483097 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.468+0000 7f5f39800640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f340cdc60 0x7f5f340c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f38e00640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f340cf120 0x7f5f3406cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f39800640 1 -- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f340cf120 msgr2=0x7f5f3406cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f39800640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f340cf120 0x7f5f3406cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f39800640 1 -- 172.21.15.44:0/1689483097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f3415aa80 msgr2=0x7f5f3406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f39800640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f3415aa80 0x7f5f3406d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f39800640 1 -- 172.21.15.44:0/1689483097 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f340b5a70 con 0x7f5f340cdc60 2024-09-13T08:47:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f39800640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f340cdc60 0x7f5f340c3560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5f280ac000 tx=0x7f5f280a5770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:52.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f38e00640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f340cf120 0x7f5f3406cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:47:52.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f31600640 1 -- 172.21.15.44:0/1689483097 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f280bc020 con 0x7f5f340cdc60 2024-09-13T08:47:52.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f31600640 1 -- 172.21.15.44:0/1689483097 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f28003a30 con 0x7f5f340cdc60 2024-09-13T08:47:52.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1689483097 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f340b5d10 con 0x7f5f340cdc60 2024-09-13T08:47:52.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f31600640 1 -- 172.21.15.44:0/1689483097 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f280b2040 con 0x7f5f340cdc60 2024-09-13T08:47:52.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.472+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1689483097 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f340b62d0 con 0x7f5f340cdc60 2024-09-13T08:47:52.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.480+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1689483097 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3406f8d0 con 0x7f5f340cdc60 2024-09-13T08:47:52.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.488+0000 7f5f31600640 1 -- 172.21.15.44:0/1689483097 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 37) ==== 98865+0+0 (secure 0 0 0) 0x7f5f280a5970 con 0x7f5f340cdc60 2024-09-13T08:47:52.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.488+0000 7f5f31600640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f1407cb50 0x7f5f1407f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:52.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.488+0000 7f5f31600640 1 -- 172.21.15.44:0/1689483097 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f5f2812d8e0 con 0x7f5f340cdc60 2024-09-13T08:47:52.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.492+0000 7f5f38e00640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f1407cb50 0x7f5f1407f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:52.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.492+0000 7f5f31600640 1 -- 172.21.15.44:0/1689483097 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5f280c7080 con 0x7f5f340cdc60 2024-09-13T08:47:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.500+0000 7f5f38e00640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f1407cb50 0x7f5f1407f010 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5f34157300 tx=0x7f5f20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:52.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:52.872+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1689483097 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone11943", "target": ["mon-mgr", ""]}) -- 0x7f5f34157c70 con 0x7f5f1407cb50 2024-09-13T08:47:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.344+0000 7f5f31600640 1 -- 172.21.15.44:0/1689483097 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5f34157c70 con 0x7f5f1407cb50 2024-09-13T08:47:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f1407cb50 msgr2=0x7f5f1407f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f1407cb50 0x7f5f1407f010 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5f34157300 tx=0x7f5f20009040 comp rx=0 tx=0).stop 2024-09-13T08:47:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f340cdc60 msgr2=0x7f5f340c3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f340cdc60 0x7f5f340c3560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5f280ac000 tx=0x7f5f280a5770 comp rx=0 tx=0).stop 2024-09-13T08:47:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1689483097 shutdown_connections 2024-09-13T08:47:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f1407cb50 0x7f5f1407f010 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f3415aa80 0x7f5f3406d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:53.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f340cf120 0x7f5f3406cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:53.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 --2- 172.21.15.44:0/1689483097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f340cdc60 0x7f5f340c3560 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:53.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1689483097 >> 172.21.15.44:0/1689483097 conn(0x7f5f340b26f0 msgr2=0x7f5f340b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:53.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1689483097 shutdown_connections 2024-09-13T08:47:53.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.348+0000 7f5f3ac00640 1 -- 172.21.15.44:0/1689483097 wait complete. 2024-09-13T08:47:53.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone25384 2024-09-13T08:47:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.652+0000 7fe445800640 1 -- 172.21.15.44:0/1921153625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe424003a10 msgr2=0x7fe424003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.652+0000 7fe445800640 1 --2- 172.21.15.44:0/1921153625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe424003a10 0x7fe424003e10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009940 tx=0x7fe42c01c760 comp rx=0 tx=0).stop 2024-09-13T08:47:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.652+0000 7fe445800640 1 -- 172.21.15.44:0/1921153625 shutdown_connections 2024-09-13T08:47:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.652+0000 7fe445800640 1 --2- 172.21.15.44:0/1921153625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe424003a10 0x7fe424003e10 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.652+0000 7fe445800640 1 -- 172.21.15.44:0/1921153625 >> 172.21.15.44:0/1921153625 conn(0x7fe4400d0cf0 msgr2=0x7fe4400d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:53.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.652+0000 7fe445800640 1 -- 172.21.15.44:0/1921153625 shutdown_connections 2024-09-13T08:47:53.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.652+0000 7fe445800640 1 -- 172.21.15.44:0/1921153625 wait complete. 2024-09-13T08:47:53.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.656+0000 7fe445800640 1 Processor -- start 2024-09-13T08:47:53.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.656+0000 7fe445800640 1 -- start start 2024-09-13T08:47:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe445800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe424003a10 0x7fe44015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe445800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe440149390 0x7fe4400b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe445800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4400b0c60 0x7fe4400b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe445800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe44006cc20 con 0x7fe4400b0c60 2024-09-13T08:47:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe445800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe44015f6d0 con 0x7fe424003a10 2024-09-13T08:47:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe445800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe44006cda0 con 0x7fe440149390 2024-09-13T08:47:53.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe43f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4400b0c60 0x7fe4400b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:53.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe43f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4400b0c60 0x7fe4400b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54030/0 (socket says 172.21.15.44:54030) 2024-09-13T08:47:53.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe43f400640 1 -- 172.21.15.44:0/533065189 learned_addr learned my addr 172.21.15.44:0/533065189 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe43f400640 1 -- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe440149390 msgr2=0x7fe4400b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe43f400640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe440149390 0x7fe4400b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe43f400640 1 -- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe424003a10 msgr2=0x7fe44015a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe43f400640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe424003a10 0x7fe44015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe43f400640 1 -- 172.21.15.44:0/533065189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4400b1c60 con 0x7fe4400b0c60 2024-09-13T08:47:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe43f400640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4400b0c60 0x7fe4400b10e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe438001cb0 tx=0x7fe43800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe436a00640 1 -- 172.21.15.44:0/533065189 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438017070 con 0x7fe4400b0c60 2024-09-13T08:47:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe436a00640 1 -- 172.21.15.44:0/533065189 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4380050d0 con 0x7fe4400b0c60 2024-09-13T08:47:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe445800640 1 -- 172.21.15.44:0/533065189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4400b6310 con 0x7fe4400b0c60 2024-09-13T08:47:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe436a00640 1 -- 172.21.15.44:0/533065189 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438005560 con 0x7fe4400b0c60 2024-09-13T08:47:53.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.660+0000 7fe445800640 1 -- 172.21.15.44:0/533065189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4400b68d0 con 0x7fe4400b0c60 2024-09-13T08:47:53.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.664+0000 7fe445800640 1 -- 172.21.15.44:0/533065189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe440074130 con 0x7fe4400b0c60 2024-09-13T08:47:53.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.664+0000 7fe436a00640 1 -- 172.21.15.44:0/533065189 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7fe43801a070 con 0x7fe4400b0c60 2024-09-13T08:47:53.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.668+0000 7fe436a00640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe420074420 0x7fe4200768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:53.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.668+0000 7fe436a00640 1 -- 172.21.15.44:0/533065189 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fe438092550 con 0x7fe4400b0c60 2024-09-13T08:47:53.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.672+0000 7fe43ea00640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe420074420 0x7fe4200768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:53.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.676+0000 7fe436a00640 1 -- 172.21.15.44:0/533065189 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe43805ee50 con 0x7fe4400b0c60 2024-09-13T08:47:53.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:53.680+0000 7fe43ea00640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe420074420 0x7fe4200768e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0ae000 tx=0x7fe42c0b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.140+0000 7fe445800640 1 -- 172.21.15.44:0/533065189 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone25384", "target": ["mon-mgr", ""]}) -- 0x7fe4400cb0b0 con 0x7fe420074420 2024-09-13T08:47:54.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.808+0000 7fe436a00640 1 -- 172.21.15.44:0/533065189 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe4400cb0b0 con 0x7fe420074420 2024-09-13T08:47:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 -- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe420074420 msgr2=0x7fe4200768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe420074420 0x7fe4200768e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0ae000 tx=0x7fe42c0b3040 comp rx=0 tx=0).stop 2024-09-13T08:47:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 -- 172.21.15.44:0/533065189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4400b0c60 msgr2=0x7fe4400b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4400b0c60 0x7fe4400b10e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe438001cb0 tx=0x7fe43800cf00 comp rx=0 tx=0).stop 2024-09-13T08:47:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 -- 172.21.15.44:0/533065189 shutdown_connections 2024-09-13T08:47:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe420074420 0x7fe4200768e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4400b0c60 0x7fe4400b10e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe440149390 0x7fe4400b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 --2- 172.21.15.44:0/533065189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe424003a10 0x7fe44015a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 -- 172.21.15.44:0/533065189 >> 172.21.15.44:0/533065189 conn(0x7fe4400d0cf0 msgr2=0x7fe44015c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 -- 172.21.15.44:0/533065189 shutdown_connections 2024-09-13T08:47:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:54.812+0000 7fe445800640 1 -- 172.21.15.44:0/533065189 wait complete. 2024-09-13T08:47:54.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:47:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.100+0000 7fa525a00640 1 -- 172.21.15.44:0/2427266461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5200d1c00 msgr2=0x7fa5200d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.100+0000 7fa525a00640 1 --2- 172.21.15.44:0/2427266461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5200d1c00 0x7fa5200d20b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa52006dd00 tx=0x7fa51801cd20 comp rx=0 tx=0).stop 2024-09-13T08:47:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.100+0000 7fa525a00640 1 -- 172.21.15.44:0/2427266461 shutdown_connections 2024-09-13T08:47:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.100+0000 7fa525a00640 1 --2- 172.21.15.44:0/2427266461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5200d1c00 0x7fa5200d20b0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.100+0000 7fa525a00640 1 --2- 172.21.15.44:0/2427266461 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5200cc780 0x7fa520160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.100+0000 7fa525a00640 1 --2- 172.21.15.44:0/2427266461 >> v2:172.21.15.129:3300/0 conn(0x7fa52015bea0 0x7fa5200d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.100+0000 7fa525a00640 1 -- 172.21.15.44:0/2427266461 >> 172.21.15.44:0/2427266461 conn(0x7fa52009aa10 msgr2=0x7fa52009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.100+0000 7fa525a00640 1 -- 172.21.15.44:0/2427266461 shutdown_connections 2024-09-13T08:47:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.100+0000 7fa525a00640 1 -- 172.21.15.44:0/2427266461 wait complete. 2024-09-13T08:47:55.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.104+0000 7fa525a00640 1 Processor -- start 2024-09-13T08:47:55.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.104+0000 7fa525a00640 1 -- start start 2024-09-13T08:47:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.108+0000 7fa525a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5200cc780 0x7fa5200c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.108+0000 7fa525a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5200d1c00 0x7fa520144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.108+0000 7fa525a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa52015bea0 0x7fa520145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.108+0000 7fa525a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5200b3560 con 0x7fa5200d1c00 2024-09-13T08:47:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.108+0000 7fa525a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa5200b33e0 con 0x7fa5200cc780 2024-09-13T08:47:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.108+0000 7fa525a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa5200ce7d0 con 0x7fa52015bea0 2024-09-13T08:47:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa51e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5200d1c00 0x7fa520144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa51e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5200d1c00 0x7fa520144c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54042/0 (socket says 172.21.15.44:54042) 2024-09-13T08:47:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa51e000640 1 -- 172.21.15.44:0/4077371853 learned_addr learned my addr 172.21.15.44:0/4077371853 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa51f400640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa52015bea0 0x7fa520145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa51e000640 1 -- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa52015bea0 msgr2=0x7fa520145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa51e000640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa52015bea0 0x7fa520145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa51e000640 1 -- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5200cc780 msgr2=0x7fa5200c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa51e000640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5200cc780 0x7fa5200c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa51e000640 1 -- 172.21.15.44:0/4077371853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5200c50e0 con 0x7fa5200d1c00 2024-09-13T08:47:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa51e000640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5200d1c00 0x7fa520144c40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00d750 tx=0x7fa50c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa516a00640 1 -- 172.21.15.44:0/4077371853 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c01c070 con 0x7fa5200d1c00 2024-09-13T08:47:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa516a00640 1 -- 172.21.15.44:0/4077371853 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa50c01a070 con 0x7fa5200d1c00 2024-09-13T08:47:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa516a00640 1 -- 172.21.15.44:0/4077371853 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c004290 con 0x7fa5200d1c00 2024-09-13T08:47:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa525a00640 1 -- 172.21.15.44:0/4077371853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52006efe0 con 0x7fa5200d1c00 2024-09-13T08:47:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.112+0000 7fa525a00640 1 -- 172.21.15.44:0/4077371853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52006f5a0 con 0x7fa5200d1c00 2024-09-13T08:47:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.116+0000 7fa525a00640 1 -- 172.21.15.44:0/4077371853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5200d20b0 con 0x7fa5200d1c00 2024-09-13T08:47:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.120+0000 7fa516a00640 1 -- 172.21.15.44:0/4077371853 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7fa50c0097c0 con 0x7fa5200d1c00 2024-09-13T08:47:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.120+0000 7fa516a00640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa500074420 0x7fa5000768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.120+0000 7fa516a00640 1 -- 172.21.15.44:0/4077371853 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fa50c09b680 con 0x7fa5200d1c00 2024-09-13T08:47:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.128+0000 7fa51ea00640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa500074420 0x7fa5000768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:55.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.144+0000 7fa51ea00640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa500074420 0x7fa5000768e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa5100058d0 tx=0x7fa510005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:55.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.148+0000 7fa516a00640 1 -- 172.21.15.44:0/4077371853 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa50c067e90 con 0x7fa5200d1c00 2024-09-13T08:47:55.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.608+0000 7fa525a00640 1 -- 172.21.15.44:0/4077371853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa5200cdce0 con 0x7fa5200d1c00 2024-09-13T08:47:55.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.612+0000 7fa516a00640 1 -- 172.21.15.44:0/4077371853 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2181 (secure 0 0 0) 0x7fa50c067790 con 0x7fa5200d1c00 2024-09-13T08:47:55.613 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:55.613 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:47:55.613 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:47:55.613 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:47:55.613 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:47:55.613 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:47:55.613 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:55.614 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 646, 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 89, 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:47:55.615 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:47:55.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:55.616 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:47:55.616 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:47:55.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:55.616 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:47:55.616 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:47:55.616 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:55.616 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:55.616 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:47:55.616 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:47:55.617 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 1229, 2024-09-13T08:47:55.617 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 5004913314, 2024-09-13T08:47:55.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 9678114816, 2024-09-13T08:47:55.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 763415998464, 2024-09-13T08:47:55.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:47:55.617 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 159693289, 2024-09-13T08:47:55.617 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 237999892, 2024-09-13T08:47:55.617 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 39, 2024-09-13T08:47:55.617 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 66 2024-09-13T08:47:55.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 57, 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:47:15:289304+0000", 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 6, 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 6, 2024-09-13T08:47:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 6942 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:47:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 15, 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:46:40.680139+0000", 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": { 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-ongoing-clones": { 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "3 ongoing clones - average progress is 15.538% (2s)\n [====........................] (remaining: 15s)", 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.1553799957036972, 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:47:55.620 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:55.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:47:55.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 -- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa500074420 msgr2=0x7fa5000768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:55.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa500074420 0x7fa5000768e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa5100058d0 tx=0x7fa510005860 comp rx=0 tx=0).stop 2024-09-13T08:47:55.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 -- 172.21.15.44:0/4077371853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5200d1c00 msgr2=0x7fa520144c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:55.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5200d1c00 0x7fa520144c40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00d750 tx=0x7fa50c00dc10 comp rx=0 tx=0).stop 2024-09-13T08:47:55.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 -- 172.21.15.44:0/4077371853 shutdown_connections 2024-09-13T08:47:55.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa500074420 0x7fa5000768e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:55.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa52015bea0 0x7fa520145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:55.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5200d1c00 0x7fa520144c40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:55.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 --2- 172.21.15.44:0/4077371853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5200cc780 0x7fa5200c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:55.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 -- 172.21.15.44:0/4077371853 >> 172.21.15.44:0/4077371853 conn(0x7fa52009aa10 msgr2=0x7fa5200ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:55.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 -- 172.21.15.44:0/4077371853 shutdown_connections 2024-09-13T08:47:55.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:55.616+0000 7fa525a00640 1 -- 172.21.15.44:0/4077371853 wait complete. 2024-09-13T08:47:56.660 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:47:56.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 -- 172.21.15.44:0/4245926398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31cc15f4d0 msgr2=0x7f31cc0b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:56.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 --2- 172.21.15.44:0/4245926398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31cc15f4d0 0x7f31cc0b1270 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0d0e30 tx=0x7f31bc01c770 comp rx=0 tx=0).stop 2024-09-13T08:47:56.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 -- 172.21.15.44:0/4245926398 shutdown_connections 2024-09-13T08:47:56.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 --2- 172.21.15.44:0/4245926398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31cc15f4d0 0x7f31cc0b1270 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:56.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 --2- 172.21.15.44:0/4245926398 >> v2:172.21.15.129:3300/0 conn(0x7f31cc0b2b50 0x7f31cc0d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:56.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 --2- 172.21.15.44:0/4245926398 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31cc15f0f0 0x7f31cc0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:56.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 -- 172.21.15.44:0/4245926398 >> 172.21.15.44:0/4245926398 conn(0x7f31cc0d1c20 msgr2=0x7f31cc0d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:56.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 -- 172.21.15.44:0/4245926398 shutdown_connections 2024-09-13T08:47:56.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 -- 172.21.15.44:0/4245926398 wait complete. 2024-09-13T08:47:56.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 Processor -- start 2024-09-13T08:47:56.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.892+0000 7f31d2800640 1 -- start start 2024-09-13T08:47:56.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d2800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31cc0b2b50 0x7f31cc0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:56.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31cc15f0f0 0x7f31cc14ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:56.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d2800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31cc15f4d0 0x7f31cc1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:56.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d2800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc0afd10 con 0x7f31cc15f0f0 2024-09-13T08:47:56.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d2800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc0afb90 con 0x7f31cc15f4d0 2024-09-13T08:47:56.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d2800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f31cc0afe90 con 0x7f31cc0b2b50 2024-09-13T08:47:56.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31cc15f0f0 0x7f31cc14ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:56.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31cc15f0f0 0x7f31cc14ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54064/0 (socket says 172.21.15.44:54064) 2024-09-13T08:47:56.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d0a00640 1 -- 172.21.15.44:0/1545245035 learned_addr learned my addr 172.21.15.44:0/1545245035 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:56.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d1400640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31cc0b2b50 0x7f31cc0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:56.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d0a00640 1 -- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31cc0b2b50 msgr2=0x7f31cc0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:56.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d0a00640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31cc0b2b50 0x7f31cc0ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:56.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d0a00640 1 -- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31cc15f4d0 msgr2=0x7f31cc1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:56.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d0a00640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31cc15f4d0 0x7f31cc1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:56.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d0a00640 1 -- 172.21.15.44:0/1545245035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31cc0b4a50 con 0x7f31cc15f0f0 2024-09-13T08:47:56.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d0a00640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31cc15f0f0 0x7f31cc14ade0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f31b800b480 tx=0x7f31b800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:56.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31c9600640 1 -- 172.21.15.44:0/1545245035 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8014070 con 0x7f31cc15f0f0 2024-09-13T08:47:56.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31c9600640 1 -- 172.21.15.44:0/1545245035 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31b8010980 con 0x7f31cc15f0f0 2024-09-13T08:47:56.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d2800640 1 -- 172.21.15.44:0/1545245035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31cc0b4d30 con 0x7f31cc15f0f0 2024-09-13T08:47:56.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31c9600640 1 -- 172.21.15.44:0/1545245035 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8010dd0 con 0x7f31cc15f0f0 2024-09-13T08:47:56.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d2800640 1 -- 172.21.15.44:0/1545245035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31cc0b55d0 con 0x7f31cc15f0f0 2024-09-13T08:47:56.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.896+0000 7f31d2800640 1 -- 172.21.15.44:0/1545245035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31cc0b1270 con 0x7f31cc15f0f0 2024-09-13T08:47:56.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.900+0000 7f31c9600640 1 -- 172.21.15.44:0/1545245035 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f31b8017070 con 0x7f31cc15f0f0 2024-09-13T08:47:56.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.900+0000 7f31c9600640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31ac074420 0x7f31ac0768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:56.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.900+0000 7f31c9600640 1 -- 172.21.15.44:0/1545245035 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f31b80931c0 con 0x7f31cc15f0f0 2024-09-13T08:47:56.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.904+0000 7f31d1400640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31ac074420 0x7f31ac0768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:56.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.904+0000 7f31c9600640 1 -- 172.21.15.44:0/1545245035 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f31b805fa60 con 0x7f31cc15f0f0 2024-09-13T08:47:56.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:56.912+0000 7f31d1400640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31ac074420 0x7f31ac0768e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009f20 tx=0x7f31c40094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:57.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.388+0000 7f31d2800640 1 -- 172.21.15.44:0/1545245035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f31cc06e420 con 0x7f31cc15f0f0 2024-09-13T08:47:57.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31c9600640 1 -- 172.21.15.44:0/1545245035 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2181 (secure 0 0 0) 0x7f31b805f360 con 0x7f31cc15f0f0 2024-09-13T08:47:57.393 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:57.393 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:47:57.393 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:47:57.394 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:47:57.394 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:47:57.394 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:47:57.394 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:47:57.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:57.394 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:47:57.395 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:47:57.395 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:47:57.395 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:47:57.395 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:47:57.395 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:57.396 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:47:57.396 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:47:57.396 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:47:57.396 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:47:57.396 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:57.396 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 648, 2024-09-13T08:47:57.397 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:47:57.397 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:47:57.397 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:47:57.397 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:47:57.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:57.398 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:47:57.398 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 89, 2024-09-13T08:47:57.398 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:47:57.399 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:47:57.399 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:47:57.399 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:47:57.399 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:47:57.399 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:47:57.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:57.400 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:47:57.400 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:47:57.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:57.400 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:47:57.401 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:47:57.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:57.401 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:57.401 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:47:57.401 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:47:57.401 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 1229, 2024-09-13T08:47:57.402 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 5004913314, 2024-09-13T08:47:57.402 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 9678114816, 2024-09-13T08:47:57.402 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 763415998464, 2024-09-13T08:47:57.402 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:47:57.402 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 159693289, 2024-09-13T08:47:57.402 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 237999892, 2024-09-13T08:47:57.403 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 39, 2024-09-13T08:47:57.403 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 66 2024-09-13T08:47:57.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:57.403 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:47:57.403 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 57, 2024-09-13T08:47:57.404 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:47:15:289304+0000", 2024-09-13T08:47:57.404 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 6, 2024-09-13T08:47:57.404 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:47:57.404 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:47:57.404 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:47:57.404 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:47:57.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:57.405 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 6, 2024-09-13T08:47:57.405 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:47:57.405 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:47:57.405 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:47:57.406 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 6942 2024-09-13T08:47:57.406 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:57.406 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:57.406 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:47:57.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:57.406 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:47:57.407 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:47:57.407 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:47:57.407 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:47:57.407 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:47:57.407 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:47:57.408 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:47:57.408 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:57.408 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:47:57.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:57.408 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:47:57.409 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 15, 2024-09-13T08:47:57.409 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:46:40.680139+0000", 2024-09-13T08:47:57.409 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:47:57.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:57.409 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": { 2024-09-13T08:47:57.409 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-ongoing-clones": { 2024-09-13T08:47:57.409 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "3 ongoing clones - average progress is 15.538% (2s)\n [====........................] (remaining: 15s)", 2024-09-13T08:47:57.409 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.1553799957036972, 2024-09-13T08:47:57.410 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:47:57.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:57.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:57.410 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:47:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 -- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31ac074420 msgr2=0x7f31ac0768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31ac074420 0x7f31ac0768e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009f20 tx=0x7f31c40094f0 comp rx=0 tx=0).stop 2024-09-13T08:47:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 -- 172.21.15.44:0/1545245035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31cc15f0f0 msgr2=0x7f31cc14ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31cc15f0f0 0x7f31cc14ade0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f31b800b480 tx=0x7f31b800b940 comp rx=0 tx=0).stop 2024-09-13T08:47:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 -- 172.21.15.44:0/1545245035 shutdown_connections 2024-09-13T08:47:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31ac074420 0x7f31ac0768e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31cc15f4d0 0x7f31cc1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31cc15f0f0 0x7f31cc14ade0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 --2- 172.21.15.44:0/1545245035 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31cc0b2b50 0x7f31cc0ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 -- 172.21.15.44:0/1545245035 >> 172.21.15.44:0/1545245035 conn(0x7f31cc0d1c20 msgr2=0x7f31cc15de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:57.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 -- 172.21.15.44:0/1545245035 shutdown_connections 2024-09-13T08:47:57.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:57.392+0000 7f31d2800640 1 -- 172.21.15.44:0/1545245035 wait complete. 2024-09-13T08:47:58.434 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:47:58.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.712+0000 7fed26e00640 1 -- 172.21.15.44:0/1105083255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed2015a150 msgr2=0x7fed2015b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:58.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.712+0000 7fed26e00640 1 --2- 172.21.15.44:0/1105083255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed2015a150 0x7fed2015b730 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fed200af7f0 tx=0x7fed1001db30 comp rx=0 tx=0).stop 2024-09-13T08:47:58.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.712+0000 7fed26e00640 1 -- 172.21.15.44:0/1105083255 shutdown_connections 2024-09-13T08:47:58.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.712+0000 7fed26e00640 1 --2- 172.21.15.44:0/1105083255 >> v2:172.21.15.129:3300/0 conn(0x7fed2015e8d0 0x7fed2015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:58.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.712+0000 7fed26e00640 1 --2- 172.21.15.44:0/1105083255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed2015a150 0x7fed2015b730 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:58.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.712+0000 7fed26e00640 1 -- 172.21.15.44:0/1105083255 >> 172.21.15.44:0/1105083255 conn(0x7fed200b2060 msgr2=0x7fed200b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:58.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.712+0000 7fed26e00640 1 -- 172.21.15.44:0/1105083255 shutdown_connections 2024-09-13T08:47:58.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.712+0000 7fed26e00640 1 -- 172.21.15.44:0/1105083255 wait complete. 2024-09-13T08:47:58.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed26e00640 1 Processor -- start 2024-09-13T08:47:58.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed26e00640 1 -- start start 2024-09-13T08:47:58.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed26e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed200ce9b0 0x7fed200b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:58.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed26e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed2015a150 0x7fed2014be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:58.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed26e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed2015e8d0 0x7fed2015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:58.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed26e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed20073c60 con 0x7fed2015e8d0 2024-09-13T08:47:58.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed26e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed20073ae0 con 0x7fed2015a150 2024-09-13T08:47:58.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed26e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fed20073de0 con 0x7fed200ce9b0 2024-09-13T08:47:58.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed24e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed2015e8d0 0x7fed2015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:58.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed24e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed2015e8d0 0x7fed2015d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54070/0 (socket says 172.21.15.44:54070) 2024-09-13T08:47:58.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed24e00640 1 -- 172.21.15.44:0/1411699505 learned_addr learned my addr 172.21.15.44:0/1411699505 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:58.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed1f400640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed2015a150 0x7fed2014be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:58.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.716+0000 7fed1fe00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed200ce9b0 0x7fed200b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:58.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed1f400640 1 -- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed200ce9b0 msgr2=0x7fed200b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:58.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed1f400640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed200ce9b0 0x7fed200b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:58.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed1f400640 1 -- 172.21.15.44:0/1411699505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed2015e8d0 msgr2=0x7fed2015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:58.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed1f400640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed2015e8d0 0x7fed2015d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:58.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed1f400640 1 -- 172.21.15.44:0/1411699505 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2015c210 con 0x7fed2015a150 2024-09-13T08:47:58.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed1fe00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed200ce9b0 0x7fed200b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:47:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed1f400640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed2015a150 0x7fed2014be50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00b410 tx=0x7fed0c00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed17e00640 1 -- 172.21.15.44:0/1411699505 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c015020 con 0x7fed2015a150 2024-09-13T08:47:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed17e00640 1 -- 172.21.15.44:0/1411699505 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed0c011920 con 0x7fed2015a150 2024-09-13T08:47:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed17e00640 1 -- 172.21.15.44:0/1411699505 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c011d70 con 0x7fed2015a150 2024-09-13T08:47:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed26e00640 1 -- 172.21.15.44:0/1411699505 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2015c4f0 con 0x7fed2015a150 2024-09-13T08:47:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.720+0000 7fed26e00640 1 -- 172.21.15.44:0/1411699505 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2015cab0 con 0x7fed2015a150 2024-09-13T08:47:58.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.724+0000 7fed26e00640 1 -- 172.21.15.44:0/1411699505 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2015b730 con 0x7fed2015a150 2024-09-13T08:47:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.736+0000 7fed17e00640 1 -- 172.21.15.44:0/1411699505 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7fed0c018020 con 0x7fed2015a150 2024-09-13T08:47:58.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.736+0000 7fed17e00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fed0007cc30 0x7fed0007f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:58.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.736+0000 7fed17e00640 1 -- 172.21.15.44:0/1411699505 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fed0c093370 con 0x7fed2015a150 2024-09-13T08:47:58.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.744+0000 7fed17e00640 1 -- 172.21.15.44:0/1411699505 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fed0c01f020 con 0x7fed2015a150 2024-09-13T08:47:58.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.756+0000 7fed1fe00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fed0007cc30 0x7fed0007f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:58.776+0000 7fed1fe00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fed0007cc30 0x7fed0007f0f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fed100b0000 tx=0x7fed1001c370 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:59.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.240+0000 7fed26e00640 1 -- 172.21.15.44:0/1411699505 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fed2014c390 con 0x7fed2015a150 2024-09-13T08:47:59.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.248+0000 7fed17e00640 1 -- 172.21.15.44:0/1411699505 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2437 (secure 0 0 0) 0x7fed0c05fc10 con 0x7fed2015a150 2024-09-13T08:47:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:47:59.256 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:47:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:47:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:47:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:47:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:47:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:47:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:47:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:47:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:47:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:47:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:47:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:47:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:47:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:47:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:47:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 649, 2024-09-13T08:47:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:47:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:47:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:47:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:47:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:47:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 89, 2024-09-13T08:47:59.259 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:47:59.259 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:47:59.259 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 1362, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 5566949862, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 10726756352, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 762367356928, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 204764662, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 248824360, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 50, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 69 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:47:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 57, 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:47:15:289304+0000", 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 6, 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 6, 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:47:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 6942 2024-09-13T08:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 15, 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:46:40.680139+0000", 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": { 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-ongoing-clones": { 2024-09-13T08:47:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "4 ongoing clones - average progress is 27.734% (6s)\n [=======.....................] (remaining: 18s)", 2024-09-13T08:47:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.27733999490737915, 2024-09-13T08:47:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:47:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:47:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-total-clones": { 2024-09-13T08:47:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "Total 5 clones - average progress is 25.885% (1s)\n [=======.....................] (remaining: 5s)", 2024-09-13T08:47:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.25885000824928284, 2024-09-13T08:47:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:47:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:47:59.267 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.256+0000 7fed26e00640 1 -- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fed0007cc30 msgr2=0x7fed0007f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.256+0000 7fed26e00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fed0007cc30 0x7fed0007f0f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fed100b0000 tx=0x7fed1001c370 comp rx=0 tx=0).stop 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.256+0000 7fed26e00640 1 -- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed2015a150 msgr2=0x7fed2014be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.256+0000 7fed26e00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed2015a150 0x7fed2014be50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00b410 tx=0x7fed0c00b810 comp rx=0 tx=0).stop 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.260+0000 7fed26e00640 1 -- 172.21.15.44:0/1411699505 shutdown_connections 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.260+0000 7fed26e00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fed0007cc30 0x7fed0007f0f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.260+0000 7fed26e00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed2015e8d0 0x7fed2015d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.260+0000 7fed26e00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed2015a150 0x7fed2014be50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.260+0000 7fed26e00640 1 --2- 172.21.15.44:0/1411699505 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed200ce9b0 0x7fed200b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.260+0000 7fed26e00640 1 -- 172.21.15.44:0/1411699505 >> 172.21.15.44:0/1411699505 conn(0x7fed200b2060 msgr2=0x7fed20070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.264+0000 7fed26e00640 1 -- 172.21.15.44:0/1411699505 shutdown_connections 2024-09-13T08:47:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.268+0000 7fed26e00640 1 -- 172.21.15.44:0/1411699505 wait complete. 2024-09-13T08:47:59.312 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 27.734% (6s)\n [=======.....................] (remaining: 18s)', 'progress': 0.27733999490737915, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 5 clones - average progress is 25.885% (1s)\n [=======.....................] (remaining: 5s)', 'progress': 0.25885000824928284, 'add_to_ceph_s': True}} 2024-09-13T08:47:59.312 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone71215 2024-09-13T08:47:59.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.576+0000 7f46e5400640 1 -- 172.21.15.44:0/1037895621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46c4004810 msgr2=0x7f46c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:59.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.576+0000 7f46e5400640 1 --2- 172.21.15.44:0/1037895621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46c4004810 0x7f46c4004c90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c4a0 tx=0x7f46cc01cf50 comp rx=0 tx=0).stop 2024-09-13T08:47:59.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.576+0000 7f46df200640 1 -- 172.21.15.44:0/1037895621 reap_dead start 2024-09-13T08:47:59.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.576+0000 7f46e5400640 1 -- 172.21.15.44:0/1037895621 shutdown_connections 2024-09-13T08:47:59.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.576+0000 7f46e5400640 1 -- 172.21.15.44:0/1037895621 >> 172.21.15.44:0/1037895621 conn(0x7f46e00d3f90 msgr2=0x7f46e00d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:47:59.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.576+0000 7f46e5400640 1 -- 172.21.15.44:0/1037895621 shutdown_connections 2024-09-13T08:47:59.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.576+0000 7f46e5400640 1 -- 172.21.15.44:0/1037895621 wait complete. 2024-09-13T08:47:59.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.576+0000 7f46e5400640 1 Processor -- start 2024-09-13T08:47:59.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.576+0000 7f46e5400640 1 -- start start 2024-09-13T08:47:59.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46e5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46c4004810 0x7f46e01fb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:59.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46e5400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f46e00a15f0 0x7f46e00b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:59.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46e5400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46e00b74f0 0x7f46e01faca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:59.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46e5400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46e00d2ea0 con 0x7f46c4004810 2024-09-13T08:47:59.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46e5400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46e00d2d20 con 0x7f46e00b74f0 2024-09-13T08:47:59.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46e5400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f46e016a6d0 con 0x7f46e00a15f0 2024-09-13T08:47:59.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46de800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46c4004810 0x7f46e01fb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:59.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46de800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46c4004810 0x7f46e01fb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54086/0 (socket says 172.21.15.44:54086) 2024-09-13T08:47:59.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46de800640 1 -- 172.21.15.44:0/1238855747 learned_addr learned my addr 172.21.15.44:0/1238855747 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:47:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46df200640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46e00b74f0 0x7f46e01faca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46de800640 1 -- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f46e00a15f0 msgr2=0x7f46e00b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46de800640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f46e00a15f0 0x7f46e00b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46de800640 1 -- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46e00b74f0 msgr2=0x7f46e01faca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:47:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46de800640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46e00b74f0 0x7f46e01faca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:47:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46de800640 1 -- 172.21.15.44:0/1238855747 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e01fbe40 con 0x7f46c4004810 2024-09-13T08:47:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46df200640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46e00b74f0 0x7f46e01faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:47:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46de800640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46c4004810 0x7f46e01fb860 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f46d000e950 tx=0x7f46d000ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:59.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46d6a00640 1 -- 172.21.15.44:0/1238855747 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d000cd80 con 0x7f46c4004810 2024-09-13T08:47:59.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46d6a00640 1 -- 172.21.15.44:0/1238855747 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46d0010430 con 0x7f46c4004810 2024-09-13T08:47:59.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46e5400640 1 -- 172.21.15.44:0/1238855747 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e01fc120 con 0x7f46c4004810 2024-09-13T08:47:59.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46d6a00640 1 -- 172.21.15.44:0/1238855747 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d00108a0 con 0x7f46c4004810 2024-09-13T08:47:59.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.580+0000 7f46e5400640 1 -- 172.21.15.44:0/1238855747 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e01fc6e0 con 0x7f46c4004810 2024-09-13T08:47:59.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.584+0000 7f46e5400640 1 -- 172.21.15.44:0/1238855747 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e00cf520 con 0x7f46c4004810 2024-09-13T08:47:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.584+0000 7f46d6a00640 1 -- 172.21.15.44:0/1238855747 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f46d0010a40 con 0x7f46c4004810 2024-09-13T08:47:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.588+0000 7f46d6a00640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f46c0074420 0x7f46c00768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:47:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.588+0000 7f46d6a00640 1 -- 172.21.15.44:0/1238855747 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f46d00936f0 con 0x7f46c4004810 2024-09-13T08:47:59.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.592+0000 7f46dde00640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f46c0074420 0x7f46c00768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:47:59.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.596+0000 7f46dde00640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f46c0074420 0x7f46c00768e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c470 tx=0x7f46c4005840 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:47:59.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.600+0000 7f46d6a00640 1 -- 172.21.15.44:0/1238855747 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f46d00603f0 con 0x7f46c4004810 2024-09-13T08:47:59.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:47:59.960+0000 7f46e5400640 1 -- 172.21.15.44:0/1238855747 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone71215", "target": ["mon-mgr", ""]}) -- 0x7f46e00b9f10 con 0x7f46c0074420 2024-09-13T08:48:00.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.860+0000 7f46d6a00640 1 -- 172.21.15.44:0/1238855747 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f46e00b9f10 con 0x7f46c0074420 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 -- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f46c0074420 msgr2=0x7f46c00768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f46c0074420 0x7f46c00768e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c470 tx=0x7f46c4005840 comp rx=0 tx=0).stop 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 -- 172.21.15.44:0/1238855747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46c4004810 msgr2=0x7f46e01fb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46c4004810 0x7f46e01fb860 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f46d000e950 tx=0x7f46d000ee10 comp rx=0 tx=0).stop 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 -- 172.21.15.44:0/1238855747 shutdown_connections 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f46c0074420 0x7f46c00768e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46e00b74f0 0x7f46e01faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f46e00a15f0 0x7f46e00b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 --2- 172.21.15.44:0/1238855747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46c4004810 0x7f46e01fb860 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 -- 172.21.15.44:0/1238855747 >> 172.21.15.44:0/1238855747 conn(0x7f46e00d3f90 msgr2=0x7f46e00b7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 -- 172.21.15.44:0/1238855747 shutdown_connections 2024-09-13T08:48:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:00.864+0000 7f46e5400640 1 -- 172.21.15.44:0/1238855747 wait complete. 2024-09-13T08:48:00.903 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone42326 2024-09-13T08:48:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.184+0000 7fb24fe00640 1 -- 172.21.15.44:0/2620537487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb24806eeb0 msgr2=0x7fb2480b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.184+0000 7fb24fe00640 1 --2- 172.21.15.44:0/2620537487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb24806eeb0 0x7fb2480b3990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009840 tx=0x7fb23c01c810 comp rx=0 tx=0).stop 2024-09-13T08:48:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.184+0000 7fb24fe00640 1 -- 172.21.15.44:0/2620537487 shutdown_connections 2024-09-13T08:48:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.184+0000 7fb24fe00640 1 --2- 172.21.15.44:0/2620537487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb24806eeb0 0x7fb2480b3990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.184+0000 7fb24fe00640 1 -- 172.21.15.44:0/2620537487 >> 172.21.15.44:0/2620537487 conn(0x7fb2480b18f0 msgr2=0x7fb2480b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.184+0000 7fb24fe00640 1 -- 172.21.15.44:0/2620537487 shutdown_connections 2024-09-13T08:48:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.184+0000 7fb24fe00640 1 -- 172.21.15.44:0/2620537487 wait complete. 2024-09-13T08:48:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.188+0000 7fb24fe00640 1 Processor -- start 2024-09-13T08:48:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.188+0000 7fb24fe00640 1 -- start start 2024-09-13T08:48:01.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.188+0000 7fb24fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24806eeb0 0x7fb24809d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:01.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.188+0000 7fb24fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2480b3ed0 0x7fb248148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.188+0000 7fb24fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb248161520 0x7fb248147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.188+0000 7fb24fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb24816c260 con 0x7fb24806eeb0 2024-09-13T08:48:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.188+0000 7fb24fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb24816c0e0 con 0x7fb2480b3ed0 2024-09-13T08:48:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.188+0000 7fb24fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb24816c3e0 con 0x7fb248161520 2024-09-13T08:48:01.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24806eeb0 0x7fb24809d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:01.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24806eeb0 0x7fb24809d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54088/0 (socket says 172.21.15.44:54088) 2024-09-13T08:48:01.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24d400640 1 -- 172.21.15.44:0/3506292068 learned_addr learned my addr 172.21.15.44:0/3506292068 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:01.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24ca00640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2480b3ed0 0x7fb248148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:01.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24de00640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb248161520 0x7fb248147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:01.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24d400640 1 -- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb248161520 msgr2=0x7fb248147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:01.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24d400640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb248161520 0x7fb248147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:01.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24d400640 1 -- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2480b3ed0 msgr2=0x7fb248148a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:01.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24d400640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2480b3ed0 0x7fb248148a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:01.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24d400640 1 -- 172.21.15.44:0/3506292068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2480d04d0 con 0x7fb24806eeb0 2024-09-13T08:48:01.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.192+0000 7fb24d400640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24806eeb0 0x7fb24809d640 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb24000cbf0 tx=0x7fb24000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.196+0000 7fb245600640 1 -- 172.21.15.44:0/3506292068 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240016020 con 0x7fb24806eeb0 2024-09-13T08:48:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.196+0000 7fb245600640 1 -- 172.21.15.44:0/3506292068 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb24000ab40 con 0x7fb24806eeb0 2024-09-13T08:48:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.196+0000 7fb24fe00640 1 -- 172.21.15.44:0/3506292068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2480d07b0 con 0x7fb24806eeb0 2024-09-13T08:48:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.196+0000 7fb24fe00640 1 -- 172.21.15.44:0/3506292068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24809d120 con 0x7fb24806eeb0 2024-09-13T08:48:01.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.196+0000 7fb245600640 1 -- 172.21.15.44:0/3506292068 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240012420 con 0x7fb24806eeb0 2024-09-13T08:48:01.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.196+0000 7fb24fe00640 1 -- 172.21.15.44:0/3506292068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2240052a0 con 0x7fb24806eeb0 2024-09-13T08:48:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.200+0000 7fb245600640 1 -- 172.21.15.44:0/3506292068 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7fb240019020 con 0x7fb24806eeb0 2024-09-13T08:48:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.200+0000 7fb245600640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb228074470 0x7fb228076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.200+0000 7fb245600640 1 -- 172.21.15.44:0/3506292068 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fb240093140 con 0x7fb24806eeb0 2024-09-13T08:48:01.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.208+0000 7fb245600640 1 -- 172.21.15.44:0/3506292068 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb240020020 con 0x7fb24806eeb0 2024-09-13T08:48:01.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.208+0000 7fb24ca00640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb228074470 0x7fb228076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:01.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.220+0000 7fb24ca00640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb228074470 0x7fb228076930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0af000 tx=0x7fb23c01b050 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:01.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:01.544+0000 7fb24fe00640 1 -- 172.21.15.44:0/3506292068 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone42326", "target": ["mon-mgr", ""]}) -- 0x7fb224000c90 con 0x7fb228074470 2024-09-13T08:48:02.366 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:48:02.370 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:48:02.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.568+0000 7fb245600640 1 -- 172.21.15.44:0/3506292068 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb224000c90 con 0x7fb228074470 2024-09-13T08:48:02.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.572+0000 7fb24fe00640 1 -- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb228074470 msgr2=0x7fb228076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:02.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.572+0000 7fb24fe00640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb228074470 0x7fb228076930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0af000 tx=0x7fb23c01b050 comp rx=0 tx=0).stop 2024-09-13T08:48:02.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.572+0000 7fb24fe00640 1 -- 172.21.15.44:0/3506292068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24806eeb0 msgr2=0x7fb24809d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:02.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.572+0000 7fb24fe00640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24806eeb0 0x7fb24809d640 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb24000cbf0 tx=0x7fb24000cce0 comp rx=0 tx=0).stop 2024-09-13T08:48:02.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.572+0000 7fb24fe00640 1 -- 172.21.15.44:0/3506292068 shutdown_connections 2024-09-13T08:48:02.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.572+0000 7fb24fe00640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb228074470 0x7fb228076930 secure :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0af000 tx=0x7fb23c01b050 comp rx=0 tx=0).stop 2024-09-13T08:48:02.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.572+0000 7fb24fe00640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb248161520 0x7fb248147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:02.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.572+0000 7fb24fe00640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2480b3ed0 0x7fb248148a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:02.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.572+0000 7fb24fe00640 1 --2- 172.21.15.44:0/3506292068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24806eeb0 0x7fb24809d640 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:02.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.572+0000 7fb24fe00640 1 -- 172.21.15.44:0/3506292068 >> 172.21.15.44:0/3506292068 conn(0x7fb2480b18f0 msgr2=0x7fb2480741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:02.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.576+0000 7fb24fe00640 1 -- 172.21.15.44:0/3506292068 shutdown_connections 2024-09-13T08:48:02.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.576+0000 7fb24fe00640 1 -- 172.21.15.44:0/3506292068 wait complete. 2024-09-13T08:48:02.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone73477 2024-09-13T08:48:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.884+0000 7f48dee00640 1 -- 172.21.15.44:0/1714339605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d80b2bf0 msgr2=0x7f48d815ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.884+0000 7f48dee00640 1 --2- 172.21.15.44:0/1714339605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d80b2bf0 0x7f48d815ab10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009900 tx=0x7f48d001c8b0 comp rx=0 tx=0).stop 2024-09-13T08:48:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.884+0000 7f48dee00640 1 -- 172.21.15.44:0/1714339605 shutdown_connections 2024-09-13T08:48:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.884+0000 7f48dee00640 1 --2- 172.21.15.44:0/1714339605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d80b2bf0 0x7f48d815ab10 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.884+0000 7f48dee00640 1 --2- 172.21.15.44:0/1714339605 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48d80cd100 0x7f48d815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.884+0000 7f48dee00640 1 -- 172.21.15.44:0/1714339605 >> 172.21.15.44:0/1714339605 conn(0x7f48d80b18f0 msgr2=0x7f48d80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.884+0000 7f48dee00640 1 -- 172.21.15.44:0/1714339605 shutdown_connections 2024-09-13T08:48:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.884+0000 7f48dee00640 1 -- 172.21.15.44:0/1714339605 wait complete. 2024-09-13T08:48:02.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.888+0000 7f48dee00640 1 Processor -- start 2024-09-13T08:48:02.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.888+0000 7f48dee00640 1 -- start start 2024-09-13T08:48:02.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.888+0000 7f48dee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d80b2bf0 0x7f48d8158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:02.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.888+0000 7f48dee00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48d80b43d0 0x7f48d80ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:02.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.888+0000 7f48dee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d80cd100 0x7f48d8168b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:02.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.888+0000 7f48dee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48d8169550 con 0x7f48d80cd100 2024-09-13T08:48:02.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.888+0000 7f48dee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48d81693d0 con 0x7f48d80b2bf0 2024-09-13T08:48:02.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.888+0000 7f48dee00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f48d81696d0 con 0x7f48d80b43d0 2024-09-13T08:48:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d80cd100 0x7f48d8168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d80cd100 0x7f48d8168b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49728/0 (socket says 172.21.15.44:49728) 2024-09-13T08:48:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dce00640 1 -- 172.21.15.44:0/19878164 learned_addr learned my addr 172.21.15.44:0/19878164 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48d7e00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d80b2bf0 0x7f48d8158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dce00640 1 -- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48d80b43d0 msgr2=0x7f48d80ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dce00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48d80b43d0 0x7f48d80ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dce00640 1 -- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d80b2bf0 msgr2=0x7f48d8158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dce00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d80b2bf0 0x7f48d8158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dce00640 1 -- 172.21.15.44:0/19878164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d8167a70 con 0x7f48d80cd100 2024-09-13T08:48:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48d7e00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d80b2bf0 0x7f48d8158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:48:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dce00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d80cd100 0x7f48d8168b10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f48d001c6e0 tx=0x7f48d00a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:02.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48d4c00640 1 -- 172.21.15.44:0/19878164 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d00b2040 con 0x7f48d80cd100 2024-09-13T08:48:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48d4c00640 1 -- 172.21.15.44:0/19878164 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48d00042c0 con 0x7f48d80cd100 2024-09-13T08:48:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dee00640 1 -- 172.21.15.44:0/19878164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d8167d50 con 0x7f48d80cd100 2024-09-13T08:48:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48d4c00640 1 -- 172.21.15.44:0/19878164 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0005030 con 0x7f48d80cd100 2024-09-13T08:48:02.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.892+0000 7f48dee00640 1 -- 172.21.15.44:0/19878164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d8168310 con 0x7f48d80cd100 2024-09-13T08:48:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.896+0000 7f48dee00640 1 -- 172.21.15.44:0/19878164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d8069410 con 0x7f48d80cd100 2024-09-13T08:48:02.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.896+0000 7f48d4c00640 1 -- 172.21.15.44:0/19878164 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f48d00051f0 con 0x7f48d80cd100 2024-09-13T08:48:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.896+0000 7f48d4c00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48b4074420 0x7f48b40768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.896+0000 7f48d4c00640 1 -- 172.21.15.44:0/19878164 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f48d012c870 con 0x7f48d80cd100 2024-09-13T08:48:02.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.904+0000 7f48d7e00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48b4074420 0x7f48b40768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:02.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.908+0000 7f48d4c00640 1 -- 172.21.15.44:0/19878164 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f48d00164d0 con 0x7f48d80cd100 2024-09-13T08:48:02.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:02.908+0000 7f48d7e00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48b4074420 0x7f48b40768e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f48c8007fd0 tx=0x7f48c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:03.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.280+0000 7f48dee00640 1 -- 172.21.15.44:0/19878164 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone73477", "target": ["mon-mgr", ""]}) -- 0x7f48d80b47b0 con 0x7f48b4074420 2024-09-13T08:48:03.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.660+0000 7f48d4c00640 1 -- 172.21.15.44:0/19878164 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f48d80b47b0 con 0x7f48b4074420 2024-09-13T08:48:03.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 -- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48b4074420 msgr2=0x7f48b40768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:03.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48b4074420 0x7f48b40768e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f48c8007fd0 tx=0x7f48c8009040 comp rx=0 tx=0).stop 2024-09-13T08:48:03.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 -- 172.21.15.44:0/19878164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d80cd100 msgr2=0x7f48d8168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:03.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d80cd100 0x7f48d8168b10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f48d001c6e0 tx=0x7f48d00a55a0 comp rx=0 tx=0).stop 2024-09-13T08:48:03.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 -- 172.21.15.44:0/19878164 shutdown_connections 2024-09-13T08:48:03.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f48b4074420 0x7f48b40768e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:03.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d80cd100 0x7f48d8168b10 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:03.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48d80b43d0 0x7f48d80ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:03.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 --2- 172.21.15.44:0/19878164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d80b2bf0 0x7f48d8158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:03.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 -- 172.21.15.44:0/19878164 >> 172.21.15.44:0/19878164 conn(0x7f48d80b18f0 msgr2=0x7f48d80af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:03.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 -- 172.21.15.44:0/19878164 shutdown_connections 2024-09-13T08:48:03.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.664+0000 7f48dee00640 1 -- 172.21.15.44:0/19878164 wait complete. 2024-09-13T08:48:03.693 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8848 2024-09-13T08:48:03.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.936+0000 7fb047000640 1 -- 172.21.15.44:0/3649334863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0400afb10 msgr2=0x7fb0400affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:03.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.936+0000 7fb047000640 1 --2- 172.21.15.44:0/3649334863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0400afb10 0x7fb0400affc0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb038009bc0 tx=0x7fb03801cba0 comp rx=0 tx=0).stop 2024-09-13T08:48:03.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.936+0000 7fb047000640 1 -- 172.21.15.44:0/3649334863 shutdown_connections 2024-09-13T08:48:03.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.936+0000 7fb047000640 1 --2- 172.21.15.44:0/3649334863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0400afb10 0x7fb0400affc0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:03.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.936+0000 7fb047000640 1 --2- 172.21.15.44:0/3649334863 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0400b1650 0x7fb0400b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:03.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.936+0000 7fb047000640 1 -- 172.21.15.44:0/3649334863 >> 172.21.15.44:0/3649334863 conn(0x7fb0400d1dd0 msgr2=0x7fb0400d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:03.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.936+0000 7fb047000640 1 -- 172.21.15.44:0/3649334863 shutdown_connections 2024-09-13T08:48:03.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb047000640 1 -- 172.21.15.44:0/3649334863 wait complete. 2024-09-13T08:48:03.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb047000640 1 Processor -- start 2024-09-13T08:48:03.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb047000640 1 -- start start 2024-09-13T08:48:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb047000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0400afb10 0x7fb04014ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb047000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0400b1650 0x7fb0401ff780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb047000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0401605d0 0x7fb04014b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb047000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb040169540 con 0x7fb0400afb10 2024-09-13T08:48:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb047000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb0400b1430 con 0x7fb0400b1650 2024-09-13T08:48:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb047000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb0401696c0 con 0x7fb0401605d0 2024-09-13T08:48:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb03fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0400afb10 0x7fb04014ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb03fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0400afb10 0x7fb04014ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49746/0 (socket says 172.21.15.44:49746) 2024-09-13T08:48:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb03fe00640 1 -- 172.21.15.44:0/3528366965 learned_addr learned my addr 172.21.15.44:0/3528366965 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.940+0000 7fb045000640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0401605d0 0x7fb04014b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb03fe00640 1 -- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0401605d0 msgr2=0x7fb04014b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb03fe00640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0401605d0 0x7fb04014b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb03fe00640 1 -- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0400b1650 msgr2=0x7fb0401ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb03fe00640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0400b1650 0x7fb0401ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb03fe00640 1 -- 172.21.15.44:0/3528366965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb040200220 con 0x7fb0400afb10 2024-09-13T08:48:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb03fe00640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0400afb10 0x7fb04014ef80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb030001ec0 tx=0x7fb03000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb037e00640 1 -- 172.21.15.44:0/3528366965 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030017070 con 0x7fb0400afb10 2024-09-13T08:48:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb037e00640 1 -- 172.21.15.44:0/3528366965 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb030003600 con 0x7fb0400afb10 2024-09-13T08:48:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb047000640 1 -- 172.21.15.44:0/3528366965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb040200500 con 0x7fb0400afb10 2024-09-13T08:48:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb037e00640 1 -- 172.21.15.44:0/3528366965 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030003a50 con 0x7fb0400afb10 2024-09-13T08:48:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb047000640 1 -- 172.21.15.44:0/3528366965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb04014ea60 con 0x7fb0400afb10 2024-09-13T08:48:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb047000640 1 -- 172.21.15.44:0/3528366965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0400affc0 con 0x7fb0400afb10 2024-09-13T08:48:03.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.944+0000 7fb037e00640 1 -- 172.21.15.44:0/3528366965 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7fb03001a070 con 0x7fb0400afb10 2024-09-13T08:48:03.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.948+0000 7fb037e00640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb020074420 0x7fb0200768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:03.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.948+0000 7fb037e00640 1 -- 172.21.15.44:0/3528366965 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fb030092240 con 0x7fb0400afb10 2024-09-13T08:48:03.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.956+0000 7fb03f400640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb020074420 0x7fb0200768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:03.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.956+0000 7fb037e00640 1 -- 172.21.15.44:0/3528366965 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb03005eae0 con 0x7fb0400afb10 2024-09-13T08:48:03.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:03.960+0000 7fb03f400640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb020074420 0x7fb0200768e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb0400c5ed0 tx=0x7fb02c006ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:04.360+0000 7fb047000640 1 -- 172.21.15.44:0/3528366965 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8848", "target": ["mon-mgr", ""]}) -- 0x7fb0400d1080 con 0x7fb020074420 2024-09-13T08:48:05.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.039+0000 7fb037e00640 1 -- 172.21.15.44:0/3528366965 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb0400d1080 con 0x7fb020074420 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 -- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb020074420 msgr2=0x7fb0200768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb020074420 0x7fb0200768e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb0400c5ed0 tx=0x7fb02c006ca0 comp rx=0 tx=0).stop 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 -- 172.21.15.44:0/3528366965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0400afb10 msgr2=0x7fb04014ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0400afb10 0x7fb04014ef80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb030001ec0 tx=0x7fb03000c890 comp rx=0 tx=0).stop 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 -- 172.21.15.44:0/3528366965 shutdown_connections 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb020074420 0x7fb0200768e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb0401605d0 0x7fb04014b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0400b1650 0x7fb0401ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 --2- 172.21.15.44:0/3528366965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0400afb10 0x7fb04014ef80 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 -- 172.21.15.44:0/3528366965 >> 172.21.15.44:0/3528366965 conn(0x7fb0400d1dd0 msgr2=0x7fb0400c4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:05.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 -- 172.21.15.44:0/3528366965 shutdown_connections 2024-09-13T08:48:05.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.043+0000 7fb047000640 1 -- 172.21.15.44:0/3528366965 wait complete. 2024-09-13T08:48:05.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone74089 2024-09-13T08:48:05.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.367+0000 7fe604a00640 1 -- 172.21.15.44:0/1908370294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60015af10 msgr2=0x7fe600073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:05.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.367+0000 7fe604a00640 1 --2- 172.21.15.44:0/1908370294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60015af10 0x7fe600073790 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009bf0 tx=0x7fe5f001f390 comp rx=0 tx=0).stop 2024-09-13T08:48:05.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.367+0000 7fe604a00640 1 -- 172.21.15.44:0/1908370294 shutdown_connections 2024-09-13T08:48:05.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.367+0000 7fe604a00640 1 --2- 172.21.15.44:0/1908370294 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe6000cfe40 0x7fe600073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:05.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.367+0000 7fe604a00640 1 --2- 172.21.15.44:0/1908370294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60015af10 0x7fe600073790 secure :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009bf0 tx=0x7fe5f001f390 comp rx=0 tx=0).stop 2024-09-13T08:48:05.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.367+0000 7fe604a00640 1 -- 172.21.15.44:0/1908370294 >> 172.21.15.44:0/1908370294 conn(0x7fe6000d0b10 msgr2=0x7fe6000d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:05.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.367+0000 7fe604a00640 1 -- 172.21.15.44:0/1908370294 shutdown_connections 2024-09-13T08:48:05.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.367+0000 7fe604a00640 1 -- 172.21.15.44:0/1908370294 wait complete. 2024-09-13T08:48:05.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.371+0000 7fe604a00640 1 Processor -- start 2024-09-13T08:48:05.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.371+0000 7fe604a00640 1 -- start start 2024-09-13T08:48:05.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe604a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe600074210 0x7fe6000af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:05.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe604a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6000cfe40 0x7fe6001572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:05.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe604a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6000b03f0 0x7fe600157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:05.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe604a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6000cb9f0 con 0x7fe6000cfe40 2024-09-13T08:48:05.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe604a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6000cb870 con 0x7fe6000b03f0 2024-09-13T08:48:05.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe604a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe6000cbb70 con 0x7fe600074210 2024-09-13T08:48:05.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe5ff400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe600074210 0x7fe6000af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:05.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe5ff400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe600074210 0x7fe6000af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:51098/0 (socket says 172.21.15.44:51098) 2024-09-13T08:48:05.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe5ff400640 1 -- 172.21.15.44:0/4070614244 learned_addr learned my addr 172.21.15.44:0/4070614244 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:05.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe5ffe00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6000b03f0 0x7fe600157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:05.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.375+0000 7fe5f6a00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6000cfe40 0x7fe6001572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:05.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe5f6a00640 1 -- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe600074210 msgr2=0x7fe6000af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:05.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe5f6a00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe600074210 0x7fe6000af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:05.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe5f6a00640 1 -- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6000b03f0 msgr2=0x7fe600157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:05.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe5f6a00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6000b03f0 0x7fe600157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:05.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe5f6a00640 1 -- 172.21.15.44:0/4070614244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe600159480 con 0x7fe6000cfe40 2024-09-13T08:48:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe5f6a00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6000cfe40 0x7fe6001572c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b410 tx=0x7fe5e800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe5f7e00640 1 -- 172.21.15.44:0/4070614244 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8015020 con 0x7fe6000cfe40 2024-09-13T08:48:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe5f7e00640 1 -- 172.21.15.44:0/4070614244 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5e8011920 con 0x7fe6000cfe40 2024-09-13T08:48:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe5f7e00640 1 -- 172.21.15.44:0/4070614244 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8011d70 con 0x7fe6000cfe40 2024-09-13T08:48:05.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe604a00640 1 -- 172.21.15.44:0/4070614244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe600159760 con 0x7fe6000cfe40 2024-09-13T08:48:05.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.379+0000 7fe604a00640 1 -- 172.21.15.44:0/4070614244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6000afc20 con 0x7fe6000cfe40 2024-09-13T08:48:05.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.383+0000 7fe604a00640 1 -- 172.21.15.44:0/4070614244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d80052a0 con 0x7fe6000cfe40 2024-09-13T08:48:05.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.383+0000 7fe5f7e00640 1 -- 172.21.15.44:0/4070614244 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7fe5e8018020 con 0x7fe6000cfe40 2024-09-13T08:48:05.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.383+0000 7fe5f7e00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe5dc074450 0x7fe5dc076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:05.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.383+0000 7fe5f7e00640 1 -- 172.21.15.44:0/4070614244 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fe5e8092c10 con 0x7fe6000cfe40 2024-09-13T08:48:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.391+0000 7fe5ff400640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe5dc074450 0x7fe5dc076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:05.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.395+0000 7fe5f7e00640 1 -- 172.21.15.44:0/4070614244 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe5e801f020 con 0x7fe6000cfe40 2024-09-13T08:48:05.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.403+0000 7fe5ff400640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe5dc074450 0x7fe5dc076910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009b10 tx=0x7fe5f001dbd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:05.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:05.763+0000 7fe604a00640 1 -- 172.21.15.44:0/4070614244 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone74089", "target": ["mon-mgr", ""]}) -- 0x7fe5d8000c90 con 0x7fe5dc074450 2024-09-13T08:48:06.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.375+0000 7fe5f7e00640 1 -- 172.21.15.44:0/4070614244 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5d8000c90 con 0x7fe5dc074450 2024-09-13T08:48:06.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 -- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe5dc074450 msgr2=0x7fe5dc076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:06.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe5dc074450 0x7fe5dc076910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009b10 tx=0x7fe5f001dbd0 comp rx=0 tx=0).stop 2024-09-13T08:48:06.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 -- 172.21.15.44:0/4070614244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6000cfe40 msgr2=0x7fe6001572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:06.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6000cfe40 0x7fe6001572c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b410 tx=0x7fe5e800b810 comp rx=0 tx=0).stop 2024-09-13T08:48:06.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 -- 172.21.15.44:0/4070614244 shutdown_connections 2024-09-13T08:48:06.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe5dc074450 0x7fe5dc076910 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:06.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6000b03f0 0x7fe600157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:06.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6000cfe40 0x7fe6001572c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:06.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 --2- 172.21.15.44:0/4070614244 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe600074210 0x7fe6000af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:06.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 -- 172.21.15.44:0/4070614244 >> 172.21.15.44:0/4070614244 conn(0x7fe6000d0b10 msgr2=0x7fe600071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:06.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.379+0000 7fe604a00640 1 -- 172.21.15.44:0/4070614244 shutdown_connections 2024-09-13T08:48:06.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.383+0000 7fe604a00640 1 -- 172.21.15.44:0/4070614244 wait complete. 2024-09-13T08:48:06.418 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone649610 2024-09-13T08:48:06.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.679+0000 7fd389400640 1 -- 172.21.15.44:0/1361001282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3840b4bd0 msgr2=0x7fd3840b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:06.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.679+0000 7fd389400640 1 --2- 172.21.15.44:0/1361001282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3840b4bd0 0x7fd3840b2bf0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009900 tx=0x7fd36c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:48:06.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.679+0000 7fd389400640 1 -- 172.21.15.44:0/1361001282 shutdown_connections 2024-09-13T08:48:06.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.679+0000 7fd389400640 1 --2- 172.21.15.44:0/1361001282 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3840b3130 0x7fd3840cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:06.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.679+0000 7fd389400640 1 --2- 172.21.15.44:0/1361001282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3840b4bd0 0x7fd3840b2bf0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:06.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.679+0000 7fd389400640 1 -- 172.21.15.44:0/1361001282 >> 172.21.15.44:0/1361001282 conn(0x7fd3840b18f0 msgr2=0x7fd3840b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:06.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.679+0000 7fd389400640 1 -- 172.21.15.44:0/1361001282 shutdown_connections 2024-09-13T08:48:06.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.679+0000 7fd389400640 1 -- 172.21.15.44:0/1361001282 wait complete. 2024-09-13T08:48:06.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.683+0000 7fd389400640 1 Processor -- start 2024-09-13T08:48:06.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.683+0000 7fd389400640 1 -- start start 2024-09-13T08:48:06.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.683+0000 7fd389400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3840b3130 0x7fd3840cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:06.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.683+0000 7fd389400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3840b4bd0 0x7fd384150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:06.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.683+0000 7fd389400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38415b580 0x7fd384149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:06.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.683+0000 7fd389400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd384165ad0 con 0x7fd3840b3130 2024-09-13T08:48:06.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.683+0000 7fd389400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd384165950 con 0x7fd38415b580 2024-09-13T08:48:06.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.683+0000 7fd389400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd384165c50 con 0x7fd3840b4bd0 2024-09-13T08:48:06.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd383200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38415b580 0x7fd384149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd383200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38415b580 0x7fd384149690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:37858/0 (socket says 172.21.15.44:37858) 2024-09-13T08:48:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd383200640 1 -- 172.21.15.44:0/3612610032 learned_addr learned my addr 172.21.15.44:0/3612610032 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd37be00640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3840b4bd0 0x7fd384150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd382800640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3840b3130 0x7fd3840cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd383200640 1 -- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3840b4bd0 msgr2=0x7fd384150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd383200640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3840b4bd0 0x7fd384150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd383200640 1 -- 172.21.15.44:0/3612610032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3840b3130 msgr2=0x7fd3840cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd383200640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3840b3130 0x7fd3840cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd383200640 1 -- 172.21.15.44:0/3612610032 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38414c810 con 0x7fd38415b580 2024-09-13T08:48:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd383200640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38415b580 0x7fd384149690 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b970 tx=0x7fd37c00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd37aa00640 1 -- 172.21.15.44:0/3612610032 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c019070 con 0x7fd38415b580 2024-09-13T08:48:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd37aa00640 1 -- 172.21.15.44:0/3612610032 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd37c00cbf0 con 0x7fd38415b580 2024-09-13T08:48:06.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd37aa00640 1 -- 172.21.15.44:0/3612610032 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c014440 con 0x7fd38415b580 2024-09-13T08:48:06.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd389400640 1 -- 172.21.15.44:0/3612610032 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38414a450 con 0x7fd38415b580 2024-09-13T08:48:06.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.687+0000 7fd389400640 1 -- 172.21.15.44:0/3612610032 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38414ca80 con 0x7fd38415b580 2024-09-13T08:48:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.691+0000 7fd389400640 1 -- 172.21.15.44:0/3612610032 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3840cc550 con 0x7fd38415b580 2024-09-13T08:48:06.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.695+0000 7fd37aa00640 1 -- 172.21.15.44:0/3612610032 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7fd37c01c070 con 0x7fd38415b580 2024-09-13T08:48:06.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.695+0000 7fd37aa00640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd360074420 0x7fd3600768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:06.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.699+0000 7fd37aa00640 1 -- 172.21.15.44:0/3612610032 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fd37c093060 con 0x7fd38415b580 2024-09-13T08:48:06.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.707+0000 7fd37aa00640 1 -- 172.21.15.44:0/3612610032 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd37c023020 con 0x7fd38415b580 2024-09-13T08:48:06.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.715+0000 7fd382800640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd360074420 0x7fd3600768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:06.795+0000 7fd382800640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd360074420 0x7fd3600768e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd3740098e0 tx=0x7fd374009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:07.075+0000 7fd389400640 1 -- 172.21.15.44:0/3612610032 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone649610", "target": ["mon-mgr", ""]}) -- 0x7fd3840b1e70 con 0x7fd360074420 2024-09-13T08:48:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.159+0000 7fd37aa00640 1 -- 172.21.15.44:0/3612610032 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd3840b1e70 con 0x7fd360074420 2024-09-13T08:48:08.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 -- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd360074420 msgr2=0x7fd3600768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:08.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd360074420 0x7fd3600768e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd3740098e0 tx=0x7fd374009060 comp rx=0 tx=0).stop 2024-09-13T08:48:08.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 -- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38415b580 msgr2=0x7fd384149690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:08.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38415b580 0x7fd384149690 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b970 tx=0x7fd37c00be30 comp rx=0 tx=0).stop 2024-09-13T08:48:08.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 -- 172.21.15.44:0/3612610032 shutdown_connections 2024-09-13T08:48:08.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd360074420 0x7fd3600768e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:08.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38415b580 0x7fd384149690 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:08.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3840b4bd0 0x7fd384150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:08.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 --2- 172.21.15.44:0/3612610032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3840b3130 0x7fd3840cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:08.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 -- 172.21.15.44:0/3612610032 >> 172.21.15.44:0/3612610032 conn(0x7fd3840b18f0 msgr2=0x7fd3840cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:08.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 -- 172.21.15.44:0/3612610032 shutdown_connections 2024-09-13T08:48:08.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.163+0000 7fd389400640 1 -- 172.21.15.44:0/3612610032 wait complete. 2024-09-13T08:48:08.213 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone677511 2024-09-13T08:48:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.479+0000 7f31cd600640 1 -- 172.21.15.44:0/150608602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c80ad5b0 msgr2=0x7f31c80d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.479+0000 7f31cd600640 1 --2- 172.21.15.44:0/150608602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c80ad5b0 0x7f31c80d07b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f31c8073fa0 tx=0x7f31c001cda0 comp rx=0 tx=0).stop 2024-09-13T08:48:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.479+0000 7f31cd600640 1 -- 172.21.15.44:0/150608602 shutdown_connections 2024-09-13T08:48:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.479+0000 7f31cd600640 1 --2- 172.21.15.44:0/150608602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c80ad5b0 0x7f31c80d07b0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:08.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.479+0000 7f31cd600640 1 --2- 172.21.15.44:0/150608602 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31c80ccc50 0x7f31c80ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:08.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.479+0000 7f31cd600640 1 --2- 172.21.15.44:0/150608602 >> v2:172.21.15.129:3300/0 conn(0x7f31c8154940 0x7f31c8068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:08.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.479+0000 7f31cd600640 1 -- 172.21.15.44:0/150608602 >> 172.21.15.44:0/150608602 conn(0x7f31c80742d0 msgr2=0x7f31c80746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:08.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.479+0000 7f31cd600640 1 -- 172.21.15.44:0/150608602 shutdown_connections 2024-09-13T08:48:08.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.479+0000 7f31cd600640 1 -- 172.21.15.44:0/150608602 wait complete. 2024-09-13T08:48:08.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.483+0000 7f31cd600640 1 Processor -- start 2024-09-13T08:48:08.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.483+0000 7f31cd600640 1 -- start start 2024-09-13T08:48:08.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31cd600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c80ad5b0 0x7f31c8151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:08.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31cd600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31c80ccc50 0x7f31c80998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:08.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31cd600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8154940 0x7f31c815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:08.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31cd600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31c815e190 con 0x7f31c80ad5b0 2024-09-13T08:48:08.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31cd600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31c815e010 con 0x7f31c8154940 2024-09-13T08:48:08.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31cd600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f31c815e310 con 0x7f31c80ccc50 2024-09-13T08:48:08.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c80ad5b0 0x7f31c8151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:08.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c80ad5b0 0x7f31c8151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49798/0 (socket says 172.21.15.44:49798) 2024-09-13T08:48:08.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31c6a00640 1 -- 172.21.15.44:0/3763496089 learned_addr learned my addr 172.21.15.44:0/3763496089 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:08.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31c7400640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8154940 0x7f31c815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:08.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31c6a00640 1 -- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31c80ccc50 msgr2=0x7f31c80998c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:48:08.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31c6a00640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31c80ccc50 0x7f31c80998c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:08.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31c6a00640 1 -- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8154940 msgr2=0x7f31c815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:08.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31c6a00640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8154940 0x7f31c815d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:08.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31c6a00640 1 -- 172.21.15.44:0/3763496089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c80c3780 con 0x7f31c80ad5b0 2024-09-13T08:48:08.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.487+0000 7f31c6a00640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c80ad5b0 0x7f31c8151f40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f31b800d950 tx=0x7f31b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:08.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.491+0000 7f31bea00640 1 -- 172.21.15.44:0/3763496089 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8017070 con 0x7f31c80ad5b0 2024-09-13T08:48:08.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.491+0000 7f31bea00640 1 -- 172.21.15.44:0/3763496089 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31b8009d20 con 0x7f31c80ad5b0 2024-09-13T08:48:08.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.491+0000 7f31bea00640 1 -- 172.21.15.44:0/3763496089 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b80106c0 con 0x7f31c80ad5b0 2024-09-13T08:48:08.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.491+0000 7f31cd600640 1 -- 172.21.15.44:0/3763496089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c815c920 con 0x7f31c80ad5b0 2024-09-13T08:48:08.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.491+0000 7f31cd600640 1 -- 172.21.15.44:0/3763496089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c80995e0 con 0x7f31c80ad5b0 2024-09-13T08:48:08.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.491+0000 7f31cd600640 1 -- 172.21.15.44:0/3763496089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a00052a0 con 0x7f31c80ad5b0 2024-09-13T08:48:08.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.495+0000 7f31bea00640 1 -- 172.21.15.44:0/3763496089 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f31b801a070 con 0x7f31c80ad5b0 2024-09-13T08:48:08.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.495+0000 7f31bea00640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31a4074420 0x7f31a40768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:08.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.495+0000 7f31bea00640 1 -- 172.21.15.44:0/3763496089 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f31b8092f00 con 0x7f31c80ad5b0 2024-09-13T08:48:08.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.503+0000 7f31bfe00640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31a4074420 0x7f31a40768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:08.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.503+0000 7f31bfe00640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31a4074420 0x7f31a40768e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f31c80b3120 tx=0x7f31b0006c30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:08.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.507+0000 7f31bea00640 1 -- 172.21.15.44:0/3763496089 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f31b805f1b0 con 0x7f31c80ad5b0 2024-09-13T08:48:08.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:08.843+0000 7f31cd600640 1 -- 172.21.15.44:0/3763496089 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone677511", "target": ["mon-mgr", ""]}) -- 0x7f31a0000c90 con 0x7f31a4074420 2024-09-13T08:48:09.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.315+0000 7f31bea00640 1 -- 172.21.15.44:0/3763496089 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f31a0000c90 con 0x7f31a4074420 2024-09-13T08:48:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 -- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31a4074420 msgr2=0x7f31a40768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31a4074420 0x7f31a40768e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f31c80b3120 tx=0x7f31b0006c30 comp rx=0 tx=0).stop 2024-09-13T08:48:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 -- 172.21.15.44:0/3763496089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c80ad5b0 msgr2=0x7f31c8151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c80ad5b0 0x7f31c8151f40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f31b800d950 tx=0x7f31b800de10 comp rx=0 tx=0).stop 2024-09-13T08:48:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 -- 172.21.15.44:0/3763496089 shutdown_connections 2024-09-13T08:48:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f31a4074420 0x7f31a40768e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8154940 0x7f31c815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f31c80ccc50 0x7f31c80998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 --2- 172.21.15.44:0/3763496089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c80ad5b0 0x7f31c8151f40 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 -- 172.21.15.44:0/3763496089 >> 172.21.15.44:0/3763496089 conn(0x7f31c80742d0 msgr2=0x7f31c80ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 -- 172.21.15.44:0/3763496089 shutdown_connections 2024-09-13T08:48:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.323+0000 7f31cd600640 1 -- 172.21.15.44:0/3763496089 wait complete. 2024-09-13T08:48:09.386 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone268612 2024-09-13T08:48:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.679+0000 7f0795c00640 1 -- 172.21.15.44:0/13850899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07900cc8f0 msgr2=0x7f07900ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.679+0000 7f0795c00640 1 --2- 172.21.15.44:0/13850899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07900cc8f0 0x7f07900ce440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f07900d0fc0 tx=0x7f078801cd80 comp rx=0 tx=0).stop 2024-09-13T08:48:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.679+0000 7f0795c00640 1 -- 172.21.15.44:0/13850899 shutdown_connections 2024-09-13T08:48:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.679+0000 7f0795c00640 1 --2- 172.21.15.44:0/13850899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07900cc8f0 0x7f07900ce440 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.679+0000 7f0795c00640 1 --2- 172.21.15.44:0/13850899 >> v2:172.21.15.129:3300/0 conn(0x7f079015bcd0 0x7f079015c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.679+0000 7f0795c00640 1 -- 172.21.15.44:0/13850899 >> 172.21.15.44:0/13850899 conn(0x7f07900b1c70 msgr2=0x7f07900b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.679+0000 7f0795c00640 1 -- 172.21.15.44:0/13850899 shutdown_connections 2024-09-13T08:48:09.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.683+0000 7f0795c00640 1 -- 172.21.15.44:0/13850899 wait complete. 2024-09-13T08:48:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.683+0000 7f0795c00640 1 Processor -- start 2024-09-13T08:48:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.683+0000 7f0795c00640 1 -- start start 2024-09-13T08:48:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f0795c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0790071210 0x7f079013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f0795c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07900cc8f0 0x7f07901537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f0795c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079015bcd0 0x7f07901f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f0795c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0790168bc0 con 0x7f0790071210 2024-09-13T08:48:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f0795c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0790168a40 con 0x7f079015bcd0 2024-09-13T08:48:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f0795c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0790168d40 con 0x7f07900cc8f0 2024-09-13T08:48:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f078f400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07900cc8f0 0x7f07901537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f0795200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079015bcd0 0x7f07901f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f078f400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07900cc8f0 0x7f07901537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:51142/0 (socket says 172.21.15.44:51142) 2024-09-13T08:48:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f078f400640 1 -- 172.21.15.44:0/1053072778 learned_addr learned my addr 172.21.15.44:0/1053072778 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f078fe00640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0790071210 0x7f079013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f078f400640 1 -- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079015bcd0 msgr2=0x7f07901f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f078f400640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079015bcd0 0x7f07901f8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f078f400640 1 -- 172.21.15.44:0/1053072778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0790071210 msgr2=0x7f079013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f078f400640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0790071210 0x7f079013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.687+0000 7f078f400640 1 -- 172.21.15.44:0/1053072778 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07901f9570 con 0x7f07900cc8f0 2024-09-13T08:48:09.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.691+0000 7f078f400640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07900cc8f0 0x7f07901537e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f077c00c500 tx=0x7f077c00c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.695+0000 7f0787e00640 1 -- 172.21.15.44:0/1053072778 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c017070 con 0x7f07900cc8f0 2024-09-13T08:48:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.695+0000 7f0787e00640 1 -- 172.21.15.44:0/1053072778 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f077c0053c0 con 0x7f07900cc8f0 2024-09-13T08:48:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.695+0000 7f0787e00640 1 -- 172.21.15.44:0/1053072778 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c012890 con 0x7f07900cc8f0 2024-09-13T08:48:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.695+0000 7f0795c00640 1 -- 172.21.15.44:0/1053072778 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07901f9850 con 0x7f07900cc8f0 2024-09-13T08:48:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.695+0000 7f0795c00640 1 -- 172.21.15.44:0/1053072778 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f079014bf20 con 0x7f07900cc8f0 2024-09-13T08:48:09.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.699+0000 7f0795c00640 1 -- 172.21.15.44:0/1053072778 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07900ce440 con 0x7f07900cc8f0 2024-09-13T08:48:09.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.703+0000 7f0787e00640 1 -- 172.21.15.44:0/1053072778 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f077c01a070 con 0x7f07900cc8f0 2024-09-13T08:48:09.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.715+0000 7f0787e00640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0770074470 0x7f0770076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:09.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.715+0000 7f0787e00640 1 -- 172.21.15.44:0/1053072778 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f077c092150 con 0x7f07900cc8f0 2024-09-13T08:48:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.715+0000 7f0787e00640 1 -- 172.21.15.44:0/1053072778 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f077c092e80 con 0x7f07900cc8f0 2024-09-13T08:48:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.715+0000 7f078fe00640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0770074470 0x7f0770076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:09.723+0000 7f078fe00640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0770074470 0x7f0770076930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f07800098f0 tx=0x7f0780006f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.047+0000 7f0795c00640 1 -- 172.21.15.44:0/1053072778 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone268612", "target": ["mon-mgr", ""]}) -- 0x7f079014c0f0 con 0x7f0770074470 2024-09-13T08:48:10.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.491+0000 7f0787e00640 1 -- 172.21.15.44:0/1053072778 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f079014c0f0 con 0x7f0770074470 2024-09-13T08:48:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.495+0000 7f0795c00640 1 -- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0770074470 msgr2=0x7f0770076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.495+0000 7f0795c00640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0770074470 0x7f0770076930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f07800098f0 tx=0x7f0780006f90 comp rx=0 tx=0).stop 2024-09-13T08:48:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.499+0000 7f0795c00640 1 -- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07900cc8f0 msgr2=0x7f07901537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.499+0000 7f0795c00640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07900cc8f0 0x7f07901537e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f077c00c500 tx=0x7f077c00c9c0 comp rx=0 tx=0).stop 2024-09-13T08:48:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.499+0000 7f0795c00640 1 -- 172.21.15.44:0/1053072778 shutdown_connections 2024-09-13T08:48:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.499+0000 7f0795c00640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0770074470 0x7f0770076930 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.499+0000 7f0795c00640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079015bcd0 0x7f07901f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.499+0000 7f0795c00640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07900cc8f0 0x7f07901537e0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f077c00c500 tx=0x7f077c00c9c0 comp rx=0 tx=0).stop 2024-09-13T08:48:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.499+0000 7f0795c00640 1 --2- 172.21.15.44:0/1053072778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0790071210 0x7f079013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.499+0000 7f0795c00640 1 -- 172.21.15.44:0/1053072778 >> 172.21.15.44:0/1053072778 conn(0x7f07900b1c70 msgr2=0x7f079014db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.499+0000 7f0795c00640 1 -- 172.21.15.44:0/1053072778 shutdown_connections 2024-09-13T08:48:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.499+0000 7f0795c00640 1 -- 172.21.15.44:0/1053072778 wait complete. 2024-09-13T08:48:10.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone680213 2024-09-13T08:48:10.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.807+0000 7f5464600640 1 -- 172.21.15.44:0/3247636710 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5444003ab0 msgr2=0x7f5444003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.807+0000 7f5464600640 1 --2- 172.21.15.44:0/3247636710 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5444003ab0 0x7f5444003ed0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f545000bb10 tx=0x7f545001cda0 comp rx=0 tx=0).stop 2024-09-13T08:48:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.807+0000 7f5464600640 1 -- 172.21.15.44:0/3247636710 shutdown_connections 2024-09-13T08:48:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.807+0000 7f5464600640 1 --2- 172.21.15.44:0/3247636710 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5444003ab0 0x7f5444003ed0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.807+0000 7f5464600640 1 -- 172.21.15.44:0/3247636710 >> 172.21.15.44:0/3247636710 conn(0x7f545c0d1dc0 msgr2=0x7f545c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.811+0000 7f5464600640 1 -- 172.21.15.44:0/3247636710 shutdown_connections 2024-09-13T08:48:10.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.811+0000 7f5464600640 1 -- 172.21.15.44:0/3247636710 wait complete. 2024-09-13T08:48:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.815+0000 7f5464600640 1 Processor -- start 2024-09-13T08:48:10.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.815+0000 7f5464600640 1 -- start start 2024-09-13T08:48:10.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.815+0000 7f5464600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5444003ab0 0x7f545c0c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5464600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f545c15f030 0x7f545c0afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5464600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f545c0b0430 0x7f545c15e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5464600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f545c15ffb0 con 0x7f545c15f030 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5464600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f545c15fe30 con 0x7f5444003ab0 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5464600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f545c160130 con 0x7f545c0b0430 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5461200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f545c15f030 0x7f545c0afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5461200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f545c15f030 0x7f545c0afef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49854/0 (socket says 172.21.15.44:49854) 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5461200640 1 -- 172.21.15.44:0/687858663 learned_addr learned my addr 172.21.15.44:0/687858663 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5462600640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f545c0b0430 0x7f545c15e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5461c00640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5444003ab0 0x7f545c0c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5461200640 1 -- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f545c0b0430 msgr2=0x7f545c15e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5461200640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f545c0b0430 0x7f545c15e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5461200640 1 -- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5444003ab0 msgr2=0x7f545c0c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5461200640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5444003ab0 0x7f545c0c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5461200640 1 -- 172.21.15.44:0/687858663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c15f470 con 0x7f545c15f030 2024-09-13T08:48:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5461200640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f545c15f030 0x7f545c0afef0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f5448011a80 tx=0x7f5448011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5459600640 1 -- 172.21.15.44:0/687858663 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544800fdf0 con 0x7f545c15f030 2024-09-13T08:48:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5459600640 1 -- 172.21.15.44:0/687858663 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5448004030 con 0x7f545c15f030 2024-09-13T08:48:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5459600640 1 -- 172.21.15.44:0/687858663 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448004460 con 0x7f545c15f030 2024-09-13T08:48:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5464600640 1 -- 172.21.15.44:0/687858663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c0c76b0 con 0x7f545c15f030 2024-09-13T08:48:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.819+0000 7f5464600640 1 -- 172.21.15.44:0/687858663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c0c7c70 con 0x7f545c15f030 2024-09-13T08:48:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.823+0000 7f5464600640 1 -- 172.21.15.44:0/687858663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545c160690 con 0x7f545c15f030 2024-09-13T08:48:10.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.823+0000 7f5459600640 1 -- 172.21.15.44:0/687858663 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f5448003600 con 0x7f545c15f030 2024-09-13T08:48:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.827+0000 7f5459600640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f543c074450 0x7f543c076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.827+0000 7f5459600640 1 -- 172.21.15.44:0/687858663 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f5448093af0 con 0x7f545c15f030 2024-09-13T08:48:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.831+0000 7f5461c00640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f543c074450 0x7f543c076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.831+0000 7f5459600640 1 -- 172.21.15.44:0/687858663 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f545c15f470 con 0x7f545c15f030 2024-09-13T08:48:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:10.831+0000 7f5461c00640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f543c074450 0x7f543c076910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f54500b0000 tx=0x7f5450019b40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:11.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:11.147+0000 7f5464600640 1 -- 172.21.15.44:0/687858663 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone680213", "target": ["mon-mgr", ""]}) -- 0x7f545c0b0810 con 0x7f543c074450 2024-09-13T08:48:12.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.087+0000 7f5459600640 1 -- 172.21.15.44:0/687858663 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f545c0b0810 con 0x7f543c074450 2024-09-13T08:48:12.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 -- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f543c074450 msgr2=0x7f543c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f543c074450 0x7f543c076910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f54500b0000 tx=0x7f5450019b40 comp rx=0 tx=0).stop 2024-09-13T08:48:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 -- 172.21.15.44:0/687858663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f545c15f030 msgr2=0x7f545c0afef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f545c15f030 0x7f545c0afef0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f5448011a80 tx=0x7f5448011f40 comp rx=0 tx=0).stop 2024-09-13T08:48:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 -- 172.21.15.44:0/687858663 shutdown_connections 2024-09-13T08:48:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f543c074450 0x7f543c076910 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f545c0b0430 0x7f545c15e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f545c15f030 0x7f545c0afef0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 --2- 172.21.15.44:0/687858663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5444003ab0 0x7f545c0c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 -- 172.21.15.44:0/687858663 >> 172.21.15.44:0/687858663 conn(0x7f545c0d1dc0 msgr2=0x7f545c0b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:12.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 -- 172.21.15.44:0/687858663 shutdown_connections 2024-09-13T08:48:12.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.091+0000 7f5464600640 1 -- 172.21.15.44:0/687858663 wait complete. 2024-09-13T08:48:12.143 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone855814 2024-09-13T08:48:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.515+0000 7f41d2e00640 1 -- 172.21.15.44:0/43604977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41b0003fa0 msgr2=0x7f41b0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.515+0000 7f41d2e00640 1 --2- 172.21.15.44:0/43604977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41b0003fa0 0x7f41b0004810 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f41cc08e6d0 tx=0x7f41b801d170 comp rx=0 tx=0).stop 2024-09-13T08:48:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.515+0000 7f41d0e00640 1 -- 172.21.15.44:0/43604977 reap_dead start 2024-09-13T08:48:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.515+0000 7f41d2e00640 1 -- 172.21.15.44:0/43604977 shutdown_connections 2024-09-13T08:48:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.515+0000 7f41d2e00640 1 -- 172.21.15.44:0/43604977 >> 172.21.15.44:0/43604977 conn(0x7f41cc0d3fb0 msgr2=0x7f41cc0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.515+0000 7f41d2e00640 1 -- 172.21.15.44:0/43604977 shutdown_connections 2024-09-13T08:48:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.515+0000 7f41d2e00640 1 -- 172.21.15.44:0/43604977 wait complete. 2024-09-13T08:48:12.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.519+0000 7f41d2e00640 1 Processor -- start 2024-09-13T08:48:12.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.519+0000 7f41d2e00640 1 -- start start 2024-09-13T08:48:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41d2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41b0003fa0 0x7f41cc1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41d2e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f41cc0ca1b0 0x7f41cc0b9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41d2e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41cc0b9850 0x7f41cc1fc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41d2e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc0d1d00 con 0x7f41b0003fa0 2024-09-13T08:48:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41d2e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc0d1b80 con 0x7f41cc0b9850 2024-09-13T08:48:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41d2e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f41cc0d1e80 con 0x7f41cc0ca1b0 2024-09-13T08:48:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41b0003fa0 0x7f41cc1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41b0003fa0 0x7f41cc1592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42594/0 (socket says 172.21.15.44:42594) 2024-09-13T08:48:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41cbe00640 1 -- 172.21.15.44:0/4069361499 learned_addr learned my addr 172.21.15.44:0/4069361499 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41cbe00640 1 -- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f41cc0ca1b0 msgr2=0x7f41cc0b9310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:48:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41cbe00640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f41cc0ca1b0 0x7f41cc0b9310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41cbe00640 1 -- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41cc0b9850 msgr2=0x7f41cc1fc960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:48:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41cbe00640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41cc0b9850 0x7f41cc1fc960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41cbe00640 1 -- 172.21.15.44:0/4069361499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc0c93f0 con 0x7f41b0003fa0 2024-09-13T08:48:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41cbe00640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41b0003fa0 0x7f41cc1592f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00f460 tx=0x7f41bc00f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41c3e00640 1 -- 172.21.15.44:0/4069361499 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc01a070 con 0x7f41b0003fa0 2024-09-13T08:48:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41c3e00640 1 -- 172.21.15.44:0/4069361499 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41bc008590 con 0x7f41b0003fa0 2024-09-13T08:48:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41c3e00640 1 -- 172.21.15.44:0/4069361499 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc006960 con 0x7f41b0003fa0 2024-09-13T08:48:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.523+0000 7f41d2e00640 1 -- 172.21.15.44:0/4069361499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41cc0c96d0 con 0x7f41b0003fa0 2024-09-13T08:48:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.527+0000 7f41d2e00640 1 -- 172.21.15.44:0/4069361499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41cc159830 con 0x7f41b0003fa0 2024-09-13T08:48:12.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.527+0000 7f41d2e00640 1 -- 172.21.15.44:0/4069361499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41cc161ff0 con 0x7f41b0003fa0 2024-09-13T08:48:12.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.527+0000 7f41c3e00640 1 -- 172.21.15.44:0/4069361499 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f41bc01d070 con 0x7f41b0003fa0 2024-09-13T08:48:12.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.531+0000 7f41c3e00640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41ac0744a0 0x7f41ac076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.531+0000 7f41c3e00640 1 -- 172.21.15.44:0/4069361499 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f41bc0951e0 con 0x7f41b0003fa0 2024-09-13T08:48:12.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.531+0000 7f41cb400640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41ac0744a0 0x7f41ac076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.535+0000 7f41c3e00640 1 -- 172.21.15.44:0/4069361499 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f41bc061a80 con 0x7f41b0003fa0 2024-09-13T08:48:12.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.539+0000 7f41cb400640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41ac0744a0 0x7f41ac076960 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f41b80ae000 tx=0x7f41b8019f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:12.899+0000 7f41d2e00640 1 -- 172.21.15.44:0/4069361499 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone855814", "target": ["mon-mgr", ""]}) -- 0x7f41cc0b9c80 con 0x7f41ac0744a0 2024-09-13T08:48:13.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.707+0000 7f41c3e00640 1 -- 172.21.15.44:0/4069361499 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f41b8005160 con 0x7f41ac0744a0 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 -- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41ac0744a0 msgr2=0x7f41ac076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41ac0744a0 0x7f41ac076960 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f41b80ae000 tx=0x7f41b8019f10 comp rx=0 tx=0).stop 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 -- 172.21.15.44:0/4069361499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41b0003fa0 msgr2=0x7f41cc1592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41b0003fa0 0x7f41cc1592f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00f460 tx=0x7f41bc00f920 comp rx=0 tx=0).stop 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 -- 172.21.15.44:0/4069361499 shutdown_connections 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f41ac0744a0 0x7f41ac076960 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41cc0b9850 0x7f41cc1fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f41cc0ca1b0 0x7f41cc0b9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 --2- 172.21.15.44:0/4069361499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41b0003fa0 0x7f41cc1592f0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 -- 172.21.15.44:0/4069361499 >> 172.21.15.44:0/4069361499 conn(0x7f41cc0d3fb0 msgr2=0x7f41cc0b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 -- 172.21.15.44:0/4069361499 shutdown_connections 2024-09-13T08:48:13.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:13.711+0000 7f41d2e00640 1 -- 172.21.15.44:0/4069361499 wait complete. 2024-09-13T08:48:13.746 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone58715 2024-09-13T08:48:14.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.055+0000 7f22f7c00640 1 -- 172.21.15.44:0/1732116767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22f00cf560 msgr2=0x7f22f0073820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:14.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.055+0000 7f22f7c00640 1 --2- 172.21.15.44:0/1732116767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22f00cf560 0x7f22f0073820 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009840 tx=0x7f22e401c700 comp rx=0 tx=0).stop 2024-09-13T08:48:14.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.055+0000 7f22f7c00640 1 -- 172.21.15.44:0/1732116767 shutdown_connections 2024-09-13T08:48:14.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.055+0000 7f22f7c00640 1 --2- 172.21.15.44:0/1732116767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22f00cf560 0x7f22f0073820 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:14.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.055+0000 7f22f7c00640 1 -- 172.21.15.44:0/1732116767 >> 172.21.15.44:0/1732116767 conn(0x7f22f00b2860 msgr2=0x7f22f00b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:14.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.055+0000 7f22f7c00640 1 -- 172.21.15.44:0/1732116767 shutdown_connections 2024-09-13T08:48:14.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.055+0000 7f22f7c00640 1 -- 172.21.15.44:0/1732116767 wait complete. 2024-09-13T08:48:14.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.055+0000 7f22f7c00640 1 Processor -- start 2024-09-13T08:48:14.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.055+0000 7f22f7c00640 1 -- start start 2024-09-13T08:48:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22f7c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f00cebb0 0x7f22f0158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22f7c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22f014e0c0 0x7f22f014d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22f7c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22f014e5f0 0x7f22f015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:14.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22f7c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22f00ce6e0 con 0x7f22f014e0c0 2024-09-13T08:48:14.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22f7c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f22f00ce560 con 0x7f22f00cebb0 2024-09-13T08:48:14.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22f7c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f22f016a490 con 0x7f22f014e5f0 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22f014e0c0 0x7f22f014d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22f014e0c0 0x7f22f014d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42620/0 (socket says 172.21.15.44:42620) 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22efe00640 1 -- 172.21.15.44:0/2689510840 learned_addr learned my addr 172.21.15.44:0/2689510840 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22f5c00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22f014e5f0 0x7f22f015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22f5200640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f00cebb0 0x7f22f0158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22efe00640 1 -- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22f014e5f0 msgr2=0x7f22f015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22efe00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22f014e5f0 0x7f22f015d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22efe00640 1 -- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f00cebb0 msgr2=0x7f22f0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22efe00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f00cebb0 0x7f22f0158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22efe00640 1 -- 172.21.15.44:0/2689510840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f015e680 con 0x7f22f014e0c0 2024-09-13T08:48:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22efe00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22f014e0c0 0x7f22f014d7e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c3c0 tx=0x7f22dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:14.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.059+0000 7f22ecc00640 1 -- 172.21.15.44:0/2689510840 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc011790 con 0x7f22f014e0c0 2024-09-13T08:48:14.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.063+0000 7f22ecc00640 1 -- 172.21.15.44:0/2689510840 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22dc017070 con 0x7f22f014e0c0 2024-09-13T08:48:14.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.063+0000 7f22ecc00640 1 -- 172.21.15.44:0/2689510840 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc011ce0 con 0x7f22f014e0c0 2024-09-13T08:48:14.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.063+0000 7f22f7c00640 1 -- 172.21.15.44:0/2689510840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f0150290 con 0x7f22f014e0c0 2024-09-13T08:48:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.063+0000 7f22f7c00640 1 -- 172.21.15.44:0/2689510840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f0150850 con 0x7f22f014e0c0 2024-09-13T08:48:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.063+0000 7f22f7c00640 1 -- 172.21.15.44:0/2689510840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f0073820 con 0x7f22f014e0c0 2024-09-13T08:48:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.071+0000 7f22ecc00640 1 -- 172.21.15.44:0/2689510840 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f22dc01c460 con 0x7f22f014e0c0 2024-09-13T08:48:14.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.075+0000 7f22ecc00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22d0074420 0x7f22d00768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:14.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.075+0000 7f22ecc00640 1 -- 172.21.15.44:0/2689510840 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f22dc0937c0 con 0x7f22f014e0c0 2024-09-13T08:48:14.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.079+0000 7f22ecc00640 1 -- 172.21.15.44:0/2689510840 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f22dc060060 con 0x7f22f014e0c0 2024-09-13T08:48:14.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.079+0000 7f22f5200640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22d0074420 0x7f22d00768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.087+0000 7f22f5200640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22d0074420 0x7f22d00768e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f22e40af000 tx=0x7f22e401af40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:14.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:14.535+0000 7f22f7c00640 1 -- 172.21.15.44:0/2689510840 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone58715", "target": ["mon-mgr", ""]}) -- 0x7f22f0071dc0 con 0x7f22d0074420 2024-09-13T08:48:15.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.427+0000 7f22ecc00640 1 -- 172.21.15.44:0/2689510840 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f22f0071dc0 con 0x7f22d0074420 2024-09-13T08:48:15.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 -- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22d0074420 msgr2=0x7f22d00768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:15.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22d0074420 0x7f22d00768e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f22e40af000 tx=0x7f22e401af40 comp rx=0 tx=0).stop 2024-09-13T08:48:15.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 -- 172.21.15.44:0/2689510840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22f014e0c0 msgr2=0x7f22f014d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:15.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22f014e0c0 0x7f22f014d7e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c3c0 tx=0x7f22dc00c880 comp rx=0 tx=0).stop 2024-09-13T08:48:15.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 -- 172.21.15.44:0/2689510840 shutdown_connections 2024-09-13T08:48:15.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f22d0074420 0x7f22d00768e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:15.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f22f014e5f0 0x7f22f015d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:15.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22f014e0c0 0x7f22f014d7e0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:15.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 --2- 172.21.15.44:0/2689510840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f00cebb0 0x7f22f0158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:15.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 -- 172.21.15.44:0/2689510840 >> 172.21.15.44:0/2689510840 conn(0x7f22f00b2860 msgr2=0x7f22f00cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:15.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 -- 172.21.15.44:0/2689510840 shutdown_connections 2024-09-13T08:48:15.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.431+0000 7f22f7c00640 1 -- 172.21.15.44:0/2689510840 wait complete. 2024-09-13T08:48:15.486 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone945516 2024-09-13T08:48:15.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.839+0000 7ff6a0a00640 1 -- 172.21.15.44:0/3188681205 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6940063d0 con 0x7ff688003a60 2024-09-13T08:48:15.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.839+0000 7ff6a8a00640 1 -- 172.21.15.44:0/3188681205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff688003a60 msgr2=0x7ff688003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:15.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.839+0000 7ff6a8a00640 1 --2- 172.21.15.44:0/3188681205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff688003a60 0x7ff688003e80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff6940016c0 tx=0x7ff69401c730 comp rx=0 tx=0).stop 2024-09-13T08:48:15.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.839+0000 7ff6a8a00640 1 -- 172.21.15.44:0/3188681205 shutdown_connections 2024-09-13T08:48:15.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.839+0000 7ff6a8a00640 1 --2- 172.21.15.44:0/3188681205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff688003a60 0x7ff688003e80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:15.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.839+0000 7ff6a8a00640 1 -- 172.21.15.44:0/3188681205 >> 172.21.15.44:0/3188681205 conn(0x7ff6a40b18f0 msgr2=0x7ff6a40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:15.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.839+0000 7ff6a8a00640 1 -- 172.21.15.44:0/3188681205 shutdown_connections 2024-09-13T08:48:15.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.839+0000 7ff6a8a00640 1 -- 172.21.15.44:0/3188681205 wait complete. 2024-09-13T08:48:15.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.843+0000 7ff6a8a00640 1 Processor -- start 2024-09-13T08:48:15.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.843+0000 7ff6a8a00640 1 -- start start 2024-09-13T08:48:15.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.843+0000 7ff6a8a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff688003a60 0x7ff6a414a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:15.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.843+0000 7ff6a8a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff6a4156a70 0x7ff6a4156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:15.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.843+0000 7ff6a8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41542b0 0x7ff6a4154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:15.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.843+0000 7ff6a8a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a40b3ee0 con 0x7ff6a41542b0 2024-09-13T08:48:15.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.843+0000 7ff6a8a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a40b3d60 con 0x7ff688003a60 2024-09-13T08:48:15.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.843+0000 7ff6a8a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a406e7a0 con 0x7ff6a4156a70 2024-09-13T08:48:15.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41542b0 0x7ff6a4154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:15.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41542b0 0x7ff6a4154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42648/0 (socket says 172.21.15.44:42648) 2024-09-13T08:48:15.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a2800640 1 -- 172.21.15.44:0/2607062187 learned_addr learned my addr 172.21.15.44:0/2607062187 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:15.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a1400640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff6a4156a70 0x7ff6a4156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a2800640 1 -- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff6a4156a70 msgr2=0x7ff6a4156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a2800640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff6a4156a70 0x7ff6a4156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a2800640 1 -- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff688003a60 msgr2=0x7ff6a414a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a2800640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff688003a60 0x7ff6a414a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a2800640 1 -- 172.21.15.44:0/2607062187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a414ab60 con 0x7ff6a41542b0 2024-09-13T08:48:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a2800640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41542b0 0x7ff6a4154710 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40b44c0 tx=0x7ff69000b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff69a000640 1 -- 172.21.15.44:0/2607062187 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690005220 con 0x7ff6a41542b0 2024-09-13T08:48:15.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a8a00640 1 -- 172.21.15.44:0/2607062187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a415e9b0 con 0x7ff6a41542b0 2024-09-13T08:48:15.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff69a000640 1 -- 172.21.15.44:0/2607062187 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff690015070 con 0x7ff6a41542b0 2024-09-13T08:48:15.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff6a8a00640 1 -- 172.21.15.44:0/2607062187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a415ef70 con 0x7ff6a41542b0 2024-09-13T08:48:15.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.847+0000 7ff69a000640 1 -- 172.21.15.44:0/2607062187 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900047b0 con 0x7ff6a41542b0 2024-09-13T08:48:15.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.851+0000 7ff6a8a00640 1 -- 172.21.15.44:0/2607062187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff67c0052a0 con 0x7ff6a41542b0 2024-09-13T08:48:15.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.851+0000 7ff69a000640 1 -- 172.21.15.44:0/2607062187 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7ff690004950 con 0x7ff6a41542b0 2024-09-13T08:48:15.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.851+0000 7ff69a000640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff684074620 0x7ff684076ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:15.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.855+0000 7ff69a000640 1 -- 172.21.15.44:0/2607062187 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7ff690005420 con 0x7ff6a41542b0 2024-09-13T08:48:15.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.855+0000 7ff6a1e00640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff684074620 0x7ff684076ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:15.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.859+0000 7ff69a000640 1 -- 172.21.15.44:0/2607062187 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff690020020 con 0x7ff6a41542b0 2024-09-13T08:48:15.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:15.883+0000 7ff6a1e00640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff684074620 0x7ff684076ae0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff688004790 tx=0x7ff688005840 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:16.255+0000 7ff6a8a00640 1 -- 172.21.15.44:0/2607062187 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone945516", "target": ["mon-mgr", ""]}) -- 0x7ff67c000c90 con 0x7ff684074620 2024-09-13T08:48:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.095+0000 7ff69a000640 1 -- 172.21.15.44:0/2607062187 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff67c000c90 con 0x7ff684074620 2024-09-13T08:48:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 -- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff684074620 msgr2=0x7ff684076ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff684074620 0x7ff684076ae0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff688004790 tx=0x7ff688005840 comp rx=0 tx=0).stop 2024-09-13T08:48:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 -- 172.21.15.44:0/2607062187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41542b0 msgr2=0x7ff6a4154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41542b0 0x7ff6a4154710 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40b44c0 tx=0x7ff69000b9a0 comp rx=0 tx=0).stop 2024-09-13T08:48:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 -- 172.21.15.44:0/2607062187 shutdown_connections 2024-09-13T08:48:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff684074620 0x7ff684076ae0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41542b0 0x7ff6a4154710 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff6a4156a70 0x7ff6a4156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 --2- 172.21.15.44:0/2607062187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff688003a60 0x7ff6a414a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 -- 172.21.15.44:0/2607062187 >> 172.21.15.44:0/2607062187 conn(0x7ff6a40b18f0 msgr2=0x7ff6a4072900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.099+0000 7ff6a8a00640 1 -- 172.21.15.44:0/2607062187 shutdown_connections 2024-09-13T08:48:17.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.103+0000 7ff6a8a00640 1 -- 172.21.15.44:0/2607062187 wait complete. 2024-09-13T08:48:17.131 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone453217 2024-09-13T08:48:17.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.427+0000 7f1553e00640 1 -- 172.21.15.44:0/1745124396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154c16ed30 msgr2=0x7f154c16f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:17.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.427+0000 7f1553e00640 1 --2- 172.21.15.44:0/1745124396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154c16ed30 0x7f154c16f180 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f154c069980 tx=0x7f153801cd20 comp rx=0 tx=0).stop 2024-09-13T08:48:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.431+0000 7f1553e00640 1 -- 172.21.15.44:0/1745124396 shutdown_connections 2024-09-13T08:48:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.431+0000 7f1553e00640 1 --2- 172.21.15.44:0/1745124396 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f154c06e7a0 0x7f154c06ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.431+0000 7f1553e00640 1 --2- 172.21.15.44:0/1745124396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154c16ed30 0x7f154c16f180 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.431+0000 7f1553e00640 1 --2- 172.21.15.44:0/1745124396 >> v2:172.21.15.129:3300/0 conn(0x7f154c16ab60 0x7f154c16e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.431+0000 7f1553e00640 1 -- 172.21.15.44:0/1745124396 >> 172.21.15.44:0/1745124396 conn(0x7f154c0b18f0 msgr2=0x7f154c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.431+0000 7f1553e00640 1 -- 172.21.15.44:0/1745124396 shutdown_connections 2024-09-13T08:48:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.431+0000 7f1553e00640 1 -- 172.21.15.44:0/1745124396 wait complete. 2024-09-13T08:48:17.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.443+0000 7f1553e00640 1 Processor -- start 2024-09-13T08:48:17.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.443+0000 7f1553e00640 1 -- start start 2024-09-13T08:48:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1553e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154c06e7a0 0x7f154c1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1553e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154c16ab60 0x7f154c09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1553e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f154c16ed30 0x7f154c09af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1553e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f154c0b3f20 con 0x7f154c16ab60 2024-09-13T08:48:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1553e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f154c0b3da0 con 0x7f154c06e7a0 2024-09-13T08:48:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1553e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f154c0b40a0 con 0x7f154c16ed30 2024-09-13T08:48:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1550a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154c16ab60 0x7f154c09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:17.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1550a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154c16ab60 0x7f154c09aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42656/0 (socket says 172.21.15.44:42656) 2024-09-13T08:48:17.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1550a00640 1 -- 172.21.15.44:0/2669701829 learned_addr learned my addr 172.21.15.44:0/2669701829 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:17.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1551e00640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f154c16ed30 0x7f154c09af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:17.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1551400640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154c06e7a0 0x7f154c1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:17.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1551e00640 1 -- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154c06e7a0 msgr2=0x7f154c1584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:17.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1551e00640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154c06e7a0 0x7f154c1584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:17.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1551e00640 1 -- 172.21.15.44:0/2669701829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154c16ab60 msgr2=0x7f154c09aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:17.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1551e00640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154c16ab60 0x7f154c09aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:17.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.451+0000 7f1551e00640 1 -- 172.21.15.44:0/2669701829 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154c1589e0 con 0x7f154c16ed30 2024-09-13T08:48:17.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.455+0000 7f1551e00640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f154c16ed30 0x7f154c09af50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f154400b3e0 tx=0x7f1544009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:17.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.459+0000 7f1549600640 1 -- 172.21.15.44:0/2669701829 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544015020 con 0x7f154c16ed30 2024-09-13T08:48:17.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.459+0000 7f1549600640 1 -- 172.21.15.44:0/2669701829 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1544011920 con 0x7f154c16ed30 2024-09-13T08:48:17.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.459+0000 7f1549600640 1 -- 172.21.15.44:0/2669701829 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544011d50 con 0x7f154c16ed30 2024-09-13T08:48:17.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.459+0000 7f1553e00640 1 -- 172.21.15.44:0/2669701829 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154c15f9b0 con 0x7f154c16ed30 2024-09-13T08:48:17.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.459+0000 7f1553e00640 1 -- 172.21.15.44:0/2669701829 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f154c072da0 con 0x7f154c16ed30 2024-09-13T08:48:17.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.459+0000 7f1553e00640 1 -- 172.21.15.44:0/2669701829 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f154c16e7f0 con 0x7f154c16ed30 2024-09-13T08:48:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.467+0000 7f1549600640 1 -- 172.21.15.44:0/2669701829 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f1544018020 con 0x7f154c16ed30 2024-09-13T08:48:17.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.471+0000 7f1549600640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f152c074470 0x7f152c076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.471+0000 7f1549600640 1 -- 172.21.15.44:0/2669701829 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f1544093010 con 0x7f154c16ed30 2024-09-13T08:48:17.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.483+0000 7f1551400640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f152c074470 0x7f152c076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:17.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.499+0000 7f1551400640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f152c074470 0x7f152c076930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1540009f60 tx=0x7f15400094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.503+0000 7f1549600640 1 -- 172.21.15.44:0/2669701829 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f154401f020 con 0x7f154c16ed30 2024-09-13T08:48:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:17.935+0000 7f1553e00640 1 -- 172.21.15.44:0/2669701829 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone453217", "target": ["mon-mgr", ""]}) -- 0x7f154c072870 con 0x7f152c074470 2024-09-13T08:48:18.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.631+0000 7f1549600640 1 -- 172.21.15.44:0/2669701829 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f154c072870 con 0x7f152c074470 2024-09-13T08:48:18.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.635+0000 7f1553e00640 1 -- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f152c074470 msgr2=0x7f152c076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:18.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.635+0000 7f1553e00640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f152c074470 0x7f152c076930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1540009f60 tx=0x7f15400094f0 comp rx=0 tx=0).stop 2024-09-13T08:48:18.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.635+0000 7f1553e00640 1 -- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f154c16ed30 msgr2=0x7f154c09af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:18.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.635+0000 7f1553e00640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f154c16ed30 0x7f154c09af50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f154400b3e0 tx=0x7f1544009f90 comp rx=0 tx=0).stop 2024-09-13T08:48:18.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.635+0000 7f1553e00640 1 -- 172.21.15.44:0/2669701829 shutdown_connections 2024-09-13T08:48:18.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.639+0000 7f1553e00640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f152c074470 0x7f152c076930 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:18.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.639+0000 7f1553e00640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f154c16ed30 0x7f154c09af50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:18.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.639+0000 7f1553e00640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154c16ab60 0x7f154c09aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:18.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.639+0000 7f1553e00640 1 --2- 172.21.15.44:0/2669701829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154c06e7a0 0x7f154c1584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:18.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.639+0000 7f1553e00640 1 -- 172.21.15.44:0/2669701829 >> 172.21.15.44:0/2669701829 conn(0x7f154c0b18f0 msgr2=0x7f154c168b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:18.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.643+0000 7f1553e00640 1 -- 172.21.15.44:0/2669701829 shutdown_connections 2024-09-13T08:48:18.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:18.643+0000 7f1553e00640 1 -- 172.21.15.44:0/2669701829 wait complete. 2024-09-13T08:48:18.706 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone300218 2024-09-13T08:48:19.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.007+0000 7fe5ab400640 1 -- 172.21.15.44:0/92479208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5a40d05b0 msgr2=0x7fe5a40d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.007+0000 7fe5ab400640 1 --2- 172.21.15.44:0/92479208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5a40d05b0 0x7fe5a40d0990 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40b33c0 tx=0x7fe59401c770 comp rx=0 tx=0).stop 2024-09-13T08:48:19.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.007+0000 7fe5ab400640 1 -- 172.21.15.44:0/92479208 shutdown_connections 2024-09-13T08:48:19.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.007+0000 7fe5ab400640 1 --2- 172.21.15.44:0/92479208 >> v2:172.21.15.129:3300/0 conn(0x7fe5a40b3f70 0x7fe5a40cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:19.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.007+0000 7fe5ab400640 1 --2- 172.21.15.44:0/92479208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5a40d05b0 0x7fe5a40d0990 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:19.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.007+0000 7fe5ab400640 1 -- 172.21.15.44:0/92479208 >> 172.21.15.44:0/92479208 conn(0x7fe5a40d1e00 msgr2=0x7fe5a40d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:19.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.011+0000 7fe5ab400640 1 -- 172.21.15.44:0/92479208 shutdown_connections 2024-09-13T08:48:19.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.011+0000 7fe5ab400640 1 -- 172.21.15.44:0/92479208 wait complete. 2024-09-13T08:48:19.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.015+0000 7fe5ab400640 1 Processor -- start 2024-09-13T08:48:19.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.015+0000 7fe5ab400640 1 -- start start 2024-09-13T08:48:19.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5ab400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe5a40b2990 0x7fe5a40caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:19.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5ab400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5a40b3f70 0x7fe5a40af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:19.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5ab400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5a40d05b0 0x7fe5a40c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5ab400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4169150 con 0x7fe5a40d05b0 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5ab400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a40cdbc0 con 0x7fe5a40b3f70 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5ab400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a41692d0 con 0x7fe5a40b2990 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5a9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5a40d05b0 0x7fe5a40c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5a9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5a40d05b0 0x7fe5a40c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42672/0 (socket says 172.21.15.44:42672) 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5a9400640 1 -- 172.21.15.44:0/480069040 learned_addr learned my addr 172.21.15.44:0/480069040 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5a9400640 1 -- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe5a40b2990 msgr2=0x7fe5a40caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5a9400640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe5a40b2990 0x7fe5a40caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.023+0000 7fe5a9400640 1 -- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5a40b3f70 msgr2=0x7fe5a40af4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.027+0000 7fe5a3e00640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5a40b3f70 0x7fe5a40af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.027+0000 7fe5a9400640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5a40b3f70 0x7fe5a40af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.027+0000 7fe5a9400640 1 -- 172.21.15.44:0/480069040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a40cb410 con 0x7fe5a40d05b0 2024-09-13T08:48:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.035+0000 7fe5a9400640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5a40d05b0 0x7fe5a40c58a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00b970 tx=0x7fe59c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.039+0000 7fe5a0c00640 1 -- 172.21.15.44:0/480069040 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c019070 con 0x7fe5a40d05b0 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.039+0000 7fe5a0c00640 1 -- 172.21.15.44:0/480069040 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe59c00cc50 con 0x7fe5a40d05b0 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.039+0000 7fe5a0c00640 1 -- 172.21.15.44:0/480069040 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0146a0 con 0x7fe5a40d05b0 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.039+0000 7fe5ab400640 1 -- 172.21.15.44:0/480069040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a40af9f0 con 0x7fe5a40d05b0 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.039+0000 7fe5ab400640 1 -- 172.21.15.44:0/480069040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a40affa0 con 0x7fe5a40d05b0 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.043+0000 7fe5ab400640 1 -- 172.21.15.44:0/480069040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a40cc850 con 0x7fe5a40d05b0 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.047+0000 7fe5a0c00640 1 -- 172.21.15.44:0/480069040 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7fe59c01c070 con 0x7fe5a40d05b0 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.051+0000 7fe5a0c00640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe580074470 0x7fe580076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.051+0000 7fe5a0c00640 1 -- 172.21.15.44:0/480069040 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fe59c05b180 con 0x7fe5a40d05b0 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.055+0000 7fe5a8a00640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe580074470 0x7fe580076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:19.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.059+0000 7fe5a0c00640 1 -- 172.21.15.44:0/480069040 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe59c05f610 con 0x7fe5a40d05b0 2024-09-13T08:48:19.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.059+0000 7fe5a8a00640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe580074470 0x7fe580076930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe5940af000 tx=0x7fe594019510 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:19.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:19.463+0000 7fe5ab400640 1 -- 172.21.15.44:0/480069040 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone300218", "target": ["mon-mgr", ""]}) -- 0x7fe5a4071ff0 con 0x7fe580074470 2024-09-13T08:48:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5a0c00640 1 -- 172.21.15.44:0/480069040 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5a4071ff0 con 0x7fe580074470 2024-09-13T08:48:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5ab400640 1 -- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe580074470 msgr2=0x7fe580076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5ab400640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe580074470 0x7fe580076930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe5940af000 tx=0x7fe594019510 comp rx=0 tx=0).stop 2024-09-13T08:48:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5ab400640 1 -- 172.21.15.44:0/480069040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5a40d05b0 msgr2=0x7fe5a40c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5ab400640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5a40d05b0 0x7fe5a40c58a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00b970 tx=0x7fe59c00be30 comp rx=0 tx=0).stop 2024-09-13T08:48:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5ab400640 1 -- 172.21.15.44:0/480069040 shutdown_connections 2024-09-13T08:48:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5ab400640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe580074470 0x7fe580076930 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5ab400640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5a40d05b0 0x7fe5a40c58a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5ab400640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5a40b3f70 0x7fe5a40af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5ab400640 1 --2- 172.21.15.44:0/480069040 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe5a40b2990 0x7fe5a40caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:20.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.391+0000 7fe5ab400640 1 -- 172.21.15.44:0/480069040 >> 172.21.15.44:0/480069040 conn(0x7fe5a40d1e00 msgr2=0x7fe5a4166700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:20.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.395+0000 7fe5ab400640 1 -- 172.21.15.44:0/480069040 shutdown_connections 2024-09-13T08:48:20.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.395+0000 7fe5ab400640 1 -- 172.21.15.44:0/480069040 wait complete. 2024-09-13T08:48:20.456 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone632019 2024-09-13T08:48:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.811+0000 7f3595200640 1 -- 172.21.15.44:0/1127405892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35900b3d30 msgr2=0x7f35900b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.811+0000 7f3595200640 1 --2- 172.21.15.44:0/1127405892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35900b3d30 0x7f35900b41e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f358800afd0 tx=0x7f358801d860 comp rx=0 tx=0).stop 2024-09-13T08:48:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.811+0000 7f3595200640 1 -- 172.21.15.44:0/1127405892 shutdown_connections 2024-09-13T08:48:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.811+0000 7f3595200640 1 --2- 172.21.15.44:0/1127405892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35900b3d30 0x7f35900b41e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.811+0000 7f3595200640 1 --2- 172.21.15.44:0/1127405892 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f359006f8c0 0x7f359006fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.811+0000 7f3595200640 1 -- 172.21.15.44:0/1127405892 >> 172.21.15.44:0/1127405892 conn(0x7f35900b18f0 msgr2=0x7f35900b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.811+0000 7f3595200640 1 -- 172.21.15.44:0/1127405892 shutdown_connections 2024-09-13T08:48:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.811+0000 7f3595200640 1 -- 172.21.15.44:0/1127405892 wait complete. 2024-09-13T08:48:20.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f3595200640 1 Processor -- start 2024-09-13T08:48:20.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f3595200640 1 -- start start 2024-09-13T08:48:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f3595200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006f8c0 0x7f35901fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:20.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f3595200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35900b3d30 0x7f3590073fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:20.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f3595200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f35900cc740 0x7f3590163960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:20.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f3595200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3590169740 con 0x7f35900b3d30 2024-09-13T08:48:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f3595200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f35901695c0 con 0x7f359006f8c0 2024-09-13T08:48:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f3595200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f35901698c0 con 0x7f35900cc740 2024-09-13T08:48:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f358d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35900b3d30 0x7f3590073fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f358d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35900b3d30 0x7f3590073fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42680/0 (socket says 172.21.15.44:42680) 2024-09-13T08:48:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.815+0000 7f358d400640 1 -- 172.21.15.44:0/982490822 learned_addr learned my addr 172.21.15.44:0/982490822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f358d400640 1 -- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f35900cc740 msgr2=0x7f3590163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f358d400640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f35900cc740 0x7f3590163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f358d400640 1 -- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006f8c0 msgr2=0x7f35901fbd00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:48:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f358d400640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006f8c0 0x7f35901fbd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f358d400640 1 -- 172.21.15.44:0/982490822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3590143a00 con 0x7f35900b3d30 2024-09-13T08:48:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f358d400640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35900b3d30 0x7f3590073fa0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f357c00b410 tx=0x7f357c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f3586000640 1 -- 172.21.15.44:0/982490822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c015020 con 0x7f35900b3d30 2024-09-13T08:48:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f3595200640 1 -- 172.21.15.44:0/982490822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35901fc240 con 0x7f35900b3d30 2024-09-13T08:48:20.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f3586000640 1 -- 172.21.15.44:0/982490822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f357c011940 con 0x7f35900b3d30 2024-09-13T08:48:20.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f3586000640 1 -- 172.21.15.44:0/982490822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c011d90 con 0x7f35900b3d30 2024-09-13T08:48:20.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f3595200640 1 -- 172.21.15.44:0/982490822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3590132290 con 0x7f35900b3d30 2024-09-13T08:48:20.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.819+0000 7f3595200640 1 -- 172.21.15.44:0/982490822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359006fca0 con 0x7f35900b3d30 2024-09-13T08:48:20.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.823+0000 7f3586000640 1 -- 172.21.15.44:0/982490822 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f357c018020 con 0x7f35900b3d30 2024-09-13T08:48:20.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.823+0000 7f3586000640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3570074420 0x7f35700768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:20.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.823+0000 7f3586000640 1 -- 172.21.15.44:0/982490822 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f357c093470 con 0x7f35900b3d30 2024-09-13T08:48:20.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.827+0000 7f3586000640 1 -- 172.21.15.44:0/982490822 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f357c01f020 con 0x7f35900b3d30 2024-09-13T08:48:20.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.843+0000 7f358de00640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3570074420 0x7f35700768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:20.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:20.851+0000 7f358de00640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3570074420 0x7f35700768e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3580009a70 tx=0x7f35800091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:21.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:21.231+0000 7f3595200640 1 -- 172.21.15.44:0/982490822 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone632019", "target": ["mon-mgr", ""]}) -- 0x7f359015d300 con 0x7f3570074420 2024-09-13T08:48:22.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.179+0000 7f3586000640 1 -- 172.21.15.44:0/982490822 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f359015d300 con 0x7f3570074420 2024-09-13T08:48:22.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.183+0000 7f3595200640 1 -- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3570074420 msgr2=0x7f35700768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:22.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.183+0000 7f3595200640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3570074420 0x7f35700768e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3580009a70 tx=0x7f35800091e0 comp rx=0 tx=0).stop 2024-09-13T08:48:22.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.183+0000 7f3595200640 1 -- 172.21.15.44:0/982490822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35900b3d30 msgr2=0x7f3590073fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:22.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.183+0000 7f3595200640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35900b3d30 0x7f3590073fa0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f357c00b410 tx=0x7f357c00b8d0 comp rx=0 tx=0).stop 2024-09-13T08:48:22.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.187+0000 7f3595200640 1 -- 172.21.15.44:0/982490822 shutdown_connections 2024-09-13T08:48:22.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.187+0000 7f3595200640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3570074420 0x7f35700768e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:22.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.187+0000 7f3595200640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f35900cc740 0x7f3590163960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:22.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.187+0000 7f3595200640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35900b3d30 0x7f3590073fa0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:22.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.187+0000 7f3595200640 1 --2- 172.21.15.44:0/982490822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006f8c0 0x7f35901fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:22.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.187+0000 7f3595200640 1 -- 172.21.15.44:0/982490822 >> 172.21.15.44:0/982490822 conn(0x7f35900b18f0 msgr2=0x7f35900b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:22.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.187+0000 7f3595200640 1 -- 172.21.15.44:0/982490822 shutdown_connections 2024-09-13T08:48:22.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:22.191+0000 7f3595200640 1 -- 172.21.15.44:0/982490822 wait complete. 2024-09-13T08:48:24.240 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:48:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.487+0000 7f38d6a00640 1 -- 172.21.15.44:0/1837769127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38d015e030 msgr2=0x7f38d006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.487+0000 7f38d6a00640 1 --2- 172.21.15.44:0/1837769127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38d015e030 0x7f38d006ecb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009900 tx=0x7f38c001f0f0 comp rx=0 tx=0).stop 2024-09-13T08:48:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.487+0000 7f38d6a00640 1 -- 172.21.15.44:0/1837769127 shutdown_connections 2024-09-13T08:48:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.487+0000 7f38d6a00640 1 --2- 172.21.15.44:0/1837769127 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38d00d00c0 0x7f38d00ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.487+0000 7f38d6a00640 1 --2- 172.21.15.44:0/1837769127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38d015e030 0x7f38d006ecb0 secure :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009900 tx=0x7f38c001f0f0 comp rx=0 tx=0).stop 2024-09-13T08:48:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.487+0000 7f38d6a00640 1 -- 172.21.15.44:0/1837769127 >> 172.21.15.44:0/1837769127 conn(0x7f38d00d1da0 msgr2=0x7f38d00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.487+0000 7f38d6a00640 1 -- 172.21.15.44:0/1837769127 shutdown_connections 2024-09-13T08:48:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.487+0000 7f38d6a00640 1 -- 172.21.15.44:0/1837769127 wait complete. 2024-09-13T08:48:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38d6a00640 1 Processor -- start 2024-09-13T08:48:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38d6a00640 1 -- start start 2024-09-13T08:48:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38d6a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38d00d00c0 0x7f38d0160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38d6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d01fd200 0x7f38d00b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38d6a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38d01fd680 0x7f38d00b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38d6a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f38d00cc8f0 con 0x7f38d01fd200 2024-09-13T08:48:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38d6a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38d00cc770 con 0x7f38d01fd680 2024-09-13T08:48:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38d6a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f38d00cca70 con 0x7f38d00d00c0 2024-09-13T08:48:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d01fd200 0x7f38d00b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d01fd200 0x7f38d00b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58102/0 (socket says 172.21.15.44:58102) 2024-09-13T08:48:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38cf400640 1 -- 172.21.15.44:0/4279591547 learned_addr learned my addr 172.21.15.44:0/4279591547 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38cf400640 1 -- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38d00d00c0 msgr2=0x7f38d0160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38cf400640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38d00d00c0 0x7f38d0160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38cf400640 1 -- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38d01fd680 msgr2=0x7f38d00b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38cf400640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38d01fd680 0x7f38d00b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38cf400640 1 -- 172.21.15.44:0/4279591547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d01f97a0 con 0x7f38d01fd200 2024-09-13T08:48:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.491+0000 7f38cf400640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d01fd200 0x7f38d00b1200 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f38bc001ec0 tx=0x7f38bc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.495+0000 7f38c7e00640 1 -- 172.21.15.44:0/4279591547 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc017070 con 0x7f38d01fd200 2024-09-13T08:48:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.495+0000 7f38c7e00640 1 -- 172.21.15.44:0/4279591547 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38bc0053c0 con 0x7f38d01fd200 2024-09-13T08:48:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.495+0000 7f38d6a00640 1 -- 172.21.15.44:0/4279591547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d01f9a80 con 0x7f38d01fd200 2024-09-13T08:48:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.495+0000 7f38c7e00640 1 -- 172.21.15.44:0/4279591547 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc003860 con 0x7f38d01fd200 2024-09-13T08:48:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.495+0000 7f38d6a00640 1 -- 172.21.15.44:0/4279591547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d01fa040 con 0x7f38d01fd200 2024-09-13T08:48:24.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.495+0000 7f38d6a00640 1 -- 172.21.15.44:0/4279591547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d01fa370 con 0x7f38d01fd200 2024-09-13T08:48:24.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.499+0000 7f38c7e00640 1 -- 172.21.15.44:0/4279591547 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f38bc01a070 con 0x7f38d01fd200 2024-09-13T08:48:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.499+0000 7f38c7e00640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f38b0074450 0x7f38b0076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.499+0000 7f38c7e00640 1 -- 172.21.15.44:0/4279591547 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f38bc092300 con 0x7f38d01fd200 2024-09-13T08:48:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.499+0000 7f38cfe00640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f38b0074450 0x7f38b0076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:24.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.503+0000 7f38cfe00640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f38b0074450 0x7f38b0076910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f38c00b2000 tx=0x7f38c001be90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.507+0000 7f38c7e00640 1 -- 172.21.15.44:0/4279591547 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f38bc05eba0 con 0x7f38d01fd200 2024-09-13T08:48:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.971+0000 7f38d6a00640 1 -- 172.21.15.44:0/4279591547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f38d0070350 con 0x7f38d01fd200 2024-09-13T08:48:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.971+0000 7f38c7e00640 1 -- 172.21.15.44:0/4279591547 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2443 (secure 0 0 0) 0x7f38bc05e4a0 con 0x7f38d01fd200 2024-09-13T08:48:24.974 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:48:24.974 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:48:24.974 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:48:24.974 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:48:24.974 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:48:24.974 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:48:24.974 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:48:24.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:24.974 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 675, 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:48:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 89, 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:24.976 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 6089, 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 25336405908, 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 50435309568, 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 722658803712, 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:48:24.977 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 824533488, 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 827415223, 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 200, 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 224 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 57, 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:47:15:289304+0000", 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 6, 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:24.978 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 6, 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 6942 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:48:24.979 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 17, 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:48:14.719960+0000", 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": { 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-ongoing-clones": { 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "4 ongoing clones - average progress is 41.081% (32s)\n [===========.................] (remaining: 45s)", 2024-09-13T08:48:24.980 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.41080999374389648, 2024-09-13T08:48:24.981 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:48:24.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:24.981 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr-vol-total-clones": { 2024-09-13T08:48:24.981 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "Total 13 clones - average progress is 18.99% (27s)\n [=====.......................] (remaining: 115s)", 2024-09-13T08:48:24.981 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.1898999959230423, 2024-09-13T08:48:24.981 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:48:24.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:48:24.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:48:24.981 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:48:24.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 -- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f38b0074450 msgr2=0x7f38b0076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:24.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f38b0074450 0x7f38b0076910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f38c00b2000 tx=0x7f38c001be90 comp rx=0 tx=0).stop 2024-09-13T08:48:24.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 -- 172.21.15.44:0/4279591547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d01fd200 msgr2=0x7f38d00b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:24.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d01fd200 0x7f38d00b1200 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f38bc001ec0 tx=0x7f38bc00c890 comp rx=0 tx=0).stop 2024-09-13T08:48:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 -- 172.21.15.44:0/4279591547 shutdown_connections 2024-09-13T08:48:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f38b0074450 0x7f38b0076910 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38d01fd680 0x7f38d00b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d01fd200 0x7f38d00b1200 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 --2- 172.21.15.44:0/4279591547 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38d00d00c0 0x7f38d0160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 -- 172.21.15.44:0/4279591547 >> 172.21.15.44:0/4279591547 conn(0x7f38d00d1da0 msgr2=0x7f38d00d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 -- 172.21.15.44:0/4279591547 shutdown_connections 2024-09-13T08:48:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:24.975+0000 7f38d6a00640 1 -- 172.21.15.44:0/4279591547 wait complete. 2024-09-13T08:48:25.021 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 41.081% (32s)\n [===========.................] (remaining: 45s)', 'progress': 0.4108099937438965, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 13 clones - average progress is 18.99% (27s)\n [=====.......................] (remaining: 115s)', 'progress': 0.1898999959230423, 'add_to_ceph_s': True}} 2024-09-13T08:48:25.021 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-09-13T08:48:25.021 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.1898999959230423; onpen_p = 0.25885000824928284 2024-09-13T08:48:25.021 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone6700 2024-09-13T08:48:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.299+0000 7f5cb7000640 1 -- 172.21.15.44:0/2138449716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cb00b1650 msgr2=0x7f5cb00b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.299+0000 7f5cb7000640 1 --2- 172.21.15.44:0/2138449716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cb00b1650 0x7f5cb00b1a30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009900 tx=0x7f5c9801c8e0 comp rx=0 tx=0).stop 2024-09-13T08:48:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.299+0000 7f5cb7000640 1 -- 172.21.15.44:0/2138449716 shutdown_connections 2024-09-13T08:48:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.299+0000 7f5cb7000640 1 --2- 172.21.15.44:0/2138449716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cb00b1650 0x7f5cb00b1a30 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.299+0000 7f5cb7000640 1 --2- 172.21.15.44:0/2138449716 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5cb01605d0 0x7f5cb00d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.299+0000 7f5cb7000640 1 -- 172.21.15.44:0/2138449716 >> 172.21.15.44:0/2138449716 conn(0x7f5cb00d1dd0 msgr2=0x7f5cb00d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.299+0000 7f5cb7000640 1 -- 172.21.15.44:0/2138449716 shutdown_connections 2024-09-13T08:48:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.299+0000 7f5cb7000640 1 -- 172.21.15.44:0/2138449716 wait complete. 2024-09-13T08:48:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cb7000640 1 Processor -- start 2024-09-13T08:48:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cb7000640 1 -- start start 2024-09-13T08:48:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cb7000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cb00aeb00 0x7f5cb00cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cb7000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cb00b1650 0x7f5cb00cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cb7000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5cb01605d0 0x7f5cb015e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cb7000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb00b1220 con 0x7f5cb00aeb00 2024-09-13T08:48:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cb7000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb00b0ee0 con 0x7f5cb00b1650 2024-09-13T08:48:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cb7000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb00b13a0 con 0x7f5cb01605d0 2024-09-13T08:48:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cafe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cb00aeb00 0x7f5cb00cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cafe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cb00aeb00 0x7f5cb00cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58128/0 (socket says 172.21.15.44:58128) 2024-09-13T08:48:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cafe00640 1 -- 172.21.15.44:0/2838925002 learned_addr learned my addr 172.21.15.44:0/2838925002 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cafe00640 1 -- 172.21.15.44:0/2838925002 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5cb01605d0 msgr2=0x7f5cb015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cafe00640 1 --2- 172.21.15.44:0/2838925002 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5cb01605d0 0x7f5cb015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cafe00640 1 -- 172.21.15.44:0/2838925002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cb00b1650 msgr2=0x7f5cb00cb160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:48:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cafe00640 1 --2- 172.21.15.44:0/2838925002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cb00b1650 0x7f5cb00cb160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cafe00640 1 -- 172.21.15.44:0/2838925002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb0155fe0 con 0x7f5cb00aeb00 2024-09-13T08:48:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cafe00640 1 --2- 172.21.15.44:0/2838925002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cb00aeb00 0x7f5cb00cc630 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000b410 tx=0x7f5ca000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cacc00640 1 -- 172.21.15.44:0/2838925002 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0015020 con 0x7f5cb00aeb00 2024-09-13T08:48:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cacc00640 1 -- 172.21.15.44:0/2838925002 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ca0011920 con 0x7f5cb00aeb00 2024-09-13T08:48:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cacc00640 1 -- 172.21.15.44:0/2838925002 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0011d70 con 0x7f5cb00aeb00 2024-09-13T08:48:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cb7000640 1 -- 172.21.15.44:0/2838925002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb015faa0 con 0x7f5cb00aeb00 2024-09-13T08:48:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.303+0000 7f5cb7000640 1 -- 172.21.15.44:0/2838925002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb015df50 con 0x7f5cb00aeb00 2024-09-13T08:48:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.307+0000 7f5cb7000640 1 -- 172.21.15.44:0/2838925002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb00b1a30 con 0x7f5cb00aeb00 2024-09-13T08:48:25.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.311+0000 7f5cacc00640 1 -- 172.21.15.44:0/2838925002 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f5ca0018020 con 0x7f5cb00aeb00 2024-09-13T08:48:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.311+0000 7f5cacc00640 1 --2- 172.21.15.44:0/2838925002 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5c8c074420 0x7f5c8c0768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.311+0000 7f5cacc00640 1 -- 172.21.15.44:0/2838925002 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f5ca00932e0 con 0x7f5cb00aeb00 2024-09-13T08:48:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.315+0000 7f5ca7e00640 1 --2- 172.21.15.44:0/2838925002 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5c8c074420 0x7f5c8c0768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:25.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.315+0000 7f5ca7e00640 1 --2- 172.21.15.44:0/2838925002 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5c8c074420 0x7f5c8c0768e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5c980af000 tx=0x7f5c9801b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:25.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.315+0000 7f5cacc00640 1 -- 172.21.15.44:0/2838925002 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5ca001f020 con 0x7f5cb00aeb00 2024-09-13T08:48:25.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:25.711+0000 7f5cb7000640 1 -- 172.21.15.44:0/2838925002 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone6700", "target": ["mon-mgr", ""]}) -- 0x7f5cb0164ef0 con 0x7f5c8c074420 2024-09-13T08:48:26.059 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:26.058+0000 7f8ac3e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.067+0000 7f5cacc00640 1 -- 172.21.15.44:0/2838925002 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f5cb0164ef0 con 0x7f5c8c074420 2024-09-13T08:48:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:26.085 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T08:48:26.086 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone5221 2024-09-13T08:48:26.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.339+0000 7f7281a00640 1 -- 172.21.15.44:0/406683710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f727c0b1650 msgr2=0x7f727c0b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:26.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.339+0000 7f7281a00640 1 --2- 172.21.15.44:0/406683710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f727c0b1650 0x7f727c0b1a30 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f726c009bc0 tx=0x7f726c01cba0 comp rx=0 tx=0).stop 2024-09-13T08:48:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.339+0000 7f7281a00640 1 -- 172.21.15.44:0/406683710 shutdown_connections 2024-09-13T08:48:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.339+0000 7f7281a00640 1 --2- 172.21.15.44:0/406683710 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f727c0afb10 0x7f727c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.339+0000 7f7281a00640 1 --2- 172.21.15.44:0/406683710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f727c0b1650 0x7f727c0b1a30 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.339+0000 7f7281a00640 1 -- 172.21.15.44:0/406683710 >> 172.21.15.44:0/406683710 conn(0x7f727c0d1dd0 msgr2=0x7f727c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.339+0000 7f7281a00640 1 -- 172.21.15.44:0/406683710 shutdown_connections 2024-09-13T08:48:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.339+0000 7f7281a00640 1 -- 172.21.15.44:0/406683710 wait complete. 2024-09-13T08:48:26.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f7281a00640 1 Processor -- start 2024-09-13T08:48:26.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f7281a00640 1 -- start start 2024-09-13T08:48:26.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f7281a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f727c0afb10 0x7f727c1f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:26.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f7281a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f727c0b1650 0x7f727c15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f7281a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f727c1605d0 0x7f727c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f7281a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f727c0b0ec0 con 0x7f727c0afb10 2024-09-13T08:48:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f7281a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f727c0b1430 con 0x7f727c0b1650 2024-09-13T08:48:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f7281a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f727c169540 con 0x7f727c1605d0 2024-09-13T08:48:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f727aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f727c0afb10 0x7f727c1f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f727aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f727c0afb10 0x7f727c1f8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58144/0 (socket says 172.21.15.44:58144) 2024-09-13T08:48:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.343+0000 7f727aa00640 1 -- 172.21.15.44:0/2560637016 learned_addr learned my addr 172.21.15.44:0/2560637016 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:26.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f727aa00640 1 -- 172.21.15.44:0/2560637016 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f727c1605d0 msgr2=0x7f727c0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:26.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f727aa00640 1 --2- 172.21.15.44:0/2560637016 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f727c1605d0 0x7f727c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:26.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f727aa00640 1 -- 172.21.15.44:0/2560637016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f727c0b1650 msgr2=0x7f727c15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:26.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f727aa00640 1 --2- 172.21.15.44:0/2560637016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f727c0b1650 0x7f727c15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:26.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f727aa00640 1 -- 172.21.15.44:0/2560637016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c15de40 con 0x7f727c0afb10 2024-09-13T08:48:26.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f727aa00640 1 --2- 172.21.15.44:0/2560637016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f727c0afb10 0x7f727c1f8930 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f726400c430 tx=0x7f726400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:26.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f7272a00640 1 -- 172.21.15.44:0/2560637016 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264011790 con 0x7f727c0afb10 2024-09-13T08:48:26.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f7272a00640 1 -- 172.21.15.44:0/2560637016 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7264017070 con 0x7f727c0afb10 2024-09-13T08:48:26.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f7272a00640 1 -- 172.21.15.44:0/2560637016 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264011ce0 con 0x7f727c0afb10 2024-09-13T08:48:26.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f7281a00640 1 -- 172.21.15.44:0/2560637016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c15e120 con 0x7f727c0afb10 2024-09-13T08:48:26.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f7281a00640 1 -- 172.21.15.44:0/2560637016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c15e6e0 con 0x7f727c0afb10 2024-09-13T08:48:26.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.347+0000 7f7281a00640 1 -- 172.21.15.44:0/2560637016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c0affc0 con 0x7f727c0afb10 2024-09-13T08:48:26.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.351+0000 7f7272a00640 1 -- 172.21.15.44:0/2560637016 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f726401c460 con 0x7f727c0afb10 2024-09-13T08:48:26.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.351+0000 7f7272a00640 1 --2- 172.21.15.44:0/2560637016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7258074420 0x7f72580768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:26.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.355+0000 7f7272a00640 1 -- 172.21.15.44:0/2560637016 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f7264093810 con 0x7f727c0afb10 2024-09-13T08:48:26.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.355+0000 7f7272a00640 1 -- 172.21.15.44:0/2560637016 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f72640601b0 con 0x7f727c0afb10 2024-09-13T08:48:26.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.383+0000 7f727a000640 1 --2- 172.21.15.44:0/2560637016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7258074420 0x7f72580768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.391+0000 7f727a000640 1 --2- 172.21.15.44:0/2560637016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7258074420 0x7f72580768e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f726c0af000 tx=0x7f726c019940 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:26.775+0000 7f7281a00640 1 -- 172.21.15.44:0/2560637016 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone5221", "target": ["mon-mgr", ""]}) -- 0x7f727c0d1080 con 0x7f7258074420 2024-09-13T08:48:26.968 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:26.966+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:26.968 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:26.966+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:26.968 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:26.966+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:26.968 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:26.966+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:26.969 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:26.966+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:27.206 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:27.206+0000 7f8ac3e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:27.223 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:27.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.219+0000 7f7272a00640 1 -- 172.21.15.44:0/2560637016 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f727c0d1080 con 0x7f7258074420 2024-09-13T08:48:27.250 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T08:48:27.251 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone25122 2024-09-13T08:48:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.547+0000 7f473c200640 1 -- 172.21.15.44:0/3861373479 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47341584d0 msgr2=0x7f473415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.547+0000 7f473c200640 1 --2- 172.21.15.44:0/3861373479 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47341584d0 0x7f473415c220 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4728009900 tx=0x7f472801c8e0 comp rx=0 tx=0).stop 2024-09-13T08:48:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.547+0000 7f473c200640 1 -- 172.21.15.44:0/3861373479 shutdown_connections 2024-09-13T08:48:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.547+0000 7f473c200640 1 --2- 172.21.15.44:0/3861373479 >> v2:172.21.15.129:3300/0 conn(0x7f47340740c0 0x7f4734074520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.547+0000 7f473c200640 1 --2- 172.21.15.44:0/3861373479 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47341584d0 0x7f473415c220 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.547+0000 7f473c200640 1 -- 172.21.15.44:0/3861373479 >> 172.21.15.44:0/3861373479 conn(0x7f473406ecb0 msgr2=0x7f473406f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.547+0000 7f473c200640 1 -- 172.21.15.44:0/3861373479 shutdown_connections 2024-09-13T08:48:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.547+0000 7f473c200640 1 -- 172.21.15.44:0/3861373479 wait complete. 2024-09-13T08:48:27.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f473c200640 1 Processor -- start 2024-09-13T08:48:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f473c200640 1 -- start start 2024-09-13T08:48:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f473c200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47340740c0 0x7f47340aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f473c200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47341584d0 0x7f473415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f473c200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f473415f220 0x7f473406ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f473c200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4734167960 con 0x7f47341584d0 2024-09-13T08:48:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f473c200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47341677e0 con 0x7f47340740c0 2024-09-13T08:48:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f473c200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4734167ae0 con 0x7f473415f220 2024-09-13T08:48:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f4738e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47341584d0 0x7f473415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f4738e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47341584d0 0x7f473415a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58154/0 (socket says 172.21.15.44:58154) 2024-09-13T08:48:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f4738e00640 1 -- 172.21.15.44:0/341660678 learned_addr learned my addr 172.21.15.44:0/341660678 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f4738e00640 1 -- 172.21.15.44:0/341660678 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f473415f220 msgr2=0x7f473406ff10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:48:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f4738e00640 1 --2- 172.21.15.44:0/341660678 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f473415f220 0x7f473406ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f4738e00640 1 -- 172.21.15.44:0/341660678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47340740c0 msgr2=0x7f47340aef30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:48:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f4738e00640 1 --2- 172.21.15.44:0/341660678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47340740c0 0x7f47340aef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.551+0000 7f4738e00640 1 -- 172.21.15.44:0/341660678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4734161ac0 con 0x7f47341584d0 2024-09-13T08:48:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.555+0000 7f4738e00640 1 --2- 172.21.15.44:0/341660678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47341584d0 0x7f473415a110 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f472000c3f0 tx=0x7f472000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.555+0000 7f4731600640 1 -- 172.21.15.44:0/341660678 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720017070 con 0x7f47341584d0 2024-09-13T08:48:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.555+0000 7f4731600640 1 -- 172.21.15.44:0/341660678 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47200053c0 con 0x7f47341584d0 2024-09-13T08:48:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.555+0000 7f4731600640 1 -- 172.21.15.44:0/341660678 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720003860 con 0x7f47341584d0 2024-09-13T08:48:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.555+0000 7f473c200640 1 -- 172.21.15.44:0/341660678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4734161da0 con 0x7f47341584d0 2024-09-13T08:48:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.555+0000 7f473c200640 1 -- 172.21.15.44:0/341660678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4734162360 con 0x7f47341584d0 2024-09-13T08:48:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.555+0000 7f473c200640 1 -- 172.21.15.44:0/341660678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4734074520 con 0x7f47341584d0 2024-09-13T08:48:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.559+0000 7f4731600640 1 -- 172.21.15.44:0/341660678 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 99018+0+0 (secure 0 0 0) 0x7f472001a070 con 0x7f47341584d0 2024-09-13T08:48:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.559+0000 7f4731600640 1 --2- 172.21.15.44:0/341660678 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4714074420 0x7f47140768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.559+0000 7f4731600640 1 -- 172.21.15.44:0/341660678 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f4720092210 con 0x7f47341584d0 2024-09-13T08:48:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.567+0000 7f4731600640 1 -- 172.21.15.44:0/341660678 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4720001090 con 0x7f47341584d0 2024-09-13T08:48:27.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.599+0000 7f4739800640 1 --2- 172.21.15.44:0/341660678 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4714074420 0x7f47140768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:27.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.611+0000 7f4739800640 1 --2- 172.21.15.44:0/341660678 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4714074420 0x7f47140768e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f47280af000 tx=0x7f472801b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:27.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:27.987+0000 7f473c200640 1 -- 172.21.15.44:0/341660678 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone25122", "target": ["mon-mgr", ""]}) -- 0x7f4734162740 con 0x7f4714074420 2024-09-13T08:48:28.644 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:28.642+0000 7f8ac3e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:28.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:28.651+0000 7f4731600640 1 -- 172.21.15.44:0/341660678 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f4734162740 con 0x7f4714074420 2024-09-13T08:48:28.652 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:28.686 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T08:48:28.687 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone11943 2024-09-13T08:48:29.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.075+0000 7fe977e00640 1 -- 172.21.15.44:0/1403781943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9700ba430 msgr2=0x7fe970165d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:29.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.075+0000 7fe977e00640 1 --2- 172.21.15.44:0/1403781943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9700ba430 0x7fe970165d80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe9700be3a0 tx=0x7fe96801ca00 comp rx=0 tx=0).stop 2024-09-13T08:48:29.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.075+0000 7fe977e00640 1 -- 172.21.15.44:0/1403781943 shutdown_connections 2024-09-13T08:48:29.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.075+0000 7fe977e00640 1 --2- 172.21.15.44:0/1403781943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9700ba430 0x7fe970165d80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:29.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.075+0000 7fe977e00640 1 --2- 172.21.15.44:0/1403781943 >> v2:172.21.15.44:3300/0 conn(0x7fe9700d6040 0x7fe9700b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:29.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.075+0000 7fe977e00640 1 --2- 172.21.15.44:0/1403781943 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe97015c9a0 0x7fe9700d7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:29.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.075+0000 7fe977e00640 1 -- 172.21.15.44:0/1403781943 >> 172.21.15.44:0/1403781943 conn(0x7fe9700d4ef0 msgr2=0x7fe9700d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:29.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.079+0000 7fe977e00640 1 -- 172.21.15.44:0/1403781943 shutdown_connections 2024-09-13T08:48:29.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.079+0000 7fe977e00640 1 -- 172.21.15.44:0/1403781943 wait complete. 2024-09-13T08:48:29.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.079+0000 7fe977e00640 1 Processor -- start 2024-09-13T08:48:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.079+0000 7fe977e00640 1 -- start start 2024-09-13T08:48:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.079+0000 7fe977e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9700ba430 0x7fe9700cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:29.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.079+0000 7fe977e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9700d6040 0x7fe97015e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe977e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97015c9a0 0x7fe9700bc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe977e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe97006eb80 con 0x7fe9700ba430 2024-09-13T08:48:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe977e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe97006ea00 con 0x7fe97015c9a0 2024-09-13T08:48:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe977e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe97006ed00 con 0x7fe9700d6040 2024-09-13T08:48:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe975400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9700ba430 0x7fe9700cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:29.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe975400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9700ba430 0x7fe9700cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58158/0 (socket says 172.21.15.44:58158) 2024-09-13T08:48:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe975400640 1 -- 172.21.15.44:0/3765849359 learned_addr learned my addr 172.21.15.44:0/3765849359 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe975400640 1 -- 172.21.15.44:0/3765849359 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9700d6040 msgr2=0x7fe97015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe975400640 1 --2- 172.21.15.44:0/3765849359 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9700d6040 0x7fe97015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe975400640 1 -- 172.21.15.44:0/3765849359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97015c9a0 msgr2=0x7fe9700bc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:29.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe975400640 1 --2- 172.21.15.44:0/3765849359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97015c9a0 0x7fe9700bc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:29.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe975400640 1 -- 172.21.15.44:0/3765849359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe970165a10 con 0x7fe9700ba430 2024-09-13T08:48:29.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.083+0000 7fe975400640 1 --2- 172.21.15.44:0/3765849359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9700ba430 0x7fe9700cc3f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e970 tx=0x7fe96400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:29.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.099+0000 7fe96d600640 1 -- 172.21.15.44:0/3765849359 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400cd80 con 0x7fe9700ba430 2024-09-13T08:48:29.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.099+0000 7fe96d600640 1 -- 172.21.15.44:0/3765849359 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe964004020 con 0x7fe9700ba430 2024-09-13T08:48:29.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.099+0000 7fe96d600640 1 -- 172.21.15.44:0/3765849359 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964004470 con 0x7fe9700ba430 2024-09-13T08:48:29.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.099+0000 7fe977e00640 1 -- 172.21.15.44:0/3765849359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97015f020 con 0x7fe9700ba430 2024-09-13T08:48:29.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.099+0000 7fe977e00640 1 -- 172.21.15.44:0/3765849359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe970161ec0 con 0x7fe9700ba430 2024-09-13T08:48:29.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.099+0000 7fe977e00640 1 -- 172.21.15.44:0/3765849359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9700b9ef0 con 0x7fe9700ba430 2024-09-13T08:48:29.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.099+0000 7fe96d600640 1 -- 172.21.15.44:0/3765849359 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7fe964003600 con 0x7fe9700ba430 2024-09-13T08:48:29.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.103+0000 7fe96d600640 1 --2- 172.21.15.44:0/3765849359 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe9500744e0 0x7fe9500769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:29.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.103+0000 7fe96d600640 1 -- 172.21.15.44:0/3765849359 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fe9640934d0 con 0x7fe9700ba430 2024-09-13T08:48:29.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.103+0000 7fe974a00640 1 --2- 172.21.15.44:0/3765849359 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe9500744e0 0x7fe9500769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:29.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.107+0000 7fe974a00640 1 --2- 172.21.15.44:0/3765849359 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe9500744e0 0x7fe9500769a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe95c005d10 tx=0x7fe95c005c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:29.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.111+0000 7fe96d600640 1 -- 172.21.15.44:0/3765849359 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe9640601d0 con 0x7fe9700ba430 2024-09-13T08:48:29.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.539+0000 7fe977e00640 1 -- 172.21.15.44:0/3765849359 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone11943", "target": ["mon-mgr", ""]}) -- 0x7fe970161550 con 0x7fe9500744e0 2024-09-13T08:48:29.909 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:29.910+0000 7f8ac3e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:29.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:29.907+0000 7fe96d600640 1 -- 172.21.15.44:0/3765849359 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fe970161550 con 0x7fe9500744e0 2024-09-13T08:48:29.910 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:29.928 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T08:48:29.929 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone25384 2024-09-13T08:48:30.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.247+0000 7f5e15c00640 1 -- 172.21.15.44:0/2038029695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e1015e420 msgr2=0x7f5e100cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:30.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.247+0000 7f5e15c00640 1 --2- 172.21.15.44:0/2038029695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e1015e420 0x7f5e100cd150 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5e1006eca0 tx=0x7f5dfc01c850 comp rx=0 tx=0).stop 2024-09-13T08:48:30.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.247+0000 7f5e15c00640 1 -- 172.21.15.44:0/2038029695 shutdown_connections 2024-09-13T08:48:30.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.247+0000 7f5e15c00640 1 --2- 172.21.15.44:0/2038029695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e1015e420 0x7f5e100cd150 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:30.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.247+0000 7f5e15c00640 1 --2- 172.21.15.44:0/2038029695 >> v2:172.21.15.129:3300/0 conn(0x7f5e101518a0 0x7f5e1015b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:30.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.247+0000 7f5e15c00640 1 -- 172.21.15.44:0/2038029695 >> 172.21.15.44:0/2038029695 conn(0x7f5e100b2680 msgr2=0x7f5e100b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:30.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.247+0000 7f5e15c00640 1 -- 172.21.15.44:0/2038029695 shutdown_connections 2024-09-13T08:48:30.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.247+0000 7f5e15c00640 1 -- 172.21.15.44:0/2038029695 wait complete. 2024-09-13T08:48:30.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.247+0000 7f5e15c00640 1 Processor -- start 2024-09-13T08:48:30.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.247+0000 7f5e15c00640 1 -- start start 2024-09-13T08:48:30.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e15c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101518a0 0x7f5e1014e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:30.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e15c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e1015e420 0x7f5e10148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:30.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e15c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e1015e8b0 0x7f5e1014c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:30.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e15c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e101661f0 con 0x7f5e101518a0 2024-09-13T08:48:30.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e15c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5e10166070 con 0x7f5e1015e8b0 2024-09-13T08:48:30.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e15c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5e10166370 con 0x7f5e1015e420 2024-09-13T08:48:30.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101518a0 0x7f5e1014e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:30.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101518a0 0x7f5e1014e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58184/0 (socket says 172.21.15.44:58184) 2024-09-13T08:48:30.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e0fe00640 1 -- 172.21.15.44:0/2290730111 learned_addr learned my addr 172.21.15.44:0/2290730111 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:30.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e0fe00640 1 -- 172.21.15.44:0/2290730111 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e1015e420 msgr2=0x7f5e10148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:30.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e0fe00640 1 --2- 172.21.15.44:0/2290730111 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e1015e420 0x7f5e10148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:30.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e0fe00640 1 -- 172.21.15.44:0/2290730111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e1015e8b0 msgr2=0x7f5e1014c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:30.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e0fe00640 1 --2- 172.21.15.44:0/2290730111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e1015e8b0 0x7f5e1014c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:30.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e0fe00640 1 -- 172.21.15.44:0/2290730111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e10050f50 con 0x7f5e101518a0 2024-09-13T08:48:30.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e0fe00640 1 --2- 172.21.15.44:0/2290730111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101518a0 0x7f5e1014e6a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000b410 tx=0x7f5e0000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:30.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e07e00640 1 -- 172.21.15.44:0/2290730111 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00015020 con 0x7f5e101518a0 2024-09-13T08:48:30.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e07e00640 1 -- 172.21.15.44:0/2290730111 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e00011920 con 0x7f5e101518a0 2024-09-13T08:48:30.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e15c00640 1 -- 172.21.15.44:0/2290730111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e100b06b0 con 0x7f5e101518a0 2024-09-13T08:48:30.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e07e00640 1 -- 172.21.15.44:0/2290730111 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00011d70 con 0x7f5e101518a0 2024-09-13T08:48:30.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.251+0000 7f5e15c00640 1 -- 172.21.15.44:0/2290730111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e100b4670 con 0x7f5e101518a0 2024-09-13T08:48:30.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.255+0000 7f5e15c00640 1 -- 172.21.15.44:0/2290730111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1015b800 con 0x7f5e101518a0 2024-09-13T08:48:30.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.255+0000 7f5e07e00640 1 -- 172.21.15.44:0/2290730111 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7f5e00018020 con 0x7f5e101518a0 2024-09-13T08:48:30.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.259+0000 7f5e07e00640 1 --2- 172.21.15.44:0/2290730111 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5df00744b0 0x7f5df0076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:30.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.259+0000 7f5e07e00640 1 -- 172.21.15.44:0/2290730111 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f5e00092f90 con 0x7f5e101518a0 2024-09-13T08:48:30.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.259+0000 7f5e0f400640 1 --2- 172.21.15.44:0/2290730111 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5df00744b0 0x7f5df0076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:30.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.263+0000 7f5e0f400640 1 --2- 172.21.15.44:0/2290730111 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5df00744b0 0x7f5df0076970 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0af000 tx=0x7f5dfc0195f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.263+0000 7f5e07e00640 1 -- 172.21.15.44:0/2290730111 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5e0001f020 con 0x7f5e101518a0 2024-09-13T08:48:30.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.635+0000 7f5e15c00640 1 -- 172.21.15.44:0/2290730111 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone25384", "target": ["mon-mgr", ""]}) -- 0x7f5e10165020 con 0x7f5df00744b0 2024-09-13T08:48:30.907 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:30.906+0000 7f8ac3e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:30.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:30.903+0000 7f5e07e00640 1 -- 172.21.15.44:0/2290730111 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f5e10165020 con 0x7f5df00744b0 2024-09-13T08:48:30.907 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:30.933 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T08:48:30.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone71215 2024-09-13T08:48:31.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.215+0000 7f1d3c200640 1 -- 172.21.15.44:0/623278358 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d340b1650 msgr2=0x7f1d340b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:31.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.215+0000 7f1d3c200640 1 --2- 172.21.15.44:0/623278358 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d340b1650 0x7f1d340b1a30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009840 tx=0x7f1d2801c840 comp rx=0 tx=0).stop 2024-09-13T08:48:31.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.215+0000 7f1d3c200640 1 -- 172.21.15.44:0/623278358 shutdown_connections 2024-09-13T08:48:31.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.215+0000 7f1d3c200640 1 --2- 172.21.15.44:0/623278358 >> v2:172.21.15.129:3300/0 conn(0x7f1d340afb10 0x7f1d340affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:31.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.215+0000 7f1d3c200640 1 --2- 172.21.15.44:0/623278358 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d340b1650 0x7f1d340b1a30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:31.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.215+0000 7f1d3c200640 1 -- 172.21.15.44:0/623278358 >> 172.21.15.44:0/623278358 conn(0x7f1d340d1dd0 msgr2=0x7f1d340d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:31.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.215+0000 7f1d3c200640 1 -- 172.21.15.44:0/623278358 shutdown_connections 2024-09-13T08:48:31.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.215+0000 7f1d3c200640 1 -- 172.21.15.44:0/623278358 wait complete. 2024-09-13T08:48:31.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.219+0000 7f1d3c200640 1 Processor -- start 2024-09-13T08:48:31.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.219+0000 7f1d3c200640 1 -- start start 2024-09-13T08:48:31.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.219+0000 7f1d3c200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d340afb10 0x7f1d3406e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:31.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.219+0000 7f1d3c200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d340b1650 0x7f1d340c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:31.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.219+0000 7f1d3c200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d341605d0 0x7f1d340c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:31.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.219+0000 7f1d3c200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34169540 con 0x7f1d341605d0 2024-09-13T08:48:31.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.219+0000 7f1d3c200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1d340b1430 con 0x7f1d340b1650 2024-09-13T08:48:31.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.219+0000 7f1d3c200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1d341696c0 con 0x7f1d340afb10 2024-09-13T08:48:31.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d3a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d341605d0 0x7f1d340c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:31.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d3a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d341605d0 0x7f1d340c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58190/0 (socket says 172.21.15.44:58190) 2024-09-13T08:48:31.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d3a200640 1 -- 172.21.15.44:0/4252338322 learned_addr learned my addr 172.21.15.44:0/4252338322 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:31.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d38e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d340b1650 0x7f1d340c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:31.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d3a200640 1 -- 172.21.15.44:0/4252338322 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d340afb10 msgr2=0x7f1d3406e720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:48:31.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d3a200640 1 --2- 172.21.15.44:0/4252338322 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1d340afb10 0x7f1d3406e720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:31.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d3a200640 1 -- 172.21.15.44:0/4252338322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d340b1650 msgr2=0x7f1d340c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:31.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d3a200640 1 --2- 172.21.15.44:0/4252338322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d340b1650 0x7f1d340c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:31.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d3a200640 1 -- 172.21.15.44:0/4252338322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d340b6540 con 0x7f1d341605d0 2024-09-13T08:48:31.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d3a200640 1 --2- 172.21.15.44:0/4252338322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d341605d0 0x7f1d340c5ed0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00ea20 tx=0x7f1d2c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:31.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d31600640 1 -- 172.21.15.44:0/4252338322 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c00cde0 con 0x7f1d341605d0 2024-09-13T08:48:31.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d31600640 1 -- 172.21.15.44:0/4252338322 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d2c004c20 con 0x7f1d341605d0 2024-09-13T08:48:31.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.223+0000 7f1d31600640 1 -- 172.21.15.44:0/4252338322 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c005280 con 0x7f1d341605d0 2024-09-13T08:48:31.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.227+0000 7f1d3c200640 1 -- 172.21.15.44:0/4252338322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d340b6820 con 0x7f1d341605d0 2024-09-13T08:48:31.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.227+0000 7f1d3c200640 1 -- 172.21.15.44:0/4252338322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d340b6de0 con 0x7f1d341605d0 2024-09-13T08:48:31.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.227+0000 7f1d3c200640 1 -- 172.21.15.44:0/4252338322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d340affc0 con 0x7f1d341605d0 2024-09-13T08:48:31.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.231+0000 7f1d31600640 1 -- 172.21.15.44:0/4252338322 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7f1d2c005420 con 0x7f1d341605d0 2024-09-13T08:48:31.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.231+0000 7f1d31600640 1 --2- 172.21.15.44:0/4252338322 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1d14074500 0x7f1d140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:31.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.231+0000 7f1d31600640 1 -- 172.21.15.44:0/4252338322 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f1d2c0939c0 con 0x7f1d341605d0 2024-09-13T08:48:31.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.235+0000 7f1d39800640 1 --2- 172.21.15.44:0/4252338322 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1d14074500 0x7f1d140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:31.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.235+0000 7f1d31600640 1 -- 172.21.15.44:0/4252338322 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1d2c016030 con 0x7f1d341605d0 2024-09-13T08:48:31.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.239+0000 7f1d39800640 1 --2- 172.21.15.44:0/4252338322 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1d14074500 0x7f1d140769c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009690 tx=0x7f1d20006b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:31.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.667+0000 7f1d3c200640 1 -- 172.21.15.44:0/4252338322 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone71215", "target": ["mon-mgr", ""]}) -- 0x7f1d340d1080 con 0x7f1d14074500 2024-09-13T08:48:31.810 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:31.810+0000 7f8ac3e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:31.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:31.807+0000 7f1d31600640 1 -- 172.21.15.44:0/4252338322 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f1d340d1080 con 0x7f1d14074500 2024-09-13T08:48:31.812 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:31.848 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T08:48:31.849 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone42326 2024-09-13T08:48:32.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.147+0000 7fd2e1c00640 1 -- 172.21.15.44:0/1052865259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc0cfcc0 msgr2=0x7fd2dc073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:32.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.147+0000 7fd2e1c00640 1 --2- 172.21.15.44:0/1052865259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc0cfcc0 0x7fd2dc073cd0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc06f220 tx=0x7fd2c801c7e0 comp rx=0 tx=0).stop 2024-09-13T08:48:32.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.147+0000 7fd2e1c00640 1 -- 172.21.15.44:0/1052865259 shutdown_connections 2024-09-13T08:48:32.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.147+0000 7fd2e1c00640 1 --2- 172.21.15.44:0/1052865259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc0cfcc0 0x7fd2dc073cd0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:32.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.147+0000 7fd2e1c00640 1 --2- 172.21.15.44:0/1052865259 >> v2:172.21.15.129:3300/0 conn(0x7fd2dc15a9d0 0x7fd2dc073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:32.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.147+0000 7fd2e1c00640 1 -- 172.21.15.44:0/1052865259 >> 172.21.15.44:0/1052865259 conn(0x7fd2dc0d0a90 msgr2=0x7fd2dc0d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:32.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.147+0000 7fd2e1c00640 1 -- 172.21.15.44:0/1052865259 shutdown_connections 2024-09-13T08:48:32.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.147+0000 7fd2e1c00640 1 -- 172.21.15.44:0/1052865259 wait complete. 2024-09-13T08:48:32.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.147+0000 7fd2e1c00640 1 Processor -- start 2024-09-13T08:48:32.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.147+0000 7fd2e1c00640 1 -- start start 2024-09-13T08:48:32.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2dc074210 0x7fd2dc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:32.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd2dc0cfcc0 0x7fd2dc2078c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:32.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc15a9d0 0x7fd2dc207e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:32.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc06cfb0 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc06ce30 con 0x7fd2dc074210 2024-09-13T08:48:32.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd2dc06d130 con 0x7fd2dc0cfcc0 2024-09-13T08:48:32.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc15a9d0 0x7fd2dc207e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:32.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc15a9d0 0x7fd2dc207e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51212/0 (socket says 172.21.15.44:51212) 2024-09-13T08:48:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1200640 1 -- 172.21.15.44:0/3647713336 learned_addr learned my addr 172.21.15.44:0/3647713336 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2db400640 1 --2- 172.21.15.44:0/3647713336 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd2dc0cfcc0 0x7fd2dc2078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1200640 1 -- 172.21.15.44:0/3647713336 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd2dc0cfcc0 msgr2=0x7fd2dc2078c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1200640 1 --2- 172.21.15.44:0/3647713336 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd2dc0cfcc0 0x7fd2dc2078c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1200640 1 -- 172.21.15.44:0/3647713336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2dc074210 msgr2=0x7fd2dc15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1200640 1 --2- 172.21.15.44:0/3647713336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2dc074210 0x7fd2dc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1200640 1 -- 172.21.15.44:0/3647713336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2dc15b430 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1200640 1 --2- 172.21.15.44:0/3647713336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc15a9d0 0x7fd2dc207e00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400cbc0 tx=0x7fd2d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:32.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2d3e00640 1 -- 172.21.15.44:0/3647713336 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4016020 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2d3e00640 1 -- 172.21.15.44:0/3647713336 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2d400ab40 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2d3e00640 1 -- 172.21.15.44:0/3647713336 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4012420 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1c00640 1 -- 172.21.15.44:0/3647713336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2dc0c6660 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.151+0000 7fd2e1c00640 1 -- 172.21.15.44:0/3647713336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2dc0c6c20 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.155+0000 7fd2e1c00640 1 -- 172.21.15.44:0/3647713336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b40052a0 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.159+0000 7fd2d3e00640 1 -- 172.21.15.44:0/3647713336 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7fd2d4019020 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.159+0000 7fd2d3e00640 1 --2- 172.21.15.44:0/3647713336 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd2bc0744b0 0x7fd2bc076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:32.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.159+0000 7fd2d3e00640 1 -- 172.21.15.44:0/3647713336 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fd2d40930b0 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.171+0000 7fd2d3e00640 1 -- 172.21.15.44:0/3647713336 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd2d4020020 con 0x7fd2dc15a9d0 2024-09-13T08:48:32.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.179+0000 7fd2dbe00640 1 --2- 172.21.15.44:0/3647713336 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd2bc0744b0 0x7fd2bc076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:32.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.215+0000 7fd2dbe00640 1 --2- 172.21.15.44:0/3647713336 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd2bc0744b0 0x7fd2bc076970 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009690 tx=0x7fd2cc006c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:32.480 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:48:32.485 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:48:32.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:32.595+0000 7fd2e1c00640 1 -- 172.21.15.44:0/3647713336 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone42326", "target": ["mon-mgr", ""]}) -- 0x7fd2b4000bd0 con 0x7fd2bc0744b0 2024-09-13T08:48:33.138 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:33.138+0000 7f8ac3e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:33.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.159+0000 7fd2d3e00640 1 -- 172.21.15.44:0/3647713336 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fd2b4000bd0 con 0x7fd2bc0744b0 2024-09-13T08:48:33.165 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:33.178 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T08:48:33.178 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone73477 2024-09-13T08:48:33.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.495+0000 7f5324000640 1 -- 172.21.15.44:0/289068925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c070960 msgr2=0x7f531c06f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.495+0000 7f5324000640 1 --2- 172.21.15.44:0/289068925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c070960 0x7f531c06f8c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f531c072500 tx=0x7f531001c770 comp rx=0 tx=0).stop 2024-09-13T08:48:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.495+0000 7f5324000640 1 -- 172.21.15.44:0/289068925 shutdown_connections 2024-09-13T08:48:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.495+0000 7f5324000640 1 --2- 172.21.15.44:0/289068925 >> v2:172.21.15.129:3300/0 conn(0x7f531c06fe00 0x7f531c0cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.495+0000 7f5324000640 1 --2- 172.21.15.44:0/289068925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c070960 0x7f531c06f8c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.495+0000 7f5324000640 1 -- 172.21.15.44:0/289068925 >> 172.21.15.44:0/289068925 conn(0x7f531c0b18f0 msgr2=0x7f531c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.495+0000 7f5324000640 1 -- 172.21.15.44:0/289068925 shutdown_connections 2024-09-13T08:48:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.495+0000 7f5324000640 1 -- 172.21.15.44:0/289068925 wait complete. 2024-09-13T08:48:33.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.499+0000 7f5324000640 1 Processor -- start 2024-09-13T08:48:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.499+0000 7f5324000640 1 -- start start 2024-09-13T08:48:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.499+0000 7f5324000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f531c06fe00 0x7f531c162590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5324000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f531c070960 0x7f531c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5324000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c0ccc80 0x7f531c0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5324000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1671e0 con 0x7f531c0ccc80 2024-09-13T08:48:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5324000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f531c0cd250 con 0x7f531c070960 2024-09-13T08:48:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5324000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f531c167360 con 0x7f531c06fe00 2024-09-13T08:48:33.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5322000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c0ccc80 0x7f531c0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:33.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5322000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c0ccc80 0x7f531c0ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51236/0 (socket says 172.21.15.44:51236) 2024-09-13T08:48:33.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5322000640 1 -- 172.21.15.44:0/1455059686 learned_addr learned my addr 172.21.15.44:0/1455059686 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:33.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5322000640 1 -- 172.21.15.44:0/1455059686 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f531c06fe00 msgr2=0x7f531c162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:33.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5322000640 1 --2- 172.21.15.44:0/1455059686 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f531c06fe00 0x7f531c162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:33.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5322000640 1 -- 172.21.15.44:0/1455059686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f531c070960 msgr2=0x7f531c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:33.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5322000640 1 --2- 172.21.15.44:0/1455059686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f531c070960 0x7f531c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:33.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5322000640 1 -- 172.21.15.44:0/1455059686 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c09abb0 con 0x7f531c0ccc80 2024-09-13T08:48:33.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.503+0000 7f5322000640 1 --2- 172.21.15.44:0/1455059686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c0ccc80 0x7f531c0ce2a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f531400b990 tx=0x7f531400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.507+0000 7f5319600640 1 -- 172.21.15.44:0/1455059686 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314019070 con 0x7f531c0ccc80 2024-09-13T08:48:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.507+0000 7f5319600640 1 -- 172.21.15.44:0/1455059686 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f531400cc90 con 0x7f531c0ccc80 2024-09-13T08:48:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.507+0000 7f5319600640 1 -- 172.21.15.44:0/1455059686 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53140146a0 con 0x7f531c0ccc80 2024-09-13T08:48:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.507+0000 7f5324000640 1 -- 172.21.15.44:0/1455059686 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531c09ae90 con 0x7f531c0ccc80 2024-09-13T08:48:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.507+0000 7f5324000640 1 -- 172.21.15.44:0/1455059686 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531c09b450 con 0x7f531c0ccc80 2024-09-13T08:48:33.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.515+0000 7f5324000640 1 -- 172.21.15.44:0/1455059686 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531c0cc740 con 0x7f531c0ccc80 2024-09-13T08:48:33.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.515+0000 7f5319600640 1 -- 172.21.15.44:0/1455059686 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7f531401c070 con 0x7f531c0ccc80 2024-09-13T08:48:33.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.519+0000 7f5319600640 1 --2- 172.21.15.44:0/1455059686 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f52fc074500 0x7f52fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:33.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.523+0000 7f5319600640 1 -- 172.21.15.44:0/1455059686 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f531405b170 con 0x7f531c0ccc80 2024-09-13T08:48:33.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.535+0000 7f5319600640 1 -- 172.21.15.44:0/1455059686 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f531405f600 con 0x7f531c0ccc80 2024-09-13T08:48:33.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.543+0000 7f5321600640 1 --2- 172.21.15.44:0/1455059686 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f52fc074500 0x7f52fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:33.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.571+0000 7f5321600640 1 --2- 172.21.15.44:0/1455059686 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f52fc074500 0x7f52fc0769c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f53100af000 tx=0x7f5310019510 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:33.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:33.987+0000 7f5324000640 1 -- 172.21.15.44:0/1455059686 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone73477", "target": ["mon-mgr", ""]}) -- 0x7f531c15e030 con 0x7f52fc074500 2024-09-13T08:48:34.528 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:34.530+0000 7f8ac3e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.551+0000 7f5319600640 1 -- 172.21.15.44:0/1455059686 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f531c15e030 con 0x7f52fc074500 2024-09-13T08:48:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-13T08:48:34.570 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T08:48:34.570 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8848 2024-09-13T08:48:34.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.855+0000 7fd760a00640 1 -- 172.21.15.44:0/3065210362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c15c250 msgr2=0x7fd75c1648a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:34.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.855+0000 7fd760a00640 1 --2- 172.21.15.44:0/3065210362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c15c250 0x7fd75c1648a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd754009bc0 tx=0x7fd75401cb70 comp rx=0 tx=0).stop 2024-09-13T08:48:34.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.855+0000 7fd760a00640 1 -- 172.21.15.44:0/3065210362 shutdown_connections 2024-09-13T08:48:34.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.855+0000 7fd760a00640 1 --2- 172.21.15.44:0/3065210362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c15c250 0x7fd75c1648a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:34.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.855+0000 7fd760a00640 1 -- 172.21.15.44:0/3065210362 >> 172.21.15.44:0/3065210362 conn(0x7fd75c06cc20 msgr2=0x7fd75c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.855+0000 7fd760a00640 1 -- 172.21.15.44:0/3065210362 shutdown_connections 2024-09-13T08:48:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.855+0000 7fd760a00640 1 -- 172.21.15.44:0/3065210362 wait complete. 2024-09-13T08:48:34.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.859+0000 7fd760a00640 1 Processor -- start 2024-09-13T08:48:34.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.863+0000 7fd760a00640 1 -- start start 2024-09-13T08:48:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.863+0000 7fd760a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd75c159c10 0x7fd75c14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd760a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c15c250 0x7fd75c1fc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd760a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd75c1fc790 0x7fd75c153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd760a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1692c0 con 0x7fd75c15c250 2024-09-13T08:48:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd760a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c169140 con 0x7fd75c1fc790 2024-09-13T08:48:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd760a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd75c169440 con 0x7fd75c159c10 2024-09-13T08:48:34.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd759e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c15c250 0x7fd75c1fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:34.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd759e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c15c250 0x7fd75c1fc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51262/0 (socket says 172.21.15.44:51262) 2024-09-13T08:48:34.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd759e00640 1 -- 172.21.15.44:0/3427577013 learned_addr learned my addr 172.21.15.44:0/3427577013 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd759e00640 1 -- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd75c159c10 msgr2=0x7fd75c14c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd759e00640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd75c159c10 0x7fd75c14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd759e00640 1 -- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd75c1fc790 msgr2=0x7fd75c153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd759e00640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd75c1fc790 0x7fd75c153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd759e00640 1 -- 172.21.15.44:0/3427577013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c1fcca0 con 0x7fd75c15c250 2024-09-13T08:48:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd759e00640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c15c250 0x7fd75c1fc250 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd74800c410 tx=0x7fd74800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.867+0000 7fd752a00640 1 -- 172.21.15.44:0/3427577013 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748017070 con 0x7fd75c15c250 2024-09-13T08:48:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.871+0000 7fd752a00640 1 -- 172.21.15.44:0/3427577013 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd748003600 con 0x7fd75c15c250 2024-09-13T08:48:34.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.871+0000 7fd752a00640 1 -- 172.21.15.44:0/3427577013 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748003a50 con 0x7fd75c15c250 2024-09-13T08:48:34.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.871+0000 7fd760a00640 1 -- 172.21.15.44:0/3427577013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c1fcf80 con 0x7fd75c15c250 2024-09-13T08:48:34.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.871+0000 7fd760a00640 1 -- 172.21.15.44:0/3427577013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c14a040 con 0x7fd75c15c250 2024-09-13T08:48:34.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.879+0000 7fd760a00640 1 -- 172.21.15.44:0/3427577013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd75c1fd1c0 con 0x7fd75c15c250 2024-09-13T08:48:34.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.887+0000 7fd752a00640 1 -- 172.21.15.44:0/3427577013 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7fd74801a070 con 0x7fd75c15c250 2024-09-13T08:48:34.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.887+0000 7fd752a00640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd73c0745b0 0x7fd73c076a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:34.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.887+0000 7fd752a00640 1 -- 172.21.15.44:0/3427577013 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fd7480920b0 con 0x7fd75c15c250 2024-09-13T08:48:34.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.899+0000 7fd752a00640 1 -- 172.21.15.44:0/3427577013 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd75c1fd1c0 con 0x7fd75c15c250 2024-09-13T08:48:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.911+0000 7fd75a800640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd73c0745b0 0x7fd73c076a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:34.947+0000 7fd75a800640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd73c0745b0 0x7fd73c076a70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0015d0 tx=0x7fd74c007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:35.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:35.355+0000 7fd760a00640 1 -- 172.21.15.44:0/3427577013 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8848", "target": ["mon-mgr", ""]}) -- 0x7fd75c06ebf0 con 0x7fd73c0745b0 2024-09-13T08:48:36.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd752a00640 1 -- 172.21.15.44:0/3427577013 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd75c06ebf0 con 0x7fd73c0745b0 2024-09-13T08:48:36.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 -- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd73c0745b0 msgr2=0x7fd73c076a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:36.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd73c0745b0 0x7fd73c076a70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0015d0 tx=0x7fd74c007a40 comp rx=0 tx=0).stop 2024-09-13T08:48:36.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 -- 172.21.15.44:0/3427577013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c15c250 msgr2=0x7fd75c1fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:36.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c15c250 0x7fd75c1fc250 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd74800c410 tx=0x7fd74800c8d0 comp rx=0 tx=0).stop 2024-09-13T08:48:36.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 -- 172.21.15.44:0/3427577013 shutdown_connections 2024-09-13T08:48:36.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd73c0745b0 0x7fd73c076a70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:36.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd75c1fc790 0x7fd75c153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:36.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c15c250 0x7fd75c1fc250 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:36.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 --2- 172.21.15.44:0/3427577013 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd75c159c10 0x7fd75c14c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:36.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 -- 172.21.15.44:0/3427577013 >> 172.21.15.44:0/3427577013 conn(0x7fd75c06cc20 msgr2=0x7fd75c157ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:36.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 -- 172.21.15.44:0/3427577013 shutdown_connections 2024-09-13T08:48:36.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:36.959+0000 7fd760a00640 1 -- 172.21.15.44:0/3427577013 wait complete. 2024-09-13T08:48:37.014 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone74089 2024-09-13T08:48:37.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.315+0000 7fda7ca00640 1 -- 172.21.15.44:0/3274493860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5c004dc0 msgr2=0x7fda5c005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:37.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.315+0000 7fda7ca00640 1 --2- 172.21.15.44:0/3274493860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5c004dc0 0x7fda5c005240 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fda70009e30 tx=0x7fda7001d180 comp rx=0 tx=0).stop 2024-09-13T08:48:37.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.315+0000 7fda76800640 1 -- 172.21.15.44:0/3274493860 reap_dead start 2024-09-13T08:48:37.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.315+0000 7fda7ca00640 1 -- 172.21.15.44:0/3274493860 shutdown_connections 2024-09-13T08:48:37.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.315+0000 7fda7ca00640 1 -- 172.21.15.44:0/3274493860 >> 172.21.15.44:0/3274493860 conn(0x7fda780b18f0 msgr2=0x7fda780b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:37.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.315+0000 7fda7ca00640 1 -- 172.21.15.44:0/3274493860 shutdown_connections 2024-09-13T08:48:37.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.315+0000 7fda7ca00640 1 -- 172.21.15.44:0/3274493860 wait complete. 2024-09-13T08:48:37.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.319+0000 7fda7ca00640 1 Processor -- start 2024-09-13T08:48:37.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.319+0000 7fda7ca00640 1 -- start start 2024-09-13T08:48:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.319+0000 7fda7ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda5c004dc0 0x7fda781543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.319+0000 7fda7ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda781f4920 0x7fda781434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.319+0000 7fda7ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda781f3ca0 0x7fda781f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.319+0000 7fda7ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda780b4af0 con 0x7fda781f4920 2024-09-13T08:48:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.319+0000 7fda7ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda780b4970 con 0x7fda5c004dc0 2024-09-13T08:48:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.319+0000 7fda7ca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fda780b4c70 con 0x7fda781f3ca0 2024-09-13T08:48:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda75400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda781f4920 0x7fda781434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda75400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda781f4920 0x7fda781434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51278/0 (socket says 172.21.15.44:51278) 2024-09-13T08:48:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda75400640 1 -- 172.21.15.44:0/3983304705 learned_addr learned my addr 172.21.15.44:0/3983304705 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda75400640 1 -- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda781f3ca0 msgr2=0x7fda781f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:37.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda75400640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda781f3ca0 0x7fda781f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:37.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda75400640 1 -- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda5c004dc0 msgr2=0x7fda781543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:37.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda75400640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda5c004dc0 0x7fda781543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:37.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda75400640 1 -- 172.21.15.44:0/3983304705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda780af860 con 0x7fda781f4920 2024-09-13T08:48:37.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda75400640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda781f4920 0x7fda781434f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fda6400db20 tx=0x7fda6400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:37.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda6e000640 1 -- 172.21.15.44:0/3983304705 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64008dd0 con 0x7fda781f4920 2024-09-13T08:48:37.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda7ca00640 1 -- 172.21.15.44:0/3983304705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7809d950 con 0x7fda781f4920 2024-09-13T08:48:37.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda6e000640 1 -- 172.21.15.44:0/3983304705 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda64004020 con 0x7fda781f4920 2024-09-13T08:48:37.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda6e000640 1 -- 172.21.15.44:0/3983304705 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64004470 con 0x7fda781f4920 2024-09-13T08:48:37.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda7ca00640 1 -- 172.21.15.44:0/3983304705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda781e49b0 con 0x7fda781f4920 2024-09-13T08:48:37.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.323+0000 7fda7ca00640 1 -- 172.21.15.44:0/3983304705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda78069410 con 0x7fda781f4920 2024-09-13T08:48:37.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.327+0000 7fda6e000640 1 -- 172.21.15.44:0/3983304705 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7fda6400c040 con 0x7fda781f4920 2024-09-13T08:48:37.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.327+0000 7fda6e000640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda580744b0 0x7fda58076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:37.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.331+0000 7fda6e000640 1 -- 172.21.15.44:0/3983304705 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fda64093580 con 0x7fda781f4920 2024-09-13T08:48:37.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.335+0000 7fda75e00640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda580744b0 0x7fda58076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:37.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.339+0000 7fda6e000640 1 -- 172.21.15.44:0/3983304705 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fda6405ff30 con 0x7fda781f4920 2024-09-13T08:48:37.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.355+0000 7fda75e00640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda580744b0 0x7fda58076970 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fda6800b7e0 tx=0x7fda680015c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:37.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:37.795+0000 7fda7ca00640 1 -- 172.21.15.44:0/3983304705 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone74089", "target": ["mon-mgr", ""]}) -- 0x7fda780cc660 con 0x7fda580744b0 2024-09-13T08:48:38.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.839+0000 7fda6e000640 1 -- 172.21.15.44:0/3983304705 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda780cc660 con 0x7fda580744b0 2024-09-13T08:48:38.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.839+0000 7fda7ca00640 1 -- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda580744b0 msgr2=0x7fda58076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:38.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.839+0000 7fda7ca00640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda580744b0 0x7fda58076970 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fda6800b7e0 tx=0x7fda680015c0 comp rx=0 tx=0).stop 2024-09-13T08:48:38.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.839+0000 7fda7ca00640 1 -- 172.21.15.44:0/3983304705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda781f4920 msgr2=0x7fda781434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:38.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.839+0000 7fda7ca00640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda781f4920 0x7fda781434f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fda6400db20 tx=0x7fda6400bf60 comp rx=0 tx=0).stop 2024-09-13T08:48:38.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.843+0000 7fda7ca00640 1 -- 172.21.15.44:0/3983304705 shutdown_connections 2024-09-13T08:48:38.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.843+0000 7fda7ca00640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fda580744b0 0x7fda58076970 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:38.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.843+0000 7fda7ca00640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fda781f3ca0 0x7fda781f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:38.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.843+0000 7fda7ca00640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda781f4920 0x7fda781434f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:38.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.843+0000 7fda7ca00640 1 --2- 172.21.15.44:0/3983304705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda5c004dc0 0x7fda781543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:38.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.843+0000 7fda7ca00640 1 -- 172.21.15.44:0/3983304705 >> 172.21.15.44:0/3983304705 conn(0x7fda780b18f0 msgr2=0x7fda780cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:38.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.843+0000 7fda7ca00640 1 -- 172.21.15.44:0/3983304705 shutdown_connections 2024-09-13T08:48:38.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:38.843+0000 7fda7ca00640 1 -- 172.21.15.44:0/3983304705 wait complete. 2024-09-13T08:48:38.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone649610 2024-09-13T08:48:39.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.231+0000 7fcc26a00640 1 -- 172.21.15.44:0/526336094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc04004810 msgr2=0x7fcc04004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:39.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.231+0000 7fcc26a00640 1 --2- 172.21.15.44:0/526336094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc04004810 0x7fcc04004c90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009b60 tx=0x7fcc0801d2d0 comp rx=0 tx=0).stop 2024-09-13T08:48:39.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.231+0000 7fcc26a00640 1 -- 172.21.15.44:0/526336094 shutdown_connections 2024-09-13T08:48:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.231+0000 7fcc26a00640 1 --2- 172.21.15.44:0/526336094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc04004810 0x7fcc04004c90 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.231+0000 7fcc26a00640 1 -- 172.21.15.44:0/526336094 >> 172.21.15.44:0/526336094 conn(0x7fcc2006ecd0 msgr2=0x7fcc2006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.231+0000 7fcc26a00640 1 -- 172.21.15.44:0/526336094 shutdown_connections 2024-09-13T08:48:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.231+0000 7fcc26a00640 1 -- 172.21.15.44:0/526336094 wait complete. 2024-09-13T08:48:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc26a00640 1 Processor -- start 2024-09-13T08:48:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc26a00640 1 -- start start 2024-09-13T08:48:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc26a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200b4980 0x7fcc20150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc26a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc201f56a0 0x7fcc20150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc26a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcc200b4d60 0x7fcc201334c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc26a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20161e50 con 0x7fcc200b4980 2024-09-13T08:48:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc26a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20161cd0 con 0x7fcc201f56a0 2024-09-13T08:48:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc26a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcc20161fd0 con 0x7fcc200b4d60 2024-09-13T08:48:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200b4980 0x7fcc20150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200b4980 0x7fcc20150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51300/0 (socket says 172.21.15.44:51300) 2024-09-13T08:48:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc1fe00640 1 -- 172.21.15.44:0/239914060 learned_addr learned my addr 172.21.15.44:0/239914060 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc1fe00640 1 -- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcc200b4d60 msgr2=0x7fcc201334c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc1fe00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcc200b4d60 0x7fcc201334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc1fe00640 1 -- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc201f56a0 msgr2=0x7fcc20150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc1fe00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc201f56a0 0x7fcc20150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc1fe00640 1 -- 172.21.15.44:0/239914060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc201341d0 con 0x7fcc200b4980 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.235+0000 7fcc1fe00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200b4980 0x7fcc20150830 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000f4f0 tx=0x7fcc1000f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.239+0000 7fcc1cc00640 1 -- 172.21.15.44:0/239914060 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10003600 con 0x7fcc200b4980 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.239+0000 7fcc1cc00640 1 -- 172.21.15.44:0/239914060 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc100037a0 con 0x7fcc200b4980 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.239+0000 7fcc1cc00640 1 -- 172.21.15.44:0/239914060 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10003bf0 con 0x7fcc200b4980 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.239+0000 7fcc26a00640 1 -- 172.21.15.44:0/239914060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc201f4bc0 con 0x7fcc200b4980 2024-09-13T08:48:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.239+0000 7fcc26a00640 1 -- 172.21.15.44:0/239914060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc201f5180 con 0x7fcc200b4980 2024-09-13T08:48:39.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.239+0000 7fcc26a00640 1 -- 172.21.15.44:0/239914060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc201f5380 con 0x7fcc200b4980 2024-09-13T08:48:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.239+0000 7fcc1cc00640 1 -- 172.21.15.44:0/239914060 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7fcc10012070 con 0x7fcc200b4980 2024-09-13T08:48:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.243+0000 7fcc1cc00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbfc074500 0x7fcbfc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.243+0000 7fcc1cc00640 1 -- 172.21.15.44:0/239914060 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fcc10095b40 con 0x7fcc200b4980 2024-09-13T08:48:39.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.247+0000 7fcc1cc00640 1 -- 172.21.15.44:0/239914060 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcc201f5380 con 0x7fcc200b4980 2024-09-13T08:48:39.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.275+0000 7fcc17e00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbfc074500 0x7fcbfc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:39.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.283+0000 7fcc17e00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbfc074500 0x7fcbfc0769c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcc080af000 tx=0x7fcc04005840 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:39.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:39.679+0000 7fcc26a00640 1 -- 172.21.15.44:0/239914060 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone649610", "target": ["mon-mgr", ""]}) -- 0x7fcc201f49b0 con 0x7fcbfc074500 2024-09-13T08:48:40.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.555+0000 7fcc1cc00640 1 -- 172.21.15.44:0/239914060 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcc201f49b0 con 0x7fcbfc074500 2024-09-13T08:48:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.567+0000 7fcc26a00640 1 -- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbfc074500 msgr2=0x7fcbfc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.567+0000 7fcc26a00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbfc074500 0x7fcbfc0769c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcc080af000 tx=0x7fcc04005840 comp rx=0 tx=0).stop 2024-09-13T08:48:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.567+0000 7fcc26a00640 1 -- 172.21.15.44:0/239914060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200b4980 msgr2=0x7fcc20150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.567+0000 7fcc26a00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200b4980 0x7fcc20150830 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000f4f0 tx=0x7fcc1000f9b0 comp rx=0 tx=0).stop 2024-09-13T08:48:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.575+0000 7fcc26a00640 1 -- 172.21.15.44:0/239914060 shutdown_connections 2024-09-13T08:48:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.575+0000 7fcc26a00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcbfc074500 0x7fcbfc0769c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.575+0000 7fcc26a00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcc200b4d60 0x7fcc201334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.575+0000 7fcc26a00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc201f56a0 0x7fcc20150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.575+0000 7fcc26a00640 1 --2- 172.21.15.44:0/239914060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200b4980 0x7fcc20150830 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.575+0000 7fcc26a00640 1 -- 172.21.15.44:0/239914060 >> 172.21.15.44:0/239914060 conn(0x7fcc2006ecd0 msgr2=0x7fcc2015d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.575+0000 7fcc26a00640 1 -- 172.21.15.44:0/239914060 shutdown_connections 2024-09-13T08:48:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.575+0000 7fcc26a00640 1 -- 172.21.15.44:0/239914060 wait complete. 2024-09-13T08:48:40.615 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone677511 2024-09-13T08:48:40.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.899+0000 7fec74600640 1 -- 172.21.15.44:0/1042294806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec54004810 msgr2=0x7fec54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:40.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.899+0000 7fec74600640 1 --2- 172.21.15.44:0/1042294806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec54004810 0x7fec54004c90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c3c0 tx=0x7fec5c01f8a0 comp rx=0 tx=0).stop 2024-09-13T08:48:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.903+0000 7fec74600640 1 -- 172.21.15.44:0/1042294806 shutdown_connections 2024-09-13T08:48:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.903+0000 7fec74600640 1 --2- 172.21.15.44:0/1042294806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec54004810 0x7fec54004c90 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.903+0000 7fec74600640 1 -- 172.21.15.44:0/1042294806 >> 172.21.15.44:0/1042294806 conn(0x7fec6c0d1dd0 msgr2=0x7fec6c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.907+0000 7fec74600640 1 -- 172.21.15.44:0/1042294806 shutdown_connections 2024-09-13T08:48:40.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.907+0000 7fec74600640 1 -- 172.21.15.44:0/1042294806 wait complete. 2024-09-13T08:48:40.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.911+0000 7fec74600640 1 Processor -- start 2024-09-13T08:48:40.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.911+0000 7fec74600640 1 -- start start 2024-09-13T08:48:40.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.911+0000 7fec74600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec54004810 0x7fec6c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:40.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec74600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec6c15f9e0 0x7fec6c1f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec74600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec6c0c9c10 0x7fec6c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec71c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec54004810 0x7fec6c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec71c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec54004810 0x7fec6c15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:49032/0 (socket says 172.21.15.44:49032) 2024-09-13T08:48:40.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec71200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec6c15f9e0 0x7fec6c1f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec71200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec6c15f9e0 0x7fec6c1f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:42840/0 (socket says 172.21.15.44:42840) 2024-09-13T08:48:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec74600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c0b0ee0 con 0x7fec6c0c9c10 2024-09-13T08:48:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec74600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c0b2cf0 con 0x7fec54004810 2024-09-13T08:48:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec74600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fec6c0b1220 con 0x7fec6c15f9e0 2024-09-13T08:48:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec71c00640 1 -- 172.21.15.44:0/3060063328 learned_addr learned my addr 172.21.15.44:0/3060063328 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.915+0000 7fec72600640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec6c0c9c10 0x7fec6c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.919+0000 7fec71c00640 1 -- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec6c15f9e0 msgr2=0x7fec6c1f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.919+0000 7fec71c00640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec6c15f9e0 0x7fec6c1f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:40.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.919+0000 7fec71c00640 1 -- 172.21.15.44:0/3060063328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec6c0c9c10 msgr2=0x7fec6c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:40.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.919+0000 7fec71c00640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec6c0c9c10 0x7fec6c0ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:40.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.919+0000 7fec71c00640 1 -- 172.21.15.44:0/3060063328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c1f4580 con 0x7fec54004810 2024-09-13T08:48:40.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.919+0000 7fec71200640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec6c15f9e0 0x7fec6c1f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:48:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.919+0000 7fec71c00640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec54004810 0x7fec6c15a930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c340 tx=0x7fec6000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.919+0000 7fec69600640 1 -- 172.21.15.44:0/3060063328 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60017070 con 0x7fec54004810 2024-09-13T08:48:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.919+0000 7fec74600640 1 -- 172.21.15.44:0/3060063328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6c1f4860 con 0x7fec54004810 2024-09-13T08:48:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.923+0000 7fec74600640 1 -- 172.21.15.44:0/3060063328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6c0c9410 con 0x7fec54004810 2024-09-13T08:48:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.923+0000 7fec69600640 1 -- 172.21.15.44:0/3060063328 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec60006ad0 con 0x7fec54004810 2024-09-13T08:48:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.923+0000 7fec69600640 1 -- 172.21.15.44:0/3060063328 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60017070 con 0x7fec54004810 2024-09-13T08:48:40.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.927+0000 7fec74600640 1 -- 172.21.15.44:0/3060063328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec480052a0 con 0x7fec54004810 2024-09-13T08:48:40.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.931+0000 7fec69600640 1 -- 172.21.15.44:0/3060063328 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7fec6001a070 con 0x7fec54004810 2024-09-13T08:48:40.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.931+0000 7fec69600640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec500746e0 0x7fec50076ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:40.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.931+0000 7fec69600640 1 -- 172.21.15.44:0/3060063328 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fec60091c70 con 0x7fec54004810 2024-09-13T08:48:40.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.931+0000 7fec71200640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec500746e0 0x7fec50076ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:40.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.939+0000 7fec71200640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec500746e0 0x7fec50076ba0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0b0000 tx=0x7fec5c01c640 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:40.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:40.943+0000 7fec69600640 1 -- 172.21.15.44:0/3060063328 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fec6005deb0 con 0x7fec54004810 2024-09-13T08:48:41.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:41.383+0000 7fec74600640 1 -- 172.21.15.44:0/3060063328 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone677511", "target": ["mon-mgr", ""]}) -- 0x7fec48000bd0 con 0x7fec500746e0 2024-09-13T08:48:42.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.207+0000 7fec69600640 1 -- 172.21.15.44:0/3060063328 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fec48000bd0 con 0x7fec500746e0 2024-09-13T08:48:42.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 -- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec500746e0 msgr2=0x7fec50076ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec500746e0 0x7fec50076ba0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0b0000 tx=0x7fec5c01c640 comp rx=0 tx=0).stop 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 -- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec54004810 msgr2=0x7fec6c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec54004810 0x7fec6c15a930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c340 tx=0x7fec6000c800 comp rx=0 tx=0).stop 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 -- 172.21.15.44:0/3060063328 shutdown_connections 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec500746e0 0x7fec50076ba0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec6c0c9c10 0x7fec6c0ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec6c15f9e0 0x7fec6c1f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 --2- 172.21.15.44:0/3060063328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec54004810 0x7fec6c15a930 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 -- 172.21.15.44:0/3060063328 >> 172.21.15.44:0/3060063328 conn(0x7fec6c0d1dd0 msgr2=0x7fec6c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 -- 172.21.15.44:0/3060063328 shutdown_connections 2024-09-13T08:48:42.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.211+0000 7fec74600640 1 -- 172.21.15.44:0/3060063328 wait complete. 2024-09-13T08:48:42.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone268612 2024-09-13T08:48:42.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.507+0000 7fa940e00640 1 -- 172.21.15.44:0/807918485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa93c15ac70 msgr2=0x7fa93c0cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:42.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.507+0000 7fa940e00640 1 --2- 172.21.15.44:0/807918485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa93c15ac70 0x7fa93c0cf2a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0a2200 tx=0x7fa92c01c760 comp rx=0 tx=0).stop 2024-09-13T08:48:42.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.511+0000 7fa940e00640 1 -- 172.21.15.44:0/807918485 shutdown_connections 2024-09-13T08:48:42.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.511+0000 7fa940e00640 1 --2- 172.21.15.44:0/807918485 >> v2:172.21.15.129:3300/0 conn(0x7fa93c0b2e50 0x7fa93c06f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:42.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.511+0000 7fa940e00640 1 --2- 172.21.15.44:0/807918485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa93c15ac70 0x7fa93c0cf2a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:42.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.511+0000 7fa940e00640 1 -- 172.21.15.44:0/807918485 >> 172.21.15.44:0/807918485 conn(0x7fa93c0cd740 msgr2=0x7fa93c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:42.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.511+0000 7fa940e00640 1 -- 172.21.15.44:0/807918485 shutdown_connections 2024-09-13T08:48:42.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.511+0000 7fa940e00640 1 -- 172.21.15.44:0/807918485 wait complete. 2024-09-13T08:48:42.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa940e00640 1 Processor -- start 2024-09-13T08:48:42.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa940e00640 1 -- start start 2024-09-13T08:48:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa940e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa93c0b2e50 0x7fa93c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa940e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa93c0cbdb0 0x7fa93c0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa940e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa93c15ac70 0x7fa93c156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa940e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c06cfb0 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa940e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c06ce30 con 0x7fa93c0b2e50 2024-09-13T08:48:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa940e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa93c06d130 con 0x7fa93c15ac70 2024-09-13T08:48:42.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa933e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa93c0cbdb0 0x7fa93c0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:42.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa933e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa93c0cbdb0 0x7fa93c0c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49826/0 (socket says 172.21.15.44:49826) 2024-09-13T08:48:42.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa933e00640 1 -- 172.21.15.44:0/1101165072 learned_addr learned my addr 172.21.15.44:0/1101165072 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:42.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa933e00640 1 -- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa93c15ac70 msgr2=0x7fa93c156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:42.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa933e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa93c15ac70 0x7fa93c156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:42.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa933e00640 1 -- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa93c0b2e50 msgr2=0x7fa93c15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:42.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa933e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa93c0b2e50 0x7fa93c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:42.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.515+0000 7fa933e00640 1 -- 172.21.15.44:0/1101165072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93c0d1790 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.519+0000 7fa933e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa93c0cbdb0 0x7fa93c0c3ef0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c450 tx=0x7fa92400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:42.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.519+0000 7fa932000640 1 -- 172.21.15.44:0/1101165072 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924016020 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.519+0000 7fa932000640 1 -- 172.21.15.44:0/1101165072 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa924012920 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.519+0000 7fa932000640 1 -- 172.21.15.44:0/1101165072 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924012d70 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.519+0000 7fa940e00640 1 -- 172.21.15.44:0/1101165072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93c0d1a70 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.519+0000 7fa940e00640 1 -- 172.21.15.44:0/1101165072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93c1566d0 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.519+0000 7fa940e00640 1 -- 172.21.15.44:0/1101165072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93c06f8f0 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.523+0000 7fa932000640 1 -- 172.21.15.44:0/1101165072 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7fa924019020 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.523+0000 7fa932000640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9180744b0 0x7fa918076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:42.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.523+0000 7fa932000640 1 -- 172.21.15.44:0/1101165072 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fa924092ec0 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.527+0000 7fa939e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9180744b0 0x7fa918076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:42.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.527+0000 7fa939e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9180744b0 0x7fa918076970 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0af000 tx=0x7fa92c019500 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:42.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.531+0000 7fa932000640 1 -- 172.21.15.44:0/1101165072 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa924020020 con 0x7fa93c0cbdb0 2024-09-13T08:48:42.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:42.899+0000 7fa940e00640 1 -- 172.21.15.44:0/1101165072 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone268612", "target": ["mon-mgr", ""]}) -- 0x7fa93c0afea0 con 0x7fa9180744b0 2024-09-13T08:48:44.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa932000640 1 -- 172.21.15.44:0/1101165072 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa92c004040 con 0x7fa9180744b0 2024-09-13T08:48:44.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 -- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9180744b0 msgr2=0x7fa918076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:44.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9180744b0 0x7fa918076970 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0af000 tx=0x7fa92c019500 comp rx=0 tx=0).stop 2024-09-13T08:48:44.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 -- 172.21.15.44:0/1101165072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa93c0cbdb0 msgr2=0x7fa93c0c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:44.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa93c0cbdb0 0x7fa93c0c3ef0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c450 tx=0x7fa92400c910 comp rx=0 tx=0).stop 2024-09-13T08:48:44.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 -- 172.21.15.44:0/1101165072 shutdown_connections 2024-09-13T08:48:44.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa9180744b0 0x7fa918076970 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:44.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa93c15ac70 0x7fa93c156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:44.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa93c0cbdb0 0x7fa93c0c3ef0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:44.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 --2- 172.21.15.44:0/1101165072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa93c0b2e50 0x7fa93c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:44.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 -- 172.21.15.44:0/1101165072 >> 172.21.15.44:0/1101165072 conn(0x7fa93c0cd740 msgr2=0x7fa93c0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:44.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 -- 172.21.15.44:0/1101165072 shutdown_connections 2024-09-13T08:48:44.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:44.991+0000 7fa940e00640 1 -- 172.21.15.44:0/1101165072 wait complete. 2024-09-13T08:48:45.037 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone680213 2024-09-13T08:48:45.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.323+0000 7fd5c9200640 1 -- 172.21.15.44:0/2159355146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5a80041c0 msgr2=0x7fd5a8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:45.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.323+0000 7fd5c9200640 1 --2- 172.21.15.44:0/2159355146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5a80041c0 0x7fd5a8004810 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0006fc0 tx=0x7fd5b001f6a0 comp rx=0 tx=0).stop 2024-09-13T08:48:45.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.323+0000 7fd5c2800640 1 -- 172.21.15.44:0/2159355146 reap_dead start 2024-09-13T08:48:45.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.323+0000 7fd5c9200640 1 -- 172.21.15.44:0/2159355146 shutdown_connections 2024-09-13T08:48:45.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.323+0000 7fd5c9200640 1 -- 172.21.15.44:0/2159355146 >> 172.21.15.44:0/2159355146 conn(0x7fd5c40d1dd0 msgr2=0x7fd5c40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:45.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.323+0000 7fd5c9200640 1 -- 172.21.15.44:0/2159355146 shutdown_connections 2024-09-13T08:48:45.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.323+0000 7fd5c9200640 1 -- 172.21.15.44:0/2159355146 wait complete. 2024-09-13T08:48:45.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.323+0000 7fd5c9200640 1 Processor -- start 2024-09-13T08:48:45.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.323+0000 7fd5c9200640 1 -- start start 2024-09-13T08:48:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c9200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5a80041c0 0x7fd5c40c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c415caa0 0x7fd5c40c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c9200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd5c415e710 0x7fd5c415ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c9200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c40b0e50 con 0x7fd5c415caa0 2024-09-13T08:48:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c9200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c40b0cd0 con 0x7fd5a80041c0 2024-09-13T08:48:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c9200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c40b1220 con 0x7fd5c415e710 2024-09-13T08:48:45.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c415caa0 0x7fd5c40c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:45.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c415caa0 0x7fd5c40c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49852/0 (socket says 172.21.15.44:49852) 2024-09-13T08:48:45.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c1400640 1 -- 172.21.15.44:0/2777128189 learned_addr learned my addr 172.21.15.44:0/2777128189 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c2800640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd5c415e710 0x7fd5c415ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c1400640 1 -- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd5c415e710 msgr2=0x7fd5c415ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c1400640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd5c415e710 0x7fd5c415ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c1400640 1 -- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5a80041c0 msgr2=0x7fd5c40c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c1400640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5a80041c0 0x7fd5c40c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c1400640 1 -- 172.21.15.44:0/2777128189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c415faa0 con 0x7fd5c415caa0 2024-09-13T08:48:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c1400640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c415caa0 0x7fd5c40c8360 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00263b0 tx=0x7fd5b0006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:45.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5ba000640 1 -- 172.21.15.44:0/2777128189 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00c0070 con 0x7fd5c415caa0 2024-09-13T08:48:45.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5ba000640 1 -- 172.21.15.44:0/2777128189 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5b00b4040 con 0x7fd5c415caa0 2024-09-13T08:48:45.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5ba000640 1 -- 172.21.15.44:0/2777128189 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00b3030 con 0x7fd5c415caa0 2024-09-13T08:48:45.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c9200640 1 -- 172.21.15.44:0/2777128189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c415cf80 con 0x7fd5c415caa0 2024-09-13T08:48:45.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.327+0000 7fd5c9200640 1 -- 172.21.15.44:0/2777128189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c415f7c0 con 0x7fd5c415caa0 2024-09-13T08:48:45.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.331+0000 7fd5c9200640 1 -- 172.21.15.44:0/2777128189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c40d0690 con 0x7fd5c415caa0 2024-09-13T08:48:45.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.331+0000 7fd5ba000640 1 -- 172.21.15.44:0/2777128189 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 99069+0+0 (secure 0 0 0) 0x7fd5b00c3070 con 0x7fd5c415caa0 2024-09-13T08:48:45.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.335+0000 7fd5ba000640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5a40744e0 0x7fd5a40769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:45.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.335+0000 7fd5ba000640 1 -- 172.21.15.44:0/2777128189 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fd5b012feb0 con 0x7fd5c415caa0 2024-09-13T08:48:45.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.335+0000 7fd5c1e00640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5a40744e0 0x7fd5a40769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:45.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.339+0000 7fd5ba000640 1 -- 172.21.15.44:0/2777128189 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd5b00ca080 con 0x7fd5c415caa0 2024-09-13T08:48:45.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.351+0000 7fd5c1e00640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5a40744e0 0x7fd5a40769a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400ba80 tx=0x7fd5b4005d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:45.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.711+0000 7fd5c9200640 1 -- 172.21.15.44:0/2777128189 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone680213", "target": ["mon-mgr", ""]}) -- 0x7fd5c40d1080 con 0x7fd5a40744e0 2024-09-13T08:48:45.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:45.851+0000 7fd5ba000640 1 -- 172.21.15.44:0/2777128189 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7fd5b00f87b0 con 0x7fd5c415caa0 2024-09-13T08:48:46.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.363+0000 7fd5ba000640 1 -- 172.21.15.44:0/2777128189 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd5c40d1080 con 0x7fd5a40744e0 2024-09-13T08:48:46.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.371+0000 7fd5c9200640 1 -- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5a40744e0 msgr2=0x7fd5a40769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:46.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.371+0000 7fd5c9200640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5a40744e0 0x7fd5a40769a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400ba80 tx=0x7fd5b4005d10 comp rx=0 tx=0).stop 2024-09-13T08:48:46.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.371+0000 7fd5c9200640 1 -- 172.21.15.44:0/2777128189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c415caa0 msgr2=0x7fd5c40c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:46.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.371+0000 7fd5c9200640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c415caa0 0x7fd5c40c8360 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00263b0 tx=0x7fd5b0006be0 comp rx=0 tx=0).stop 2024-09-13T08:48:46.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.371+0000 7fd5c9200640 1 -- 172.21.15.44:0/2777128189 shutdown_connections 2024-09-13T08:48:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.371+0000 7fd5c9200640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5a40744e0 0x7fd5a40769a0 secure :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400ba80 tx=0x7fd5b4005d10 comp rx=0 tx=0).stop 2024-09-13T08:48:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.371+0000 7fd5c9200640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd5c415e710 0x7fd5c415ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.371+0000 7fd5c9200640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c415caa0 0x7fd5c40c8360 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.371+0000 7fd5c9200640 1 --2- 172.21.15.44:0/2777128189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5a80041c0 0x7fd5c40c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:46.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.371+0000 7fd5c9200640 1 -- 172.21.15.44:0/2777128189 >> 172.21.15.44:0/2777128189 conn(0x7fd5c40d1dd0 msgr2=0x7fd5c4070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.375+0000 7fd5c9200640 1 -- 172.21.15.44:0/2777128189 shutdown_connections 2024-09-13T08:48:46.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.375+0000 7fd5c9200640 1 -- 172.21.15.44:0/2777128189 wait complete. 2024-09-13T08:48:46.429 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone855814 2024-09-13T08:48:46.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.711+0000 7f2c4b800640 1 -- 172.21.15.44:0/1706168723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c28004810 msgr2=0x7f2c28004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:46.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.711+0000 7f2c4b800640 1 --2- 172.21.15.44:0/1706168723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c28004810 0x7f2c28004c90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2c440b7e80 tx=0x7f2c2c01ce70 comp rx=0 tx=0).stop 2024-09-13T08:48:46.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.711+0000 7f2c49800640 1 -- 172.21.15.44:0/1706168723 reap_dead start 2024-09-13T08:48:46.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.711+0000 7f2c4b800640 1 -- 172.21.15.44:0/1706168723 shutdown_connections 2024-09-13T08:48:46.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.711+0000 7f2c4b800640 1 -- 172.21.15.44:0/1706168723 >> 172.21.15.44:0/1706168723 conn(0x7f2c440d1dd0 msgr2=0x7f2c440d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:46.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.711+0000 7f2c4b800640 1 -- 172.21.15.44:0/1706168723 shutdown_connections 2024-09-13T08:48:46.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.711+0000 7f2c4b800640 1 -- 172.21.15.44:0/1706168723 wait complete. 2024-09-13T08:48:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.711+0000 7f2c4b800640 1 Processor -- start 2024-09-13T08:48:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.711+0000 7f2c4b800640 1 -- start start 2024-09-13T08:48:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c4b800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2c28004810 0x7f2c441f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c4b800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c440b5530 0x7f2c440c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c4b800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c4415d310 0x7f2c44168130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c4b800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44168f20 con 0x7f2c440b5530 2024-09-13T08:48:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c4b800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c440b24a0 con 0x7f2c4415d310 2024-09-13T08:48:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c4b800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2c441690a0 con 0x7f2c28004810 2024-09-13T08:48:46.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c440b5530 0x7f2c440c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:46.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c440b5530 0x7f2c440c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49878/0 (socket says 172.21.15.44:49878) 2024-09-13T08:48:46.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c43e00640 1 -- 172.21.15.44:0/274601886 learned_addr learned my addr 172.21.15.44:0/274601886 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:46.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c48e00640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2c28004810 0x7f2c441f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:46.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c49800640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c4415d310 0x7f2c44168130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c43e00640 1 -- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2c28004810 msgr2=0x7f2c441f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c43e00640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2c28004810 0x7f2c441f8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c43e00640 1 -- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c4415d310 msgr2=0x7f2c44168130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c43e00640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c4415d310 0x7f2c44168130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c43e00640 1 -- 172.21.15.44:0/274601886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c441f8000 con 0x7f2c440b5530 2024-09-13T08:48:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.715+0000 7f2c43e00640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c440b5530 0x7f2c440c7c60 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c01c930 tx=0x7f2c2c0a5ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.719+0000 7f2c40c00640 1 -- 172.21.15.44:0/274601886 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c000fe0 con 0x7f2c440b5530 2024-09-13T08:48:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.719+0000 7f2c40c00640 1 -- 172.21.15.44:0/274601886 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c2c0014c0 con 0x7f2c440b5530 2024-09-13T08:48:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.719+0000 7f2c40c00640 1 -- 172.21.15.44:0/274601886 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0060d0 con 0x7f2c440b5530 2024-09-13T08:48:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.719+0000 7f2c4b800640 1 -- 172.21.15.44:0/274601886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c440c83d0 con 0x7f2c440b5530 2024-09-13T08:48:46.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.719+0000 7f2c4b800640 1 -- 172.21.15.44:0/274601886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c441f8580 con 0x7f2c440b5530 2024-09-13T08:48:46.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.723+0000 7f2c4b800640 1 -- 172.21.15.44:0/274601886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1c0052a0 con 0x7f2c440b5530 2024-09-13T08:48:46.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.723+0000 7f2c40c00640 1 -- 172.21.15.44:0/274601886 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7f2c2c006270 con 0x7f2c440b5530 2024-09-13T08:48:46.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.723+0000 7f2c40c00640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c20074420 0x7f2c200768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:46.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.723+0000 7f2c40c00640 1 -- 172.21.15.44:0/274601886 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f2c2c12c310 con 0x7f2c440b5530 2024-09-13T08:48:46.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.731+0000 7f2c40c00640 1 -- 172.21.15.44:0/274601886 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2c2c016c50 con 0x7f2c440b5530 2024-09-13T08:48:46.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.739+0000 7f2c48e00640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c20074420 0x7f2c200768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:46.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:46.747+0000 7f2c48e00640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c20074420 0x7f2c200768e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005c50 tx=0x7f2c34009560 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:47.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:47.091+0000 7f2c4b800640 1 -- 172.21.15.44:0/274601886 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone855814", "target": ["mon-mgr", ""]}) -- 0x7f2c1c000bd0 con 0x7f2c20074420 2024-09-13T08:48:48.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.463+0000 7f2c40c00640 1 -- 172.21.15.44:0/274601886 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2c1c000bd0 con 0x7f2c20074420 2024-09-13T08:48:48.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.467+0000 7f2c4b800640 1 -- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c20074420 msgr2=0x7f2c200768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:48.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.467+0000 7f2c4b800640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c20074420 0x7f2c200768e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005c50 tx=0x7f2c34009560 comp rx=0 tx=0).stop 2024-09-13T08:48:48.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.467+0000 7f2c4b800640 1 -- 172.21.15.44:0/274601886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c440b5530 msgr2=0x7f2c440c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:48.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.467+0000 7f2c4b800640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c440b5530 0x7f2c440c7c60 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c01c930 tx=0x7f2c2c0a5ca0 comp rx=0 tx=0).stop 2024-09-13T08:48:48.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.467+0000 7f2c4b800640 1 -- 172.21.15.44:0/274601886 shutdown_connections 2024-09-13T08:48:48.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.467+0000 7f2c4b800640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c20074420 0x7f2c200768e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:48.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.467+0000 7f2c4b800640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c4415d310 0x7f2c44168130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:48.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.467+0000 7f2c4b800640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c440b5530 0x7f2c440c7c60 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:48.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.467+0000 7f2c4b800640 1 --2- 172.21.15.44:0/274601886 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2c28004810 0x7f2c441f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:48.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.467+0000 7f2c4b800640 1 -- 172.21.15.44:0/274601886 >> 172.21.15.44:0/274601886 conn(0x7f2c440d1dd0 msgr2=0x7f2c440cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:48.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.471+0000 7f2c4b800640 1 -- 172.21.15.44:0/274601886 shutdown_connections 2024-09-13T08:48:48.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.471+0000 7f2c4b800640 1 -- 172.21.15.44:0/274601886 wait complete. 2024-09-13T08:48:48.532 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone58715 2024-09-13T08:48:48.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.843+0000 7efc8a600640 1 -- 172.21.15.44:0/2773183476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc840afb10 msgr2=0x7efc840affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:48.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.843+0000 7efc8a600640 1 --2- 172.21.15.44:0/2773183476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc840afb10 0x7efc840affc0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009c80 tx=0x7efc7c01cda0 comp rx=0 tx=0).stop 2024-09-13T08:48:48.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.843+0000 7efc8a600640 1 -- 172.21.15.44:0/2773183476 shutdown_connections 2024-09-13T08:48:48.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.843+0000 7efc8a600640 1 --2- 172.21.15.44:0/2773183476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc840afb10 0x7efc840affc0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:48.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.843+0000 7efc8a600640 1 --2- 172.21.15.44:0/2773183476 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc840b1650 0x7efc840b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:48.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.843+0000 7efc8a600640 1 -- 172.21.15.44:0/2773183476 >> 172.21.15.44:0/2773183476 conn(0x7efc840d1dd0 msgr2=0x7efc840d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:48.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.843+0000 7efc8a600640 1 -- 172.21.15.44:0/2773183476 shutdown_connections 2024-09-13T08:48:48.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.843+0000 7efc8a600640 1 -- 172.21.15.44:0/2773183476 wait complete. 2024-09-13T08:48:48.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.847+0000 7efc8a600640 1 Processor -- start 2024-09-13T08:48:48.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.847+0000 7efc8a600640 1 -- start start 2024-09-13T08:48:48.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.847+0000 7efc8a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc840afb10 0x7efc8406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:48.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.847+0000 7efc8a600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc840b1650 0x7efc84158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:48.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.847+0000 7efc8a600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc841605d0 0x7efc841fc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:48.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.847+0000 7efc8a600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efc84169540 con 0x7efc840afb10 2024-09-13T08:48:48.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.847+0000 7efc8a600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc840b1430 con 0x7efc841605d0 2024-09-13T08:48:48.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.847+0000 7efc8a600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7efc841696c0 con 0x7efc840b1650 2024-09-13T08:48:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc83400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc840afb10 0x7efc8406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc83400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc840afb10 0x7efc8406e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49898/0 (socket says 172.21.15.44:49898) 2024-09-13T08:48:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc83400640 1 -- 172.21.15.44:0/1191514909 learned_addr learned my addr 172.21.15.44:0/1191514909 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:48.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc83400640 1 -- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc840b1650 msgr2=0x7efc84158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:48.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc83400640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc840b1650 0x7efc84158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:48.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc83400640 1 -- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc841605d0 msgr2=0x7efc841fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:48.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc83400640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc841605d0 0x7efc841fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:48.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc83400640 1 -- 172.21.15.44:0/1191514909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8406fb40 con 0x7efc840afb10 2024-09-13T08:48:48.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc83400640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc840afb10 0x7efc8406e890 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efc7400b3e0 tx=0x7efc7400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:48.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc7b400640 1 -- 172.21.15.44:0/1191514909 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74015020 con 0x7efc840afb10 2024-09-13T08:48:48.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc7b400640 1 -- 172.21.15.44:0/1191514909 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc74011940 con 0x7efc840afb10 2024-09-13T08:48:48.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc7b400640 1 -- 172.21.15.44:0/1191514909 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74011d90 con 0x7efc840afb10 2024-09-13T08:48:48.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc8a600640 1 -- 172.21.15.44:0/1191514909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8406f860 con 0x7efc840afb10 2024-09-13T08:48:48.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.851+0000 7efc8a600640 1 -- 172.21.15.44:0/1191514909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8406e090 con 0x7efc840afb10 2024-09-13T08:48:48.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.855+0000 7efc8a600640 1 -- 172.21.15.44:0/1191514909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc840affc0 con 0x7efc840afb10 2024-09-13T08:48:48.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.855+0000 7efc7b400640 1 -- 172.21.15.44:0/1191514909 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7efc74018020 con 0x7efc840afb10 2024-09-13T08:48:48.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.855+0000 7efc7b400640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc64074420 0x7efc640768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:48.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.855+0000 7efc7b400640 1 -- 172.21.15.44:0/1191514909 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7efc74092e50 con 0x7efc840afb10 2024-09-13T08:48:48.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.863+0000 7efc82a00640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc64074420 0x7efc640768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:48.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.867+0000 7efc7b400640 1 -- 172.21.15.44:0/1191514909 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7efc7401f020 con 0x7efc840afb10 2024-09-13T08:48:48.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:48.887+0000 7efc82a00640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc64074420 0x7efc640768e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efc840b1ac0 tx=0x7efc70006ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:49.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:49.319+0000 7efc8a600640 1 -- 172.21.15.44:0/1191514909 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone58715", "target": ["mon-mgr", ""]}) -- 0x7efc840d1080 con 0x7efc64074420 2024-09-13T08:48:50.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.915+0000 7efc7b400640 1 -- 172.21.15.44:0/1191514909 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc840d1080 con 0x7efc64074420 2024-09-13T08:48:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 -- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc64074420 msgr2=0x7efc640768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc64074420 0x7efc640768e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efc840b1ac0 tx=0x7efc70006ca0 comp rx=0 tx=0).stop 2024-09-13T08:48:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 -- 172.21.15.44:0/1191514909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc840afb10 msgr2=0x7efc8406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc840afb10 0x7efc8406e890 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efc7400b3e0 tx=0x7efc7400b8a0 comp rx=0 tx=0).stop 2024-09-13T08:48:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 -- 172.21.15.44:0/1191514909 shutdown_connections 2024-09-13T08:48:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7efc64074420 0x7efc640768e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc841605d0 0x7efc841fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7efc840b1650 0x7efc84158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 --2- 172.21.15.44:0/1191514909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc840afb10 0x7efc8406e890 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 -- 172.21.15.44:0/1191514909 >> 172.21.15.44:0/1191514909 conn(0x7efc840d1dd0 msgr2=0x7efc8415b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 -- 172.21.15.44:0/1191514909 shutdown_connections 2024-09-13T08:48:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:50.923+0000 7efc8a600640 1 -- 172.21.15.44:0/1191514909 wait complete. 2024-09-13T08:48:50.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone945516 2024-09-13T08:48:51.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.307+0000 7f8d94200640 1 -- 172.21.15.44:0/2971618527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d8c070950 msgr2=0x7f8d8c073d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:51.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.307+0000 7f8d94200640 1 --2- 172.21.15.44:0/2971618527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d8c070950 0x7f8d8c073d60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009c80 tx=0x7f8d8401f5a0 comp rx=0 tx=0).stop 2024-09-13T08:48:51.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.307+0000 7f8d94200640 1 -- 172.21.15.44:0/2971618527 shutdown_connections 2024-09-13T08:48:51.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.307+0000 7f8d94200640 1 --2- 172.21.15.44:0/2971618527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d8c070950 0x7f8d8c073d60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:51.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.307+0000 7f8d94200640 1 --2- 172.21.15.44:0/2971618527 >> v2:172.21.15.129:3300/0 conn(0x7f8d8c15e370 0x7f8d8c073820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:51.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.307+0000 7f8d94200640 1 -- 172.21.15.44:0/2971618527 >> 172.21.15.44:0/2971618527 conn(0x7f8d8c0d0c90 msgr2=0x7f8d8c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:51.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.307+0000 7f8d94200640 1 -- 172.21.15.44:0/2971618527 shutdown_connections 2024-09-13T08:48:51.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.307+0000 7f8d94200640 1 -- 172.21.15.44:0/2971618527 wait complete. 2024-09-13T08:48:51.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d94200640 1 Processor -- start 2024-09-13T08:48:51.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d94200640 1 -- start start 2024-09-13T08:48:51.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d94200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d8c070950 0x7f8d8c0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:51.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d94200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d8c0c4320 0x7f8d8c0c3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:51.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d94200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d8c15e370 0x7f8d8c0c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:51.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d94200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c0cb9f0 con 0x7f8d8c15e370 2024-09-13T08:48:51.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d94200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c0cea70 con 0x7f8d8c0c4320 2024-09-13T08:48:51.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d94200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8d8c0cbb70 con 0x7f8d8c070950 2024-09-13T08:48:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d92200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d8c15e370 0x7f8d8c0c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d92200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d8c15e370 0x7f8d8c0c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49910/0 (socket says 172.21.15.44:49910) 2024-09-13T08:48:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d92200640 1 -- 172.21.15.44:0/4248821801 learned_addr learned my addr 172.21.15.44:0/4248821801 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d90e00640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d8c0c4320 0x7f8d8c0c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d91800640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d8c070950 0x7f8d8c0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d92200640 1 -- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d8c070950 msgr2=0x7f8d8c0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d92200640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d8c070950 0x7f8d8c0ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d92200640 1 -- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d8c0c4320 msgr2=0x7f8d8c0c3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d92200640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d8c0c4320 0x7f8d8c0c3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d92200640 1 -- 172.21.15.44:0/4248821801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8c0c4930 con 0x7f8d8c15e370 2024-09-13T08:48:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.311+0000 7f8d91800640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d8c070950 0x7f8d8c0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:48:51.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.315+0000 7f8d92200640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d8c15e370 0x7f8d8c0c69c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009bd0 tx=0x7f8d84009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:51.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.315+0000 7f8d89600640 1 -- 172.21.15.44:0/4248821801 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840b6040 con 0x7f8d8c15e370 2024-09-13T08:48:51.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.315+0000 7f8d89600640 1 -- 172.21.15.44:0/4248821801 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d84004d10 con 0x7f8d8c15e370 2024-09-13T08:48:51.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.315+0000 7f8d89600640 1 -- 172.21.15.44:0/4248821801 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840b1b40 con 0x7f8d8c15e370 2024-09-13T08:48:51.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.315+0000 7f8d94200640 1 -- 172.21.15.44:0/4248821801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8c0afdd0 con 0x7f8d8c15e370 2024-09-13T08:48:51.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.315+0000 7f8d94200640 1 -- 172.21.15.44:0/4248821801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d8c0af640 con 0x7f8d8c15e370 2024-09-13T08:48:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.315+0000 7f8d94200640 1 -- 172.21.15.44:0/4248821801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d8c0cb2e0 con 0x7f8d8c15e370 2024-09-13T08:48:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.319+0000 7f8d89600640 1 -- 172.21.15.44:0/4248821801 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7f8d840b15c0 con 0x7f8d8c15e370 2024-09-13T08:48:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.319+0000 7f8d89600640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d6c074420 0x7f8d6c0768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.319+0000 7f8d89600640 1 -- 172.21.15.44:0/4248821801 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f8d841317b0 con 0x7f8d8c15e370 2024-09-13T08:48:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.319+0000 7f8d91800640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d6c074420 0x7f8d6c0768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.323+0000 7f8d89600640 1 -- 172.21.15.44:0/4248821801 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8d840ab070 con 0x7f8d8c15e370 2024-09-13T08:48:51.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.323+0000 7f8d91800640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d6c074420 0x7f8d6c0768e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8d78005fd0 tx=0x7f8d78005f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:51.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:51.679+0000 7f8d94200640 1 -- 172.21.15.44:0/4248821801 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone945516", "target": ["mon-mgr", ""]}) -- 0x7f8d8c0ca6b0 con 0x7f8d6c074420 2024-09-13T08:48:52.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.655+0000 7f8d89600640 1 -- 172.21.15.44:0/4248821801 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d7800fb30 con 0x7f8d6c074420 2024-09-13T08:48:52.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.671+0000 7f8d94200640 1 -- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d6c074420 msgr2=0x7f8d6c0768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:52.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.671+0000 7f8d94200640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d6c074420 0x7f8d6c0768e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8d78005fd0 tx=0x7f8d78005f00 comp rx=0 tx=0).stop 2024-09-13T08:48:52.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.671+0000 7f8d94200640 1 -- 172.21.15.44:0/4248821801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d8c15e370 msgr2=0x7f8d8c0c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:52.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.671+0000 7f8d94200640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d8c15e370 0x7f8d8c0c69c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009bd0 tx=0x7f8d84009390 comp rx=0 tx=0).stop 2024-09-13T08:48:52.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.675+0000 7f8d94200640 1 -- 172.21.15.44:0/4248821801 shutdown_connections 2024-09-13T08:48:52.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.675+0000 7f8d94200640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d6c074420 0x7f8d6c0768e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:52.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.675+0000 7f8d94200640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d8c15e370 0x7f8d8c0c69c0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:52.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.675+0000 7f8d94200640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d8c0c4320 0x7f8d8c0c3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:52.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.675+0000 7f8d94200640 1 --2- 172.21.15.44:0/4248821801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d8c070950 0x7f8d8c0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:52.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.675+0000 7f8d94200640 1 -- 172.21.15.44:0/4248821801 >> 172.21.15.44:0/4248821801 conn(0x7f8d8c0d0c90 msgr2=0x7f8d8c15cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:52.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.675+0000 7f8d94200640 1 -- 172.21.15.44:0/4248821801 shutdown_connections 2024-09-13T08:48:52.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:52.675+0000 7f8d94200640 1 -- 172.21.15.44:0/4248821801 wait complete. 2024-09-13T08:48:52.725 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone453217 2024-09-13T08:48:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.071+0000 7f76a1000640 1 -- 172.21.15.44:0/1397908265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c0cdae0 msgr2=0x7f769c0cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.071+0000 7f76a1000640 1 --2- 172.21.15.44:0/1397908265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c0cdae0 0x7f769c0cdf00 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f768c009900 tx=0x7f768c01c820 comp rx=0 tx=0).stop 2024-09-13T08:48:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.071+0000 7f76a1000640 1 -- 172.21.15.44:0/1397908265 shutdown_connections 2024-09-13T08:48:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.071+0000 7f76a1000640 1 --2- 172.21.15.44:0/1397908265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c0cdae0 0x7f769c0cdf00 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.071+0000 7f76a1000640 1 --2- 172.21.15.44:0/1397908265 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c15f300 0x7f769c0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.071+0000 7f76a1000640 1 -- 172.21.15.44:0/1397908265 >> 172.21.15.44:0/1397908265 conn(0x7f769c0b18f0 msgr2=0x7f769c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.071+0000 7f76a1000640 1 -- 172.21.15.44:0/1397908265 shutdown_connections 2024-09-13T08:48:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.071+0000 7f76a1000640 1 -- 172.21.15.44:0/1397908265 wait complete. 2024-09-13T08:48:53.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.071+0000 7f76a1000640 1 Processor -- start 2024-09-13T08:48:53.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.071+0000 7f76a1000640 1 -- start start 2024-09-13T08:48:53.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f76a1000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c0cdae0 0x7f769c153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:53.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f76a1000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c15f300 0x7f769c1f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:53.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f76a1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c15f900 0x7f769c15e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:53.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f76a1000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f769c169840 con 0x7f769c15f900 2024-09-13T08:48:53.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f76a1000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f769c1696c0 con 0x7f769c0cdae0 2024-09-13T08:48:53.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f76a1000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f769c1699c0 con 0x7f769c15f300 2024-09-13T08:48:53.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f7699400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c15f300 0x7f769c1f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:53.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f7699400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c15f300 0x7f769c1f8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:57524/0 (socket says 172.21.15.44:57524) 2024-09-13T08:48:53.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f7699400640 1 -- 172.21.15.44:0/2997780627 learned_addr learned my addr 172.21.15.44:0/2997780627 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:53.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f769a800640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c15f900 0x7f769c15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:53.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f7699e00640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c0cdae0 0x7f769c153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:53.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f769a800640 1 -- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c15f300 msgr2=0x7f769c1f8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:53.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f769a800640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c15f300 0x7f769c1f8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:53.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f769a800640 1 -- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c0cdae0 msgr2=0x7f769c153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:53.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f769a800640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c0cdae0 0x7f769c153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:53.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f769a800640 1 -- 172.21.15.44:0/2997780627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c153760 con 0x7f769c15f900 2024-09-13T08:48:53.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.075+0000 7f769a800640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c15f900 0x7f769c15e050 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f768800b410 tx=0x7f768800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:53.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.079+0000 7f7692000640 1 -- 172.21.15.44:0/2997780627 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688015020 con 0x7f769c15f900 2024-09-13T08:48:53.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.079+0000 7f7692000640 1 -- 172.21.15.44:0/2997780627 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7688011920 con 0x7f769c15f900 2024-09-13T08:48:53.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.079+0000 7f7692000640 1 -- 172.21.15.44:0/2997780627 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688011db0 con 0x7f769c15f900 2024-09-13T08:48:53.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.079+0000 7f76a1000640 1 -- 172.21.15.44:0/2997780627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c1f8b30 con 0x7f769c15f900 2024-09-13T08:48:53.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.079+0000 7f76a1000640 1 -- 172.21.15.44:0/2997780627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c0b2f00 con 0x7f769c15f900 2024-09-13T08:48:53.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.079+0000 7f76a1000640 1 -- 172.21.15.44:0/2997780627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c0b2cf0 con 0x7f769c15f900 2024-09-13T08:48:53.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.083+0000 7f7692000640 1 -- 172.21.15.44:0/2997780627 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7f7688018020 con 0x7f769c15f900 2024-09-13T08:48:53.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.087+0000 7f7692000640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f767c074420 0x7f767c0768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:53.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.087+0000 7f7692000640 1 -- 172.21.15.44:0/2997780627 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f7688092c50 con 0x7f769c15f900 2024-09-13T08:48:53.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.091+0000 7f7699e00640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f767c074420 0x7f767c0768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:53.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.091+0000 7f7692000640 1 -- 172.21.15.44:0/2997780627 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f768801f020 con 0x7f769c15f900 2024-09-13T08:48:53.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.095+0000 7f7699e00640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f767c074420 0x7f767c0768e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7694009f20 tx=0x7f76940094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:53.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:53.479+0000 7f76a1000640 1 -- 172.21.15.44:0/2997780627 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone453217", "target": ["mon-mgr", ""]}) -- 0x7f769c06f600 con 0x7f767c074420 2024-09-13T08:48:54.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.035+0000 7f7692000640 1 -- 172.21.15.44:0/2997780627 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f769c06f600 con 0x7f767c074420 2024-09-13T08:48:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 -- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f767c074420 msgr2=0x7f767c0768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f767c074420 0x7f767c0768e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7694009f20 tx=0x7f76940094f0 comp rx=0 tx=0).stop 2024-09-13T08:48:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 -- 172.21.15.44:0/2997780627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c15f900 msgr2=0x7f769c15e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c15f900 0x7f769c15e050 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f768800b410 tx=0x7f768800b8d0 comp rx=0 tx=0).stop 2024-09-13T08:48:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 -- 172.21.15.44:0/2997780627 shutdown_connections 2024-09-13T08:48:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f767c074420 0x7f767c0768e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c15f900 0x7f769c15e050 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c15f300 0x7f769c1f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 --2- 172.21.15.44:0/2997780627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c0cdae0 0x7f769c153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 -- 172.21.15.44:0/2997780627 >> 172.21.15.44:0/2997780627 conn(0x7f769c0b18f0 msgr2=0x7f769c1569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 -- 172.21.15.44:0/2997780627 shutdown_connections 2024-09-13T08:48:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.043+0000 7f76a1000640 1 -- 172.21.15.44:0/2997780627 wait complete. 2024-09-13T08:48:54.085 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone300218 2024-09-13T08:48:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.411+0000 7fe371e00640 1 -- 172.21.15.44:0/3877618329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c06ecb0 msgr2=0x7fe36c0b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.411+0000 7fe371e00640 1 --2- 172.21.15.44:0/3877618329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c06ecb0 0x7fe36c0b4a60 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0cc420 tx=0x7fe35401c820 comp rx=0 tx=0).stop 2024-09-13T08:48:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.411+0000 7fe371e00640 1 -- 172.21.15.44:0/3877618329 shutdown_connections 2024-09-13T08:48:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.411+0000 7fe371e00640 1 --2- 172.21.15.44:0/3877618329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c06ecb0 0x7fe36c0b4a60 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.411+0000 7fe371e00640 1 -- 172.21.15.44:0/3877618329 >> 172.21.15.44:0/3877618329 conn(0x7fe36c0b18f0 msgr2=0x7fe36c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.411+0000 7fe371e00640 1 -- 172.21.15.44:0/3877618329 shutdown_connections 2024-09-13T08:48:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.411+0000 7fe371e00640 1 -- 172.21.15.44:0/3877618329 wait complete. 2024-09-13T08:48:54.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.411+0000 7fe371e00640 1 Processor -- start 2024-09-13T08:48:54.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.411+0000 7fe371e00640 1 -- start start 2024-09-13T08:48:54.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe371e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c06ecb0 0x7fe36c149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:54.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe371e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe36c0b2bf0 0x7fe36c15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:54.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe371e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe36c158a20 0x7fe36c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:54.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe371e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c169120 con 0x7fe36c06ecb0 2024-09-13T08:48:54.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe371e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c168fa0 con 0x7fe36c158a20 2024-09-13T08:48:54.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe371e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe36c1692a0 con 0x7fe36c0b2bf0 2024-09-13T08:48:54.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe36aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c06ecb0 0x7fe36c149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:54.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe36aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c06ecb0 0x7fe36c149000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41286/0 (socket says 172.21.15.44:41286) 2024-09-13T08:48:54.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe36aa00640 1 -- 172.21.15.44:0/1645289205 learned_addr learned my addr 172.21.15.44:0/1645289205 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:54.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe36aa00640 1 -- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe36c0b2bf0 msgr2=0x7fe36c15c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe36aa00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe36c0b2bf0 0x7fe36c15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe36aa00640 1 -- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe36c158a20 msgr2=0x7fe36c154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe36aa00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe36c158a20 0x7fe36c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe36aa00640 1 -- 172.21.15.44:0/1645289205 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c15c950 con 0x7fe36c06ecb0 2024-09-13T08:48:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe36aa00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c06ecb0 0x7fe36c149000 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b3e0 tx=0x7fe35c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe363400640 1 -- 172.21.15.44:0/1645289205 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c015020 con 0x7fe36c06ecb0 2024-09-13T08:48:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe363400640 1 -- 172.21.15.44:0/1645289205 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe35c011940 con 0x7fe36c06ecb0 2024-09-13T08:48:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe371e00640 1 -- 172.21.15.44:0/1645289205 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c164230 con 0x7fe36c06ecb0 2024-09-13T08:48:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe363400640 1 -- 172.21.15.44:0/1645289205 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c011d90 con 0x7fe36c06ecb0 2024-09-13T08:48:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe371e00640 1 -- 172.21.15.44:0/1645289205 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c1647f0 con 0x7fe36c06ecb0 2024-09-13T08:48:54.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.415+0000 7fe371e00640 1 -- 172.21.15.44:0/1645289205 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36c069410 con 0x7fe36c06ecb0 2024-09-13T08:48:54.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.419+0000 7fe363400640 1 -- 172.21.15.44:0/1645289205 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7fe35c018020 con 0x7fe36c06ecb0 2024-09-13T08:48:54.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.419+0000 7fe363400640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe348074450 0x7fe348076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:54.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.419+0000 7fe363400640 1 -- 172.21.15.44:0/1645289205 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fe35c092e30 con 0x7fe36c06ecb0 2024-09-13T08:48:54.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.423+0000 7fe363400640 1 -- 172.21.15.44:0/1645289205 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe35c01f020 con 0x7fe36c06ecb0 2024-09-13T08:48:54.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.427+0000 7fe363e00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe348074450 0x7fe348076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.439+0000 7fe363e00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe348074450 0x7fe348076910 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe3540af000 tx=0x7fe3540195c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:54.787+0000 7fe371e00640 1 -- 172.21.15.44:0/1645289205 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone300218", "target": ["mon-mgr", ""]}) -- 0x7fe36c09dce0 con 0x7fe348074450 2024-09-13T08:48:55.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe363400640 1 -- 172.21.15.44:0/1645289205 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe36c09dce0 con 0x7fe348074450 2024-09-13T08:48:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe371e00640 1 -- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe348074450 msgr2=0x7fe348076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:55.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe371e00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe348074450 0x7fe348076910 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe3540af000 tx=0x7fe3540195c0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe371e00640 1 -- 172.21.15.44:0/1645289205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c06ecb0 msgr2=0x7fe36c149000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe371e00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c06ecb0 0x7fe36c149000 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b3e0 tx=0x7fe35c00b8a0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe371e00640 1 -- 172.21.15.44:0/1645289205 shutdown_connections 2024-09-13T08:48:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe371e00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe348074450 0x7fe348076910 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe371e00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe36c158a20 0x7fe36c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe371e00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe36c0b2bf0 0x7fe36c15c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe371e00640 1 --2- 172.21.15.44:0/1645289205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c06ecb0 0x7fe36c149000 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.195+0000 7fe371e00640 1 -- 172.21.15.44:0/1645289205 >> 172.21.15.44:0/1645289205 conn(0x7fe36c0b18f0 msgr2=0x7fe36c15e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.199+0000 7fe371e00640 1 -- 172.21.15.44:0/1645289205 shutdown_connections 2024-09-13T08:48:55.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.199+0000 7fe371e00640 1 -- 172.21.15.44:0/1645289205 wait complete. 2024-09-13T08:48:55.244 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone632019 2024-09-13T08:48:55.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.507+0000 7fc430c00640 1 -- 172.21.15.44:0/555721578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42c15cf80 msgr2=0x7fc42c0ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:55.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.507+0000 7fc430c00640 1 --2- 172.21.15.44:0/555721578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42c15cf80 0x7fc42c0ca050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc414009c80 tx=0x7fc41401cd40 comp rx=0 tx=0).stop 2024-09-13T08:48:55.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.507+0000 7fc430c00640 1 -- 172.21.15.44:0/555721578 shutdown_connections 2024-09-13T08:48:55.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.507+0000 7fc430c00640 1 --2- 172.21.15.44:0/555721578 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42c0cbd80 0x7fc42c0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.507+0000 7fc430c00640 1 --2- 172.21.15.44:0/555721578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42c15cf80 0x7fc42c0ca050 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.507+0000 7fc430c00640 1 -- 172.21.15.44:0/555721578 >> 172.21.15.44:0/555721578 conn(0x7fc42c0cfa00 msgr2=0x7fc42c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:55.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.507+0000 7fc430c00640 1 -- 172.21.15.44:0/555721578 shutdown_connections 2024-09-13T08:48:55.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.507+0000 7fc430c00640 1 -- 172.21.15.44:0/555721578 wait complete. 2024-09-13T08:48:55.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc430c00640 1 Processor -- start 2024-09-13T08:48:55.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc430c00640 1 -- start start 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc430c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc42c0cbd80 0x7fc42c0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc430c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42c15bb90 0x7fc42c146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc430c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42c15cf80 0x7fc42c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc430c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c0aede0 con 0x7fc42c0cbd80 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc430c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c0aec60 con 0x7fc42c15bb90 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc430c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc42c0aef60 con 0x7fc42c15cf80 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc42b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc42c0cbd80 0x7fc42c0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc42b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc42c0cbd80 0x7fc42c0cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41296/0 (socket says 172.21.15.44:41296) 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc42b400640 1 -- 172.21.15.44:0/907633957 learned_addr learned my addr 172.21.15.44:0/907633957 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc422a00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42c15bb90 0x7fc42c146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc42be00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42c15cf80 0x7fc42c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:55.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc42b400640 1 -- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42c15cf80 msgr2=0x7fc42c0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:55.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc42b400640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42c15cf80 0x7fc42c0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc42b400640 1 -- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42c15bb90 msgr2=0x7fc42c146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:55.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc42b400640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42c15bb90 0x7fc42c146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.511+0000 7fc42b400640 1 -- 172.21.15.44:0/907633957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c1fb6a0 con 0x7fc42c0cbd80 2024-09-13T08:48:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.515+0000 7fc42b400640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc42c0cbd80 0x7fc42c0cd5d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c500 tx=0x7fc41c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.515+0000 7fc423e00640 1 -- 172.21.15.44:0/907633957 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c017070 con 0x7fc42c0cbd80 2024-09-13T08:48:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.515+0000 7fc423e00640 1 -- 172.21.15.44:0/907633957 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc41c0053c0 con 0x7fc42c0cbd80 2024-09-13T08:48:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.515+0000 7fc430c00640 1 -- 172.21.15.44:0/907633957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c1fb960 con 0x7fc42c0cbd80 2024-09-13T08:48:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.515+0000 7fc423e00640 1 -- 172.21.15.44:0/907633957 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c012890 con 0x7fc42c0cbd80 2024-09-13T08:48:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.515+0000 7fc430c00640 1 -- 172.21.15.44:0/907633957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1fbf20 con 0x7fc42c0cbd80 2024-09-13T08:48:55.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.515+0000 7fc430c00640 1 -- 172.21.15.44:0/907633957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c160060 con 0x7fc42c0cbd80 2024-09-13T08:48:55.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.515+0000 7fc423e00640 1 -- 172.21.15.44:0/907633957 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7fc41c01a070 con 0x7fc42c0cbd80 2024-09-13T08:48:55.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.519+0000 7fc423e00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc408074420 0x7fc4080768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:55.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.519+0000 7fc423e00640 1 -- 172.21.15.44:0/907633957 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fc41c092220 con 0x7fc42c0cbd80 2024-09-13T08:48:55.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.519+0000 7fc422a00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc408074420 0x7fc4080768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:55.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.523+0000 7fc422a00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc408074420 0x7fc4080768e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc4140af000 tx=0x7fc41401b5e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:55.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.527+0000 7fc423e00640 1 -- 172.21.15.44:0/907633957 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc41c05e450 con 0x7fc42c0cbd80 2024-09-13T08:48:55.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.863+0000 7fc430c00640 1 -- 172.21.15.44:0/907633957 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone632019", "target": ["mon-mgr", ""]}) -- 0x7fc42c0b51a0 con 0x7fc408074420 2024-09-13T08:48:55.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc423e00640 1 -- 172.21.15.44:0/907633957 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc42c0b51a0 con 0x7fc408074420 2024-09-13T08:48:55.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 -- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc408074420 msgr2=0x7fc4080768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:55.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc408074420 0x7fc4080768e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc4140af000 tx=0x7fc41401b5e0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 -- 172.21.15.44:0/907633957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc42c0cbd80 msgr2=0x7fc42c0cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:55.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc42c0cbd80 0x7fc42c0cd5d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c500 tx=0x7fc41c00c9c0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 -- 172.21.15.44:0/907633957 shutdown_connections 2024-09-13T08:48:55.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc408074420 0x7fc4080768e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42c15cf80 0x7fc42c0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42c15bb90 0x7fc42c146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 --2- 172.21.15.44:0/907633957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc42c0cbd80 0x7fc42c0cd5d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:55.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 -- 172.21.15.44:0/907633957 >> 172.21.15.44:0/907633957 conn(0x7fc42c0cfa00 msgr2=0x7fc42c0b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:55.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 -- 172.21.15.44:0/907633957 shutdown_connections 2024-09-13T08:48:55.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:55.987+0000 7fc430c00640 1 -- 172.21.15.44:0/907633957 wait complete. 2024-09-13T08:48:56.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:48:56.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 -- 172.21.15.44:0/2125260420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f170815f000 msgr2=0x7f17080d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 --2- 172.21.15.44:0/2125260420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f170815f000 0x7f17080d01b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f17080c56a0 tx=0x7f16f801c7a0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 -- 172.21.15.44:0/2125260420 shutdown_connections 2024-09-13T08:48:56.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 --2- 172.21.15.44:0/2125260420 >> v2:172.21.15.129:3300/0 conn(0x7f17080cefb0 0x7f17080cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 --2- 172.21.15.44:0/2125260420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f170815f000 0x7f17080d01b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 -- 172.21.15.44:0/2125260420 >> 172.21.15.44:0/2125260420 conn(0x7f170809aa10 msgr2=0x7f170809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:56.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 -- 172.21.15.44:0/2125260420 shutdown_connections 2024-09-13T08:48:56.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 -- 172.21.15.44:0/2125260420 wait complete. 2024-09-13T08:48:56.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 Processor -- start 2024-09-13T08:48:56.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 -- start start 2024-09-13T08:48:56.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f17080ccaf0 0x7f1708165740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17080cefb0 0x7f17080b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f170815f000 0x7f1708163c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1708169240 con 0x7f170815f000 2024-09-13T08:48:56.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f17081690c0 con 0x7f17080cefb0 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.159+0000 7f170da00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f17081693c0 con 0x7f17080ccaf0 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f170d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f170815f000 0x7f1708163c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f16ffe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17080cefb0 0x7f17080b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f1707e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f17080ccaf0 0x7f1708165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f1707e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f17080ccaf0 0x7f1708165740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:57548/0 (socket says 172.21.15.44:57548) 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f16ffe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17080cefb0 0x7f17080b73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:58614/0 (socket says 172.21.15.44:58614) 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f1707e00640 1 -- 172.21.15.44:0/1817626768 learned_addr learned my addr 172.21.15.44:0/1817626768 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f170d000640 1 -- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f17080ccaf0 msgr2=0x7f1708165740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f170d000640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f17080ccaf0 0x7f1708165740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f170d000640 1 -- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17080cefb0 msgr2=0x7f17080b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f170d000640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17080cefb0 0x7f17080b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f170d000640 1 -- 172.21.15.44:0/1817626768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1708165d90 con 0x7f170815f000 2024-09-13T08:48:56.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f170d000640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f170815f000 0x7f1708163c20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f17080d00b0 tx=0x7f170000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:56.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f1704c00640 1 -- 172.21.15.44:0/1817626768 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700003ba0 con 0x7f170815f000 2024-09-13T08:48:56.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f1704c00640 1 -- 172.21.15.44:0/1817626768 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1700003d60 con 0x7f170815f000 2024-09-13T08:48:56.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f1704c00640 1 -- 172.21.15.44:0/1817626768 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700005330 con 0x7f170815f000 2024-09-13T08:48:56.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f170da00640 1 -- 172.21.15.44:0/1817626768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1708166070 con 0x7f170815f000 2024-09-13T08:48:56.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f170da00640 1 -- 172.21.15.44:0/1817626768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1708164160 con 0x7f170815f000 2024-09-13T08:48:56.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.163+0000 7f170da00640 1 -- 172.21.15.44:0/1817626768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17080cc5b0 con 0x7f170815f000 2024-09-13T08:48:56.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.167+0000 7f1704c00640 1 -- 172.21.15.44:0/1817626768 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7f17000054d0 con 0x7f170815f000 2024-09-13T08:48:56.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.167+0000 7f1704c00640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16e4074420 0x7f16e40768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.167+0000 7f1704c00640 1 -- 172.21.15.44:0/1817626768 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f1700092af0 con 0x7f170815f000 2024-09-13T08:48:56.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.167+0000 7f1707e00640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16e4074420 0x7f16e40768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.167+0000 7f1707e00640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16e4074420 0x7f16e40768e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f16f801a3d0 tx=0x7f16f80b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:56.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.171+0000 7f1704c00640 1 -- 172.21.15.44:0/1817626768 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f17080cf400 con 0x7f170815f000 2024-09-13T08:48:56.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.367+0000 7f170da00640 1 -- 172.21.15.44:0/1817626768 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f17080b78e0 con 0x7f16e4074420 2024-09-13T08:48:56.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f1704c00640 1 -- 172.21.15.44:0/1817626768 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f16f80042a0 con 0x7f16e4074420 2024-09-13T08:48:56.373 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:48:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:48:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:48:56.374 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:48:56.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 -- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16e4074420 msgr2=0x7f16e40768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16e4074420 0x7f16e40768e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f16f801a3d0 tx=0x7f16f80b3040 comp rx=0 tx=0).stop 2024-09-13T08:48:56.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 -- 172.21.15.44:0/1817626768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f170815f000 msgr2=0x7f1708163c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f170815f000 0x7f1708163c20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f17080d00b0 tx=0x7f170000cb70 comp rx=0 tx=0).stop 2024-09-13T08:48:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 -- 172.21.15.44:0/1817626768 shutdown_connections 2024-09-13T08:48:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f16e4074420 0x7f16e40768e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f170815f000 0x7f1708163c20 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17080cefb0 0x7f17080b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 --2- 172.21.15.44:0/1817626768 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f17080ccaf0 0x7f1708165740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 -- 172.21.15.44:0/1817626768 >> 172.21.15.44:0/1817626768 conn(0x7f170809aa10 msgr2=0x7f17080af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 -- 172.21.15.44:0/1817626768 shutdown_connections 2024-09-13T08:48:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.371+0000 7f170da00640 1 -- 172.21.15.44:0/1817626768 wait complete. 2024-09-13T08:48:56.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-13T08:48:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.543+0000 7ff150e00640 1 -- 172.21.15.44:0/2131421271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1300052f0 msgr2=0x7ff13000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.543+0000 7ff150e00640 1 --2- 172.21.15.44:0/2131421271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1300052f0 0x7ff13000b9d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff144002240 tx=0x7ff14401fa20 comp rx=0 tx=0).stop 2024-09-13T08:48:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.543+0000 7ff150e00640 1 -- 172.21.15.44:0/2131421271 shutdown_connections 2024-09-13T08:48:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.543+0000 7ff150e00640 1 --2- 172.21.15.44:0/2131421271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1300052f0 0x7ff13000b9d0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.543+0000 7ff150e00640 1 -- 172.21.15.44:0/2131421271 >> 172.21.15.44:0/2131421271 conn(0x7ff14c0cfc20 msgr2=0x7ff14c0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.543+0000 7ff150e00640 1 -- 172.21.15.44:0/2131421271 shutdown_connections 2024-09-13T08:48:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.543+0000 7ff150e00640 1 -- 172.21.15.44:0/2131421271 wait complete. 2024-09-13T08:48:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 Processor -- start 2024-09-13T08:48:56.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 -- start start 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1300052f0 0x7ff14c0adad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff14c0ae4b0 0x7ff14c0acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff14c1f48b0 0x7ff14c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c072970 con 0x7ff14c1f48b0 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c0cbed0 con 0x7ff1300052f0 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff14c072af0 con 0x7ff14c0ae4b0 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff14a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff14c1f48b0 0x7ff14c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff14a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff14c1f48b0 0x7ff14c0b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41334/0 (socket says 172.21.15.44:41334) 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff14a800640 1 -- 172.21.15.44:0/3680578162 learned_addr learned my addr 172.21.15.44:0/3680578162 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff149e00640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1300052f0 0x7ff14c0adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff141400640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff14c0ae4b0 0x7ff14c0acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff14a800640 1 -- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff14c0ae4b0 msgr2=0x7ff14c0acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff14a800640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff14c0ae4b0 0x7ff14c0acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff14a800640 1 -- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1300052f0 msgr2=0x7ff14c0adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff14a800640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1300052f0 0x7ff14c0adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff14a800640 1 -- 172.21.15.44:0/3680578162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14c0ae010 con 0x7ff14c1f48b0 2024-09-13T08:48:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff14a800640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff14c1f48b0 0x7ff14c0b08b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff144006040 tx=0x7ff1440a9c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff142a00640 1 -- 172.21.15.44:0/3680578162 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff144004e60 con 0x7ff14c1f48b0 2024-09-13T08:48:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff142a00640 1 -- 172.21.15.44:0/3680578162 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff14400a3e0 con 0x7ff14c1f48b0 2024-09-13T08:48:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff142a00640 1 -- 172.21.15.44:0/3680578162 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff144001e30 con 0x7ff14c1f48b0 2024-09-13T08:48:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 -- 172.21.15.44:0/3680578162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14c1f4db0 con 0x7ff14c1f48b0 2024-09-13T08:48:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 -- 172.21.15.44:0/3680578162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14c1f5370 con 0x7ff14c1f48b0 2024-09-13T08:48:56.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.547+0000 7ff150e00640 1 -- 172.21.15.44:0/3680578162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14c15cfe0 con 0x7ff14c1f48b0 2024-09-13T08:48:56.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.551+0000 7ff142a00640 1 -- 172.21.15.44:0/3680578162 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7ff1440bc060 con 0x7ff14c1f48b0 2024-09-13T08:48:56.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.551+0000 7ff142a00640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff128074420 0x7ff1280768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.551+0000 7ff142a00640 1 -- 172.21.15.44:0/3680578162 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7ff14412f0c0 con 0x7ff14c1f48b0 2024-09-13T08:48:56.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.551+0000 7ff149e00640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff128074420 0x7ff1280768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.551+0000 7ff149e00640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff128074420 0x7ff1280768e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009be0 tx=0x7ff13c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.555+0000 7ff142a00640 1 -- 172.21.15.44:0/3680578162 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff14c15d310 con 0x7ff14c1f48b0 2024-09-13T08:48:56.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.755+0000 7ff150e00640 1 -- 172.21.15.44:0/3680578162 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff14c074010 con 0x7ff128074420 2024-09-13T08:48:56.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff142a00640 1 -- 172.21.15.44:0/3680578162 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1024 (secure 0 0 0) 0x7ff14c074010 con 0x7ff128074420 2024-09-13T08:48:56.777 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:48:56.777 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:48:56.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.777 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone945516" 2024-09-13T08:48:56.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.777 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone649610" 2024-09-13T08:48:56.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone25122" 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone71215" 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sv1" 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone680213" 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone6700" 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone5221" 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone25384" 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone453217" 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone74089" 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone855814" 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone300218" 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone632019" 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone8848" 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone58715" 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone73477" 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone268612" 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone677511" 2024-09-13T08:48:56.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.782 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone11943" 2024-09-13T08:48:56.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:48:56.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:56.782 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone42326" 2024-09-13T08:48:56.782 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:48:56.782 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:48:56.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 -- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff128074420 msgr2=0x7ff1280768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff128074420 0x7ff1280768e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009be0 tx=0x7ff13c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 -- 172.21.15.44:0/3680578162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff14c1f48b0 msgr2=0x7ff14c0b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff14c1f48b0 0x7ff14c0b08b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff144006040 tx=0x7ff1440a9c50 comp rx=0 tx=0).stop 2024-09-13T08:48:56.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 -- 172.21.15.44:0/3680578162 shutdown_connections 2024-09-13T08:48:56.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff128074420 0x7ff1280768e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff14c1f48b0 0x7ff14c0b08b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff14c0ae4b0 0x7ff14c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 --2- 172.21.15.44:0/3680578162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1300052f0 0x7ff14c0adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 -- 172.21.15.44:0/3680578162 >> 172.21.15.44:0/3680578162 conn(0x7ff14c0cfc20 msgr2=0x7ff14c0d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:56.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 -- 172.21.15.44:0/3680578162 shutdown_connections 2024-09-13T08:48:56.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.775+0000 7ff150e00640 1 -- 172.21.15.44:0/3680578162 wait complete. 2024-09-13T08:48:56.798 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone945516 2024-09-13T08:48:56.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.947+0000 7ff269000640 1 -- 172.21.15.44:0/1916062606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2640d28e0 msgr2=0x7ff2640d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.947+0000 7ff269000640 1 --2- 172.21.15.44:0/1916062606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2640d28e0 0x7ff2640d0170 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009840 tx=0x7ff24c01c820 comp rx=0 tx=0).stop 2024-09-13T08:48:56.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.947+0000 7ff269000640 1 -- 172.21.15.44:0/1916062606 shutdown_connections 2024-09-13T08:48:56.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.947+0000 7ff269000640 1 --2- 172.21.15.44:0/1916062606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2640d28e0 0x7ff2640d0170 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.947+0000 7ff269000640 1 --2- 172.21.15.44:0/1916062606 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff26415f390 0x7ff2640cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.947+0000 7ff269000640 1 -- 172.21.15.44:0/1916062606 >> 172.21.15.44:0/1916062606 conn(0x7ff2640d1dc0 msgr2=0x7ff2640d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:56.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.947+0000 7ff269000640 1 -- 172.21.15.44:0/1916062606 shutdown_connections 2024-09-13T08:48:56.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.947+0000 7ff269000640 1 -- 172.21.15.44:0/1916062606 wait complete. 2024-09-13T08:48:56.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 Processor -- start 2024-09-13T08:48:56.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 -- start start 2024-09-13T08:48:56.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff2640d06b0 0x7ff26414e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2640d28e0 0x7ff264149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26415f390 0x7ff26415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff264169110 con 0x7ff2640d28e0 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff264168f90 con 0x7ff26415f390 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff264169290 con 0x7ff2640d06b0 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2640d28e0 0x7ff264149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2640d28e0 0x7ff264149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41348/0 (socket says 172.21.15.44:41348) 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25be00640 1 -- 172.21.15.44:0/519362866 learned_addr learned my addr 172.21.15.44:0/519362866 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff261e00640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff2640d06b0 0x7ff26414e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff262800640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26415f390 0x7ff26415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25be00640 1 -- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff2640d06b0 msgr2=0x7ff26414e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25be00640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff2640d06b0 0x7ff26414e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25be00640 1 -- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26415f390 msgr2=0x7ff26415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:56.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25be00640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26415f390 0x7ff26415b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:56.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25be00640 1 -- 172.21.15.44:0/519362866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26414a110 con 0x7ff2640d28e0 2024-09-13T08:48:56.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25be00640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2640d28e0 0x7ff264149bd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009790 tx=0x7ff24c0a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:56.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25a000640 1 -- 172.21.15.44:0/519362866 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c0b2040 con 0x7ff2640d28e0 2024-09-13T08:48:56.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25a000640 1 -- 172.21.15.44:0/519362866 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff24c0a5d10 con 0x7ff2640d28e0 2024-09-13T08:48:56.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff25a000640 1 -- 172.21.15.44:0/519362866 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c005280 con 0x7ff2640d28e0 2024-09-13T08:48:56.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 -- 172.21.15.44:0/519362866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2641500d0 con 0x7ff2640d28e0 2024-09-13T08:48:56.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 -- 172.21.15.44:0/519362866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff264151b10 con 0x7ff2640d28e0 2024-09-13T08:48:56.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.951+0000 7ff269000640 1 -- 172.21.15.44:0/519362866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2640b1350 con 0x7ff2640d28e0 2024-09-13T08:48:56.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.955+0000 7ff25a000640 1 -- 172.21.15.44:0/519362866 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7ff24c0be060 con 0x7ff2640d28e0 2024-09-13T08:48:56.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.955+0000 7ff25a000640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff240074450 0x7ff240076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:56.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.955+0000 7ff25a000640 1 -- 172.21.15.44:0/519362866 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7ff24c12c860 con 0x7ff2640d28e0 2024-09-13T08:48:56.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.955+0000 7ff261e00640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff240074450 0x7ff240076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:56.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.955+0000 7ff261e00640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff240074450 0x7ff240076910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff264153ce0 tx=0x7ff25400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:56.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:56.959+0000 7ff25a000640 1 -- 172.21.15.44:0/519362866 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff24c016410 con 0x7ff2640d28e0 2024-09-13T08:48:57.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.159+0000 7ff269000640 1 -- 172.21.15.44:0/519362866 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone945516", "target": ["mon-mgr", ""]}) -- 0x7ff26415d1e0 con 0x7ff240074450 2024-09-13T08:48:57.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff25a000640 1 -- 172.21.15.44:0/519362866 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff26415d1e0 con 0x7ff240074450 2024-09-13T08:48:57.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 -- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff240074450 msgr2=0x7ff240076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:57.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff240074450 0x7ff240076910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff264153ce0 tx=0x7ff25400b2e0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 -- 172.21.15.44:0/519362866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2640d28e0 msgr2=0x7ff264149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:57.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2640d28e0 0x7ff264149bd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009790 tx=0x7ff24c0a5670 comp rx=0 tx=0).stop 2024-09-13T08:48:57.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 -- 172.21.15.44:0/519362866 shutdown_connections 2024-09-13T08:48:57.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff240074450 0x7ff240076910 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26415f390 0x7ff26415b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2640d28e0 0x7ff264149bd0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 --2- 172.21.15.44:0/519362866 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff2640d06b0 0x7ff26414e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 -- 172.21.15.44:0/519362866 >> 172.21.15.44:0/519362866 conn(0x7ff2640d1dc0 msgr2=0x7ff264071e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:57.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 -- 172.21.15.44:0/519362866 shutdown_connections 2024-09-13T08:48:57.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.187+0000 7ff269000640 1 -- 172.21.15.44:0/519362866 wait complete. 2024-09-13T08:48:57.192 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:57.194+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:57.192 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:57.194+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:57.192 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:57.194+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:57.192 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:57.194+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:57.192 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:57.194+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:57.212 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone649610 2024-09-13T08:48:57.212 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:57.210+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:57.213 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:57.210+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:57.213 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:57.210+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:57.213 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:57.210+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:57.213 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:48:57.210+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:48:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.363+0000 7f6719a00640 1 -- 172.21.15.44:0/1612326402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67140b1950 msgr2=0x7f67140cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.363+0000 7f6719a00640 1 --2- 172.21.15.44:0/1612326402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67140b1950 0x7f67140cd7c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009840 tx=0x7f66fc01c6a0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.363+0000 7f6719a00640 1 -- 172.21.15.44:0/1612326402 shutdown_connections 2024-09-13T08:48:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.363+0000 7f6719a00640 1 --2- 172.21.15.44:0/1612326402 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f671415e660 0x7f67140cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.363+0000 7f6719a00640 1 --2- 172.21.15.44:0/1612326402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67140b1950 0x7f67140cd7c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.363+0000 7f6719a00640 1 -- 172.21.15.44:0/1612326402 >> 172.21.15.44:0/1612326402 conn(0x7f6714073540 msgr2=0x7f67140b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.363+0000 7f6719a00640 1 -- 172.21.15.44:0/1612326402 shutdown_connections 2024-09-13T08:48:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.363+0000 7f6719a00640 1 -- 172.21.15.44:0/1612326402 wait complete. 2024-09-13T08:48:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 Processor -- start 2024-09-13T08:48:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 -- start start 2024-09-13T08:48:57.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67140b1950 0x7f67140c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:57.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f671415a0f0 0x7f67140b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:57.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f671415e660 0x7f67140aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:57.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f67140d0920 con 0x7f671415a0f0 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67140d07a0 con 0x7f67140b1950 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f67140d0aa0 con 0x7f671415e660 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f670be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f671415a0f0 0x7f67140b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f670be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f671415a0f0 0x7f67140b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41364/0 (socket says 172.21.15.44:41364) 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f670be00640 1 -- 172.21.15.44:0/2447708260 learned_addr learned my addr 172.21.15.44:0/2447708260 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6713e00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67140b1950 0x7f67140c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719000640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f671415e660 0x7f67140aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f670be00640 1 -- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f671415e660 msgr2=0x7f67140aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f670be00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f671415e660 0x7f67140aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f670be00640 1 -- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67140b1950 msgr2=0x7f67140c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f670be00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67140b1950 0x7f67140c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f670be00640 1 -- 172.21.15.44:0/2447708260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67140c74a0 con 0x7f671415a0f0 2024-09-13T08:48:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f670be00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f671415a0f0 0x7f67140b3460 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f66fc01cb80 tx=0x7f66fc0a8690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6710c00640 1 -- 172.21.15.44:0/2447708260 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0bf070 con 0x7f671415a0f0 2024-09-13T08:48:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6710c00640 1 -- 172.21.15.44:0/2447708260 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66fc005250 con 0x7f671415a0f0 2024-09-13T08:48:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6710c00640 1 -- 172.21.15.44:0/2447708260 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0046d0 con 0x7f671415a0f0 2024-09-13T08:48:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 -- 172.21.15.44:0/2447708260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67140c7780 con 0x7f671415a0f0 2024-09-13T08:48:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 -- 172.21.15.44:0/2447708260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67140c7d40 con 0x7f671415a0f0 2024-09-13T08:48:57.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.367+0000 7f6719a00640 1 -- 172.21.15.44:0/2447708260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67140cd7c0 con 0x7f671415a0f0 2024-09-13T08:48:57.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.371+0000 7f6710c00640 1 -- 172.21.15.44:0/2447708260 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7f66fc0c2070 con 0x7f671415a0f0 2024-09-13T08:48:57.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.371+0000 7f6710c00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66f0074450 0x7f66f0076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:57.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.371+0000 7f6710c00640 1 -- 172.21.15.44:0/2447708260 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f66fc12d160 con 0x7f671415a0f0 2024-09-13T08:48:57.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.371+0000 7f6713e00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66f0074450 0x7f66f0076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:57.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.371+0000 7f6713e00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66f0074450 0x7f66f0076910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6704001910 tx=0x7f6704001420 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.375+0000 7f6710c00640 1 -- 172.21.15.44:0/2447708260 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f66fc0ac030 con 0x7f671415a0f0 2024-09-13T08:48:57.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.575+0000 7f6719a00640 1 -- 172.21.15.44:0/2447708260 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone649610", "target": ["mon-mgr", ""]}) -- 0x7f67140d1140 con 0x7f66f0074450 2024-09-13T08:48:57.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.595+0000 7f6710c00640 1 -- 172.21.15.44:0/2447708260 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f67140d1140 con 0x7f66f0074450 2024-09-13T08:48:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 -- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66f0074450 msgr2=0x7f66f0076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66f0074450 0x7f66f0076910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6704001910 tx=0x7f6704001420 comp rx=0 tx=0).stop 2024-09-13T08:48:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 -- 172.21.15.44:0/2447708260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f671415a0f0 msgr2=0x7f67140b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f671415a0f0 0x7f67140b3460 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f66fc01cb80 tx=0x7f66fc0a8690 comp rx=0 tx=0).stop 2024-09-13T08:48:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 -- 172.21.15.44:0/2447708260 shutdown_connections 2024-09-13T08:48:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66f0074450 0x7f66f0076910 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f671415e660 0x7f67140aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f671415a0f0 0x7f67140b3460 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 --2- 172.21.15.44:0/2447708260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67140b1950 0x7f67140c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 -- 172.21.15.44:0/2447708260 >> 172.21.15.44:0/2447708260 conn(0x7f6714073540 msgr2=0x7f67140caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 -- 172.21.15.44:0/2447708260 shutdown_connections 2024-09-13T08:48:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.599+0000 7f6719a00640 1 -- 172.21.15.44:0/2447708260 wait complete. 2024-09-13T08:48:57.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone25122 2024-09-13T08:48:57.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 -- 172.21.15.44:0/3213865840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6415be10 msgr2=0x7f7d640ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:57.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 --2- 172.21.15.44:0/3213865840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6415be10 0x7f7d640ce700 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7d640b0fb0 tx=0x7f7d5401f0b0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 -- 172.21.15.44:0/3213865840 shutdown_connections 2024-09-13T08:48:57.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 --2- 172.21.15.44:0/3213865840 >> v2:172.21.15.129:3300/0 conn(0x7f7d640b0b60 0x7f7d640ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 --2- 172.21.15.44:0/3213865840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6415be10 0x7f7d640ce700 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 -- 172.21.15.44:0/3213865840 >> 172.21.15.44:0/3213865840 conn(0x7f7d640d1da0 msgr2=0x7f7d640d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:57.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 -- 172.21.15.44:0/3213865840 shutdown_connections 2024-09-13T08:48:57.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 -- 172.21.15.44:0/3213865840 wait complete. 2024-09-13T08:48:57.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 Processor -- start 2024-09-13T08:48:57.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 -- start start 2024-09-13T08:48:57.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d640b0b60 0x7f7d64150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:57.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6415be10 0x7f7d641f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:57.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6415c1f0 0x7f7d640cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:57.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d640cd4a0 con 0x7f7d6415be10 2024-09-13T08:48:57.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d640cd320 con 0x7f7d6415c1f0 2024-09-13T08:48:57.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.771+0000 7f7d69200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7d640cd620 con 0x7f7d640b0b60 2024-09-13T08:48:57.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d59400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6415be10 0x7f7d641f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:57.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d61e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d640b0b60 0x7f7d64150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:57.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d62800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6415c1f0 0x7f7d640cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:57.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d61e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d640b0b60 0x7f7d64150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:57628/0 (socket says 172.21.15.44:57628) 2024-09-13T08:48:57.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d59400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6415be10 0x7f7d641f55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41370/0 (socket says 172.21.15.44:41370) 2024-09-13T08:48:57.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d61e00640 1 -- 172.21.15.44:0/505890995 learned_addr learned my addr 172.21.15.44:0/505890995 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:57.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d61e00640 1 -- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6415c1f0 msgr2=0x7f7d640cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:57.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d61e00640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6415c1f0 0x7f7d640cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d61e00640 1 -- 172.21.15.44:0/505890995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6415be10 msgr2=0x7f7d641f55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:57.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d61e00640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6415be10 0x7f7d641f55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:57.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d61e00640 1 -- 172.21.15.44:0/505890995 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d641f5ae0 con 0x7f7d640b0b60 2024-09-13T08:48:57.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d59400640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6415be10 0x7f7d641f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:48:57.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d61e00640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d640b0b60 0x7f7d64150210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7d5401bcc0 tx=0x7f7d54005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:57.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d5aa00640 1 -- 172.21.15.44:0/505890995 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d540bf070 con 0x7f7d640b0b60 2024-09-13T08:48:57.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d5aa00640 1 -- 172.21.15.44:0/505890995 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d540b08e0 con 0x7f7d640b0b60 2024-09-13T08:48:57.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d69200640 1 -- 172.21.15.44:0/505890995 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d640ca220 con 0x7f7d640b0b60 2024-09-13T08:48:57.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d5aa00640 1 -- 172.21.15.44:0/505890995 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d540b5040 con 0x7f7d640b0b60 2024-09-13T08:48:57.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d69200640 1 -- 172.21.15.44:0/505890995 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d640ca7e0 con 0x7f7d640b0b60 2024-09-13T08:48:57.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d69200640 1 -- 172.21.15.44:0/505890995 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d640cdb40 con 0x7f7d640b0b60 2024-09-13T08:48:57.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.775+0000 7f7d5aa00640 1 -- 172.21.15.44:0/505890995 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7f7d540c2070 con 0x7f7d640b0b60 2024-09-13T08:48:57.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.779+0000 7f7d5aa00640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d40074420 0x7f7d400768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:57.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.779+0000 7f7d5aa00640 1 -- 172.21.15.44:0/505890995 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f7d5412fbf0 con 0x7f7d640b0b60 2024-09-13T08:48:57.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.779+0000 7f7d59400640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d40074420 0x7f7d400768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:57.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.779+0000 7f7d59400640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d40074420 0x7f7d400768e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7d640b4f40 tx=0x7f7d4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:57.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.783+0000 7f7d5aa00640 1 -- 172.21.15.44:0/505890995 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7d641f5ae0 con 0x7f7d640b0b60 2024-09-13T08:48:57.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:57.987+0000 7f7d69200640 1 -- 172.21.15.44:0/505890995 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone25122", "target": ["mon-mgr", ""]}) -- 0x7f7d64150750 con 0x7f7d40074420 2024-09-13T08:48:58.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.003+0000 7f7d5aa00640 1 -- 172.21.15.44:0/505890995 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7d4c001c80 con 0x7f7d40074420 2024-09-13T08:48:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 -- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d40074420 msgr2=0x7f7d400768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d40074420 0x7f7d400768e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7d640b4f40 tx=0x7f7d4c009040 comp rx=0 tx=0).stop 2024-09-13T08:48:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 -- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d640b0b60 msgr2=0x7f7d64150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d640b0b60 0x7f7d64150210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7d5401bcc0 tx=0x7f7d54005e20 comp rx=0 tx=0).stop 2024-09-13T08:48:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 -- 172.21.15.44:0/505890995 shutdown_connections 2024-09-13T08:48:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d40074420 0x7f7d400768e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6415c1f0 0x7f7d640cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6415be10 0x7f7d641f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 --2- 172.21.15.44:0/505890995 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7d640b0b60 0x7f7d64150210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 -- 172.21.15.44:0/505890995 >> 172.21.15.44:0/505890995 conn(0x7f7d640d1da0 msgr2=0x7f7d64071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:58.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 -- 172.21.15.44:0/505890995 shutdown_connections 2024-09-13T08:48:58.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.007+0000 7f7d69200640 1 -- 172.21.15.44:0/505890995 wait complete. 2024-09-13T08:48:58.030 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone71215 2024-09-13T08:48:58.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.179+0000 7fec2f200640 1 -- 172.21.15.44:0/4283622661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec30073ea0 msgr2=0x7fec3015bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.179+0000 7fec2f200640 1 --2- 172.21.15.44:0/4283622661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec30073ea0 0x7fec3015bc60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fec18009840 tx=0x7fec1801f010 comp rx=0 tx=0).stop 2024-09-13T08:48:58.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.179+0000 7fec2f200640 1 -- 172.21.15.44:0/4283622661 shutdown_connections 2024-09-13T08:48:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.179+0000 7fec2f200640 1 --2- 172.21.15.44:0/4283622661 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec300cb860 0x7fec300ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.179+0000 7fec2f200640 1 --2- 172.21.15.44:0/4283622661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec30073ea0 0x7fec3015bc60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.179+0000 7fec2f200640 1 -- 172.21.15.44:0/4283622661 >> 172.21.15.44:0/4283622661 conn(0x7fec300d0b70 msgr2=0x7fec300d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.179+0000 7fec2f200640 1 -- 172.21.15.44:0/4283622661 shutdown_connections 2024-09-13T08:48:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.179+0000 7fec2f200640 1 -- 172.21.15.44:0/4283622661 wait complete. 2024-09-13T08:48:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2f200640 1 Processor -- start 2024-09-13T08:48:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2f200640 1 -- start start 2024-09-13T08:48:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec30073ea0 0x7fec30154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec300cb860 0x7fec30070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2f200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec301552d0 0x7fec30070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fec300cf280 con 0x7fec30073ea0 2024-09-13T08:48:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec300cf100 con 0x7fec300cb860 2024-09-13T08:48:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2f200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fec300cf400 con 0x7fec301552d0 2024-09-13T08:48:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec30073ea0 0x7fec30154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec30073ea0 0x7fec30154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41376/0 (socket says 172.21.15.44:41376) 2024-09-13T08:48:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2de00640 1 -- 172.21.15.44:0/3468598527 learned_addr learned my addr 172.21.15.44:0/3468598527 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:58.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec27e00640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec300cb860 0x7fec30070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:58.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2e800640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec301552d0 0x7fec30070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:58.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2de00640 1 -- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec301552d0 msgr2=0x7fec30070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2de00640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec301552d0 0x7fec30070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2de00640 1 -- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec300cb860 msgr2=0x7fec30070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2de00640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec300cb860 0x7fec30070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2de00640 1 -- 172.21.15.44:0/3468598527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec30140c30 con 0x7fec30073ea0 2024-09-13T08:48:58.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2de00640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec30073ea0 0x7fec30154110 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c430 tx=0x7fec2000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:58.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec26000640 1 -- 172.21.15.44:0/3468598527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20011790 con 0x7fec30073ea0 2024-09-13T08:48:58.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec26000640 1 -- 172.21.15.44:0/3468598527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec20017070 con 0x7fec30073ea0 2024-09-13T08:48:58.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec26000640 1 -- 172.21.15.44:0/3468598527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20011ce0 con 0x7fec30073ea0 2024-09-13T08:48:58.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2f200640 1 -- 172.21.15.44:0/3468598527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec30140f10 con 0x7fec30073ea0 2024-09-13T08:48:58.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.183+0000 7fec2f200640 1 -- 172.21.15.44:0/3468598527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec30154650 con 0x7fec30073ea0 2024-09-13T08:48:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.187+0000 7fec2f200640 1 -- 172.21.15.44:0/3468598527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec300cf990 con 0x7fec30073ea0 2024-09-13T08:48:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.187+0000 7fec26000640 1 -- 172.21.15.44:0/3468598527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7fec2001c460 con 0x7fec30073ea0 2024-09-13T08:48:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.187+0000 7fec26000640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec0c074450 0x7fec0c076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.187+0000 7fec26000640 1 -- 172.21.15.44:0/3468598527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fec20092bc0 con 0x7fec30073ea0 2024-09-13T08:48:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.187+0000 7fec27e00640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec0c074450 0x7fec0c076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:58.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.187+0000 7fec27e00640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec0c074450 0x7fec0c076910 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fec180b2000 tx=0x7fec1801d850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:58.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.191+0000 7fec26000640 1 -- 172.21.15.44:0/3468598527 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fec2005f520 con 0x7fec30073ea0 2024-09-13T08:48:58.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.395+0000 7fec2f200640 1 -- 172.21.15.44:0/3468598527 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone71215", "target": ["mon-mgr", ""]}) -- 0x7fec301409e0 con 0x7fec0c074450 2024-09-13T08:48:58.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.415+0000 7fec26000640 1 -- 172.21.15.44:0/3468598527 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fec301409e0 con 0x7fec0c074450 2024-09-13T08:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 -- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec0c074450 msgr2=0x7fec0c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec0c074450 0x7fec0c076910 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fec180b2000 tx=0x7fec1801d850 comp rx=0 tx=0).stop 2024-09-13T08:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 -- 172.21.15.44:0/3468598527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec30073ea0 msgr2=0x7fec30154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec30073ea0 0x7fec30154110 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c430 tx=0x7fec2000c830 comp rx=0 tx=0).stop 2024-09-13T08:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 -- 172.21.15.44:0/3468598527 shutdown_connections 2024-09-13T08:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec0c074450 0x7fec0c076910 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec301552d0 0x7fec30070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec300cb860 0x7fec30070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 --2- 172.21.15.44:0/3468598527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec30073ea0 0x7fec30154110 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 -- 172.21.15.44:0/3468598527 >> 172.21.15.44:0/3468598527 conn(0x7fec300d0b70 msgr2=0x7fec30159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:58.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 -- 172.21.15.44:0/3468598527 shutdown_connections 2024-09-13T08:48:58.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.419+0000 7fec2f200640 1 -- 172.21.15.44:0/3468598527 wait complete. 2024-09-13T08:48:58.448 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-13T08:48:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 -- 172.21.15.44:0/2932485972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff58c003a60 msgr2=0x7ff58c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 --2- 172.21.15.44:0/2932485972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff58c003a60 0x7ff58c003e80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c260 tx=0x7ff59801ced0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 -- 172.21.15.44:0/2932485972 shutdown_connections 2024-09-13T08:48:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 --2- 172.21.15.44:0/2932485972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff58c003a60 0x7ff58c003e80 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 -- 172.21.15.44:0/2932485972 >> 172.21.15.44:0/2932485972 conn(0x7ff5a80d1dc0 msgr2=0x7ff5a80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:58.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 -- 172.21.15.44:0/2932485972 shutdown_connections 2024-09-13T08:48:58.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 -- 172.21.15.44:0/2932485972 wait complete. 2024-09-13T08:48:58.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 Processor -- start 2024-09-13T08:48:58.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 -- start start 2024-09-13T08:48:58.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff58c003a60 0x7ff5a815e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:58.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a81eb230 0x7ff5a81fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:58.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5a815d9a0 0x7ff5a815de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:58.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a80d09e0 con 0x7ff5a81eb230 2024-09-13T08:48:58.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a80d0860 con 0x7ff5a815d9a0 2024-09-13T08:48:58.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5ae200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a80b1350 con 0x7ff58c003a60 2024-09-13T08:48:58.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a81eb230 0x7ff5a81fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:58.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a81eb230 0x7ff5a81fb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41396/0 (socket says 172.21.15.44:41396) 2024-09-13T08:48:58.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a6a00640 1 -- 172.21.15.44:0/417179930 learned_addr learned my addr 172.21.15.44:0/417179930 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:58.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a7400640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff58c003a60 0x7ff5a815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:58.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a7e00640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5a815d9a0 0x7ff5a815de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:58.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a6a00640 1 -- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff58c003a60 msgr2=0x7ff5a815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a6a00640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff58c003a60 0x7ff5a815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a6a00640 1 -- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5a815d9a0 msgr2=0x7ff5a815de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a6a00640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5a815d9a0 0x7ff5a815de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a6a00640 1 -- 172.21.15.44:0/417179930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a8154c60 con 0x7ff5a81eb230 2024-09-13T08:48:58.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.599+0000 7ff5a6a00640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a81eb230 0x7ff5a81fb6c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff59400b5f0 tx=0x7ff59400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:58.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.603+0000 7ff59f400640 1 -- 172.21.15.44:0/417179930 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594005220 con 0x7ff5a81eb230 2024-09-13T08:48:58.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.603+0000 7ff59f400640 1 -- 172.21.15.44:0/417179930 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5940053c0 con 0x7ff5a81eb230 2024-09-13T08:48:58.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.603+0000 7ff5ae200640 1 -- 172.21.15.44:0/417179930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a81eb7f0 con 0x7ff5a81eb230 2024-09-13T08:48:58.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.603+0000 7ff59f400640 1 -- 172.21.15.44:0/417179930 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5940047b0 con 0x7ff5a81eb230 2024-09-13T08:48:58.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.603+0000 7ff5ae200640 1 -- 172.21.15.44:0/417179930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a81f7c00 con 0x7ff5a81eb230 2024-09-13T08:48:58.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.603+0000 7ff5ae200640 1 -- 172.21.15.44:0/417179930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a80b18b0 con 0x7ff5a81eb230 2024-09-13T08:48:58.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.603+0000 7ff59f400640 1 -- 172.21.15.44:0/417179930 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7ff594004950 con 0x7ff5a81eb230 2024-09-13T08:48:58.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.603+0000 7ff59f400640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff588074420 0x7ff5880768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:58.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.603+0000 7ff59f400640 1 -- 172.21.15.44:0/417179930 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7ff5940934e0 con 0x7ff5a81eb230 2024-09-13T08:48:58.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.603+0000 7ff5a7400640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff588074420 0x7ff5880768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:58.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.607+0000 7ff5a7400640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff588074420 0x7ff5880768e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff5980ab000 tx=0x7ff598019d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:58.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.607+0000 7ff59f400640 1 -- 172.21.15.44:0/417179930 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff59405fb30 con 0x7ff5a81eb230 2024-09-13T08:48:58.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.815+0000 7ff5ae200640 1 -- 172.21.15.44:0/417179930 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5a80b0570 con 0x7ff588074420 2024-09-13T08:48:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.827+0000 7ff59f400640 1 -- 172.21.15.44:0/417179930 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff5a80b0570 con 0x7ff588074420 2024-09-13T08:48:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:48:58.830 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:48:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:48:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1" 2024-09-13T08:48:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:48:58.830 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:48:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 -- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff588074420 msgr2=0x7ff5880768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff588074420 0x7ff5880768e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff5980ab000 tx=0x7ff598019d10 comp rx=0 tx=0).stop 2024-09-13T08:48:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 -- 172.21.15.44:0/417179930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a81eb230 msgr2=0x7ff5a81fb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a81eb230 0x7ff5a81fb6c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff59400b5f0 tx=0x7ff59400bab0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 -- 172.21.15.44:0/417179930 shutdown_connections 2024-09-13T08:48:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff588074420 0x7ff5880768e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5a815d9a0 0x7ff5a815de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a81eb230 0x7ff5a81fb6c0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 --2- 172.21.15.44:0/417179930 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff58c003a60 0x7ff5a815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 -- 172.21.15.44:0/417179930 >> 172.21.15.44:0/417179930 conn(0x7ff5a80d1dc0 msgr2=0x7ff5a81599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 -- 172.21.15.44:0/417179930 shutdown_connections 2024-09-13T08:48:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:58.831+0000 7ff5ae200640 1 -- 172.21.15.44:0/417179930 wait complete. 2024-09-13T08:48:58.851 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-13T08:48:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.003+0000 7f8790a00640 1 -- 172.21.15.44:0/3851113676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8770004da0 msgr2=0x7f8770005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.003+0000 7f8790a00640 1 --2- 172.21.15.44:0/3851113676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8770004da0 0x7f8770005220 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8784009e00 tx=0x7f878401d130 comp rx=0 tx=0).stop 2024-09-13T08:48:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.003+0000 7f878a800640 1 -- 172.21.15.44:0/3851113676 reap_dead start 2024-09-13T08:48:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.003+0000 7f8790a00640 1 -- 172.21.15.44:0/3851113676 shutdown_connections 2024-09-13T08:48:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.003+0000 7f8790a00640 1 -- 172.21.15.44:0/3851113676 >> 172.21.15.44:0/3851113676 conn(0x7f878c0d1dd0 msgr2=0x7f878c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.003+0000 7f8790a00640 1 -- 172.21.15.44:0/3851113676 shutdown_connections 2024-09-13T08:48:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.003+0000 7f8790a00640 1 -- 172.21.15.44:0/3851113676 wait complete. 2024-09-13T08:48:59.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.003+0000 7f8790a00640 1 Processor -- start 2024-09-13T08:48:59.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.003+0000 7f8790a00640 1 -- start start 2024-09-13T08:48:59.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8790a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8770004da0 0x7f878c1f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:59.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8790a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f878c0b6b60 0x7f878c06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:59.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8790a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c06f520 0x7f878c0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:59.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8790a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f878c0b2c80 con 0x7f878c06f520 2024-09-13T08:48:59.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8790a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f878c0b0ee0 con 0x7f8770004da0 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8790a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f878c0b1220 con 0x7f878c0b6b60 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f878a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c06f520 0x7f878c0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f878a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c06f520 0x7f878c0cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41414/0 (socket says 172.21.15.44:41414) 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f878a800640 1 -- 172.21.15.44:0/1352053497 learned_addr learned my addr 172.21.15.44:0/1352053497 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8789e00640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8770004da0 0x7f878c1f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8781400640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f878c0b6b60 0x7f878c06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f878a800640 1 -- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f878c0b6b60 msgr2=0x7f878c06efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f878a800640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f878c0b6b60 0x7f878c06efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f878a800640 1 -- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8770004da0 msgr2=0x7f878c1f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f878a800640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8770004da0 0x7f878c1f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f878a800640 1 -- 172.21.15.44:0/1352053497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c06ec20 con 0x7f878c06f520 2024-09-13T08:48:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f878a800640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c06f520 0x7f878c0cdec0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f87840af040 tx=0x7f87840addc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8782a00640 1 -- 172.21.15.44:0/1352053497 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87840b1040 con 0x7f878c06f520 2024-09-13T08:48:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8782a00640 1 -- 172.21.15.44:0/1352053497 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87840bc070 con 0x7f878c06f520 2024-09-13T08:48:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8782a00640 1 -- 172.21.15.44:0/1352053497 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87840ade30 con 0x7f878c06f520 2024-09-13T08:48:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8790a00640 1 -- 172.21.15.44:0/1352053497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c06dd30 con 0x7f878c06f520 2024-09-13T08:48:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8790a00640 1 -- 172.21.15.44:0/1352053497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c06e2f0 con 0x7f878c06f520 2024-09-13T08:48:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.007+0000 7f8790a00640 1 -- 172.21.15.44:0/1352053497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c0b1a30 con 0x7f878c06f520 2024-09-13T08:48:59.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.011+0000 7f8782a00640 1 -- 172.21.15.44:0/1352053497 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 99018+0+0 (secure 0 0 0) 0x7f8784004a50 con 0x7f878c06f520 2024-09-13T08:48:59.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.011+0000 7f8782a00640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8768074420 0x7f87680768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:59.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.011+0000 7f8782a00640 1 -- 172.21.15.44:0/1352053497 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f878412dbc0 con 0x7f878c06f520 2024-09-13T08:48:59.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.011+0000 7f8789e00640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8768074420 0x7f87680768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:59.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.011+0000 7f8789e00640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8768074420 0x7f87680768e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f878c06ef70 tx=0x7f877c0067b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.015+0000 7f8782a00640 1 -- 172.21.15.44:0/1352053497 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f87840a5e20 con 0x7f878c06f520 2024-09-13T08:48:59.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.091+0000 7f8782a00640 1 -- 172.21.15.44:0/1352053497 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f87840f5ff0 con 0x7f878c06f520 2024-09-13T08:48:59.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.223+0000 7f8790a00640 1 -- 172.21.15.44:0/1352053497 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f878c0d1080 con 0x7f8768074420 2024-09-13T08:48:59.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8782a00640 1 -- 172.21.15.44:0/1352053497 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f878c0d1080 con 0x7f8768074420 2024-09-13T08:48:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:48:59.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 -- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8768074420 msgr2=0x7f87680768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:59.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8768074420 0x7f87680768e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f878c06ef70 tx=0x7f877c0067b0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 -- 172.21.15.44:0/1352053497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c06f520 msgr2=0x7f878c0cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:59.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c06f520 0x7f878c0cdec0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f87840af040 tx=0x7f87840addc0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 -- 172.21.15.44:0/1352053497 shutdown_connections 2024-09-13T08:48:59.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8768074420 0x7f87680768e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c06f520 0x7f878c0cdec0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f878c0b6b60 0x7f878c06efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 --2- 172.21.15.44:0/1352053497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8770004da0 0x7f878c1f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 -- 172.21.15.44:0/1352053497 >> 172.21.15.44:0/1352053497 conn(0x7f878c0d1dd0 msgr2=0x7f878c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:59.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 -- 172.21.15.44:0/1352053497 shutdown_connections 2024-09-13T08:48:59.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.255+0000 7f8790a00640 1 -- 172.21.15.44:0/1352053497 wait complete. 2024-09-13T08:48:59.279 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-13T08:48:59.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.451+0000 7f8ed9400640 1 -- 172.21.15.44:0/3459898054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ed40b31c0 msgr2=0x7f8ed40b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:59.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.451+0000 7f8ed9400640 1 --2- 172.21.15.44:0/3459898054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ed40b31c0 0x7f8ed40b3650 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8ed406f8e0 tx=0x7f8ecc01d820 comp rx=0 tx=0).stop 2024-09-13T08:48:59.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.451+0000 7f8ed9400640 1 -- 172.21.15.44:0/3459898054 shutdown_connections 2024-09-13T08:48:59.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.451+0000 7f8ed9400640 1 --2- 172.21.15.44:0/3459898054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ed40b31c0 0x7f8ed40b3650 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.451+0000 7f8ed9400640 1 --2- 172.21.15.44:0/3459898054 >> v2:172.21.15.129:3300/0 conn(0x7f8ed4150d80 0x7f8ed4073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.451+0000 7f8ed9400640 1 -- 172.21.15.44:0/3459898054 >> 172.21.15.44:0/3459898054 conn(0x7f8ed406ecd0 msgr2=0x7f8ed406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:59.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.451+0000 7f8ed9400640 1 -- 172.21.15.44:0/3459898054 shutdown_connections 2024-09-13T08:48:59.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.451+0000 7f8ed9400640 1 -- 172.21.15.44:0/3459898054 wait complete. 2024-09-13T08:48:59.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed9400640 1 Processor -- start 2024-09-13T08:48:59.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed9400640 1 -- start start 2024-09-13T08:48:59.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ed40b31c0 0x7f8ed4147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:59.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ed4150d80 0x7f8ed41f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:59.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed2800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ed40b31c0 0x7f8ed4147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:59.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed2800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ed40b31c0 0x7f8ed4147420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:58766/0 (socket says 172.21.15.44:58766) 2024-09-13T08:48:59.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ed4150d80 0x7f8ed41f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:59.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ed4150d80 0x7f8ed41f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41430/0 (socket says 172.21.15.44:41430) 2024-09-13T08:48:59.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed9400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ed4151160 0x7f8ed4132a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:59.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed9400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4167020 con 0x7f8ed4150d80 2024-09-13T08:48:59.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed9400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4166ea0 con 0x7f8ed40b31c0 2024-09-13T08:48:59.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed9400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed41671a0 con 0x7f8ed4151160 2024-09-13T08:48:59.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed2800640 1 -- 172.21.15.44:0/1117891168 learned_addr learned my addr 172.21.15.44:0/1117891168 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:48:59.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.455+0000 7f8ed3200640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ed4151160 0x7f8ed4132a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:59.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.459+0000 7f8ed2800640 1 -- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ed4151160 msgr2=0x7f8ed4132a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:59.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.459+0000 7f8ed2800640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ed4151160 0x7f8ed4132a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.459+0000 7f8ed2800640 1 -- 172.21.15.44:0/1117891168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ed4150d80 msgr2=0x7f8ed41f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:59.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.459+0000 7f8ed2800640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ed4150d80 0x7f8ed41f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.459+0000 7f8ed2800640 1 -- 172.21.15.44:0/1117891168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed41f7830 con 0x7f8ed40b31c0 2024-09-13T08:48:59.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.459+0000 7f8ed2800640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ed40b31c0 0x7f8ed4147420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400d950 tx=0x7f8ec400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:59.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.459+0000 7f8ecaa00640 1 -- 172.21.15.44:0/1117891168 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4017070 con 0x7f8ed40b31c0 2024-09-13T08:48:59.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.459+0000 7f8ecaa00640 1 -- 172.21.15.44:0/1117891168 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ec4009d20 con 0x7f8ed40b31c0 2024-09-13T08:48:59.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.459+0000 7f8ecaa00640 1 -- 172.21.15.44:0/1117891168 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40106c0 con 0x7f8ed40b31c0 2024-09-13T08:48:59.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.463+0000 7f8ed9400640 1 -- 172.21.15.44:0/1117891168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed4147b00 con 0x7f8ed40b31c0 2024-09-13T08:48:59.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.463+0000 7f8ed9400640 1 -- 172.21.15.44:0/1117891168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41480c0 con 0x7f8ed40b31c0 2024-09-13T08:48:59.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.463+0000 7f8ed9400640 1 -- 172.21.15.44:0/1117891168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed40b3650 con 0x7f8ed40b31c0 2024-09-13T08:48:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.467+0000 7f8ecaa00640 1 -- 172.21.15.44:0/1117891168 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f8ec4010860 con 0x7f8ed40b31c0 2024-09-13T08:48:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.467+0000 7f8ecaa00640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8eb4074640 0x7f8eb4076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:48:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.467+0000 7f8ecaa00640 1 -- 172.21.15.44:0/1117891168 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f8ec4092e10 con 0x7f8ed40b31c0 2024-09-13T08:48:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.471+0000 7f8ed1e00640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8eb4074640 0x7f8eb4076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:48:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.471+0000 7f8ed1e00640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8eb4074640 0x7f8eb4076b00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8ed41370a0 tx=0x7f8ec0009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:48:59.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.475+0000 7f8ecaa00640 1 -- 172.21.15.44:0/1117891168 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8ec405f4f0 con 0x7f8ed40b31c0 2024-09-13T08:48:59.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.867+0000 7f8ed9400640 1 -- 172.21.15.44:0/1117891168 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f8ed4159f00 con 0x7f8eb4074640 2024-09-13T08:48:59.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.899+0000 7f8ecaa00640 1 -- 172.21.15.44:0/1117891168 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ed4159f00 con 0x7f8eb4074640 2024-09-13T08:48:59.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 -- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8eb4074640 msgr2=0x7f8eb4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:59.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8eb4074640 0x7f8eb4076b00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8ed41370a0 tx=0x7f8ec0009060 comp rx=0 tx=0).stop 2024-09-13T08:48:59.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 -- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ed40b31c0 msgr2=0x7f8ed4147420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:48:59.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ed40b31c0 0x7f8ed4147420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400d950 tx=0x7f8ec400de10 comp rx=0 tx=0).stop 2024-09-13T08:48:59.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 -- 172.21.15.44:0/1117891168 shutdown_connections 2024-09-13T08:48:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8eb4074640 0x7f8eb4076b00 secure :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8ed41370a0 tx=0x7f8ec0009060 comp rx=0 tx=0).stop 2024-09-13T08:48:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ed4151160 0x7f8ed4132a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ed4150d80 0x7f8ed41f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:48:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 --2- 172.21.15.44:0/1117891168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ed40b31c0 0x7f8ed4147420 secure :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400d950 tx=0x7f8ec400de10 comp rx=0 tx=0).stop 2024-09-13T08:48:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 -- 172.21.15.44:0/1117891168 >> 172.21.15.44:0/1117891168 conn(0x7f8ed406ecd0 msgr2=0x7f8ed414da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:48:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 -- 172.21.15.44:0/1117891168 shutdown_connections 2024-09-13T08:48:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:48:59.903+0000 7f8ed9400640 1 -- 172.21.15.44:0/1117891168 wait complete. 2024-09-13T08:48:59.940 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone680213 2024-09-13T08:49:00.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.183+0000 7fd02cc00640 1 -- 172.21.15.44:0/2887695215 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd02815b6e0 msgr2=0x7fd0280cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:00.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.183+0000 7fd02cc00640 1 --2- 172.21.15.44:0/2887695215 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd02815b6e0 0x7fd0280cd530 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd020009bf0 tx=0x7fd02001cbe0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.183+0000 7fd02cc00640 1 -- 172.21.15.44:0/2887695215 shutdown_connections 2024-09-13T08:49:00.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.183+0000 7fd02cc00640 1 --2- 172.21.15.44:0/2887695215 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd02815b6e0 0x7fd0280cd530 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.183+0000 7fd02cc00640 1 -- 172.21.15.44:0/2887695215 >> 172.21.15.44:0/2887695215 conn(0x7fd0280b2900 msgr2=0x7fd0280b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:00.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.183+0000 7fd02cc00640 1 -- 172.21.15.44:0/2887695215 shutdown_connections 2024-09-13T08:49:00.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.183+0000 7fd02cc00640 1 -- 172.21.15.44:0/2887695215 wait complete. 2024-09-13T08:49:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd02cc00640 1 Processor -- start 2024-09-13T08:49:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd02cc00640 1 -- start start 2024-09-13T08:49:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd02cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0280c7740 0x7fd028150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd02cc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd02815b270 0x7fd02814a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd02cc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd02815b6e0 0x7fd0280cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd02cc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0281699b0 con 0x7fd0280c7740 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd02cc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd028169830 con 0x7fd02815b270 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd02cc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd028169b30 con 0x7fd02815b6e0 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd025e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0280c7740 0x7fd028150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd025e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0280c7740 0x7fd028150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41436/0 (socket says 172.21.15.44:41436) 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd025e00640 1 -- 172.21.15.44:0/2810432172 learned_addr learned my addr 172.21.15.44:0/2810432172 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd026800640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd02815b6e0 0x7fd0280cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd025400640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd02815b270 0x7fd02814a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd025e00640 1 -- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd02815b6e0 msgr2=0x7fd0280cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd025e00640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd02815b6e0 0x7fd0280cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd025e00640 1 -- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd02815b270 msgr2=0x7fd02814a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd025e00640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd02815b270 0x7fd02814a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.187+0000 7fd025e00640 1 -- 172.21.15.44:0/2810432172 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02809e7c0 con 0x7fd0280c7740 2024-09-13T08:49:00.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.191+0000 7fd025e00640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0280c7740 0x7fd028150c20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd01800c4b0 tx=0x7fd01800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:00.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.191+0000 7fd01e000640 1 -- 172.21.15.44:0/2810432172 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018016020 con 0x7fd0280c7740 2024-09-13T08:49:00.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.191+0000 7fd01e000640 1 -- 172.21.15.44:0/2810432172 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd018012920 con 0x7fd0280c7740 2024-09-13T08:49:00.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.191+0000 7fd01e000640 1 -- 172.21.15.44:0/2810432172 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018012d30 con 0x7fd0280c7740 2024-09-13T08:49:00.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.191+0000 7fd02cc00640 1 -- 172.21.15.44:0/2810432172 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd028152480 con 0x7fd0280c7740 2024-09-13T08:49:00.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.191+0000 7fd02cc00640 1 -- 172.21.15.44:0/2810432172 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02814ae20 con 0x7fd0280c7740 2024-09-13T08:49:00.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.191+0000 7fd02cc00640 1 -- 172.21.15.44:0/2810432172 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0280cd530 con 0x7fd0280c7740 2024-09-13T08:49:00.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.195+0000 7fd01e000640 1 -- 172.21.15.44:0/2810432172 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fd018019020 con 0x7fd0280c7740 2024-09-13T08:49:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.195+0000 7fd01e000640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd008074590 0x7fd008076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.195+0000 7fd01e000640 1 -- 172.21.15.44:0/2810432172 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fd018092950 con 0x7fd0280c7740 2024-09-13T08:49:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.195+0000 7fd025400640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd008074590 0x7fd008076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.199+0000 7fd025400640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd008074590 0x7fd008076a50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd0280cd2f0 tx=0x7fd014008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.203+0000 7fd01e000640 1 -- 172.21.15.44:0/2810432172 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd018020030 con 0x7fd0280c7740 2024-09-13T08:49:00.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.491+0000 7fd02cc00640 1 -- 172.21.15.44:0/2810432172 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone680213", "target": ["mon-mgr", ""]}) -- 0x7fd02809eaa0 con 0x7fd008074590 2024-09-13T08:49:00.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.531+0000 7fd01e000640 1 -- 172.21.15.44:0/2810432172 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd02809eaa0 con 0x7fd008074590 2024-09-13T08:49:00.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 -- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd008074590 msgr2=0x7fd008076a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:00.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd008074590 0x7fd008076a50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd0280cd2f0 tx=0x7fd014008040 comp rx=0 tx=0).stop 2024-09-13T08:49:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 -- 172.21.15.44:0/2810432172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0280c7740 msgr2=0x7fd028150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0280c7740 0x7fd028150c20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd01800c4b0 tx=0x7fd01800c970 comp rx=0 tx=0).stop 2024-09-13T08:49:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 -- 172.21.15.44:0/2810432172 shutdown_connections 2024-09-13T08:49:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd008074590 0x7fd008076a50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd02815b6e0 0x7fd0280cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd02815b270 0x7fd02814a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 --2- 172.21.15.44:0/2810432172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0280c7740 0x7fd028150c20 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 -- 172.21.15.44:0/2810432172 >> 172.21.15.44:0/2810432172 conn(0x7fd0280b2900 msgr2=0x7fd02806f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 -- 172.21.15.44:0/2810432172 shutdown_connections 2024-09-13T08:49:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.535+0000 7fd02cc00640 1 -- 172.21.15.44:0/2810432172 wait complete. 2024-09-13T08:49:00.568 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone6700 2024-09-13T08:49:00.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.819+0000 7f76a0a00640 1 -- 172.21.15.44:0/3502405256 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c15de40 msgr2=0x7f769c0b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:00.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.819+0000 7f76a0a00640 1 --2- 172.21.15.44:0/3502405256 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c15de40 0x7f769c0b35a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f768c009870 tx=0x7f768c01c5a0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.819+0000 7f76a0a00640 1 -- 172.21.15.44:0/3502405256 shutdown_connections 2024-09-13T08:49:00.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.819+0000 7f76a0a00640 1 --2- 172.21.15.44:0/3502405256 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c15de40 0x7f769c0b35a0 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f768c009870 tx=0x7f768c01c5a0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.819+0000 7f76a0a00640 1 -- 172.21.15.44:0/3502405256 >> 172.21.15.44:0/3502405256 conn(0x7f769c0d3f90 msgr2=0x7f769c0d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:00.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.823+0000 7f76a0a00640 1 -- 172.21.15.44:0/3502405256 shutdown_connections 2024-09-13T08:49:00.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.823+0000 7f76a0a00640 1 -- 172.21.15.44:0/3502405256 wait complete. 2024-09-13T08:49:00.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.827+0000 7f76a0a00640 1 Processor -- start 2024-09-13T08:49:00.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.827+0000 7f76a0a00640 1 -- start start 2024-09-13T08:49:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.827+0000 7f76a0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c0d2540 0x7f769c0b9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.831+0000 7f76a0a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c15e220 0x7f769c1f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.831+0000 7f76a0a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c152fa0 0x7f769c09dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.831+0000 7f76a0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f769c0cffa0 con 0x7f769c0d2540 2024-09-13T08:49:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.831+0000 7f76a0a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f769c0cfe20 con 0x7f769c15e220 2024-09-13T08:49:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.831+0000 7f76a0a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f769c0d29d0 con 0x7f769c152fa0 2024-09-13T08:49:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.831+0000 7f7699400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c15e220 0x7f769c1f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.831+0000 7f7699400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c15e220 0x7f769c1f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:58786/0 (socket says 172.21.15.44:58786) 2024-09-13T08:49:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.831+0000 7f7699400640 1 -- 172.21.15.44:0/1252794716 learned_addr learned my addr 172.21.15.44:0/1252794716 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f769a800640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c152fa0 0x7f769c09dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7699e00640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c0d2540 0x7f769c0b9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7699400640 1 -- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c152fa0 msgr2=0x7f769c09dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7699400640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c152fa0 0x7f769c09dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7699400640 1 -- 172.21.15.44:0/1252794716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c0d2540 msgr2=0x7f769c0b9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7699400640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c0d2540 0x7f769c0b9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7699400640 1 -- 172.21.15.44:0/1252794716 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c1f6440 con 0x7f769c15e220 2024-09-13T08:49:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7699e00640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c0d2540 0x7f769c0b9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7699400640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c15e220 0x7f769c1f5f00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7684009cd0 tx=0x7f768400c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7692000640 1 -- 172.21.15.44:0/1252794716 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684004030 con 0x7f769c15e220 2024-09-13T08:49:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7692000640 1 -- 172.21.15.44:0/1252794716 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76840042e0 con 0x7f769c15e220 2024-09-13T08:49:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f76a0a00640 1 -- 172.21.15.44:0/1252794716 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c153b60 con 0x7f769c15e220 2024-09-13T08:49:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f7692000640 1 -- 172.21.15.44:0/1252794716 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684003860 con 0x7f769c15e220 2024-09-13T08:49:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.835+0000 7f76a0a00640 1 -- 172.21.15.44:0/1252794716 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c0b8690 con 0x7f769c15e220 2024-09-13T08:49:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.839+0000 7f76a0a00640 1 -- 172.21.15.44:0/1252794716 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c0b35a0 con 0x7f769c15e220 2024-09-13T08:49:00.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.839+0000 7f7692000640 1 -- 172.21.15.44:0/1252794716 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f7684003a00 con 0x7f769c15e220 2024-09-13T08:49:00.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.839+0000 7f7692000640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7678074540 0x7f7678076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:00.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.839+0000 7f7692000640 1 -- 172.21.15.44:0/1252794716 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f76840956e0 con 0x7f769c15e220 2024-09-13T08:49:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.843+0000 7f7699e00640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7678074540 0x7f7678076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:00.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.843+0000 7f7699e00640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7678074540 0x7f7678076a00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f768c0ae000 tx=0x7f768c01ade0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:00.847+0000 7f7692000640 1 -- 172.21.15.44:0/1252794716 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f769c1f6440 con 0x7f769c15e220 2024-09-13T08:49:01.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.055+0000 7f76a0a00640 1 -- 172.21.15.44:0/1252794716 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone6700", "target": ["mon-mgr", ""]}) -- 0x7f769c0b9f10 con 0x7f7678074540 2024-09-13T08:49:01.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.075+0000 7f7692000640 1 -- 172.21.15.44:0/1252794716 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f769c0b9f10 con 0x7f7678074540 2024-09-13T08:49:01.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 -- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7678074540 msgr2=0x7f7678076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7678074540 0x7f7678076a00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f768c0ae000 tx=0x7f768c01ade0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 -- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c15e220 msgr2=0x7f769c1f5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c15e220 0x7f769c1f5f00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7684009cd0 tx=0x7f768400c430 comp rx=0 tx=0).stop 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 -- 172.21.15.44:0/1252794716 shutdown_connections 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7678074540 0x7f7678076a00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f769c152fa0 0x7f769c09dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f769c15e220 0x7f769c1f5f00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 --2- 172.21.15.44:0/1252794716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f769c0d2540 0x7f769c0b9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 -- 172.21.15.44:0/1252794716 >> 172.21.15.44:0/1252794716 conn(0x7f769c0d3f90 msgr2=0x7f769c0ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 -- 172.21.15.44:0/1252794716 shutdown_connections 2024-09-13T08:49:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.079+0000 7f76a0a00640 1 -- 172.21.15.44:0/1252794716 wait complete. 2024-09-13T08:49:01.106 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone5221 2024-09-13T08:49:01.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.255+0000 7f98a6800640 1 -- 172.21.15.44:0/3907372344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98a015c1f0 msgr2=0x7f98a00cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:01.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.255+0000 7f98a6800640 1 --2- 172.21.15.44:0/3907372344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98a015c1f0 0x7f98a00cee10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9888009840 tx=0x7f988801f010 comp rx=0 tx=0).stop 2024-09-13T08:49:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.255+0000 7f98a6800640 1 -- 172.21.15.44:0/3907372344 shutdown_connections 2024-09-13T08:49:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.255+0000 7f98a6800640 1 --2- 172.21.15.44:0/3907372344 >> v2:172.21.15.129:3300/0 conn(0x7f98a00b0b60 0x7f98a00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.255+0000 7f98a6800640 1 --2- 172.21.15.44:0/3907372344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98a015c1f0 0x7f98a00cee10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.255+0000 7f98a6800640 1 -- 172.21.15.44:0/3907372344 >> 172.21.15.44:0/3907372344 conn(0x7f98a00d1da0 msgr2=0x7f98a00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.255+0000 7f98a6800640 1 -- 172.21.15.44:0/3907372344 shutdown_connections 2024-09-13T08:49:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.255+0000 7f98a6800640 1 -- 172.21.15.44:0/3907372344 wait complete. 2024-09-13T08:49:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f98a6800640 1 Processor -- start 2024-09-13T08:49:01.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f98a6800640 1 -- start start 2024-09-13T08:49:01.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f98a6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a00b0b60 0x7f98a00c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:01.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f98a6800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98a015be10 0x7f98a00cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f98a6800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98a015c1f0 0x7f98a00cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f98a6800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00cd320 con 0x7f98a00b0b60 2024-09-13T08:49:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f98a6800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00cf590 con 0x7f98a015be10 2024-09-13T08:49:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f98a6800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f98a00cd4a0 con 0x7f98a015c1f0 2024-09-13T08:49:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f989f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a00b0b60 0x7f98a00c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f989f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a00b0b60 0x7f98a00c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41454/0 (socket says 172.21.15.44:41454) 2024-09-13T08:49:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f989f400640 1 -- 172.21.15.44:0/263116565 learned_addr learned my addr 172.21.15.44:0/263116565 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f9897e00640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98a015be10 0x7f98a00cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f989fe00640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98a015c1f0 0x7f98a00cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:01.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f989f400640 1 -- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98a015c1f0 msgr2=0x7f98a00cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f989f400640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98a015c1f0 0x7f98a00cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f989f400640 1 -- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98a015be10 msgr2=0x7f98a00cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f989f400640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98a015be10 0x7f98a00cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f989f400640 1 -- 172.21.15.44:0/263116565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a00cbd00 con 0x7f98a00b0b60 2024-09-13T08:49:01.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f989f400640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a00b0b60 0x7f98a00c9640 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f989000c490 tx=0x7f989000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f9897400640 1 -- 172.21.15.44:0/263116565 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890016020 con 0x7f98a00b0b60 2024-09-13T08:49:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f9897400640 1 -- 172.21.15.44:0/263116565 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9890012940 con 0x7f98a00b0b60 2024-09-13T08:49:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f9897400640 1 -- 172.21.15.44:0/263116565 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890012d90 con 0x7f98a00b0b60 2024-09-13T08:49:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f98a6800640 1 -- 172.21.15.44:0/263116565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a01561e0 con 0x7f98a00b0b60 2024-09-13T08:49:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.259+0000 7f98a6800640 1 -- 172.21.15.44:0/263116565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a01567a0 con 0x7f98a00b0b60 2024-09-13T08:49:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.263+0000 7f98a6800640 1 -- 172.21.15.44:0/263116565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a00ccbd0 con 0x7f98a00b0b60 2024-09-13T08:49:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.263+0000 7f9897400640 1 -- 172.21.15.44:0/263116565 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f9890019020 con 0x7f98a00b0b60 2024-09-13T08:49:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.263+0000 7f9897400640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f987c074540 0x7f987c076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.263+0000 7f9897400640 1 -- 172.21.15.44:0/263116565 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f9890092a20 con 0x7f98a00b0b60 2024-09-13T08:49:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.263+0000 7f9897e00640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f987c074540 0x7f987c076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.267+0000 7f9897e00640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f987c074540 0x7f987c076a00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f98880b2000 tx=0x7f988801d850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:01.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.267+0000 7f9897400640 1 -- 172.21.15.44:0/263116565 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9890020020 con 0x7f98a00b0b60 2024-09-13T08:49:01.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.539+0000 7f98a6800640 1 -- 172.21.15.44:0/263116565 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone5221", "target": ["mon-mgr", ""]}) -- 0x7f98a0151930 con 0x7f987c074540 2024-09-13T08:49:01.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.575+0000 7f9897400640 1 -- 172.21.15.44:0/263116565 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98a0151930 con 0x7f987c074540 2024-09-13T08:49:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 -- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f987c074540 msgr2=0x7f987c076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f987c074540 0x7f987c076a00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f98880b2000 tx=0x7f988801d850 comp rx=0 tx=0).stop 2024-09-13T08:49:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 -- 172.21.15.44:0/263116565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a00b0b60 msgr2=0x7f98a00c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a00b0b60 0x7f98a00c9640 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f989000c490 tx=0x7f989000c950 comp rx=0 tx=0).stop 2024-09-13T08:49:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 -- 172.21.15.44:0/263116565 shutdown_connections 2024-09-13T08:49:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f987c074540 0x7f987c076a00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98a015c1f0 0x7f98a00cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98a015be10 0x7f98a00cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 --2- 172.21.15.44:0/263116565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a00b0b60 0x7f98a00c9640 secure :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f989000c490 tx=0x7f989000c950 comp rx=0 tx=0).stop 2024-09-13T08:49:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 -- 172.21.15.44:0/263116565 >> 172.21.15.44:0/263116565 conn(0x7f98a00d1da0 msgr2=0x7f98a0070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 -- 172.21.15.44:0/263116565 shutdown_connections 2024-09-13T08:49:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.579+0000 7f98a6800640 1 -- 172.21.15.44:0/263116565 wait complete. 2024-09-13T08:49:01.619 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone25384 2024-09-13T08:49:01.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.859+0000 7f5fcd200640 1 -- 172.21.15.44:0/1231351166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fc809dad0 msgr2=0x7f5fc815f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.859+0000 7f5fcd200640 1 --2- 172.21.15.44:0/1231351166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fc809dad0 0x7f5fc815f060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009930 tx=0x7f5fb401c920 comp rx=0 tx=0).stop 2024-09-13T08:49:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.859+0000 7f5fcd200640 1 -- 172.21.15.44:0/1231351166 shutdown_connections 2024-09-13T08:49:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.859+0000 7f5fcd200640 1 --2- 172.21.15.44:0/1231351166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fc809dad0 0x7f5fc815f060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.859+0000 7f5fcd200640 1 -- 172.21.15.44:0/1231351166 >> 172.21.15.44:0/1231351166 conn(0x7f5fc80cfbc0 msgr2=0x7f5fc80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.859+0000 7f5fcd200640 1 -- 172.21.15.44:0/1231351166 shutdown_connections 2024-09-13T08:49:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.859+0000 7f5fcd200640 1 -- 172.21.15.44:0/1231351166 wait complete. 2024-09-13T08:49:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fcd200640 1 Processor -- start 2024-09-13T08:49:01.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fcd200640 1 -- start start 2024-09-13T08:49:01.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fcd200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fc809dad0 0x7f5fc809bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:01.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fcd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fc80ad3c0 0x7f5fc81fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:01.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fcd200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc815ade0 0x7f5fc81454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:01.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fcd200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc81691c0 con 0x7f5fc80ad3c0 2024-09-13T08:49:01.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fcd200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc8169040 con 0x7f5fc815ade0 2024-09-13T08:49:01.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fcd200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc8169340 con 0x7f5fc809dad0 2024-09-13T08:49:01.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fc80ad3c0 0x7f5fc81fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:01.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fc80ad3c0 0x7f5fc81fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41470/0 (socket says 172.21.15.44:41470) 2024-09-13T08:49:01.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc5400640 1 -- 172.21.15.44:0/2119626312 learned_addr learned my addr 172.21.15.44:0/2119626312 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:01.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc5e00640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fc809dad0 0x7f5fc809bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:01.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc6800640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc815ade0 0x7f5fc81454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:01.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc5400640 1 -- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fc809dad0 msgr2=0x7f5fc809bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:01.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc5400640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fc809dad0 0x7f5fc809bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc5400640 1 -- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc815ade0 msgr2=0x7f5fc81454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:01.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc5400640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc815ade0 0x7f5fc81454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:01.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc5400640 1 -- 172.21.15.44:0/2119626312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc81fc0c0 con 0x7f5fc80ad3c0 2024-09-13T08:49:01.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc6800640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc815ade0 0x7f5fc81454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:01.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fc5400640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fc80ad3c0 0x7f5fc81fbb80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40b0040 tx=0x7f5fb40a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:01.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fbe000640 1 -- 172.21.15.44:0/2119626312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40b2040 con 0x7f5fc80ad3c0 2024-09-13T08:49:01.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fbe000640 1 -- 172.21.15.44:0/2119626312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fb4004470 con 0x7f5fc80ad3c0 2024-09-13T08:49:01.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fbe000640 1 -- 172.21.15.44:0/2119626312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4005290 con 0x7f5fc80ad3c0 2024-09-13T08:49:01.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fcd200640 1 -- 172.21.15.44:0/2119626312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc809aab0 con 0x7f5fc80ad3c0 2024-09-13T08:49:01.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.863+0000 7f5fcd200640 1 -- 172.21.15.44:0/2119626312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc809b070 con 0x7f5fc80ad3c0 2024-09-13T08:49:01.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.867+0000 7f5fcd200640 1 -- 172.21.15.44:0/2119626312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc8072870 con 0x7f5fc80ad3c0 2024-09-13T08:49:01.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.867+0000 7f5fbe000640 1 -- 172.21.15.44:0/2119626312 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f5fb40bf060 con 0x7f5fc80ad3c0 2024-09-13T08:49:01.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.867+0000 7f5fbe000640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8074540 0x7f5fa8076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:01.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.867+0000 7f5fbe000640 1 -- 172.21.15.44:0/2119626312 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f5fb412caa0 con 0x7f5fc80ad3c0 2024-09-13T08:49:01.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.867+0000 7f5fc5e00640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8074540 0x7f5fa8076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:01.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.867+0000 7f5fc5e00640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8074540 0x7f5fa8076a00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8007fd0 tx=0x7f5fb8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:01.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:01.871+0000 7f5fbe000640 1 -- 172.21.15.44:0/2119626312 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5fc81fc0c0 con 0x7f5fc80ad3c0 2024-09-13T08:49:02.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.071+0000 7f5fcd200640 1 -- 172.21.15.44:0/2119626312 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone25384", "target": ["mon-mgr", ""]}) -- 0x7f5fc8165030 con 0x7f5fa8074540 2024-09-13T08:49:02.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fbe000640 1 -- 172.21.15.44:0/2119626312 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5fc8165030 con 0x7f5fa8074540 2024-09-13T08:49:02.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 -- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8074540 msgr2=0x7f5fa8076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:02.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8074540 0x7f5fa8076a00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8007fd0 tx=0x7f5fb8009040 comp rx=0 tx=0).stop 2024-09-13T08:49:02.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 -- 172.21.15.44:0/2119626312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fc80ad3c0 msgr2=0x7f5fc81fbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:02.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fc80ad3c0 0x7f5fc81fbb80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40b0040 tx=0x7f5fb40a56b0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 -- 172.21.15.44:0/2119626312 shutdown_connections 2024-09-13T08:49:02.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8074540 0x7f5fa8076a00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc815ade0 0x7f5fc81454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fc80ad3c0 0x7f5fc81fbb80 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 --2- 172.21.15.44:0/2119626312 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fc809dad0 0x7f5fc809bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 -- 172.21.15.44:0/2119626312 >> 172.21.15.44:0/2119626312 conn(0x7f5fc80cfbc0 msgr2=0x7f5fc80b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:02.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 -- 172.21.15.44:0/2119626312 shutdown_connections 2024-09-13T08:49:02.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.091+0000 7f5fcd200640 1 -- 172.21.15.44:0/2119626312 wait complete. 2024-09-13T08:49:02.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone453217 2024-09-13T08:49:02.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.267+0000 7fab5ba00640 1 -- 172.21.15.44:0/2879297122 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fab540b5bf0 msgr2=0x7fab5406f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:02.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.267+0000 7fab5ba00640 1 --2- 172.21.15.44:0/2879297122 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fab540b5bf0 0x7fab5406f690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009bf0 tx=0x7fab4c01cb50 comp rx=0 tx=0).stop 2024-09-13T08:49:02.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.267+0000 7fab5ba00640 1 -- 172.21.15.44:0/2879297122 shutdown_connections 2024-09-13T08:49:02.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.267+0000 7fab5ba00640 1 --2- 172.21.15.44:0/2879297122 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fab540b5bf0 0x7fab5406f690 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.267+0000 7fab5ba00640 1 -- 172.21.15.44:0/2879297122 >> 172.21.15.44:0/2879297122 conn(0x7fab5406dd30 msgr2=0x7fab5406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:02.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.267+0000 7fab5ba00640 1 -- 172.21.15.44:0/2879297122 shutdown_connections 2024-09-13T08:49:02.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.267+0000 7fab5ba00640 1 -- 172.21.15.44:0/2879297122 wait complete. 2024-09-13T08:49:02.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.267+0000 7fab5ba00640 1 Processor -- start 2024-09-13T08:49:02.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab5ba00640 1 -- start start 2024-09-13T08:49:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab5ba00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540b5bf0 0x7fab5415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab5ba00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fab5415b960 0x7fab5415a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab5ba00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab5415e2f0 0x7fab54144a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab5ba00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fab54147200 con 0x7fab540b5bf0 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab5ba00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab54147080 con 0x7fab5415e2f0 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab5ba00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fab54147380 con 0x7fab5415b960 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab59000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540b5bf0 0x7fab5415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab59000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540b5bf0 0x7fab5415aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59954/0 (socket says 172.21.15.44:59954) 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab59000640 1 -- 172.21.15.44:0/638287832 learned_addr learned my addr 172.21.15.44:0/638287832 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab59a00640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab5415e2f0 0x7fab54144a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab4be00640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fab5415b960 0x7fab5415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab59000640 1 -- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fab5415b960 msgr2=0x7fab5415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab59000640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fab5415b960 0x7fab5415a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab59000640 1 -- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab5415e2f0 msgr2=0x7fab54144a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab59000640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab5415e2f0 0x7fab54144a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab59000640 1 -- 172.21.15.44:0/638287832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5413fd10 con 0x7fab540b5bf0 2024-09-13T08:49:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab59000640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540b5bf0 0x7fab5415aba0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fab4400c460 tx=0x7fab4400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab51600640 1 -- 172.21.15.44:0/638287832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44017070 con 0x7fab540b5bf0 2024-09-13T08:49:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab51600640 1 -- 172.21.15.44:0/638287832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab44003600 con 0x7fab540b5bf0 2024-09-13T08:49:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab51600640 1 -- 172.21.15.44:0/638287832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44003a50 con 0x7fab540b5bf0 2024-09-13T08:49:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab5ba00640 1 -- 172.21.15.44:0/638287832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5413fff0 con 0x7fab540b5bf0 2024-09-13T08:49:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab5ba00640 1 -- 172.21.15.44:0/638287832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab540cf860 con 0x7fab540b5bf0 2024-09-13T08:49:02.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.271+0000 7fab5ba00640 1 -- 172.21.15.44:0/638287832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5406f690 con 0x7fab540b5bf0 2024-09-13T08:49:02.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.275+0000 7fab51600640 1 -- 172.21.15.44:0/638287832 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fab4401a070 con 0x7fab540b5bf0 2024-09-13T08:49:02.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.275+0000 7fab51600640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab30074540 0x7fab30076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:02.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.275+0000 7fab51600640 1 -- 172.21.15.44:0/638287832 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7fab44091e50 con 0x7fab540b5bf0 2024-09-13T08:49:02.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.275+0000 7fab4be00640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab30074540 0x7fab30076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.275+0000 7fab4be00640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab30074540 0x7fab30076a00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fab5413f0e0 tx=0x7fab3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.279+0000 7fab51600640 1 -- 172.21.15.44:0/638287832 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fab4405e4b0 con 0x7fab540b5bf0 2024-09-13T08:49:02.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.543+0000 7fab5ba00640 1 -- 172.21.15.44:0/638287832 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone453217", "target": ["mon-mgr", ""]}) -- 0x7fab5413fac0 con 0x7fab30074540 2024-09-13T08:49:02.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.563+0000 7fab51600640 1 -- 172.21.15.44:0/638287832 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab5413fac0 con 0x7fab30074540 2024-09-13T08:49:02.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 -- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab30074540 msgr2=0x7fab30076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:02.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab30074540 0x7fab30076a00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fab5413f0e0 tx=0x7fab3c009040 comp rx=0 tx=0).stop 2024-09-13T08:49:02.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 -- 172.21.15.44:0/638287832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540b5bf0 msgr2=0x7fab5415aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:02.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540b5bf0 0x7fab5415aba0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fab4400c460 tx=0x7fab4400c920 comp rx=0 tx=0).stop 2024-09-13T08:49:02.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 -- 172.21.15.44:0/638287832 shutdown_connections 2024-09-13T08:49:02.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fab30074540 0x7fab30076a00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab5415e2f0 0x7fab54144a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fab5415b960 0x7fab5415a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 --2- 172.21.15.44:0/638287832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540b5bf0 0x7fab5415aba0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 -- 172.21.15.44:0/638287832 >> 172.21.15.44:0/638287832 conn(0x7fab5406dd30 msgr2=0x7fab54070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:02.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 -- 172.21.15.44:0/638287832 shutdown_connections 2024-09-13T08:49:02.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.567+0000 7fab5ba00640 1 -- 172.21.15.44:0/638287832 wait complete. 2024-09-13T08:49:02.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:49:02.585 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:49:02.591 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone74089 2024-09-13T08:49:02.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 -- 172.21.15.44:0/727551231 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73f415be10 msgr2=0x7f73f40ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:02.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 --2- 172.21.15.44:0/727551231 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73f415be10 0x7f73f40ce700 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009870 tx=0x7f73e401f050 comp rx=0 tx=0).stop 2024-09-13T08:49:02.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 -- 172.21.15.44:0/727551231 shutdown_connections 2024-09-13T08:49:02.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 --2- 172.21.15.44:0/727551231 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73f415be10 0x7f73f40ce700 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 -- 172.21.15.44:0/727551231 >> 172.21.15.44:0/727551231 conn(0x7f73f40d1da0 msgr2=0x7f73f40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:02.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 -- 172.21.15.44:0/727551231 shutdown_connections 2024-09-13T08:49:02.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 -- 172.21.15.44:0/727551231 wait complete. 2024-09-13T08:49:02.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 Processor -- start 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 -- start start 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73f40b0b60 0x7f73f4152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73f415be10 0x7f73f406f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73f415c1f0 0x7f73f40af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73f40cd670 con 0x7f73f40b0b60 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73f40cd4f0 con 0x7f73f415c1f0 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73fa400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f73f40cd7f0 con 0x7f73f415be10 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73f3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73f40b0b60 0x7f73f4152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73f3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73f40b0b60 0x7f73f4152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59966/0 (socket says 172.21.15.44:59966) 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73f3400640 1 -- 172.21.15.44:0/3869631118 learned_addr learned my addr 172.21.15.44:0/3869631118 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:02.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73f2a00640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73f415be10 0x7f73f406f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:02.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73f3e00640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73f415c1f0 0x7f73f40af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:02.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73f3400640 1 -- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73f415be10 msgr2=0x7f73f406f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:02.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73f3400640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73f415be10 0x7f73f406f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73f3400640 1 -- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73f415c1f0 msgr2=0x7f73f40af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:02.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73f3400640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73f415c1f0 0x7f73f40af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:02.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.823+0000 7f73f3400640 1 -- 172.21.15.44:0/3869631118 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f41527c0 con 0x7f73f40b0b60 2024-09-13T08:49:02.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.827+0000 7f73f3400640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73f40b0b60 0x7f73f4152280 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f73e401bc60 tx=0x7f73e4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:02.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.827+0000 7f73eb400640 1 -- 172.21.15.44:0/3869631118 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40b5040 con 0x7f73f40b0b60 2024-09-13T08:49:02.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.827+0000 7f73eb400640 1 -- 172.21.15.44:0/3869631118 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73e4003600 con 0x7f73f40b0b60 2024-09-13T08:49:02.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.827+0000 7f73eb400640 1 -- 172.21.15.44:0/3869631118 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40039f0 con 0x7f73f40b0b60 2024-09-13T08:49:02.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.827+0000 7f73fa400640 1 -- 172.21.15.44:0/3869631118 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f41f7d00 con 0x7f73f40b0b60 2024-09-13T08:49:02.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.827+0000 7f73fa400640 1 -- 172.21.15.44:0/3869631118 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f41f7fa0 con 0x7f73f40b0b60 2024-09-13T08:49:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.827+0000 7f73fa400640 1 -- 172.21.15.44:0/3869631118 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f41f89a0 con 0x7f73f40b0b60 2024-09-13T08:49:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.827+0000 7f73eb400640 1 -- 172.21.15.44:0/3869631118 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f73e40c0030 con 0x7f73f40b0b60 2024-09-13T08:49:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.827+0000 7f73eb400640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f73d4074540 0x7f73d4076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.827+0000 7f73eb400640 1 -- 172.21.15.44:0/3869631118 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5793+0+0 (secure 0 0 0) 0x7f73e412fdb0 con 0x7f73f40b0b60 2024-09-13T08:49:02.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.831+0000 7f73f2a00640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f73d4074540 0x7f73d4076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:02.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.831+0000 7f73f2a00640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f73d4074540 0x7f73d4076a00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f73f4071320 tx=0x7f73e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:02.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:02.831+0000 7f73eb400640 1 -- 172.21.15.44:0/3869631118 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f73e40c7020 con 0x7f73f40b0b60 2024-09-13T08:49:03.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.031+0000 7f73fa400640 1 -- 172.21.15.44:0/3869631118 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone74089", "target": ["mon-mgr", ""]}) -- 0x7f73f4076410 con 0x7f73d4074540 2024-09-13T08:49:03.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.051+0000 7f73eb400640 1 -- 172.21.15.44:0/3869631118 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f73f4076410 con 0x7f73d4074540 2024-09-13T08:49:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 -- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f73d4074540 msgr2=0x7f73d4076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f73d4074540 0x7f73d4076a00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f73f4071320 tx=0x7f73e0009040 comp rx=0 tx=0).stop 2024-09-13T08:49:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 -- 172.21.15.44:0/3869631118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73f40b0b60 msgr2=0x7f73f4152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73f40b0b60 0x7f73f4152280 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f73e401bc60 tx=0x7f73e4004620 comp rx=0 tx=0).stop 2024-09-13T08:49:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 -- 172.21.15.44:0/3869631118 shutdown_connections 2024-09-13T08:49:03.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f73d4074540 0x7f73d4076a00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73f415c1f0 0x7f73f40af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73f415be10 0x7f73f406f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 --2- 172.21.15.44:0/3869631118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73f40b0b60 0x7f73f4152280 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 -- 172.21.15.44:0/3869631118 >> 172.21.15.44:0/3869631118 conn(0x7f73f40d1da0 msgr2=0x7f73f40aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 -- 172.21.15.44:0/3869631118 shutdown_connections 2024-09-13T08:49:03.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.055+0000 7f73fa400640 1 -- 172.21.15.44:0/3869631118 wait complete. 2024-09-13T08:49:03.076 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone855814 2024-09-13T08:49:03.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.231+0000 7f3677e00640 1 -- 172.21.15.44:0/1611587495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3658005220 msgr2=0x7f365800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.231+0000 7f3677e00640 1 --2- 172.21.15.44:0/1611587495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3658005220 0x7f365800b9d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f366800e400 tx=0x7f366801d7d0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.231+0000 7f3677e00640 1 -- 172.21.15.44:0/1611587495 shutdown_connections 2024-09-13T08:49:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.231+0000 7f3677e00640 1 --2- 172.21.15.44:0/1611587495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3658005220 0x7f365800b9d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.231+0000 7f3677e00640 1 -- 172.21.15.44:0/1611587495 >> 172.21.15.44:0/1611587495 conn(0x7f36700d1da0 msgr2=0x7f36700d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.231+0000 7f3677e00640 1 -- 172.21.15.44:0/1611587495 shutdown_connections 2024-09-13T08:49:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.231+0000 7f3677e00640 1 -- 172.21.15.44:0/1611587495 wait complete. 2024-09-13T08:49:03.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3677e00640 1 Processor -- start 2024-09-13T08:49:03.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3677e00640 1 -- start start 2024-09-13T08:49:03.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3677e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3658005220 0x7f36701f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3677e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f36701501c0 0x7f36700c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3677e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36700afd60 0x7f36700b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3677e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f36700cea90 con 0x7f36700afd60 2024-09-13T08:49:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3677e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36700ce910 con 0x7f3658005220 2024-09-13T08:49:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3677e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f36700cec10 con 0x7f36701501c0 2024-09-13T08:49:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3675e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36700afd60 0x7f36700b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3675e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36700afd60 0x7f36700b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59980/0 (socket says 172.21.15.44:59980) 2024-09-13T08:49:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3675e00640 1 -- 172.21.15.44:0/2395999573 learned_addr learned my addr 172.21.15.44:0/2395999573 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3674a00640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f36701501c0 0x7f36700c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3675400640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3658005220 0x7f36701f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3675e00640 1 -- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f36701501c0 msgr2=0x7f36700c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3675e00640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f36701501c0 0x7f36700c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3675e00640 1 -- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3658005220 msgr2=0x7f36701f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3675e00640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3658005220 0x7f36701f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3675e00640 1 -- 172.21.15.44:0/2395999573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36700c7a80 con 0x7f36700afd60 2024-09-13T08:49:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3675e00640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36700afd60 0x7f36700b0210 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3668009fb0 tx=0x7f3668009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f366d600640 1 -- 172.21.15.44:0/2395999573 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36680b3040 con 0x7f36700afd60 2024-09-13T08:49:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f3677e00640 1 -- 172.21.15.44:0/2395999573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36700c7d60 con 0x7f36700afd60 2024-09-13T08:49:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f366d600640 1 -- 172.21.15.44:0/2395999573 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3668004d40 con 0x7f36700afd60 2024-09-13T08:49:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.235+0000 7f366d600640 1 -- 172.21.15.44:0/2395999573 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668009c60 con 0x7f36700afd60 2024-09-13T08:49:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.239+0000 7f3677e00640 1 -- 172.21.15.44:0/2395999573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36700c9c10 con 0x7f36700afd60 2024-09-13T08:49:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.239+0000 7f3677e00640 1 -- 172.21.15.44:0/2395999573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36700cf170 con 0x7f36700afd60 2024-09-13T08:49:03.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.239+0000 7f366d600640 1 -- 172.21.15.44:0/2395999573 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f36680a6760 con 0x7f36700afd60 2024-09-13T08:49:03.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.239+0000 7f366d600640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3650074540 0x7f3650076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:03.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.239+0000 7f366d600640 1 -- 172.21.15.44:0/2395999573 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5849+0+0 (secure 0 0 0) 0x7f366812cdd0 con 0x7f36700afd60 2024-09-13T08:49:03.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.239+0000 7f3675400640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3650074540 0x7f3650076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:03.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.243+0000 7f3675400640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3650074540 0x7f3650076a00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3664000f70 tx=0x7f3664000ee0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:03.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.243+0000 7f366d600640 1 -- 172.21.15.44:0/2395999573 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f36680be030 con 0x7f36700afd60 2024-09-13T08:49:03.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.507+0000 7f3677e00640 1 -- 172.21.15.44:0/2395999573 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone855814", "target": ["mon-mgr", ""]}) -- 0x7f3670070350 con 0x7f3650074540 2024-09-13T08:49:03.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.539+0000 7f366d600640 1 -- 172.21.15.44:0/2395999573 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3670070350 con 0x7f3650074540 2024-09-13T08:49:03.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 -- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3650074540 msgr2=0x7f3650076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:03.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3650074540 0x7f3650076a00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3664000f70 tx=0x7f3664000ee0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 -- 172.21.15.44:0/2395999573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36700afd60 msgr2=0x7f36700b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:03.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36700afd60 0x7f36700b0210 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3668009fb0 tx=0x7f3668009700 comp rx=0 tx=0).stop 2024-09-13T08:49:03.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 -- 172.21.15.44:0/2395999573 shutdown_connections 2024-09-13T08:49:03.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3650074540 0x7f3650076a00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36700afd60 0x7f36700b0210 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f36701501c0 0x7f36700c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 --2- 172.21.15.44:0/2395999573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3658005220 0x7f36701f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 -- 172.21.15.44:0/2395999573 >> 172.21.15.44:0/2395999573 conn(0x7f36700d1da0 msgr2=0x7f36700b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:03.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 -- 172.21.15.44:0/2395999573 shutdown_connections 2024-09-13T08:49:03.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.543+0000 7f3677e00640 1 -- 172.21.15.44:0/2395999573 wait complete. 2024-09-13T08:49:03.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone300218 2024-09-13T08:49:03.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.831+0000 7f67f3e00640 1 -- 172.21.15.44:0/3287688927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f40cb8f0 msgr2=0x7f67f40ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:03.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.831+0000 7f67f3e00640 1 --2- 172.21.15.44:0/3287688927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f40cb8f0 0x7f67f40ce660 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009840 tx=0x7f67e401c760 comp rx=0 tx=0).stop 2024-09-13T08:49:03.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.831+0000 7f67f3e00640 1 -- 172.21.15.44:0/3287688927 shutdown_connections 2024-09-13T08:49:03.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.831+0000 7f67f3e00640 1 --2- 172.21.15.44:0/3287688927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f40cb8f0 0x7f67f40ce660 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.831+0000 7f67f3e00640 1 -- 172.21.15.44:0/3287688927 >> 172.21.15.44:0/3287688927 conn(0x7f67f40cc500 msgr2=0x7f67f40cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:03.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.831+0000 7f67f3e00640 1 -- 172.21.15.44:0/3287688927 shutdown_connections 2024-09-13T08:49:03.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.831+0000 7f67f3e00640 1 -- 172.21.15.44:0/3287688927 wait complete. 2024-09-13T08:49:03.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f3e00640 1 Processor -- start 2024-09-13T08:49:03.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f3e00640 1 -- start start 2024-09-13T08:49:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f3e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67f40ae760 0x7f67f40b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67f40cb8f0 0x7f67f40b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f415ae20 0x7f67f414c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f67f4169560 con 0x7f67f40cb8f0 2024-09-13T08:49:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f3e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67f415f740 con 0x7f67f415ae20 2024-09-13T08:49:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f3e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f67f41696e0 con 0x7f67f40ae760 2024-09-13T08:49:03.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f2a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67f40ae760 0x7f67f40b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:03.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67f40cb8f0 0x7f67f40b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:03.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67f40cb8f0 0x7f67f40b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59992/0 (socket says 172.21.15.44:59992) 2024-09-13T08:49:03.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f2000640 1 -- 172.21.15.44:0/3117432080 learned_addr learned my addr 172.21.15.44:0/3117432080 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:03.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.835+0000 7f67f3400640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f415ae20 0x7f67f414c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:03.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67f2000640 1 -- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67f40ae760 msgr2=0x7f67f40b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:03.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67f2000640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67f40ae760 0x7f67f40b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67f2000640 1 -- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f415ae20 msgr2=0x7f67f414c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67f2000640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f415ae20 0x7f67f414c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67f2000640 1 -- 172.21.15.44:0/3117432080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f415bd40 con 0x7f67f40cb8f0 2024-09-13T08:49:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67f2000640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67f40cb8f0 0x7f67f40b4560 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009790 tx=0x7f67e40a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67eaa00640 1 -- 172.21.15.44:0/3117432080 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40b2040 con 0x7f67f40cb8f0 2024-09-13T08:49:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67eaa00640 1 -- 172.21.15.44:0/3117432080 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67e4004430 con 0x7f67f40cb8f0 2024-09-13T08:49:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67eaa00640 1 -- 172.21.15.44:0/3117432080 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4005280 con 0x7f67f40cb8f0 2024-09-13T08:49:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67f3e00640 1 -- 172.21.15.44:0/3117432080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f415c000 con 0x7f67f40cb8f0 2024-09-13T08:49:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67f3e00640 1 -- 172.21.15.44:0/3117432080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f415c5c0 con 0x7f67f40cb8f0 2024-09-13T08:49:03.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.839+0000 7f67f3e00640 1 -- 172.21.15.44:0/3117432080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f40cb1e0 con 0x7f67f40cb8f0 2024-09-13T08:49:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.843+0000 7f67eaa00640 1 -- 172.21.15.44:0/3117432080 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f67e40be060 con 0x7f67f40cb8f0 2024-09-13T08:49:03.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.843+0000 7f67eaa00640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f67d4074610 0x7f67d4076ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:03.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.843+0000 7f67eaa00640 1 -- 172.21.15.44:0/3117432080 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5849+0+0 (secure 0 0 0) 0x7f67e412c650 con 0x7f67f40cb8f0 2024-09-13T08:49:03.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.847+0000 7f67f2a00640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f67d4074610 0x7f67d4076ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:03.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.847+0000 7f67f2a00640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f67d4074610 0x7f67d4076ad0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f67e0007fd0 tx=0x7f67e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:03.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:03.847+0000 7f67eaa00640 1 -- 172.21.15.44:0/3117432080 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f67e4016410 con 0x7f67f40cb8f0 2024-09-13T08:49:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:04.239+0000 7f67f3e00640 1 -- 172.21.15.44:0/3117432080 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone300218", "target": ["mon-mgr", ""]}) -- 0x7f67f40b5a00 con 0x7f67d4074610 2024-09-13T08:49:07.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.831+0000 7f67eaa00640 1 -- 172.21.15.44:0/3117432080 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f67f40b5a00 con 0x7f67d4074610 2024-09-13T08:49:07.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 -- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f67d4074610 msgr2=0x7f67d4076ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:07.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f67d4074610 0x7f67d4076ad0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f67e0007fd0 tx=0x7f67e0009040 comp rx=0 tx=0).stop 2024-09-13T08:49:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 -- 172.21.15.44:0/3117432080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67f40cb8f0 msgr2=0x7f67f40b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67f40cb8f0 0x7f67f40b4560 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009790 tx=0x7f67e40a5750 comp rx=0 tx=0).stop 2024-09-13T08:49:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 -- 172.21.15.44:0/3117432080 shutdown_connections 2024-09-13T08:49:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f67d4074610 0x7f67d4076ad0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f415ae20 0x7f67f414c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67f40cb8f0 0x7f67f40b4560 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 --2- 172.21.15.44:0/3117432080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67f40ae760 0x7f67f40b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 -- 172.21.15.44:0/3117432080 >> 172.21.15.44:0/3117432080 conn(0x7f67f40cc500 msgr2=0x7f67f40cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 -- 172.21.15.44:0/3117432080 shutdown_connections 2024-09-13T08:49:07.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:07.835+0000 7f67f3e00640 1 -- 172.21.15.44:0/3117432080 wait complete. 2024-09-13T08:49:07.870 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone632019 2024-09-13T08:49:08.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.023+0000 7f913ba00640 1 -- 172.21.15.44:0/4291579910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f913415be10 msgr2=0x7f91340ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.023+0000 7f913ba00640 1 --2- 172.21.15.44:0/4291579910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f913415be10 0x7f91340ce700 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9128009840 tx=0x7f912801c820 comp rx=0 tx=0).stop 2024-09-13T08:49:08.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.023+0000 7f913ba00640 1 -- 172.21.15.44:0/4291579910 shutdown_connections 2024-09-13T08:49:08.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.023+0000 7f913ba00640 1 --2- 172.21.15.44:0/4291579910 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91340b0b60 0x7f91340ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.023+0000 7f913ba00640 1 --2- 172.21.15.44:0/4291579910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f913415be10 0x7f91340ce700 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.023+0000 7f913ba00640 1 -- 172.21.15.44:0/4291579910 >> 172.21.15.44:0/4291579910 conn(0x7f91340d1da0 msgr2=0x7f91340d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:08.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.023+0000 7f913ba00640 1 -- 172.21.15.44:0/4291579910 shutdown_connections 2024-09-13T08:49:08.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.023+0000 7f913ba00640 1 -- 172.21.15.44:0/4291579910 wait complete. 2024-09-13T08:49:08.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f913ba00640 1 Processor -- start 2024-09-13T08:49:08.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f913ba00640 1 -- start start 2024-09-13T08:49:08.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f913ba00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91340b0b60 0x7f913415af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f913ba00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f913415be10 0x7f91340c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f913ba00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f913415c1f0 0x7f9134153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f913ba00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9134169040 con 0x7f91340b0b60 2024-09-13T08:49:08.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f913ba00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9134168ec0 con 0x7f913415be10 2024-09-13T08:49:08.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f913ba00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f91341691c0 con 0x7f913415c1f0 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91340b0b60 0x7f913415af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91340b0b60 0x7f913415af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60010/0 (socket says 172.21.15.44:60010) 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139000640 1 -- 172.21.15.44:0/4131385319 learned_addr learned my addr 172.21.15.44:0/4131385319 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139a00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f913415c1f0 0x7f9134153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9133e00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f913415be10 0x7f91340c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139000640 1 -- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f913415c1f0 msgr2=0x7f9134153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139000640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f913415c1f0 0x7f9134153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139000640 1 -- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f913415be10 msgr2=0x7f91340c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139000640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f913415be10 0x7f91340c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139000640 1 -- 172.21.15.44:0/4131385319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9134154150 con 0x7f91340b0b60 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139a00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f913415c1f0 0x7f9134153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9133e00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f913415be10 0x7f91340c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9139000640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91340b0b60 0x7f913415af50 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f9128009790 tx=0x7f91280a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9130c00640 1 -- 172.21.15.44:0/4131385319 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91280b2040 con 0x7f91340b0b60 2024-09-13T08:49:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9130c00640 1 -- 172.21.15.44:0/4131385319 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91280043b0 con 0x7f91340b0b60 2024-09-13T08:49:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f9130c00640 1 -- 172.21.15.44:0/4131385319 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128005240 con 0x7f91340b0b60 2024-09-13T08:49:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f913ba00640 1 -- 172.21.15.44:0/4131385319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9134154410 con 0x7f91340b0b60 2024-09-13T08:49:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.027+0000 7f913ba00640 1 -- 172.21.15.44:0/4131385319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91341f6510 con 0x7f91340b0b60 2024-09-13T08:49:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.031+0000 7f913ba00640 1 -- 172.21.15.44:0/4131385319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91340ccbd0 con 0x7f91340b0b60 2024-09-13T08:49:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.031+0000 7f9130c00640 1 -- 172.21.15.44:0/4131385319 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f91280053e0 con 0x7f91340b0b60 2024-09-13T08:49:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.031+0000 7f9130c00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9114074540 0x7f9114076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.031+0000 7f9130c00640 1 -- 172.21.15.44:0/4131385319 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5849+0+0 (secure 0 0 0) 0x7f912812c200 con 0x7f91340b0b60 2024-09-13T08:49:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.031+0000 7f9133e00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9114074540 0x7f9114076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.031+0000 7f9133e00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9114074540 0x7f9114076a00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f91340afbb0 tx=0x7f9120009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.035+0000 7f9130c00640 1 -- 172.21.15.44:0/4131385319 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9128016410 con 0x7f91340b0b60 2024-09-13T08:49:08.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.235+0000 7f913ba00640 1 -- 172.21.15.44:0/4131385319 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone632019", "target": ["mon-mgr", ""]}) -- 0x7f9134070350 con 0x7f9114074540 2024-09-13T08:49:08.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.255+0000 7f9130c00640 1 -- 172.21.15.44:0/4131385319 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9134070350 con 0x7f9114074540 2024-09-13T08:49:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 -- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9114074540 msgr2=0x7f9114076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9114074540 0x7f9114076a00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f91340afbb0 tx=0x7f9120009040 comp rx=0 tx=0).stop 2024-09-13T08:49:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 -- 172.21.15.44:0/4131385319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91340b0b60 msgr2=0x7f913415af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91340b0b60 0x7f913415af50 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f9128009790 tx=0x7f91280a5650 comp rx=0 tx=0).stop 2024-09-13T08:49:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 -- 172.21.15.44:0/4131385319 shutdown_connections 2024-09-13T08:49:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9114074540 0x7f9114076a00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f913415c1f0 0x7f9134153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f913415be10 0x7f91340c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 --2- 172.21.15.44:0/4131385319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91340b0b60 0x7f913415af50 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 -- 172.21.15.44:0/4131385319 >> 172.21.15.44:0/4131385319 conn(0x7f91340d1da0 msgr2=0x7f91340cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 -- 172.21.15.44:0/4131385319 shutdown_connections 2024-09-13T08:49:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.259+0000 7f913ba00640 1 -- 172.21.15.44:0/4131385319 wait complete. 2024-09-13T08:49:08.285 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8848 2024-09-13T08:49:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 -- 172.21.15.44:0/2623992101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb80b5170 msgr2=0x7f4cb80aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 --2- 172.21.15.44:0/2623992101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb80b5170 0x7f4cb80aedf0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009900 tx=0x7f4ca401c810 comp rx=0 tx=0).stop 2024-09-13T08:49:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 -- 172.21.15.44:0/2623992101 shutdown_connections 2024-09-13T08:49:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 --2- 172.21.15.44:0/2623992101 >> v2:172.21.15.129:3300/0 conn(0x7f4cb80af330 0x7f4cb80cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 --2- 172.21.15.44:0/2623992101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb80b5170 0x7f4cb80aedf0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 -- 172.21.15.44:0/2623992101 >> 172.21.15.44:0/2623992101 conn(0x7f4cb80cfbc0 msgr2=0x7f4cb80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 -- 172.21.15.44:0/2623992101 shutdown_connections 2024-09-13T08:49:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 -- 172.21.15.44:0/2623992101 wait complete. 2024-09-13T08:49:08.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 Processor -- start 2024-09-13T08:49:08.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 -- start start 2024-09-13T08:49:08.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb80af330 0x7f4cb815f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4cb80b5170 0x7f4cb81fbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cb815bf60 0x7f4cb806ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb8169240 con 0x7f4cb80af330 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb81690c0 con 0x7f4cb815bf60 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbf800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb81693c0 con 0x7f4cb80b5170 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.435+0000 7f4cbce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb80af330 0x7f4cb815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb80af330 0x7f4cb815f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60026/0 (socket says 172.21.15.44:60026) 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbce00640 1 -- 172.21.15.44:0/1243924194 learned_addr learned my addr 172.21.15.44:0/1243924194 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cb7e00640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4cb80b5170 0x7f4cb81fbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbd800640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cb815bf60 0x7f4cb806ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbce00640 1 -- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4cb80b5170 msgr2=0x7f4cb81fbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbce00640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4cb80b5170 0x7f4cb81fbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbce00640 1 -- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cb815bf60 msgr2=0x7f4cb806ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbce00640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cb815bf60 0x7f4cb806ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbce00640 1 -- 172.21.15.44:0/1243924194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb81f8110 con 0x7f4cb80af330 2024-09-13T08:49:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbd800640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cb815bf60 0x7f4cb806ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbce00640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb80af330 0x7f4cb815f140 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00c3f0 tx=0x7f4cac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cb4c00640 1 -- 172.21.15.44:0/1243924194 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac017070 con 0x7f4cb80af330 2024-09-13T08:49:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cb4c00640 1 -- 172.21.15.44:0/1243924194 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cac0053c0 con 0x7f4cb80af330 2024-09-13T08:49:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cb4c00640 1 -- 172.21.15.44:0/1243924194 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac003860 con 0x7f4cb80af330 2024-09-13T08:49:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbf800640 1 -- 172.21.15.44:0/1243924194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb81f83f0 con 0x7f4cb80af330 2024-09-13T08:49:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbf800640 1 -- 172.21.15.44:0/1243924194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb81f89b0 con 0x7f4cb80af330 2024-09-13T08:49:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cbf800640 1 -- 172.21.15.44:0/1243924194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb8069410 con 0x7f4cb80af330 2024-09-13T08:49:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.439+0000 7f4cb4c00640 1 -- 172.21.15.44:0/1243924194 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f4cac01a070 con 0x7f4cb80af330 2024-09-13T08:49:08.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.443+0000 7f4cb4c00640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c98074540 0x7f4c98076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.443+0000 7f4cb4c00640 1 -- 172.21.15.44:0/1243924194 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5849+0+0 (secure 0 0 0) 0x7f4cac091e40 con 0x7f4cb80af330 2024-09-13T08:49:08.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.443+0000 7f4cb7e00640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c98074540 0x7f4c98076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.443+0000 7f4cb7e00640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c98074540 0x7f4c98076a00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40af000 tx=0x7f4ca401b050 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:08.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.447+0000 7f4cb4c00640 1 -- 172.21.15.44:0/1243924194 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4cac05e4a0 con 0x7f4cb80af330 2024-09-13T08:49:08.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.643+0000 7f4cbf800640 1 -- 172.21.15.44:0/1243924194 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8848", "target": ["mon-mgr", ""]}) -- 0x7f4cb81fc440 con 0x7f4c98074540 2024-09-13T08:49:08.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cb4c00640 1 -- 172.21.15.44:0/1243924194 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4cb81fc440 con 0x7f4c98074540 2024-09-13T08:49:08.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cbf800640 1 -- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c98074540 msgr2=0x7f4c98076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cbf800640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c98074540 0x7f4c98076a00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40af000 tx=0x7f4ca401b050 comp rx=0 tx=0).stop 2024-09-13T08:49:08.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cbf800640 1 -- 172.21.15.44:0/1243924194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb80af330 msgr2=0x7f4cb815f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cbf800640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb80af330 0x7f4cb815f140 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00c3f0 tx=0x7f4cac00c8b0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cbf800640 1 -- 172.21.15.44:0/1243924194 shutdown_connections 2024-09-13T08:49:08.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cbf800640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c98074540 0x7f4c98076a00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cbf800640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cb815bf60 0x7f4cb806ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cbf800640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4cb80b5170 0x7f4cb81fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cbf800640 1 --2- 172.21.15.44:0/1243924194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb80af330 0x7f4cb815f140 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.667+0000 7f4cbf800640 1 -- 172.21.15.44:0/1243924194 >> 172.21.15.44:0/1243924194 conn(0x7f4cb80cfbc0 msgr2=0x7f4cb8165ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:08.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.671+0000 7f4cbf800640 1 -- 172.21.15.44:0/1243924194 shutdown_connections 2024-09-13T08:49:08.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.671+0000 7f4cbf800640 1 -- 172.21.15.44:0/1243924194 wait complete. 2024-09-13T08:49:08.694 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone58715 2024-09-13T08:49:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.843+0000 7f0d8ac00640 1 -- 172.21.15.44:0/4231351357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d84068960 msgr2=0x7f0d8415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.843+0000 7f0d8ac00640 1 --2- 172.21.15.44:0/4231351357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d84068960 0x7f0d8415dfc0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f0d840cf920 tx=0x7f0d6c01c8c0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.843+0000 7f0d8ac00640 1 -- 172.21.15.44:0/4231351357 shutdown_connections 2024-09-13T08:49:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.843+0000 7f0d8ac00640 1 --2- 172.21.15.44:0/4231351357 >> v2:172.21.15.129:3300/0 conn(0x7f0d84068dc0 0x7f0d8415e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.843+0000 7f0d8ac00640 1 --2- 172.21.15.44:0/4231351357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d84068960 0x7f0d8415dfc0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.843+0000 7f0d8ac00640 1 --2- 172.21.15.44:0/4231351357 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d840ce050 0x7f0d8415d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.843+0000 7f0d8ac00640 1 -- 172.21.15.44:0/4231351357 >> 172.21.15.44:0/4231351357 conn(0x7f0d840b2060 msgr2=0x7f0d840b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.843+0000 7f0d8ac00640 1 -- 172.21.15.44:0/4231351357 shutdown_connections 2024-09-13T08:49:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.843+0000 7f0d8ac00640 1 -- 172.21.15.44:0/4231351357 wait complete. 2024-09-13T08:49:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d8ac00640 1 Processor -- start 2024-09-13T08:49:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d8ac00640 1 -- start start 2024-09-13T08:49:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d8ac00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d84068960 0x7f0d841f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d8ac00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d84068dc0 0x7f0d8406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d8ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d840ce050 0x7f0d84155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d8ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d840cf410 con 0x7f0d840ce050 2024-09-13T08:49:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d8ac00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d840cf290 con 0x7f0d84068dc0 2024-09-13T08:49:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d8ac00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0d84167360 con 0x7f0d84068960 2024-09-13T08:49:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d88c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d840ce050 0x7f0d84155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d88c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d840ce050 0x7f0d84155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60044/0 (socket says 172.21.15.44:60044) 2024-09-13T08:49:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d88c00640 1 -- 172.21.15.44:0/2345688310 learned_addr learned my addr 172.21.15.44:0/2345688310 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d83e00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d84068960 0x7f0d841f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d7b400640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d84068dc0 0x7f0d8406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d88c00640 1 -- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d84068960 msgr2=0x7f0d841f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d88c00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d84068960 0x7f0d841f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d88c00640 1 -- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d84068dc0 msgr2=0x7f0d8406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d88c00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d84068dc0 0x7f0d8406f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d88c00640 1 -- 172.21.15.44:0/2345688310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d840b6290 con 0x7f0d840ce050 2024-09-13T08:49:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d88c00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d840ce050 0x7f0d84155000 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00d990 tx=0x7f0d7c00de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d80c00640 1 -- 172.21.15.44:0/2345688310 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c016070 con 0x7f0d840ce050 2024-09-13T08:49:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d80c00640 1 -- 172.21.15.44:0/2345688310 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d7c00bde0 con 0x7f0d840ce050 2024-09-13T08:49:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d8ac00640 1 -- 172.21.15.44:0/2345688310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d840b5a50 con 0x7f0d840ce050 2024-09-13T08:49:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d80c00640 1 -- 172.21.15.44:0/2345688310 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0106c0 con 0x7f0d840ce050 2024-09-13T08:49:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.847+0000 7f0d8ac00640 1 -- 172.21.15.44:0/2345688310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d840b5cd0 con 0x7f0d840ce050 2024-09-13T08:49:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.851+0000 7f0d8ac00640 1 -- 172.21.15.44:0/2345688310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8415d960 con 0x7f0d840ce050 2024-09-13T08:49:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.851+0000 7f0d80c00640 1 -- 172.21.15.44:0/2345688310 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f0d7c019070 con 0x7f0d840ce050 2024-09-13T08:49:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.851+0000 7f0d80c00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d60074570 0x7f0d60076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.851+0000 7f0d80c00640 1 -- 172.21.15.44:0/2345688310 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5849+0+0 (secure 0 0 0) 0x7f0d7c0931f0 con 0x7f0d840ce050 2024-09-13T08:49:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.851+0000 7f0d83e00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d60074570 0x7f0d60076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.851+0000 7f0d83e00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d60074570 0x7f0d60076a30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0d84071240 tx=0x7f0d740091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:08.855+0000 7f0d80c00640 1 -- 172.21.15.44:0/2345688310 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0d840b5a50 con 0x7f0d840ce050 2024-09-13T08:49:09.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.055+0000 7f0d8ac00640 1 -- 172.21.15.44:0/2345688310 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone58715", "target": ["mon-mgr", ""]}) -- 0x7f0d84074800 con 0x7f0d60074570 2024-09-13T08:49:09.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.075+0000 7f0d80c00640 1 -- 172.21.15.44:0/2345688310 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d84074800 con 0x7f0d60074570 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 -- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d60074570 msgr2=0x7f0d60076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d60074570 0x7f0d60076a30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0d84071240 tx=0x7f0d740091e0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 -- 172.21.15.44:0/2345688310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d840ce050 msgr2=0x7f0d84155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d840ce050 0x7f0d84155000 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00d990 tx=0x7f0d7c00de50 comp rx=0 tx=0).stop 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 -- 172.21.15.44:0/2345688310 shutdown_connections 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d60074570 0x7f0d60076a30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d840ce050 0x7f0d84155000 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d84068dc0 0x7f0d8406f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 --2- 172.21.15.44:0/2345688310 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d84068960 0x7f0d841f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 -- 172.21.15.44:0/2345688310 >> 172.21.15.44:0/2345688310 conn(0x7f0d840b2060 msgr2=0x7f0d84073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 -- 172.21.15.44:0/2345688310 shutdown_connections 2024-09-13T08:49:09.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.079+0000 7f0d8ac00640 1 -- 172.21.15.44:0/2345688310 wait complete. 2024-09-13T08:49:09.102 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone73477 2024-09-13T08:49:09.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 -- 172.21.15.44:0/905380123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c2c004d50 msgr2=0x7f4c2c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 --2- 172.21.15.44:0/905380123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c2c004d50 0x7f4c2c0051d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4c480738c0 tx=0x7f4c4001d1a0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c46800640 1 -- 172.21.15.44:0/905380123 reap_dead start 2024-09-13T08:49:09.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 -- 172.21.15.44:0/905380123 shutdown_connections 2024-09-13T08:49:09.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 -- 172.21.15.44:0/905380123 >> 172.21.15.44:0/905380123 conn(0x7f4c4806dd30 msgr2=0x7f4c4806e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:09.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 -- 172.21.15.44:0/905380123 shutdown_connections 2024-09-13T08:49:09.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 -- 172.21.15.44:0/905380123 wait complete. 2024-09-13T08:49:09.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 Processor -- start 2024-09-13T08:49:09.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 -- start start 2024-09-13T08:49:09.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c2c004d50 0x7f4c481486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:09.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c48158720 0x7f4c48149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:09.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c48149f90 0x7f4c4806f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:09.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4c48167cb0 con 0x7f4c48149f90 2024-09-13T08:49:09.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c48167b30 con 0x7f4c48158720 2024-09-13T08:49:09.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.255+0000 7f4c4d000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4c48167e30 con 0x7f4c2c004d50 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c45e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c2c004d50 0x7f4c481486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c46800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c48149f90 0x7f4c4806f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c45e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c2c004d50 0x7f4c481486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:46468/0 (socket says 172.21.15.44:46468) 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c3d400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c48158720 0x7f4c48149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c45e00640 1 -- 172.21.15.44:0/4278518847 learned_addr learned my addr 172.21.15.44:0/4278518847 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c46800640 1 -- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c2c004d50 msgr2=0x7f4c481486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c46800640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c2c004d50 0x7f4c481486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c46800640 1 -- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c48158720 msgr2=0x7f4c48149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c46800640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c48158720 0x7f4c48149a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c46800640 1 -- 172.21.15.44:0/4278518847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c480b0a30 con 0x7f4c48149f90 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c46800640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c48149f90 0x7f4c4806f690 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00bb90 tx=0x7f4c400add70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c3ea00640 1 -- 172.21.15.44:0/4278518847 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c400b1040 con 0x7f4c48149f90 2024-09-13T08:49:09.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c3ea00640 1 -- 172.21.15.44:0/4278518847 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c400be070 con 0x7f4c48149f90 2024-09-13T08:49:09.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c4d000640 1 -- 172.21.15.44:0/4278518847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c480b0d10 con 0x7f4c48149f90 2024-09-13T08:49:09.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c3ea00640 1 -- 172.21.15.44:0/4278518847 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40009730 con 0x7f4c48149f90 2024-09-13T08:49:09.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c4d000640 1 -- 172.21.15.44:0/4278518847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c480b12d0 con 0x7f4c48149f90 2024-09-13T08:49:09.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c4d000640 1 -- 172.21.15.44:0/4278518847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4815c070 con 0x7f4c48149f90 2024-09-13T08:49:09.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.259+0000 7f4c3ea00640 1 -- 172.21.15.44:0/4278518847 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f4c400b7470 con 0x7f4c48149f90 2024-09-13T08:49:09.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.263+0000 7f4c3ea00640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c24074570 0x7f4c24076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:09.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.263+0000 7f4c3ea00640 1 -- 172.21.15.44:0/4278518847 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f4c4012dee0 con 0x7f4c48149f90 2024-09-13T08:49:09.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.263+0000 7f4c45e00640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c24074570 0x7f4c24076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:09.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.263+0000 7f4c45e00640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c24074570 0x7f4c24076a30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4c38006260 tx=0x7f4c38006f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.267+0000 7f4c3ea00640 1 -- 172.21.15.44:0/4278518847 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4c40016ee0 con 0x7f4c48149f90 2024-09-13T08:49:09.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.531+0000 7f4c4d000640 1 -- 172.21.15.44:0/4278518847 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone73477", "target": ["mon-mgr", ""]}) -- 0x7f4c4806adb0 con 0x7f4c24074570 2024-09-13T08:49:09.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c3ea00640 1 -- 172.21.15.44:0/4278518847 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4c4806adb0 con 0x7f4c24074570 2024-09-13T08:49:09.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 -- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c24074570 msgr2=0x7f4c24076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c24074570 0x7f4c24076a30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4c38006260 tx=0x7f4c38006f00 comp rx=0 tx=0).stop 2024-09-13T08:49:09.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 -- 172.21.15.44:0/4278518847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c48149f90 msgr2=0x7f4c4806f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c48149f90 0x7f4c4806f690 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00bb90 tx=0x7f4c400add70 comp rx=0 tx=0).stop 2024-09-13T08:49:09.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 -- 172.21.15.44:0/4278518847 shutdown_connections 2024-09-13T08:49:09.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c24074570 0x7f4c24076a30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c48149f90 0x7f4c4806f690 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c48158720 0x7f4c48149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 --2- 172.21.15.44:0/4278518847 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c2c004d50 0x7f4c481486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 -- 172.21.15.44:0/4278518847 >> 172.21.15.44:0/4278518847 conn(0x7f4c4806dd30 msgr2=0x7f4c4814cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:09.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 -- 172.21.15.44:0/4278518847 shutdown_connections 2024-09-13T08:49:09.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.559+0000 7f4c4d000640 1 -- 172.21.15.44:0/4278518847 wait complete. 2024-09-13T08:49:09.584 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone268612 2024-09-13T08:49:09.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 -- 172.21.15.44:0/3648130443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f180b0b60 msgr2=0x7f4f180ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 --2- 172.21.15.44:0/3648130443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f180b0b60 0x7f4f180ccbd0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009c80 tx=0x7f4f1001f5a0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 -- 172.21.15.44:0/3648130443 shutdown_connections 2024-09-13T08:49:09.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 --2- 172.21.15.44:0/3648130443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f180b0b60 0x7f4f180ccbd0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 --2- 172.21.15.44:0/3648130443 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f1815be10 0x7f4f180ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 -- 172.21.15.44:0/3648130443 >> 172.21.15.44:0/3648130443 conn(0x7f4f180d1da0 msgr2=0x7f4f180d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:09.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 -- 172.21.15.44:0/3648130443 shutdown_connections 2024-09-13T08:49:09.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 -- 172.21.15.44:0/3648130443 wait complete. 2024-09-13T08:49:09.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 Processor -- start 2024-09-13T08:49:09.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 -- start start 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f180b0b60 0x7f4f1806e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f1815be10 0x7f4f18156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f1815c1f0 0x7f4f181f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4f180cd4a0 con 0x7f4f1815c1f0 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f180cd320 con 0x7f4f1815be10 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1f800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4f180cd620 con 0x7f4f180b0b60 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f1815c1f0 0x7f4f181f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1ce00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f180b0b60 0x7f4f1806e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f1815c1f0 0x7f4f181f4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60084/0 (socket says 172.21.15.44:60084) 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1d800640 1 -- 172.21.15.44:0/3228284352 learned_addr learned my addr 172.21.15.44:0/3228284352 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f0fe00640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f1815be10 0x7f4f18156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:09.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1ce00640 1 -- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f1815be10 msgr2=0x7f4f18156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1ce00640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f1815be10 0x7f4f18156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1ce00640 1 -- 172.21.15.44:0/3228284352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f1815c1f0 msgr2=0x7f4f181f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1ce00640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f1815c1f0 0x7f4f181f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.735+0000 7f4f1ce00640 1 -- 172.21.15.44:0/3228284352 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f180b6820 con 0x7f4f180b0b60 2024-09-13T08:49:09.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f1ce00640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f180b0b60 0x7f4f1806e690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800e9f0 tx=0x7f4f0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:09.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f15600640 1 -- 172.21.15.44:0/3228284352 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800ccc0 con 0x7f4f180b0b60 2024-09-13T08:49:09.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f15600640 1 -- 172.21.15.44:0/3228284352 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f08010430 con 0x7f4f180b0b60 2024-09-13T08:49:09.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f15600640 1 -- 172.21.15.44:0/3228284352 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08010860 con 0x7f4f180b0b60 2024-09-13T08:49:09.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f1f800640 1 -- 172.21.15.44:0/3228284352 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1806deb0 con 0x7f4f180b0b60 2024-09-13T08:49:09.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f1f800640 1 -- 172.21.15.44:0/3228284352 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f181f5650 con 0x7f4f180b0b60 2024-09-13T08:49:09.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f1f800640 1 -- 172.21.15.44:0/3228284352 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f180cdb10 con 0x7f4f180b0b60 2024-09-13T08:49:09.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f15600640 1 -- 172.21.15.44:0/3228284352 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f4f08010a00 con 0x7f4f180b0b60 2024-09-13T08:49:09.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f15600640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ef4074540 0x7f4ef4076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:09.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f15600640 1 -- 172.21.15.44:0/3228284352 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f4f08093610 con 0x7f4f180b0b60 2024-09-13T08:49:09.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.739+0000 7f4f0fe00640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ef4074540 0x7f4ef4076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:09.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.743+0000 7f4f0fe00640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ef4074540 0x7f4ef4076a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4f1809fc90 tx=0x7f4f00009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:09.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.743+0000 7f4f15600640 1 -- 172.21.15.44:0/3228284352 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4f0805fc70 con 0x7f4f180b0b60 2024-09-13T08:49:09.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.943+0000 7f4f1f800640 1 -- 172.21.15.44:0/3228284352 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone268612", "target": ["mon-mgr", ""]}) -- 0x7f4f18070350 con 0x7f4ef4074540 2024-09-13T08:49:09.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.967+0000 7f4f15600640 1 -- 172.21.15.44:0/3228284352 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4f18070350 con 0x7f4ef4074540 2024-09-13T08:49:09.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 -- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ef4074540 msgr2=0x7f4ef4076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ef4074540 0x7f4ef4076a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4f1809fc90 tx=0x7f4f00009040 comp rx=0 tx=0).stop 2024-09-13T08:49:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 -- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f180b0b60 msgr2=0x7f4f1806e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f180b0b60 0x7f4f1806e690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800e9f0 tx=0x7f4f0800eeb0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 -- 172.21.15.44:0/3228284352 shutdown_connections 2024-09-13T08:49:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ef4074540 0x7f4ef4076a00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f1815c1f0 0x7f4f181f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f1815be10 0x7f4f18156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 --2- 172.21.15.44:0/3228284352 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f180b0b60 0x7f4f1806e690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 -- 172.21.15.44:0/3228284352 >> 172.21.15.44:0/3228284352 conn(0x7f4f180d1da0 msgr2=0x7f4f180ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 -- 172.21.15.44:0/3228284352 shutdown_connections 2024-09-13T08:49:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:09.971+0000 7f4f1f800640 1 -- 172.21.15.44:0/3228284352 wait complete. 2024-09-13T08:49:09.994 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone677511 2024-09-13T08:49:10.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.143+0000 7f154b200640 1 -- 172.21.15.44:0/2105721933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154415be10 msgr2=0x7f15440ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:10.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.143+0000 7f154b200640 1 --2- 172.21.15.44:0/2105721933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154415be10 0x7f15440ce700 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f15440746b0 tx=0x7f153401eff0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.143+0000 7f154b200640 1 -- 172.21.15.44:0/2105721933 shutdown_connections 2024-09-13T08:49:10.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.143+0000 7f154b200640 1 --2- 172.21.15.44:0/2105721933 >> v2:172.21.15.129:3300/0 conn(0x7f154415c1f0 0x7f15440cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.143+0000 7f154b200640 1 --2- 172.21.15.44:0/2105721933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154415be10 0x7f15440ce700 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.143+0000 7f154b200640 1 -- 172.21.15.44:0/2105721933 >> 172.21.15.44:0/2105721933 conn(0x7f15440d1da0 msgr2=0x7f15440d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:10.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f154b200640 1 -- 172.21.15.44:0/2105721933 shutdown_connections 2024-09-13T08:49:10.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f154b200640 1 -- 172.21.15.44:0/2105721933 wait complete. 2024-09-13T08:49:10.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f154b200640 1 Processor -- start 2024-09-13T08:49:10.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f154b200640 1 -- start start 2024-09-13T08:49:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f154b200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f15440b0b60 0x7f1544153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f154b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154415be10 0x7f15440c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f154b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154415c1f0 0x7f15440c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f154b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15440cf510 con 0x7f154415be10 2024-09-13T08:49:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f154b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15440cf390 con 0x7f154415c1f0 2024-09-13T08:49:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f154b200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1544160290 con 0x7f15440b0b60 2024-09-13T08:49:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1543400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154415be10 0x7f15440c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1543400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154415be10 0x7f15440c5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60112/0 (socket says 172.21.15.44:60112) 2024-09-13T08:49:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1543400640 1 -- 172.21.15.44:0/126999309 learned_addr learned my addr 172.21.15.44:0/126999309 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1549200640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154415c1f0 0x7f15440c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1543e00640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f15440b0b60 0x7f1544153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1543400640 1 -- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f15440b0b60 msgr2=0x7f1544153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1543400640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f15440b0b60 0x7f1544153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1543400640 1 -- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154415c1f0 msgr2=0x7f15440c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1543400640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154415c1f0 0x7f15440c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1543400640 1 -- 172.21.15.44:0/126999309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15440cb940 con 0x7f154415be10 2024-09-13T08:49:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.147+0000 7f1543400640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154415be10 0x7f15440c5ba0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f153000e7a0 tx=0x7f153000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.151+0000 7f153be00640 1 -- 172.21.15.44:0/126999309 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153000ccc0 con 0x7f154415be10 2024-09-13T08:49:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.151+0000 7f153be00640 1 -- 172.21.15.44:0/126999309 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1530010430 con 0x7f154415be10 2024-09-13T08:49:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.151+0000 7f154b200640 1 -- 172.21.15.44:0/126999309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15440b2020 con 0x7f154415be10 2024-09-13T08:49:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.151+0000 7f153be00640 1 -- 172.21.15.44:0/126999309 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530010860 con 0x7f154415be10 2024-09-13T08:49:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.151+0000 7f154b200640 1 -- 172.21.15.44:0/126999309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15440b25e0 con 0x7f154415be10 2024-09-13T08:49:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.151+0000 7f154b200640 1 -- 172.21.15.44:0/126999309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15441607b0 con 0x7f154415be10 2024-09-13T08:49:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.151+0000 7f153be00640 1 -- 172.21.15.44:0/126999309 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f1530010a00 con 0x7f154415be10 2024-09-13T08:49:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.151+0000 7f153be00640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1524074570 0x7f1524076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.151+0000 7f153be00640 1 -- 172.21.15.44:0/126999309 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f1530093450 con 0x7f154415be10 2024-09-13T08:49:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.151+0000 7f1543e00640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1524074570 0x7f1524076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.155+0000 7f1543e00640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1524074570 0x7f1524076a30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f15340b2000 tx=0x7f153401d830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.155+0000 7f153be00640 1 -- 172.21.15.44:0/126999309 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f153005fab0 con 0x7f154415be10 2024-09-13T08:49:10.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.355+0000 7f154b200640 1 -- 172.21.15.44:0/126999309 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone677511", "target": ["mon-mgr", ""]}) -- 0x7f15440b29e0 con 0x7f1524074570 2024-09-13T08:49:10.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.375+0000 7f153be00640 1 -- 172.21.15.44:0/126999309 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f15440b29e0 con 0x7f1524074570 2024-09-13T08:49:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 -- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1524074570 msgr2=0x7f1524076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1524074570 0x7f1524076a30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f15340b2000 tx=0x7f153401d830 comp rx=0 tx=0).stop 2024-09-13T08:49:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 -- 172.21.15.44:0/126999309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154415be10 msgr2=0x7f15440c5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154415be10 0x7f15440c5ba0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f153000e7a0 tx=0x7f153000ec60 comp rx=0 tx=0).stop 2024-09-13T08:49:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 -- 172.21.15.44:0/126999309 shutdown_connections 2024-09-13T08:49:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1524074570 0x7f1524076a30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154415c1f0 0x7f15440c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f154415be10 0x7f15440c5ba0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 --2- 172.21.15.44:0/126999309 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f15440b0b60 0x7f1544153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 -- 172.21.15.44:0/126999309 >> 172.21.15.44:0/126999309 conn(0x7f15440d1da0 msgr2=0x7f15440ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:10.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 -- 172.21.15.44:0/126999309 shutdown_connections 2024-09-13T08:49:10.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.379+0000 7f154b200640 1 -- 172.21.15.44:0/126999309 wait complete. 2024-09-13T08:49:10.402 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone11943 2024-09-13T08:49:10.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.643+0000 7f1543600640 1 -- 172.21.15.44:0/2887149523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f153c15c1f0 msgr2=0x7f153c0cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:10.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.643+0000 7f1543600640 1 --2- 172.21.15.44:0/2887149523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f153c15c1f0 0x7f153c0cee10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f153c0746b0 tx=0x7f152801ee30 comp rx=0 tx=0).stop 2024-09-13T08:49:10.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.647+0000 7f1543600640 1 -- 172.21.15.44:0/2887149523 shutdown_connections 2024-09-13T08:49:10.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.647+0000 7f1543600640 1 --2- 172.21.15.44:0/2887149523 >> v2:172.21.15.44:3300/0 conn(0x7f153c0b0b60 0x7f153c0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.647+0000 7f1543600640 1 --2- 172.21.15.44:0/2887149523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f153c15c1f0 0x7f153c0cee10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.647+0000 7f1543600640 1 -- 172.21.15.44:0/2887149523 >> 172.21.15.44:0/2887149523 conn(0x7f153c0d1da0 msgr2=0x7f153c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:10.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.647+0000 7f1543600640 1 -- 172.21.15.44:0/2887149523 shutdown_connections 2024-09-13T08:49:10.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.647+0000 7f1543600640 1 -- 172.21.15.44:0/2887149523 wait complete. 2024-09-13T08:49:10.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1543600640 1 Processor -- start 2024-09-13T08:49:10.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1543600640 1 -- start start 2024-09-13T08:49:10.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1543600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f153c0b0b60 0x7f153c0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:10.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1543600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153c15be10 0x7f153c0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:10.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1543600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153c15c1f0 0x7f153c15d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:10.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1543600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f153c0cd4a0 con 0x7f153c15be10 2024-09-13T08:49:10.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1543600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f153c0cd320 con 0x7f153c0b0b60 2024-09-13T08:49:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1543600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f153c0cd620 con 0x7f153c15c1f0 2024-09-13T08:49:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1540c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f153c0b0b60 0x7f153c0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f153be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153c15be10 0x7f153c0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1540c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f153c0b0b60 0x7f153c0c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:58970/0 (socket says 172.21.15.44:58970) 2024-09-13T08:49:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1540c00640 1 -- 172.21.15.44:0/3648223166 learned_addr learned my addr 172.21.15.44:0/3648223166 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.651+0000 7f1541600640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153c15c1f0 0x7f153c15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f153be00640 1 -- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153c15c1f0 msgr2=0x7f153c15d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:10.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f153be00640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153c15c1f0 0x7f153c15d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f153be00640 1 -- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f153c0b0b60 msgr2=0x7f153c0c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:10.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f153be00640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f153c0b0b60 0x7f153c0c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f153be00640 1 -- 172.21.15.44:0/3648223166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c0c7580 con 0x7f153c15be10 2024-09-13T08:49:10.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f153be00640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153c15be10 0x7f153c0b5d70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f15280af000 tx=0x7f1528005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:10.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f1538c00640 1 -- 172.21.15.44:0/3648223166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15280be070 con 0x7f153c15be10 2024-09-13T08:49:10.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f1538c00640 1 -- 172.21.15.44:0/3648223166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1528005400 con 0x7f153c15be10 2024-09-13T08:49:10.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f1538c00640 1 -- 172.21.15.44:0/3648223166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15280b5040 con 0x7f153c15be10 2024-09-13T08:49:10.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f1543600640 1 -- 172.21.15.44:0/3648223166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153c0b55b0 con 0x7f153c15be10 2024-09-13T08:49:10.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.655+0000 7f1543600640 1 -- 172.21.15.44:0/3648223166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153c0b7850 con 0x7f153c15be10 2024-09-13T08:49:10.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.659+0000 7f1543600640 1 -- 172.21.15.44:0/3648223166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153c0cdb40 con 0x7f153c15be10 2024-09-13T08:49:10.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.663+0000 7f1538c00640 1 -- 172.21.15.44:0/3648223166 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f15280046d0 con 0x7f153c15be10 2024-09-13T08:49:10.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.663+0000 7f1538c00640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f151c074560 0x7f151c076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:10.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.663+0000 7f1538c00640 1 -- 172.21.15.44:0/3648223166 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f1528130160 con 0x7f153c15be10 2024-09-13T08:49:10.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.667+0000 7f1540c00640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f151c074560 0x7f151c076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:10.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.667+0000 7f1540c00640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f151c074560 0x7f151c076a20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f152c000fc0 tx=0x7f152c010ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:10.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.671+0000 7f1538c00640 1 -- 172.21.15.44:0/3648223166 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f153c0c7580 con 0x7f153c15be10 2024-09-13T08:49:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.923+0000 7f1543600640 1 -- 172.21.15.44:0/3648223166 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone11943", "target": ["mon-mgr", ""]}) -- 0x7f153c070350 con 0x7f151c074560 2024-09-13T08:49:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.943+0000 7f1538c00640 1 -- 172.21.15.44:0/3648223166 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f153c070350 con 0x7f151c074560 2024-09-13T08:49:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 -- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f151c074560 msgr2=0x7f151c076a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f151c074560 0x7f151c076a20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f152c000fc0 tx=0x7f152c010ce0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 -- 172.21.15.44:0/3648223166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153c15be10 msgr2=0x7f153c0b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153c15be10 0x7f153c0b5d70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f15280af000 tx=0x7f1528005d20 comp rx=0 tx=0).stop 2024-09-13T08:49:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 -- 172.21.15.44:0/3648223166 shutdown_connections 2024-09-13T08:49:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f151c074560 0x7f151c076a20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f153c15c1f0 0x7f153c15d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f153c15be10 0x7f153c0b5d70 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 --2- 172.21.15.44:0/3648223166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f153c0b0b60 0x7f153c0c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 -- 172.21.15.44:0/3648223166 >> 172.21.15.44:0/3648223166 conn(0x7f153c0d1da0 msgr2=0x7f153c070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 -- 172.21.15.44:0/3648223166 shutdown_connections 2024-09-13T08:49:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:10.947+0000 7f1543600640 1 -- 172.21.15.44:0/3648223166 wait complete. 2024-09-13T08:49:10.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone42326 2024-09-13T08:49:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 -- 172.21.15.44:0/3909652263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b415a7e0 msgr2=0x7f56b40ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 --2- 172.21.15.44:0/3909652263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b415a7e0 0x7f56b40ae4d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f56b40b4020 tx=0x7f56a401c8a0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 -- 172.21.15.44:0/3909652263 shutdown_connections 2024-09-13T08:49:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 --2- 172.21.15.44:0/3909652263 >> v2:172.21.15.129:3300/0 conn(0x7f56b415abc0 0x7f56b40d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 --2- 172.21.15.44:0/3909652263 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b40aea10 0x7f56b40d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 --2- 172.21.15.44:0/3909652263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b415a7e0 0x7f56b40ae4d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 -- 172.21.15.44:0/3909652263 >> 172.21.15.44:0/3909652263 conn(0x7f56b406dd30 msgr2=0x7f56b406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 -- 172.21.15.44:0/3909652263 shutdown_connections 2024-09-13T08:49:11.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 -- 172.21.15.44:0/3909652263 wait complete. 2024-09-13T08:49:11.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 Processor -- start 2024-09-13T08:49:11.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.119+0000 7f56ba800640 1 -- start start 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ba800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b40aea10 0x7f56b41477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ba800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b415a7e0 0x7f56b41f8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b415abc0 0x7f56b41f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ba800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f56b415ee10 con 0x7f56b415abc0 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ba800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56b415ec90 con 0x7f56b415a7e0 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ba800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f56b415ef90 con 0x7f56b40aea10 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b415abc0 0x7f56b41f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b415abc0 0x7f56b41f88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60154/0 (socket says 172.21.15.44:60154) 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56b3e00640 1 -- 172.21.15.44:0/1690911282 learned_addr learned my addr 172.21.15.44:0/1690911282 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56abe00640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b415a7e0 0x7f56b41f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56b3400640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b40aea10 0x7f56b41477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56b3e00640 1 -- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b40aea10 msgr2=0x7f56b41477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56b3e00640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b40aea10 0x7f56b41477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56b3e00640 1 -- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b415a7e0 msgr2=0x7f56b41f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56b3e00640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b415a7e0 0x7f56b41f8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56b3e00640 1 -- 172.21.15.44:0/1690911282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b41f5880 con 0x7f56b415abc0 2024-09-13T08:49:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56abe00640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b415a7e0 0x7f56b41f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56b3e00640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b415abc0 0x7f56b41f88b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00da00 tx=0x7f56ac00dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:11.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ab400640 1 -- 172.21.15.44:0/1690911282 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac017070 con 0x7f56b415abc0 2024-09-13T08:49:11.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ab400640 1 -- 172.21.15.44:0/1690911282 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56ac004e20 con 0x7f56b415abc0 2024-09-13T08:49:11.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ab400640 1 -- 172.21.15.44:0/1690911282 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00bd50 con 0x7f56b415abc0 2024-09-13T08:49:11.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ba800640 1 -- 172.21.15.44:0/1690911282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b41f5b60 con 0x7f56b415abc0 2024-09-13T08:49:11.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ba800640 1 -- 172.21.15.44:0/1690911282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b415f750 con 0x7f56b415abc0 2024-09-13T08:49:11.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.123+0000 7f56ba800640 1 -- 172.21.15.44:0/1690911282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b40ae4d0 con 0x7f56b415abc0 2024-09-13T08:49:11.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.127+0000 7f56ab400640 1 -- 172.21.15.44:0/1690911282 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f56ac005000 con 0x7f56b415abc0 2024-09-13T08:49:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.127+0000 7f56ab400640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5690074540 0x7f5690076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.127+0000 7f56ab400640 1 -- 172.21.15.44:0/1690911282 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f56ac092dd0 con 0x7f56b415abc0 2024-09-13T08:49:11.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.127+0000 7f56b3400640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5690074540 0x7f5690076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:11.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.127+0000 7f56b3400640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5690074540 0x7f5690076a00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f56a40af000 tx=0x7f56a401b0e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:11.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.131+0000 7f56ab400640 1 -- 172.21.15.44:0/1690911282 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f56ac020070 con 0x7f56b415abc0 2024-09-13T08:49:11.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.335+0000 7f56ba800640 1 -- 172.21.15.44:0/1690911282 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone42326", "target": ["mon-mgr", ""]}) -- 0x7f56b406e1b0 con 0x7f5690074540 2024-09-13T08:49:11.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.351+0000 7f56ab400640 1 -- 172.21.15.44:0/1690911282 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f56b406e1b0 con 0x7f5690074540 2024-09-13T08:49:11.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 -- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5690074540 msgr2=0x7f5690076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:11.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5690074540 0x7f5690076a00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f56a40af000 tx=0x7f56a401b0e0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 -- 172.21.15.44:0/1690911282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b415abc0 msgr2=0x7f56b41f88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:11.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b415abc0 0x7f56b41f88b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00da00 tx=0x7f56ac00dec0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 -- 172.21.15.44:0/1690911282 shutdown_connections 2024-09-13T08:49:11.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5690074540 0x7f5690076a00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b415abc0 0x7f56b41f88b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b415a7e0 0x7f56b41f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 --2- 172.21.15.44:0/1690911282 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f56b40aea10 0x7f56b41477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 -- 172.21.15.44:0/1690911282 >> 172.21.15.44:0/1690911282 conn(0x7f56b406dd30 msgr2=0x7f56b4073cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:11.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 -- 172.21.15.44:0/1690911282 shutdown_connections 2024-09-13T08:49:11.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.355+0000 7f56ba800640 1 -- 172.21.15.44:0/1690911282 wait complete. 2024-09-13T08:49:11.377 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-13T08:49:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:49:11.445 INFO:teuthology.orchestra.run.smithi044.stdout:2 2024-09-13T08:49:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:49:11.457 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:49:11.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.715+0000 7f6dd8600640 1 -- 172.21.15.44:0/2617646143 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dd015ac70 msgr2=0x7f6dd00cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.715+0000 7f6dd8600640 1 --2- 172.21.15.44:0/2617646143 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dd015ac70 0x7f6dd00cf2a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc401c820 comp rx=0 tx=0).stop 2024-09-13T08:49:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.715+0000 7f6dd8600640 1 -- 172.21.15.44:0/2617646143 shutdown_connections 2024-09-13T08:49:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.715+0000 7f6dd8600640 1 --2- 172.21.15.44:0/2617646143 >> v2:172.21.15.44:3300/0 conn(0x7f6dd00cbdb0 0x7f6dd00b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.715+0000 7f6dd8600640 1 --2- 172.21.15.44:0/2617646143 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dd015ac70 0x7f6dd00cf2a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.715+0000 7f6dd8600640 1 -- 172.21.15.44:0/2617646143 >> 172.21.15.44:0/2617646143 conn(0x7f6dd00cd740 msgr2=0x7f6dd00cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:11.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.715+0000 7f6dd8600640 1 -- 172.21.15.44:0/2617646143 shutdown_connections 2024-09-13T08:49:11.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.715+0000 7f6dd8600640 1 -- 172.21.15.44:0/2617646143 wait complete. 2024-09-13T08:49:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd8600640 1 Processor -- start 2024-09-13T08:49:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd8600640 1 -- start start 2024-09-13T08:49:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd8600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dd00b2c80 0x7f6dd00ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd8600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dd00cbdb0 0x7f6dd00c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd8600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd015ac70 0x7f6dd00c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd8600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0167c00 con 0x7f6dd015ac70 2024-09-13T08:49:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd8600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0167a80 con 0x7f6dd00cbdb0 2024-09-13T08:49:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd8600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd0167d80 con 0x7f6dd00b2c80 2024-09-13T08:49:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd5c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dd00b2c80 0x7f6dd00ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd6600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd015ac70 0x7f6dd00c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd5200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dd00cbdb0 0x7f6dd00c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd5200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dd00cbdb0 0x7f6dd00c3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:58994/0 (socket says 172.21.15.44:58994) 2024-09-13T08:49:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd5200640 1 -- 172.21.15.44:0/3689479556 learned_addr learned my addr 172.21.15.44:0/3689479556 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.719+0000 7f6dd6600640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd015ac70 0x7f6dd00c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60184/0 (socket says 172.21.15.44:60184) 2024-09-13T08:49:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dd5c00640 1 -- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dd00cbdb0 msgr2=0x7f6dd00c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dd5c00640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dd00cbdb0 0x7f6dd00c3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dd5c00640 1 -- 172.21.15.44:0/3689479556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd015ac70 msgr2=0x7f6dd00c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dd5c00640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd015ac70 0x7f6dd00c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dd5c00640 1 -- 172.21.15.44:0/3689479556 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd01ff110 con 0x7f6dd00b2c80 2024-09-13T08:49:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dd6600640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd015ac70 0x7f6dd00c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:49:11.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dd5c00640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dd00b2c80 0x7f6dd00ca150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40ac000 tx=0x7f6dc40a6700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:11.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dcd600640 1 -- 172.21.15.44:0/3689479556 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4004380 con 0x7f6dd00b2c80 2024-09-13T08:49:11.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dcd600640 1 -- 172.21.15.44:0/3689479556 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dc4005070 con 0x7f6dd00b2c80 2024-09-13T08:49:11.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dcd600640 1 -- 172.21.15.44:0/3689479556 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40054c0 con 0x7f6dd00b2c80 2024-09-13T08:49:11.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dd8600640 1 -- 172.21.15.44:0/3689479556 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd01ff3f0 con 0x7f6dd00b2c80 2024-09-13T08:49:11.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.723+0000 7f6dd8600640 1 -- 172.21.15.44:0/3689479556 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd01ff9b0 con 0x7f6dd00b2c80 2024-09-13T08:49:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.727+0000 7f6dd8600640 1 -- 172.21.15.44:0/3689479556 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd00c4590 con 0x7f6dd00b2c80 2024-09-13T08:49:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.731+0000 7f6dcd600640 1 -- 172.21.15.44:0/3689479556 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f6dc40a6870 con 0x7f6dd00b2c80 2024-09-13T08:49:11.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.731+0000 7f6dcd600640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6db0074540 0x7f6db0076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:11.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.731+0000 7f6dcd600640 1 -- 172.21.15.44:0/3689479556 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f6dc40bd070 con 0x7f6dd00b2c80 2024-09-13T08:49:11.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.731+0000 7f6dd5200640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6db0074540 0x7f6db0076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.735+0000 7f6dd5200640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6db0074540 0x7f6db0076a00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00b17b0 tx=0x7f6dbc00b440 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:11.735+0000 7f6dcd600640 1 -- 172.21.15.44:0/3689479556 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6dc4016410 con 0x7f6dd00b2c80 2024-09-13T08:49:12.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.099+0000 7f6dd8600640 1 -- 172.21.15.44:0/3689479556 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f6dd0070e50 con 0x7f6dd00b2c80 2024-09-13T08:49:12.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.099+0000 7f6dcd600640 1 -- 172.21.15.44:0/3689479556 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v43)=0 v43) ==== 131+0+0 (secure 0 0 0) 0x7f6dc40f92f0 con 0x7f6dd00b2c80 2024-09-13T08:49:12.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 -- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6db0074540 msgr2=0x7f6db0076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6db0074540 0x7f6db0076a00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00b17b0 tx=0x7f6dbc00b440 comp rx=0 tx=0).stop 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 -- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dd00b2c80 msgr2=0x7f6dd00ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dd00b2c80 0x7f6dd00ca150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40ac000 tx=0x7f6dc40a6700 comp rx=0 tx=0).stop 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 -- 172.21.15.44:0/3689479556 shutdown_connections 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6db0074540 0x7f6db0076a00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd015ac70 0x7f6dd00c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dd00cbdb0 0x7f6dd00c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 --2- 172.21.15.44:0/3689479556 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dd00b2c80 0x7f6dd00ca150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 -- 172.21.15.44:0/3689479556 >> 172.21.15.44:0/3689479556 conn(0x7f6dd00cd740 msgr2=0x7f6dd0075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 -- 172.21.15.44:0/3689479556 shutdown_connections 2024-09-13T08:49:12.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.107+0000 7f6dd8600640 1 -- 172.21.15.44:0/3689479556 wait complete. 2024-09-13T08:49:12.136 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:49:12.136 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:49:12.222 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:49:12.310 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:49:12.310 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:49:12.310 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:49:12.310 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:12.319 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:49:12.319 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:12.371 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:49:12.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.667+0000 7f015f200640 1 -- 172.21.15.44:0/1936232747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01540087f0 msgr2=0x7f01540e1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:12.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.667+0000 7f015f200640 1 --2- 172.21.15.44:0/1936232747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01540087f0 0x7f01540e1ac0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0148009900 tx=0x7f014801c8e0 comp rx=0 tx=0).stop 2024-09-13T08:49:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.667+0000 7f015f200640 1 -- 172.21.15.44:0/1936232747 shutdown_connections 2024-09-13T08:49:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.667+0000 7f015f200640 1 --2- 172.21.15.44:0/1936232747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01540087f0 0x7f01540e1ac0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.667+0000 7f015f200640 1 --2- 172.21.15.44:0/1936232747 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0154008410 0x7f01540e1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.667+0000 7f015f200640 1 -- 172.21.15.44:0/1936232747 >> 172.21.15.44:0/1936232747 conn(0x7f0154056800 msgr2=0x7f0154056be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.667+0000 7f015f200640 1 -- 172.21.15.44:0/1936232747 shutdown_connections 2024-09-13T08:49:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.667+0000 7f015f200640 1 -- 172.21.15.44:0/1936232747 wait complete. 2024-09-13T08:49:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.671+0000 7f015f200640 1 Processor -- start 2024-09-13T08:49:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.671+0000 7f015f200640 1 -- start start 2024-09-13T08:49:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015f200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01540079b0 0x7f01540e2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:12.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0154008410 0x7f015404cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:12.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01540087f0 0x7f015404a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:12.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01540e4bf0 con 0x7f0154008410 2024-09-13T08:49:12.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01540e4a70 con 0x7f01540087f0 2024-09-13T08:49:12.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015f200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f01540e4d70 con 0x7f01540079b0 2024-09-13T08:49:12.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015de00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01540079b0 0x7f01540e2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:12.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015de00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01540079b0 0x7f01540e2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:36610/0 (socket says 172.21.15.44:36610) 2024-09-13T08:49:12.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015de00640 1 -- 172.21.15.44:0/135366092 learned_addr learned my addr 172.21.15.44:0/135366092 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:12.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015e800640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01540087f0 0x7f015404a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:12.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015de00640 1 -- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01540087f0 msgr2=0x7f015404a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:12.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015de00640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01540087f0 0x7f015404a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015de00640 1 -- 172.21.15.44:0/135366092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0154008410 msgr2=0x7f015404cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015de00640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0154008410 0x7f015404cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015de00640 1 -- 172.21.15.44:0/135366092 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01540e2f00 con 0x7f01540079b0 2024-09-13T08:49:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015de00640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01540079b0 0x7f01540e2620 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f014c00e9f0 tx=0x7f014c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f0152000640 1 -- 172.21.15.44:0/135366092 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c00cd80 con 0x7f01540079b0 2024-09-13T08:49:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f0152000640 1 -- 172.21.15.44:0/135366092 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f014c010430 con 0x7f01540079b0 2024-09-13T08:49:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f0152000640 1 -- 172.21.15.44:0/135366092 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c010860 con 0x7f01540079b0 2024-09-13T08:49:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015f200640 1 -- 172.21.15.44:0/135366092 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01540e31e0 con 0x7f01540079b0 2024-09-13T08:49:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.675+0000 7f015f200640 1 -- 172.21.15.44:0/135366092 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f01540e37a0 con 0x7f01540079b0 2024-09-13T08:49:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.679+0000 7f015f200640 1 -- 172.21.15.44:0/135366092 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01540e6830 con 0x7f01540079b0 2024-09-13T08:49:12.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.679+0000 7f0152000640 1 -- 172.21.15.44:0/135366092 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f014c010a00 con 0x7f01540079b0 2024-09-13T08:49:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.683+0000 7f0152000640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f013c074540 0x7f013c076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.683+0000 7f0152000640 1 -- 172.21.15.44:0/135366092 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f014c093780 con 0x7f01540079b0 2024-09-13T08:49:12.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.683+0000 7f015d400640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f013c074540 0x7f013c076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:12.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.683+0000 7f015d400640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f013c074540 0x7f013c076a00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f01480af000 tx=0x7f0148019680 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:12.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:12.683+0000 7f0152000640 1 -- 172.21.15.44:0/135366092 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f014c0604b0 con 0x7f01540079b0 2024-09-13T08:49:13.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.055+0000 7f015f200640 1 -- 172.21.15.44:0/135366092 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f01540dd840 con 0x7f01540079b0 2024-09-13T08:49:13.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.055+0000 7f0152000640 1 -- 172.21.15.44:0/135366092 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) ==== 80+0+0 (secure 0 0 0) 0x7f014c05fdb0 con 0x7f01540079b0 2024-09-13T08:49:13.060 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:49:13.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 -- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f013c074540 msgr2=0x7f013c076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:13.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f013c074540 0x7f013c076a00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f01480af000 tx=0x7f0148019680 comp rx=0 tx=0).stop 2024-09-13T08:49:13.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 -- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01540079b0 msgr2=0x7f01540e2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:13.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01540079b0 0x7f01540e2620 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f014c00e9f0 tx=0x7f014c00eeb0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 -- 172.21.15.44:0/135366092 shutdown_connections 2024-09-13T08:49:13.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f013c074540 0x7f013c076a00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01540087f0 0x7f015404a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0154008410 0x7f015404cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 --2- 172.21.15.44:0/135366092 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01540079b0 0x7f01540e2620 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 -- 172.21.15.44:0/135366092 >> 172.21.15.44:0/135366092 conn(0x7f0154056800 msgr2=0x7f01540e5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:13.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 -- 172.21.15.44:0/135366092 shutdown_connections 2024-09-13T08:49:13.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.059+0000 7f015f200640 1 -- 172.21.15.44:0/135366092 wait complete. 2024-09-13T08:49:13.088 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T08:49:13.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:13.089 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T08:49:13.138 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:49:13.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:13.139 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T08:49:13.174 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:49:13.175 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:49:13.175 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:13.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:49:13.184 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:49:13.184 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:49:13.184 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:13.225 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:49:13.225 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:49:13.225 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:49:13.225 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:13.235 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:49:13.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:49:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.419+0000 7f6a3e800640 1 -- 172.21.15.44:0/998057798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3815bd80 msgr2=0x7f6a380ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.419+0000 7f6a3e800640 1 --2- 172.21.15.44:0/998057798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3815bd80 0x7f6a380ccbc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009840 tx=0x7f6a2801c740 comp rx=0 tx=0).stop 2024-09-13T08:49:13.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.419+0000 7f6a3e800640 1 -- 172.21.15.44:0/998057798 shutdown_connections 2024-09-13T08:49:13.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.419+0000 7f6a3e800640 1 --2- 172.21.15.44:0/998057798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3815bd80 0x7f6a380ccbc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.419+0000 7f6a3e800640 1 -- 172.21.15.44:0/998057798 >> 172.21.15.44:0/998057798 conn(0x7f6a380d0cd0 msgr2=0x7f6a380d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:13.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.419+0000 7f6a3e800640 1 -- 172.21.15.44:0/998057798 shutdown_connections 2024-09-13T08:49:13.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.419+0000 7f6a3e800640 1 -- 172.21.15.44:0/998057798 wait complete. 2024-09-13T08:49:13.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a3e800640 1 Processor -- start 2024-09-13T08:49:13.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a3e800640 1 -- start start 2024-09-13T08:49:13.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a3e800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a38071b60 0x7f6a381fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:13.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a3e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a38073820 0x7f6a380c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:13.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a3e800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3815bd80 0x7f6a3815de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a3e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3806cda0 con 0x7f6a38073820 2024-09-13T08:49:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a3e800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3806cc20 con 0x7f6a3815bd80 2024-09-13T08:49:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a3e800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3806cf20 con 0x7f6a38071b60 2024-09-13T08:49:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a37400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a38071b60 0x7f6a381fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a37400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a38071b60 0x7f6a381fd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:36622/0 (socket says 172.21.15.44:36622) 2024-09-13T08:49:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.423+0000 7f6a37400640 1 -- 172.21.15.44:0/3255251808 learned_addr learned my addr 172.21.15.44:0/3255251808 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.427+0000 7f6a37e00640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3815bd80 0x7f6a3815de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.427+0000 7f6a37400640 1 -- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3815bd80 msgr2=0x7f6a3815de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:13.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.427+0000 7f6a37400640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3815bd80 0x7f6a3815de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.427+0000 7f6a37400640 1 -- 172.21.15.44:0/3255251808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a38073820 msgr2=0x7f6a380c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:13.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.427+0000 7f6a37400640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a38073820 0x7f6a380c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.427+0000 7f6a37400640 1 -- 172.21.15.44:0/3255251808 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3815e350 con 0x7f6a38071b60 2024-09-13T08:49:13.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.427+0000 7f6a37e00640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3815bd80 0x7f6a3815de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:13.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.427+0000 7f6a37400640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a38071b60 0x7f6a381fd8a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009790 tx=0x7f6a28005220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:13.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.431+0000 7f6a2f400640 1 -- 172.21.15.44:0/3255251808 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a280ac070 con 0x7f6a38071b60 2024-09-13T08:49:13.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.431+0000 7f6a2f400640 1 -- 172.21.15.44:0/3255251808 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a28004030 con 0x7f6a38071b60 2024-09-13T08:49:13.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.431+0000 7f6a2f400640 1 -- 172.21.15.44:0/3255251808 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28004440 con 0x7f6a38071b60 2024-09-13T08:49:13.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.431+0000 7f6a3e800640 1 -- 172.21.15.44:0/3255251808 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a380b46a0 con 0x7f6a38071b60 2024-09-13T08:49:13.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.431+0000 7f6a3e800640 1 -- 172.21.15.44:0/3255251808 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a38159df0 con 0x7f6a38071b60 2024-09-13T08:49:13.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.431+0000 7f6a2cc00640 1 -- 172.21.15.44:0/3255251808 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c0052a0 con 0x7f6a38071b60 2024-09-13T08:49:13.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.435+0000 7f6a2f400640 1 -- 172.21.15.44:0/3255251808 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f6a280af070 con 0x7f6a38071b60 2024-09-13T08:49:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.439+0000 7f6a2f400640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074640 0x7f6a18076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.439+0000 7f6a2f400640 1 -- 172.21.15.44:0/3255251808 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f6a2812c5f0 con 0x7f6a38071b60 2024-09-13T08:49:13.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.439+0000 7f6a36a00640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074640 0x7f6a18076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.439+0000 7f6a36a00640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074640 0x7f6a18076b00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6a24007fd0 tx=0x7f6a240078e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.443+0000 7f6a2f400640 1 -- 172.21.15.44:0/3255251808 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6a280b6070 con 0x7f6a38071b60 2024-09-13T08:49:13.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.847+0000 7f6a2cc00640 1 -- 172.21.15.44:0/3255251808 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a0c005860 con 0x7f6a38071b60 2024-09-13T08:49:13.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.847+0000 7f6a2f400640 1 -- 172.21.15.44:0/3255251808 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7f6a28016330 con 0x7f6a38071b60 2024-09-13T08:49:13.851 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:13.851 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":57,"btime":"2024-09-13T08:47:15:289304+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/833446483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":833446483},{"type":"v1","addr":"172.21.15.129:6836","nonce":833446483}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1844578230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1844578230},{"type":"v1","addr":"172.21.15.129:6837","nonce":1844578230}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2007417535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2007417535},{"type":"v1","addr":"172.21.15.44:6833","nonce":2007417535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:47:06.650500+0000","modified":"2024-09-13T08:47:09.677516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6942":{"gid":6942,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1796790158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1796790158},{"type":"v1","addr":"172.21.15.44:6835","nonce":1796790158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6942,"qdb_cluster":[6942]},"id":6}]} 2024-09-13T08:49:13.851 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 57 2024-09-13T08:49:13.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 -- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074640 msgr2=0x7f6a18076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:13.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074640 0x7f6a18076b00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6a24007fd0 tx=0x7f6a240078e0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 -- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a38071b60 msgr2=0x7f6a381fd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:13.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a38071b60 0x7f6a381fd8a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009790 tx=0x7f6a28005220 comp rx=0 tx=0).stop 2024-09-13T08:49:13.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 -- 172.21.15.44:0/3255251808 shutdown_connections 2024-09-13T08:49:13.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074640 0x7f6a18076b00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3815bd80 0x7f6a3815de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a38073820 0x7f6a380c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 --2- 172.21.15.44:0/3255251808 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a38071b60 0x7f6a381fd8a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:13.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 -- 172.21.15.44:0/3255251808 >> 172.21.15.44:0/3255251808 conn(0x7f6a380d0cd0 msgr2=0x7f6a380d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:13.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 -- 172.21.15.44:0/3255251808 shutdown_connections 2024-09-13T08:49:13.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:13.851+0000 7f6a2cc00640 1 -- 172.21.15.44:0/3255251808 wait complete. 2024-09-13T08:49:13.873 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:49:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 -- 172.21.15.44:0/943886509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c415bea0 msgr2=0x7f19c40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 --2- 172.21.15.44:0/943886509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c415bea0 0x7f19c40ce670 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009900 tx=0x7f19b401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 -- 172.21.15.44:0/943886509 shutdown_connections 2024-09-13T08:49:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 --2- 172.21.15.44:0/943886509 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19c40d28e0 0x7f19c40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 --2- 172.21.15.44:0/943886509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c415bea0 0x7f19c40ce670 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 -- 172.21.15.44:0/943886509 >> 172.21.15.44:0/943886509 conn(0x7f19c40d1dc0 msgr2=0x7f19c40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 -- 172.21.15.44:0/943886509 shutdown_connections 2024-09-13T08:49:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 -- 172.21.15.44:0/943886509 wait complete. 2024-09-13T08:49:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 Processor -- start 2024-09-13T08:49:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 -- start start 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c40cdb30 0x7f19c41f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19c40d28e0 0x7f19c40b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c415bea0 0x7f19c40b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f19c4168fc0 con 0x7f19c40cdb30 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19c4168e40 con 0x7f19c415bea0 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19ca400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f19c4169140 con 0x7f19c40d28e0 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19c3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c40cdb30 0x7f19c41f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19c3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c40cdb30 0x7f19c41f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56722/0 (socket says 172.21.15.44:56722) 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19c3400640 1 -- 172.21.15.44:0/2512589423 learned_addr learned my addr 172.21.15.44:0/2512589423 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19c3e00640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c415bea0 0x7f19c40b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19bbe00640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19c40d28e0 0x7f19c40b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19c3400640 1 -- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19c40d28e0 msgr2=0x7f19c40b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19c3400640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19c40d28e0 0x7f19c40b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19c3400640 1 -- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c415bea0 msgr2=0x7f19c40b7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19c3400640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c415bea0 0x7f19c40b7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19c3400640 1 -- 172.21.15.44:0/2512589423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c40b41a0 con 0x7f19c40cdb30 2024-09-13T08:49:14.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19bbe00640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19c40d28e0 0x7f19c40b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19c3400640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c40cdb30 0x7f19c41f6670 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009850 tx=0x7f19b40a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.019+0000 7f19bb400640 1 -- 172.21.15.44:0/2512589423 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b40b2040 con 0x7f19c40cdb30 2024-09-13T08:49:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.023+0000 7f19bb400640 1 -- 172.21.15.44:0/2512589423 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19b40042c0 con 0x7f19c40cdb30 2024-09-13T08:49:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.023+0000 7f19bb400640 1 -- 172.21.15.44:0/2512589423 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4005030 con 0x7f19c40cdb30 2024-09-13T08:49:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.023+0000 7f19ca400640 1 -- 172.21.15.44:0/2512589423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c40b4480 con 0x7f19c40cdb30 2024-09-13T08:49:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.023+0000 7f19ca400640 1 -- 172.21.15.44:0/2512589423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c41f5e70 con 0x7f19c40cdb30 2024-09-13T08:49:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.023+0000 7f19ca400640 1 -- 172.21.15.44:0/2512589423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c40cc9d0 con 0x7f19c40cdb30 2024-09-13T08:49:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.023+0000 7f19bb400640 1 -- 172.21.15.44:0/2512589423 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f19b40051d0 con 0x7f19c40cdb30 2024-09-13T08:49:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.023+0000 7f19bb400640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f19a0074540 0x7f19a0076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.023+0000 7f19bb400640 1 -- 172.21.15.44:0/2512589423 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f19b412c650 con 0x7f19c40cdb30 2024-09-13T08:49:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.023+0000 7f19bbe00640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f19a0074540 0x7f19a0076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.027+0000 7f19bbe00640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f19a0074540 0x7f19a0076a00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f19c409fc20 tx=0x7f19ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.027+0000 7f19bb400640 1 -- 172.21.15.44:0/2512589423 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f19b40164d0 con 0x7f19c40cdb30 2024-09-13T08:49:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.243+0000 7f19ca400640 1 -- 172.21.15.44:0/2512589423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19c40d2640 con 0x7f19c40cdb30 2024-09-13T08:49:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.243+0000 7f19bb400640 1 -- 172.21.15.44:0/2512589423 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7f19b40f9280 con 0x7f19c40cdb30 2024-09-13T08:49:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:14.246 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":57,"btime":"2024-09-13T08:47:15:289304+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/833446483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":833446483},{"type":"v1","addr":"172.21.15.129:6836","nonce":833446483}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1844578230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1844578230},{"type":"v1","addr":"172.21.15.129:6837","nonce":1844578230}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2007417535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2007417535},{"type":"v1","addr":"172.21.15.44:6833","nonce":2007417535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:47:06.650500+0000","modified":"2024-09-13T08:47:09.677516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6942":{"gid":6942,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1796790158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1796790158},{"type":"v1","addr":"172.21.15.44:6835","nonce":1796790158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6942,"qdb_cluster":[6942]},"id":6}]} 2024-09-13T08:49:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 57 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 -- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f19a0074540 msgr2=0x7f19a0076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f19a0074540 0x7f19a0076a00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f19c409fc20 tx=0x7f19ac009040 comp rx=0 tx=0).stop 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 -- 172.21.15.44:0/2512589423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c40cdb30 msgr2=0x7f19c41f6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c40cdb30 0x7f19c41f6670 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009850 tx=0x7f19b40a5730 comp rx=0 tx=0).stop 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 -- 172.21.15.44:0/2512589423 shutdown_connections 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f19a0074540 0x7f19a0076a00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c415bea0 0x7f19c40b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19c40d28e0 0x7f19c40b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 --2- 172.21.15.44:0/2512589423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c40cdb30 0x7f19c41f6670 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 -- 172.21.15.44:0/2512589423 >> 172.21.15.44:0/2512589423 conn(0x7f19c40d1dc0 msgr2=0x7f19c40b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 -- 172.21.15.44:0/2512589423 shutdown_connections 2024-09-13T08:49:14.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.247+0000 7f19ca400640 1 -- 172.21.15.44:0/2512589423 wait complete. 2024-09-13T08:49:14.269 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:49:14.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 -- 172.21.15.44:0/3338364661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8073170 msgr2=0x7fdfe806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 --2- 172.21.15.44:0/3338364661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8073170 0x7fdfe806d160 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fdfe806d950 tx=0x7fdfd401c7e0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 -- 172.21.15.44:0/3338364661 shutdown_connections 2024-09-13T08:49:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 --2- 172.21.15.44:0/3338364661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe8073170 0x7fdfe806d160 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 --2- 172.21.15.44:0/3338364661 >> v2:172.21.15.129:3300/0 conn(0x7fdfe80ce800 0x7fdfe806cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 -- 172.21.15.44:0/3338364661 >> 172.21.15.44:0/3338364661 conn(0x7fdfe80b2240 msgr2=0x7fdfe80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 -- 172.21.15.44:0/3338364661 shutdown_connections 2024-09-13T08:49:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 -- 172.21.15.44:0/3338364661 wait complete. 2024-09-13T08:49:14.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 Processor -- start 2024-09-13T08:49:14.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 -- start start 2024-09-13T08:49:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdfe8073170 0x7fdfe815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe80ce800 0x7fdfe81434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe8143fe0 0x7fdfe80b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8168290 con 0x7fdfe80ce800 2024-09-13T08:49:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8168110 con 0x7fdfe8143fe0 2024-09-13T08:49:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdff0600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe8168410 con 0x7fdfe8073170 2024-09-13T08:49:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.415+0000 7fdfed200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe80ce800 0x7fdfe81434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfed200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe80ce800 0x7fdfe81434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56746/0 (socket says 172.21.15.44:56746) 2024-09-13T08:49:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfed200640 1 -- 172.21.15.44:0/1262098576 learned_addr learned my addr 172.21.15.44:0/1262098576 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:14.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfee600640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe8143fe0 0x7fdfe80b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfedc00640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdfe8073170 0x7fdfe815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfed200640 1 -- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdfe8073170 msgr2=0x7fdfe815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfed200640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdfe8073170 0x7fdfe815abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfed200640 1 -- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe8143fe0 msgr2=0x7fdfe80b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfed200640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe8143fe0 0x7fdfe80b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfed200640 1 -- 172.21.15.44:0/1262098576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe80b3cb0 con 0x7fdfe80ce800 2024-09-13T08:49:14.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfed200640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe80ce800 0x7fdfe81434a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009790 tx=0x7fdfd40051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:14.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfe5600640 1 -- 172.21.15.44:0/1262098576 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4005510 con 0x7fdfe80ce800 2024-09-13T08:49:14.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfe5600640 1 -- 172.21.15.44:0/1262098576 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfd40bd070 con 0x7fdfe80ce800 2024-09-13T08:49:14.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdff0600640 1 -- 172.21.15.44:0/1262098576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe80b3f90 con 0x7fdfe80ce800 2024-09-13T08:49:14.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfe5600640 1 -- 172.21.15.44:0/1262098576 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4004930 con 0x7fdfe80ce800 2024-09-13T08:49:14.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdff0600640 1 -- 172.21.15.44:0/1262098576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe80b4550 con 0x7fdfe80ce800 2024-09-13T08:49:14.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdff0600640 1 -- 172.21.15.44:0/1262098576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe815e8b0 con 0x7fdfe80ce800 2024-09-13T08:49:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.419+0000 7fdfe5600640 1 -- 172.21.15.44:0/1262098576 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fdfd4004ad0 con 0x7fdfe80ce800 2024-09-13T08:49:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.423+0000 7fdfe5600640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdfc8074540 0x7fdfc8076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.423+0000 7fdfe5600640 1 -- 172.21.15.44:0/1262098576 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7fdfd412d690 con 0x7fdfe80ce800 2024-09-13T08:49:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.423+0000 7fdfedc00640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdfc8074540 0x7fdfc8076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.423+0000 7fdfedc00640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdfc8074540 0x7fdfc8076a00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009b70 tx=0x7fdfdc0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:14.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.427+0000 7fdfe5600640 1 -- 172.21.15.44:0/1262098576 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdfd40163d0 con 0x7fdfe80ce800 2024-09-13T08:49:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.603+0000 7fdff0600640 1 -- 172.21.15.44:0/1262098576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfe806e150 con 0x7fdfe80ce800 2024-09-13T08:49:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdfe5600640 1 -- 172.21.15.44:0/1262098576 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16942 (secure 0 0 0) 0x7fdfd40f9d90 con 0x7fdfe80ce800 2024-09-13T08:49:14.608 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:14.609 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":91,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:49:09.165275+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:47:04.371378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T08:47:05.614930+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:49:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 -- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdfc8074540 msgr2=0x7fdfc8076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdfc8074540 0x7fdfc8076a00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009b70 tx=0x7fdfdc0091e0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 -- 172.21.15.44:0/1262098576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe80ce800 msgr2=0x7fdfe81434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe80ce800 0x7fdfe81434a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009790 tx=0x7fdfd40051a0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 -- 172.21.15.44:0/1262098576 shutdown_connections 2024-09-13T08:49:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdfc8074540 0x7fdfc8076a00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe8143fe0 0x7fdfe80b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe80ce800 0x7fdfe81434a0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 --2- 172.21.15.44:0/1262098576 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdfe8073170 0x7fdfe815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 -- 172.21.15.44:0/1262098576 >> 172.21.15.44:0/1262098576 conn(0x7fdfe80b2240 msgr2=0x7fdfe80b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:14.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 -- 172.21.15.44:0/1262098576 shutdown_connections 2024-09-13T08:49:14.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.607+0000 7fdff0600640 1 -- 172.21.15.44:0/1262098576 wait complete. 2024-09-13T08:49:14.631 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:49:14.631 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:49:14.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.775+0000 7f6d74200640 1 -- 172.21.15.44:0/592935958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d6c15bea0 msgr2=0x7f6d6c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.775+0000 7f6d74200640 1 --2- 172.21.15.44:0/592935958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d6c15bea0 0x7f6d6c0ce670 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c073d10 tx=0x7f6d6001c830 comp rx=0 tx=0).stop 2024-09-13T08:49:14.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.775+0000 7f6d74200640 1 -- 172.21.15.44:0/592935958 shutdown_connections 2024-09-13T08:49:14.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.775+0000 7f6d74200640 1 --2- 172.21.15.44:0/592935958 >> v2:172.21.15.129:3300/0 conn(0x7f6d6c0cdb30 0x7f6d6c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.775+0000 7f6d74200640 1 --2- 172.21.15.44:0/592935958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d6c15bea0 0x7f6d6c0ce670 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.775+0000 7f6d74200640 1 -- 172.21.15.44:0/592935958 >> 172.21.15.44:0/592935958 conn(0x7f6d6c0d1dc0 msgr2=0x7f6d6c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.775+0000 7f6d74200640 1 -- 172.21.15.44:0/592935958 shutdown_connections 2024-09-13T08:49:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.775+0000 7f6d74200640 1 -- 172.21.15.44:0/592935958 wait complete. 2024-09-13T08:49:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d74200640 1 Processor -- start 2024-09-13T08:49:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d74200640 1 -- start start 2024-09-13T08:49:14.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d74200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6c0cdb30 0x7f6d6c0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d74200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d6c0d28e0 0x7f6d6c151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d74200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6d6c15bea0 0x7f6d6c153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d74200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c168fc0 con 0x7f6d6c0d28e0 2024-09-13T08:49:14.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d74200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c168e40 con 0x7f6d6c0cdb30 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d74200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6d6c169140 con 0x7f6d6c15bea0 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d70e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d6c0d28e0 0x7f6d6c151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d70e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d6c0d28e0 0x7f6d6c151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56760/0 (socket says 172.21.15.44:56760) 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d70e00640 1 -- 172.21.15.44:0/111623542 learned_addr learned my addr 172.21.15.44:0/111623542 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d71800640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6c0cdb30 0x7f6d6c0b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d72200640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6d6c15bea0 0x7f6d6c153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d70e00640 1 -- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6d6c15bea0 msgr2=0x7f6d6c153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d70e00640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6d6c15bea0 0x7f6d6c153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d70e00640 1 -- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6c0cdb30 msgr2=0x7f6d6c0b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d70e00640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6c0cdb30 0x7f6d6c0b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d70e00640 1 -- 172.21.15.44:0/111623542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c156410 con 0x7f6d6c0d28e0 2024-09-13T08:49:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d70e00640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d6c0d28e0 0x7f6d6c151d10 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800b550 tx=0x7f6d5800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:14.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d69600640 1 -- 172.21.15.44:0/111623542 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58014070 con 0x7f6d6c0d28e0 2024-09-13T08:49:14.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d69600640 1 -- 172.21.15.44:0/111623542 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d58010980 con 0x7f6d6c0d28e0 2024-09-13T08:49:14.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d69600640 1 -- 172.21.15.44:0/111623542 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58010dd0 con 0x7f6d6c0d28e0 2024-09-13T08:49:14.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d74200640 1 -- 172.21.15.44:0/111623542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6c1566f0 con 0x7f6d6c0d28e0 2024-09-13T08:49:14.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.779+0000 7f6d74200640 1 -- 172.21.15.44:0/111623542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6c0ca3c0 con 0x7f6d6c0d28e0 2024-09-13T08:49:14.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.783+0000 7f6d74200640 1 -- 172.21.15.44:0/111623542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6c0cc9d0 con 0x7f6d6c0d28e0 2024-09-13T08:49:14.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.783+0000 7f6d69600640 1 -- 172.21.15.44:0/111623542 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f6d58017070 con 0x7f6d6c0d28e0 2024-09-13T08:49:14.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.783+0000 7f6d69600640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d4c074540 0x7f6d4c076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:14.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.783+0000 7f6d69600640 1 -- 172.21.15.44:0/111623542 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f6d58092900 con 0x7f6d6c0d28e0 2024-09-13T08:49:14.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.783+0000 7f6d71800640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d4c074540 0x7f6d4c076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:14.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.783+0000 7f6d71800640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d4c074540 0x7f6d4c076a00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6d600af000 tx=0x7f6d6001b070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:14.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.787+0000 7f6d69600640 1 -- 172.21.15.44:0/111623542 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6d5805ef60 con 0x7f6d6c0d28e0 2024-09-13T08:49:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.999+0000 7f6d74200640 1 -- 172.21.15.44:0/111623542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d6c072200 con 0x7f6d6c0d28e0 2024-09-13T08:49:15.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:14.999+0000 7f6d69600640 1 -- 172.21.15.44:0/111623542 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7f6d58063990 con 0x7f6d6c0d28e0 2024-09-13T08:49:15.003 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:15.003 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":57,"btime":"2024-09-13T08:47:15:289304+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/833446483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":833446483},{"type":"v1","addr":"172.21.15.129:6836","nonce":833446483}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1844578230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1844578230},{"type":"v1","addr":"172.21.15.129:6837","nonce":1844578230}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2007417535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2007417535},{"type":"v1","addr":"172.21.15.44:6833","nonce":2007417535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:47:06.650500+0000","modified":"2024-09-13T08:47:09.677516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6942":{"gid":6942,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1796790158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1796790158},{"type":"v1","addr":"172.21.15.44:6835","nonce":1796790158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6942,"qdb_cluster":[6942]},"id":6}]} 2024-09-13T08:49:15.003 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 57 2024-09-13T08:49:15.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 -- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d4c074540 msgr2=0x7f6d4c076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d4c074540 0x7f6d4c076a00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6d600af000 tx=0x7f6d6001b070 comp rx=0 tx=0).stop 2024-09-13T08:49:15.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 -- 172.21.15.44:0/111623542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d6c0d28e0 msgr2=0x7f6d6c151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d6c0d28e0 0x7f6d6c151d10 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800b550 tx=0x7f6d5800ba10 comp rx=0 tx=0).stop 2024-09-13T08:49:15.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 -- 172.21.15.44:0/111623542 shutdown_connections 2024-09-13T08:49:15.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d4c074540 0x7f6d4c076a00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6d6c15bea0 0x7f6d6c153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d6c0d28e0 0x7f6d6c151d10 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 --2- 172.21.15.44:0/111623542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6c0cdb30 0x7f6d6c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 -- 172.21.15.44:0/111623542 >> 172.21.15.44:0/111623542 conn(0x7f6d6c0d1dc0 msgr2=0x7f6d6c0cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:15.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 -- 172.21.15.44:0/111623542 shutdown_connections 2024-09-13T08:49:15.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.003+0000 7f6d74200640 1 -- 172.21.15.44:0/111623542 wait complete. 2024-09-13T08:49:15.024 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:49:15.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.167+0000 7fed21a00640 1 -- 172.21.15.44:0/1504874380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed1c0cd540 msgr2=0x7fed1c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.167+0000 7fed21a00640 1 --2- 172.21.15.44:0/1504874380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed1c0cd540 0x7fed1c06e7a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fed14009bc0 tx=0x7fed1401f3b0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.167+0000 7fed21a00640 1 -- 172.21.15.44:0/1504874380 shutdown_connections 2024-09-13T08:49:15.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.167+0000 7fed21a00640 1 --2- 172.21.15.44:0/1504874380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed1c0cd540 0x7fed1c06e7a0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.167+0000 7fed21a00640 1 --2- 172.21.15.44:0/1504874380 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed1c15bfa0 0x7fed1c0cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.167+0000 7fed21a00640 1 -- 172.21.15.44:0/1504874380 >> 172.21.15.44:0/1504874380 conn(0x7fed1c0b18f0 msgr2=0x7fed1c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:15.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.167+0000 7fed21a00640 1 -- 172.21.15.44:0/1504874380 shutdown_connections 2024-09-13T08:49:15.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.167+0000 7fed21a00640 1 -- 172.21.15.44:0/1504874380 wait complete. 2024-09-13T08:49:15.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed21a00640 1 Processor -- start 2024-09-13T08:49:15.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed21a00640 1 -- start start 2024-09-13T08:49:15.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed21a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed1c0cd540 0x7fed1c158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed21a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed1c15bfa0 0x7fed1c0b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed21a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed1c15c3e0 0x7fed1c154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed21a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c06f120 con 0x7fed1c15c3e0 2024-09-13T08:49:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed21a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c06efa0 con 0x7fed1c15bfa0 2024-09-13T08:49:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed21a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fed1c06f2a0 con 0x7fed1c0cd540 2024-09-13T08:49:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed1c15c3e0 0x7fed1c154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed1c15c3e0 0x7fed1c154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56780/0 (socket says 172.21.15.44:56780) 2024-09-13T08:49:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1b400640 1 -- 172.21.15.44:0/3164775884 learned_addr learned my addr 172.21.15.44:0/3164775884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:15.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1aa00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed1c0cd540 0x7fed1c158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1b400640 1 -- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed1c0cd540 msgr2=0x7fed1c158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1b400640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed1c0cd540 0x7fed1c158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1b400640 1 -- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed1c15bfa0 msgr2=0x7fed1c0b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed13e00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed1c15bfa0 0x7fed1c0b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1b400640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed1c15bfa0 0x7fed1c0b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1b400640 1 -- 172.21.15.44:0/3164775884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c158510 con 0x7fed1c15c3e0 2024-09-13T08:49:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed13e00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed1c15bfa0 0x7fed1c0b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1aa00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed1c0cd540 0x7fed1c158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed1b400640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed1c15c3e0 0x7fed1c154690 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fed1401cfe0 tx=0x7fed14009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed12a00640 1 -- 172.21.15.44:0/3164775884 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed140b4040 con 0x7fed1c15c3e0 2024-09-13T08:49:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed12a00640 1 -- 172.21.15.44:0/3164775884 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed14004b40 con 0x7fed1c15c3e0 2024-09-13T08:49:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed12a00640 1 -- 172.21.15.44:0/3164775884 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed140af8a0 con 0x7fed1c15c3e0 2024-09-13T08:49:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed21a00640 1 -- 172.21.15.44:0/3164775884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1c0ae540 con 0x7fed1c15c3e0 2024-09-13T08:49:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.171+0000 7fed21a00640 1 -- 172.21.15.44:0/3164775884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1c159210 con 0x7fed1c15c3e0 2024-09-13T08:49:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.175+0000 7fed21a00640 1 -- 172.21.15.44:0/3164775884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed1c069410 con 0x7fed1c15c3e0 2024-09-13T08:49:15.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.175+0000 7fed12a00640 1 -- 172.21.15.44:0/3164775884 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fed140c1020 con 0x7fed1c15c3e0 2024-09-13T08:49:15.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.175+0000 7fed12a00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf8074540 0x7fecf8076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.175+0000 7fed12a00640 1 -- 172.21.15.44:0/3164775884 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7fed14130630 con 0x7fed1c15c3e0 2024-09-13T08:49:15.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.175+0000 7fed1aa00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf8074540 0x7fecf8076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.175+0000 7fed1aa00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf8074540 0x7fecf8076a00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fed0c001230 tx=0x7fed0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:15.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.179+0000 7fed12a00640 1 -- 172.21.15.44:0/3164775884 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fed14018f10 con 0x7fed1c15c3e0 2024-09-13T08:49:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.391+0000 7fed21a00640 1 -- 172.21.15.44:0/3164775884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed1c0cbf00 con 0x7fed1c15c3e0 2024-09-13T08:49:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.391+0000 7fed12a00640 1 -- 172.21.15.44:0/3164775884 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7fed140fd2e0 con 0x7fed1c15c3e0 2024-09-13T08:49:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:15.396 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":57,"btime":"2024-09-13T08:47:15:289304+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/833446483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":833446483},{"type":"v1","addr":"172.21.15.129:6836","nonce":833446483}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1844578230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1844578230},{"type":"v1","addr":"172.21.15.129:6837","nonce":1844578230}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2007417535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2007417535},{"type":"v1","addr":"172.21.15.44:6833","nonce":2007417535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:47:06.650500+0000","modified":"2024-09-13T08:47:09.677516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6942":{"gid":6942,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1796790158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1796790158},{"type":"v1","addr":"172.21.15.44:6835","nonce":1796790158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6942,"qdb_cluster":[6942]},"id":6}]} 2024-09-13T08:49:15.396 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 57 2024-09-13T08:49:15.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 -- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf8074540 msgr2=0x7fecf8076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf8074540 0x7fecf8076a00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fed0c001230 tx=0x7fed0c009040 comp rx=0 tx=0).stop 2024-09-13T08:49:15.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 -- 172.21.15.44:0/3164775884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed1c15c3e0 msgr2=0x7fed1c154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed1c15c3e0 0x7fed1c154690 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fed1401cfe0 tx=0x7fed14009300 comp rx=0 tx=0).stop 2024-09-13T08:49:15.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 -- 172.21.15.44:0/3164775884 shutdown_connections 2024-09-13T08:49:15.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf8074540 0x7fecf8076a00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed1c15c3e0 0x7fed1c154690 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed1c15bfa0 0x7fed1c0b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 --2- 172.21.15.44:0/3164775884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed1c0cd540 0x7fed1c158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 -- 172.21.15.44:0/3164775884 >> 172.21.15.44:0/3164775884 conn(0x7fed1c0b18f0 msgr2=0x7fed1c0ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:15.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 -- 172.21.15.44:0/3164775884 shutdown_connections 2024-09-13T08:49:15.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.395+0000 7fed21a00640 1 -- 172.21.15.44:0/3164775884 wait complete. 2024-09-13T08:49:15.418 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:49:15.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.563+0000 7f5847000640 1 -- 172.21.15.44:0/2058113937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f584015bea0 msgr2=0x7f58400ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.563+0000 7f5847000640 1 --2- 172.21.15.44:0/2058113937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f584015bea0 0x7f58400ce670 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5830009840 tx=0x7f583001c820 comp rx=0 tx=0).stop 2024-09-13T08:49:15.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.563+0000 7f5847000640 1 -- 172.21.15.44:0/2058113937 shutdown_connections 2024-09-13T08:49:15.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.563+0000 7f5847000640 1 --2- 172.21.15.44:0/2058113937 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f58400d28e0 0x7f58400cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.563+0000 7f5847000640 1 --2- 172.21.15.44:0/2058113937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f584015bea0 0x7f58400ce670 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.563+0000 7f5847000640 1 -- 172.21.15.44:0/2058113937 >> 172.21.15.44:0/2058113937 conn(0x7f58400d1dc0 msgr2=0x7f58400d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:15.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.563+0000 7f5847000640 1 -- 172.21.15.44:0/2058113937 shutdown_connections 2024-09-13T08:49:15.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.563+0000 7f5847000640 1 -- 172.21.15.44:0/2058113937 wait complete. 2024-09-13T08:49:15.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.563+0000 7f5847000640 1 Processor -- start 2024-09-13T08:49:15.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.563+0000 7f5847000640 1 -- start start 2024-09-13T08:49:15.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5847000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58400cdb30 0x7f5840157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5847000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58400d28e0 0x7f5840155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5847000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f584015bea0 0x7f58400b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5847000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5840168fc0 con 0x7f58400cdb30 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5847000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5840168e40 con 0x7f58400d28e0 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5847000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5840169140 con 0x7f584015bea0 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58400cdb30 0x7f5840157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58400cdb30 0x7f5840157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56808/0 (socket says 172.21.15.44:56808) 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583fe00640 1 -- 172.21.15.44:0/1986250790 learned_addr learned my addr 172.21.15.44:0/1986250790 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5845000640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f584015bea0 0x7f58400b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5837e00640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58400d28e0 0x7f5840155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583fe00640 1 -- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f584015bea0 msgr2=0x7f58400b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583fe00640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f584015bea0 0x7f58400b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583fe00640 1 -- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58400d28e0 msgr2=0x7f5840155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583fe00640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58400d28e0 0x7f5840155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583fe00640 1 -- 172.21.15.44:0/1986250790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58400c59a0 con 0x7f58400cdb30 2024-09-13T08:49:15.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583fe00640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58400cdb30 0x7f5840157040 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f5830009790 tx=0x7f58300a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:15.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583cc00640 1 -- 172.21.15.44:0/1986250790 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58300bb070 con 0x7f58400cdb30 2024-09-13T08:49:15.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583cc00640 1 -- 172.21.15.44:0/1986250790 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5830004470 con 0x7f58400cdb30 2024-09-13T08:49:15.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5847000640 1 -- 172.21.15.44:0/1986250790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58400c89b0 con 0x7f58400cdb30 2024-09-13T08:49:15.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583cc00640 1 -- 172.21.15.44:0/1986250790 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58300b2040 con 0x7f58400cdb30 2024-09-13T08:49:15.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5847000640 1 -- 172.21.15.44:0/1986250790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58400c8100 con 0x7f58400cdb30 2024-09-13T08:49:15.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f5847000640 1 -- 172.21.15.44:0/1986250790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58400cc9d0 con 0x7f58400cdb30 2024-09-13T08:49:15.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.567+0000 7f583cc00640 1 -- 172.21.15.44:0/1986250790 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f58300be070 con 0x7f58400cdb30 2024-09-13T08:49:15.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.571+0000 7f583cc00640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f581c074540 0x7f581c076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.571+0000 7f583cc00640 1 -- 172.21.15.44:0/1986250790 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f583012c2f0 con 0x7f58400cdb30 2024-09-13T08:49:15.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.571+0000 7f5837e00640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f581c074540 0x7f581c076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.571+0000 7f5837e00640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f581c074540 0x7f581c076a00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5840157580 tx=0x7f5828009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:15.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.575+0000 7f583cc00640 1 -- 172.21.15.44:0/1986250790 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5830016410 con 0x7f58400cdb30 2024-09-13T08:49:15.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.751+0000 7f5847000640 1 -- 172.21.15.44:0/1986250790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58400cb750 con 0x7f58400cdb30 2024-09-13T08:49:15.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.751+0000 7f583cc00640 1 -- 172.21.15.44:0/1986250790 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16942 (secure 0 0 0) 0x7f58300f8fc0 con 0x7f58400cdb30 2024-09-13T08:49:15.755 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:15.755 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":91,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:49:09.165275+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:47:04.371378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T08:47:05.614930+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:49:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 -- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f581c074540 msgr2=0x7f581c076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f581c074540 0x7f581c076a00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5840157580 tx=0x7f5828009040 comp rx=0 tx=0).stop 2024-09-13T08:49:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 -- 172.21.15.44:0/1986250790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58400cdb30 msgr2=0x7f5840157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58400cdb30 0x7f5840157040 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f5830009790 tx=0x7f58300a5670 comp rx=0 tx=0).stop 2024-09-13T08:49:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 -- 172.21.15.44:0/1986250790 shutdown_connections 2024-09-13T08:49:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f581c074540 0x7f581c076a00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f584015bea0 0x7f58400b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58400d28e0 0x7f5840155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 --2- 172.21.15.44:0/1986250790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58400cdb30 0x7f5840157040 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 -- 172.21.15.44:0/1986250790 >> 172.21.15.44:0/1986250790 conn(0x7f58400d1dc0 msgr2=0x7f58400b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:15.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 -- 172.21.15.44:0/1986250790 shutdown_connections 2024-09-13T08:49:15.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.755+0000 7f5847000640 1 -- 172.21.15.44:0/1986250790 wait complete. 2024-09-13T08:49:15.777 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T08:49:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 -- 172.21.15.44:0/2583210411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba440cf4e0 msgr2=0x7fba440cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 --2- 172.21.15.44:0/2583210411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba440cf4e0 0x7fba440cd950 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009bc0 tx=0x7fba3c01ca80 comp rx=0 tx=0).stop 2024-09-13T08:49:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 -- 172.21.15.44:0/2583210411 shutdown_connections 2024-09-13T08:49:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 --2- 172.21.15.44:0/2583210411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba440cf4e0 0x7fba440cd950 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 --2- 172.21.15.44:0/2583210411 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba4415bd50 0x7fba440ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 -- 172.21.15.44:0/2583210411 >> 172.21.15.44:0/2583210411 conn(0x7fba440d1ea0 msgr2=0x7fba440d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 -- 172.21.15.44:0/2583210411 shutdown_connections 2024-09-13T08:49:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 -- 172.21.15.44:0/2583210411 wait complete. 2024-09-13T08:49:15.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 Processor -- start 2024-09-13T08:49:15.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 -- start start 2024-09-13T08:49:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba440cf4e0 0x7fba440b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba4415bd50 0x7fba4414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba4415c130 0x7fba440b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fba44168f00 con 0x7fba440cf4e0 2024-09-13T08:49:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fba44168d80 con 0x7fba4415c130 2024-09-13T08:49:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba4ae00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fba44169080 con 0x7fba4415bd50 2024-09-13T08:49:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba440cf4e0 0x7fba440b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba440cf4e0 0x7fba440b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56824/0 (socket says 172.21.15.44:56824) 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba43e00640 1 -- 172.21.15.44:0/2137072344 learned_addr learned my addr 172.21.15.44:0/2137072344 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba48e00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba4415c130 0x7fba440b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba3be00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba4415bd50 0x7fba4414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba43e00640 1 -- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba4415bd50 msgr2=0x7fba4414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba43e00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba4415bd50 0x7fba4414e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba43e00640 1 -- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba4415c130 msgr2=0x7fba440b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba43e00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba4415c130 0x7fba440b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba43e00640 1 -- 172.21.15.44:0/2137072344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba440b57a0 con 0x7fba440cf4e0 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.923+0000 7fba48e00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba4415c130 0x7fba440b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba43e00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba440cf4e0 0x7fba440b6840 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fba3400e9f0 tx=0x7fba3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba40c00640 1 -- 172.21.15.44:0/2137072344 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3400cd80 con 0x7fba440cf4e0 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba40c00640 1 -- 172.21.15.44:0/2137072344 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba34010430 con 0x7fba440cf4e0 2024-09-13T08:49:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba4ae00640 1 -- 172.21.15.44:0/2137072344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba440b5a80 con 0x7fba440cf4e0 2024-09-13T08:49:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba40c00640 1 -- 172.21.15.44:0/2137072344 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34010860 con 0x7fba440cf4e0 2024-09-13T08:49:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba4ae00640 1 -- 172.21.15.44:0/2137072344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba440b6040 con 0x7fba440cf4e0 2024-09-13T08:49:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba4ae00640 1 -- 172.21.15.44:0/2137072344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba440b6480 con 0x7fba440cf4e0 2024-09-13T08:49:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba40c00640 1 -- 172.21.15.44:0/2137072344 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fba34010a00 con 0x7fba440cf4e0 2024-09-13T08:49:15.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba40c00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fba20074540 0x7fba20076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:15.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba40c00640 1 -- 172.21.15.44:0/2137072344 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7fba34093690 con 0x7fba440cf4e0 2024-09-13T08:49:15.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.927+0000 7fba3be00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fba20074540 0x7fba20076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:15.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.931+0000 7fba3be00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fba20074540 0x7fba20076a00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0046c0 tx=0x7fba2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:15.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:15.931+0000 7fba40c00640 1 -- 172.21.15.44:0/2137072344 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fba340604a0 con 0x7fba440cf4e0 2024-09-13T08:49:16.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.151+0000 7fba4ae00640 1 -- 172.21.15.44:0/2137072344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fba440ce590 con 0x7fba440cf4e0 2024-09-13T08:49:16.155 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T08:49:16.151+0000 7f03b3c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T08:49:16.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.155+0000 7fba40c00640 1 -- 172.21.15.44:0/2137072344 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7fba3405fda0 con 0x7fba440cf4e0 2024-09-13T08:49:16.157 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T08:49:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 -- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fba20074540 msgr2=0x7fba20076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fba20074540 0x7fba20076a00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0046c0 tx=0x7fba2c009040 comp rx=0 tx=0).stop 2024-09-13T08:49:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 -- 172.21.15.44:0/2137072344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba440cf4e0 msgr2=0x7fba440b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba440cf4e0 0x7fba440b6840 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fba3400e9f0 tx=0x7fba3400eeb0 comp rx=0 tx=0).stop 2024-09-13T08:49:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 -- 172.21.15.44:0/2137072344 shutdown_connections 2024-09-13T08:49:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fba20074540 0x7fba20076a00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba4415c130 0x7fba440b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba4415bd50 0x7fba4414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:16.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 --2- 172.21.15.44:0/2137072344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba440cf4e0 0x7fba440b6840 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:16.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 -- 172.21.15.44:0/2137072344 >> 172.21.15.44:0/2137072344 conn(0x7fba440d1ea0 msgr2=0x7fba4415b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:16.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 -- 172.21.15.44:0/2137072344 shutdown_connections 2024-09-13T08:49:16.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.159+0000 7fba4ae00640 1 -- 172.21.15.44:0/2137072344 wait complete. 2024-09-13T08:49:16.184 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T08:49:16.231 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:49:16.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 -- 172.21.15.44:0/16340554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901603f0 msgr2=0x7fe1900b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:16.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 --2- 172.21.15.44:0/16340554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901603f0 0x7fe1900b2390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe184009840 tx=0x7fe18401c7f0 comp rx=0 tx=0).stop 2024-09-13T08:49:16.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 -- 172.21.15.44:0/16340554 shutdown_connections 2024-09-13T08:49:16.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 --2- 172.21.15.44:0/16340554 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1900ce980 0x7fe1900cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:16.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 --2- 172.21.15.44:0/16340554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901603f0 0x7fe1900b2390 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:16.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 -- 172.21.15.44:0/16340554 >> 172.21.15.44:0/16340554 conn(0x7fe1900d1df0 msgr2=0x7fe1900d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:16.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 -- 172.21.15.44:0/16340554 shutdown_connections 2024-09-13T08:49:16.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 -- 172.21.15.44:0/16340554 wait complete. 2024-09-13T08:49:16.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 Processor -- start 2024-09-13T08:49:16.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 -- start start 2024-09-13T08:49:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1900b28d0 0x7fe190150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1900ce980 0x7fe1901662e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1901603f0 0x7fe1901f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe190169670 con 0x7fe1900b28d0 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901694f0 con 0x7fe1901603f0 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe198400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe1901697f0 con 0x7fe1900ce980 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe195a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1900b28d0 0x7fe190150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe195a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1900b28d0 0x7fe190150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56850/0 (socket says 172.21.15.44:56850) 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe195a00640 1 -- 172.21.15.44:0/2177467075 learned_addr learned my addr 172.21.15.44:0/2177467075 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe196400640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1901603f0 0x7fe1901f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe195a00640 1 -- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1900ce980 msgr2=0x7fe1901662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe195a00640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1900ce980 0x7fe1901662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe195a00640 1 -- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1901603f0 msgr2=0x7fe1901f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe195a00640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1901603f0 0x7fe1901f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe195a00640 1 -- 172.21.15.44:0/2177467075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19006fac0 con 0x7fe1900b28d0 2024-09-13T08:49:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.335+0000 7fe196400640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1901603f0 0x7fe1901f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe195a00640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1900b28d0 0x7fe190150f70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe184009790 tx=0x7fe1840a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe18d600640 1 -- 172.21.15.44:0/2177467075 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840bb070 con 0x7fe1900b28d0 2024-09-13T08:49:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe18d600640 1 -- 172.21.15.44:0/2177467075 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe184005030 con 0x7fe1900b28d0 2024-09-13T08:49:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe18d600640 1 -- 172.21.15.44:0/2177467075 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840b2040 con 0x7fe1900b28d0 2024-09-13T08:49:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe198400640 1 -- 172.21.15.44:0/2177467075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe190157d10 con 0x7fe1900b28d0 2024-09-13T08:49:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe198400640 1 -- 172.21.15.44:0/2177467075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1901582d0 con 0x7fe1900b28d0 2024-09-13T08:49:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe198400640 1 -- 172.21.15.44:0/2177467075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1900cf9c0 con 0x7fe1900b28d0 2024-09-13T08:49:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe18d600640 1 -- 172.21.15.44:0/2177467075 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fe1840be070 con 0x7fe1900b28d0 2024-09-13T08:49:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe18d600640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170074540 0x7fe170076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe18d600640 1 -- 172.21.15.44:0/2177467075 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5879+0+0 (secure 0 0 0) 0x7fe18412c190 con 0x7fe1900b28d0 2024-09-13T08:49:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.339+0000 7fe195000640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170074540 0x7fe170076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.343+0000 7fe195000640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170074540 0x7fe170076a00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe17c007fd0 tx=0x7fe17c0078e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.343+0000 7fe18d600640 1 -- 172.21.15.44:0/2177467075 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe184016410 con 0x7fe1900b28d0 2024-09-13T08:49:16.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:16.559+0000 7fe198400640 1 -- 172.21.15.44:0/2177467075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe1900b7d70 con 0x7fe1900b28d0 2024-09-13T08:49:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.195+0000 7fe18d600640 1 -- 172.21.15.44:0/2177467075 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) ==== 104+0+0 (secure 0 0 0) 0x7fe1840f8e40 con 0x7fe1900b28d0 2024-09-13T08:49:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 -- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170074540 msgr2=0x7fe170076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170074540 0x7fe170076a00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe17c007fd0 tx=0x7fe17c0078e0 comp rx=0 tx=0).stop 2024-09-13T08:49:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 -- 172.21.15.44:0/2177467075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1900b28d0 msgr2=0x7fe190150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1900b28d0 0x7fe190150f70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe184009790 tx=0x7fe1840a5780 comp rx=0 tx=0).stop 2024-09-13T08:49:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 -- 172.21.15.44:0/2177467075 shutdown_connections 2024-09-13T08:49:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe170074540 0x7fe170076a00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1901603f0 0x7fe1901f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe1900ce980 0x7fe1901662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 --2- 172.21.15.44:0/2177467075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1900b28d0 0x7fe190150f70 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 -- 172.21.15.44:0/2177467075 >> 172.21.15.44:0/2177467075 conn(0x7fe1900d1df0 msgr2=0x7fe1900b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 -- 172.21.15.44:0/2177467075 shutdown_connections 2024-09-13T08:49:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.199+0000 7fe198400640 1 -- 172.21.15.44:0/2177467075 wait complete. 2024-09-13T08:49:17.228 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T08:49:17.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.427+0000 7fc6ce000640 1 -- 172.21.15.44:0/698802923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ac004d50 msgr2=0x7fc6ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.427+0000 7fc6ce000640 1 --2- 172.21.15.44:0/698802923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ac004d50 0x7fc6ac0051d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000c640 tx=0x7fc6c001d320 comp rx=0 tx=0).stop 2024-09-13T08:49:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.427+0000 7fc6c7e00640 1 -- 172.21.15.44:0/698802923 reap_dead start 2024-09-13T08:49:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.427+0000 7fc6ce000640 1 -- 172.21.15.44:0/698802923 shutdown_connections 2024-09-13T08:49:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.427+0000 7fc6ce000640 1 -- 172.21.15.44:0/698802923 >> 172.21.15.44:0/698802923 conn(0x7fc6c80d1da0 msgr2=0x7fc6c80d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.427+0000 7fc6ce000640 1 -- 172.21.15.44:0/698802923 shutdown_connections 2024-09-13T08:49:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.427+0000 7fc6ce000640 1 -- 172.21.15.44:0/698802923 wait complete. 2024-09-13T08:49:17.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6ce000640 1 Processor -- start 2024-09-13T08:49:17.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6ce000640 1 -- start start 2024-09-13T08:49:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6ce000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6ac004d50 0x7fc6c80c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6ce000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6c81f5950 0x7fc6c81f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6ce000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c8158340 0x7fc6c81587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6ce000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c80cea90 con 0x7fc6c81f5950 2024-09-13T08:49:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6ce000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c80ce910 con 0x7fc6c8158340 2024-09-13T08:49:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6ce000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c80cddf0 con 0x7fc6ac004d50 2024-09-13T08:49:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6c81f5950 0x7fc6c81f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6c81f5950 0x7fc6c81f62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56860/0 (socket says 172.21.15.44:56860) 2024-09-13T08:49:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c6a00640 1 -- 172.21.15.44:0/2685673257 learned_addr learned my addr 172.21.15.44:0/2685673257 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c7e00640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c8158340 0x7fc6c81587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c7400640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6ac004d50 0x7fc6c80c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c6a00640 1 -- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6ac004d50 msgr2=0x7fc6c80c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c6a00640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6ac004d50 0x7fc6c80c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c6a00640 1 -- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c8158340 msgr2=0x7fc6c81587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c6a00640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c8158340 0x7fc6c81587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c6a00640 1 -- 172.21.15.44:0/2685673257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c81591a0 con 0x7fc6c81f5950 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6c6a00640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6c81f5950 0x7fc6c81f62f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4011ae0 tx=0x7fc6b4009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6bf400640 1 -- 172.21.15.44:0/2685673257 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400fcd0 con 0x7fc6c81f5950 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6bf400640 1 -- 172.21.15.44:0/2685673257 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6b400fe70 con 0x7fc6c81f5950 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6bf400640 1 -- 172.21.15.44:0/2685673257 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4004260 con 0x7fc6c81f5950 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6ce000640 1 -- 172.21.15.44:0/2685673257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c81f6830 con 0x7fc6c81f5950 2024-09-13T08:49:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.431+0000 7fc6ce000640 1 -- 172.21.15.44:0/2685673257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c80c7350 con 0x7fc6c81f5950 2024-09-13T08:49:17.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.435+0000 7fc6ce000640 1 -- 172.21.15.44:0/2685673257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c80cec10 con 0x7fc6c81f5950 2024-09-13T08:49:17.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.435+0000 7fc6bf400640 1 -- 172.21.15.44:0/2685673257 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fc6b4003600 con 0x7fc6c81f5950 2024-09-13T08:49:17.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.435+0000 7fc6bf400640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6a8074570 0x7fc6a8076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:17.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.435+0000 7fc6bf400640 1 -- 172.21.15.44:0/2685673257 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5879+0+0 (secure 0 0 0) 0x7fc6b4093ec0 con 0x7fc6c81f5950 2024-09-13T08:49:17.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.435+0000 7fc6c7400640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6a8074570 0x7fc6a8076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:17.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.439+0000 7fc6c7400640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6a8074570 0x7fc6a8076a30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009ba0 tx=0x7fc6b80091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:17.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.439+0000 7fc6bf400640 1 -- 172.21.15.44:0/2685673257 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc6b4060bf0 con 0x7fc6c81f5950 2024-09-13T08:49:17.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:17.615+0000 7fc6ce000640 1 -- 172.21.15.44:0/2685673257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc6c8070350 con 0x7fc6c81f5950 2024-09-13T08:49:18.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6bf400640 1 -- 172.21.15.44:0/2685673257 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7fc6b40604f0 con 0x7fc6c81f5950 2024-09-13T08:49:18.208 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T08:49:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 -- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6a8074570 msgr2=0x7fc6a8076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6a8074570 0x7fc6a8076a30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009ba0 tx=0x7fc6b80091e0 comp rx=0 tx=0).stop 2024-09-13T08:49:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 -- 172.21.15.44:0/2685673257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6c81f5950 msgr2=0x7fc6c81f62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6c81f5950 0x7fc6c81f62f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4011ae0 tx=0x7fc6b4009f20 comp rx=0 tx=0).stop 2024-09-13T08:49:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 -- 172.21.15.44:0/2685673257 shutdown_connections 2024-09-13T08:49:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6a8074570 0x7fc6a8076a30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c8158340 0x7fc6c81587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6c81f5950 0x7fc6c81f62f0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 --2- 172.21.15.44:0/2685673257 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc6ac004d50 0x7fc6c80c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 -- 172.21.15.44:0/2685673257 >> 172.21.15.44:0/2685673257 conn(0x7fc6c80d1da0 msgr2=0x7fc6c8154ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 -- 172.21.15.44:0/2685673257 shutdown_connections 2024-09-13T08:49:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.207+0000 7fc6ce000640 1 -- 172.21.15.44:0/2685673257 wait complete. 2024-09-13T08:49:18.237 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T08:49:18.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 -- 172.21.15.44:0/4078374024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e500051d0 msgr2=0x7f6e5000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:18.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 --2- 172.21.15.44:0/4078374024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e500051d0 0x7f6e5000b9d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009ee0 tx=0x7f6e6401d630 comp rx=0 tx=0).stop 2024-09-13T08:49:18.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 -- 172.21.15.44:0/4078374024 shutdown_connections 2024-09-13T08:49:18.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 --2- 172.21.15.44:0/4078374024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e500051d0 0x7f6e5000b9d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:18.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 -- 172.21.15.44:0/4078374024 >> 172.21.15.44:0/4078374024 conn(0x7f6e6c0d1da0 msgr2=0x7f6e6c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:18.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 -- 172.21.15.44:0/4078374024 shutdown_connections 2024-09-13T08:49:18.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 -- 172.21.15.44:0/4078374024 wait complete. 2024-09-13T08:49:18.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 Processor -- start 2024-09-13T08:49:18.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 -- start start 2024-09-13T08:49:18.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e500051d0 0x7f6e6c0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:18.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e6c1f7e80 0x7f6e6c0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:18.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e6c0cb430 0x7f6e6c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:18.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c0cea90 con 0x7f6e500051d0 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c0ce910 con 0x7f6e6c0cb430 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e73800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6e6c0cec10 con 0x7f6e6c1f7e80 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e70e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e500051d0 0x7f6e6c0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e70e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e500051d0 0x7f6e6c0c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56872/0 (socket says 172.21.15.44:56872) 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e70e00640 1 -- 172.21.15.44:0/840807711 learned_addr learned my addr 172.21.15.44:0/840807711 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e63e00640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e6c1f7e80 0x7f6e6c0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e71800640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e6c0cb430 0x7f6e6c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e70e00640 1 -- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e6c1f7e80 msgr2=0x7f6e6c0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e70e00640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e6c1f7e80 0x7f6e6c0b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e70e00640 1 -- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e6c0cb430 msgr2=0x7f6e6c0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e70e00640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e6c0cb430 0x7f6e6c0cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:18.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.399+0000 7f6e70e00640 1 -- 172.21.15.44:0/840807711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c1f8660 con 0x7f6e500051d0 2024-09-13T08:49:18.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.403+0000 7f6e70e00640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e500051d0 0x7f6e6c0c6050 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0126c0 tx=0x7f6e5c012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:18.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.403+0000 7f6e69600640 1 -- 172.21.15.44:0/840807711 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c006ca0 con 0x7f6e500051d0 2024-09-13T08:49:18.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.403+0000 7f6e69600640 1 -- 172.21.15.44:0/840807711 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e5c010ae0 con 0x7f6e500051d0 2024-09-13T08:49:18.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.403+0000 7f6e69600640 1 -- 172.21.15.44:0/840807711 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c013450 con 0x7f6e500051d0 2024-09-13T08:49:18.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.403+0000 7f6e73800640 1 -- 172.21.15.44:0/840807711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c1f8940 con 0x7f6e500051d0 2024-09-13T08:49:18.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.403+0000 7f6e73800640 1 -- 172.21.15.44:0/840807711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c0caf10 con 0x7f6e500051d0 2024-09-13T08:49:18.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.403+0000 7f6e73800640 1 -- 172.21.15.44:0/840807711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c0cf170 con 0x7f6e500051d0 2024-09-13T08:49:18.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.403+0000 7f6e69600640 1 -- 172.21.15.44:0/840807711 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f6e5c0135f0 con 0x7f6e500051d0 2024-09-13T08:49:18.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.403+0000 7f6e69600640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e48074540 0x7f6e48076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:18.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.403+0000 7f6e69600640 1 -- 172.21.15.44:0/840807711 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5441+0+0 (secure 0 0 0) 0x7f6e5c096460 con 0x7f6e500051d0 2024-09-13T08:49:18.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.407+0000 7f6e63e00640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e48074540 0x7f6e48076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:18.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.407+0000 7f6e63e00640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e48074540 0x7f6e48076a00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6e54000c10 tx=0x7f6e54006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:18.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.407+0000 7f6e69600640 1 -- 172.21.15.44:0/840807711 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6e5c062ac0 con 0x7f6e500051d0 2024-09-13T08:49:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:18.583+0000 7f6e73800640 1 -- 172.21.15.44:0/840807711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6e6c070350 con 0x7f6e500051d0 2024-09-13T08:49:19.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.211+0000 7f6e69600640 1 -- 172.21.15.44:0/840807711 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f6e5c0674f0 con 0x7f6e500051d0 2024-09-13T08:49:19.214 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T08:49:19.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 -- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e48074540 msgr2=0x7f6e48076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e48074540 0x7f6e48076a00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6e54000c10 tx=0x7f6e54006b80 comp rx=0 tx=0).stop 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 -- 172.21.15.44:0/840807711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e500051d0 msgr2=0x7f6e6c0c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e500051d0 0x7f6e6c0c6050 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0126c0 tx=0x7f6e5c012b80 comp rx=0 tx=0).stop 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 -- 172.21.15.44:0/840807711 shutdown_connections 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6e48074540 0x7f6e48076a00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e6c0cb430 0x7f6e6c0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6e6c1f7e80 0x7f6e6c0b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 --2- 172.21.15.44:0/840807711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e500051d0 0x7f6e6c0c6050 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 -- 172.21.15.44:0/840807711 >> 172.21.15.44:0/840807711 conn(0x7f6e6c0d1da0 msgr2=0x7f6e6c071ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 -- 172.21.15.44:0/840807711 shutdown_connections 2024-09-13T08:49:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.215+0000 7f6e73800640 1 -- 172.21.15.44:0/840807711 wait complete. 2024-09-13T08:49:19.249 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-13T08:49:19.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.395+0000 7f0568000640 1 -- 172.21.15.44:0/3465121190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0560072c90 msgr2=0x7f056015c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:19.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.395+0000 7f0568000640 1 --2- 172.21.15.44:0/3465121190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0560072c90 0x7f056015c9c0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f054c009840 tx=0x7f054c01c7f0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.395+0000 7f0568000640 1 -- 172.21.15.44:0/3465121190 shutdown_connections 2024-09-13T08:49:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.395+0000 7f0568000640 1 --2- 172.21.15.44:0/3465121190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0560072c90 0x7f056015c9c0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.395+0000 7f0568000640 1 --2- 172.21.15.44:0/3465121190 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f056015c5e0 0x7f056006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.395+0000 7f0568000640 1 -- 172.21.15.44:0/3465121190 >> 172.21.15.44:0/3465121190 conn(0x7f05600b1c20 msgr2=0x7f05600b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0568000640 1 -- 172.21.15.44:0/3465121190 shutdown_connections 2024-09-13T08:49:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0568000640 1 -- 172.21.15.44:0/3465121190 wait complete. 2024-09-13T08:49:19.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0568000640 1 Processor -- start 2024-09-13T08:49:19.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0568000640 1 -- start start 2024-09-13T08:49:19.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0568000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0560072c90 0x7f05601553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:19.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0568000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05600744c0 0x7f056013a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:19.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0568000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f056015c5e0 0x7f056013a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:19.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0568000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0560152bf0 con 0x7f05600744c0 2024-09-13T08:49:19.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0568000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0560152a70 con 0x7f0560072c90 2024-09-13T08:49:19.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0568000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0560152d70 con 0x7f056015c5e0 2024-09-13T08:49:19.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0564c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05600744c0 0x7f056013a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:19.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0564c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05600744c0 0x7f056013a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56884/0 (socket says 172.21.15.44:56884) 2024-09-13T08:49:19.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0564c00640 1 -- 172.21.15.44:0/2035752187 learned_addr learned my addr 172.21.15.44:0/2035752187 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:19.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0565600640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0560072c90 0x7f05601553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:19.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0566000640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f056015c5e0 0x7f056013a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:19.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0564c00640 1 -- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f056015c5e0 msgr2=0x7f056013a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:19.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0564c00640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f056015c5e0 0x7f056013a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0564c00640 1 -- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0560072c90 msgr2=0x7f05601553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:19.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0564c00640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0560072c90 0x7f05601553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0564c00640 1 -- 172.21.15.44:0/2035752187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05601562a0 con 0x7f05600744c0 2024-09-13T08:49:19.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0565600640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0560072c90 0x7f05601553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:19.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0566000640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f056015c5e0 0x7f056013a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:19.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f0564c00640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05600744c0 0x7f056013a4a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f054c01aea0 tx=0x7f054c0a5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:19.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.399+0000 7f055d600640 1 -- 172.21.15.44:0/2035752187 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c0b2040 con 0x7f05600744c0 2024-09-13T08:49:19.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.403+0000 7f055d600640 1 -- 172.21.15.44:0/2035752187 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f054c0044b0 con 0x7f05600744c0 2024-09-13T08:49:19.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.403+0000 7f0568000640 1 -- 172.21.15.44:0/2035752187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056013af20 con 0x7f05600744c0 2024-09-13T08:49:19.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.403+0000 7f055d600640 1 -- 172.21.15.44:0/2035752187 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c005260 con 0x7f05600744c0 2024-09-13T08:49:19.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.403+0000 7f0568000640 1 -- 172.21.15.44:0/2035752187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0560155a00 con 0x7f05600744c0 2024-09-13T08:49:19.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.403+0000 7f0568000640 1 -- 172.21.15.44:0/2035752187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056015c9c0 con 0x7f05600744c0 2024-09-13T08:49:19.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.403+0000 7f055d600640 1 -- 172.21.15.44:0/2035752187 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f054c0be060 con 0x7f05600744c0 2024-09-13T08:49:19.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.403+0000 7f055d600640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0540074540 0x7f0540076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:19.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.403+0000 7f055d600640 1 -- 172.21.15.44:0/2035752187 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5027+0+0 (secure 0 0 0) 0x7f054c12c5a0 con 0x7f05600744c0 2024-09-13T08:49:19.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.403+0000 7f0565600640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0540074540 0x7f0540076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:19.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.407+0000 7f0565600640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0540074540 0x7f0540076a00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0554005d10 tx=0x7f0554005de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:19.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.407+0000 7f055d600640 1 -- 172.21.15.44:0/2035752187 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f054c016410 con 0x7f05600744c0 2024-09-13T08:49:19.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.595+0000 7f0568000640 1 -- 172.21.15.44:0/2035752187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f056006a330 con 0x7f05600744c0 2024-09-13T08:49:19.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f055d600640 1 -- 172.21.15.44:0/2035752187 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7f054c0f91d0 con 0x7f05600744c0 2024-09-13T08:49:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 -- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0540074540 msgr2=0x7f0540076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0540074540 0x7f0540076a00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0554005d10 tx=0x7f0554005de0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 -- 172.21.15.44:0/2035752187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05600744c0 msgr2=0x7f056013a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05600744c0 0x7f056013a4a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f054c01aea0 tx=0x7f054c0a5840 comp rx=0 tx=0).stop 2024-09-13T08:49:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 -- 172.21.15.44:0/2035752187 shutdown_connections 2024-09-13T08:49:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0540074540 0x7f0540076a00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f056015c5e0 0x7f056013a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05600744c0 0x7f056013a4a0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 --2- 172.21.15.44:0/2035752187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0560072c90 0x7f05601553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 -- 172.21.15.44:0/2035752187 >> 172.21.15.44:0/2035752187 conn(0x7f05600b1c20 msgr2=0x7f05600b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 -- 172.21.15.44:0/2035752187 shutdown_connections 2024-09-13T08:49:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.599+0000 7f0568000640 1 -- 172.21.15.44:0/2035752187 wait complete. 2024-09-13T08:49:19.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-13T08:49:19.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.767+0000 7fdbe0600640 1 -- 172.21.15.44:0/2692174861 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbd80b49e0 msgr2=0x7fdbd806f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:19.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.767+0000 7fdbe0600640 1 --2- 172.21.15.44:0/2692174861 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbd80b49e0 0x7fdbd806f7b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009c80 tx=0x7fdbd001cd10 comp rx=0 tx=0).stop 2024-09-13T08:49:19.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.767+0000 7fdbe0600640 1 -- 172.21.15.44:0/2692174861 shutdown_connections 2024-09-13T08:49:19.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.767+0000 7fdbe0600640 1 --2- 172.21.15.44:0/2692174861 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbd80b49e0 0x7fdbd806f7b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.767+0000 7fdbe0600640 1 --2- 172.21.15.44:0/2692174861 >> v2:172.21.15.129:3300/0 conn(0x7fdbd815dae0 0x7fdbd80acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.767+0000 7fdbe0600640 1 -- 172.21.15.44:0/2692174861 >> 172.21.15.44:0/2692174861 conn(0x7fdbd80b18f0 msgr2=0x7fdbd80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:19.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.767+0000 7fdbe0600640 1 -- 172.21.15.44:0/2692174861 shutdown_connections 2024-09-13T08:49:19.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.767+0000 7fdbe0600640 1 -- 172.21.15.44:0/2692174861 wait complete. 2024-09-13T08:49:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 Processor -- start 2024-09-13T08:49:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 -- start start 2024-09-13T08:49:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbd80b49e0 0x7fdbd81564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd80cd100 0x7fdbd8153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbd815dae0 0x7fdbd8153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd80cf120 con 0x7fdbd815dae0 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd80cefa0 con 0x7fdbd80cd100 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd80cf2a0 con 0x7fdbd80b49e0 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbde600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbd815dae0 0x7fdbd8153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbde600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbd815dae0 0x7fdbd8153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56900/0 (socket says 172.21.15.44:56900) 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbde600640 1 -- 172.21.15.44:0/3427577206 learned_addr learned my addr 172.21.15.44:0/3427577206 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbddc00640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbd80b49e0 0x7fdbd81564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbdd200640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd80cd100 0x7fdbd8153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbde600640 1 -- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbd80b49e0 msgr2=0x7fdbd81564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbde600640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbd80b49e0 0x7fdbd81564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbde600640 1 -- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd80cd100 msgr2=0x7fdbd8153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbde600640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd80cd100 0x7fdbd8153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbde600640 1 -- 172.21.15.44:0/3427577206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd8164ea0 con 0x7fdbd815dae0 2024-09-13T08:49:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbde600640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbd815dae0 0x7fdbd8153e90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fdbd001c780 tx=0x7fdbd0005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbd5600640 1 -- 172.21.15.44:0/3427577206 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd00b2040 con 0x7fdbd815dae0 2024-09-13T08:49:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbd5600640 1 -- 172.21.15.44:0/3427577206 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbd00bb070 con 0x7fdbd815dae0 2024-09-13T08:49:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbd5600640 1 -- 172.21.15.44:0/3427577206 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0004330 con 0x7fdbd815dae0 2024-09-13T08:49:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 -- 172.21.15.44:0/3427577206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd80b0b20 con 0x7fdbd815dae0 2024-09-13T08:49:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 -- 172.21.15.44:0/3427577206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd8164980 con 0x7fdbd815dae0 2024-09-13T08:49:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.771+0000 7fdbe0600640 1 -- 172.21.15.44:0/3427577206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd8069410 con 0x7fdbd815dae0 2024-09-13T08:49:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.775+0000 7fdbd5600640 1 -- 172.21.15.44:0/3427577206 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fdbd00044d0 con 0x7fdbd815dae0 2024-09-13T08:49:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.775+0000 7fdbd5600640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbb8074540 0x7fdbb8076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.775+0000 7fdbd5600640 1 -- 172.21.15.44:0/3427577206 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5027+0+0 (secure 0 0 0) 0x7fdbd012d1b0 con 0x7fdbd815dae0 2024-09-13T08:49:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.775+0000 7fdbddc00640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbb8074540 0x7fdbb8076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.775+0000 7fdbddc00640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbb8074540 0x7fdbb8076a00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009600 tx=0x7fdbcc0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:19.779+0000 7fdbd5600640 1 -- 172.21.15.44:0/3427577206 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdbd0016990 con 0x7fdbd815dae0 2024-09-13T08:49:20.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.039+0000 7fdbe0600640 1 -- 172.21.15.44:0/3427577206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7fdbd8158c70 con 0x7fdbd815dae0 2024-09-13T08:49:20.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbd5600640 1 -- 172.21.15.44:0/3427577206 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v527) ==== 174+0+0 (secure 0 0 0) 0x7fdbd00f9f70 con 0x7fdbd815dae0 2024-09-13T08:49:20.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 -- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbb8074540 msgr2=0x7fdbb8076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbb8074540 0x7fdbb8076a00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009600 tx=0x7fdbcc0091e0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 -- 172.21.15.44:0/3427577206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbd815dae0 msgr2=0x7fdbd8153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbd815dae0 0x7fdbd8153e90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fdbd001c780 tx=0x7fdbd0005c20 comp rx=0 tx=0).stop 2024-09-13T08:49:20.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 -- 172.21.15.44:0/3427577206 shutdown_connections 2024-09-13T08:49:20.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdbb8074540 0x7fdbb8076a00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbd815dae0 0x7fdbd8153e90 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd80cd100 0x7fdbd8153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 --2- 172.21.15.44:0/3427577206 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdbd80b49e0 0x7fdbd81564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 -- 172.21.15.44:0/3427577206 >> 172.21.15.44:0/3427577206 conn(0x7fdbd80b18f0 msgr2=0x7fdbd8154ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:20.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 -- 172.21.15.44:0/3427577206 shutdown_connections 2024-09-13T08:49:20.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.215+0000 7fdbe0600640 1 -- 172.21.15.44:0/3427577206 wait complete. 2024-09-13T08:49:20.239 INFO:tasks.cephfs_test_runner:Test that progress indicated by progress bar for ongoing+pending clones ... ok 2024-09-13T08:49:20.239 INFO:tasks.cephfs_test_runner:Starting test: test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the command "ceph fs clone status" prints progress stats 2024-09-13T08:49:20.239 INFO:tasks.cephfs_test_runner:test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-13T08:49:20.240 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:49:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.383+0000 7f8091200640 1 -- 172.21.15.44:0/2115853364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f808c15dae0 msgr2=0x7f808c0acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.383+0000 7f8091200640 1 --2- 172.21.15.44:0/2115853364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f808c15dae0 0x7f808c0acd40 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f807c009840 tx=0x7f807c01c700 comp rx=0 tx=0).stop 2024-09-13T08:49:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.383+0000 7f8091200640 1 -- 172.21.15.44:0/2115853364 shutdown_connections 2024-09-13T08:49:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.383+0000 7f8091200640 1 --2- 172.21.15.44:0/2115853364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f808c15dae0 0x7f808c0acd40 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.383+0000 7f8091200640 1 -- 172.21.15.44:0/2115853364 >> 172.21.15.44:0/2115853364 conn(0x7f808c0b18f0 msgr2=0x7f808c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.383+0000 7f8091200640 1 -- 172.21.15.44:0/2115853364 shutdown_connections 2024-09-13T08:49:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.383+0000 7f8091200640 1 -- 172.21.15.44:0/2115853364 wait complete. 2024-09-13T08:49:20.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8091200640 1 Processor -- start 2024-09-13T08:49:20.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8091200640 1 -- start start 2024-09-13T08:49:20.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8091200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f808c0b49e0 0x7f808c09c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:20.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8091200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f808c0cd100 0x7f808c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:20.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8091200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f808c15dae0 0x7f808c1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:20.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8091200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0ccbb0 con 0x7f808c0b49e0 2024-09-13T08:49:20.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8091200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0cca30 con 0x7f808c0cd100 2024-09-13T08:49:20.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8091200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f808c0ccd30 con 0x7f808c15dae0 2024-09-13T08:49:20.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8089e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f808c0b49e0 0x7f808c09c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:20.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8089e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f808c0b49e0 0x7f808c09c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56914/0 (socket says 172.21.15.44:56914) 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8089e00640 1 -- 172.21.15.44:0/1097377841 learned_addr learned my addr 172.21.15.44:0/1097377841 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8083e00640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f808c0cd100 0x7f808c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f808a800640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f808c15dae0 0x7f808c1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8089e00640 1 -- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f808c15dae0 msgr2=0x7f808c1552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8089e00640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f808c15dae0 0x7f808c1552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8089e00640 1 -- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f808c0cd100 msgr2=0x7f808c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8089e00640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f808c0cd100 0x7f808c154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8089e00640 1 -- 172.21.15.44:0/1097377841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c09b600 con 0x7f808c0b49e0 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f808a800640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f808c15dae0 0x7f808c1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8089e00640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f808c0b49e0 0x7f808c09c980 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f807c000c10 tx=0x7f807c0a86e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:20.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8082000640 1 -- 172.21.15.44:0/1097377841 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c0a8a70 con 0x7f808c0b49e0 2024-09-13T08:49:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8082000640 1 -- 172.21.15.44:0/1097377841 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f807c0a8c10 con 0x7f808c0b49e0 2024-09-13T08:49:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8082000640 1 -- 172.21.15.44:0/1097377841 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c005140 con 0x7f808c0b49e0 2024-09-13T08:49:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8091200640 1 -- 172.21.15.44:0/1097377841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c09b8e0 con 0x7f808c0b49e0 2024-09-13T08:49:20.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.387+0000 7f8091200640 1 -- 172.21.15.44:0/1097377841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c09bea0 con 0x7f808c0b49e0 2024-09-13T08:49:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.391+0000 7f8091200640 1 -- 172.21.15.44:0/1097377841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c069410 con 0x7f808c0b49e0 2024-09-13T08:49:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.391+0000 7f8082000640 1 -- 172.21.15.44:0/1097377841 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f807c0052e0 con 0x7f808c0b49e0 2024-09-13T08:49:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.391+0000 7f8082000640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8068074540 0x7f8068076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.391+0000 7f8082000640 1 -- 172.21.15.44:0/1097377841 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5027+0+0 (secure 0 0 0) 0x7f807c12cd10 con 0x7f808c0b49e0 2024-09-13T08:49:20.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.391+0000 7f8083e00640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8068074540 0x7f8068076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:20.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.391+0000 7f8083e00640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8068074540 0x7f8068076a00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f808c070520 tx=0x7f8074009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:20.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.395+0000 7f8082000640 1 -- 172.21.15.44:0/1097377841 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f807c0c1030 con 0x7f808c0b49e0 2024-09-13T08:49:20.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.575+0000 7f8091200640 1 -- 172.21.15.44:0/1097377841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f808c06adb0 con 0x7f808c0b49e0 2024-09-13T08:49:20.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.575+0000 7f8082000640 1 -- 172.21.15.44:0/1097377841 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+13177 (secure 0 0 0) 0x7f807c0ac030 con 0x7f808c0b49e0 2024-09-13T08:49:20.579 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:20.580 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":94,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:49:19.211161+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/1796790158":"2024-09-14T08:49:16.157998+0000","172.21.15.44:6834/1796790158":"2024-09-14T08:49:16.157998+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:49:20.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 -- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8068074540 msgr2=0x7f8068076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8068074540 0x7f8068076a00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f808c070520 tx=0x7f8074009040 comp rx=0 tx=0).stop 2024-09-13T08:49:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 -- 172.21.15.44:0/1097377841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f808c0b49e0 msgr2=0x7f808c09c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f808c0b49e0 0x7f808c09c980 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f807c000c10 tx=0x7f807c0a86e0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 -- 172.21.15.44:0/1097377841 shutdown_connections 2024-09-13T08:49:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8068074540 0x7f8068076a00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f808c15dae0 0x7f808c1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f808c0cd100 0x7f808c154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 --2- 172.21.15.44:0/1097377841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f808c0b49e0 0x7f808c09c980 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 -- 172.21.15.44:0/1097377841 >> 172.21.15.44:0/1097377841 conn(0x7f808c0b18f0 msgr2=0x7f808c0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 -- 172.21.15.44:0/1097377841 shutdown_connections 2024-09-13T08:49:20.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.579+0000 7f8091200640 1 -- 172.21.15.44:0/1097377841 wait complete. 2024-09-13T08:49:20.602 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T08:37:24.563344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T08:49:20.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T08:49:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 -- 172.21.15.44:0/1033192368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d780b0d10 msgr2=0x7f9d780d1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 --2- 172.21.15.44:0/1033192368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d780b0d10 0x7f9d780d1bb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009840 tx=0x7f9d6001c820 comp rx=0 tx=0).stop 2024-09-13T08:49:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 -- 172.21.15.44:0/1033192368 shutdown_connections 2024-09-13T08:49:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 --2- 172.21.15.44:0/1033192368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d780b0d10 0x7f9d780d1bb0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 -- 172.21.15.44:0/1033192368 >> 172.21.15.44:0/1033192368 conn(0x7f9d7809aa10 msgr2=0x7f9d7809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 -- 172.21.15.44:0/1033192368 shutdown_connections 2024-09-13T08:49:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 -- 172.21.15.44:0/1033192368 wait complete. 2024-09-13T08:49:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 Processor -- start 2024-09-13T08:49:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 -- start start 2024-09-13T08:49:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d780b0d10 0x7f9d781fb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d780d20f0 0x7f9d78158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d7815bd40 0x7f9d780ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78169e80 con 0x7f9d780b0d10 2024-09-13T08:49:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78169d00 con 0x7f9d780d20f0 2024-09-13T08:49:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d77200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7816a000 con 0x7f9d7815bd40 2024-09-13T08:49:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d75e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d780b0d10 0x7f9d781fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d75e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d780b0d10 0x7f9d781fb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56942/0 (socket says 172.21.15.44:56942) 2024-09-13T08:49:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.751+0000 7f9d75e00640 1 -- 172.21.15.44:0/1670762702 learned_addr learned my addr 172.21.15.44:0/1670762702 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d6fe00640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d780d20f0 0x7f9d78158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d76800640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d7815bd40 0x7f9d780ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d75e00640 1 -- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d7815bd40 msgr2=0x7f9d780ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d75e00640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d7815bd40 0x7f9d780ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d75e00640 1 -- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d780d20f0 msgr2=0x7f9d78158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d75e00640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d780d20f0 0x7f9d78158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d75e00640 1 -- 172.21.15.44:0/1670762702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d781fba70 con 0x7f9d780b0d10 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d76800640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d7815bd40 0x7f9d780ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d75e00640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d780b0d10 0x7f9d781fb530 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c3f0 tx=0x7f9d6800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d6e000640 1 -- 172.21.15.44:0/1670762702 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68017070 con 0x7f9d780b0d10 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d6e000640 1 -- 172.21.15.44:0/1670762702 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d68003600 con 0x7f9d780b0d10 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d6e000640 1 -- 172.21.15.44:0/1670762702 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68003a50 con 0x7f9d780b0d10 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d77200640 1 -- 172.21.15.44:0/1670762702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7815c890 con 0x7f9d780b0d10 2024-09-13T08:49:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d77200640 1 -- 172.21.15.44:0/1670762702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7815ce50 con 0x7f9d780b0d10 2024-09-13T08:49:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d77200640 1 -- 172.21.15.44:0/1670762702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d7815d560 con 0x7f9d780b0d10 2024-09-13T08:49:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.755+0000 7f9d6e000640 1 -- 172.21.15.44:0/1670762702 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f9d6801a070 con 0x7f9d780b0d10 2024-09-13T08:49:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.759+0000 7f9d6e000640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d54074570 0x7f9d54076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:20.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.759+0000 7f9d6e000640 1 -- 172.21.15.44:0/1670762702 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5027+0+0 (secure 0 0 0) 0x7f9d68091d10 con 0x7f9d780b0d10 2024-09-13T08:49:20.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.759+0000 7f9d6fe00640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d54074570 0x7f9d54076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:20.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.759+0000 7f9d6fe00640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d54074570 0x7f9d54076a30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9d600af000 tx=0x7f9d6001b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:20.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.759+0000 7f9d6e000640 1 -- 172.21.15.44:0/1670762702 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9d781fba70 con 0x7f9d780b0d10 2024-09-13T08:49:20.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.935+0000 7f9d77200640 1 -- 172.21.15.44:0/1670762702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9d780d1bb0 con 0x7f9d780b0d10 2024-09-13T08:49:20.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.935+0000 7f9d6e000640 1 -- 172.21.15.44:0/1670762702 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f9d6805ea60 con 0x7f9d780b0d10 2024-09-13T08:49:20.939 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T08:49:20.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 -- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d54074570 msgr2=0x7f9d54076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d54074570 0x7f9d54076a30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9d600af000 tx=0x7f9d6001b060 comp rx=0 tx=0).stop 2024-09-13T08:49:20.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 -- 172.21.15.44:0/1670762702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d780b0d10 msgr2=0x7f9d781fb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d780b0d10 0x7f9d781fb530 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c3f0 tx=0x7f9d6800c8b0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 -- 172.21.15.44:0/1670762702 shutdown_connections 2024-09-13T08:49:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9d54074570 0x7f9d54076a30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9d7815bd40 0x7f9d780ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d780d20f0 0x7f9d78158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 --2- 172.21.15.44:0/1670762702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d780b0d10 0x7f9d781fb530 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 -- 172.21.15.44:0/1670762702 >> 172.21.15.44:0/1670762702 conn(0x7f9d7809aa10 msgr2=0x7f9d7806ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 -- 172.21.15.44:0/1670762702 shutdown_connections 2024-09-13T08:49:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:20.939+0000 7f9d77200640 1 -- 172.21.15.44:0/1670762702 wait complete. 2024-09-13T08:49:20.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-13T08:49:21.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.103+0000 7fc42cc00640 1 -- 172.21.15.44:0/1372775503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc42815e330 msgr2=0x7fc42806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:21.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.103+0000 7fc42cc00640 1 --2- 172.21.15.44:0/1372775503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc42815e330 0x7fc42806ec80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fc410009900 tx=0x7fc41001c7c0 comp rx=0 tx=0).stop 2024-09-13T08:49:21.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.103+0000 7fc42cc00640 1 -- 172.21.15.44:0/1372775503 shutdown_connections 2024-09-13T08:49:21.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.103+0000 7fc42cc00640 1 --2- 172.21.15.44:0/1372775503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc42815e330 0x7fc42806ec80 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:21.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.103+0000 7fc42cc00640 1 --2- 172.21.15.44:0/1372775503 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42815df50 0x7fc42806e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:21.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.103+0000 7fc42cc00640 1 -- 172.21.15.44:0/1372775503 >> 172.21.15.44:0/1372775503 conn(0x7fc4280d1dc0 msgr2=0x7fc4280d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:21.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.103+0000 7fc42cc00640 1 -- 172.21.15.44:0/1372775503 shutdown_connections 2024-09-13T08:49:21.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.103+0000 7fc42cc00640 1 -- 172.21.15.44:0/1372775503 wait complete. 2024-09-13T08:49:21.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc42cc00640 1 Processor -- start 2024-09-13T08:49:21.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc42cc00640 1 -- start start 2024-09-13T08:49:21.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc42cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4280b11d0 0x7fc428156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc42cc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42815df50 0x7fc4280c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc42cc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42815e330 0x7fc42809bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc42cc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc4280d0790 con 0x7fc4280b11d0 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc42cc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4280d0610 con 0x7fc42815e330 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc42cc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc4280d0910 con 0x7fc42815df50 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc425e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4280b11d0 0x7fc428156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc425e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4280b11d0 0x7fc428156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56968/0 (socket says 172.21.15.44:56968) 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc425e00640 1 -- 172.21.15.44:0/1529291547 learned_addr learned my addr 172.21.15.44:0/1529291547 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc41d400640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42815df50 0x7fc4280c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc426800640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42815e330 0x7fc42809bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc425e00640 1 -- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42815df50 msgr2=0x7fc4280c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:21.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc425e00640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42815df50 0x7fc4280c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:21.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc425e00640 1 -- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42815e330 msgr2=0x7fc42809bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:21.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc425e00640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42815e330 0x7fc42809bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:21.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc425e00640 1 -- 172.21.15.44:0/1529291547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4280c6c50 con 0x7fc4280b11d0 2024-09-13T08:49:21.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc425e00640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4280b11d0 0x7fc428156370 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d980 tx=0x7fc41800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:21.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc41ea00640 1 -- 172.21.15.44:0/1529291547 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418017070 con 0x7fc4280b11d0 2024-09-13T08:49:21.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc41ea00640 1 -- 172.21.15.44:0/1529291547 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc418009d20 con 0x7fc4280b11d0 2024-09-13T08:49:21.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc41ea00640 1 -- 172.21.15.44:0/1529291547 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4180106c0 con 0x7fc4280b11d0 2024-09-13T08:49:21.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc42cc00640 1 -- 172.21.15.44:0/1529291547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4280c6f30 con 0x7fc4280b11d0 2024-09-13T08:49:21.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.107+0000 7fc42cc00640 1 -- 172.21.15.44:0/1529291547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4280c74f0 con 0x7fc4280b11d0 2024-09-13T08:49:21.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.111+0000 7fc42cc00640 1 -- 172.21.15.44:0/1529291547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4280afcd0 con 0x7fc4280b11d0 2024-09-13T08:49:21.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.111+0000 7fc41ea00640 1 -- 172.21.15.44:0/1529291547 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fc418010860 con 0x7fc4280b11d0 2024-09-13T08:49:21.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.111+0000 7fc41ea00640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc404074610 0x7fc404076ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:21.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.111+0000 7fc41ea00640 1 -- 172.21.15.44:0/1529291547 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5027+0+0 (secure 0 0 0) 0x7fc418092ec0 con 0x7fc4280b11d0 2024-09-13T08:49:21.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.111+0000 7fc41d400640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc404074610 0x7fc404076ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:21.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.111+0000 7fc41d400640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc404074610 0x7fc404076ad0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc4100af000 tx=0x7fc410019560 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:21.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.115+0000 7fc41ea00640 1 -- 172.21.15.44:0/1529291547 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc41805fbf0 con 0x7fc4280b11d0 2024-09-13T08:49:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:21.351+0000 7fc42cc00640 1 -- 172.21.15.44:0/1529291547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7fc428157ab0 con 0x7fc4280b11d0 2024-09-13T08:49:22.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.223+0000 7fc41ea00640 1 -- 172.21.15.44:0/1529291547 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v529) ==== 181+0+0 (secure 0 0 0) 0x7fc41805f4f0 con 0x7fc4280b11d0 2024-09-13T08:49:22.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 -- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc404074610 msgr2=0x7fc404076ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc404074610 0x7fc404076ad0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc4100af000 tx=0x7fc410019560 comp rx=0 tx=0).stop 2024-09-13T08:49:22.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 -- 172.21.15.44:0/1529291547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4280b11d0 msgr2=0x7fc428156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4280b11d0 0x7fc428156370 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d980 tx=0x7fc41800de40 comp rx=0 tx=0).stop 2024-09-13T08:49:22.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 -- 172.21.15.44:0/1529291547 shutdown_connections 2024-09-13T08:49:22.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc404074610 0x7fc404076ad0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc42815e330 0x7fc42809bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc42815df50 0x7fc4280c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 --2- 172.21.15.44:0/1529291547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4280b11d0 0x7fc428156370 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 -- 172.21.15.44:0/1529291547 >> 172.21.15.44:0/1529291547 conn(0x7fc4280d1dc0 msgr2=0x7fc428071680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:22.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 -- 172.21.15.44:0/1529291547 shutdown_connections 2024-09-13T08:49:22.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.227+0000 7fc42cc00640 1 -- 172.21.15.44:0/1529291547 wait complete. 2024-09-13T08:49:22.252 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-13T08:49:22.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 -- 172.21.15.44:0/4058641956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01800d28c0 msgr2=0x7f01800cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 --2- 172.21.15.44:0/4058641956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01800d28c0 0x7f01800cea20 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0180074290 tx=0x7f016c01c8c0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 -- 172.21.15.44:0/4058641956 shutdown_connections 2024-09-13T08:49:22.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 --2- 172.21.15.44:0/4058641956 >> v2:172.21.15.129:3300/0 conn(0x7f01800b53f0 0x7f01800b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 --2- 172.21.15.44:0/4058641956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01800d28c0 0x7f01800cea20 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 --2- 172.21.15.44:0/4058641956 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f018015be80 0x7f01800ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 -- 172.21.15.44:0/4058641956 >> 172.21.15.44:0/4058641956 conn(0x7f01800d1da0 msgr2=0x7f01800d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:22.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 -- 172.21.15.44:0/4058641956 shutdown_connections 2024-09-13T08:49:22.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 -- 172.21.15.44:0/4058641956 wait complete. 2024-09-13T08:49:22.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 Processor -- start 2024-09-13T08:49:22.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 -- start start 2024-09-13T08:49:22.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01800b53f0 0x7f01801598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:22.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01800d28c0 0x7f0180156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f018015be80 0x7f01800caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01800ccfa0 con 0x7f01800b53f0 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01800cce20 con 0x7f018015be80 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0187a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f01800cd120 con 0x7f01800d28c0 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0185000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01800b53f0 0x7f01801598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0185000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01800b53f0 0x7f01801598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59956/0 (socket says 172.21.15.44:59956) 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0185000640 1 -- 172.21.15.44:0/1177728601 learned_addr learned my addr 172.21.15.44:0/1177728601 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f0185a00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f018015be80 0x7f01800caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.403+0000 7f017fe00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01800d28c0 0x7f0180156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f0185000640 1 -- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01800d28c0 msgr2=0x7f0180156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f0185000640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01800d28c0 0x7f0180156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f0185000640 1 -- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f018015be80 msgr2=0x7f01800caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f0185000640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f018015be80 0x7f01800caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f0185000640 1 -- 172.21.15.44:0/1177728601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0180166840 con 0x7f01800b53f0 2024-09-13T08:49:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f0185000640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01800b53f0 0x7f01801598b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f017400c9f0 tx=0x7f017400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f017cc00640 1 -- 172.21.15.44:0/1177728601 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174007b50 con 0x7f01800b53f0 2024-09-13T08:49:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f017cc00640 1 -- 172.21.15.44:0/1177728601 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0174007de0 con 0x7f01800b53f0 2024-09-13T08:49:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f017cc00640 1 -- 172.21.15.44:0/1177728601 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017400f6b0 con 0x7f01800b53f0 2024-09-13T08:49:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f0187a00640 1 -- 172.21.15.44:0/1177728601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01801586f0 con 0x7f01800b53f0 2024-09-13T08:49:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f0187a00640 1 -- 172.21.15.44:0/1177728601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0180157e50 con 0x7f01800b53f0 2024-09-13T08:49:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f0187a00640 1 -- 172.21.15.44:0/1177728601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01800b5880 con 0x7f01800b53f0 2024-09-13T08:49:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f017cc00640 1 -- 172.21.15.44:0/1177728601 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f017400f850 con 0x7f01800b53f0 2024-09-13T08:49:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f017cc00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0160074570 0x7f0160076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.407+0000 7f017cc00640 1 -- 172.21.15.44:0/1177728601 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5027+0+0 (secure 0 0 0) 0x7f01740931c0 con 0x7f01800b53f0 2024-09-13T08:49:22.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.411+0000 7f017fe00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0160074570 0x7f0160076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:22.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.411+0000 7f017fe00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0160074570 0x7f0160076a30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f016c0af000 tx=0x7f016c01b100 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:22.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.411+0000 7f017cc00640 1 -- 172.21.15.44:0/1177728601 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0174014030 con 0x7f01800b53f0 2024-09-13T08:49:22.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 -- 172.21.15.44:0/1177728601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f0180159df0 con 0x7f01800b53f0 2024-09-13T08:49:22.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f017cc00640 1 -- 172.21.15.44:0/1177728601 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+1998 (secure 0 0 0) 0x7f017405ff10 con 0x7f01800b53f0 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4094.620852 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3964.014083 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3978.263337 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4002.298162 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4667.000623 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4664.046790 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6403.016405 2024-09-13T08:49:22.593 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4696.908546 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 2/2 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-09-13T08:49:22.594 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-09-13T08:49:22.595 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-09-13T08:49:22.595 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-09-13T08:49:22.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 -- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0160074570 msgr2=0x7f0160076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0160074570 0x7f0160076a30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f016c0af000 tx=0x7f016c01b100 comp rx=0 tx=0).stop 2024-09-13T08:49:22.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 -- 172.21.15.44:0/1177728601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01800b53f0 msgr2=0x7f01801598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01800b53f0 0x7f01801598b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f017400c9f0 tx=0x7f017400ceb0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 -- 172.21.15.44:0/1177728601 shutdown_connections 2024-09-13T08:49:22.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0160074570 0x7f0160076a30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f018015be80 0x7f01800caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f01800d28c0 0x7f0180156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 --2- 172.21.15.44:0/1177728601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01800b53f0 0x7f01801598b0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 -- 172.21.15.44:0/1177728601 >> 172.21.15.44:0/1177728601 conn(0x7f01800d1da0 msgr2=0x7f01800b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:22.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 -- 172.21.15.44:0/1177728601 shutdown_connections 2024-09-13T08:49:22.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.591+0000 7f0187a00640 1 -- 172.21.15.44:0/1177728601 wait complete. 2024-09-13T08:49:22.615 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:49:22.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.763+0000 7f42e9200640 1 -- 172.21.15.44:0/1990619127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e40d22c0 msgr2=0x7f42e40cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.763+0000 7f42e9200640 1 --2- 172.21.15.44:0/1990619127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e40d22c0 0x7f42e40cc620 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009bc0 tx=0x7f42dc01cba0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.763+0000 7f42e9200640 1 -- 172.21.15.44:0/1990619127 shutdown_connections 2024-09-13T08:49:22.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.763+0000 7f42e9200640 1 --2- 172.21.15.44:0/1990619127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e40d22c0 0x7f42e40cc620 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.763+0000 7f42e9200640 1 --2- 172.21.15.44:0/1990619127 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42e415bd40 0x7f42e40d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.763+0000 7f42e9200640 1 -- 172.21.15.44:0/1990619127 >> 172.21.15.44:0/1990619127 conn(0x7f42e409aa10 msgr2=0x7f42e409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.763+0000 7f42e9200640 1 -- 172.21.15.44:0/1990619127 shutdown_connections 2024-09-13T08:49:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.763+0000 7f42e9200640 1 -- 172.21.15.44:0/1990619127 wait complete. 2024-09-13T08:49:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 Processor -- start 2024-09-13T08:49:22.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 -- start start 2024-09-13T08:49:22.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e40b0d10 0x7f42e4159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:22.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42e40d22c0 0x7f42e41587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:22.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e415bd40 0x7f42e4158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:22.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4168d00 con 0x7f42e40b0d10 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4168b80 con 0x7f42e415bd40 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f42e4168e80 con 0x7f42e40d22c0 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e40b0d10 0x7f42e4159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e40b0d10 0x7f42e4159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59962/0 (socket says 172.21.15.44:59962) 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3400640 1 -- 172.21.15.44:0/3284829870 learned_addr learned my addr 172.21.15.44:0/3284829870 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3e00640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e415bd40 0x7f42e4158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42daa00640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42e40d22c0 0x7f42e41587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3400640 1 -- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42e40d22c0 msgr2=0x7f42e41587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3400640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42e40d22c0 0x7f42e41587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3400640 1 -- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e415bd40 msgr2=0x7f42e4158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3400640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e415bd40 0x7f42e4158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3400640 1 -- 172.21.15.44:0/3284829870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e415c2a0 con 0x7f42e40b0d10 2024-09-13T08:49:22.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3e00640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e415bd40 0x7f42e4158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e3400640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e40b0d10 0x7f42e4159ba0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f42d400e7a0 tx=0x7f42d400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42dbe00640 1 -- 172.21.15.44:0/3284829870 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d400ccc0 con 0x7f42e40b0d10 2024-09-13T08:49:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42dbe00640 1 -- 172.21.15.44:0/3284829870 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42d4010430 con 0x7f42e40b0d10 2024-09-13T08:49:22.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 -- 172.21.15.44:0/3284829870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e415c560 con 0x7f42e40b0d10 2024-09-13T08:49:22.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42dbe00640 1 -- 172.21.15.44:0/3284829870 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4010860 con 0x7f42e40b0d10 2024-09-13T08:49:22.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 -- 172.21.15.44:0/3284829870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e415a0e0 con 0x7f42e40b0d10 2024-09-13T08:49:22.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.767+0000 7f42e9200640 1 -- 172.21.15.44:0/3284829870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e415edf0 con 0x7f42e40b0d10 2024-09-13T08:49:22.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.771+0000 7f42dbe00640 1 -- 172.21.15.44:0/3284829870 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f42d4010a00 con 0x7f42e40b0d10 2024-09-13T08:49:22.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.771+0000 7f42dbe00640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f42c0074540 0x7f42c0076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:22.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.771+0000 7f42dbe00640 1 -- 172.21.15.44:0/3284829870 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5027+0+0 (secure 0 0 0) 0x7f42d4093560 con 0x7f42e40b0d10 2024-09-13T08:49:22.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.771+0000 7f42daa00640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f42c0074540 0x7f42c0076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.771+0000 7f42daa00640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f42c0074540 0x7f42c0076a00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f42e40b7820 tx=0x7f42cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.775+0000 7f42dbe00640 1 -- 172.21.15.44:0/3284829870 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f42d4060210 con 0x7f42e40b0d10 2024-09-13T08:49:22.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.951+0000 7f42e9200640 1 -- 172.21.15.44:0/3284829870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f42e40cc620 con 0x7f42e40b0d10 2024-09-13T08:49:22.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.955+0000 7f42dbe00640 1 -- 172.21.15.44:0/3284829870 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f42d405fb10 con 0x7f42e40b0d10 2024-09-13T08:49:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 -- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f42c0074540 msgr2=0x7f42c0076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f42c0074540 0x7f42c0076a00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f42e40b7820 tx=0x7f42cc009040 comp rx=0 tx=0).stop 2024-09-13T08:49:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 -- 172.21.15.44:0/3284829870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e40b0d10 msgr2=0x7f42e4159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e40b0d10 0x7f42e4159ba0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f42d400e7a0 tx=0x7f42d400ec60 comp rx=0 tx=0).stop 2024-09-13T08:49:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 -- 172.21.15.44:0/3284829870 shutdown_connections 2024-09-13T08:49:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f42c0074540 0x7f42c0076a00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e415bd40 0x7f42e4158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42e40d22c0 0x7f42e41587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 --2- 172.21.15.44:0/3284829870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e40b0d10 0x7f42e4159ba0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:22.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 -- 172.21.15.44:0/3284829870 >> 172.21.15.44:0/3284829870 conn(0x7f42e409aa10 msgr2=0x7f42e406df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:22.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 -- 172.21.15.44:0/3284829870 shutdown_connections 2024-09-13T08:49:22.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:22.959+0000 7f42e9200640 1 -- 172.21.15.44:0/3284829870 wait complete. 2024-09-13T08:49:22.979 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:49:22.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:49:23.070 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:49:23.163 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:49:23.163 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:49:23.163 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:23.172 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:49:23.172 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:49:23.172 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:23.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:49:23.183 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:49:23.183 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:23.223 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:49:23.223 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:49:23.223 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:23.236 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:49:23.237 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T08:49:23.237 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:49:23.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.415+0000 7f66de600640 1 -- 172.21.15.44:0/1845593732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d80cdb30 msgr2=0x7f66d80aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:23.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.415+0000 7f66de600640 1 --2- 172.21.15.44:0/1845593732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d80cdb30 0x7f66d80aeef0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009bc0 tx=0x7f66d001cba0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.415+0000 7f66de600640 1 -- 172.21.15.44:0/1845593732 shutdown_connections 2024-09-13T08:49:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.415+0000 7f66de600640 1 --2- 172.21.15.44:0/1845593732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d80cdb30 0x7f66d80aeef0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.415+0000 7f66de600640 1 --2- 172.21.15.44:0/1845593732 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66d80ced80 0x7f66d80ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.415+0000 7f66de600640 1 -- 172.21.15.44:0/1845593732 >> 172.21.15.44:0/1845593732 conn(0x7f66d80d1dc0 msgr2=0x7f66d80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.415+0000 7f66de600640 1 -- 172.21.15.44:0/1845593732 shutdown_connections 2024-09-13T08:49:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66de600640 1 -- 172.21.15.44:0/1845593732 wait complete. 2024-09-13T08:49:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66de600640 1 Processor -- start 2024-09-13T08:49:23.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66de600640 1 -- start start 2024-09-13T08:49:23.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66de600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66d80cdb30 0x7f66d814e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:23.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66de600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d80ced80 0x7f66d80b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:23.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66de600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d815bea0 0x7f66d80c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:23.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66de600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66d8169340 con 0x7f66d80ced80 2024-09-13T08:49:23.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66de600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81691c0 con 0x7f66d815bea0 2024-09-13T08:49:23.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66de600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f66d81694c0 con 0x7f66d80cdb30 2024-09-13T08:49:23.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d80ced80 0x7f66d80b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:23.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d80ced80 0x7f66d80b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59972/0 (socket says 172.21.15.44:59972) 2024-09-13T08:49:23.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cea00640 1 -- 172.21.15.44:0/3554478522 learned_addr learned my addr 172.21.15.44:0/3554478522 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:23.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66d7e00640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d815bea0 0x7f66d80c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:23.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66d7400640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66d80cdb30 0x7f66d814e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cea00640 1 -- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66d80cdb30 msgr2=0x7f66d814e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cea00640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66d80cdb30 0x7f66d814e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cea00640 1 -- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d815bea0 msgr2=0x7f66d80c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cea00640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d815bea0 0x7f66d80c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cea00640 1 -- 172.21.15.44:0/3554478522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d80c9920 con 0x7f66d80ced80 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cea00640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d80ced80 0x7f66d80b6500 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f66c000ea40 tx=0x7f66c000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cfe00640 1 -- 172.21.15.44:0/3554478522 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c000ce40 con 0x7f66d80ced80 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cfe00640 1 -- 172.21.15.44:0/3554478522 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66c0004020 con 0x7f66d80ced80 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66cfe00640 1 -- 172.21.15.44:0/3554478522 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0004470 con 0x7f66d80ced80 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.419+0000 7f66de600640 1 -- 172.21.15.44:0/3554478522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d80c6a70 con 0x7f66d80ced80 2024-09-13T08:49:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.423+0000 7f66de600640 1 -- 172.21.15.44:0/3554478522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d80c7030 con 0x7f66d80ced80 2024-09-13T08:49:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.423+0000 7f66de600640 1 -- 172.21.15.44:0/3554478522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d80ce670 con 0x7f66d80ced80 2024-09-13T08:49:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.423+0000 7f66cfe00640 1 -- 172.21.15.44:0/3554478522 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f66c0003600 con 0x7f66d80ced80 2024-09-13T08:49:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.423+0000 7f66cfe00640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66b4074540 0x7f66b4076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.423+0000 7f66cfe00640 1 -- 172.21.15.44:0/3554478522 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5027+0+0 (secure 0 0 0) 0x7f66c0093440 con 0x7f66d80ced80 2024-09-13T08:49:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.423+0000 7f66d7400640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66b4074540 0x7f66b4076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.427+0000 7f66d7400640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66b4074540 0x7f66b4076a00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f66d80b7920 tx=0x7f66c80013a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:23.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.427+0000 7f66cfe00640 1 -- 172.21.15.44:0/3554478522 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f66c005faa0 con 0x7f66d80ced80 2024-09-13T08:49:23.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 -- 172.21.15.44:0/3554478522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66d8072780 con 0x7f66d80ced80 2024-09-13T08:49:23.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66cfe00640 1 -- 172.21.15.44:0/3554478522 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+3911 (secure 0 0 0) 0x7f66c00644d0 con 0x7f66d80ced80 2024-09-13T08:49:23.645 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:23.646 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":59,"btime":"2024-09-13T08:49:17:193452+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/833446483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":833446483},{"type":"v1","addr":"172.21.15.129:6836","nonce":833446483}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1844578230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1844578230},{"type":"v1","addr":"172.21.15.129:6837","nonce":1844578230}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2007417535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2007417535},{"type":"v1","addr":"172.21.15.44:6833","nonce":2007417535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7668,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/463557674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":463557674},{"type":"v1","addr":"172.21.15.44:6835","nonce":463557674}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-09-13T08:49:23.646 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 59 2024-09-13T08:49:23.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 -- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66b4074540 msgr2=0x7f66b4076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:23.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66b4074540 0x7f66b4076a00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f66d80b7920 tx=0x7f66c80013a0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 -- 172.21.15.44:0/3554478522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d80ced80 msgr2=0x7f66d80b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:23.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d80ced80 0x7f66d80b6500 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f66c000ea40 tx=0x7f66c000ef00 comp rx=0 tx=0).stop 2024-09-13T08:49:23.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 -- 172.21.15.44:0/3554478522 shutdown_connections 2024-09-13T08:49:23.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f66b4074540 0x7f66b4076a00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d815bea0 0x7f66d80c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d80ced80 0x7f66d80b6500 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 --2- 172.21.15.44:0/3554478522 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f66d80cdb30 0x7f66d814e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 -- 172.21.15.44:0/3554478522 >> 172.21.15.44:0/3554478522 conn(0x7f66d80d1dc0 msgr2=0x7f66d80d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:23.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 -- 172.21.15.44:0/3554478522 shutdown_connections 2024-09-13T08:49:23.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.643+0000 7f66de600640 1 -- 172.21.15.44:0/3554478522 wait complete. 2024-09-13T08:49:23.669 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T08:49:23.669 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T08:49:23.669 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T08:49:23.669 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:49:23.669 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T08:49:23.669 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T08:49:23.670 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T08:49:23.670 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:49:23.670 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T08:49:23.670 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T08:49:23.670 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T08:49:23.670 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:49:23.670 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T08:49:23.670 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T08:49:23.671 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T08:49:23.671 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:49:23.682 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T08:49:23.682 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T08:49:23.684 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T08:49:23.684 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T08:49:23.685 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T08:49:23.685 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T08:49:23.686 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T08:49:23.686 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T08:49:23.729 INFO:tasks.ceph.mds.a:Started 2024-09-13T08:49:23.730 DEBUG:teuthology.parallel:result is None 2024-09-13T08:49:23.732 INFO:tasks.ceph.mds.c:Started 2024-09-13T08:49:23.732 DEBUG:teuthology.parallel:result is None 2024-09-13T08:49:23.734 INFO:tasks.ceph.mds.d:Started 2024-09-13T08:49:23.734 DEBUG:teuthology.parallel:result is None 2024-09-13T08:49:23.735 INFO:tasks.ceph.mds.b:Started 2024-09-13T08:49:23.736 DEBUG:teuthology.parallel:result is None 2024-09-13T08:49:23.736 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T08:49:23.817 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:49:23.818 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:49:23.825 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:49:23.826 INFO:tasks.ceph.mds.b.smithi129.stdout:starting mds.b at 2024-09-13T08:49:23.829 INFO:tasks.ceph.mds.b.smithi129.stderr:2024-09-13T08:49:23.829+0000 7fdd2ca00640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2024-09-13T08:49:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.887+0000 7f0a6c200640 1 -- 172.21.15.44:0/856147765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a6415bea0 msgr2=0x7f0a640ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.887+0000 7f0a6c200640 1 --2- 172.21.15.44:0/856147765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a6415bea0 0x7f0a640ce670 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009900 tx=0x7f0a5801c850 comp rx=0 tx=0).stop 2024-09-13T08:49:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.887+0000 7f0a6c200640 1 -- 172.21.15.44:0/856147765 shutdown_connections 2024-09-13T08:49:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.887+0000 7f0a6c200640 1 --2- 172.21.15.44:0/856147765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a6415bea0 0x7f0a640ce670 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.887+0000 7f0a6c200640 1 -- 172.21.15.44:0/856147765 >> 172.21.15.44:0/856147765 conn(0x7f0a640d1dc0 msgr2=0x7f0a640d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.887+0000 7f0a6c200640 1 -- 172.21.15.44:0/856147765 shutdown_connections 2024-09-13T08:49:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.887+0000 7f0a6c200640 1 -- 172.21.15.44:0/856147765 wait complete. 2024-09-13T08:49:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.887+0000 7f0a6c200640 1 Processor -- start 2024-09-13T08:49:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.887+0000 7f0a6c200640 1 -- start start 2024-09-13T08:49:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a6c200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a640cdb30 0x7f0a64167230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a6c200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a640d28e0 0x7f0a641530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a6c200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a6415bea0 0x7f0a64151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a6c200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6416a930 con 0x7f0a640d28e0 2024-09-13T08:49:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a6c200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6416a7b0 con 0x7f0a6415bea0 2024-09-13T08:49:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a6c200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6416aab0 con 0x7f0a640cdb30 2024-09-13T08:49:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a68e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a640d28e0 0x7f0a641530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a68e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a640d28e0 0x7f0a641530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60002/0 (socket says 172.21.15.44:60002) 2024-09-13T08:49:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a68e00640 1 -- 172.21.15.44:0/613644869 learned_addr learned my addr 172.21.15.44:0/613644869 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a6a200640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a6415bea0 0x7f0a64151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a69800640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a640cdb30 0x7f0a64167230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a68e00640 1 -- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a640cdb30 msgr2=0x7f0a64167230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a68e00640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a640cdb30 0x7f0a64167230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a68e00640 1 -- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a6415bea0 msgr2=0x7f0a64151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a68e00640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a6415bea0 0x7f0a64151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a68e00640 1 -- 172.21.15.44:0/613644869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a64158930 con 0x7f0a640d28e0 2024-09-13T08:49:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a68e00640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a640d28e0 0x7f0a641530e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000c500 tx=0x7f0a5000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:23.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a61600640 1 -- 172.21.15.44:0/613644869 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50017070 con 0x7f0a640d28e0 2024-09-13T08:49:23.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a61600640 1 -- 172.21.15.44:0/613644869 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a500053c0 con 0x7f0a640d28e0 2024-09-13T08:49:23.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a61600640 1 -- 172.21.15.44:0/613644869 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50012890 con 0x7f0a640d28e0 2024-09-13T08:49:23.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a6c200640 1 -- 172.21.15.44:0/613644869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a64158bf0 con 0x7f0a640d28e0 2024-09-13T08:49:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a6c200640 1 -- 172.21.15.44:0/613644869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a641591b0 con 0x7f0a640d28e0 2024-09-13T08:49:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.891+0000 7f0a6c200640 1 -- 172.21.15.44:0/613644869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a640ce670 con 0x7f0a640d28e0 2024-09-13T08:49:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.895+0000 7f0a61600640 1 -- 172.21.15.44:0/613644869 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f0a5001a070 con 0x7f0a640d28e0 2024-09-13T08:49:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.895+0000 7f0a61600640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a44074540 0x7f0a44076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.895+0000 7f0a61600640 1 -- 172.21.15.44:0/613644869 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5027+0+0 (secure 0 0 0) 0x7f0a50091da0 con 0x7f0a640d28e0 2024-09-13T08:49:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.895+0000 7f0a69800640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a44074540 0x7f0a44076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.895+0000 7f0a69800640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a44074540 0x7f0a44076a00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0a580af000 tx=0x7f0a5801b090 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:23.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:23.899+0000 7f0a61600640 1 -- 172.21.15.44:0/613644869 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0a5005e400 con 0x7f0a640d28e0 2024-09-13T08:49:24.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.087+0000 7f0a6c200640 1 -- 172.21.15.44:0/613644869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0a640d2480 con 0x7f0a640d28e0 2024-09-13T08:49:24.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.247+0000 7f0a61600640 1 -- 172.21.15.44:0/613644869 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v96) ==== 97+0+0 (secure 0 0 0) 0x7f0a50062e30 con 0x7f0a640d28e0 2024-09-13T08:49:24.250 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-09-13T08:49:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 -- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a44074540 msgr2=0x7f0a44076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a44074540 0x7f0a44076a00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0a580af000 tx=0x7f0a5801b090 comp rx=0 tx=0).stop 2024-09-13T08:49:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 -- 172.21.15.44:0/613644869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a640d28e0 msgr2=0x7f0a641530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a640d28e0 0x7f0a641530e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000c500 tx=0x7f0a5000c9c0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 -- 172.21.15.44:0/613644869 shutdown_connections 2024-09-13T08:49:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a44074540 0x7f0a44076a00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a6415bea0 0x7f0a64151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a640d28e0 0x7f0a641530e0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 --2- 172.21.15.44:0/613644869 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a640cdb30 0x7f0a64167230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 -- 172.21.15.44:0/613644869 >> 172.21.15.44:0/613644869 conn(0x7f0a640d1dc0 msgr2=0x7f0a640b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 -- 172.21.15.44:0/613644869 shutdown_connections 2024-09-13T08:49:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.251+0000 7f0a6c200640 1 -- 172.21.15.44:0/613644869 wait complete. 2024-09-13T08:49:24.275 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T08:49:24.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 -- 172.21.15.44:0/2621020473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec015bd50 msgr2=0x7f9ec00ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:24.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 --2- 172.21.15.44:0/2621020473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec015bd50 0x7f9ec00ce5e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009900 tx=0x7f9eb401c7c0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 -- 172.21.15.44:0/2621020473 shutdown_connections 2024-09-13T08:49:24.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 --2- 172.21.15.44:0/2621020473 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ec015c130 0x7f9ec00ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 --2- 172.21.15.44:0/2621020473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec015bd50 0x7f9ec00ce5e0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 -- 172.21.15.44:0/2621020473 >> 172.21.15.44:0/2621020473 conn(0x7f9ec00d1ea0 msgr2=0x7f9ec00d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:24.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 -- 172.21.15.44:0/2621020473 shutdown_connections 2024-09-13T08:49:24.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 -- 172.21.15.44:0/2621020473 wait complete. 2024-09-13T08:49:24.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 Processor -- start 2024-09-13T08:49:24.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 -- start start 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec00cf4e0 0x7f9ec00c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ec015bd50 0x7f9ec006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ec015c130 0x7f9ec0154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec0168e50 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec0168cd0 con 0x7f9ec015bd50 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec8200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec0168fd0 con 0x7f9ec015c130 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec00cf4e0 0x7f9ec00c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec00cf4e0 0x7f9ec00c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60020/0 (socket says 172.21.15.44:60020) 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec5800640 1 -- 172.21.15.44:0/3681060621 learned_addr learned my addr 172.21.15.44:0/3681060621 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec6200640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ec015c130 0x7f9ec0154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:24.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec4e00640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ec015bd50 0x7f9ec006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec5800640 1 -- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ec015c130 msgr2=0x7f9ec0154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec5800640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ec015c130 0x7f9ec0154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec5800640 1 -- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ec015bd50 msgr2=0x7f9ec006f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec5800640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ec015bd50 0x7f9ec006f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec5800640 1 -- 172.21.15.44:0/3681060621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec006fc60 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.423+0000 7f9ec4e00640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ec015bd50 0x7f9ec006f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ec5800640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec00cf4e0 0x7f9ec00c8f70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009850 tx=0x7f9eb40a5590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ebd600640 1 -- 172.21.15.44:0/3681060621 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb40b2040 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ebd600640 1 -- 172.21.15.44:0/3681060621 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9eb4004470 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ec8200640 1 -- 172.21.15.44:0/3681060621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec00b5e00 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ebd600640 1 -- 172.21.15.44:0/3681060621 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4005290 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ec8200640 1 -- 172.21.15.44:0/3681060621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec00b63c0 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ec8200640 1 -- 172.21.15.44:0/3681060621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec00c5dd0 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ebd600640 1 -- 172.21.15.44:0/3681060621 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f9eb40bd060 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ebd600640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ea0074540 0x7f9ea0076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:24.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ebd600640 1 -- 172.21.15.44:0/3681060621 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4941+0+0 (secure 0 0 0) 0x7f9eb412c310 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.427+0000 7f9ec4e00640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ea0074540 0x7f9ea0076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:24.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.431+0000 7f9ec4e00640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ea0074540 0x7f9ea0076a00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9ec006f3b0 tx=0x7f9eac0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:24.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.431+0000 7f9ebd600640 1 -- 172.21.15.44:0/3681060621 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9eb40164d0 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.671+0000 7f9ec8200640 1 -- 172.21.15.44:0/3681060621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9ec00c8800 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.671+0000 7f9ebd600640 1 -- 172.21.15.44:0/3681060621 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f9eb40f8fc0 con 0x7f9ec00cf4e0 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw==", 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.675 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ==", 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-09-13T08:49:24.676 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA==", 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw==", 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:49:24.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg==", 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg==", 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw==", 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig==", 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ==", 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g==", 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:49:24.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg==", 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.682 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg==", 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA==", 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T08:49:24.683 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w==", 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-09-13T08:49:24.684 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA==", 2024-09-13T08:49:24.685 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.685 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:49:24.685 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:49:24.685 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:49:24.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:49:24.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.685 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-09-13T08:49:24.685 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w==", 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmDD2LJxAA3s0jEb2EfPsLcu0w145G3Q==", 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:49:24.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ==", 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:49:24.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:24.688 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-09-13T08:49:24.688 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg==", 2024-09-13T08:49:24.688 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:49:24.688 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:49:24.688 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:49:24.688 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:49:24.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:24.688 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T08:49:24.688 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:49:24.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 -- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ea0074540 msgr2=0x7f9ea0076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ea0074540 0x7f9ea0076a00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9ec006f3b0 tx=0x7f9eac0091e0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 -- 172.21.15.44:0/3681060621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec00cf4e0 msgr2=0x7f9ec00c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec00cf4e0 0x7f9ec00c8f70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009850 tx=0x7f9eb40a5590 comp rx=0 tx=0).stop 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 -- 172.21.15.44:0/3681060621 shutdown_connections 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ea0074540 0x7f9ea0076a00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ec015c130 0x7f9ec0154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ec015bd50 0x7f9ec006f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 --2- 172.21.15.44:0/3681060621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec00cf4e0 0x7f9ec00c8f70 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 -- 172.21.15.44:0/3681060621 >> 172.21.15.44:0/3681060621 conn(0x7f9ec00d1ea0 msgr2=0x7f9ec00d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 -- 172.21.15.44:0/3681060621 shutdown_connections 2024-09-13T08:49:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.675+0000 7f9ec8200640 1 -- 172.21.15.44:0/3681060621 wait complete. 2024-09-13T08:49:24.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T08:49:24.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.843+0000 7fc510e00640 1 -- 172.21.15.44:0/3299466303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f0004810 msgr2=0x7fc4f0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:24.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.843+0000 7fc510e00640 1 --2- 172.21.15.44:0/3299466303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f0004810 0x7fc4f0004c90 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009b60 tx=0x7fc4f401cf90 comp rx=0 tx=0).stop 2024-09-13T08:49:24.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.843+0000 7fc50a800640 1 -- 172.21.15.44:0/3299466303 reap_dead start 2024-09-13T08:49:24.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.843+0000 7fc510e00640 1 -- 172.21.15.44:0/3299466303 shutdown_connections 2024-09-13T08:49:24.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.843+0000 7fc510e00640 1 -- 172.21.15.44:0/3299466303 >> 172.21.15.44:0/3299466303 conn(0x7fc50c0d1ea0 msgr2=0x7fc50c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:24.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.843+0000 7fc510e00640 1 -- 172.21.15.44:0/3299466303 shutdown_connections 2024-09-13T08:49:24.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.843+0000 7fc510e00640 1 -- 172.21.15.44:0/3299466303 wait complete. 2024-09-13T08:49:24.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 Processor -- start 2024-09-13T08:49:24.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 -- start start 2024-09-13T08:49:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f0004810 0x7fc50c151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc50c165490 0x7fc50c0b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc50c0b56e0 0x7fc50c1f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c166e40 con 0x7fc50c0b56e0 2024-09-13T08:49:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c166cc0 con 0x7fc50c165490 2024-09-13T08:49:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc50c166fc0 con 0x7fc4f0004810 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc50a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc50c0b56e0 0x7fc50c1f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc50a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc50c0b56e0 0x7fc50c1f7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60042/0 (socket says 172.21.15.44:60042) 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc50a800640 1 -- 172.21.15.44:0/3566396691 learned_addr learned my addr 172.21.15.44:0/3566396691 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc509e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f0004810 0x7fc50c151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc503e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc50c165490 0x7fc50c0b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc50a800640 1 -- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f0004810 msgr2=0x7fc50c151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc50a800640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f0004810 0x7fc50c151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc50a800640 1 -- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc50c165490 msgr2=0x7fc50c0b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc50a800640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc50c165490 0x7fc50c0b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc50a800640 1 -- 172.21.15.44:0/3566396691 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c0b7570 con 0x7fc50c0b56e0 2024-09-13T08:49:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc509e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f0004810 0x7fc50c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc50a800640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc50c0b56e0 0x7fc50c1f7d20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc504001340 tx=0x7fc504006670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc502000640 1 -- 172.21.15.44:0/3566396691 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504012470 con 0x7fc50c0b56e0 2024-09-13T08:49:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc502000640 1 -- 172.21.15.44:0/3566396691 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc504012700 con 0x7fc50c0b56e0 2024-09-13T08:49:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc502000640 1 -- 172.21.15.44:0/3566396691 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504012b50 con 0x7fc50c0b56e0 2024-09-13T08:49:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 -- 172.21.15.44:0/3566396691 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50c0b7850 con 0x7fc50c0b56e0 2024-09-13T08:49:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 -- 172.21.15.44:0/3566396691 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc50c162e80 con 0x7fc50c0b56e0 2024-09-13T08:49:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.847+0000 7fc510e00640 1 -- 172.21.15.44:0/3566396691 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc50c0ceb00 con 0x7fc50c0b56e0 2024-09-13T08:49:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.851+0000 7fc502000640 1 -- 172.21.15.44:0/3566396691 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fc504012cf0 con 0x7fc50c0b56e0 2024-09-13T08:49:24.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.851+0000 7fc502000640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4e8074540 0x7fc4e8076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:24.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.851+0000 7fc502000640 1 -- 172.21.15.44:0/3566396691 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4941+0+0 (secure 0 0 0) 0x7fc504093ef0 con 0x7fc50c0b56e0 2024-09-13T08:49:24.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.851+0000 7fc509e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4e8074540 0x7fc4e8076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:24.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.851+0000 7fc509e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4e8074540 0x7fc4e8076a00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009b80 tx=0x7fc4fc0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:24.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:24.855+0000 7fc502000640 1 -- 172.21.15.44:0/3566396691 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc504060ba0 con 0x7fc50c0b56e0 2024-09-13T08:49:25.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 -- 172.21.15.44:0/3566396691 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc50c071340 con 0x7fc50c0b56e0 2024-09-13T08:49:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc502000640 1 -- 172.21.15.44:0/3566396691 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) ==== 84+0+65 (secure 0 0 0) 0x7fc5040604a0 con 0x7fc50c0b56e0 2024-09-13T08:49:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:25.033 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:49:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T08:49:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T08:49:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:25.033 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:49:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 -- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4e8074540 msgr2=0x7fc4e8076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4e8074540 0x7fc4e8076a00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009b80 tx=0x7fc4fc0091e0 comp rx=0 tx=0).stop 2024-09-13T08:49:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 -- 172.21.15.44:0/3566396691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc50c0b56e0 msgr2=0x7fc50c1f7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc50c0b56e0 0x7fc50c1f7d20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc504001340 tx=0x7fc504006670 comp rx=0 tx=0).stop 2024-09-13T08:49:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 -- 172.21.15.44:0/3566396691 shutdown_connections 2024-09-13T08:49:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4e8074540 0x7fc4e8076a00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc50c0b56e0 0x7fc50c1f7d20 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc50c165490 0x7fc50c0b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:25.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 --2- 172.21.15.44:0/3566396691 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f0004810 0x7fc50c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:25.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 -- 172.21.15.44:0/3566396691 >> 172.21.15.44:0/3566396691 conn(0x7fc50c0d1ea0 msgr2=0x7fc50c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:25.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 -- 172.21.15.44:0/3566396691 shutdown_connections 2024-09-13T08:49:25.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.031+0000 7fc510e00640 1 -- 172.21.15.44:0/3566396691 wait complete. 2024-09-13T08:49:25.054 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T08:49:25.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T08:49:25.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.199+0000 7fb291a00640 1 -- 172.21.15.44:0/775937210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb28c15be10 msgr2=0x7fb28c0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:25.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.199+0000 7fb291a00640 1 --2- 172.21.15.44:0/775937210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb28c15be10 0x7fb28c0ce700 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0746b0 tx=0x7fb27c01eff0 comp rx=0 tx=0).stop 2024-09-13T08:49:25.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.199+0000 7fb291a00640 1 -- 172.21.15.44:0/775937210 shutdown_connections 2024-09-13T08:49:25.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 --2- 172.21.15.44:0/775937210 >> v2:172.21.15.129:3300/0 conn(0x7fb28c15c1f0 0x7fb28c0cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:25.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 --2- 172.21.15.44:0/775937210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb28c15be10 0x7fb28c0ce700 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:25.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 -- 172.21.15.44:0/775937210 >> 172.21.15.44:0/775937210 conn(0x7fb28c0d1da0 msgr2=0x7fb28c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:25.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 -- 172.21.15.44:0/775937210 shutdown_connections 2024-09-13T08:49:25.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 -- 172.21.15.44:0/775937210 wait complete. 2024-09-13T08:49:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 Processor -- start 2024-09-13T08:49:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 -- start start 2024-09-13T08:49:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb28c0b0b60 0x7fb28c1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb28c15be10 0x7fb28c0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb28c15c1f0 0x7fb28c06e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c0cf510 con 0x7fb28c15c1f0 2024-09-13T08:49:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c0cf390 con 0x7fb28c0b0b60 2024-09-13T08:49:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb291a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb28c160290 con 0x7fb28c15be10 2024-09-13T08:49:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb28b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb28c15c1f0 0x7fb28c06e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb28b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb28c15c1f0 0x7fb28c06e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60060/0 (socket says 172.21.15.44:60060) 2024-09-13T08:49:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb28b400640 1 -- 172.21.15.44:0/4256529647 learned_addr learned my addr 172.21.15.44:0/4256529647 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb28aa00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb28c0b0b60 0x7fb28c1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb283e00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb28c15be10 0x7fb28c0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb28b400640 1 -- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb28c15be10 msgr2=0x7fb28c0c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb28b400640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb28c15be10 0x7fb28c0c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb28b400640 1 -- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb28c0b0b60 msgr2=0x7fb28c1522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb28b400640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb28c0b0b60 0x7fb28c1522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb28b400640 1 -- 172.21.15.44:0/4256529647 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c1527e0 con 0x7fb28c15c1f0 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.203+0000 7fb28b400640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb28c15c1f0 0x7fb28c06e3e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c730 tx=0x7fb28400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.207+0000 7fb282a00640 1 -- 172.21.15.44:0/4256529647 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284003db0 con 0x7fb28c15c1f0 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.207+0000 7fb282a00640 1 -- 172.21.15.44:0/4256529647 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb28400ce40 con 0x7fb28c15c1f0 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.207+0000 7fb291a00640 1 -- 172.21.15.44:0/4256529647 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28c0c6100 con 0x7fb28c15c1f0 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.207+0000 7fb282a00640 1 -- 172.21.15.44:0/4256529647 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2840052d0 con 0x7fb28c15c1f0 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.207+0000 7fb291a00640 1 -- 172.21.15.44:0/4256529647 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28c0c66c0 con 0x7fb28c15c1f0 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.207+0000 7fb291a00640 1 -- 172.21.15.44:0/4256529647 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28c1607b0 con 0x7fb28c15c1f0 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.207+0000 7fb282a00640 1 -- 172.21.15.44:0/4256529647 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fb284005470 con 0x7fb28c15c1f0 2024-09-13T08:49:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.207+0000 7fb282a00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb268074540 0x7fb268076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.207+0000 7fb282a00640 1 -- 172.21.15.44:0/4256529647 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4941+0+0 (secure 0 0 0) 0x7fb284092990 con 0x7fb28c15c1f0 2024-09-13T08:49:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.207+0000 7fb28aa00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb268074540 0x7fb268076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.211+0000 7fb28aa00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb268074540 0x7fb268076a00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0b2000 tx=0x7fb27c01d830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.211+0000 7fb282a00640 1 -- 172.21.15.44:0/4256529647 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb28405f640 con 0x7fb28c15c1f0 2024-09-13T08:49:25.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:25.391+0000 7fb291a00640 1 -- 172.21.15.44:0/4256529647 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb28c0b1960 con 0x7fb28c15c1f0 2024-09-13T08:49:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.259+0000 7fb282a00640 1 -- 172.21.15.44:0/4256529647 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v99) ==== 138+0+0 (secure 0 0 0) 0x7fb28405ef40 con 0x7fb28c15c1f0 2024-09-13T08:49:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T08:49:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 -- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb268074540 msgr2=0x7fb268076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb268074540 0x7fb268076a00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0b2000 tx=0x7fb27c01d830 comp rx=0 tx=0).stop 2024-09-13T08:49:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 -- 172.21.15.44:0/4256529647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb28c15c1f0 msgr2=0x7fb28c06e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb28c15c1f0 0x7fb28c06e3e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c730 tx=0x7fb28400cbf0 comp rx=0 tx=0).stop 2024-09-13T08:49:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 -- 172.21.15.44:0/4256529647 shutdown_connections 2024-09-13T08:49:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb268074540 0x7fb268076a00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb28c15c1f0 0x7fb28c06e3e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb28c15be10 0x7fb28c0c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 --2- 172.21.15.44:0/4256529647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb28c0b0b60 0x7fb28c1522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 -- 172.21.15.44:0/4256529647 >> 172.21.15.44:0/4256529647 conn(0x7fb28c0d1da0 msgr2=0x7fb28c0712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 -- 172.21.15.44:0/4256529647 shutdown_connections 2024-09-13T08:49:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.263+0000 7fb291a00640 1 -- 172.21.15.44:0/4256529647 wait complete. 2024-09-13T08:49:26.296 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T08:49:26.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.451+0000 7f0114600640 1 -- 172.21.15.44:0/3007909879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c073e40 msgr2=0x7f010c0cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:26.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.451+0000 7f0114600640 1 --2- 172.21.15.44:0/3007909879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c073e40 0x7f010c0cb8c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f0104009bc0 tx=0x7f010401cba0 comp rx=0 tx=0).stop 2024-09-13T08:49:26.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.451+0000 7f0114600640 1 -- 172.21.15.44:0/3007909879 shutdown_connections 2024-09-13T08:49:26.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.451+0000 7f0114600640 1 --2- 172.21.15.44:0/3007909879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c073e40 0x7f010c0cb8c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:26.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.451+0000 7f0114600640 1 --2- 172.21.15.44:0/3007909879 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f010c151f40 0x7f010c15bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:26.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.451+0000 7f0114600640 1 -- 172.21.15.44:0/3007909879 >> 172.21.15.44:0/3007909879 conn(0x7f010c0d0c70 msgr2=0x7f010c0d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:26.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.451+0000 7f0114600640 1 -- 172.21.15.44:0/3007909879 shutdown_connections 2024-09-13T08:49:26.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.451+0000 7f0114600640 1 -- 172.21.15.44:0/3007909879 wait complete. 2024-09-13T08:49:26.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0114600640 1 Processor -- start 2024-09-13T08:49:26.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0114600640 1 -- start start 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0114600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c073a60 0x7f010c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0114600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f010c073e40 0x7f010c148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0114600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f010c151f40 0x7f010c1f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0114600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f010c0cf070 con 0x7f010c073a60 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0114600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f010c0ceef0 con 0x7f010c073e40 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0114600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f010c0cf1f0 con 0x7f010c151f40 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0111c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c073a60 0x7f010c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0111c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c073a60 0x7f010c148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60074/0 (socket says 172.21.15.44:60074) 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0111c00640 1 -- 172.21.15.44:0/355909453 learned_addr learned my addr 172.21.15.44:0/355909453 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0112600640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f010c151f40 0x7f010c1f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0111200640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f010c073e40 0x7f010c148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0111c00640 1 -- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f010c151f40 msgr2=0x7f010c1f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:26.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0111c00640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f010c151f40 0x7f010c1f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:26.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0111c00640 1 -- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f010c073e40 msgr2=0x7f010c148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:26.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0111c00640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f010c073e40 0x7f010c148400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:26.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0111c00640 1 -- 172.21.15.44:0/355909453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c1f9a30 con 0x7f010c073a60 2024-09-13T08:49:26.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0111c00640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c073a60 0x7f010c148ff0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f010000c500 tx=0x7f010000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:26.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0109600640 1 -- 172.21.15.44:0/355909453 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100017070 con 0x7f010c073a60 2024-09-13T08:49:26.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0109600640 1 -- 172.21.15.44:0/355909453 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0100003600 con 0x7f010c073a60 2024-09-13T08:49:26.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0109600640 1 -- 172.21.15.44:0/355909453 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100003a50 con 0x7f010c073a60 2024-09-13T08:49:26.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0114600640 1 -- 172.21.15.44:0/355909453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c155ae0 con 0x7f010c073a60 2024-09-13T08:49:26.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.455+0000 7f0114600640 1 -- 172.21.15.44:0/355909453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c148c80 con 0x7f010c073a60 2024-09-13T08:49:26.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.459+0000 7f0114600640 1 -- 172.21.15.44:0/355909453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c0cb8c0 con 0x7f010c073a60 2024-09-13T08:49:26.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.459+0000 7f0109600640 1 -- 172.21.15.44:0/355909453 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f010001a070 con 0x7f010c073a60 2024-09-13T08:49:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.459+0000 7f0109600640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00ec074540 0x7f00ec076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.459+0000 7f0109600640 1 -- 172.21.15.44:0/355909453 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5311+0+0 (secure 0 0 0) 0x7f01000922e0 con 0x7f010c073a60 2024-09-13T08:49:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.463+0000 7f0111200640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00ec074540 0x7f00ec076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:26.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.463+0000 7f0111200640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00ec074540 0x7f00ec076a00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f010c14ed00 tx=0x7f00f800b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:26.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.463+0000 7f0109600640 1 -- 172.21.15.44:0/355909453 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0100021020 con 0x7f010c073a60 2024-09-13T08:49:26.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:26.643+0000 7f0114600640 1 -- 172.21.15.44:0/355909453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f010c0d1890 con 0x7f010c073a60 2024-09-13T08:49:27.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.271+0000 7f0109600640 1 -- 172.21.15.44:0/355909453 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v100) ==== 170+0+0 (secure 0 0 0) 0x7f010005eb00 con 0x7f010c073a60 2024-09-13T08:49:27.274 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T08:49:27.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 -- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00ec074540 msgr2=0x7f00ec076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:27.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00ec074540 0x7f00ec076a00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f010c14ed00 tx=0x7f00f800b2e0 comp rx=0 tx=0).stop 2024-09-13T08:49:27.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 -- 172.21.15.44:0/355909453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c073a60 msgr2=0x7f010c148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:27.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c073a60 0x7f010c148ff0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f010000c500 tx=0x7f010000c9c0 comp rx=0 tx=0).stop 2024-09-13T08:49:27.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 -- 172.21.15.44:0/355909453 shutdown_connections 2024-09-13T08:49:27.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00ec074540 0x7f00ec076a00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:27.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f010c151f40 0x7f010c1f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:27.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f010c073e40 0x7f010c148400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:27.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 --2- 172.21.15.44:0/355909453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c073a60 0x7f010c148ff0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:27.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 -- 172.21.15.44:0/355909453 >> 172.21.15.44:0/355909453 conn(0x7f010c0d0c70 msgr2=0x7f010c14feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:27.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 -- 172.21.15.44:0/355909453 shutdown_connections 2024-09-13T08:49:27.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.275+0000 7f0114600640 1 -- 172.21.15.44:0/355909453 wait complete. 2024-09-13T08:49:27.315 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T08:49:27.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.467+0000 7fb83fc00640 1 -- 172.21.15.44:0/3495820091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8381605d0 msgr2=0x7fb8380d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:27.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.467+0000 7fb83fc00640 1 --2- 172.21.15.44:0/3495820091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8381605d0 0x7fb8380d0690 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009840 tx=0x7fb82c01c820 comp rx=0 tx=0).stop 2024-09-13T08:49:27.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.467+0000 7fb83fc00640 1 -- 172.21.15.44:0/3495820091 shutdown_connections 2024-09-13T08:49:27.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.467+0000 7fb83fc00640 1 --2- 172.21.15.44:0/3495820091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8381605d0 0x7fb8380d0690 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:27.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.467+0000 7fb83fc00640 1 -- 172.21.15.44:0/3495820091 >> 172.21.15.44:0/3495820091 conn(0x7fb8380d1dd0 msgr2=0x7fb8380d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:27.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.467+0000 7fb83fc00640 1 -- 172.21.15.44:0/3495820091 shutdown_connections 2024-09-13T08:49:27.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.467+0000 7fb83fc00640 1 -- 172.21.15.44:0/3495820091 wait complete. 2024-09-13T08:49:27.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.467+0000 7fb83fc00640 1 Processor -- start 2024-09-13T08:49:27.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.467+0000 7fb83fc00640 1 -- start start 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.467+0000 7fb83fc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8380afb10 0x7fb838150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83fc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8380b1650 0x7fb8381f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83fc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8381605d0 0x7fb838165850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83fc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8380b1220 con 0x7fb8381605d0 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83fc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb8380b0ee0 con 0x7fb8380afb10 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83fc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb8380b13a0 con 0x7fb8380b1650 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8381605d0 0x7fb838165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8381605d0 0x7fb838165850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60100/0 (socket says 172.21.15.44:60100) 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83dc00640 1 -- 172.21.15.44:0/4142730921 learned_addr learned my addr 172.21.15.44:0/4142730921 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83d200640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8380afb10 0x7fb838150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb837e00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8380b1650 0x7fb8381f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83dc00640 1 -- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8380b1650 msgr2=0x7fb8381f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83dc00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8380b1650 0x7fb8381f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83dc00640 1 -- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8380afb10 msgr2=0x7fb838150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83dc00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8380afb10 0x7fb838150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83dc00640 1 -- 172.21.15.44:0/4142730921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8381f85a0 con 0x7fb8381605d0 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83dc00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8381605d0 0x7fb838165850 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb83000cbf0 tx=0x7fb83000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb834c00640 1 -- 172.21.15.44:0/4142730921 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830016020 con 0x7fb8381605d0 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb834c00640 1 -- 172.21.15.44:0/4142730921 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb83000ab00 con 0x7fb8381605d0 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83fc00640 1 -- 172.21.15.44:0/4142730921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8381f8880 con 0x7fb8381605d0 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb834c00640 1 -- 172.21.15.44:0/4142730921 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830012420 con 0x7fb8381605d0 2024-09-13T08:49:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83fc00640 1 -- 172.21.15.44:0/4142730921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb838151680 con 0x7fb8381605d0 2024-09-13T08:49:27.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.471+0000 7fb83fc00640 1 -- 172.21.15.44:0/4142730921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8380affc0 con 0x7fb8381605d0 2024-09-13T08:49:27.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.475+0000 7fb834c00640 1 -- 172.21.15.44:0/4142730921 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fb830019020 con 0x7fb8381605d0 2024-09-13T08:49:27.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.475+0000 7fb834c00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb818074540 0x7fb818076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:27.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.475+0000 7fb834c00640 1 -- 172.21.15.44:0/4142730921 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5693+0+0 (secure 0 0 0) 0x7fb830093170 con 0x7fb8381605d0 2024-09-13T08:49:27.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.475+0000 7fb83d200640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb818074540 0x7fb818076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:27.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.475+0000 7fb83d200640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb818074540 0x7fb818076a00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0af000 tx=0x7fb82c01b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:27.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.479+0000 7fb834c00640 1 -- 172.21.15.44:0/4142730921 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb830020020 con 0x7fb8381605d0 2024-09-13T08:49:27.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:27.691+0000 7fb83fc00640 1 -- 172.21.15.44:0/4142730921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb8381f8a50 con 0x7fb8381605d0 2024-09-13T08:49:28.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.291+0000 7fb834c00640 1 -- 172.21.15.44:0/4142730921 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v62) ==== 174+0+0 (secure 0 0 0) 0x7fb83005f850 con 0x7fb8381605d0 2024-09-13T08:49:28.299 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 14 and data pool 15 2024-09-13T08:49:28.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 -- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb818074540 msgr2=0x7fb818076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:28.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb818074540 0x7fb818076a00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0af000 tx=0x7fb82c01b060 comp rx=0 tx=0).stop 2024-09-13T08:49:28.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 -- 172.21.15.44:0/4142730921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8381605d0 msgr2=0x7fb838165850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:28.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8381605d0 0x7fb838165850 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb83000cbf0 tx=0x7fb83000cce0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 -- 172.21.15.44:0/4142730921 shutdown_connections 2024-09-13T08:49:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb818074540 0x7fb818076a00 secure :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0af000 tx=0x7fb82c01b060 comp rx=0 tx=0).stop 2024-09-13T08:49:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8381605d0 0x7fb838165850 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8380b1650 0x7fb8381f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 --2- 172.21.15.44:0/4142730921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8380afb10 0x7fb838150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 -- 172.21.15.44:0/4142730921 >> 172.21.15.44:0/4142730921 conn(0x7fb8380d1dd0 msgr2=0x7fb8380721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 -- 172.21.15.44:0/4142730921 shutdown_connections 2024-09-13T08:49:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.307+0000 7fb83fc00640 1 -- 172.21.15.44:0/4142730921 wait complete. 2024-09-13T08:49:28.337 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:49:28.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 -- 172.21.15.44:0/1226831568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2540cecf0 msgr2=0x7ff2540cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:28.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 --2- 172.21.15.44:0/1226831568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2540cecf0 0x7ff2540cc940 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff2540d0780 tx=0x7ff24001c700 comp rx=0 tx=0).stop 2024-09-13T08:49:28.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 -- 172.21.15.44:0/1226831568 shutdown_connections 2024-09-13T08:49:28.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 --2- 172.21.15.44:0/1226831568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2540cecf0 0x7ff2540cc940 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 -- 172.21.15.44:0/1226831568 >> 172.21.15.44:0/1226831568 conn(0x7ff2540d1ea0 msgr2=0x7ff2540d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:28.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 -- 172.21.15.44:0/1226831568 shutdown_connections 2024-09-13T08:49:28.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 -- 172.21.15.44:0/1226831568 wait complete. 2024-09-13T08:49:28.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 Processor -- start 2024-09-13T08:49:28.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 -- start start 2024-09-13T08:49:28.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2540cecf0 0x7ff25414ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:28.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff25415bd50 0x7ff2540b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:28.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff25415c130 0x7ff2540b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:28.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff25416a5b0 con 0x7ff2540cecf0 2024-09-13T08:49:28.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff25416a430 con 0x7ff25415c130 2024-09-13T08:49:28.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25c400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff25416a730 con 0x7ff25415bd50 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff259a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2540cecf0 0x7ff25414ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff259a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2540cecf0 0x7ff25414ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60124/0 (socket says 172.21.15.44:60124) 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff259a00640 1 -- 172.21.15.44:0/1897512221 learned_addr learned my addr 172.21.15.44:0/1897512221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff25a400640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff25415c130 0x7ff2540b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff259000640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff25415bd50 0x7ff2540b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff259a00640 1 -- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff25415bd50 msgr2=0x7ff2540b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff259a00640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff25415bd50 0x7ff2540b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff259a00640 1 -- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff25415c130 msgr2=0x7ff2540b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff259a00640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff25415c130 0x7ff2540b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff259a00640 1 -- 172.21.15.44:0/1897512221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25414fa70 con 0x7ff2540cecf0 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.483+0000 7ff259000640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff25415bd50 0x7ff2540b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:28.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.487+0000 7ff259a00640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2540cecf0 0x7ff25414ebe0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff24800c500 tx=0x7ff24800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:28.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.487+0000 7ff251600640 1 -- 172.21.15.44:0/1897512221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248017070 con 0x7ff2540cecf0 2024-09-13T08:49:28.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.487+0000 7ff251600640 1 -- 172.21.15.44:0/1897512221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff248003600 con 0x7ff2540cecf0 2024-09-13T08:49:28.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.487+0000 7ff25c400640 1 -- 172.21.15.44:0/1897512221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2541fb540 con 0x7ff2540cecf0 2024-09-13T08:49:28.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.487+0000 7ff251600640 1 -- 172.21.15.44:0/1897512221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248003a90 con 0x7ff2540cecf0 2024-09-13T08:49:28.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.487+0000 7ff25c400640 1 -- 172.21.15.44:0/1897512221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2541fbb00 con 0x7ff2540cecf0 2024-09-13T08:49:28.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.487+0000 7ff25c400640 1 -- 172.21.15.44:0/1897512221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2540ce5e0 con 0x7ff2540cecf0 2024-09-13T08:49:28.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.487+0000 7ff251600640 1 -- 172.21.15.44:0/1897512221 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7ff24801a070 con 0x7ff2540cecf0 2024-09-13T08:49:28.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.491+0000 7ff251600640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff234074540 0x7ff234076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:28.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.491+0000 7ff251600640 1 -- 172.21.15.44:0/1897512221 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5793+0+0 (secure 0 0 0) 0x7ff248091fe0 con 0x7ff2540cecf0 2024-09-13T08:49:28.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.491+0000 7ff259000640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff234074540 0x7ff234076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:28.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.491+0000 7ff259000640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff234074540 0x7ff234076a00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff2400af000 tx=0x7ff24001af40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:28.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.495+0000 7ff251600640 1 -- 172.21.15.44:0/1897512221 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff24805edd0 con 0x7ff2540cecf0 2024-09-13T08:49:28.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 -- 172.21.15.44:0/1897512221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff254071020 con 0x7ff2540cecf0 2024-09-13T08:49:28.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff251600640 1 -- 172.21.15.44:0/1897512221 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16836 (secure 0 0 0) 0x7ff24805e6d0 con 0x7ff2540cecf0 2024-09-13T08:49:28.673 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:28.673 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":101,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:49:27.697135+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:49:25.408514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-13T08:49:26.657487+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:49:28.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 -- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff234074540 msgr2=0x7ff234076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:28.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff234074540 0x7ff234076a00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff2400af000 tx=0x7ff24001af40 comp rx=0 tx=0).stop 2024-09-13T08:49:28.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 -- 172.21.15.44:0/1897512221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2540cecf0 msgr2=0x7ff25414ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:28.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2540cecf0 0x7ff25414ebe0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff24800c500 tx=0x7ff24800c9c0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 -- 172.21.15.44:0/1897512221 shutdown_connections 2024-09-13T08:49:28.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff234074540 0x7ff234076a00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff25415c130 0x7ff2540b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff25415bd50 0x7ff2540b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 --2- 172.21.15.44:0/1897512221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2540cecf0 0x7ff25414ebe0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.671+0000 7ff25c400640 1 -- 172.21.15.44:0/1897512221 >> 172.21.15.44:0/1897512221 conn(0x7ff2540d1ea0 msgr2=0x7ff25415b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:28.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.675+0000 7ff25c400640 1 -- 172.21.15.44:0/1897512221 shutdown_connections 2024-09-13T08:49:28.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.675+0000 7ff25c400640 1 -- 172.21.15.44:0/1897512221 wait complete. 2024-09-13T08:49:28.695 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:49:28.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.839+0000 7fad8dc00640 1 -- 172.21.15.44:0/567622269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004d50 msgr2=0x7fad6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:28.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.839+0000 7fad8dc00640 1 --2- 172.21.15.44:0/567622269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004d50 0x7fad6c0051d0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fad80009f10 tx=0x7fad8001d5d0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.839+0000 7fad87400640 1 -- 172.21.15.44:0/567622269 reap_dead start 2024-09-13T08:49:28.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.839+0000 7fad8dc00640 1 -- 172.21.15.44:0/567622269 shutdown_connections 2024-09-13T08:49:28.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.839+0000 7fad8dc00640 1 -- 172.21.15.44:0/567622269 >> 172.21.15.44:0/567622269 conn(0x7fad880d1dc0 msgr2=0x7fad880d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:28.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.839+0000 7fad8dc00640 1 -- 172.21.15.44:0/567622269 shutdown_connections 2024-09-13T08:49:28.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.839+0000 7fad8dc00640 1 -- 172.21.15.44:0/567622269 wait complete. 2024-09-13T08:49:28.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad8dc00640 1 Processor -- start 2024-09-13T08:49:28.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad8dc00640 1 -- start start 2024-09-13T08:49:28.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad8dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c004d50 0x7fad881522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:28.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad8dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad880ca980 0x7fad880cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:28.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad8dc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad880b2660 0x7fad880b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:28.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad8dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad8815c400 con 0x7fad880ca980 2024-09-13T08:49:28.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad8dc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad8815c280 con 0x7fad6c004d50 2024-09-13T08:49:28.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad8dc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fad880cebb0 con 0x7fad880b2660 2024-09-13T08:49:28.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad880ca980 0x7fad880cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:28.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad880ca980 0x7fad880cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60144/0 (socket says 172.21.15.44:60144) 2024-09-13T08:49:28.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7fe00640 1 -- 172.21.15.44:0/2044734624 learned_addr learned my addr 172.21.15.44:0/2044734624 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:28.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad86a00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c004d50 0x7fad881522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:28.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad87400640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad880b2660 0x7fad880b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:28.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7fe00640 1 -- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad880b2660 msgr2=0x7fad880b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:28.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7fe00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad880b2660 0x7fad880b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7fe00640 1 -- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c004d50 msgr2=0x7fad881522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:28.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7fe00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c004d50 0x7fad881522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:28.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7fe00640 1 -- 172.21.15.44:0/2044734624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad880b18a0 con 0x7fad880ca980 2024-09-13T08:49:28.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7fe00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad880ca980 0x7fad880cae00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fad70006670 tx=0x7fad700039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:28.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7ea00640 1 -- 172.21.15.44:0/2044734624 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70015070 con 0x7fad880ca980 2024-09-13T08:49:28.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7ea00640 1 -- 172.21.15.44:0/2044734624 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad70011900 con 0x7fad880ca980 2024-09-13T08:49:28.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad8dc00640 1 -- 172.21.15.44:0/2044734624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad880b1b80 con 0x7fad880ca980 2024-09-13T08:49:28.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad7ea00640 1 -- 172.21.15.44:0/2044734624 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70011d50 con 0x7fad880ca980 2024-09-13T08:49:28.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.843+0000 7fad8dc00640 1 -- 172.21.15.44:0/2044734624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad880c9160 con 0x7fad880ca980 2024-09-13T08:49:28.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.847+0000 7fad8dc00640 1 -- 172.21.15.44:0/2044734624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad880cfcf0 con 0x7fad880ca980 2024-09-13T08:49:28.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.847+0000 7fad7ea00640 1 -- 172.21.15.44:0/2044734624 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fad70018070 con 0x7fad880ca980 2024-09-13T08:49:28.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.847+0000 7fad7ea00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad64074540 0x7fad64076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:28.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.847+0000 7fad7ea00640 1 -- 172.21.15.44:0/2044734624 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5793+0+0 (secure 0 0 0) 0x7fad70093780 con 0x7fad880ca980 2024-09-13T08:49:28.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.847+0000 7fad86a00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad64074540 0x7fad64076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:28.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.847+0000 7fad86a00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad64074540 0x7fad64076a00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fad78004610 tx=0x7fad7800b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:28.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:28.851+0000 7fad7ea00640 1 -- 172.21.15.44:0/2044734624 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fad7005fdd0 con 0x7fad880ca980 2024-09-13T08:49:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.027+0000 7fad8dc00640 1 -- 172.21.15.44:0/2044734624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad880cc2f0 con 0x7fad880ca980 2024-09-13T08:49:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.027+0000 7fad7ea00640 1 -- 172.21.15.44:0/2044734624 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16836 (secure 0 0 0) 0x7fad70064800 con 0x7fad880ca980 2024-09-13T08:49:29.030 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:29.031 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":101,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:49:27.697135+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:49:25.408514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-13T08:49:26.657487+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:49:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 -- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad64074540 msgr2=0x7fad64076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad64074540 0x7fad64076a00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fad78004610 tx=0x7fad7800b2e0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 -- 172.21.15.44:0/2044734624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad880ca980 msgr2=0x7fad880cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad880ca980 0x7fad880cae00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fad70006670 tx=0x7fad700039d0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 -- 172.21.15.44:0/2044734624 shutdown_connections 2024-09-13T08:49:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad64074540 0x7fad64076a00 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad880b2660 0x7fad880b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad880ca980 0x7fad880cae00 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 --2- 172.21.15.44:0/2044734624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c004d50 0x7fad881522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 -- 172.21.15.44:0/2044734624 >> 172.21.15.44:0/2044734624 conn(0x7fad880d1dc0 msgr2=0x7fad880cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 -- 172.21.15.44:0/2044734624 shutdown_connections 2024-09-13T08:49:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.031+0000 7fad8dc00640 1 -- 172.21.15.44:0/2044734624 wait complete. 2024-09-13T08:49:29.053 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T08:49:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 -- 172.21.15.44:0/3071433916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40b1650 msgr2=0x7f13c40b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 --2- 172.21.15.44:0/3071433916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40b1650 0x7f13c40b1a30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f13c4074840 tx=0x7f13ac01c8a0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 -- 172.21.15.44:0/3071433916 shutdown_connections 2024-09-13T08:49:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 --2- 172.21.15.44:0/3071433916 >> v2:172.21.15.129:3300/0 conn(0x7f13c40aeb00 0x7f13c40aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 --2- 172.21.15.44:0/3071433916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40b1650 0x7f13c40b1a30 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 --2- 172.21.15.44:0/3071433916 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f13c41605d0 0x7f13c40d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 -- 172.21.15.44:0/3071433916 >> 172.21.15.44:0/3071433916 conn(0x7f13c40d1dd0 msgr2=0x7f13c40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 -- 172.21.15.44:0/3071433916 shutdown_connections 2024-09-13T08:49:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 -- 172.21.15.44:0/3071433916 wait complete. 2024-09-13T08:49:29.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 Processor -- start 2024-09-13T08:49:29.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 -- start start 2024-09-13T08:49:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40aeb00 0x7f13c4155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f13c40b1650 0x7f13c4157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13c41605d0 0x7f13c4158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13c40b13a0 con 0x7f13c40aeb00 2024-09-13T08:49:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f13c40b1220 con 0x7f13c41605d0 2024-09-13T08:49:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c9000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f13c40afd00 con 0x7f13c40b1650 2024-09-13T08:49:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40aeb00 0x7f13c4155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40aeb00 0x7f13c4155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60162/0 (socket says 172.21.15.44:60162) 2024-09-13T08:49:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c1e00640 1 -- 172.21.15.44:0/768374007 learned_addr learned my addr 172.21.15.44:0/768374007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.203+0000 7f13c2800640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13c41605d0 0x7f13c4158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13c1e00640 1 -- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f13c40b1650 msgr2=0x7f13c4157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13c1e00640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f13c40b1650 0x7f13c4157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13c1e00640 1 -- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13c41605d0 msgr2=0x7f13c4158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13c1e00640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13c41605d0 0x7f13c4158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13c1e00640 1 -- 172.21.15.44:0/768374007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c4159820 con 0x7f13c40aeb00 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13c1e00640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40aeb00 0x7f13c4155320 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f13b400c9f0 tx=0x7f13b400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13ba000640 1 -- 172.21.15.44:0/768374007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4007bb0 con 0x7f13c40aeb00 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13ba000640 1 -- 172.21.15.44:0/768374007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13b4007e40 con 0x7f13c40aeb00 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13ba000640 1 -- 172.21.15.44:0/768374007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b400f690 con 0x7f13c40aeb00 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13c9000640 1 -- 172.21.15.44:0/768374007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c4159b00 con 0x7f13c40aeb00 2024-09-13T08:49:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13c9000640 1 -- 172.21.15.44:0/768374007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c415a0c0 con 0x7f13c40aeb00 2024-09-13T08:49:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13c9000640 1 -- 172.21.15.44:0/768374007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c40b1a30 con 0x7f13c40aeb00 2024-09-13T08:49:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13ba000640 1 -- 172.21.15.44:0/768374007 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f13b400f830 con 0x7f13c40aeb00 2024-09-13T08:49:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13ba000640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f13a0074540 0x7f13a0076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.207+0000 7f13ba000640 1 -- 172.21.15.44:0/768374007 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5793+0+0 (secure 0 0 0) 0x7f13b4093280 con 0x7f13c40aeb00 2024-09-13T08:49:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.211+0000 7f13bbe00640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f13a0074540 0x7f13a0076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:29.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.211+0000 7f13bbe00640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f13a0074540 0x7f13a0076a00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0af000 tx=0x7f13ac019640 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:29.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.211+0000 7f13ba000640 1 -- 172.21.15.44:0/768374007 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f13b4014030 con 0x7f13c40aeb00 2024-09-13T08:49:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:29.427+0000 7f13c9000640 1 -- 172.21.15.44:0/768374007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f13c4157760 con 0x7f13c40aeb00 2024-09-13T08:49:30.261 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-09-13T08:49:30.263+0000 7f0514e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:49:30.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.291+0000 7f13ba000640 1 -- 172.21.15.44:0/768374007 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) ==== 118+0+0 (secure 0 0 0) 0x7f13b405ff30 con 0x7f13c40aeb00 2024-09-13T08:49:30.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 -- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f13a0074540 msgr2=0x7f13a0076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:30.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f13a0074540 0x7f13a0076a00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0af000 tx=0x7f13ac019640 comp rx=0 tx=0).stop 2024-09-13T08:49:30.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 -- 172.21.15.44:0/768374007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40aeb00 msgr2=0x7f13c4155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:30.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40aeb00 0x7f13c4155320 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f13b400c9f0 tx=0x7f13b400ceb0 comp rx=0 tx=0).stop 2024-09-13T08:49:30.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 -- 172.21.15.44:0/768374007 shutdown_connections 2024-09-13T08:49:30.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f13a0074540 0x7f13a0076a00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:30.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13c41605d0 0x7f13c4158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:30.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f13c40b1650 0x7f13c4157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:30.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 --2- 172.21.15.44:0/768374007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40aeb00 0x7f13c4155320 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:30.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 -- 172.21.15.44:0/768374007 >> 172.21.15.44:0/768374007 conn(0x7f13c40d1dd0 msgr2=0x7f13c40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:30.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 -- 172.21.15.44:0/768374007 shutdown_connections 2024-09-13T08:49:30.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.295+0000 7f13c9000640 1 -- 172.21.15.44:0/768374007 wait complete. 2024-09-13T08:49:30.321 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T08:49:30.321 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T08:49:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.475+0000 7f2f32400640 1 -- 172.21.15.44:0/272591532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0d0770 msgr2=0x7f2f2c06d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.475+0000 7f2f32400640 1 --2- 172.21.15.44:0/272591532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0d0770 0x7f2f2c06d090 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0b3820 tx=0x7f2f2401ccf0 comp rx=0 tx=0).stop 2024-09-13T08:49:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.475+0000 7f2f32400640 1 -- 172.21.15.44:0/272591532 shutdown_connections 2024-09-13T08:49:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.475+0000 7f2f32400640 1 --2- 172.21.15.44:0/272591532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0d0770 0x7f2f2c06d090 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.475+0000 7f2f32400640 1 --2- 172.21.15.44:0/272591532 >> v2:172.21.15.129:3300/0 conn(0x7f2f2c0cd060 0x7f2f2c161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.475+0000 7f2f32400640 1 -- 172.21.15.44:0/272591532 >> 172.21.15.44:0/272591532 conn(0x7f2f2c0cd740 msgr2=0x7f2f2c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.475+0000 7f2f32400640 1 -- 172.21.15.44:0/272591532 shutdown_connections 2024-09-13T08:49:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.475+0000 7f2f32400640 1 -- 172.21.15.44:0/272591532 wait complete. 2024-09-13T08:49:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.475+0000 7f2f32400640 1 Processor -- start 2024-09-13T08:49:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.475+0000 7f2f32400640 1 -- start start 2024-09-13T08:49:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f32400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f2c06cc20 0x7f2f2c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f32400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0cd060 0x7f2f2c158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f32400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f2c0d0770 0x7f2f2c159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f32400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c16b140 con 0x7f2f2c0cd060 2024-09-13T08:49:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f32400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c16afc0 con 0x7f2f2c06cc20 2024-09-13T08:49:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f32400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2f2c16b2c0 con 0x7f2f2c0d0770 2024-09-13T08:49:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0cd060 0x7f2f2c158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0cd060 0x7f2f2c158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60188/0 (socket says 172.21.15.44:60188) 2024-09-13T08:49:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23e00640 1 -- 172.21.15.44:0/1259437108 learned_addr learned my addr 172.21.15.44:0/1259437108 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f2b400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f2c06cc20 0x7f2f2c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:30.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f2be00640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f2c0d0770 0x7f2f2c159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:30.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23e00640 1 -- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f2c0d0770 msgr2=0x7f2f2c159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:30.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23e00640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f2c0d0770 0x7f2f2c159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:30.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23e00640 1 -- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f2c06cc20 msgr2=0x7f2f2c155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:30.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23e00640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f2c06cc20 0x7f2f2c155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:30.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23e00640 1 -- 172.21.15.44:0/1259437108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c159b80 con 0x7f2f2c0cd060 2024-09-13T08:49:30.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f2b400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f2c06cc20 0x7f2f2c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:30.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23e00640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0cd060 0x7f2f2c158fd0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c3c0 tx=0x7f2f1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23400640 1 -- 172.21.15.44:0/1259437108 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14017070 con 0x7f2f2c0cd060 2024-09-13T08:49:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23400640 1 -- 172.21.15.44:0/1259437108 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f140053c0 con 0x7f2f2c0cd060 2024-09-13T08:49:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f32400640 1 -- 172.21.15.44:0/1259437108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2c159e60 con 0x7f2f2c0cd060 2024-09-13T08:49:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f23400640 1 -- 172.21.15.44:0/1259437108 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14003860 con 0x7f2f2c0cd060 2024-09-13T08:49:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f32400640 1 -- 172.21.15.44:0/1259437108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2c0c5930 con 0x7f2f2c0cd060 2024-09-13T08:49:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.479+0000 7f2f32400640 1 -- 172.21.15.44:0/1259437108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2c0b2520 con 0x7f2f2c0cd060 2024-09-13T08:49:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.483+0000 7f2f23400640 1 -- 172.21.15.44:0/1259437108 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f2f1401a070 con 0x7f2f2c0cd060 2024-09-13T08:49:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.483+0000 7f2f23400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074570 0x7f2f08076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.483+0000 7f2f23400640 1 -- 172.21.15.44:0/1259437108 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f2f14091ea0 con 0x7f2f2c0cd060 2024-09-13T08:49:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.483+0000 7f2f2b400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074570 0x7f2f08076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.483+0000 7f2f2b400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074570 0x7f2f08076a30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0058d0 tx=0x7f2f1c005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.487+0000 7f2f23400640 1 -- 172.21.15.44:0/1259437108 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2f2c0cd440 con 0x7f2f2c0cd060 2024-09-13T08:49:30.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:30.695+0000 7f2f32400640 1 -- 172.21.15.44:0/1259437108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2f2c0b6c40 con 0x7f2f2c0cd060 2024-09-13T08:49:31.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f23400640 1 -- 172.21.15.44:0/1259437108 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) ==== 122+0+0 (secure 0 0 0) 0x7f2f1405eb50 con 0x7f2f2c0cd060 2024-09-13T08:49:31.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f32400640 1 -- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074570 msgr2=0x7f2f08076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:31.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f32400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074570 0x7f2f08076a30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0058d0 tx=0x7f2f1c005860 comp rx=0 tx=0).stop 2024-09-13T08:49:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f32400640 1 -- 172.21.15.44:0/1259437108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0cd060 msgr2=0x7f2f2c158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f32400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0cd060 0x7f2f2c158fd0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c3c0 tx=0x7f2f1400c880 comp rx=0 tx=0).stop 2024-09-13T08:49:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f32400640 1 -- 172.21.15.44:0/1259437108 shutdown_connections 2024-09-13T08:49:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f32400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074570 0x7f2f08076a30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f32400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f2c0d0770 0x7f2f2c159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f32400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0cd060 0x7f2f2c158fd0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f32400640 1 --2- 172.21.15.44:0/1259437108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f2c06cc20 0x7f2f2c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.299+0000 7f2f32400640 1 -- 172.21.15.44:0/1259437108 >> 172.21.15.44:0/1259437108 conn(0x7f2f2c0cd740 msgr2=0x7f2f2c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.303+0000 7f2f32400640 1 -- 172.21.15.44:0/1259437108 shutdown_connections 2024-09-13T08:49:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.303+0000 7f2f32400640 1 -- 172.21.15.44:0/1259437108 wait complete. 2024-09-13T08:49:31.327 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T08:49:31.328 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:49:31.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.475+0000 7f73be200640 1 -- 172.21.15.44:0/2331459168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b80737c0 msgr2=0x7f73b8073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:31.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.475+0000 7f73be200640 1 --2- 172.21.15.44:0/2331459168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b80737c0 0x7f73b8073c20 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f73b8074c60 tx=0x7f73a001c8c0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 -- 172.21.15.44:0/2331459168 shutdown_connections 2024-09-13T08:49:31.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 --2- 172.21.15.44:0/2331459168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b80737c0 0x7f73b8073c20 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 --2- 172.21.15.44:0/2331459168 >> v2:172.21.15.129:3300/0 conn(0x7f73b815af60 0x7f73b80cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 -- 172.21.15.44:0/2331459168 >> 172.21.15.44:0/2331459168 conn(0x7f73b80d0c90 msgr2=0x7f73b80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:31.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 -- 172.21.15.44:0/2331459168 shutdown_connections 2024-09-13T08:49:31.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 -- 172.21.15.44:0/2331459168 wait complete. 2024-09-13T08:49:31.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 Processor -- start 2024-09-13T08:49:31.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 -- start start 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b80737c0 0x7f73b80c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b8074160 0x7f73b80c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b815af60 0x7f73b80c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8167b40 con 0x7f73b8074160 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8070b40 con 0x7f73b80737c0 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73be200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f73b8167cc0 con 0x7f73b815af60 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73aea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b8074160 0x7f73b80c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73aea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b8074160 0x7f73b80c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60200/0 (socket says 172.21.15.44:60200) 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73aea00640 1 -- 172.21.15.44:0/1181886369 learned_addr learned my addr 172.21.15.44:0/1181886369 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73b7400640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b80737c0 0x7f73b80c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73b7e00640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b815af60 0x7f73b80c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:31.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73aea00640 1 -- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b815af60 msgr2=0x7f73b80c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:31.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73aea00640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b815af60 0x7f73b80c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73aea00640 1 -- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b80737c0 msgr2=0x7f73b80c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:31.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73aea00640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b80737c0 0x7f73b80c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73aea00640 1 -- 172.21.15.44:0/1181886369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b80b0790 con 0x7f73b8074160 2024-09-13T08:49:31.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.479+0000 7f73aea00640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b8074160 0x7f73b80c5360 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f73a00b0040 tx=0x7f73a00a5610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:31.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.483+0000 7f73afe00640 1 -- 172.21.15.44:0/1181886369 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a00b2040 con 0x7f73b8074160 2024-09-13T08:49:31.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.483+0000 7f73afe00640 1 -- 172.21.15.44:0/1181886369 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73a0004380 con 0x7f73b8074160 2024-09-13T08:49:31.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.483+0000 7f73afe00640 1 -- 172.21.15.44:0/1181886369 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0005280 con 0x7f73b8074160 2024-09-13T08:49:31.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.483+0000 7f73be200640 1 -- 172.21.15.44:0/1181886369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b8154f10 con 0x7f73b8074160 2024-09-13T08:49:31.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.483+0000 7f73be200640 1 -- 172.21.15.44:0/1181886369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b80c4860 con 0x7f73b8074160 2024-09-13T08:49:31.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.483+0000 7f73be200640 1 -- 172.21.15.44:0/1181886369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b80b0960 con 0x7f73b8074160 2024-09-13T08:49:31.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.483+0000 7f73afe00640 1 -- 172.21.15.44:0/1181886369 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f73a0005420 con 0x7f73b8074160 2024-09-13T08:49:31.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.483+0000 7f73afe00640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7394074540 0x7f7394076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:31.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.483+0000 7f73afe00640 1 -- 172.21.15.44:0/1181886369 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f73a012ccb0 con 0x7f73b8074160 2024-09-13T08:49:31.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.483+0000 7f73b7400640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7394074540 0x7f7394076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:31.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.487+0000 7f73b7400640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7394074540 0x7f7394076a00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f73a800bb50 tx=0x7f73a800b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:31.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.487+0000 7f73afe00640 1 -- 172.21.15.44:0/1181886369 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f73a00164b0 con 0x7f73b8074160 2024-09-13T08:49:31.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.699+0000 7f73be200640 1 -- 172.21.15.44:0/1181886369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73b80cced0 con 0x7f73b8074160 2024-09-13T08:49:31.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.699+0000 7f73afe00640 1 -- 172.21.15.44:0/1181886369 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5225 (secure 0 0 0) 0x7f73a00f9290 con 0x7f73b8074160 2024-09-13T08:49:31.702 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:31.702 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":65,"btime":"2024-09-13T08:49:31:296465+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1987308515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1987308515},{"type":"v1","addr":"172.21.15.129:6837","nonce":1987308515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1224713128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1224713128},{"type":"v1","addr":"172.21.15.129:6836","nonce":1224713128}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7755,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/380853907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":380853907},{"type":"v1","addr":"172.21.15.44:6833","nonce":380853907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:49:27.697092+0000","modified":"2024-09-13T08:49:30.701664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/409896091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":409896091},{"type":"v1","addr":"172.21.15.44:6835","nonce":409896091}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7758,"qdb_cluster":[7758]},"id":7}]} 2024-09-13T08:49:31.703 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 65 2024-09-13T08:49:31.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 -- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7394074540 msgr2=0x7f7394076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:31.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7394074540 0x7f7394076a00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f73a800bb50 tx=0x7f73a800b2e0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 -- 172.21.15.44:0/1181886369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b8074160 msgr2=0x7f73b80c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:31.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b8074160 0x7f73b80c5360 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f73a00b0040 tx=0x7f73a00a5610 comp rx=0 tx=0).stop 2024-09-13T08:49:31.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 -- 172.21.15.44:0/1181886369 shutdown_connections 2024-09-13T08:49:31.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7394074540 0x7f7394076a00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b815af60 0x7f73b80c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b8074160 0x7f73b80c5360 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 --2- 172.21.15.44:0/1181886369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b80737c0 0x7f73b80c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 -- 172.21.15.44:0/1181886369 >> 172.21.15.44:0/1181886369 conn(0x7f73b80d0c90 msgr2=0x7f73b8157530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:31.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 -- 172.21.15.44:0/1181886369 shutdown_connections 2024-09-13T08:49:31.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.703+0000 7f73be200640 1 -- 172.21.15.44:0/1181886369 wait complete. 2024-09-13T08:49:31.724 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:49:31.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 -- 172.21.15.44:0/446614369 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16440ced80 msgr2=0x7f16440ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:31.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 --2- 172.21.15.44:0/446614369 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16440ced80 0x7f16440ae9b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1630009870 tx=0x7f163001c860 comp rx=0 tx=0).stop 2024-09-13T08:49:31.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 -- 172.21.15.44:0/446614369 shutdown_connections 2024-09-13T08:49:31.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 --2- 172.21.15.44:0/446614369 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16440ced80 0x7f16440ae9b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 -- 172.21.15.44:0/446614369 >> 172.21.15.44:0/446614369 conn(0x7f16440d1dc0 msgr2=0x7f16440d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:31.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 -- 172.21.15.44:0/446614369 shutdown_connections 2024-09-13T08:49:31.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 -- 172.21.15.44:0/446614369 wait complete. 2024-09-13T08:49:31.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 Processor -- start 2024-09-13T08:49:31.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 -- start start 2024-09-13T08:49:31.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16440cdb30 0x7f16441582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:31.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16440ced80 0x7f164414a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:31.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f164415bea0 0x7f16440b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f164416a8f0 con 0x7f164415bea0 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f164416a770 con 0x7f16440ced80 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164c600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f164416aa70 con 0x7f16440cdb30 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f164415bea0 0x7f16440b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f164415bea0 0x7f16440b2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60208/0 (socket says 172.21.15.44:60208) 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f164a600640 1 -- 172.21.15.44:0/3498515527 learned_addr learned my addr 172.21.15.44:0/3498515527 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f1649c00640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16440cdb30 0x7f16441582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.871+0000 7f1649200640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16440ced80 0x7f164414a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f164a600640 1 -- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16440cdb30 msgr2=0x7f16441582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f164a600640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16440cdb30 0x7f16441582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f164a600640 1 -- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16440ced80 msgr2=0x7f164414a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f164a600640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16440ced80 0x7f164414a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f164a600640 1 -- 172.21.15.44:0/3498515527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16440b2ec0 con 0x7f164415bea0 2024-09-13T08:49:31.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f164a600640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f164415bea0 0x7f16440b2980 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f163c00cb30 tx=0x7f163c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:31.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f1641600640 1 -- 172.21.15.44:0/3498515527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c016020 con 0x7f164415bea0 2024-09-13T08:49:31.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f1641600640 1 -- 172.21.15.44:0/3498515527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f163c00ab40 con 0x7f164415bea0 2024-09-13T08:49:31.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f1641600640 1 -- 172.21.15.44:0/3498515527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c012420 con 0x7f164415bea0 2024-09-13T08:49:31.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f164c600640 1 -- 172.21.15.44:0/3498515527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16440b18e0 con 0x7f164415bea0 2024-09-13T08:49:31.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f164c600640 1 -- 172.21.15.44:0/3498515527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16440b1ea0 con 0x7f164415bea0 2024-09-13T08:49:31.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f164c600640 1 -- 172.21.15.44:0/3498515527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16440ce670 con 0x7f164415bea0 2024-09-13T08:49:31.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f1641600640 1 -- 172.21.15.44:0/3498515527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f163c019020 con 0x7f164415bea0 2024-09-13T08:49:31.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.875+0000 7f1641600640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1624074540 0x7f1624076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:31.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.879+0000 7f1641600640 1 -- 172.21.15.44:0/3498515527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f163c092b80 con 0x7f164415bea0 2024-09-13T08:49:31.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.879+0000 7f1649c00640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1624074540 0x7f1624076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:31.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.879+0000 7f1649c00640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1624074540 0x7f1624076a00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f164406fa30 tx=0x7f1638009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:31.883+0000 7f1641600640 1 -- 172.21.15.44:0/3498515527 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f163c020020 con 0x7f164415bea0 2024-09-13T08:49:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.055+0000 7f164c600640 1 -- 172.21.15.44:0/3498515527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16440cdfa0 con 0x7f164415bea0 2024-09-13T08:49:32.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f1641600640 1 -- 172.21.15.44:0/3498515527 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16823 (secure 0 0 0) 0x7f163c05f280 con 0x7f164415bea0 2024-09-13T08:49:32.060 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:32.060 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":102,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:49:29.279209+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:49:25.408514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-13T08:49:26.657487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:49:32.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 -- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1624074540 msgr2=0x7f1624076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1624074540 0x7f1624076a00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f164406fa30 tx=0x7f1638009040 comp rx=0 tx=0).stop 2024-09-13T08:49:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 -- 172.21.15.44:0/3498515527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f164415bea0 msgr2=0x7f16440b2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f164415bea0 0x7f16440b2980 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f163c00cb30 tx=0x7f163c00cc20 comp rx=0 tx=0).stop 2024-09-13T08:49:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 -- 172.21.15.44:0/3498515527 shutdown_connections 2024-09-13T08:49:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1624074540 0x7f1624076a00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f164415bea0 0x7f16440b2980 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16440ced80 0x7f164414a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 --2- 172.21.15.44:0/3498515527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16440cdb30 0x7f16441582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 -- 172.21.15.44:0/3498515527 >> 172.21.15.44:0/3498515527 conn(0x7f16440d1dc0 msgr2=0x7f16440d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 -- 172.21.15.44:0/3498515527 shutdown_connections 2024-09-13T08:49:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.059+0000 7f164c600640 1 -- 172.21.15.44:0/3498515527 wait complete. 2024-09-13T08:49:32.084 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:49:32.084 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:49:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 -- 172.21.15.44:0/932417205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4778003c90 msgr2=0x7f4778004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 --2- 172.21.15.44:0/932417205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4778003c90 0x7f4778004090 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4784001f50 tx=0x7f478401cc20 comp rx=0 tx=0).stop 2024-09-13T08:49:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 -- 172.21.15.44:0/932417205 shutdown_connections 2024-09-13T08:49:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 --2- 172.21.15.44:0/932417205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4778003c90 0x7f4778004090 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 -- 172.21.15.44:0/932417205 >> 172.21.15.44:0/932417205 conn(0x7f47940d1dc0 msgr2=0x7f47940d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 -- 172.21.15.44:0/932417205 shutdown_connections 2024-09-13T08:49:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 -- 172.21.15.44:0/932417205 wait complete. 2024-09-13T08:49:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 Processor -- start 2024-09-13T08:49:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 -- start start 2024-09-13T08:49:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4778003c90 0x7f47941faae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47940b1dd0 0x7f47940b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47940b3e90 0x7f479406e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f47940cea80 con 0x7f47940b3e90 2024-09-13T08:49:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47940ce900 con 0x7f47940b1dd0 2024-09-13T08:49:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4799000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f47940cec00 con 0x7f4778003c90 2024-09-13T08:49:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4792800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47940b3e90 0x7f479406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4792800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47940b3e90 0x7f479406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42506/0 (socket says 172.21.15.44:42506) 2024-09-13T08:49:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4792800640 1 -- 172.21.15.44:0/3921714752 learned_addr learned my addr 172.21.15.44:0/3921714752 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4789400640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47940b1dd0 0x7f47940b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4791e00640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4778003c90 0x7f47941faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4792800640 1 -- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4778003c90 msgr2=0x7f47941faae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4792800640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4778003c90 0x7f47941faae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4792800640 1 -- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47940b1dd0 msgr2=0x7f47940b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4792800640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47940b1dd0 0x7f47940b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.231+0000 7f4792800640 1 -- 172.21.15.44:0/3921714752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47940cbd40 con 0x7f47940b3e90 2024-09-13T08:49:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f4792800640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47940b3e90 0x7f479406e940 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f478c001ec0 tx=0x7f478c00e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f478aa00640 1 -- 172.21.15.44:0/3921714752 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c015020 con 0x7f47940b3e90 2024-09-13T08:49:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f478aa00640 1 -- 172.21.15.44:0/3921714752 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f478c011960 con 0x7f47940b3e90 2024-09-13T08:49:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f478aa00640 1 -- 172.21.15.44:0/3921714752 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c011db0 con 0x7f47940b3e90 2024-09-13T08:49:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f4799000640 1 -- 172.21.15.44:0/3921714752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479406de60 con 0x7f47940b3e90 2024-09-13T08:49:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f4799000640 1 -- 172.21.15.44:0/3921714752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47940b4310 con 0x7f47940b3e90 2024-09-13T08:49:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f4799000640 1 -- 172.21.15.44:0/3921714752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47940cd4f0 con 0x7f47940b3e90 2024-09-13T08:49:32.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f478aa00640 1 -- 172.21.15.44:0/3921714752 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f478c018020 con 0x7f47940b3e90 2024-09-13T08:49:32.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f478aa00640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4770074570 0x7f4770076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:32.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f478aa00640 1 -- 172.21.15.44:0/3921714752 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f478c092d80 con 0x7f47940b3e90 2024-09-13T08:49:32.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.235+0000 7f4791e00640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4770074570 0x7f4770076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:32.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.239+0000 7f4791e00640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4770074570 0x7f4770076a30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f47840a9000 tx=0x7f4784019a60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:32.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.239+0000 7f478aa00640 1 -- 172.21.15.44:0/3921714752 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f478c01f030 con 0x7f47940b3e90 2024-09-13T08:49:32.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.415+0000 7f4799000640 1 -- 172.21.15.44:0/3921714752 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47940b47b0 con 0x7f4770074570 2024-09-13T08:49:32.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f478aa00640 1 -- 172.21.15.44:0/3921714752 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252421 (secure 0 0 0) 0x7f47840fba20 con 0x7f4770074570 2024-09-13T08:49:32.427 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:32.428 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 -- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4770074570 msgr2=0x7f4770076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4770074570 0x7f4770076a30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f47840a9000 tx=0x7f4784019a60 comp rx=0 tx=0).stop 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 -- 172.21.15.44:0/3921714752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47940b3e90 msgr2=0x7f479406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47940b3e90 0x7f479406e940 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f478c001ec0 tx=0x7f478c00e770 comp rx=0 tx=0).stop 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 -- 172.21.15.44:0/3921714752 shutdown_connections 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4770074570 0x7f4770076a30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47940b3e90 0x7f479406e940 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47940b1dd0 0x7f47940b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 --2- 172.21.15.44:0/3921714752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4778003c90 0x7f47941faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 -- 172.21.15.44:0/3921714752 >> 172.21.15.44:0/3921714752 conn(0x7f47940d1dc0 msgr2=0x7f47940d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.423+0000 7f4799000640 1 -- 172.21.15.44:0/3921714752 shutdown_connections 2024-09-13T08:49:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.427+0000 7f4799000640 1 -- 172.21.15.44:0/3921714752 wait complete. 2024-09-13T08:49:32.447 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-09-13T08:49:30.774078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":859688,"kb_used_data":10920,"kb_used_omap":12,"kb_used_meta":848563,"kb_avail":754115032,"statfs":{"total":773094113280,"available":772213792768,"internally_reserved":0,"allocated":11182080,"data_stored":5503786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":868929073},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.505046"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309764+0000","last_change":"2024-09-13T08:49:28.308301+0000","last_active":"2024-09-13T08:49:28.309764+0000","last_peered":"2024-09-13T08:49:28.309764+0000","last_clean":"2024-09-13T08:49:28.309764+0000","last_became_active":"2024-09-13T08:49:28.307210+0000","last_became_peered":"2024-09-13T08:49:28.307210+0000","last_unstale":"2024-09-13T08:49:28.309764+0000","last_undegraded":"2024-09-13T08:49:28.309764+0000","last_fullsized":"2024-09-13T08:49:28.309764+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:54.439629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318313+0000","last_change":"2024-09-13T08:49:28.311455+0000","last_active":"2024-09-13T08:49:28.318313+0000","last_peered":"2024-09-13T08:49:28.318313+0000","last_clean":"2024-09-13T08:49:28.318313+0000","last_became_active":"2024-09-13T08:49:28.310745+0000","last_became_peered":"2024-09-13T08:49:28.310745+0000","last_unstale":"2024-09-13T08:49:28.318313+0000","last_undegraded":"2024-09-13T08:49:28.318313+0000","last_fullsized":"2024-09-13T08:49:28.318313+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:09.660148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309299+0000","last_change":"2024-09-13T08:49:28.298501+0000","last_active":"2024-09-13T08:49:29.309299+0000","last_peered":"2024-09-13T08:49:29.309299+0000","last_clean":"2024-09-13T08:49:29.309299+0000","last_became_active":"2024-09-13T08:49:28.298245+0000","last_became_peered":"2024-09-13T08:49:28.298245+0000","last_unstale":"2024-09-13T08:49:29.309299+0000","last_undegraded":"2024-09-13T08:49:29.309299+0000","last_fullsized":"2024-09-13T08:49:29.309299+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:01.526002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.305420+0000","last_change":"2024-09-13T08:49:28.304773+0000","last_active":"2024-09-13T08:49:28.305420+0000","last_peered":"2024-09-13T08:49:28.305420+0000","last_clean":"2024-09-13T08:49:28.305420+0000","last_became_active":"2024-09-13T08:49:28.304456+0000","last_became_peered":"2024-09-13T08:49:28.304456+0000","last_unstale":"2024-09-13T08:49:28.305420+0000","last_undegraded":"2024-09-13T08:49:28.305420+0000","last_fullsized":"2024-09-13T08:49:28.305420+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:13.924359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.302091+0000","last_change":"2024-09-13T08:49:28.298893+0000","last_active":"2024-09-13T08:49:28.302091+0000","last_peered":"2024-09-13T08:49:28.302091+0000","last_clean":"2024-09-13T08:49:28.302091+0000","last_became_active":"2024-09-13T08:49:28.298620+0000","last_became_peered":"2024-09-13T08:49:28.298620+0000","last_unstale":"2024-09-13T08:49:28.302091+0000","last_undegraded":"2024-09-13T08:49:28.302091+0000","last_fullsized":"2024-09-13T08:49:28.302091+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:34.628036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.300167+0000","last_change":"2024-09-13T08:49:28.295832+0000","last_active":"2024-09-13T08:49:28.300167+0000","last_peered":"2024-09-13T08:49:28.300167+0000","last_clean":"2024-09-13T08:49:28.300167+0000","last_became_active":"2024-09-13T08:49:28.294324+0000","last_became_peered":"2024-09-13T08:49:28.294324+0000","last_unstale":"2024-09-13T08:49:28.300167+0000","last_undegraded":"2024-09-13T08:49:28.300167+0000","last_fullsized":"2024-09-13T08:49:28.300167+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:33.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.303316+0000","last_change":"2024-09-13T08:49:28.301121+0000","last_active":"2024-09-13T08:49:28.303316+0000","last_peered":"2024-09-13T08:49:28.303316+0000","last_clean":"2024-09-13T08:49:28.303316+0000","last_became_active":"2024-09-13T08:49:28.300815+0000","last_became_peered":"2024-09-13T08:49:28.300815+0000","last_unstale":"2024-09-13T08:49:28.303316+0000","last_undegraded":"2024-09-13T08:49:28.303316+0000","last_fullsized":"2024-09-13T08:49:28.303316+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:04:59.423182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309053+0000","last_change":"2024-09-13T08:49:28.293945+0000","last_active":"2024-09-13T08:49:29.309053+0000","last_peered":"2024-09-13T08:49:29.309053+0000","last_clean":"2024-09-13T08:49:29.309053+0000","last_became_active":"2024-09-13T08:49:28.293673+0000","last_became_peered":"2024-09-13T08:49:28.293673+0000","last_unstale":"2024-09-13T08:49:29.309053+0000","last_undegraded":"2024-09-13T08:49:29.309053+0000","last_fullsized":"2024-09-13T08:49:29.309053+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:54.895240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.312361+0000","last_change":"2024-09-13T08:49:28.309249+0000","last_active":"2024-09-13T08:49:28.312361+0000","last_peered":"2024-09-13T08:49:28.312361+0000","last_clean":"2024-09-13T08:49:28.312361+0000","last_became_active":"2024-09-13T08:49:28.308969+0000","last_became_peered":"2024-09-13T08:49:28.308969+0000","last_unstale":"2024-09-13T08:49:28.312361+0000","last_undegraded":"2024-09-13T08:49:28.312361+0000","last_fullsized":"2024-09-13T08:49:28.312361+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.463148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.312463+0000","last_change":"2024-09-13T08:49:28.310940+0000","last_active":"2024-09-13T08:49:28.312463+0000","last_peered":"2024-09-13T08:49:28.312463+0000","last_clean":"2024-09-13T08:49:28.312463+0000","last_became_active":"2024-09-13T08:49:28.310689+0000","last_became_peered":"2024-09-13T08:49:28.310689+0000","last_unstale":"2024-09-13T08:49:28.312463+0000","last_undegraded":"2024-09-13T08:49:28.312463+0000","last_fullsized":"2024-09-13T08:49:28.312463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:29.813282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.303111+0000","last_change":"2024-09-13T08:49:28.300689+0000","last_active":"2024-09-13T08:49:28.303111+0000","last_peered":"2024-09-13T08:49:28.303111+0000","last_clean":"2024-09-13T08:49:28.303111+0000","last_became_active":"2024-09-13T08:49:28.300327+0000","last_became_peered":"2024-09-13T08:49:28.300327+0000","last_unstale":"2024-09-13T08:49:28.303111+0000","last_undegraded":"2024-09-13T08:49:28.303111+0000","last_fullsized":"2024-09-13T08:49:28.303111+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:50:03.833588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.310517+0000","last_change":"2024-09-13T08:49:28.309720+0000","last_active":"2024-09-13T08:49:28.310517+0000","last_peered":"2024-09-13T08:49:28.310517+0000","last_clean":"2024-09-13T08:49:28.310517+0000","last_became_active":"2024-09-13T08:49:28.309258+0000","last_became_peered":"2024-09-13T08:49:28.309258+0000","last_unstale":"2024-09-13T08:49:28.310517+0000","last_undegraded":"2024-09-13T08:49:28.310517+0000","last_fullsized":"2024-09-13T08:49:28.310517+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:46.425416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318285+0000","last_change":"2024-09-13T08:49:28.317239+0000","last_active":"2024-09-13T08:49:28.318285+0000","last_peered":"2024-09-13T08:49:28.318285+0000","last_clean":"2024-09-13T08:49:28.318285+0000","last_became_active":"2024-09-13T08:49:28.316923+0000","last_became_peered":"2024-09-13T08:49:28.316923+0000","last_unstale":"2024-09-13T08:49:28.318285+0000","last_undegraded":"2024-09-13T08:49:28.318285+0000","last_fullsized":"2024-09-13T08:49:28.318285+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:14.508154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.319762+0000","last_change":"2024-09-13T08:49:28.312290+0000","last_active":"2024-09-13T08:49:28.319762+0000","last_peered":"2024-09-13T08:49:28.319762+0000","last_clean":"2024-09-13T08:49:28.319762+0000","last_became_active":"2024-09-13T08:49:28.311643+0000","last_became_peered":"2024-09-13T08:49:28.311643+0000","last_unstale":"2024-09-13T08:49:28.319762+0000","last_undegraded":"2024-09-13T08:49:28.319762+0000","last_fullsized":"2024-09-13T08:49:28.319762+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:13.418708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309619+0000","last_change":"2024-09-13T08:49:28.307566+0000","last_active":"2024-09-13T08:49:28.309619+0000","last_peered":"2024-09-13T08:49:28.309619+0000","last_clean":"2024-09-13T08:49:28.309619+0000","last_became_active":"2024-09-13T08:49:28.306626+0000","last_became_peered":"2024-09-13T08:49:28.306626+0000","last_unstale":"2024-09-13T08:49:28.309619+0000","last_undegraded":"2024-09-13T08:49:28.309619+0000","last_fullsized":"2024-09-13T08:49:28.309619+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:53.143087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.302430+0000","last_change":"2024-09-13T08:49:28.298983+0000","last_active":"2024-09-13T08:49:28.302430+0000","last_peered":"2024-09-13T08:49:28.302430+0000","last_clean":"2024-09-13T08:49:28.302430+0000","last_became_active":"2024-09-13T08:49:28.293845+0000","last_became_peered":"2024-09-13T08:49:28.293845+0000","last_unstale":"2024-09-13T08:49:28.302430+0000","last_undegraded":"2024-09-13T08:49:28.302430+0000","last_fullsized":"2024-09-13T08:49:28.302430+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:42.579804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287971+0000","last_change":"2024-09-13T08:49:28.307169+0000","last_active":"2024-09-13T08:49:29.287971+0000","last_peered":"2024-09-13T08:49:29.287971+0000","last_clean":"2024-09-13T08:49:29.287971+0000","last_became_active":"2024-09-13T08:49:28.306905+0000","last_became_peered":"2024-09-13T08:49:28.306905+0000","last_unstale":"2024-09-13T08:49:29.287971+0000","last_undegraded":"2024-09-13T08:49:29.287971+0000","last_fullsized":"2024-09-13T08:49:29.287971+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:34:43.033131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318263+0000","last_change":"2024-09-13T08:49:28.311396+0000","last_active":"2024-09-13T08:49:28.318263+0000","last_peered":"2024-09-13T08:49:28.318263+0000","last_clean":"2024-09-13T08:49:28.318263+0000","last_became_active":"2024-09-13T08:49:28.310454+0000","last_became_peered":"2024-09-13T08:49:28.310454+0000","last_unstale":"2024-09-13T08:49:28.318263+0000","last_undegraded":"2024-09-13T08:49:28.318263+0000","last_fullsized":"2024-09-13T08:49:28.318263+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:21.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287539+0000","last_change":"2024-09-13T08:49:28.313300+0000","last_active":"2024-09-13T08:49:29.287539+0000","last_peered":"2024-09-13T08:49:29.287539+0000","last_clean":"2024-09-13T08:49:29.287539+0000","last_became_active":"2024-09-13T08:49:28.313046+0000","last_became_peered":"2024-09-13T08:49:28.313046+0000","last_unstale":"2024-09-13T08:49:29.287539+0000","last_undegraded":"2024-09-13T08:49:29.287539+0000","last_fullsized":"2024-09-13T08:49:29.287539+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:10.595581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309877+0000","last_change":"2024-09-13T08:49:28.307967+0000","last_active":"2024-09-13T08:49:28.309877+0000","last_peered":"2024-09-13T08:49:28.309877+0000","last_clean":"2024-09-13T08:49:28.309877+0000","last_became_active":"2024-09-13T08:49:28.307639+0000","last_became_peered":"2024-09-13T08:49:28.307639+0000","last_unstale":"2024-09-13T08:49:28.309877+0000","last_undegraded":"2024-09-13T08:49:28.309877+0000","last_fullsized":"2024-09-13T08:49:28.309877+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:27:31.001532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309665+0000","last_change":"2024-09-13T08:49:28.308961+0000","last_active":"2024-09-13T08:49:29.309665+0000","last_peered":"2024-09-13T08:49:29.309665+0000","last_clean":"2024-09-13T08:49:29.309665+0000","last_became_active":"2024-09-13T08:49:28.307859+0000","last_became_peered":"2024-09-13T08:49:28.307859+0000","last_unstale":"2024-09-13T08:49:29.309665+0000","last_undegraded":"2024-09-13T08:49:29.309665+0000","last_fullsized":"2024-09-13T08:49:29.309665+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:18.993392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309166+0000","last_change":"2024-09-13T08:49:28.304589+0000","last_active":"2024-09-13T08:49:29.309166+0000","last_peered":"2024-09-13T08:49:29.309166+0000","last_clean":"2024-09-13T08:49:29.309166+0000","last_became_active":"2024-09-13T08:49:28.304315+0000","last_became_peered":"2024-09-13T08:49:28.304315+0000","last_unstale":"2024-09-13T08:49:29.309166+0000","last_undegraded":"2024-09-13T08:49:29.309166+0000","last_fullsized":"2024-09-13T08:49:29.309166+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:23:39.301642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.319321+0000","last_change":"2024-09-13T08:49:28.311959+0000","last_active":"2024-09-13T08:49:28.319321+0000","last_peered":"2024-09-13T08:49:28.319321+0000","last_clean":"2024-09-13T08:49:28.319321+0000","last_became_active":"2024-09-13T08:49:28.311608+0000","last_became_peered":"2024-09-13T08:49:28.311608+0000","last_unstale":"2024-09-13T08:49:28.319321+0000","last_undegraded":"2024-09-13T08:49:28.319321+0000","last_fullsized":"2024-09-13T08:49:28.319321+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:30:49.273165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.307904+0000","last_change":"2024-09-13T08:49:28.302145+0000","last_active":"2024-09-13T08:49:28.307904+0000","last_peered":"2024-09-13T08:49:28.307904+0000","last_clean":"2024-09-13T08:49:28.307904+0000","last_became_active":"2024-09-13T08:49:28.301663+0000","last_became_peered":"2024-09-13T08:49:28.301663+0000","last_unstale":"2024-09-13T08:49:28.307904+0000","last_undegraded":"2024-09-13T08:49:28.307904+0000","last_fullsized":"2024-09-13T08:49:28.307904+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:48.882320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.300170+0000","last_change":"2024-09-13T08:49:28.295011+0000","last_active":"2024-09-13T08:49:28.300170+0000","last_peered":"2024-09-13T08:49:28.300170+0000","last_clean":"2024-09-13T08:49:28.300170+0000","last_became_active":"2024-09-13T08:49:28.294708+0000","last_became_peered":"2024-09-13T08:49:28.294708+0000","last_unstale":"2024-09-13T08:49:28.300170+0000","last_undegraded":"2024-09-13T08:49:28.300170+0000","last_fullsized":"2024-09-13T08:49:28.300170+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:18.993392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.311236+0000","last_change":"2024-09-13T08:49:28.307615+0000","last_active":"2024-09-13T08:49:28.311236+0000","last_peered":"2024-09-13T08:49:28.311236+0000","last_clean":"2024-09-13T08:49:28.311236+0000","last_became_active":"2024-09-13T08:49:28.306711+0000","last_became_peered":"2024-09-13T08:49:28.306711+0000","last_unstale":"2024-09-13T08:49:28.311236+0000","last_undegraded":"2024-09-13T08:49:28.311236+0000","last_fullsized":"2024-09-13T08:49:28.311236+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:31.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.311943+0000","last_change":"2024-09-13T08:49:28.310812+0000","last_active":"2024-09-13T08:49:28.311943+0000","last_peered":"2024-09-13T08:49:28.311943+0000","last_clean":"2024-09-13T08:49:28.311943+0000","last_became_active":"2024-09-13T08:49:28.310384+0000","last_became_peered":"2024-09-13T08:49:28.310384+0000","last_unstale":"2024-09-13T08:49:28.311943+0000","last_undegraded":"2024-09-13T08:49:28.311943+0000","last_fullsized":"2024-09-13T08:49:28.311943+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:17.492931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.296410+0000","last_change":"2024-09-13T08:49:28.290617+0000","last_active":"2024-09-13T08:49:28.296410+0000","last_peered":"2024-09-13T08:49:28.296410+0000","last_clean":"2024-09-13T08:49:28.296410+0000","last_became_active":"2024-09-13T08:49:28.290356+0000","last_became_peered":"2024-09-13T08:49:28.290356+0000","last_unstale":"2024-09-13T08:49:28.296410+0000","last_undegraded":"2024-09-13T08:49:28.296410+0000","last_fullsized":"2024-09-13T08:49:28.296410+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:13:44.081989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.311212+0000","last_change":"2024-09-13T08:49:28.307187+0000","last_active":"2024-09-13T08:49:28.311212+0000","last_peered":"2024-09-13T08:49:28.311212+0000","last_clean":"2024-09-13T08:49:28.311212+0000","last_became_active":"2024-09-13T08:49:28.306919+0000","last_became_peered":"2024-09-13T08:49:28.306919+0000","last_unstale":"2024-09-13T08:49:28.311212+0000","last_undegraded":"2024-09-13T08:49:28.311212+0000","last_fullsized":"2024-09-13T08:49:28.311212+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:16:01.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.314869+0000","last_change":"2024-09-13T08:49:28.307751+0000","last_active":"2024-09-13T08:49:28.314869+0000","last_peered":"2024-09-13T08:49:28.314869+0000","last_clean":"2024-09-13T08:49:28.314869+0000","last_became_active":"2024-09-13T08:49:28.306690+0000","last_became_peered":"2024-09-13T08:49:28.306690+0000","last_unstale":"2024-09-13T08:49:28.314869+0000","last_undegraded":"2024-09-13T08:49:28.314869+0000","last_fullsized":"2024-09-13T08:49:28.314869+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:01:22.163556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309395+0000","last_change":"2024-09-13T08:49:28.292684+0000","last_active":"2024-09-13T08:49:29.309395+0000","last_peered":"2024-09-13T08:49:29.309395+0000","last_clean":"2024-09-13T08:49:29.309395+0000","last_became_active":"2024-09-13T08:49:28.292370+0000","last_became_peered":"2024-09-13T08:49:28.292370+0000","last_unstale":"2024-09-13T08:49:29.309395+0000","last_undegraded":"2024-09-13T08:49:29.309395+0000","last_fullsized":"2024-09-13T08:49:29.309395+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:39.842892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.304073+0000","last_change":"2024-09-13T08:49:28.301383+0000","last_active":"2024-09-13T08:49:28.304073+0000","last_peered":"2024-09-13T08:49:28.304073+0000","last_clean":"2024-09-13T08:49:28.304073+0000","last_became_active":"2024-09-13T08:49:28.300938+0000","last_became_peered":"2024-09-13T08:49:28.300938+0000","last_unstale":"2024-09-13T08:49:28.304073+0000","last_undegraded":"2024-09-13T08:49:28.304073+0000","last_fullsized":"2024-09-13T08:49:28.304073+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:21:11.937699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287857+0000","last_change":"2024-09-13T08:49:28.304025+0000","last_active":"2024-09-13T08:49:29.287857+0000","last_peered":"2024-09-13T08:49:29.287857+0000","last_clean":"2024-09-13T08:49:29.287857+0000","last_became_active":"2024-09-13T08:49:28.303484+0000","last_became_peered":"2024-09-13T08:49:28.303484+0000","last_unstale":"2024-09-13T08:49:29.287857+0000","last_undegraded":"2024-09-13T08:49:29.287857+0000","last_fullsized":"2024-09-13T08:49:29.287857+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:54:14.094928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.321983+0000","last_change":"2024-09-13T08:49:27.304233+0000","last_active":"2024-09-13T08:49:28.321983+0000","last_peered":"2024-09-13T08:49:28.321983+0000","last_clean":"2024-09-13T08:49:28.321983+0000","last_became_active":"2024-09-13T08:49:27.303922+0000","last_became_peered":"2024-09-13T08:49:27.303922+0000","last_unstale":"2024-09-13T08:49:28.321983+0000","last_undegraded":"2024-09-13T08:49:28.321983+0000","last_fullsized":"2024-09-13T08:49:28.321983+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:01.166230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.308586+0000","last_change":"2024-09-13T08:49:28.304603+0000","last_active":"2024-09-13T08:49:28.308586+0000","last_peered":"2024-09-13T08:49:28.308586+0000","last_clean":"2024-09-13T08:49:28.308586+0000","last_became_active":"2024-09-13T08:49:28.304315+0000","last_became_peered":"2024-09-13T08:49:28.304315+0000","last_unstale":"2024-09-13T08:49:28.308586+0000","last_undegraded":"2024-09-13T08:49:28.308586+0000","last_fullsized":"2024-09-13T08:49:28.308586+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:30.501323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.321641+0000","last_change":"2024-09-13T08:49:27.291602+0000","last_active":"2024-09-13T08:49:28.321641+0000","last_peered":"2024-09-13T08:49:28.321641+0000","last_clean":"2024-09-13T08:49:28.321641+0000","last_became_active":"2024-09-13T08:49:27.291157+0000","last_became_peered":"2024-09-13T08:49:27.291157+0000","last_unstale":"2024-09-13T08:49:28.321641+0000","last_undegraded":"2024-09-13T08:49:28.321641+0000","last_fullsized":"2024-09-13T08:49:28.321641+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:06:20.232249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.304245+0000","last_change":"2024-09-13T08:49:28.302518+0000","last_active":"2024-09-13T08:49:28.304245+0000","last_peered":"2024-09-13T08:49:28.304245+0000","last_clean":"2024-09-13T08:49:28.304245+0000","last_became_active":"2024-09-13T08:49:28.302275+0000","last_became_peered":"2024-09-13T08:49:28.302275+0000","last_unstale":"2024-09-13T08:49:28.304245+0000","last_undegraded":"2024-09-13T08:49:28.304245+0000","last_fullsized":"2024-09-13T08:49:28.304245+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:47:33.775174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309323+0000","last_change":"2024-09-13T08:49:27.307806+0000","last_active":"2024-09-13T08:49:29.309323+0000","last_peered":"2024-09-13T08:49:29.309323+0000","last_clean":"2024-09-13T08:49:29.309323+0000","last_became_active":"2024-09-13T08:49:27.307441+0000","last_became_peered":"2024-09-13T08:49:27.307441+0000","last_unstale":"2024-09-13T08:49:29.309323+0000","last_undegraded":"2024-09-13T08:49:29.309323+0000","last_fullsized":"2024-09-13T08:49:29.309323+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:08.620519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.299724+0000","last_change":"2024-09-13T08:49:28.295293+0000","last_active":"2024-09-13T08:49:28.299724+0000","last_peered":"2024-09-13T08:49:28.299724+0000","last_clean":"2024-09-13T08:49:28.299724+0000","last_became_active":"2024-09-13T08:49:28.294933+0000","last_became_peered":"2024-09-13T08:49:28.294933+0000","last_unstale":"2024-09-13T08:49:28.299724+0000","last_undegraded":"2024-09-13T08:49:28.299724+0000","last_fullsized":"2024-09-13T08:49:28.299724+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:50:30.506890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.324102+0000","last_change":"2024-09-13T08:49:27.304809+0000","last_active":"2024-09-13T08:49:28.324102+0000","last_peered":"2024-09-13T08:49:28.324102+0000","last_clean":"2024-09-13T08:49:28.324102+0000","last_became_active":"2024-09-13T08:49:27.304460+0000","last_became_peered":"2024-09-13T08:49:27.304460+0000","last_unstale":"2024-09-13T08:49:28.324102+0000","last_undegraded":"2024-09-13T08:49:28.324102+0000","last_fullsized":"2024-09-13T08:49:28.324102+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:01:56.309772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.304967+0000","last_change":"2024-09-13T08:49:28.303525+0000","last_active":"2024-09-13T08:49:28.304967+0000","last_peered":"2024-09-13T08:49:28.304967+0000","last_clean":"2024-09-13T08:49:28.304967+0000","last_became_active":"2024-09-13T08:49:28.303276+0000","last_became_peered":"2024-09-13T08:49:28.303276+0000","last_unstale":"2024-09-13T08:49:28.304967+0000","last_undegraded":"2024-09-13T08:49:28.304967+0000","last_fullsized":"2024-09-13T08:49:28.304967+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:09:11.535610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.323437+0000","last_change":"2024-09-13T08:49:27.306908+0000","last_active":"2024-09-13T08:49:28.323437+0000","last_peered":"2024-09-13T08:49:28.323437+0000","last_clean":"2024-09-13T08:49:28.323437+0000","last_became_active":"2024-09-13T08:49:27.306187+0000","last_became_peered":"2024-09-13T08:49:27.306187+0000","last_unstale":"2024-09-13T08:49:28.323437+0000","last_undegraded":"2024-09-13T08:49:28.323437+0000","last_fullsized":"2024-09-13T08:49:28.323437+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:44.144108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.303251+0000","last_change":"2024-09-13T08:49:28.300754+0000","last_active":"2024-09-13T08:49:28.303251+0000","last_peered":"2024-09-13T08:49:28.303251+0000","last_clean":"2024-09-13T08:49:28.303251+0000","last_became_active":"2024-09-13T08:49:28.300409+0000","last_became_peered":"2024-09-13T08:49:28.300409+0000","last_unstale":"2024-09-13T08:49:28.303251+0000","last_undegraded":"2024-09-13T08:49:28.303251+0000","last_fullsized":"2024-09-13T08:49:28.303251+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:42:12.765747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309086+0000","last_change":"2024-09-13T08:49:27.300301+0000","last_active":"2024-09-13T08:49:29.309086+0000","last_peered":"2024-09-13T08:49:29.309086+0000","last_clean":"2024-09-13T08:49:29.309086+0000","last_became_active":"2024-09-13T08:49:27.299991+0000","last_became_peered":"2024-09-13T08:49:27.299991+0000","last_unstale":"2024-09-13T08:49:29.309086+0000","last_undegraded":"2024-09-13T08:49:29.309086+0000","last_fullsized":"2024-09-13T08:49:29.309086+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:48:27.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.315807+0000","last_change":"2024-09-13T08:49:28.310176+0000","last_active":"2024-09-13T08:49:28.315807+0000","last_peered":"2024-09-13T08:49:28.315807+0000","last_clean":"2024-09-13T08:49:28.315807+0000","last_became_active":"2024-09-13T08:49:28.309838+0000","last_became_peered":"2024-09-13T08:49:28.309838+0000","last_unstale":"2024-09-13T08:49:28.315807+0000","last_undegraded":"2024-09-13T08:49:28.315807+0000","last_fullsized":"2024-09-13T08:49:28.315807+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:30.570457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.288333+0000","last_change":"2024-09-13T08:49:27.297539+0000","last_active":"2024-09-13T08:49:28.288333+0000","last_peered":"2024-09-13T08:49:28.288333+0000","last_clean":"2024-09-13T08:49:28.288333+0000","last_became_active":"2024-09-13T08:49:27.297241+0000","last_became_peered":"2024-09-13T08:49:27.297241+0000","last_unstale":"2024-09-13T08:49:28.288333+0000","last_undegraded":"2024-09-13T08:49:28.288333+0000","last_fullsized":"2024-09-13T08:49:28.288333+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:55.812887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.301018+0000","last_change":"2024-09-13T08:49:28.295461+0000","last_active":"2024-09-13T08:49:28.301018+0000","last_peered":"2024-09-13T08:49:28.301018+0000","last_clean":"2024-09-13T08:49:28.301018+0000","last_became_active":"2024-09-13T08:49:28.295200+0000","last_became_peered":"2024-09-13T08:49:28.295200+0000","last_unstale":"2024-09-13T08:49:28.301018+0000","last_undegraded":"2024-09-13T08:49:28.301018+0000","last_fullsized":"2024-09-13T08:49:28.301018+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:05:37.274424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.286333+0000","last_change":"2024-09-13T08:49:27.305047+0000","last_active":"2024-09-13T08:49:29.286333+0000","last_peered":"2024-09-13T08:49:29.286333+0000","last_clean":"2024-09-13T08:49:29.286333+0000","last_became_active":"2024-09-13T08:49:27.304783+0000","last_became_peered":"2024-09-13T08:49:27.304783+0000","last_unstale":"2024-09-13T08:49:29.286333+0000","last_undegraded":"2024-09-13T08:49:29.286333+0000","last_fullsized":"2024-09-13T08:49:29.286333+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:47.228172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318236+0000","last_change":"2024-09-13T08:49:28.311153+0000","last_active":"2024-09-13T08:49:28.318236+0000","last_peered":"2024-09-13T08:49:28.318236+0000","last_clean":"2024-09-13T08:49:28.318236+0000","last_became_active":"2024-09-13T08:49:28.310070+0000","last_became_peered":"2024-09-13T08:49:28.310070+0000","last_unstale":"2024-09-13T08:49:28.318236+0000","last_undegraded":"2024-09-13T08:49:28.318236+0000","last_fullsized":"2024-09-13T08:49:28.318236+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:58.780865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309483+0000","last_change":"2024-09-13T08:49:27.301210+0000","last_active":"2024-09-13T08:49:29.309483+0000","last_peered":"2024-09-13T08:49:29.309483+0000","last_clean":"2024-09-13T08:49:29.309483+0000","last_became_active":"2024-09-13T08:49:27.297621+0000","last_became_peered":"2024-09-13T08:49:27.297621+0000","last_unstale":"2024-09-13T08:49:29.309483+0000","last_undegraded":"2024-09-13T08:49:29.309483+0000","last_fullsized":"2024-09-13T08:49:29.309483+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:21.001638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.299245+0000","last_change":"2024-09-13T08:49:28.295106+0000","last_active":"2024-09-13T08:49:28.299245+0000","last_peered":"2024-09-13T08:49:28.299245+0000","last_clean":"2024-09-13T08:49:28.299245+0000","last_became_active":"2024-09-13T08:49:28.291915+0000","last_became_peered":"2024-09-13T08:49:28.291915+0000","last_unstale":"2024-09-13T08:49:28.299245+0000","last_undegraded":"2024-09-13T08:49:28.299245+0000","last_fullsized":"2024-09-13T08:49:28.299245+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:49.365537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.308893+0000","last_change":"2024-09-13T08:49:27.308011+0000","last_active":"2024-09-13T08:49:28.308893+0000","last_peered":"2024-09-13T08:49:28.308893+0000","last_clean":"2024-09-13T08:49:28.308893+0000","last_became_active":"2024-09-13T08:49:27.307735+0000","last_became_peered":"2024-09-13T08:49:27.307735+0000","last_unstale":"2024-09-13T08:49:28.308893+0000","last_undegraded":"2024-09-13T08:49:28.308893+0000","last_fullsized":"2024-09-13T08:49:28.308893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:49:16.620549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.307447+0000","last_change":"2024-09-13T08:49:28.306278+0000","last_active":"2024-09-13T08:49:28.307447+0000","last_peered":"2024-09-13T08:49:28.307447+0000","last_clean":"2024-09-13T08:49:28.307447+0000","last_became_active":"2024-09-13T08:49:28.305531+0000","last_became_peered":"2024-09-13T08:49:28.305531+0000","last_unstale":"2024-09-13T08:49:28.307447+0000","last_undegraded":"2024-09-13T08:49:28.307447+0000","last_fullsized":"2024-09-13T08:49:28.307447+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:54:52.031333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318562+0000","last_change":"2024-09-13T08:49:27.297795+0000","last_active":"2024-09-13T08:49:28.318562+0000","last_peered":"2024-09-13T08:49:28.318562+0000","last_clean":"2024-09-13T08:49:28.318562+0000","last_became_active":"2024-09-13T08:49:27.297504+0000","last_became_peered":"2024-09-13T08:49:27.297504+0000","last_unstale":"2024-09-13T08:49:28.318562+0000","last_undegraded":"2024-09-13T08:49:28.318562+0000","last_fullsized":"2024-09-13T08:49:28.318562+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:07:27.376375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.300586+0000","last_change":"2024-09-13T08:49:28.295590+0000","last_active":"2024-09-13T08:49:28.300586+0000","last_peered":"2024-09-13T08:49:28.300586+0000","last_clean":"2024-09-13T08:49:28.300586+0000","last_became_active":"2024-09-13T08:49:28.295322+0000","last_became_peered":"2024-09-13T08:49:28.295322+0000","last_unstale":"2024-09-13T08:49:28.300586+0000","last_undegraded":"2024-09-13T08:49:28.300586+0000","last_fullsized":"2024-09-13T08:49:28.300586+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:55.355769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.324258+0000","last_change":"2024-09-13T08:49:27.303146+0000","last_active":"2024-09-13T08:49:28.324258+0000","last_peered":"2024-09-13T08:49:28.324258+0000","last_clean":"2024-09-13T08:49:28.324258+0000","last_became_active":"2024-09-13T08:49:27.302845+0000","last_became_peered":"2024-09-13T08:49:27.302845+0000","last_unstale":"2024-09-13T08:49:28.324258+0000","last_undegraded":"2024-09-13T08:49:28.324258+0000","last_fullsized":"2024-09-13T08:49:28.324258+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:17:00.617948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309672+0000","last_change":"2024-09-13T08:49:28.305629+0000","last_active":"2024-09-13T08:49:28.309672+0000","last_peered":"2024-09-13T08:49:28.309672+0000","last_clean":"2024-09-13T08:49:28.309672+0000","last_became_active":"2024-09-13T08:49:28.305295+0000","last_became_peered":"2024-09-13T08:49:28.305295+0000","last_unstale":"2024-09-13T08:49:28.309672+0000","last_undegraded":"2024-09-13T08:49:28.309672+0000","last_fullsized":"2024-09-13T08:49:28.309672+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:52.786482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287345+0000","last_change":"2024-09-13T08:49:27.301916+0000","last_active":"2024-09-13T08:49:29.287345+0000","last_peered":"2024-09-13T08:49:29.287345+0000","last_clean":"2024-09-13T08:49:29.287345+0000","last_became_active":"2024-09-13T08:49:27.301629+0000","last_became_peered":"2024-09-13T08:49:27.301629+0000","last_unstale":"2024-09-13T08:49:29.287345+0000","last_undegraded":"2024-09-13T08:49:29.287345+0000","last_fullsized":"2024-09-13T08:49:29.287345+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:34.501901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309776+0000","last_change":"2024-09-13T08:49:28.303461+0000","last_active":"2024-09-13T08:49:29.309776+0000","last_peered":"2024-09-13T08:49:29.309776+0000","last_clean":"2024-09-13T08:49:29.309776+0000","last_became_active":"2024-09-13T08:49:28.303125+0000","last_became_peered":"2024-09-13T08:49:28.303125+0000","last_unstale":"2024-09-13T08:49:29.309776+0000","last_undegraded":"2024-09-13T08:49:29.309776+0000","last_fullsized":"2024-09-13T08:49:29.309776+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:50:30.506890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.296613+0000","last_change":"2024-09-13T08:49:27.305735+0000","last_active":"2024-09-13T08:49:28.296613+0000","last_peered":"2024-09-13T08:49:28.296613+0000","last_clean":"2024-09-13T08:49:28.296613+0000","last_became_active":"2024-09-13T08:49:27.304538+0000","last_became_peered":"2024-09-13T08:49:27.304538+0000","last_unstale":"2024-09-13T08:49:28.296613+0000","last_undegraded":"2024-09-13T08:49:28.296613+0000","last_fullsized":"2024-09-13T08:49:28.296613+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:22.837897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.302404+0000","last_change":"2024-09-13T08:49:28.296332+0000","last_active":"2024-09-13T08:49:28.302404+0000","last_peered":"2024-09-13T08:49:28.302404+0000","last_clean":"2024-09-13T08:49:28.302404+0000","last_became_active":"2024-09-13T08:49:28.294152+0000","last_became_peered":"2024-09-13T08:49:28.294152+0000","last_unstale":"2024-09-13T08:49:28.302404+0000","last_undegraded":"2024-09-13T08:49:28.302404+0000","last_fullsized":"2024-09-13T08:49:28.302404+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:54.477778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.285981+0000","last_change":"2024-09-13T08:49:27.295451+0000","last_active":"2024-09-13T08:49:28.285981+0000","last_peered":"2024-09-13T08:49:28.285981+0000","last_clean":"2024-09-13T08:49:28.285981+0000","last_became_active":"2024-09-13T08:49:27.295025+0000","last_became_peered":"2024-09-13T08:49:27.295025+0000","last_unstale":"2024-09-13T08:49:28.285981+0000","last_undegraded":"2024-09-13T08:49:28.285981+0000","last_fullsized":"2024-09-13T08:49:28.285981+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:01:21.160415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309208+0000","last_change":"2024-09-13T08:49:28.302164+0000","last_active":"2024-09-13T08:49:29.309208+0000","last_peered":"2024-09-13T08:49:29.309208+0000","last_clean":"2024-09-13T08:49:29.309208+0000","last_became_active":"2024-09-13T08:49:28.301826+0000","last_became_peered":"2024-09-13T08:49:28.301826+0000","last_unstale":"2024-09-13T08:49:29.309208+0000","last_undegraded":"2024-09-13T08:49:29.309208+0000","last_fullsized":"2024-09-13T08:49:29.309208+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:42:10.631635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":21,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.325278+0000","last_change":"2024-09-13T08:49:27.308147+0000","last_active":"2024-09-13T08:49:28.325278+0000","last_peered":"2024-09-13T08:49:28.325278+0000","last_clean":"2024-09-13T08:49:28.325278+0000","last_became_active":"2024-09-13T08:49:27.306431+0000","last_became_peered":"2024-09-13T08:49:27.306431+0000","last_unstale":"2024-09-13T08:49:28.325278+0000","last_undegraded":"2024-09-13T08:49:28.325278+0000","last_fullsized":"2024-09-13T08:49:28.325278+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:09.200022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.297778+0000","last_change":"2024-09-13T08:49:28.289688+0000","last_active":"2024-09-13T08:49:28.297778+0000","last_peered":"2024-09-13T08:49:28.297778+0000","last_clean":"2024-09-13T08:49:28.297778+0000","last_became_active":"2024-09-13T08:49:28.289376+0000","last_became_peered":"2024-09-13T08:49:28.289376+0000","last_unstale":"2024-09-13T08:49:28.297778+0000","last_undegraded":"2024-09-13T08:49:28.297778+0000","last_fullsized":"2024-09-13T08:49:28.297778+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:15:23.757716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.322059+0000","last_change":"2024-09-13T08:49:27.298388+0000","last_active":"2024-09-13T08:49:28.322059+0000","last_peered":"2024-09-13T08:49:28.322059+0000","last_clean":"2024-09-13T08:49:28.322059+0000","last_became_active":"2024-09-13T08:49:27.298108+0000","last_became_peered":"2024-09-13T08:49:27.298108+0000","last_unstale":"2024-09-13T08:49:28.322059+0000","last_undegraded":"2024-09-13T08:49:28.322059+0000","last_fullsized":"2024-09-13T08:49:28.322059+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:30.882957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.297370+0000","last_change":"2024-09-13T08:49:28.293206+0000","last_active":"2024-09-13T08:49:28.297370+0000","last_peered":"2024-09-13T08:49:28.297370+0000","last_clean":"2024-09-13T08:49:28.297370+0000","last_became_active":"2024-09-13T08:49:28.292934+0000","last_became_peered":"2024-09-13T08:49:28.292934+0000","last_unstale":"2024-09-13T08:49:28.297370+0000","last_undegraded":"2024-09-13T08:49:28.297370+0000","last_fullsized":"2024-09-13T08:49:28.297370+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:15.584522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.317698+0000","last_change":"2024-09-13T08:49:27.294920+0000","last_active":"2024-09-13T08:49:28.317698+0000","last_peered":"2024-09-13T08:49:28.317698+0000","last_clean":"2024-09-13T08:49:28.317698+0000","last_became_active":"2024-09-13T08:49:27.293444+0000","last_became_peered":"2024-09-13T08:49:27.293444+0000","last_unstale":"2024-09-13T08:49:28.317698+0000","last_undegraded":"2024-09-13T08:49:28.317698+0000","last_fullsized":"2024-09-13T08:49:28.317698+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:31:47.656651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309421+0000","last_change":"2024-09-13T08:49:28.308062+0000","last_active":"2024-09-13T08:49:28.309421+0000","last_peered":"2024-09-13T08:49:28.309421+0000","last_clean":"2024-09-13T08:49:28.309421+0000","last_became_active":"2024-09-13T08:49:28.304255+0000","last_became_peered":"2024-09-13T08:49:28.304255+0000","last_unstale":"2024-09-13T08:49:28.309421+0000","last_undegraded":"2024-09-13T08:49:28.309421+0000","last_fullsized":"2024-09-13T08:49:28.309421+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:39.130420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287517+0000","last_change":"2024-09-13T08:49:27.295352+0000","last_active":"2024-09-13T08:49:29.287517+0000","last_peered":"2024-09-13T08:49:29.287517+0000","last_clean":"2024-09-13T08:49:29.287517+0000","last_became_active":"2024-09-13T08:49:27.295072+0000","last_became_peered":"2024-09-13T08:49:27.295072+0000","last_unstale":"2024-09-13T08:49:29.287517+0000","last_undegraded":"2024-09-13T08:49:29.287517+0000","last_fullsized":"2024-09-13T08:49:29.287517+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:16.489790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.286280+0000","last_change":"2024-09-13T08:49:28.308145+0000","last_active":"2024-09-13T08:49:29.286280+0000","last_peered":"2024-09-13T08:49:29.286280+0000","last_clean":"2024-09-13T08:49:29.286280+0000","last_became_active":"2024-09-13T08:49:28.307835+0000","last_became_peered":"2024-09-13T08:49:28.307835+0000","last_unstale":"2024-09-13T08:49:29.286280+0000","last_undegraded":"2024-09-13T08:49:29.286280+0000","last_fullsized":"2024-09-13T08:49:29.286280+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:29:32.858641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.324135+0000","last_change":"2024-09-13T08:49:27.301401+0000","last_active":"2024-09-13T08:49:28.324135+0000","last_peered":"2024-09-13T08:49:28.324135+0000","last_clean":"2024-09-13T08:49:28.324135+0000","last_became_active":"2024-09-13T08:49:27.300443+0000","last_became_peered":"2024-09-13T08:49:27.300443+0000","last_unstale":"2024-09-13T08:49:28.324135+0000","last_undegraded":"2024-09-13T08:49:28.324135+0000","last_fullsized":"2024-09-13T08:49:28.324135+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:08.144557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287381+0000","last_change":"2024-09-13T08:49:28.311582+0000","last_active":"2024-09-13T08:49:29.287381+0000","last_peered":"2024-09-13T08:49:29.287381+0000","last_clean":"2024-09-13T08:49:29.287381+0000","last_became_active":"2024-09-13T08:49:28.311296+0000","last_became_peered":"2024-09-13T08:49:28.311296+0000","last_unstale":"2024-09-13T08:49:29.287381+0000","last_undegraded":"2024-09-13T08:49:29.287381+0000","last_fullsized":"2024-09-13T08:49:29.287381+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:03.789700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.281266+0000","last_change":"2024-09-13T08:49:27.302457+0000","last_active":"2024-09-13T08:49:28.281266+0000","last_peered":"2024-09-13T08:49:28.281266+0000","last_clean":"2024-09-13T08:49:28.281266+0000","last_became_active":"2024-09-13T08:49:27.302080+0000","last_became_peered":"2024-09-13T08:49:27.302080+0000","last_unstale":"2024-09-13T08:49:28.281266+0000","last_undegraded":"2024-09-13T08:49:28.281266+0000","last_fullsized":"2024-09-13T08:49:28.281266+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:57.166670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.319669+0000","last_change":"2024-09-13T08:49:28.313087+0000","last_active":"2024-09-13T08:49:28.319669+0000","last_peered":"2024-09-13T08:49:28.319669+0000","last_clean":"2024-09-13T08:49:28.319669+0000","last_became_active":"2024-09-13T08:49:28.312748+0000","last_became_peered":"2024-09-13T08:49:28.312748+0000","last_unstale":"2024-09-13T08:49:28.319669+0000","last_undegraded":"2024-09-13T08:49:28.319669+0000","last_fullsized":"2024-09-13T08:49:28.319669+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:28.155055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.285418+0000","last_change":"2024-09-13T08:49:27.297279+0000","last_active":"2024-09-13T08:49:28.285418+0000","last_peered":"2024-09-13T08:49:28.285418+0000","last_clean":"2024-09-13T08:49:28.285418+0000","last_became_active":"2024-09-13T08:49:27.296888+0000","last_became_peered":"2024-09-13T08:49:27.296888+0000","last_unstale":"2024-09-13T08:49:28.285418+0000","last_undegraded":"2024-09-13T08:49:28.285418+0000","last_fullsized":"2024-09-13T08:49:28.285418+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:37.775891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.294424+0000","last_change":"2024-09-13T08:49:28.291798+0000","last_active":"2024-09-13T08:49:28.294424+0000","last_peered":"2024-09-13T08:49:28.294424+0000","last_clean":"2024-09-13T08:49:28.294424+0000","last_became_active":"2024-09-13T08:49:28.291128+0000","last_became_peered":"2024-09-13T08:49:28.291128+0000","last_unstale":"2024-09-13T08:49:28.294424+0000","last_undegraded":"2024-09-13T08:49:28.294424+0000","last_fullsized":"2024-09-13T08:49:28.294424+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:04.105940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309099+0000","last_change":"2024-09-13T08:49:27.293707+0000","last_active":"2024-09-13T08:49:29.309099+0000","last_peered":"2024-09-13T08:49:29.309099+0000","last_clean":"2024-09-13T08:49:29.309099+0000","last_became_active":"2024-09-13T08:49:27.293309+0000","last_became_peered":"2024-09-13T08:49:27.293309+0000","last_unstale":"2024-09-13T08:49:29.309099+0000","last_undegraded":"2024-09-13T08:49:29.309099+0000","last_fullsized":"2024-09-13T08:49:29.309099+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:00:29.660465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287880+0000","last_change":"2024-09-13T08:49:28.303736+0000","last_active":"2024-09-13T08:49:29.287880+0000","last_peered":"2024-09-13T08:49:29.287880+0000","last_clean":"2024-09-13T08:49:29.287880+0000","last_became_active":"2024-09-13T08:49:28.303128+0000","last_became_peered":"2024-09-13T08:49:28.303128+0000","last_unstale":"2024-09-13T08:49:29.287880+0000","last_undegraded":"2024-09-13T08:49:29.287880+0000","last_fullsized":"2024-09-13T08:49:29.287880+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:16:06.842512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318190+0000","last_change":"2024-09-13T08:49:27.296205+0000","last_active":"2024-09-13T08:49:28.318190+0000","last_peered":"2024-09-13T08:49:28.318190+0000","last_clean":"2024-09-13T08:49:28.318190+0000","last_became_active":"2024-09-13T08:49:27.295922+0000","last_became_peered":"2024-09-13T08:49:27.295922+0000","last_unstale":"2024-09-13T08:49:28.318190+0000","last_undegraded":"2024-09-13T08:49:28.318190+0000","last_fullsized":"2024-09-13T08:49:28.318190+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:13.505013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.288027+0000","last_change":"2024-09-13T08:49:28.314734+0000","last_active":"2024-09-13T08:49:29.288027+0000","last_peered":"2024-09-13T08:49:29.288027+0000","last_clean":"2024-09-13T08:49:29.288027+0000","last_became_active":"2024-09-13T08:49:28.314439+0000","last_became_peered":"2024-09-13T08:49:28.314439+0000","last_unstale":"2024-09-13T08:49:29.288027+0000","last_undegraded":"2024-09-13T08:49:29.288027+0000","last_fullsized":"2024-09-13T08:49:29.288027+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:50.516830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"102'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.308239+0000","last_change":"2024-09-13T08:49:27.297035+0000","last_active":"2024-09-13T08:49:29.308239+0000","last_peered":"2024-09-13T08:49:29.308239+0000","last_clean":"2024-09-13T08:49:29.308239+0000","last_became_active":"2024-09-13T08:49:27.296377+0000","last_became_peered":"2024-09-13T08:49:27.296377+0000","last_unstale":"2024-09-13T08:49:29.308239+0000","last_undegraded":"2024-09-13T08:49:29.308239+0000","last_fullsized":"2024-09-13T08:49:29.308239+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:55:27.031358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309592+0000","last_change":"2024-09-13T08:49:28.295786+0000","last_active":"2024-09-13T08:49:29.309592+0000","last_peered":"2024-09-13T08:49:29.309592+0000","last_clean":"2024-09-13T08:49:29.309592+0000","last_became_active":"2024-09-13T08:49:28.295463+0000","last_became_peered":"2024-09-13T08:49:28.295463+0000","last_unstale":"2024-09-13T08:49:29.309592+0000","last_undegraded":"2024-09-13T08:49:29.309592+0000","last_fullsized":"2024-09-13T08:49:29.309592+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:38.779032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287688+0000","last_change":"2024-09-13T08:49:27.298941+0000","last_active":"2024-09-13T08:49:29.287688+0000","last_peered":"2024-09-13T08:49:29.287688+0000","last_clean":"2024-09-13T08:49:29.287688+0000","last_became_active":"2024-09-13T08:49:27.296342+0000","last_became_peered":"2024-09-13T08:49:27.296342+0000","last_unstale":"2024-09-13T08:49:29.287688+0000","last_undegraded":"2024-09-13T08:49:29.287688+0000","last_fullsized":"2024-09-13T08:49:29.287688+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:47:07.117691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.303382+0000","last_change":"2024-09-13T08:49:28.301482+0000","last_active":"2024-09-13T08:49:28.303382+0000","last_peered":"2024-09-13T08:49:28.303382+0000","last_clean":"2024-09-13T08:49:28.303382+0000","last_became_active":"2024-09-13T08:49:28.301216+0000","last_became_peered":"2024-09-13T08:49:28.301216+0000","last_unstale":"2024-09-13T08:49:28.303382+0000","last_undegraded":"2024-09-13T08:49:28.303382+0000","last_fullsized":"2024-09-13T08:49:28.303382+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:44:13.777035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.281747+0000","last_change":"2024-09-13T08:49:27.297259+0000","last_active":"2024-09-13T08:49:28.281747+0000","last_peered":"2024-09-13T08:49:28.281747+0000","last_clean":"2024-09-13T08:49:28.281747+0000","last_became_active":"2024-09-13T08:49:27.296996+0000","last_became_peered":"2024-09-13T08:49:27.296996+0000","last_unstale":"2024-09-13T08:49:28.281747+0000","last_undegraded":"2024-09-13T08:49:28.281747+0000","last_fullsized":"2024-09-13T08:49:28.281747+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:23.151811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287256+0000","last_change":"2024-09-13T08:49:28.303409+0000","last_active":"2024-09-13T08:49:29.287256+0000","last_peered":"2024-09-13T08:49:29.287256+0000","last_clean":"2024-09-13T08:49:29.287256+0000","last_became_active":"2024-09-13T08:49:28.303111+0000","last_became_peered":"2024-09-13T08:49:28.303111+0000","last_unstale":"2024-09-13T08:49:29.287256+0000","last_undegraded":"2024-09-13T08:49:29.287256+0000","last_fullsized":"2024-09-13T08:49:29.287256+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:03.350204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309727+0000","last_change":"2024-09-13T08:49:27.305906+0000","last_active":"2024-09-13T08:49:29.309727+0000","last_peered":"2024-09-13T08:49:29.309727+0000","last_clean":"2024-09-13T08:49:29.309727+0000","last_became_active":"2024-09-13T08:49:27.305623+0000","last_became_peered":"2024-09-13T08:49:27.305623+0000","last_unstale":"2024-09-13T08:49:29.309727+0000","last_undegraded":"2024-09-13T08:49:29.309727+0000","last_fullsized":"2024-09-13T08:49:29.309727+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:19:42.944227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309752+0000","last_change":"2024-09-13T08:49:28.307343+0000","last_active":"2024-09-13T08:49:28.309752+0000","last_peered":"2024-09-13T08:49:28.309752+0000","last_clean":"2024-09-13T08:49:28.309752+0000","last_became_active":"2024-09-13T08:49:28.307067+0000","last_became_peered":"2024-09-13T08:49:28.307067+0000","last_unstale":"2024-09-13T08:49:28.309752+0000","last_undegraded":"2024-09-13T08:49:28.309752+0000","last_fullsized":"2024-09-13T08:49:28.309752+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:49.541352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.281506+0000","last_change":"2024-09-13T08:49:27.296163+0000","last_active":"2024-09-13T08:49:28.281506+0000","last_peered":"2024-09-13T08:49:28.281506+0000","last_clean":"2024-09-13T08:49:28.281506+0000","last_became_active":"2024-09-13T08:49:27.295765+0000","last_became_peered":"2024-09-13T08:49:27.295765+0000","last_unstale":"2024-09-13T08:49:28.281506+0000","last_undegraded":"2024-09-13T08:49:28.281506+0000","last_fullsized":"2024-09-13T08:49:28.281506+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:48:56.292419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.286227+0000","last_change":"2024-09-13T08:49:28.296017+0000","last_active":"2024-09-13T08:49:29.286227+0000","last_peered":"2024-09-13T08:49:29.286227+0000","last_clean":"2024-09-13T08:49:29.286227+0000","last_became_active":"2024-09-13T08:49:28.295704+0000","last_became_peered":"2024-09-13T08:49:28.295704+0000","last_unstale":"2024-09-13T08:49:29.286227+0000","last_undegraded":"2024-09-13T08:49:29.286227+0000","last_fullsized":"2024-09-13T08:49:29.286227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:14:55.145831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.297160+0000","last_change":"2024-09-13T08:49:27.305544+0000","last_active":"2024-09-13T08:49:28.297160+0000","last_peered":"2024-09-13T08:49:28.297160+0000","last_clean":"2024-09-13T08:49:28.297160+0000","last_became_active":"2024-09-13T08:49:27.304766+0000","last_became_peered":"2024-09-13T08:49:27.304766+0000","last_unstale":"2024-09-13T08:49:28.297160+0000","last_undegraded":"2024-09-13T08:49:28.297160+0000","last_fullsized":"2024-09-13T08:49:28.297160+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:10.382062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.311952+0000","last_change":"2024-09-13T08:49:28.310728+0000","last_active":"2024-09-13T08:49:28.311952+0000","last_peered":"2024-09-13T08:49:28.311952+0000","last_clean":"2024-09-13T08:49:28.311952+0000","last_became_active":"2024-09-13T08:49:28.309830+0000","last_became_peered":"2024-09-13T08:49:28.309830+0000","last_unstale":"2024-09-13T08:49:28.311952+0000","last_undegraded":"2024-09-13T08:49:28.311952+0000","last_fullsized":"2024-09-13T08:49:28.311952+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:55.183512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309202+0000","last_change":"2024-09-13T08:49:27.292002+0000","last_active":"2024-09-13T08:49:29.309202+0000","last_peered":"2024-09-13T08:49:29.309202+0000","last_clean":"2024-09-13T08:49:29.309202+0000","last_became_active":"2024-09-13T08:49:27.291714+0000","last_became_peered":"2024-09-13T08:49:27.291714+0000","last_unstale":"2024-09-13T08:49:29.309202+0000","last_undegraded":"2024-09-13T08:49:29.309202+0000","last_fullsized":"2024-09-13T08:49:29.309202+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:54:52.903433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309556+0000","last_change":"2024-09-13T08:49:28.315033+0000","last_active":"2024-09-13T08:49:29.309556+0000","last_peered":"2024-09-13T08:49:29.309556+0000","last_clean":"2024-09-13T08:49:29.309556+0000","last_became_active":"2024-09-13T08:49:28.313861+0000","last_became_peered":"2024-09-13T08:49:28.313861+0000","last_unstale":"2024-09-13T08:49:29.309556+0000","last_undegraded":"2024-09-13T08:49:29.309556+0000","last_fullsized":"2024-09-13T08:49:29.309556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:16.225504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.321750+0000","last_change":"2024-09-13T08:49:27.299657+0000","last_active":"2024-09-13T08:49:28.321750+0000","last_peered":"2024-09-13T08:49:28.321750+0000","last_clean":"2024-09-13T08:49:28.321750+0000","last_became_active":"2024-09-13T08:49:27.299236+0000","last_became_peered":"2024-09-13T08:49:27.299236+0000","last_unstale":"2024-09-13T08:49:28.321750+0000","last_undegraded":"2024-09-13T08:49:28.321750+0000","last_fullsized":"2024-09-13T08:49:28.321750+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:29.567316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":264,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.285941+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:49:28.285941+0000","last_peered":"2024-09-13T08:49:28.285941+0000","last_clean":"2024-09-13T08:49:28.285941+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:49:28.285941+0000","last_undegraded":"2024-09-13T08:49:28.285941+0000","last_fullsized":"2024-09-13T08:49:28.285941+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509587,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80924,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290916,"statfs":{"total":96636764160,"available":96553897984,"internally_reserved":0,"allocated":1245184,"data_stored":539587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":81590730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509587,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123188,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248652,"statfs":{"total":96636764160,"available":96510619648,"internally_reserved":0,"allocated":1867776,"data_stored":1137425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":124254662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509588,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122580,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249260,"statfs":{"total":96636764160,"available":96511242240,"internally_reserved":0,"allocated":1245184,"data_stored":539587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122576,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249264,"statfs":{"total":96636764160,"available":96511246336,"internally_reserved":0,"allocated":1241088,"data_stored":539005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509587,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118424,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253416,"statfs":{"total":96636764160,"available":96515497984,"internally_reserved":0,"allocated":1245184,"data_stored":539075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509589,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64880,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306960,"statfs":{"total":96636764160,"available":96570327040,"internally_reserved":0,"allocated":1855488,"data_stored":1131097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509587,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82324,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289516,"statfs":{"total":96636764160,"available":96552464384,"internally_reserved":0,"allocated":1241088,"data_stored":539005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542293,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144792,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227048,"statfs":{"total":96636764160,"available":96488497152,"internally_reserved":0,"allocated":1241088,"data_stored":539005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:49:32.453 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:49:32.628 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:49:32.631 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:49:32.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.635+0000 7f7b7d600640 1 -- 172.21.15.44:0/2588977376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b78071940 msgr2=0x7f7b780d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.635+0000 7f7b7d600640 1 --2- 172.21.15.44:0/2588977376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b78071940 0x7f7b780d0170 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6401c7c0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.635+0000 7f7b7d600640 1 -- 172.21.15.44:0/2588977376 shutdown_connections 2024-09-13T08:49:32.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.635+0000 7f7b7d600640 1 --2- 172.21.15.44:0/2588977376 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b780b42d0 0x7f7b780af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.635+0000 7f7b7d600640 1 --2- 172.21.15.44:0/2588977376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b78071940 0x7f7b780d0170 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.635+0000 7f7b7d600640 1 -- 172.21.15.44:0/2588977376 >> 172.21.15.44:0/2588977376 conn(0x7f7b780b37b0 msgr2=0x7f7b780b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:32.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.635+0000 7f7b7d600640 1 -- 172.21.15.44:0/2588977376 shutdown_connections 2024-09-13T08:49:32.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.635+0000 7f7b7d600640 1 -- 172.21.15.44:0/2588977376 wait complete. 2024-09-13T08:49:32.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.635+0000 7f7b7d600640 1 Processor -- start 2024-09-13T08:49:32.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.635+0000 7f7b7d600640 1 -- start start 2024-09-13T08:49:32.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7d600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b78071940 0x7f7b780c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:32.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7d600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b780b42d0 0x7f7b7815cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:32.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b78160310 0x7f7b780c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:32.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7b780b0b60 con 0x7f7b78160310 2024-09-13T08:49:32.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7d600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b780b09e0 con 0x7f7b780b42d0 2024-09-13T08:49:32.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7d600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7b78168c50 con 0x7f7b78071940 2024-09-13T08:49:32.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b78160310 0x7f7b780c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:32.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b78160310 0x7f7b780c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42526/0 (socket says 172.21.15.44:42526) 2024-09-13T08:49:32.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7cc00640 1 -- 172.21.15.44:0/295329415 learned_addr learned my addr 172.21.15.44:0/295329415 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b77e00640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b78071940 0x7f7b780c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b77400640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b780b42d0 0x7f7b7815cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7cc00640 1 -- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b78071940 msgr2=0x7f7b780c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7cc00640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b78071940 0x7f7b780c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7cc00640 1 -- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b780b42d0 msgr2=0x7f7b7815cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7cc00640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b780b42d0 0x7f7b7815cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7cc00640 1 -- 172.21.15.44:0/295329415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7815d140 con 0x7f7b78160310 2024-09-13T08:49:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7cc00640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b78160310 0x7f7b780c7960 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000cce0 tx=0x7f7b7000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:32.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b6fe00640 1 -- 172.21.15.44:0/295329415 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000f450 con 0x7f7b78160310 2024-09-13T08:49:32.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b6fe00640 1 -- 172.21.15.44:0/295329415 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b7000f6e0 con 0x7f7b78160310 2024-09-13T08:49:32.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7d600640 1 -- 172.21.15.44:0/295329415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7815d3e0 con 0x7f7b78160310 2024-09-13T08:49:32.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b6fe00640 1 -- 172.21.15.44:0/295329415 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000fb70 con 0x7f7b78160310 2024-09-13T08:49:32.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7d600640 1 -- 172.21.15.44:0/295329415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7815d9a0 con 0x7f7b78160310 2024-09-13T08:49:32.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.639+0000 7f7b7d600640 1 -- 172.21.15.44:0/295329415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b780c5550 con 0x7f7b78160310 2024-09-13T08:49:32.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.643+0000 7f7b6fe00640 1 -- 172.21.15.44:0/295329415 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f7b7000fd10 con 0x7f7b78160310 2024-09-13T08:49:32.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.643+0000 7f7b6fe00640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b58074540 0x7f7b58076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:32.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.643+0000 7f7b6fe00640 1 -- 172.21.15.44:0/295329415 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f7b700934e0 con 0x7f7b78160310 2024-09-13T08:49:32.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.643+0000 7f7b77e00640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b58074540 0x7f7b58076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:32.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.647+0000 7f7b77e00640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b58074540 0x7f7b58076a00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7b780b3dd0 tx=0x7f7b68009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:32.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.647+0000 7f7b6fe00640 1 -- 172.21.15.44:0/295329415 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7b70014030 con 0x7f7b78160310 2024-09-13T08:49:32.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.823+0000 7f7b7d600640 1 -- 172.21.15.44:0/295329415 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b780b7c00 con 0x7f7b58074540 2024-09-13T08:49:32.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.831+0000 7f7b6fe00640 1 -- 172.21.15.44:0/295329415 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252421 (secure 0 0 0) 0x7f7b780b7c00 con 0x7f7b58074540 2024-09-13T08:49:32.834 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:32.836 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:49:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 -- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b58074540 msgr2=0x7f7b58076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b58074540 0x7f7b58076a00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7b780b3dd0 tx=0x7f7b68009060 comp rx=0 tx=0).stop 2024-09-13T08:49:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 -- 172.21.15.44:0/295329415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b78160310 msgr2=0x7f7b780c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b78160310 0x7f7b780c7960 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000cce0 tx=0x7f7b7000a620 comp rx=0 tx=0).stop 2024-09-13T08:49:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 -- 172.21.15.44:0/295329415 shutdown_connections 2024-09-13T08:49:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7b58074540 0x7f7b58076a00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b78160310 0x7f7b780c7960 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b780b42d0 0x7f7b7815cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 --2- 172.21.15.44:0/295329415 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b78071940 0x7f7b780c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:32.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 -- 172.21.15.44:0/295329415 >> 172.21.15.44:0/295329415 conn(0x7f7b780b37b0 msgr2=0x7f7b7815af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:32.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 -- 172.21.15.44:0/295329415 shutdown_connections 2024-09-13T08:49:32.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:32.835+0000 7f7b7d600640 1 -- 172.21.15.44:0/295329415 wait complete. 2024-09-13T08:49:32.856 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-13T08:49:32.774636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":859688,"kb_used_data":10920,"kb_used_omap":12,"kb_used_meta":848563,"kb_avail":754115032,"statfs":{"total":773094113280,"available":772213792768,"internally_reserved":0,"allocated":11182080,"data_stored":5503786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":868929073},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001600"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309764+0000","last_change":"2024-09-13T08:49:28.308301+0000","last_active":"2024-09-13T08:49:28.309764+0000","last_peered":"2024-09-13T08:49:28.309764+0000","last_clean":"2024-09-13T08:49:28.309764+0000","last_became_active":"2024-09-13T08:49:28.307210+0000","last_became_peered":"2024-09-13T08:49:28.307210+0000","last_unstale":"2024-09-13T08:49:28.309764+0000","last_undegraded":"2024-09-13T08:49:28.309764+0000","last_fullsized":"2024-09-13T08:49:28.309764+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:54.439629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318313+0000","last_change":"2024-09-13T08:49:28.311455+0000","last_active":"2024-09-13T08:49:28.318313+0000","last_peered":"2024-09-13T08:49:28.318313+0000","last_clean":"2024-09-13T08:49:28.318313+0000","last_became_active":"2024-09-13T08:49:28.310745+0000","last_became_peered":"2024-09-13T08:49:28.310745+0000","last_unstale":"2024-09-13T08:49:28.318313+0000","last_undegraded":"2024-09-13T08:49:28.318313+0000","last_fullsized":"2024-09-13T08:49:28.318313+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:09.660148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309299+0000","last_change":"2024-09-13T08:49:28.298501+0000","last_active":"2024-09-13T08:49:29.309299+0000","last_peered":"2024-09-13T08:49:29.309299+0000","last_clean":"2024-09-13T08:49:29.309299+0000","last_became_active":"2024-09-13T08:49:28.298245+0000","last_became_peered":"2024-09-13T08:49:28.298245+0000","last_unstale":"2024-09-13T08:49:29.309299+0000","last_undegraded":"2024-09-13T08:49:29.309299+0000","last_fullsized":"2024-09-13T08:49:29.309299+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:01.526002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.305420+0000","last_change":"2024-09-13T08:49:28.304773+0000","last_active":"2024-09-13T08:49:28.305420+0000","last_peered":"2024-09-13T08:49:28.305420+0000","last_clean":"2024-09-13T08:49:28.305420+0000","last_became_active":"2024-09-13T08:49:28.304456+0000","last_became_peered":"2024-09-13T08:49:28.304456+0000","last_unstale":"2024-09-13T08:49:28.305420+0000","last_undegraded":"2024-09-13T08:49:28.305420+0000","last_fullsized":"2024-09-13T08:49:28.305420+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:13.924359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.302091+0000","last_change":"2024-09-13T08:49:28.298893+0000","last_active":"2024-09-13T08:49:28.302091+0000","last_peered":"2024-09-13T08:49:28.302091+0000","last_clean":"2024-09-13T08:49:28.302091+0000","last_became_active":"2024-09-13T08:49:28.298620+0000","last_became_peered":"2024-09-13T08:49:28.298620+0000","last_unstale":"2024-09-13T08:49:28.302091+0000","last_undegraded":"2024-09-13T08:49:28.302091+0000","last_fullsized":"2024-09-13T08:49:28.302091+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:34.628036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.300167+0000","last_change":"2024-09-13T08:49:28.295832+0000","last_active":"2024-09-13T08:49:28.300167+0000","last_peered":"2024-09-13T08:49:28.300167+0000","last_clean":"2024-09-13T08:49:28.300167+0000","last_became_active":"2024-09-13T08:49:28.294324+0000","last_became_peered":"2024-09-13T08:49:28.294324+0000","last_unstale":"2024-09-13T08:49:28.300167+0000","last_undegraded":"2024-09-13T08:49:28.300167+0000","last_fullsized":"2024-09-13T08:49:28.300167+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:33.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.303316+0000","last_change":"2024-09-13T08:49:28.301121+0000","last_active":"2024-09-13T08:49:28.303316+0000","last_peered":"2024-09-13T08:49:28.303316+0000","last_clean":"2024-09-13T08:49:28.303316+0000","last_became_active":"2024-09-13T08:49:28.300815+0000","last_became_peered":"2024-09-13T08:49:28.300815+0000","last_unstale":"2024-09-13T08:49:28.303316+0000","last_undegraded":"2024-09-13T08:49:28.303316+0000","last_fullsized":"2024-09-13T08:49:28.303316+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:04:59.423182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309053+0000","last_change":"2024-09-13T08:49:28.293945+0000","last_active":"2024-09-13T08:49:29.309053+0000","last_peered":"2024-09-13T08:49:29.309053+0000","last_clean":"2024-09-13T08:49:29.309053+0000","last_became_active":"2024-09-13T08:49:28.293673+0000","last_became_peered":"2024-09-13T08:49:28.293673+0000","last_unstale":"2024-09-13T08:49:29.309053+0000","last_undegraded":"2024-09-13T08:49:29.309053+0000","last_fullsized":"2024-09-13T08:49:29.309053+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:54.895240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.312361+0000","last_change":"2024-09-13T08:49:28.309249+0000","last_active":"2024-09-13T08:49:28.312361+0000","last_peered":"2024-09-13T08:49:28.312361+0000","last_clean":"2024-09-13T08:49:28.312361+0000","last_became_active":"2024-09-13T08:49:28.308969+0000","last_became_peered":"2024-09-13T08:49:28.308969+0000","last_unstale":"2024-09-13T08:49:28.312361+0000","last_undegraded":"2024-09-13T08:49:28.312361+0000","last_fullsized":"2024-09-13T08:49:28.312361+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.463148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.312463+0000","last_change":"2024-09-13T08:49:28.310940+0000","last_active":"2024-09-13T08:49:28.312463+0000","last_peered":"2024-09-13T08:49:28.312463+0000","last_clean":"2024-09-13T08:49:28.312463+0000","last_became_active":"2024-09-13T08:49:28.310689+0000","last_became_peered":"2024-09-13T08:49:28.310689+0000","last_unstale":"2024-09-13T08:49:28.312463+0000","last_undegraded":"2024-09-13T08:49:28.312463+0000","last_fullsized":"2024-09-13T08:49:28.312463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:29.813282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.303111+0000","last_change":"2024-09-13T08:49:28.300689+0000","last_active":"2024-09-13T08:49:28.303111+0000","last_peered":"2024-09-13T08:49:28.303111+0000","last_clean":"2024-09-13T08:49:28.303111+0000","last_became_active":"2024-09-13T08:49:28.300327+0000","last_became_peered":"2024-09-13T08:49:28.300327+0000","last_unstale":"2024-09-13T08:49:28.303111+0000","last_undegraded":"2024-09-13T08:49:28.303111+0000","last_fullsized":"2024-09-13T08:49:28.303111+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:50:03.833588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.310517+0000","last_change":"2024-09-13T08:49:28.309720+0000","last_active":"2024-09-13T08:49:28.310517+0000","last_peered":"2024-09-13T08:49:28.310517+0000","last_clean":"2024-09-13T08:49:28.310517+0000","last_became_active":"2024-09-13T08:49:28.309258+0000","last_became_peered":"2024-09-13T08:49:28.309258+0000","last_unstale":"2024-09-13T08:49:28.310517+0000","last_undegraded":"2024-09-13T08:49:28.310517+0000","last_fullsized":"2024-09-13T08:49:28.310517+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:46.425416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318285+0000","last_change":"2024-09-13T08:49:28.317239+0000","last_active":"2024-09-13T08:49:28.318285+0000","last_peered":"2024-09-13T08:49:28.318285+0000","last_clean":"2024-09-13T08:49:28.318285+0000","last_became_active":"2024-09-13T08:49:28.316923+0000","last_became_peered":"2024-09-13T08:49:28.316923+0000","last_unstale":"2024-09-13T08:49:28.318285+0000","last_undegraded":"2024-09-13T08:49:28.318285+0000","last_fullsized":"2024-09-13T08:49:28.318285+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:14.508154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.319762+0000","last_change":"2024-09-13T08:49:28.312290+0000","last_active":"2024-09-13T08:49:28.319762+0000","last_peered":"2024-09-13T08:49:28.319762+0000","last_clean":"2024-09-13T08:49:28.319762+0000","last_became_active":"2024-09-13T08:49:28.311643+0000","last_became_peered":"2024-09-13T08:49:28.311643+0000","last_unstale":"2024-09-13T08:49:28.319762+0000","last_undegraded":"2024-09-13T08:49:28.319762+0000","last_fullsized":"2024-09-13T08:49:28.319762+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:13.418708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309619+0000","last_change":"2024-09-13T08:49:28.307566+0000","last_active":"2024-09-13T08:49:28.309619+0000","last_peered":"2024-09-13T08:49:28.309619+0000","last_clean":"2024-09-13T08:49:28.309619+0000","last_became_active":"2024-09-13T08:49:28.306626+0000","last_became_peered":"2024-09-13T08:49:28.306626+0000","last_unstale":"2024-09-13T08:49:28.309619+0000","last_undegraded":"2024-09-13T08:49:28.309619+0000","last_fullsized":"2024-09-13T08:49:28.309619+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:53.143087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.302430+0000","last_change":"2024-09-13T08:49:28.298983+0000","last_active":"2024-09-13T08:49:28.302430+0000","last_peered":"2024-09-13T08:49:28.302430+0000","last_clean":"2024-09-13T08:49:28.302430+0000","last_became_active":"2024-09-13T08:49:28.293845+0000","last_became_peered":"2024-09-13T08:49:28.293845+0000","last_unstale":"2024-09-13T08:49:28.302430+0000","last_undegraded":"2024-09-13T08:49:28.302430+0000","last_fullsized":"2024-09-13T08:49:28.302430+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:42.579804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287971+0000","last_change":"2024-09-13T08:49:28.307169+0000","last_active":"2024-09-13T08:49:29.287971+0000","last_peered":"2024-09-13T08:49:29.287971+0000","last_clean":"2024-09-13T08:49:29.287971+0000","last_became_active":"2024-09-13T08:49:28.306905+0000","last_became_peered":"2024-09-13T08:49:28.306905+0000","last_unstale":"2024-09-13T08:49:29.287971+0000","last_undegraded":"2024-09-13T08:49:29.287971+0000","last_fullsized":"2024-09-13T08:49:29.287971+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:34:43.033131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318263+0000","last_change":"2024-09-13T08:49:28.311396+0000","last_active":"2024-09-13T08:49:28.318263+0000","last_peered":"2024-09-13T08:49:28.318263+0000","last_clean":"2024-09-13T08:49:28.318263+0000","last_became_active":"2024-09-13T08:49:28.310454+0000","last_became_peered":"2024-09-13T08:49:28.310454+0000","last_unstale":"2024-09-13T08:49:28.318263+0000","last_undegraded":"2024-09-13T08:49:28.318263+0000","last_fullsized":"2024-09-13T08:49:28.318263+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:21.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287539+0000","last_change":"2024-09-13T08:49:28.313300+0000","last_active":"2024-09-13T08:49:29.287539+0000","last_peered":"2024-09-13T08:49:29.287539+0000","last_clean":"2024-09-13T08:49:29.287539+0000","last_became_active":"2024-09-13T08:49:28.313046+0000","last_became_peered":"2024-09-13T08:49:28.313046+0000","last_unstale":"2024-09-13T08:49:29.287539+0000","last_undegraded":"2024-09-13T08:49:29.287539+0000","last_fullsized":"2024-09-13T08:49:29.287539+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:10.595581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309877+0000","last_change":"2024-09-13T08:49:28.307967+0000","last_active":"2024-09-13T08:49:28.309877+0000","last_peered":"2024-09-13T08:49:28.309877+0000","last_clean":"2024-09-13T08:49:28.309877+0000","last_became_active":"2024-09-13T08:49:28.307639+0000","last_became_peered":"2024-09-13T08:49:28.307639+0000","last_unstale":"2024-09-13T08:49:28.309877+0000","last_undegraded":"2024-09-13T08:49:28.309877+0000","last_fullsized":"2024-09-13T08:49:28.309877+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:27:31.001532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309665+0000","last_change":"2024-09-13T08:49:28.308961+0000","last_active":"2024-09-13T08:49:29.309665+0000","last_peered":"2024-09-13T08:49:29.309665+0000","last_clean":"2024-09-13T08:49:29.309665+0000","last_became_active":"2024-09-13T08:49:28.307859+0000","last_became_peered":"2024-09-13T08:49:28.307859+0000","last_unstale":"2024-09-13T08:49:29.309665+0000","last_undegraded":"2024-09-13T08:49:29.309665+0000","last_fullsized":"2024-09-13T08:49:29.309665+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:18.993392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309166+0000","last_change":"2024-09-13T08:49:28.304589+0000","last_active":"2024-09-13T08:49:29.309166+0000","last_peered":"2024-09-13T08:49:29.309166+0000","last_clean":"2024-09-13T08:49:29.309166+0000","last_became_active":"2024-09-13T08:49:28.304315+0000","last_became_peered":"2024-09-13T08:49:28.304315+0000","last_unstale":"2024-09-13T08:49:29.309166+0000","last_undegraded":"2024-09-13T08:49:29.309166+0000","last_fullsized":"2024-09-13T08:49:29.309166+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:23:39.301642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.319321+0000","last_change":"2024-09-13T08:49:28.311959+0000","last_active":"2024-09-13T08:49:28.319321+0000","last_peered":"2024-09-13T08:49:28.319321+0000","last_clean":"2024-09-13T08:49:28.319321+0000","last_became_active":"2024-09-13T08:49:28.311608+0000","last_became_peered":"2024-09-13T08:49:28.311608+0000","last_unstale":"2024-09-13T08:49:28.319321+0000","last_undegraded":"2024-09-13T08:49:28.319321+0000","last_fullsized":"2024-09-13T08:49:28.319321+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:30:49.273165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.307904+0000","last_change":"2024-09-13T08:49:28.302145+0000","last_active":"2024-09-13T08:49:28.307904+0000","last_peered":"2024-09-13T08:49:28.307904+0000","last_clean":"2024-09-13T08:49:28.307904+0000","last_became_active":"2024-09-13T08:49:28.301663+0000","last_became_peered":"2024-09-13T08:49:28.301663+0000","last_unstale":"2024-09-13T08:49:28.307904+0000","last_undegraded":"2024-09-13T08:49:28.307904+0000","last_fullsized":"2024-09-13T08:49:28.307904+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:48.882320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.300170+0000","last_change":"2024-09-13T08:49:28.295011+0000","last_active":"2024-09-13T08:49:28.300170+0000","last_peered":"2024-09-13T08:49:28.300170+0000","last_clean":"2024-09-13T08:49:28.300170+0000","last_became_active":"2024-09-13T08:49:28.294708+0000","last_became_peered":"2024-09-13T08:49:28.294708+0000","last_unstale":"2024-09-13T08:49:28.300170+0000","last_undegraded":"2024-09-13T08:49:28.300170+0000","last_fullsized":"2024-09-13T08:49:28.300170+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:18.993392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.311236+0000","last_change":"2024-09-13T08:49:28.307615+0000","last_active":"2024-09-13T08:49:28.311236+0000","last_peered":"2024-09-13T08:49:28.311236+0000","last_clean":"2024-09-13T08:49:28.311236+0000","last_became_active":"2024-09-13T08:49:28.306711+0000","last_became_peered":"2024-09-13T08:49:28.306711+0000","last_unstale":"2024-09-13T08:49:28.311236+0000","last_undegraded":"2024-09-13T08:49:28.311236+0000","last_fullsized":"2024-09-13T08:49:28.311236+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:31.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.311943+0000","last_change":"2024-09-13T08:49:28.310812+0000","last_active":"2024-09-13T08:49:28.311943+0000","last_peered":"2024-09-13T08:49:28.311943+0000","last_clean":"2024-09-13T08:49:28.311943+0000","last_became_active":"2024-09-13T08:49:28.310384+0000","last_became_peered":"2024-09-13T08:49:28.310384+0000","last_unstale":"2024-09-13T08:49:28.311943+0000","last_undegraded":"2024-09-13T08:49:28.311943+0000","last_fullsized":"2024-09-13T08:49:28.311943+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:17.492931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.296410+0000","last_change":"2024-09-13T08:49:28.290617+0000","last_active":"2024-09-13T08:49:28.296410+0000","last_peered":"2024-09-13T08:49:28.296410+0000","last_clean":"2024-09-13T08:49:28.296410+0000","last_became_active":"2024-09-13T08:49:28.290356+0000","last_became_peered":"2024-09-13T08:49:28.290356+0000","last_unstale":"2024-09-13T08:49:28.296410+0000","last_undegraded":"2024-09-13T08:49:28.296410+0000","last_fullsized":"2024-09-13T08:49:28.296410+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:13:44.081989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.311212+0000","last_change":"2024-09-13T08:49:28.307187+0000","last_active":"2024-09-13T08:49:28.311212+0000","last_peered":"2024-09-13T08:49:28.311212+0000","last_clean":"2024-09-13T08:49:28.311212+0000","last_became_active":"2024-09-13T08:49:28.306919+0000","last_became_peered":"2024-09-13T08:49:28.306919+0000","last_unstale":"2024-09-13T08:49:28.311212+0000","last_undegraded":"2024-09-13T08:49:28.311212+0000","last_fullsized":"2024-09-13T08:49:28.311212+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:16:01.168882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.314869+0000","last_change":"2024-09-13T08:49:28.307751+0000","last_active":"2024-09-13T08:49:28.314869+0000","last_peered":"2024-09-13T08:49:28.314869+0000","last_clean":"2024-09-13T08:49:28.314869+0000","last_became_active":"2024-09-13T08:49:28.306690+0000","last_became_peered":"2024-09-13T08:49:28.306690+0000","last_unstale":"2024-09-13T08:49:28.314869+0000","last_undegraded":"2024-09-13T08:49:28.314869+0000","last_fullsized":"2024-09-13T08:49:28.314869+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:01:22.163556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309395+0000","last_change":"2024-09-13T08:49:28.292684+0000","last_active":"2024-09-13T08:49:29.309395+0000","last_peered":"2024-09-13T08:49:29.309395+0000","last_clean":"2024-09-13T08:49:29.309395+0000","last_became_active":"2024-09-13T08:49:28.292370+0000","last_became_peered":"2024-09-13T08:49:28.292370+0000","last_unstale":"2024-09-13T08:49:29.309395+0000","last_undegraded":"2024-09-13T08:49:29.309395+0000","last_fullsized":"2024-09-13T08:49:29.309395+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:39.842892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.304073+0000","last_change":"2024-09-13T08:49:28.301383+0000","last_active":"2024-09-13T08:49:28.304073+0000","last_peered":"2024-09-13T08:49:28.304073+0000","last_clean":"2024-09-13T08:49:28.304073+0000","last_became_active":"2024-09-13T08:49:28.300938+0000","last_became_peered":"2024-09-13T08:49:28.300938+0000","last_unstale":"2024-09-13T08:49:28.304073+0000","last_undegraded":"2024-09-13T08:49:28.304073+0000","last_fullsized":"2024-09-13T08:49:28.304073+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:21:11.937699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287857+0000","last_change":"2024-09-13T08:49:28.304025+0000","last_active":"2024-09-13T08:49:29.287857+0000","last_peered":"2024-09-13T08:49:29.287857+0000","last_clean":"2024-09-13T08:49:29.287857+0000","last_became_active":"2024-09-13T08:49:28.303484+0000","last_became_peered":"2024-09-13T08:49:28.303484+0000","last_unstale":"2024-09-13T08:49:29.287857+0000","last_undegraded":"2024-09-13T08:49:29.287857+0000","last_fullsized":"2024-09-13T08:49:29.287857+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:54:14.094928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.321983+0000","last_change":"2024-09-13T08:49:27.304233+0000","last_active":"2024-09-13T08:49:28.321983+0000","last_peered":"2024-09-13T08:49:28.321983+0000","last_clean":"2024-09-13T08:49:28.321983+0000","last_became_active":"2024-09-13T08:49:27.303922+0000","last_became_peered":"2024-09-13T08:49:27.303922+0000","last_unstale":"2024-09-13T08:49:28.321983+0000","last_undegraded":"2024-09-13T08:49:28.321983+0000","last_fullsized":"2024-09-13T08:49:28.321983+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:01.166230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.308586+0000","last_change":"2024-09-13T08:49:28.304603+0000","last_active":"2024-09-13T08:49:28.308586+0000","last_peered":"2024-09-13T08:49:28.308586+0000","last_clean":"2024-09-13T08:49:28.308586+0000","last_became_active":"2024-09-13T08:49:28.304315+0000","last_became_peered":"2024-09-13T08:49:28.304315+0000","last_unstale":"2024-09-13T08:49:28.308586+0000","last_undegraded":"2024-09-13T08:49:28.308586+0000","last_fullsized":"2024-09-13T08:49:28.308586+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:30.501323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.321641+0000","last_change":"2024-09-13T08:49:27.291602+0000","last_active":"2024-09-13T08:49:28.321641+0000","last_peered":"2024-09-13T08:49:28.321641+0000","last_clean":"2024-09-13T08:49:28.321641+0000","last_became_active":"2024-09-13T08:49:27.291157+0000","last_became_peered":"2024-09-13T08:49:27.291157+0000","last_unstale":"2024-09-13T08:49:28.321641+0000","last_undegraded":"2024-09-13T08:49:28.321641+0000","last_fullsized":"2024-09-13T08:49:28.321641+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:06:20.232249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.304245+0000","last_change":"2024-09-13T08:49:28.302518+0000","last_active":"2024-09-13T08:49:28.304245+0000","last_peered":"2024-09-13T08:49:28.304245+0000","last_clean":"2024-09-13T08:49:28.304245+0000","last_became_active":"2024-09-13T08:49:28.302275+0000","last_became_peered":"2024-09-13T08:49:28.302275+0000","last_unstale":"2024-09-13T08:49:28.304245+0000","last_undegraded":"2024-09-13T08:49:28.304245+0000","last_fullsized":"2024-09-13T08:49:28.304245+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:47:33.775174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309323+0000","last_change":"2024-09-13T08:49:27.307806+0000","last_active":"2024-09-13T08:49:29.309323+0000","last_peered":"2024-09-13T08:49:29.309323+0000","last_clean":"2024-09-13T08:49:29.309323+0000","last_became_active":"2024-09-13T08:49:27.307441+0000","last_became_peered":"2024-09-13T08:49:27.307441+0000","last_unstale":"2024-09-13T08:49:29.309323+0000","last_undegraded":"2024-09-13T08:49:29.309323+0000","last_fullsized":"2024-09-13T08:49:29.309323+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:08.620519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.299724+0000","last_change":"2024-09-13T08:49:28.295293+0000","last_active":"2024-09-13T08:49:28.299724+0000","last_peered":"2024-09-13T08:49:28.299724+0000","last_clean":"2024-09-13T08:49:28.299724+0000","last_became_active":"2024-09-13T08:49:28.294933+0000","last_became_peered":"2024-09-13T08:49:28.294933+0000","last_unstale":"2024-09-13T08:49:28.299724+0000","last_undegraded":"2024-09-13T08:49:28.299724+0000","last_fullsized":"2024-09-13T08:49:28.299724+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:50:30.506890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.324102+0000","last_change":"2024-09-13T08:49:27.304809+0000","last_active":"2024-09-13T08:49:28.324102+0000","last_peered":"2024-09-13T08:49:28.324102+0000","last_clean":"2024-09-13T08:49:28.324102+0000","last_became_active":"2024-09-13T08:49:27.304460+0000","last_became_peered":"2024-09-13T08:49:27.304460+0000","last_unstale":"2024-09-13T08:49:28.324102+0000","last_undegraded":"2024-09-13T08:49:28.324102+0000","last_fullsized":"2024-09-13T08:49:28.324102+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:01:56.309772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.304967+0000","last_change":"2024-09-13T08:49:28.303525+0000","last_active":"2024-09-13T08:49:28.304967+0000","last_peered":"2024-09-13T08:49:28.304967+0000","last_clean":"2024-09-13T08:49:28.304967+0000","last_became_active":"2024-09-13T08:49:28.303276+0000","last_became_peered":"2024-09-13T08:49:28.303276+0000","last_unstale":"2024-09-13T08:49:28.304967+0000","last_undegraded":"2024-09-13T08:49:28.304967+0000","last_fullsized":"2024-09-13T08:49:28.304967+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:09:11.535610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.323437+0000","last_change":"2024-09-13T08:49:27.306908+0000","last_active":"2024-09-13T08:49:28.323437+0000","last_peered":"2024-09-13T08:49:28.323437+0000","last_clean":"2024-09-13T08:49:28.323437+0000","last_became_active":"2024-09-13T08:49:27.306187+0000","last_became_peered":"2024-09-13T08:49:27.306187+0000","last_unstale":"2024-09-13T08:49:28.323437+0000","last_undegraded":"2024-09-13T08:49:28.323437+0000","last_fullsized":"2024-09-13T08:49:28.323437+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:44.144108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.303251+0000","last_change":"2024-09-13T08:49:28.300754+0000","last_active":"2024-09-13T08:49:28.303251+0000","last_peered":"2024-09-13T08:49:28.303251+0000","last_clean":"2024-09-13T08:49:28.303251+0000","last_became_active":"2024-09-13T08:49:28.300409+0000","last_became_peered":"2024-09-13T08:49:28.300409+0000","last_unstale":"2024-09-13T08:49:28.303251+0000","last_undegraded":"2024-09-13T08:49:28.303251+0000","last_fullsized":"2024-09-13T08:49:28.303251+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:42:12.765747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309086+0000","last_change":"2024-09-13T08:49:27.300301+0000","last_active":"2024-09-13T08:49:29.309086+0000","last_peered":"2024-09-13T08:49:29.309086+0000","last_clean":"2024-09-13T08:49:29.309086+0000","last_became_active":"2024-09-13T08:49:27.299991+0000","last_became_peered":"2024-09-13T08:49:27.299991+0000","last_unstale":"2024-09-13T08:49:29.309086+0000","last_undegraded":"2024-09-13T08:49:29.309086+0000","last_fullsized":"2024-09-13T08:49:29.309086+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:48:27.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.315807+0000","last_change":"2024-09-13T08:49:28.310176+0000","last_active":"2024-09-13T08:49:28.315807+0000","last_peered":"2024-09-13T08:49:28.315807+0000","last_clean":"2024-09-13T08:49:28.315807+0000","last_became_active":"2024-09-13T08:49:28.309838+0000","last_became_peered":"2024-09-13T08:49:28.309838+0000","last_unstale":"2024-09-13T08:49:28.315807+0000","last_undegraded":"2024-09-13T08:49:28.315807+0000","last_fullsized":"2024-09-13T08:49:28.315807+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:30.570457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.288333+0000","last_change":"2024-09-13T08:49:27.297539+0000","last_active":"2024-09-13T08:49:28.288333+0000","last_peered":"2024-09-13T08:49:28.288333+0000","last_clean":"2024-09-13T08:49:28.288333+0000","last_became_active":"2024-09-13T08:49:27.297241+0000","last_became_peered":"2024-09-13T08:49:27.297241+0000","last_unstale":"2024-09-13T08:49:28.288333+0000","last_undegraded":"2024-09-13T08:49:28.288333+0000","last_fullsized":"2024-09-13T08:49:28.288333+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:55.812887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.301018+0000","last_change":"2024-09-13T08:49:28.295461+0000","last_active":"2024-09-13T08:49:28.301018+0000","last_peered":"2024-09-13T08:49:28.301018+0000","last_clean":"2024-09-13T08:49:28.301018+0000","last_became_active":"2024-09-13T08:49:28.295200+0000","last_became_peered":"2024-09-13T08:49:28.295200+0000","last_unstale":"2024-09-13T08:49:28.301018+0000","last_undegraded":"2024-09-13T08:49:28.301018+0000","last_fullsized":"2024-09-13T08:49:28.301018+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:05:37.274424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.286333+0000","last_change":"2024-09-13T08:49:27.305047+0000","last_active":"2024-09-13T08:49:29.286333+0000","last_peered":"2024-09-13T08:49:29.286333+0000","last_clean":"2024-09-13T08:49:29.286333+0000","last_became_active":"2024-09-13T08:49:27.304783+0000","last_became_peered":"2024-09-13T08:49:27.304783+0000","last_unstale":"2024-09-13T08:49:29.286333+0000","last_undegraded":"2024-09-13T08:49:29.286333+0000","last_fullsized":"2024-09-13T08:49:29.286333+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:47.228172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318236+0000","last_change":"2024-09-13T08:49:28.311153+0000","last_active":"2024-09-13T08:49:28.318236+0000","last_peered":"2024-09-13T08:49:28.318236+0000","last_clean":"2024-09-13T08:49:28.318236+0000","last_became_active":"2024-09-13T08:49:28.310070+0000","last_became_peered":"2024-09-13T08:49:28.310070+0000","last_unstale":"2024-09-13T08:49:28.318236+0000","last_undegraded":"2024-09-13T08:49:28.318236+0000","last_fullsized":"2024-09-13T08:49:28.318236+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:58.780865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309483+0000","last_change":"2024-09-13T08:49:27.301210+0000","last_active":"2024-09-13T08:49:29.309483+0000","last_peered":"2024-09-13T08:49:29.309483+0000","last_clean":"2024-09-13T08:49:29.309483+0000","last_became_active":"2024-09-13T08:49:27.297621+0000","last_became_peered":"2024-09-13T08:49:27.297621+0000","last_unstale":"2024-09-13T08:49:29.309483+0000","last_undegraded":"2024-09-13T08:49:29.309483+0000","last_fullsized":"2024-09-13T08:49:29.309483+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:21.001638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.299245+0000","last_change":"2024-09-13T08:49:28.295106+0000","last_active":"2024-09-13T08:49:28.299245+0000","last_peered":"2024-09-13T08:49:28.299245+0000","last_clean":"2024-09-13T08:49:28.299245+0000","last_became_active":"2024-09-13T08:49:28.291915+0000","last_became_peered":"2024-09-13T08:49:28.291915+0000","last_unstale":"2024-09-13T08:49:28.299245+0000","last_undegraded":"2024-09-13T08:49:28.299245+0000","last_fullsized":"2024-09-13T08:49:28.299245+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:49.365537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.308893+0000","last_change":"2024-09-13T08:49:27.308011+0000","last_active":"2024-09-13T08:49:28.308893+0000","last_peered":"2024-09-13T08:49:28.308893+0000","last_clean":"2024-09-13T08:49:28.308893+0000","last_became_active":"2024-09-13T08:49:27.307735+0000","last_became_peered":"2024-09-13T08:49:27.307735+0000","last_unstale":"2024-09-13T08:49:28.308893+0000","last_undegraded":"2024-09-13T08:49:28.308893+0000","last_fullsized":"2024-09-13T08:49:28.308893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:49:16.620549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.307447+0000","last_change":"2024-09-13T08:49:28.306278+0000","last_active":"2024-09-13T08:49:28.307447+0000","last_peered":"2024-09-13T08:49:28.307447+0000","last_clean":"2024-09-13T08:49:28.307447+0000","last_became_active":"2024-09-13T08:49:28.305531+0000","last_became_peered":"2024-09-13T08:49:28.305531+0000","last_unstale":"2024-09-13T08:49:28.307447+0000","last_undegraded":"2024-09-13T08:49:28.307447+0000","last_fullsized":"2024-09-13T08:49:28.307447+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:54:52.031333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318562+0000","last_change":"2024-09-13T08:49:27.297795+0000","last_active":"2024-09-13T08:49:28.318562+0000","last_peered":"2024-09-13T08:49:28.318562+0000","last_clean":"2024-09-13T08:49:28.318562+0000","last_became_active":"2024-09-13T08:49:27.297504+0000","last_became_peered":"2024-09-13T08:49:27.297504+0000","last_unstale":"2024-09-13T08:49:28.318562+0000","last_undegraded":"2024-09-13T08:49:28.318562+0000","last_fullsized":"2024-09-13T08:49:28.318562+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:07:27.376375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.300586+0000","last_change":"2024-09-13T08:49:28.295590+0000","last_active":"2024-09-13T08:49:28.300586+0000","last_peered":"2024-09-13T08:49:28.300586+0000","last_clean":"2024-09-13T08:49:28.300586+0000","last_became_active":"2024-09-13T08:49:28.295322+0000","last_became_peered":"2024-09-13T08:49:28.295322+0000","last_unstale":"2024-09-13T08:49:28.300586+0000","last_undegraded":"2024-09-13T08:49:28.300586+0000","last_fullsized":"2024-09-13T08:49:28.300586+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:55.355769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.324258+0000","last_change":"2024-09-13T08:49:27.303146+0000","last_active":"2024-09-13T08:49:28.324258+0000","last_peered":"2024-09-13T08:49:28.324258+0000","last_clean":"2024-09-13T08:49:28.324258+0000","last_became_active":"2024-09-13T08:49:27.302845+0000","last_became_peered":"2024-09-13T08:49:27.302845+0000","last_unstale":"2024-09-13T08:49:28.324258+0000","last_undegraded":"2024-09-13T08:49:28.324258+0000","last_fullsized":"2024-09-13T08:49:28.324258+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:17:00.617948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309672+0000","last_change":"2024-09-13T08:49:28.305629+0000","last_active":"2024-09-13T08:49:28.309672+0000","last_peered":"2024-09-13T08:49:28.309672+0000","last_clean":"2024-09-13T08:49:28.309672+0000","last_became_active":"2024-09-13T08:49:28.305295+0000","last_became_peered":"2024-09-13T08:49:28.305295+0000","last_unstale":"2024-09-13T08:49:28.309672+0000","last_undegraded":"2024-09-13T08:49:28.309672+0000","last_fullsized":"2024-09-13T08:49:28.309672+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:52.786482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287345+0000","last_change":"2024-09-13T08:49:27.301916+0000","last_active":"2024-09-13T08:49:29.287345+0000","last_peered":"2024-09-13T08:49:29.287345+0000","last_clean":"2024-09-13T08:49:29.287345+0000","last_became_active":"2024-09-13T08:49:27.301629+0000","last_became_peered":"2024-09-13T08:49:27.301629+0000","last_unstale":"2024-09-13T08:49:29.287345+0000","last_undegraded":"2024-09-13T08:49:29.287345+0000","last_fullsized":"2024-09-13T08:49:29.287345+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:34.501901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309776+0000","last_change":"2024-09-13T08:49:28.303461+0000","last_active":"2024-09-13T08:49:29.309776+0000","last_peered":"2024-09-13T08:49:29.309776+0000","last_clean":"2024-09-13T08:49:29.309776+0000","last_became_active":"2024-09-13T08:49:28.303125+0000","last_became_peered":"2024-09-13T08:49:28.303125+0000","last_unstale":"2024-09-13T08:49:29.309776+0000","last_undegraded":"2024-09-13T08:49:29.309776+0000","last_fullsized":"2024-09-13T08:49:29.309776+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:50:30.506890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.296613+0000","last_change":"2024-09-13T08:49:27.305735+0000","last_active":"2024-09-13T08:49:28.296613+0000","last_peered":"2024-09-13T08:49:28.296613+0000","last_clean":"2024-09-13T08:49:28.296613+0000","last_became_active":"2024-09-13T08:49:27.304538+0000","last_became_peered":"2024-09-13T08:49:27.304538+0000","last_unstale":"2024-09-13T08:49:28.296613+0000","last_undegraded":"2024-09-13T08:49:28.296613+0000","last_fullsized":"2024-09-13T08:49:28.296613+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:22.837897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.302404+0000","last_change":"2024-09-13T08:49:28.296332+0000","last_active":"2024-09-13T08:49:28.302404+0000","last_peered":"2024-09-13T08:49:28.302404+0000","last_clean":"2024-09-13T08:49:28.302404+0000","last_became_active":"2024-09-13T08:49:28.294152+0000","last_became_peered":"2024-09-13T08:49:28.294152+0000","last_unstale":"2024-09-13T08:49:28.302404+0000","last_undegraded":"2024-09-13T08:49:28.302404+0000","last_fullsized":"2024-09-13T08:49:28.302404+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:54.477778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.285981+0000","last_change":"2024-09-13T08:49:27.295451+0000","last_active":"2024-09-13T08:49:28.285981+0000","last_peered":"2024-09-13T08:49:28.285981+0000","last_clean":"2024-09-13T08:49:28.285981+0000","last_became_active":"2024-09-13T08:49:27.295025+0000","last_became_peered":"2024-09-13T08:49:27.295025+0000","last_unstale":"2024-09-13T08:49:28.285981+0000","last_undegraded":"2024-09-13T08:49:28.285981+0000","last_fullsized":"2024-09-13T08:49:28.285981+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:01:21.160415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309208+0000","last_change":"2024-09-13T08:49:28.302164+0000","last_active":"2024-09-13T08:49:29.309208+0000","last_peered":"2024-09-13T08:49:29.309208+0000","last_clean":"2024-09-13T08:49:29.309208+0000","last_became_active":"2024-09-13T08:49:28.301826+0000","last_became_peered":"2024-09-13T08:49:28.301826+0000","last_unstale":"2024-09-13T08:49:29.309208+0000","last_undegraded":"2024-09-13T08:49:29.309208+0000","last_fullsized":"2024-09-13T08:49:29.309208+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:42:10.631635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":21,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.325278+0000","last_change":"2024-09-13T08:49:27.308147+0000","last_active":"2024-09-13T08:49:28.325278+0000","last_peered":"2024-09-13T08:49:28.325278+0000","last_clean":"2024-09-13T08:49:28.325278+0000","last_became_active":"2024-09-13T08:49:27.306431+0000","last_became_peered":"2024-09-13T08:49:27.306431+0000","last_unstale":"2024-09-13T08:49:28.325278+0000","last_undegraded":"2024-09-13T08:49:28.325278+0000","last_fullsized":"2024-09-13T08:49:28.325278+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:09.200022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.297778+0000","last_change":"2024-09-13T08:49:28.289688+0000","last_active":"2024-09-13T08:49:28.297778+0000","last_peered":"2024-09-13T08:49:28.297778+0000","last_clean":"2024-09-13T08:49:28.297778+0000","last_became_active":"2024-09-13T08:49:28.289376+0000","last_became_peered":"2024-09-13T08:49:28.289376+0000","last_unstale":"2024-09-13T08:49:28.297778+0000","last_undegraded":"2024-09-13T08:49:28.297778+0000","last_fullsized":"2024-09-13T08:49:28.297778+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:15:23.757716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.322059+0000","last_change":"2024-09-13T08:49:27.298388+0000","last_active":"2024-09-13T08:49:28.322059+0000","last_peered":"2024-09-13T08:49:28.322059+0000","last_clean":"2024-09-13T08:49:28.322059+0000","last_became_active":"2024-09-13T08:49:27.298108+0000","last_became_peered":"2024-09-13T08:49:27.298108+0000","last_unstale":"2024-09-13T08:49:28.322059+0000","last_undegraded":"2024-09-13T08:49:28.322059+0000","last_fullsized":"2024-09-13T08:49:28.322059+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:30.882957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.297370+0000","last_change":"2024-09-13T08:49:28.293206+0000","last_active":"2024-09-13T08:49:28.297370+0000","last_peered":"2024-09-13T08:49:28.297370+0000","last_clean":"2024-09-13T08:49:28.297370+0000","last_became_active":"2024-09-13T08:49:28.292934+0000","last_became_peered":"2024-09-13T08:49:28.292934+0000","last_unstale":"2024-09-13T08:49:28.297370+0000","last_undegraded":"2024-09-13T08:49:28.297370+0000","last_fullsized":"2024-09-13T08:49:28.297370+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:15.584522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.317698+0000","last_change":"2024-09-13T08:49:27.294920+0000","last_active":"2024-09-13T08:49:28.317698+0000","last_peered":"2024-09-13T08:49:28.317698+0000","last_clean":"2024-09-13T08:49:28.317698+0000","last_became_active":"2024-09-13T08:49:27.293444+0000","last_became_peered":"2024-09-13T08:49:27.293444+0000","last_unstale":"2024-09-13T08:49:28.317698+0000","last_undegraded":"2024-09-13T08:49:28.317698+0000","last_fullsized":"2024-09-13T08:49:28.317698+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:31:47.656651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309421+0000","last_change":"2024-09-13T08:49:28.308062+0000","last_active":"2024-09-13T08:49:28.309421+0000","last_peered":"2024-09-13T08:49:28.309421+0000","last_clean":"2024-09-13T08:49:28.309421+0000","last_became_active":"2024-09-13T08:49:28.304255+0000","last_became_peered":"2024-09-13T08:49:28.304255+0000","last_unstale":"2024-09-13T08:49:28.309421+0000","last_undegraded":"2024-09-13T08:49:28.309421+0000","last_fullsized":"2024-09-13T08:49:28.309421+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:39.130420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287517+0000","last_change":"2024-09-13T08:49:27.295352+0000","last_active":"2024-09-13T08:49:29.287517+0000","last_peered":"2024-09-13T08:49:29.287517+0000","last_clean":"2024-09-13T08:49:29.287517+0000","last_became_active":"2024-09-13T08:49:27.295072+0000","last_became_peered":"2024-09-13T08:49:27.295072+0000","last_unstale":"2024-09-13T08:49:29.287517+0000","last_undegraded":"2024-09-13T08:49:29.287517+0000","last_fullsized":"2024-09-13T08:49:29.287517+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:16.489790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.286280+0000","last_change":"2024-09-13T08:49:28.308145+0000","last_active":"2024-09-13T08:49:29.286280+0000","last_peered":"2024-09-13T08:49:29.286280+0000","last_clean":"2024-09-13T08:49:29.286280+0000","last_became_active":"2024-09-13T08:49:28.307835+0000","last_became_peered":"2024-09-13T08:49:28.307835+0000","last_unstale":"2024-09-13T08:49:29.286280+0000","last_undegraded":"2024-09-13T08:49:29.286280+0000","last_fullsized":"2024-09-13T08:49:29.286280+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:29:32.858641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.324135+0000","last_change":"2024-09-13T08:49:27.301401+0000","last_active":"2024-09-13T08:49:28.324135+0000","last_peered":"2024-09-13T08:49:28.324135+0000","last_clean":"2024-09-13T08:49:28.324135+0000","last_became_active":"2024-09-13T08:49:27.300443+0000","last_became_peered":"2024-09-13T08:49:27.300443+0000","last_unstale":"2024-09-13T08:49:28.324135+0000","last_undegraded":"2024-09-13T08:49:28.324135+0000","last_fullsized":"2024-09-13T08:49:28.324135+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:08.144557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287381+0000","last_change":"2024-09-13T08:49:28.311582+0000","last_active":"2024-09-13T08:49:29.287381+0000","last_peered":"2024-09-13T08:49:29.287381+0000","last_clean":"2024-09-13T08:49:29.287381+0000","last_became_active":"2024-09-13T08:49:28.311296+0000","last_became_peered":"2024-09-13T08:49:28.311296+0000","last_unstale":"2024-09-13T08:49:29.287381+0000","last_undegraded":"2024-09-13T08:49:29.287381+0000","last_fullsized":"2024-09-13T08:49:29.287381+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:03.789700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.281266+0000","last_change":"2024-09-13T08:49:27.302457+0000","last_active":"2024-09-13T08:49:28.281266+0000","last_peered":"2024-09-13T08:49:28.281266+0000","last_clean":"2024-09-13T08:49:28.281266+0000","last_became_active":"2024-09-13T08:49:27.302080+0000","last_became_peered":"2024-09-13T08:49:27.302080+0000","last_unstale":"2024-09-13T08:49:28.281266+0000","last_undegraded":"2024-09-13T08:49:28.281266+0000","last_fullsized":"2024-09-13T08:49:28.281266+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:57.166670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.319669+0000","last_change":"2024-09-13T08:49:28.313087+0000","last_active":"2024-09-13T08:49:28.319669+0000","last_peered":"2024-09-13T08:49:28.319669+0000","last_clean":"2024-09-13T08:49:28.319669+0000","last_became_active":"2024-09-13T08:49:28.312748+0000","last_became_peered":"2024-09-13T08:49:28.312748+0000","last_unstale":"2024-09-13T08:49:28.319669+0000","last_undegraded":"2024-09-13T08:49:28.319669+0000","last_fullsized":"2024-09-13T08:49:28.319669+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:28.155055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.285418+0000","last_change":"2024-09-13T08:49:27.297279+0000","last_active":"2024-09-13T08:49:28.285418+0000","last_peered":"2024-09-13T08:49:28.285418+0000","last_clean":"2024-09-13T08:49:28.285418+0000","last_became_active":"2024-09-13T08:49:27.296888+0000","last_became_peered":"2024-09-13T08:49:27.296888+0000","last_unstale":"2024-09-13T08:49:28.285418+0000","last_undegraded":"2024-09-13T08:49:28.285418+0000","last_fullsized":"2024-09-13T08:49:28.285418+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:37.775891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.294424+0000","last_change":"2024-09-13T08:49:28.291798+0000","last_active":"2024-09-13T08:49:28.294424+0000","last_peered":"2024-09-13T08:49:28.294424+0000","last_clean":"2024-09-13T08:49:28.294424+0000","last_became_active":"2024-09-13T08:49:28.291128+0000","last_became_peered":"2024-09-13T08:49:28.291128+0000","last_unstale":"2024-09-13T08:49:28.294424+0000","last_undegraded":"2024-09-13T08:49:28.294424+0000","last_fullsized":"2024-09-13T08:49:28.294424+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:04.105940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309099+0000","last_change":"2024-09-13T08:49:27.293707+0000","last_active":"2024-09-13T08:49:29.309099+0000","last_peered":"2024-09-13T08:49:29.309099+0000","last_clean":"2024-09-13T08:49:29.309099+0000","last_became_active":"2024-09-13T08:49:27.293309+0000","last_became_peered":"2024-09-13T08:49:27.293309+0000","last_unstale":"2024-09-13T08:49:29.309099+0000","last_undegraded":"2024-09-13T08:49:29.309099+0000","last_fullsized":"2024-09-13T08:49:29.309099+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:00:29.660465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287880+0000","last_change":"2024-09-13T08:49:28.303736+0000","last_active":"2024-09-13T08:49:29.287880+0000","last_peered":"2024-09-13T08:49:29.287880+0000","last_clean":"2024-09-13T08:49:29.287880+0000","last_became_active":"2024-09-13T08:49:28.303128+0000","last_became_peered":"2024-09-13T08:49:28.303128+0000","last_unstale":"2024-09-13T08:49:29.287880+0000","last_undegraded":"2024-09-13T08:49:29.287880+0000","last_fullsized":"2024-09-13T08:49:29.287880+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:16:06.842512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.318190+0000","last_change":"2024-09-13T08:49:27.296205+0000","last_active":"2024-09-13T08:49:28.318190+0000","last_peered":"2024-09-13T08:49:28.318190+0000","last_clean":"2024-09-13T08:49:28.318190+0000","last_became_active":"2024-09-13T08:49:27.295922+0000","last_became_peered":"2024-09-13T08:49:27.295922+0000","last_unstale":"2024-09-13T08:49:28.318190+0000","last_undegraded":"2024-09-13T08:49:28.318190+0000","last_fullsized":"2024-09-13T08:49:28.318190+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:13:13.505013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.288027+0000","last_change":"2024-09-13T08:49:28.314734+0000","last_active":"2024-09-13T08:49:29.288027+0000","last_peered":"2024-09-13T08:49:29.288027+0000","last_clean":"2024-09-13T08:49:29.288027+0000","last_became_active":"2024-09-13T08:49:28.314439+0000","last_became_peered":"2024-09-13T08:49:28.314439+0000","last_unstale":"2024-09-13T08:49:29.288027+0000","last_undegraded":"2024-09-13T08:49:29.288027+0000","last_fullsized":"2024-09-13T08:49:29.288027+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:50.516830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"102'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.308239+0000","last_change":"2024-09-13T08:49:27.297035+0000","last_active":"2024-09-13T08:49:29.308239+0000","last_peered":"2024-09-13T08:49:29.308239+0000","last_clean":"2024-09-13T08:49:29.308239+0000","last_became_active":"2024-09-13T08:49:27.296377+0000","last_became_peered":"2024-09-13T08:49:27.296377+0000","last_unstale":"2024-09-13T08:49:29.308239+0000","last_undegraded":"2024-09-13T08:49:29.308239+0000","last_fullsized":"2024-09-13T08:49:29.308239+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:55:27.031358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309592+0000","last_change":"2024-09-13T08:49:28.295786+0000","last_active":"2024-09-13T08:49:29.309592+0000","last_peered":"2024-09-13T08:49:29.309592+0000","last_clean":"2024-09-13T08:49:29.309592+0000","last_became_active":"2024-09-13T08:49:28.295463+0000","last_became_peered":"2024-09-13T08:49:28.295463+0000","last_unstale":"2024-09-13T08:49:29.309592+0000","last_undegraded":"2024-09-13T08:49:29.309592+0000","last_fullsized":"2024-09-13T08:49:29.309592+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:38.779032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287688+0000","last_change":"2024-09-13T08:49:27.298941+0000","last_active":"2024-09-13T08:49:29.287688+0000","last_peered":"2024-09-13T08:49:29.287688+0000","last_clean":"2024-09-13T08:49:29.287688+0000","last_became_active":"2024-09-13T08:49:27.296342+0000","last_became_peered":"2024-09-13T08:49:27.296342+0000","last_unstale":"2024-09-13T08:49:29.287688+0000","last_undegraded":"2024-09-13T08:49:29.287688+0000","last_fullsized":"2024-09-13T08:49:29.287688+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:47:07.117691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.303382+0000","last_change":"2024-09-13T08:49:28.301482+0000","last_active":"2024-09-13T08:49:28.303382+0000","last_peered":"2024-09-13T08:49:28.303382+0000","last_clean":"2024-09-13T08:49:28.303382+0000","last_became_active":"2024-09-13T08:49:28.301216+0000","last_became_peered":"2024-09-13T08:49:28.301216+0000","last_unstale":"2024-09-13T08:49:28.303382+0000","last_undegraded":"2024-09-13T08:49:28.303382+0000","last_fullsized":"2024-09-13T08:49:28.303382+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:44:13.777035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.281747+0000","last_change":"2024-09-13T08:49:27.297259+0000","last_active":"2024-09-13T08:49:28.281747+0000","last_peered":"2024-09-13T08:49:28.281747+0000","last_clean":"2024-09-13T08:49:28.281747+0000","last_became_active":"2024-09-13T08:49:27.296996+0000","last_became_peered":"2024-09-13T08:49:27.296996+0000","last_unstale":"2024-09-13T08:49:28.281747+0000","last_undegraded":"2024-09-13T08:49:28.281747+0000","last_fullsized":"2024-09-13T08:49:28.281747+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:23.151811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.287256+0000","last_change":"2024-09-13T08:49:28.303409+0000","last_active":"2024-09-13T08:49:29.287256+0000","last_peered":"2024-09-13T08:49:29.287256+0000","last_clean":"2024-09-13T08:49:29.287256+0000","last_became_active":"2024-09-13T08:49:28.303111+0000","last_became_peered":"2024-09-13T08:49:28.303111+0000","last_unstale":"2024-09-13T08:49:29.287256+0000","last_undegraded":"2024-09-13T08:49:29.287256+0000","last_fullsized":"2024-09-13T08:49:29.287256+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:03.350204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309727+0000","last_change":"2024-09-13T08:49:27.305906+0000","last_active":"2024-09-13T08:49:29.309727+0000","last_peered":"2024-09-13T08:49:29.309727+0000","last_clean":"2024-09-13T08:49:29.309727+0000","last_became_active":"2024-09-13T08:49:27.305623+0000","last_became_peered":"2024-09-13T08:49:27.305623+0000","last_unstale":"2024-09-13T08:49:29.309727+0000","last_undegraded":"2024-09-13T08:49:29.309727+0000","last_fullsized":"2024-09-13T08:49:29.309727+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:19:42.944227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.309752+0000","last_change":"2024-09-13T08:49:28.307343+0000","last_active":"2024-09-13T08:49:28.309752+0000","last_peered":"2024-09-13T08:49:28.309752+0000","last_clean":"2024-09-13T08:49:28.309752+0000","last_became_active":"2024-09-13T08:49:28.307067+0000","last_became_peered":"2024-09-13T08:49:28.307067+0000","last_unstale":"2024-09-13T08:49:28.309752+0000","last_undegraded":"2024-09-13T08:49:28.309752+0000","last_fullsized":"2024-09-13T08:49:28.309752+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:49.541352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.281506+0000","last_change":"2024-09-13T08:49:27.296163+0000","last_active":"2024-09-13T08:49:28.281506+0000","last_peered":"2024-09-13T08:49:28.281506+0000","last_clean":"2024-09-13T08:49:28.281506+0000","last_became_active":"2024-09-13T08:49:27.295765+0000","last_became_peered":"2024-09-13T08:49:27.295765+0000","last_unstale":"2024-09-13T08:49:28.281506+0000","last_undegraded":"2024-09-13T08:49:28.281506+0000","last_fullsized":"2024-09-13T08:49:28.281506+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:48:56.292419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.286227+0000","last_change":"2024-09-13T08:49:28.296017+0000","last_active":"2024-09-13T08:49:29.286227+0000","last_peered":"2024-09-13T08:49:29.286227+0000","last_clean":"2024-09-13T08:49:29.286227+0000","last_became_active":"2024-09-13T08:49:28.295704+0000","last_became_peered":"2024-09-13T08:49:28.295704+0000","last_unstale":"2024-09-13T08:49:29.286227+0000","last_undegraded":"2024-09-13T08:49:29.286227+0000","last_fullsized":"2024-09-13T08:49:29.286227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:14:55.145831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.297160+0000","last_change":"2024-09-13T08:49:27.305544+0000","last_active":"2024-09-13T08:49:28.297160+0000","last_peered":"2024-09-13T08:49:28.297160+0000","last_clean":"2024-09-13T08:49:28.297160+0000","last_became_active":"2024-09-13T08:49:27.304766+0000","last_became_peered":"2024-09-13T08:49:27.304766+0000","last_unstale":"2024-09-13T08:49:28.297160+0000","last_undegraded":"2024-09-13T08:49:28.297160+0000","last_fullsized":"2024-09-13T08:49:28.297160+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:10.382062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.311952+0000","last_change":"2024-09-13T08:49:28.310728+0000","last_active":"2024-09-13T08:49:28.311952+0000","last_peered":"2024-09-13T08:49:28.311952+0000","last_clean":"2024-09-13T08:49:28.311952+0000","last_became_active":"2024-09-13T08:49:28.309830+0000","last_became_peered":"2024-09-13T08:49:28.309830+0000","last_unstale":"2024-09-13T08:49:28.311952+0000","last_undegraded":"2024-09-13T08:49:28.311952+0000","last_fullsized":"2024-09-13T08:49:28.311952+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:55.183512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309202+0000","last_change":"2024-09-13T08:49:27.292002+0000","last_active":"2024-09-13T08:49:29.309202+0000","last_peered":"2024-09-13T08:49:29.309202+0000","last_clean":"2024-09-13T08:49:29.309202+0000","last_became_active":"2024-09-13T08:49:27.291714+0000","last_became_peered":"2024-09-13T08:49:27.291714+0000","last_unstale":"2024-09-13T08:49:29.309202+0000","last_undegraded":"2024-09-13T08:49:29.309202+0000","last_fullsized":"2024-09-13T08:49:29.309202+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:54:52.903433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-13T08:49:29.309556+0000","last_change":"2024-09-13T08:49:28.315033+0000","last_active":"2024-09-13T08:49:29.309556+0000","last_peered":"2024-09-13T08:49:29.309556+0000","last_clean":"2024-09-13T08:49:29.309556+0000","last_became_active":"2024-09-13T08:49:28.313861+0000","last_became_peered":"2024-09-13T08:49:28.313861+0000","last_unstale":"2024-09-13T08:49:29.309556+0000","last_undegraded":"2024-09-13T08:49:29.309556+0000","last_fullsized":"2024-09-13T08:49:29.309556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:27.267093+0000","last_clean_scrub_stamp":"2024-09-13T08:49:27.267093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:16.225504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.321750+0000","last_change":"2024-09-13T08:49:27.299657+0000","last_active":"2024-09-13T08:49:28.321750+0000","last_peered":"2024-09-13T08:49:28.321750+0000","last_clean":"2024-09-13T08:49:28.321750+0000","last_became_active":"2024-09-13T08:49:27.299236+0000","last_became_peered":"2024-09-13T08:49:27.299236+0000","last_unstale":"2024-09-13T08:49:28.321750+0000","last_undegraded":"2024-09-13T08:49:28.321750+0000","last_fullsized":"2024-09-13T08:49:28.321750+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:49:26.263952+0000","last_clean_scrub_stamp":"2024-09-13T08:49:26.263952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:29.567316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":264,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T08:49:28.285941+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:49:28.285941+0000","last_peered":"2024-09-13T08:49:28.285941+0000","last_clean":"2024-09-13T08:49:28.285941+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:49:28.285941+0000","last_undegraded":"2024-09-13T08:49:28.285941+0000","last_fullsized":"2024-09-13T08:49:28.285941+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509587,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80924,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290916,"statfs":{"total":96636764160,"available":96553897984,"internally_reserved":0,"allocated":1245184,"data_stored":539587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":81590730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509587,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123188,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248652,"statfs":{"total":96636764160,"available":96510619648,"internally_reserved":0,"allocated":1867776,"data_stored":1137425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":124254662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509588,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122580,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249260,"statfs":{"total":96636764160,"available":96511242240,"internally_reserved":0,"allocated":1245184,"data_stored":539587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122576,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249264,"statfs":{"total":96636764160,"available":96511246336,"internally_reserved":0,"allocated":1241088,"data_stored":539005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509587,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118424,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253416,"statfs":{"total":96636764160,"available":96515497984,"internally_reserved":0,"allocated":1245184,"data_stored":539075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509589,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64880,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306960,"statfs":{"total":96636764160,"available":96570327040,"internally_reserved":0,"allocated":1855488,"data_stored":1131097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509587,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82324,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289516,"statfs":{"total":96636764160,"available":96552464384,"internally_reserved":0,"allocated":1241088,"data_stored":539005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542293,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144792,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227048,"statfs":{"total":96636764160,"available":96488497152,"internally_reserved":0,"allocated":1241088,"data_stored":539005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:49:32.860 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:49:32.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T08:49:33.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 -- 172.21.15.44:0/838982725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f200c0b4810 msgr2=0x7f200c0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:33.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 --2- 172.21.15.44:0/838982725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f200c0b4810 0x7f200c0ce810 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f200c06aab0 tx=0x7f200401cd30 comp rx=0 tx=0).stop 2024-09-13T08:49:33.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 -- 172.21.15.44:0/838982725 shutdown_connections 2024-09-13T08:49:33.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 --2- 172.21.15.44:0/838982725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f200c0b4810 0x7f200c0ce810 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 --2- 172.21.15.44:0/838982725 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f200c0cd100 0x7f200c0b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 --2- 172.21.15.44:0/838982725 >> v2:172.21.15.129:3300/0 conn(0x7f200c15dae0 0x7f200c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 -- 172.21.15.44:0/838982725 >> 172.21.15.44:0/838982725 conn(0x7f200c0b18f0 msgr2=0x7f200c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:33.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 -- 172.21.15.44:0/838982725 shutdown_connections 2024-09-13T08:49:33.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 -- 172.21.15.44:0/838982725 wait complete. 2024-09-13T08:49:33.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 Processor -- start 2024-09-13T08:49:33.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 -- start start 2024-09-13T08:49:33.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f200c0b4810 0x7f200c1503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:33.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f200c0cd100 0x7f200c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:33.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f200c15dae0 0x7f200c09bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:33.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f200c06e7a0 con 0x7f200c0b4810 2024-09-13T08:49:33.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f200c0cccb0 con 0x7f200c15dae0 2024-09-13T08:49:33.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.055+0000 7f2012c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f200c06e920 con 0x7f200c0cd100 2024-09-13T08:49:33.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f200be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f200c0b4810 0x7f200c1503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:33.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f200be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f200c0b4810 0x7f200c1503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42534/0 (socket says 172.21.15.44:42534) 2024-09-13T08:49:33.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f200be00640 1 -- 172.21.15.44:0/895120084 learned_addr learned my addr 172.21.15.44:0/895120084 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f2010c00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f200c15dae0 0x7f200c09bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f200be00640 1 -- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f200c0cd100 msgr2=0x7f200c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f200be00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f200c0cd100 0x7f200c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f200be00640 1 -- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f200c15dae0 msgr2=0x7f200c09bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f200be00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f200c15dae0 0x7f200c09bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f200be00640 1 -- 172.21.15.44:0/895120084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c1556f0 con 0x7f200c0b4810 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f2010c00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f200c15dae0 0x7f200c09bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f200be00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f200c0b4810 0x7f200c1503c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00b410 tx=0x7f1ffc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f2008c00640 1 -- 172.21.15.44:0/895120084 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc015020 con 0x7f200c0b4810 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f2008c00640 1 -- 172.21.15.44:0/895120084 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ffc011920 con 0x7f200c0b4810 2024-09-13T08:49:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f2008c00640 1 -- 172.21.15.44:0/895120084 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc011d70 con 0x7f200c0b4810 2024-09-13T08:49:33.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f2012c00640 1 -- 172.21.15.44:0/895120084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200c0afa00 con 0x7f200c0b4810 2024-09-13T08:49:33.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f2012c00640 1 -- 172.21.15.44:0/895120084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200c0affc0 con 0x7f200c0b4810 2024-09-13T08:49:33.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f2012c00640 1 -- 172.21.15.44:0/895120084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200c069410 con 0x7f200c0b4810 2024-09-13T08:49:33.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.059+0000 7f2008c00640 1 -- 172.21.15.44:0/895120084 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f1ffc018020 con 0x7f200c0b4810 2024-09-13T08:49:33.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.063+0000 7f2008c00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fe8074540 0x7f1fe8076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:33.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.063+0000 7f2008c00640 1 -- 172.21.15.44:0/895120084 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f1ffc092ac0 con 0x7f200c0b4810 2024-09-13T08:49:33.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.063+0000 7f2003e00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fe8074540 0x7f1fe8076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:33.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.063+0000 7f2008c00640 1 -- 172.21.15.44:0/895120084 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1ffc01f020 con 0x7f200c0b4810 2024-09-13T08:49:33.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.067+0000 7f2003e00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fe8074540 0x7f1fe8076a00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f200c0ad780 tx=0x7f1ff4006c50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:33.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.307+0000 7f2012c00640 1 -- 172.21.15.44:0/895120084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f200c0b01c0 con 0x7f200c0b4810 2024-09-13T08:49:33.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2008c00640 1 -- 172.21.15.44:0/895120084 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f1ffc05f770 con 0x7f200c0b4810 2024-09-13T08:49:33.316 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T08:49:33.316 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:49:33.316 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-09-13T08:49:33.317 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-09-13T08:49:33.317 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-13T08:49:33.317 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-09-13T08:49:33.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 -- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fe8074540 msgr2=0x7f1fe8076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:33.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fe8074540 0x7f1fe8076a00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f200c0ad780 tx=0x7f1ff4006c50 comp rx=0 tx=0).stop 2024-09-13T08:49:33.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 -- 172.21.15.44:0/895120084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f200c0b4810 msgr2=0x7f200c1503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:33.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f200c0b4810 0x7f200c1503c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00b410 tx=0x7f1ffc00b8d0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 -- 172.21.15.44:0/895120084 shutdown_connections 2024-09-13T08:49:33.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fe8074540 0x7f1fe8076a00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f200c15dae0 0x7f200c09bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f200c0cd100 0x7f200c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 --2- 172.21.15.44:0/895120084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f200c0b4810 0x7f200c1503c0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 -- 172.21.15.44:0/895120084 >> 172.21.15.44:0/895120084 conn(0x7f200c0b18f0 msgr2=0x7f200c06fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:33.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 -- 172.21.15.44:0/895120084 shutdown_connections 2024-09-13T08:49:33.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.315+0000 7f2012c00640 1 -- 172.21.15.44:0/895120084 wait complete. 2024-09-13T08:49:33.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:49:33.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.483+0000 7f44f3000640 1 -- 172.21.15.44:0/30117050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44ec15bea0 msgr2=0x7f44ec0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:33.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.483+0000 7f44f3000640 1 --2- 172.21.15.44:0/30117050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44ec15bea0 0x7f44ec0ce670 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0cc3a0 tx=0x7f44dc01c830 comp rx=0 tx=0).stop 2024-09-13T08:49:33.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.483+0000 7f44f3000640 1 -- 172.21.15.44:0/30117050 shutdown_connections 2024-09-13T08:49:33.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.483+0000 7f44f3000640 1 --2- 172.21.15.44:0/30117050 >> v2:172.21.15.129:3300/0 conn(0x7f44ec0d28e0 0x7f44ec0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.483+0000 7f44f3000640 1 --2- 172.21.15.44:0/30117050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44ec15bea0 0x7f44ec0ce670 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.483+0000 7f44f3000640 1 -- 172.21.15.44:0/30117050 >> 172.21.15.44:0/30117050 conn(0x7f44ec0d1dc0 msgr2=0x7f44ec0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:33.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.483+0000 7f44f3000640 1 -- 172.21.15.44:0/30117050 shutdown_connections 2024-09-13T08:49:33.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.483+0000 7f44f3000640 1 -- 172.21.15.44:0/30117050 wait complete. 2024-09-13T08:49:33.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 Processor -- start 2024-09-13T08:49:33.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 -- start start 2024-09-13T08:49:33.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f44ec0cdb30 0x7f44ec0c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:33.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ec0d28e0 0x7f44ec15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44ec15bea0 0x7f44ec0c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec168fc0 con 0x7f44ec15bea0 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec168e40 con 0x7f44ec0d28e0 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec169140 con 0x7f44ec0cdb30 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44ec15bea0 0x7f44ec0c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44ebe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f44ec0cdb30 0x7f44ec0c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44ec15bea0 0x7f44ec0c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42548/0 (socket says 172.21.15.44:42548) 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f1000640 1 -- 172.21.15.44:0/3629976262 learned_addr learned my addr 172.21.15.44:0/3629976262 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44e3e00640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ec0d28e0 0x7f44ec15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f1000640 1 -- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f44ec0cdb30 msgr2=0x7f44ec0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f1000640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f44ec0cdb30 0x7f44ec0c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f1000640 1 -- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ec0d28e0 msgr2=0x7f44ec15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:33.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f1000640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ec0d28e0 0x7f44ec15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f1000640 1 -- 172.21.15.44:0/3629976262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec0c56c0 con 0x7f44ec15bea0 2024-09-13T08:49:33.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f1000640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44ec15bea0 0x7f44ec0c6db0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f44e400cce0 tx=0x7f44e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:33.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44e8c00640 1 -- 172.21.15.44:0/3629976262 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e400f450 con 0x7f44ec15bea0 2024-09-13T08:49:33.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44e8c00640 1 -- 172.21.15.44:0/3629976262 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44e400f700 con 0x7f44ec15bea0 2024-09-13T08:49:33.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44e8c00640 1 -- 172.21.15.44:0/3629976262 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e400fb50 con 0x7f44ec15bea0 2024-09-13T08:49:33.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 -- 172.21.15.44:0/3629976262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec0c5960 con 0x7f44ec15bea0 2024-09-13T08:49:33.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 -- 172.21.15.44:0/3629976262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec0c5f20 con 0x7f44ec15bea0 2024-09-13T08:49:33.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.487+0000 7f44f3000640 1 -- 172.21.15.44:0/3629976262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ec0cc9d0 con 0x7f44ec15bea0 2024-09-13T08:49:33.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.491+0000 7f44e8c00640 1 -- 172.21.15.44:0/3629976262 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f44e400fcf0 con 0x7f44ec15bea0 2024-09-13T08:49:33.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.491+0000 7f44e8c00640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f44c8074570 0x7f44c8076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:33.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.491+0000 7f44e8c00640 1 -- 172.21.15.44:0/3629976262 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f44e4093570 con 0x7f44ec15bea0 2024-09-13T08:49:33.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.491+0000 7f44ebe00640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f44c8074570 0x7f44c8076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:33.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.491+0000 7f44ebe00640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f44c8074570 0x7f44c8076a30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0af000 tx=0x7f44dc01b070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:33.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.495+0000 7f44e8c00640 1 -- 172.21.15.44:0/3629976262 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f44e4014030 con 0x7f44ec15bea0 2024-09-13T08:49:33.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.711+0000 7f44f3000640 1 -- 172.21.15.44:0/3629976262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44ec0762f0 con 0x7f44ec15bea0 2024-09-13T08:49:33.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.711+0000 7f44e8c00640 1 -- 172.21.15.44:0/3629976262 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5225 (secure 0 0 0) 0x7f44e405fbd0 con 0x7f44ec15bea0 2024-09-13T08:49:33.715 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:49:33.715 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":65,"btime":"2024-09-13T08:49:31:296465+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1987308515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1987308515},{"type":"v1","addr":"172.21.15.129:6837","nonce":1987308515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1224713128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1224713128},{"type":"v1","addr":"172.21.15.129:6836","nonce":1224713128}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7755,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/380853907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":380853907},{"type":"v1","addr":"172.21.15.44:6833","nonce":380853907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:49:27.697092+0000","modified":"2024-09-13T08:49:30.701664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/409896091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":409896091},{"type":"v1","addr":"172.21.15.44:6835","nonce":409896091}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7758,"qdb_cluster":[7758]},"id":7}]} 2024-09-13T08:49:33.715 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 65 2024-09-13T08:49:33.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 -- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f44c8074570 msgr2=0x7f44c8076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:33.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f44c8074570 0x7f44c8076a30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0af000 tx=0x7f44dc01b070 comp rx=0 tx=0).stop 2024-09-13T08:49:33.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 -- 172.21.15.44:0/3629976262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44ec15bea0 msgr2=0x7f44ec0c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:33.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44ec15bea0 0x7f44ec0c6db0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f44e400cce0 tx=0x7f44e400a620 comp rx=0 tx=0).stop 2024-09-13T08:49:33.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 -- 172.21.15.44:0/3629976262 shutdown_connections 2024-09-13T08:49:33.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f44c8074570 0x7f44c8076a30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44ec15bea0 0x7f44ec0c6db0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ec0d28e0 0x7f44ec15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 --2- 172.21.15.44:0/3629976262 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f44ec0cdb30 0x7f44ec0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:33.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 -- 172.21.15.44:0/3629976262 >> 172.21.15.44:0/3629976262 conn(0x7f44ec0d1dc0 msgr2=0x7f44ec0b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:33.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 -- 172.21.15.44:0/3629976262 shutdown_connections 2024-09-13T08:49:33.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:33.715+0000 7f44f3000640 1 -- 172.21.15.44:0/3629976262 wait complete. 2024-09-13T08:49:34.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T08:49:27.697092+0000', 'modified': '2024-09-13T08:49:30.701664+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7758}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7758': {'gid': 7758, 'name': 'c', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/409896091', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 409896091}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 409896091}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7758, 'qdb_cluster': [7758]} 2024-09-13T08:49:34.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T08:49:34.138 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T08:49:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.287+0000 7f5b17e00640 1 -- 172.21.15.44:0/3627423139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b18155080 msgr2=0x7f5b180cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.287+0000 7f5b17e00640 1 --2- 172.21.15.44:0/3627423139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b18155080 0x7f5b180cd510 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f5b18074b80 tx=0x7f5b0801c800 comp rx=0 tx=0).stop 2024-09-13T08:49:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.287+0000 7f5b17e00640 1 -- 172.21.15.44:0/3627423139 shutdown_connections 2024-09-13T08:49:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.287+0000 7f5b17e00640 1 --2- 172.21.15.44:0/3627423139 >> v2:172.21.15.129:3300/0 conn(0x7f5b180cda50 0x7f5b180cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.287+0000 7f5b17e00640 1 --2- 172.21.15.44:0/3627423139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b18155080 0x7f5b180cd510 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.287+0000 7f5b17e00640 1 -- 172.21.15.44:0/3627423139 >> 172.21.15.44:0/3627423139 conn(0x7f5b1806dd30 msgr2=0x7f5b1806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.287+0000 7f5b17e00640 1 -- 172.21.15.44:0/3627423139 shutdown_connections 2024-09-13T08:49:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.287+0000 7f5b17e00640 1 -- 172.21.15.44:0/3627423139 wait complete. 2024-09-13T08:49:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17e00640 1 Processor -- start 2024-09-13T08:49:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17e00640 1 -- start start 2024-09-13T08:49:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b180cb8a0 0x7f5b181f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5b180cda50 0x7f5b181f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b18155080 0x7f5b180c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18167cf0 con 0x7f5b18155080 2024-09-13T08:49:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18167b70 con 0x7f5b180cb8a0 2024-09-13T08:49:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5b18167e70 con 0x7f5b180cda50 2024-09-13T08:49:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b18155080 0x7f5b180c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b18155080 0x7f5b180c2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42566/0 (socket says 172.21.15.44:42566) 2024-09-13T08:49:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17400640 1 -- 172.21.15.44:0/2062955362 learned_addr learned my addr 172.21.15.44:0/2062955362 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b0fe00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5b180cda50 0x7f5b181f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b16a00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b180cb8a0 0x7f5b181f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17400640 1 -- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5b180cda50 msgr2=0x7f5b181f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17400640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5b180cda50 0x7f5b181f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17400640 1 -- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b180cb8a0 msgr2=0x7f5b181f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17400640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b180cb8a0 0x7f5b181f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17400640 1 -- 172.21.15.44:0/2062955362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b18159410 con 0x7f5b18155080 2024-09-13T08:49:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17400640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b18155080 0x7f5b180c2fb0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000ca80 tx=0x7f5b1000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b0ea00640 1 -- 172.21.15.44:0/2062955362 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10016020 con 0x7f5b18155080 2024-09-13T08:49:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b0ea00640 1 -- 172.21.15.44:0/2062955362 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b1000a930 con 0x7f5b18155080 2024-09-13T08:49:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b0ea00640 1 -- 172.21.15.44:0/2062955362 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1000ad80 con 0x7f5b18155080 2024-09-13T08:49:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17e00640 1 -- 172.21.15.44:0/2062955362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b181f9db0 con 0x7f5b18155080 2024-09-13T08:49:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.291+0000 7f5b17e00640 1 -- 172.21.15.44:0/2062955362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b181f8dc0 con 0x7f5b18155080 2024-09-13T08:49:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b0ea00640 1 -- 172.21.15.44:0/2062955362 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f5b10019020 con 0x7f5b18155080 2024-09-13T08:49:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b0ea00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5af4074540 0x7f5af4076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b20a1c000 1 Processor -- start 2024-09-13T08:49:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b0ea00640 1 -- 172.21.15.44:0/2062955362 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f5b100928e0 con 0x7f5b18155080 2024-09-13T08:49:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b20a1c000 1 -- start start 2024-09-13T08:49:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b16a00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5af4074540 0x7f5af4076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 0x557c0c85d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c810ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c811220 0x557c0c811600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557c0c72d3f0 con 0x557c0c85d6e0 2024-09-13T08:49:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c0c66ab20 con 0x557c0c85cdc0 2024-09-13T08:49:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x557c0c659f50 con 0x557c0c811220 2024-09-13T08:49:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b16a00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5af4074540 0x7f5af4076a00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5b080af000 tx=0x7f5b0801b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c810ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c810ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42578/0 (socket says 172.21.15.44:42578) 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b0fe00640 1 -- 172.21.15.44:0/1917425443 learned_addr learned my addr 172.21.15.44:0/1917425443 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b16a00640 1 --2- 172.21.15.44:0/1917425443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 0x557c0c85d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.295+0000 7f5b17400640 1 --2- 172.21.15.44:0/1917425443 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c811220 0x557c0c811600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b0fe00640 1 -- 172.21.15.44:0/1917425443 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c811220 msgr2=0x557c0c811600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b0fe00640 1 --2- 172.21.15.44:0/1917425443 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c811220 0x557c0c811600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b0fe00640 1 -- 172.21.15.44:0/1917425443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 msgr2=0x557c0c85d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b0fe00640 1 --2- 172.21.15.44:0/1917425443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 0x557c0c85d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b0fe00640 1 -- 172.21.15.44:0/1917425443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0c621590 con 0x557c0c85d6e0 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b0fe00640 1 --2- 172.21.15.44:0/1917425443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c810ce0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000bbd0 tx=0x7f5b000107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d283e9265c622d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b07400640 1 -- 172.21.15.44:0/1917425443 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0001c480 con 0x557c0c85d6e0 2024-09-13T08:49:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b07400640 1 -- 172.21.15.44:0/1917425443 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b0001c710 con 0x557c0c85d6e0 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b07400640 1 -- 172.21.15.44:0/1917425443 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0001cb40 con 0x557c0c85d6e0 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 -- 172.21.15.44:0/1917425443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 msgr2=0x557c0c810ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/1917425443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c810ce0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000bbd0 tx=0x7f5b000107a0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 -- 172.21.15.44:0/1917425443 shutdown_connections 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/1917425443 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c811220 0x557c0c811600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/1917425443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c810ce0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/1917425443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 0x557c0c85d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 -- 172.21.15.44:0/1917425443 >> 172.21.15.44:0/1917425443 conn(0x557c0c6f6130 msgr2=0x557c0c83be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 -- 172.21.15.44:0/1917425443 shutdown_connections 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 -- 172.21.15.44:0/1917425443 wait complete. 2024-09-13T08:49:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 -- start start 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 Processor -- start 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c811220 0x557c0c81d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85cdc0 0x557c0c81d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c81dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557c0c5f45d0 con 0x557c0c85d6e0 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c0c65dd60 con 0x557c0c811220 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x557c0c600500 con 0x557c0c85cdc0 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b20a1c000 10 client.0 _ll_register_callbacks cb 0x557c0c604810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b17400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c81dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b17400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c81dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42586/0 (socket says 172.21.15.44:42586) 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b17400640 1 -- 172.21.15.44:0/2334134552 learned_addr learned my addr 172.21.15.44:0/2334134552 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b16a00640 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c811220 0x557c0c81d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b0fe00640 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85cdc0 0x557c0c81d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b17400640 1 -- 172.21.15.44:0/2334134552 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85cdc0 msgr2=0x557c0c81d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:49:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b17400640 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85cdc0 0x557c0c81d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b17400640 1 -- 172.21.15.44:0/2334134552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c811220 msgr2=0x557c0c81d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:49:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b17400640 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c811220 0x557c0c81d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.299+0000 7f5b17400640 1 -- 172.21.15.44:0/2334134552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0c5f8170 con 0x557c0c85d6e0 2024-09-13T08:49:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b17400640 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c81dcb0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f5b1005eed0 tx=0x7f5b1005edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1db82dcbda9b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5afbe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:49:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5afbe00640 1 -- 172.21.15.44:0/2334134552 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1005ef40 con 0x557c0c85d6e0 2024-09-13T08:49:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5afbe00640 1 -- 172.21.15.44:0/2334134552 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f5b10064b00 con 0x557c0c85d6e0 2024-09-13T08:49:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2334134552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557c0c5f20f0 con 0x557c0c85d6e0 2024-09-13T08:49:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5afbe00640 1 -- 172.21.15.44:0/2334134552 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f5b10080050 con 0x557c0c85d6e0 2024-09-13T08:49:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 10 client.7893 fetch_fsmap learned FSMap version 65 2024-09-13T08:49:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2334134552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x557c0c6deef0 con 0x557c0c85d6e0 2024-09-13T08:49:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5afbe00640 1 -- 172.21.15.44:0/2334134552 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557c0c5f20f0 con 0x557c0c85d6e0 2024-09-13T08:49:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5afbe00640 1 -- 172.21.15.44:0/2334134552 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 65) ==== 3502+0+0 (secure 0 0 0) 0x7f5b100801f0 con 0x557c0c85d6e0 2024-09-13T08:49:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 10 client.7893 fetch_fsmap finished waiting for FSMap version 65 2024-09-13T08:49:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 10 client.7893 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 20 client.7893 populate_metadata read hostname 'smithi044' 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.44:6834/409896091,v1:172.21.15.44:6835/409896091] conn(0x557c0c86c060 0x557c0c86e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 4 client.7893 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2334134552 --> [v2:172.21.15.44:6834/409896091,v1:172.21.15.44:6835/409896091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c0c66ca70 con 0x557c0c86c060 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b16a00640 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.44:6834/409896091,v1:172.21.15.44:6835/409896091] conn(0x557c0c86c060 0x557c0c86e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b16a00640 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.44:6834/409896091,v1:172.21.15.44:6835/409896091] conn(0x557c0c86c060 0x557c0c86e450 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d0082d2c5522e6 server_cookie=176b3d10253f71f2 in_seq=0 out_seq=0 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5afbe00640 10 client.7893 ms_handle_connect on v2:172.21.15.44:6834/409896091 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5afbe00640 1 -- 172.21.15.44:0/2334134552 <== mds.0 v2:172.21.15.44:6834/409896091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c0c66ca70 con 0x557c0c86c060 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5afbe00640 10 client.7893 handle_command_reply: tid=0 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 client.7893 shutdown 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2334134552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 msgr2=0x557c0c81dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c81dcb0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f5b1005eed0 tx=0x7f5b1005edf0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2334134552 shutdown_connections 2024-09-13T08:49:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.44:6834/409896091,v1:172.21.15.44:6835/409896091] conn(0x557c0c86c060 0x557c0c86e450 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c85d6e0 0x557c0c81dcb0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85cdc0 0x557c0c81d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2334134552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c811220 0x557c0c81d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5afbe00640 0 client.7893 ms_handle_reset on v2:172.21.15.44:6834/409896091 2024-09-13T08:49:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2334134552 >> 172.21.15.44:0/2334134552 conn(0x557c0c6f6130 msgr2=0x557c0c80b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2334134552 shutdown_connections 2024-09-13T08:49:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2334134552 wait complete. 2024-09-13T08:49:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.303+0000 7f5b20a1c000 20 client.7893 trim_cache size 0 max 16384 2024-09-13T08:49:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.323+0000 7f5b20a1c000 1 Processor -- start 2024-09-13T08:49:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.323+0000 7f5b20a1c000 1 -- start start 2024-09-13T08:49:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c838ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 0x557c0c839000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85d6e0 0x557c0c86c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1815d9a0 con 0x557c0c811220 2024-09-13T08:49:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c0c662390 con 0x557c0c85cdc0 2024-09-13T08:49:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x557c0c6dea60 con 0x557c0c85d6e0 2024-09-13T08:49:34.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b16a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c838ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b16a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c838ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42594/0 (socket says 172.21.15.44:42594) 2024-09-13T08:49:34.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b16a00640 1 -- 172.21.15.44:0/1405455726 learned_addr learned my addr 172.21.15.44:0/1405455726 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b0fe00640 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 0x557c0c839000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b17400640 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85d6e0 0x557c0c86c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b16a00640 1 -- 172.21.15.44:0/1405455726 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85d6e0 msgr2=0x557c0c86c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:49:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b16a00640 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85d6e0 0x557c0c86c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b16a00640 1 -- 172.21.15.44:0/1405455726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 msgr2=0x557c0c839000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:49:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b16a00640 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 0x557c0c839000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b16a00640 1 -- 172.21.15.44:0/1405455726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0c6b5d90 con 0x557c0c811220 2024-09-13T08:49:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b17400640 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85d6e0 0x557c0c86c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b0fe00640 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 0x557c0c839000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b16a00640 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c838ac0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f5b0801a430 tx=0x7f5b080bd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33a467d80ed4d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b06a00640 1 -- 172.21.15.44:0/1405455726 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b080048d0 con 0x557c0c811220 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b06a00640 1 -- 172.21.15.44:0/1405455726 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b080d5070 con 0x557c0c811220 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b06a00640 1 -- 172.21.15.44:0/1405455726 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08004c70 con 0x557c0c811220 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 -- 172.21.15.44:0/1405455726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 msgr2=0x557c0c838ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c838ac0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f5b0801a430 tx=0x7f5b080bd000 comp rx=0 tx=0).stop 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 -- 172.21.15.44:0/1405455726 shutdown_connections 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85d6e0 0x557c0c86c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85cdc0 0x557c0c839000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/1405455726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c838ac0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 -- 172.21.15.44:0/1405455726 >> 172.21.15.44:0/1405455726 conn(0x557c0c6f6130 msgr2=0x557c0c85c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 -- 172.21.15.44:0/1405455726 shutdown_connections 2024-09-13T08:49:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 -- 172.21.15.44:0/1405455726 wait complete. 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 -- start start 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.327+0000 7f5b20a1c000 1 Processor -- start 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c8d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85cdc0 0x557c0c8d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85d6e0 0x557c0c8d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557c0c719730 con 0x557c0c811220 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c0c65d070 con 0x557c0c85d6e0 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x557c0c625ec0 con 0x557c0c85cdc0 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 10 client.0 _ll_register_callbacks cb 0x7f5b1000af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b16a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c8d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b16a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c8d25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42600/0 (socket says 172.21.15.44:42600) 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b16a00640 1 -- 172.21.15.44:0/2354951677 learned_addr learned my addr 172.21.15.44:0/2354951677 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b0fe00640 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85cdc0 0x557c0c8d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b17400640 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85d6e0 0x557c0c8d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b16a00640 1 -- 172.21.15.44:0/2354951677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85cdc0 msgr2=0x557c0c8d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b16a00640 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85cdc0 0x557c0c8d2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b16a00640 1 -- 172.21.15.44:0/2354951677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85d6e0 msgr2=0x557c0c8d3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b16a00640 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85d6e0 0x557c0c8d3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b16a00640 1 -- 172.21.15.44:0/2354951677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0c620a90 con 0x557c0c811220 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b16a00640 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c8d25d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f5b080d2040 tx=0x7f5b08005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a890a5d8bf7e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5afbe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5afbe00640 1 -- 172.21.15.44:0/2354951677 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08004b10 con 0x557c0c811220 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5afbe00640 1 -- 172.21.15.44:0/2354951677 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f5b08004cb0 con 0x557c0c811220 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2354951677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b080d5070 con 0x557c0c811220 2024-09-13T08:49:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5afbe00640 1 -- 172.21.15.44:0/2354951677 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f5b08003600 con 0x557c0c811220 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 10 client.7899 fetch_fsmap learned FSMap version 65 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2354951677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x557c0c5ff050 con 0x557c0c811220 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5afbe00640 1 -- 172.21.15.44:0/2354951677 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b080d5070 con 0x557c0c811220 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5afbe00640 1 -- 172.21.15.44:0/2354951677 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 65) ==== 3502+0+0 (secure 0 0 0) 0x7f5b080037a0 con 0x557c0c811220 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 10 client.7899 fetch_fsmap finished waiting for FSMap version 65 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 10 client.7899 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 20 client.7899 populate_metadata read hostname 'smithi044' 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.44:6834/409896091,v1:172.21.15.44:6835/409896091] conn(0x557c0c86fab0 0x557c0c86fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 4 client.7899 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2354951677 --> [v2:172.21.15.44:6834/409896091,v1:172.21.15.44:6835/409896091] -- command(tid 0: {"prefix": "status"}) -- 0x557c0c668520 con 0x557c0c86fab0 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b0fe00640 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.44:6834/409896091,v1:172.21.15.44:6835/409896091] conn(0x557c0c86fab0 0x557c0c86fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b0fe00640 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.44:6834/409896091,v1:172.21.15.44:6835/409896091] conn(0x557c0c86fab0 0x557c0c86fe90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9867d7a531606e37 server_cookie=cdd73aad2340e0a3 in_seq=0 out_seq=0 2024-09-13T08:49:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5afbe00640 10 client.7899 ms_handle_connect on v2:172.21.15.44:6834/409896091 2024-09-13T08:49:34.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5afbe00640 1 -- 172.21.15.44:0/2354951677 <== mds.0 v2:172.21.15.44:6834/409896091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557c0c668520 con 0x557c0c86fab0 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5afbe00640 10 client.7899 handle_command_reply: tid=0 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 client.7899 shutdown 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2354951677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 msgr2=0x557c0c8d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c8d25d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f5b080d2040 tx=0x7f5b08005850 comp rx=0 tx=0).stop 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2354951677 shutdown_connections 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.44:6834/409896091,v1:172.21.15.44:6835/409896091] conn(0x557c0c86fab0 0x557c0c86fe90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c85d6e0 0x557c0c8d3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x557c0c85cdc0 0x557c0c8d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 --2- 172.21.15.44:0/2354951677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c0c811220 0x557c0c8d25d0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5afbe00640 0 client.7899 ms_handle_reset on v2:172.21.15.44:6834/409896091 2024-09-13T08:49:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2354951677 >> 172.21.15.44:0/2354951677 conn(0x557c0c6f6130 msgr2=0x557c0c85bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2354951677 shutdown_connections 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 1 -- 172.21.15.44:0/2354951677 wait complete. 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.331+0000 7f5b20a1c000 20 client.7899 trim_cache size 0 max 16384 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 7758, 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T08:49:34.339 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.0427031949999996, 2024-09-13T08:49:34.340 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 65, 2024-09-13T08:49:34.340 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 102, 2024-09-13T08:49:34.340 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 102, 2024-09-13T08:49:34.340 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 10.510928648 2024-09-13T08:49:34.340 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:49:34.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 -- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5af4074540 msgr2=0x7f5af4076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5af4074540 0x7f5af4076a00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5b080af000 tx=0x7f5b0801b040 comp rx=0 tx=0).stop 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 -- 172.21.15.44:0/2062955362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b18155080 msgr2=0x7f5b180c2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b18155080 0x7f5b180c2fb0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000ca80 tx=0x7f5b1000cb70 comp rx=0 tx=0).stop 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 -- 172.21.15.44:0/2062955362 shutdown_connections 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5af4074540 0x7f5af4076a00 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b18155080 0x7f5b180c2fb0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5b180cda50 0x7f5b181f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 --2- 172.21.15.44:0/2062955362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b180cb8a0 0x7f5b181f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 -- 172.21.15.44:0/2062955362 >> 172.21.15.44:0/2062955362 conn(0x7f5b1806dd30 msgr2=0x7f5b18154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 -- 172.21.15.44:0/2062955362 shutdown_connections 2024-09-13T08:49:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.335+0000 7f5b17e00640 1 -- 172.21.15.44:0/2062955362 wait complete. 2024-09-13T08:49:34.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:49:34.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.499+0000 7f7981c00640 1 -- 172.21.15.44:0/2169251643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797c0ced80 msgr2=0x7f797c0ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.499+0000 7f7981c00640 1 --2- 172.21.15.44:0/2169251643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797c0ced80 0x7f797c0ae9b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f797c0cf340 tx=0x7f796401c800 comp rx=0 tx=0).stop 2024-09-13T08:49:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.499+0000 7f7981c00640 1 -- 172.21.15.44:0/2169251643 shutdown_connections 2024-09-13T08:49:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.499+0000 7f7981c00640 1 --2- 172.21.15.44:0/2169251643 >> v2:172.21.15.129:3300/0 conn(0x7f797c0cdb30 0x7f797c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.499+0000 7f7981c00640 1 --2- 172.21.15.44:0/2169251643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797c0ced80 0x7f797c0ae9b0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.499+0000 7f7981c00640 1 -- 172.21.15.44:0/2169251643 >> 172.21.15.44:0/2169251643 conn(0x7f797c0d1dc0 msgr2=0x7f797c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.499+0000 7f7981c00640 1 -- 172.21.15.44:0/2169251643 shutdown_connections 2024-09-13T08:49:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.499+0000 7f7981c00640 1 -- 172.21.15.44:0/2169251643 wait complete. 2024-09-13T08:49:34.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7981c00640 1 Processor -- start 2024-09-13T08:49:34.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7981c00640 1 -- start start 2024-09-13T08:49:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7981c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f797c0cdb30 0x7f797c070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7981c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f797c0ced80 0x7f797c071110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7981c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797c15bea0 0x7f797c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7981c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f797c169240 con 0x7f797c15bea0 2024-09-13T08:49:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7981c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1690c0 con 0x7f797c0ced80 2024-09-13T08:49:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7981c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f797c1693c0 con 0x7f797c0cdb30 2024-09-13T08:49:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797c15bea0 0x7f797c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797c15bea0 0x7f797c06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42614/0 (socket says 172.21.15.44:42614) 2024-09-13T08:49:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797b400640 1 -- 172.21.15.44:0/2833522320 learned_addr learned my addr 172.21.15.44:0/2833522320 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7972000640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f797c0ced80 0x7f797c071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797aa00640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f797c0cdb30 0x7f797c070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797b400640 1 -- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f797c0cdb30 msgr2=0x7f797c070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797b400640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f797c0cdb30 0x7f797c070bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797b400640 1 -- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f797c0ced80 msgr2=0x7f797c071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797b400640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f797c0ced80 0x7f797c071110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797b400640 1 -- 172.21.15.44:0/2833522320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c06f000 con 0x7f797c15bea0 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797aa00640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f797c0cdb30 0x7f797c070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f797b400640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797c15bea0 0x7f797c06dd30 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f797400ea20 tx=0x7f797400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7973400640 1 -- 172.21.15.44:0/2833522320 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797400cde0 con 0x7f797c15bea0 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7973400640 1 -- 172.21.15.44:0/2833522320 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7974004c40 con 0x7f797c15bea0 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7973400640 1 -- 172.21.15.44:0/2833522320 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974005280 con 0x7f797c15bea0 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7981c00640 1 -- 172.21.15.44:0/2833522320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797c06f2e0 con 0x7f797c15bea0 2024-09-13T08:49:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.503+0000 7f7981c00640 1 -- 172.21.15.44:0/2833522320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797c14cb80 con 0x7f797c15bea0 2024-09-13T08:49:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.507+0000 7f7981c00640 1 -- 172.21.15.44:0/2833522320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797c06f520 con 0x7f797c15bea0 2024-09-13T08:49:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.507+0000 7f7973400640 1 -- 172.21.15.44:0/2833522320 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f7974005420 con 0x7f797c15bea0 2024-09-13T08:49:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.507+0000 7f7973400640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7958074540 0x7f7958076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.507+0000 7f7973400640 1 -- 172.21.15.44:0/2833522320 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f7974093490 con 0x7f797c15bea0 2024-09-13T08:49:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.507+0000 7f797aa00640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7958074540 0x7f7958076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.507+0000 7f797aa00640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7958074540 0x7f7958076a00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f796c000f70 tx=0x7f796c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:34.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.511+0000 7f7973400640 1 -- 172.21.15.44:0/2833522320 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f797c06f520 con 0x7f797c15bea0 2024-09-13T08:49:34.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.723+0000 7f7981c00640 1 -- 172.21.15.44:0/2833522320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f797c0ce670 con 0x7f797c15bea0 2024-09-13T08:49:34.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.723+0000 7f7973400640 1 -- 172.21.15.44:0/2833522320 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) ==== 53+0+73 (secure 0 0 0) 0x7f7974016030 con 0x7f797c15bea0 2024-09-13T08:49:34.727 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:49:34.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 -- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7958074540 msgr2=0x7f7958076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7958074540 0x7f7958076a00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f796c000f70 tx=0x7f796c009040 comp rx=0 tx=0).stop 2024-09-13T08:49:34.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 -- 172.21.15.44:0/2833522320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797c15bea0 msgr2=0x7f797c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:34.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797c15bea0 0x7f797c06dd30 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f797400ea20 tx=0x7f797400eee0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 -- 172.21.15.44:0/2833522320 shutdown_connections 2024-09-13T08:49:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7958074540 0x7f7958076a00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797c15bea0 0x7f797c06dd30 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f797c0ced80 0x7f797c071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 --2- 172.21.15.44:0/2833522320 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f797c0cdb30 0x7f797c070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 -- 172.21.15.44:0/2833522320 >> 172.21.15.44:0/2833522320 conn(0x7f797c0d1dc0 msgr2=0x7f797c0b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 -- 172.21.15.44:0/2833522320 shutdown_connections 2024-09-13T08:49:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34.727+0000 7f7981c00640 1 -- 172.21.15.44:0/2833522320 wait complete. 2024-09-13T08:49:34.749 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T08:49:34.749 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:49:34.749 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:49:34.749 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T08:49:34.749 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:49:34.749 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T08:49:34.749 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T08:49:34.749 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:49:34.749 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:49:34.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:34.750 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:49:34.758 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:49:34.759 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41708sec preferred_lft 41708sec 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:49:34.760 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T08:49:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:49:34.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:34.763 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:49:34.829 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:49:34.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:34.831 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:49:34.884 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:49:34.885 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:49:34.885 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:49:34.885 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:49:34.885 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:49:34.885 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:49:34.885 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:49:34.885 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:49:34.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:49:34.886 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:49:34.886 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:49:34.886 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:49:34.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:49:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:49:35.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:35.024 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:49:35.080 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:49:35.080 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T08:49:35.081 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:49:35.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:35.081 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T08:49:35.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:49:35.148 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T08:49:35.203 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:49:35.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:49:35.203 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:49:35.258 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:49:35.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:35.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:49:35.348 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:49:35.349 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:49:35.349 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:49:35.364 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:49:35.364 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:49:35.365 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:49:35.365 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:49:35.365 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:49:35.373 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:49:35.373 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:49:35.373 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:49:35.373 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:49:35.373 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:49:35.374 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:49:35.374 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:49:35.374 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:49:35.377 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:49:35.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:35.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:49:35.397 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:49:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:49:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:49:35.508 INFO:teuthology.orchestra.run.smithi044.stdout:client7423 2024-09-13T08:49:35.508 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:49:35.508 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:35.517 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-13T08:49:35.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.707+0000 7f6152800640 1 -- 172.21.15.44:0/1280465944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6130004010 msgr2=0x7f6130004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:35.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.707+0000 7f6152800640 1 --2- 172.21.15.44:0/1280465944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6130004010 0x7f6130004810 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f613400ac70 tx=0x7f613401e090 comp rx=0 tx=0).stop 2024-09-13T08:49:35.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.707+0000 7f614be00640 1 -- 172.21.15.44:0/1280465944 reap_dead start 2024-09-13T08:49:35.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.707+0000 7f6152800640 1 -- 172.21.15.44:0/1280465944 shutdown_connections 2024-09-13T08:49:35.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.707+0000 7f6152800640 1 -- 172.21.15.44:0/1280465944 >> 172.21.15.44:0/1280465944 conn(0x7f614c0cfbc0 msgr2=0x7f614c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:35.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.707+0000 7f6152800640 1 -- 172.21.15.44:0/1280465944 shutdown_connections 2024-09-13T08:49:35.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.707+0000 7f6152800640 1 -- 172.21.15.44:0/1280465944 wait complete. 2024-09-13T08:49:35.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.707+0000 7f6152800640 1 Processor -- start 2024-09-13T08:49:35.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.707+0000 7f6152800640 1 -- start start 2024-09-13T08:49:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6152800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6130004010 0x7f614c143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6152800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f614c1f8f70 0x7f614c0b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6152800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c0b0aa0 0x7f614c0b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6152800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f614c06f6f0 con 0x7f614c0b0aa0 2024-09-13T08:49:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6152800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f614c06f570 con 0x7f614c1f8f70 2024-09-13T08:49:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6152800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f614c06f870 con 0x7f6130004010 2024-09-13T08:49:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f614be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c0b0aa0 0x7f614c0b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f614b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6130004010 0x7f614c143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:35.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f614be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c0b0aa0 0x7f614c0b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42636/0 (socket says 172.21.15.44:42636) 2024-09-13T08:49:35.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f614be00640 1 -- 172.21.15.44:0/3955251577 learned_addr learned my addr 172.21.15.44:0/3955251577 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:35.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6142a00640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f614c1f8f70 0x7f614c0b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f614be00640 1 -- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6130004010 msgr2=0x7f614c143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f614be00640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6130004010 0x7f614c143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f614be00640 1 -- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f614c1f8f70 msgr2=0x7f614c0b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f614be00640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f614c1f8f70 0x7f614c0b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f614be00640 1 -- 172.21.15.44:0/3955251577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0b5680 con 0x7f614c0b0aa0 2024-09-13T08:49:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6142a00640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f614c1f8f70 0x7f614c0b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f614be00640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c0b0aa0 0x7f614c0b0f20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f614c069890 tx=0x7f6144011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6143e00640 1 -- 172.21.15.44:0/3955251577 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614400f980 con 0x7f614c0b0aa0 2024-09-13T08:49:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6143e00640 1 -- 172.21.15.44:0/3955251577 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f614400fc10 con 0x7f614c0b0aa0 2024-09-13T08:49:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6143e00640 1 -- 172.21.15.44:0/3955251577 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144013450 con 0x7f614c0b0aa0 2024-09-13T08:49:35.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6152800640 1 -- 172.21.15.44:0/3955251577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c0b4b10 con 0x7f614c0b0aa0 2024-09-13T08:49:35.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6152800640 1 -- 172.21.15.44:0/3955251577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c0b50c0 con 0x7f614c0b0aa0 2024-09-13T08:49:35.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.711+0000 7f6152800640 1 -- 172.21.15.44:0/3955251577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614c0b1460 con 0x7f614c0b0aa0 2024-09-13T08:49:35.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.715+0000 7f6143e00640 1 -- 172.21.15.44:0/3955251577 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f61440135f0 con 0x7f614c0b0aa0 2024-09-13T08:49:35.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.715+0000 7f6143e00640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6128074570 0x7f6128076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:35.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.715+0000 7f6143e00640 1 -- 172.21.15.44:0/3955251577 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f6144096510 con 0x7f614c0b0aa0 2024-09-13T08:49:35.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.715+0000 7f614b400640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6128074570 0x7f6128076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:35.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.715+0000 7f614b400640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6128074570 0x7f6128076a30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f613c000c10 tx=0x7f613c006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:35.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.719+0000 7f6143e00640 1 -- 172.21.15.44:0/3955251577 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f61440631c0 con 0x7f614c0b0aa0 2024-09-13T08:49:35.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:35.927+0000 7f6152800640 1 -- 172.21.15.44:0/3955251577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f614c069410 con 0x7f614c0b0aa0 2024-09-13T08:49:36.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.319+0000 7f6143e00640 1 -- 172.21.15.44:0/3955251577 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v66) ==== 136+0+0 (secure 0 0 0) 0x7f6144062ac0 con 0x7f614c0b0aa0 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 -- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6128074570 msgr2=0x7f6128076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6128074570 0x7f6128076a30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f613c000c10 tx=0x7f613c006b80 comp rx=0 tx=0).stop 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 -- 172.21.15.44:0/3955251577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c0b0aa0 msgr2=0x7f614c0b0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c0b0aa0 0x7f614c0b0f20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f614c069890 tx=0x7f6144011a30 comp rx=0 tx=0).stop 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 -- 172.21.15.44:0/3955251577 shutdown_connections 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6128074570 0x7f6128076a30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c0b0aa0 0x7f614c0b0f20 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f614c1f8f70 0x7f614c0b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 --2- 172.21.15.44:0/3955251577 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6130004010 0x7f614c143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 -- 172.21.15.44:0/3955251577 >> 172.21.15.44:0/3955251577 conn(0x7f614c0cfbc0 msgr2=0x7f614c075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:36.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 -- 172.21.15.44:0/3955251577 shutdown_connections 2024-09-13T08:49:36.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.323+0000 7f6152800640 1 -- 172.21.15.44:0/3955251577 wait complete. 2024-09-13T08:49:36.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:49:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 -- 172.21.15.44:0/3683450763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f694815f7b0 msgr2=0x7f694815fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 --2- 172.21.15.44:0/3683450763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f694815f7b0 0x7f694815fc40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f694806ec30 tx=0x7f694001cd50 comp rx=0 tx=0).stop 2024-09-13T08:49:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 -- 172.21.15.44:0/3683450763 shutdown_connections 2024-09-13T08:49:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 --2- 172.21.15.44:0/3683450763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f694815f7b0 0x7f694815fc40 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 --2- 172.21.15.44:0/3683450763 >> v2:172.21.15.129:3300/0 conn(0x7f694815c1c0 0x7f694806d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 --2- 172.21.15.44:0/3683450763 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f694806cc20 0x7f69480737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 -- 172.21.15.44:0/3683450763 >> 172.21.15.44:0/3683450763 conn(0x7f69480d0c90 msgr2=0x7f69480d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 -- 172.21.15.44:0/3683450763 shutdown_connections 2024-09-13T08:49:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 -- 172.21.15.44:0/3683450763 wait complete. 2024-09-13T08:49:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 Processor -- start 2024-09-13T08:49:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.495+0000 7f694d400640 1 -- start start 2024-09-13T08:49:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f694d400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f694806cc20 0x7f69480afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f694d400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f694815c1c0 0x7f6948156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f694d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f694815f7b0 0x7f6948157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f694d400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69480719c0 con 0x7f694815f7b0 2024-09-13T08:49:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f694d400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6948071840 con 0x7f694806cc20 2024-09-13T08:49:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f694d400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6948071b40 con 0x7f694815c1c0 2024-09-13T08:49:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6947200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f694815f7b0 0x7f6948157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6947200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f694815f7b0 0x7f6948157530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42654/0 (socket says 172.21.15.44:42654) 2024-09-13T08:49:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6947200640 1 -- 172.21.15.44:0/2742162080 learned_addr learned my addr 172.21.15.44:0/2742162080 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6947200640 1 -- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f694815c1c0 msgr2=0x7f6948156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6947200640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f694815c1c0 0x7f6948156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6946800640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f694806cc20 0x7f69480afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6947200640 1 -- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f694806cc20 msgr2=0x7f69480afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6947200640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f694806cc20 0x7f69480afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6947200640 1 -- 172.21.15.44:0/2742162080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69481f7340 con 0x7f694815f7b0 2024-09-13T08:49:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6946800640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f694806cc20 0x7f69480afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:49:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f6947200640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f694815f7b0 0x7f6948157530 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f694001b400 tx=0x7f6940009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f693ea00640 1 -- 172.21.15.44:0/2742162080 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69400bb070 con 0x7f694815f7b0 2024-09-13T08:49:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f693ea00640 1 -- 172.21.15.44:0/2742162080 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6940004d40 con 0x7f694815f7b0 2024-09-13T08:49:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f694d400640 1 -- 172.21.15.44:0/2742162080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69481f7620 con 0x7f694815f7b0 2024-09-13T08:49:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f693ea00640 1 -- 172.21.15.44:0/2742162080 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69400b2040 con 0x7f694815f7b0 2024-09-13T08:49:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f694d400640 1 -- 172.21.15.44:0/2742162080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69481f6ff0 con 0x7f694815f7b0 2024-09-13T08:49:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f694d400640 1 -- 172.21.15.44:0/2742162080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694806d000 con 0x7f694815f7b0 2024-09-13T08:49:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.499+0000 7f693ea00640 1 -- 172.21.15.44:0/2742162080 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f69400be070 con 0x7f694815f7b0 2024-09-13T08:49:36.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.503+0000 7f693ea00640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6924074540 0x7f6924076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:36.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.503+0000 7f693ea00640 1 -- 172.21.15.44:0/2742162080 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f694012c790 con 0x7f694815f7b0 2024-09-13T08:49:36.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.503+0000 7f6946800640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6924074540 0x7f6924076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:36.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.503+0000 7f6946800640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6924074540 0x7f6924076a00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6938005fd0 tx=0x7f6938005d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:36.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.507+0000 7f693ea00640 1 -- 172.21.15.44:0/2742162080 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6940016970 con 0x7f694815f7b0 2024-09-13T08:49:36.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.703+0000 7f694d400640 1 -- 172.21.15.44:0/2742162080 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f69480b6c10 con 0x7f6924074540 2024-09-13T08:49:36.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f693ea00640 1 -- 172.21.15.44:0/2742162080 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f69480b6c10 con 0x7f6924074540 2024-09-13T08:49:36.707 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:49:36.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:49:36.708 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:49:36.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:49:36.708 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:49:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 -- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6924074540 msgr2=0x7f6924076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6924074540 0x7f6924076a00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6938005fd0 tx=0x7f6938005d40 comp rx=0 tx=0).stop 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 -- 172.21.15.44:0/2742162080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f694815f7b0 msgr2=0x7f6948157530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f694815f7b0 0x7f6948157530 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f694001b400 tx=0x7f6940009370 comp rx=0 tx=0).stop 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 -- 172.21.15.44:0/2742162080 shutdown_connections 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6924074540 0x7f6924076a00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f694815f7b0 0x7f6948157530 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f694815c1c0 0x7f6948156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 --2- 172.21.15.44:0/2742162080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f694806cc20 0x7f69480afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 -- 172.21.15.44:0/2742162080 >> 172.21.15.44:0/2742162080 conn(0x7f69480d0c90 msgr2=0x7f694815df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 -- 172.21.15.44:0/2742162080 shutdown_connections 2024-09-13T08:49:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.707+0000 7f694d400640 1 -- 172.21.15.44:0/2742162080 wait complete. 2024-09-13T08:49:36.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:49:36.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.875+0000 7f2f2f600640 1 -- 172.21.15.44:0/3352042548 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f280b2900 msgr2=0x7f2f280b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.875+0000 7f2f2f600640 1 --2- 172.21.15.44:0/3352042548 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f280b2900 0x7f2f280b3f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009bc0 tx=0x7f2f2001f3b0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 -- 172.21.15.44:0/3352042548 shutdown_connections 2024-09-13T08:49:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 --2- 172.21.15.44:0/3352042548 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f280b2900 0x7f2f280b3f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 --2- 172.21.15.44:0/3352042548 >> v2:172.21.15.129:3300/0 conn(0x7f2f280d02c0 0x7f2f280d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 -- 172.21.15.44:0/3352042548 >> 172.21.15.44:0/3352042548 conn(0x7f2f280d1d70 msgr2=0x7f2f280d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 -- 172.21.15.44:0/3352042548 shutdown_connections 2024-09-13T08:49:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 -- 172.21.15.44:0/3352042548 wait complete. 2024-09-13T08:49:36.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 Processor -- start 2024-09-13T08:49:36.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 -- start start 2024-09-13T08:49:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f280b2900 0x7f2f2814d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f280b2ec0 0x7f2f2814f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f280d02c0 0x7f2f280c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f280cdc80 con 0x7f2f280b2900 2024-09-13T08:49:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f280cdb00 con 0x7f2f280d02c0 2024-09-13T08:49:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2f280d2aa0 con 0x7f2f280b2ec0 2024-09-13T08:49:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f280b2900 0x7f2f2814d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f280b2900 0x7f2f2814d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42660/0 (socket says 172.21.15.44:42660) 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2cc00640 1 -- 172.21.15.44:0/2488762400 learned_addr learned my addr 172.21.15.44:0/2488762400 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f27e00640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f280b2ec0 0x7f2f2814f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2d600640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f280d02c0 0x7f2f280c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2cc00640 1 -- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f280b2ec0 msgr2=0x7f2f2814f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2cc00640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f280b2ec0 0x7f2f2814f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2cc00640 1 -- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f280d02c0 msgr2=0x7f2f280c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2cc00640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f280d02c0 0x7f2f280c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2cc00640 1 -- 172.21.15.44:0/2488762400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2814f9c0 con 0x7f2f280b2900 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2cc00640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f280b2900 0x7f2f2814d0e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00e7a0 tx=0x7f2f1c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f24c00640 1 -- 172.21.15.44:0/2488762400 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00ccc0 con 0x7f2f280b2900 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f24c00640 1 -- 172.21.15.44:0/2488762400 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f1c010430 con 0x7f2f280b2900 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f2f600640 1 -- 172.21.15.44:0/2488762400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2814fca0 con 0x7f2f280b2900 2024-09-13T08:49:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.879+0000 7f2f24c00640 1 -- 172.21.15.44:0/2488762400 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c010860 con 0x7f2f280b2900 2024-09-13T08:49:36.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.883+0000 7f2f2f600640 1 -- 172.21.15.44:0/2488762400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f280cafc0 con 0x7f2f280b2900 2024-09-13T08:49:36.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.883+0000 7f2f2f600640 1 -- 172.21.15.44:0/2488762400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f280b44e0 con 0x7f2f280b2900 2024-09-13T08:49:36.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.883+0000 7f2f24c00640 1 -- 172.21.15.44:0/2488762400 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f2f1c010a00 con 0x7f2f280b2900 2024-09-13T08:49:36.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.883+0000 7f2f24c00640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074540 0x7f2f08076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:36.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.883+0000 7f2f24c00640 1 -- 172.21.15.44:0/2488762400 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f2f1c093630 con 0x7f2f280b2900 2024-09-13T08:49:36.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.883+0000 7f2f27e00640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074540 0x7f2f08076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:36.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.887+0000 7f2f27e00640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074540 0x7f2f08076a00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2f280c8d10 tx=0x7f2f14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:36.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:36.887+0000 7f2f24c00640 1 -- 172.21.15.44:0/2488762400 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2f1c060260 con 0x7f2f280b2900 2024-09-13T08:49:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.071+0000 7f2f2f600640 1 -- 172.21.15.44:0/2488762400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2f280d0730 con 0x7f2f280b2900 2024-09-13T08:49:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.071+0000 7f2f24c00640 1 -- 172.21.15.44:0/2488762400 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f2f1c05fb60 con 0x7f2f280b2900 2024-09-13T08:49:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 -- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074540 msgr2=0x7f2f08076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074540 0x7f2f08076a00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2f280c8d10 tx=0x7f2f14009040 comp rx=0 tx=0).stop 2024-09-13T08:49:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 -- 172.21.15.44:0/2488762400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f280b2900 msgr2=0x7f2f2814d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f280b2900 0x7f2f2814d0e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00e7a0 tx=0x7f2f1c00ec60 comp rx=0 tx=0).stop 2024-09-13T08:49:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 -- 172.21.15.44:0/2488762400 shutdown_connections 2024-09-13T08:49:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f08074540 0x7f2f08076a00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f280d02c0 0x7f2f280c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f280b2ec0 0x7f2f2814f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 --2- 172.21.15.44:0/2488762400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f280b2900 0x7f2f2814d0e0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 -- 172.21.15.44:0/2488762400 >> 172.21.15.44:0/2488762400 conn(0x7f2f280d1d70 msgr2=0x7f2f280b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 -- 172.21.15.44:0/2488762400 shutdown_connections 2024-09-13T08:49:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.075+0000 7f2f2f600640 1 -- 172.21.15.44:0/2488762400 wait complete. 2024-09-13T08:49:37.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-13T08:49:37.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.243+0000 7f4860400640 1 -- 172.21.15.44:0/2747879651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4858074760 msgr2=0x7f48580cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.243+0000 7f4860400640 1 --2- 172.21.15.44:0/2747879651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4858074760 0x7f48580cdb20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4850009c80 tx=0x7f485001cbc0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.243+0000 7f4860400640 1 -- 172.21.15.44:0/2747879651 shutdown_connections 2024-09-13T08:49:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.243+0000 7f4860400640 1 --2- 172.21.15.44:0/2747879651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4858074760 0x7f48580cdb20 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.243+0000 7f4860400640 1 --2- 172.21.15.44:0/2747879651 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48581564d0 0x7f485815cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.243+0000 7f4860400640 1 -- 172.21.15.44:0/2747879651 >> 172.21.15.44:0/2747879651 conn(0x7f48580cfc20 msgr2=0x7f48580d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.243+0000 7f4860400640 1 -- 172.21.15.44:0/2747879651 shutdown_connections 2024-09-13T08:49:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.243+0000 7f4860400640 1 -- 172.21.15.44:0/2747879651 wait complete. 2024-09-13T08:49:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 Processor -- start 2024-09-13T08:49:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 -- start start 2024-09-13T08:49:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4858074760 0x7f485806f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48580cadf0 0x7f48580b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48581564d0 0x7f48580c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48580cec00 con 0x7f48581564d0 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48580cea80 con 0x7f48580cadf0 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f48580ced80 con 0x7f4858074760 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48581564d0 0x7f48580c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48581564d0 0x7f48580c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42686/0 (socket says 172.21.15.44:42686) 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485e400640 1 -- 172.21.15.44:0/1150664290 learned_addr learned my addr 172.21.15.44:0/1150664290 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485da00640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4858074760 0x7f485806f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485d000640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48580cadf0 0x7f48580b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485e400640 1 -- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4858074760 msgr2=0x7f485806f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485e400640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4858074760 0x7f485806f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485e400640 1 -- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48580cadf0 msgr2=0x7f48580b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485e400640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48580cadf0 0x7f48580b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485e400640 1 -- 172.21.15.44:0/1150664290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48580b3360 con 0x7f48581564d0 2024-09-13T08:49:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f485e400640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48581564d0 0x7f48580c8f70 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f4850009bd0 tx=0x7f48500a5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4855600640 1 -- 172.21.15.44:0/1150664290 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500bb070 con 0x7f48581564d0 2024-09-13T08:49:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4855600640 1 -- 172.21.15.44:0/1150664290 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4850004d40 con 0x7f48581564d0 2024-09-13T08:49:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4855600640 1 -- 172.21.15.44:0/1150664290 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500b2040 con 0x7f48581564d0 2024-09-13T08:49:37.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 -- 172.21.15.44:0/1150664290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48580b3640 con 0x7f48581564d0 2024-09-13T08:49:37.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 -- 172.21.15.44:0/1150664290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48580b3c00 con 0x7f48581564d0 2024-09-13T08:49:37.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.247+0000 7f4860400640 1 -- 172.21.15.44:0/1150664290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48580cdb20 con 0x7f48581564d0 2024-09-13T08:49:37.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.251+0000 7f4855600640 1 -- 172.21.15.44:0/1150664290 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f48500be070 con 0x7f48581564d0 2024-09-13T08:49:37.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.251+0000 7f4855600640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4838074540 0x7f4838076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.251+0000 7f4855600640 1 -- 172.21.15.44:0/1150664290 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f485012c8a0 con 0x7f48581564d0 2024-09-13T08:49:37.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.251+0000 7f485da00640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4838074540 0x7f4838076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.251+0000 7f485da00640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4838074540 0x7f4838076a00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f484c005fd0 tx=0x7f484c005d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:37.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.255+0000 7f4855600640 1 -- 172.21.15.44:0/1150664290 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f485815d310 con 0x7f48581564d0 2024-09-13T08:49:37.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 -- 172.21.15.44:0/1150664290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f48580cf230 con 0x7f48581564d0 2024-09-13T08:49:37.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4855600640 1 -- 172.21.15.44:0/1150664290 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v45) ==== 116+0+2 (secure 0 0 0) 0x7f48500f9550 con 0x7f48581564d0 2024-09-13T08:49:37.437 INFO:teuthology.orchestra.run.smithi044.stdout:4 2024-09-13T08:49:37.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 -- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4838074540 msgr2=0x7f4838076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4838074540 0x7f4838076a00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f484c005fd0 tx=0x7f484c005d10 comp rx=0 tx=0).stop 2024-09-13T08:49:37.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 -- 172.21.15.44:0/1150664290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48581564d0 msgr2=0x7f48580c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48581564d0 0x7f48580c8f70 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f4850009bd0 tx=0x7f48500a5920 comp rx=0 tx=0).stop 2024-09-13T08:49:37.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 -- 172.21.15.44:0/1150664290 shutdown_connections 2024-09-13T08:49:37.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4838074540 0x7f4838076a00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48581564d0 0x7f48580c8f70 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48580cadf0 0x7f48580b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 --2- 172.21.15.44:0/1150664290 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4858074760 0x7f485806f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 -- 172.21.15.44:0/1150664290 >> 172.21.15.44:0/1150664290 conn(0x7f48580cfc20 msgr2=0x7f485815ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 -- 172.21.15.44:0/1150664290 shutdown_connections 2024-09-13T08:49:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.435+0000 7f4860400640 1 -- 172.21.15.44:0/1150664290 wait complete. 2024-09-13T08:49:37.459 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:49:37.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.607+0000 7f1675800640 1 -- 172.21.15.44:0/444100742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1670158a20 msgr2=0x7f167006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.607+0000 7f1675800640 1 --2- 172.21.15.44:0/444100742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1670158a20 0x7f167006ecb0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f1660009900 tx=0x7f166001c8e0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.607+0000 7f1675800640 1 -- 172.21.15.44:0/444100742 shutdown_connections 2024-09-13T08:49:37.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.607+0000 7f1675800640 1 --2- 172.21.15.44:0/444100742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1670158a20 0x7f167006ecb0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.607+0000 7f1675800640 1 -- 172.21.15.44:0/444100742 >> 172.21.15.44:0/444100742 conn(0x7f16700b18f0 msgr2=0x7f16700b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:37.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.607+0000 7f1675800640 1 -- 172.21.15.44:0/444100742 shutdown_connections 2024-09-13T08:49:37.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.607+0000 7f1675800640 1 -- 172.21.15.44:0/444100742 wait complete. 2024-09-13T08:49:37.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.607+0000 7f1675800640 1 Processor -- start 2024-09-13T08:49:37.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.607+0000 7f1675800640 1 -- start start 2024-09-13T08:49:37.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1675800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16700b2bf0 0x7f16701529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1675800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16700b4a60 0x7f167015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1675800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1670158a20 0x7f16700738c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1675800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16701691d0 con 0x7f1670158a20 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1675800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1670169050 con 0x7f16700b4a60 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1675800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1670169350 con 0x7f16700b2bf0 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1670158a20 0x7f16700738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1670158a20 0x7f16700738c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42692/0 (socket says 172.21.15.44:42692) 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166f400640 1 -- 172.21.15.44:0/3677203666 learned_addr learned my addr 172.21.15.44:0/3677203666 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166ea00640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16700b2bf0 0x7f16701529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166e000640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16700b4a60 0x7f167015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166f400640 1 -- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16700b2bf0 msgr2=0x7f16701529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166f400640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16700b2bf0 0x7f16701529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166f400640 1 -- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16700b4a60 msgr2=0x7f167015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166f400640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16700b4a60 0x7f167015c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166f400640 1 -- 172.21.15.44:0/3677203666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1670165ee0 con 0x7f1670158a20 2024-09-13T08:49:37.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166ea00640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16700b2bf0 0x7f16701529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:49:37.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f166f400640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1670158a20 0x7f16700738c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f166800c780 tx=0x7f166800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:37.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1666a00640 1 -- 172.21.15.44:0/3677203666 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668017070 con 0x7f1670158a20 2024-09-13T08:49:37.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1666a00640 1 -- 172.21.15.44:0/3677203666 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1668005070 con 0x7f1670158a20 2024-09-13T08:49:37.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1666a00640 1 -- 172.21.15.44:0/3677203666 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16680054c0 con 0x7f1670158a20 2024-09-13T08:49:37.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1675800640 1 -- 172.21.15.44:0/3677203666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16701661c0 con 0x7f1670158a20 2024-09-13T08:49:37.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1675800640 1 -- 172.21.15.44:0/3677203666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1670166780 con 0x7f1670158a20 2024-09-13T08:49:37.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.611+0000 7f1675800640 1 -- 172.21.15.44:0/3677203666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1670069410 con 0x7f1670158a20 2024-09-13T08:49:37.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.615+0000 7f1666a00640 1 -- 172.21.15.44:0/3677203666 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7f1668005660 con 0x7f1670158a20 2024-09-13T08:49:37.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.615+0000 7f1666a00640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1650074540 0x7f1650076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.615+0000 7f1666a00640 1 -- 172.21.15.44:0/3677203666 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f16680920a0 con 0x7f1670158a20 2024-09-13T08:49:37.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.615+0000 7f166ea00640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1650074540 0x7f1650076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.615+0000 7f166ea00640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1650074540 0x7f1650076a00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f16600af000 tx=0x7f166001b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:37.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.619+0000 7f1666a00640 1 -- 172.21.15.44:0/3677203666 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f166805ed50 con 0x7f1670158a20 2024-09-13T08:49:37.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.795+0000 7f1675800640 1 -- 172.21.15.44:0/3677203666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f16700b1340 con 0x7f1670158a20 2024-09-13T08:49:37.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1666a00640 1 -- 172.21.15.44:0/3677203666 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7f166805e650 con 0x7f1670158a20 2024-09-13T08:49:37.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 -- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1650074540 msgr2=0x7f1650076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1650074540 0x7f1650076a00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f16600af000 tx=0x7f166001b120 comp rx=0 tx=0).stop 2024-09-13T08:49:37.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 -- 172.21.15.44:0/3677203666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1670158a20 msgr2=0x7f16700738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1670158a20 0x7f16700738c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f166800c780 tx=0x7f166800cc40 comp rx=0 tx=0).stop 2024-09-13T08:49:37.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 -- 172.21.15.44:0/3677203666 shutdown_connections 2024-09-13T08:49:37.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1650074540 0x7f1650076a00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1670158a20 0x7f16700738c0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16700b4a60 0x7f167015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 --2- 172.21.15.44:0/3677203666 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f16700b2bf0 0x7f16701529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 -- 172.21.15.44:0/3677203666 >> 172.21.15.44:0/3677203666 conn(0x7f16700b18f0 msgr2=0x7f1670075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:37.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 -- 172.21.15.44:0/3677203666 shutdown_connections 2024-09-13T08:49:37.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.799+0000 7f1675800640 1 -- 172.21.15.44:0/3677203666 wait complete. 2024-09-13T08:49:37.820 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:49:37.821+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:49:37.820 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:49:37.821+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:49:37.820 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:49:37.821+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:49:37.820 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:49:37.821+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:49:37.821 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:49:37.821+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:49:37.824 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-13T08:49:37.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.971+0000 7fee03a00640 1 -- 172.21.15.44:0/3702017926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede4003ab0 msgr2=0x7fede4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.971+0000 7fee03a00640 1 --2- 172.21.15.44:0/3702017926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede4003ab0 0x7fede4003ed0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fedf00016c0 tx=0x7fedf001c730 comp rx=0 tx=0).stop 2024-09-13T08:49:37.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.971+0000 7fee03a00640 1 -- 172.21.15.44:0/3702017926 shutdown_connections 2024-09-13T08:49:37.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.971+0000 7fee03a00640 1 --2- 172.21.15.44:0/3702017926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede4003ab0 0x7fede4003ed0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.971+0000 7fee03a00640 1 -- 172.21.15.44:0/3702017926 >> 172.21.15.44:0/3702017926 conn(0x7fedfc0d1da0 msgr2=0x7fedfc0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:37.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.971+0000 7fee03a00640 1 -- 172.21.15.44:0/3702017926 shutdown_connections 2024-09-13T08:49:37.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.971+0000 7fee03a00640 1 -- 172.21.15.44:0/3702017926 wait complete. 2024-09-13T08:49:37.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee03a00640 1 Processor -- start 2024-09-13T08:49:37.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee03a00640 1 -- start start 2024-09-13T08:49:37.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee03a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fede4003ab0 0x7fedfc14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee03a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0c7ab0 0x7fedfc0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee03a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fedfc0b4bf0 0x7fedfc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee03a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0ceb40 con 0x7fedfc0c7ab0 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee03a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0ce9c0 con 0x7fede4003ab0 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee03a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc0cecc0 con 0x7fedfc0b4bf0 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedfbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0c7ab0 0x7fedfc0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedfbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0c7ab0 0x7fedfc0c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42704/0 (socket says 172.21.15.44:42704) 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedfbe00640 1 -- 172.21.15.44:0/3059968125 learned_addr learned my addr 172.21.15.44:0/3059968125 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee01000640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fede4003ab0 0x7fedfc14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee01a00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fedfc0b4bf0 0x7fedfc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedfbe00640 1 -- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fedfc0b4bf0 msgr2=0x7fedfc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedfbe00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fedfc0b4bf0 0x7fedfc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedfbe00640 1 -- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fede4003ab0 msgr2=0x7fedfc14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:37.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedfbe00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fede4003ab0 0x7fedfc14d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:37.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedfbe00640 1 -- 172.21.15.44:0/3059968125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc1546b0 con 0x7fedfc0c7ab0 2024-09-13T08:49:37.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedfbe00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0c7ab0 0x7fedfc0c7f10 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fede80126c0 tx=0x7fede8012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:37.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedf8c00640 1 -- 172.21.15.44:0/3059968125 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8010b40 con 0x7fedfc0c7ab0 2024-09-13T08:49:37.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedf8c00640 1 -- 172.21.15.44:0/3059968125 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fede8017030 con 0x7fedfc0c7ab0 2024-09-13T08:49:37.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee03a00640 1 -- 172.21.15.44:0/3059968125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc0c58a0 con 0x7fedfc0c7ab0 2024-09-13T08:49:37.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fedf8c00640 1 -- 172.21.15.44:0/3059968125 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8013450 con 0x7fedfc0c7ab0 2024-09-13T08:49:37.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.975+0000 7fee03a00640 1 -- 172.21.15.44:0/3059968125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc0b5bc0 con 0x7fedfc0c7ab0 2024-09-13T08:49:37.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.979+0000 7fee03a00640 1 -- 172.21.15.44:0/3059968125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedfc0cf220 con 0x7fedfc0c7ab0 2024-09-13T08:49:37.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.979+0000 7fedf8c00640 1 -- 172.21.15.44:0/3059968125 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fede80135f0 con 0x7fedfc0c7ab0 2024-09-13T08:49:37.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.979+0000 7fedf8c00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feddc074540 0x7feddc076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:37.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.979+0000 7fedf8c00640 1 -- 172.21.15.44:0/3059968125 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7fede8096710 con 0x7fedfc0c7ab0 2024-09-13T08:49:37.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.979+0000 7fee01000640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feddc074540 0x7feddc076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:37.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.979+0000 7fee01000640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feddc074540 0x7feddc076a00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fede4004790 tx=0x7fedf0019520 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:37.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:37.983+0000 7fedf8c00640 1 -- 172.21.15.44:0/3059968125 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fede8062d70 con 0x7fedfc0c7ab0 2024-09-13T08:49:38.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.187+0000 7fee03a00640 1 -- 172.21.15.44:0/3059968125 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fedfc070350 con 0x7feddc074540 2024-09-13T08:49:38.200 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:49:38.201+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:49:38.200 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:49:38.201+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:49:38.200 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:49:38.201+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:49:38.200 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:49:38.201+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:49:38.200 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:49:38.201+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:49:38.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fedf8c00640 1 -- 172.21.15.44:0/3059968125 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fedfc070350 con 0x7feddc074540 2024-09-13T08:49:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 -- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feddc074540 msgr2=0x7feddc076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feddc074540 0x7feddc076a00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fede4004790 tx=0x7fedf0019520 comp rx=0 tx=0).stop 2024-09-13T08:49:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 -- 172.21.15.44:0/3059968125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0c7ab0 msgr2=0x7fedfc0c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0c7ab0 0x7fedfc0c7f10 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fede80126c0 tx=0x7fede8012b80 comp rx=0 tx=0).stop 2024-09-13T08:49:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 -- 172.21.15.44:0/3059968125 shutdown_connections 2024-09-13T08:49:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7feddc074540 0x7feddc076a00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fedfc0b4bf0 0x7fedfc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0c7ab0 0x7fedfc0c7f10 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 --2- 172.21.15.44:0/3059968125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fede4003ab0 0x7fedfc14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 -- 172.21.15.44:0/3059968125 >> 172.21.15.44:0/3059968125 conn(0x7fedfc0d1da0 msgr2=0x7fedfc0b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 -- 172.21.15.44:0/3059968125 shutdown_connections 2024-09-13T08:49:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.235+0000 7fee03a00640 1 -- 172.21.15.44:0/3059968125 wait complete. 2024-09-13T08:49:38.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:49:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.407+0000 7fcb0a600640 1 -- 172.21.15.44:0/402472690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaec003a10 msgr2=0x7fcaec003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.407+0000 7fcb0a600640 1 --2- 172.21.15.44:0/402472690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaec003a10 0x7fcaec003df0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8001770 tx=0x7fcaf801c820 comp rx=0 tx=0).stop 2024-09-13T08:49:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.407+0000 7fcb0a600640 1 -- 172.21.15.44:0/402472690 shutdown_connections 2024-09-13T08:49:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.407+0000 7fcb0a600640 1 --2- 172.21.15.44:0/402472690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaec003a10 0x7fcaec003df0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.407+0000 7fcb0a600640 1 -- 172.21.15.44:0/402472690 >> 172.21.15.44:0/402472690 conn(0x7fcb0409aa10 msgr2=0x7fcb0409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.407+0000 7fcb0a600640 1 -- 172.21.15.44:0/402472690 shutdown_connections 2024-09-13T08:49:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.407+0000 7fcb0a600640 1 -- 172.21.15.44:0/402472690 wait complete. 2024-09-13T08:49:38.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.407+0000 7fcb0a600640 1 Processor -- start 2024-09-13T08:49:38.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.407+0000 7fcb0a600640 1 -- start start 2024-09-13T08:49:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb0a600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcaec003a10 0x7fcb041f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:38.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb0a600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406ed40 0x7fcb0406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb0a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb0415e160 0x7fcb0415e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb0a600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040d0870 con 0x7fcb0415e160 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb0a600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040d06f0 con 0x7fcb0406ed40 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb0a600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcb040ccd60 con 0x7fcaec003a10 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb09c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb0415e160 0x7fcb0415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb09c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb0415e160 0x7fcb0415e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42720/0 (socket says 172.21.15.44:42720) 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb09c00640 1 -- 172.21.15.44:0/2342778139 learned_addr learned my addr 172.21.15.44:0/2342778139 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb09200640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcaec003a10 0x7fcb041f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb03e00640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406ed40 0x7fcb0406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb09c00640 1 -- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcaec003a10 msgr2=0x7fcb041f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb09c00640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcaec003a10 0x7fcb041f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb09c00640 1 -- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406ed40 msgr2=0x7fcb0406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb09c00640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406ed40 0x7fcb0406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb09c00640 1 -- 172.21.15.44:0/2342778139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb04152a80 con 0x7fcb0415e160 2024-09-13T08:49:38.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb09c00640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb0415e160 0x7fcb0415e610 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fcb04070240 tx=0x7fcafc010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:38.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb00c00640 1 -- 172.21.15.44:0/2342778139 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00ee40 con 0x7fcb0415e160 2024-09-13T08:49:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb00c00640 1 -- 172.21.15.44:0/2342778139 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcafc015070 con 0x7fcb0415e160 2024-09-13T08:49:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb00c00640 1 -- 172.21.15.44:0/2342778139 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc003d10 con 0x7fcb0415e160 2024-09-13T08:49:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb0a600640 1 -- 172.21.15.44:0/2342778139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0406df80 con 0x7fcb0415e160 2024-09-13T08:49:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb0a600640 1 -- 172.21.15.44:0/2342778139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0406e540 con 0x7fcb0415e160 2024-09-13T08:49:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.411+0000 7fcb0a600640 1 -- 172.21.15.44:0/2342778139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb040cd2c0 con 0x7fcb0415e160 2024-09-13T08:49:38.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.415+0000 7fcb00c00640 1 -- 172.21.15.44:0/2342778139 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 99120+0+0 (secure 0 0 0) 0x7fcafc004030 con 0x7fcb0415e160 2024-09-13T08:49:38.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.415+0000 7fcb00c00640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae4074540 0x7fcae4076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:49:38.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.415+0000 7fcb00c00640 1 -- 172.21.15.44:0/2342778139 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7fcafc093220 con 0x7fcb0415e160 2024-09-13T08:49:38.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.415+0000 7fcb09200640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae4074540 0x7fcae4076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:49:38.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.415+0000 7fcb09200640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae4074540 0x7fcae4076a00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8005000 tx=0x7fcaf80195c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:49:38.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.419+0000 7fcb00c00640 1 -- 172.21.15.44:0/2342778139 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcafc05f880 con 0x7fcb0415e160 2024-09-13T08:49:38.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.619+0000 7fcb0a600640 1 -- 172.21.15.44:0/2342778139 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fcb0406dd30 con 0x7fcae4074540 2024-09-13T08:49:38.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb00c00640 1 -- 172.21.15.44:0/2342778139 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fcaf80a9050 con 0x7fcae4074540 2024-09-13T08:49:38.628 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/24f33b94-a680-4f9a-9f2a-22efda418d40 2024-09-13T08:49:38.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 -- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae4074540 msgr2=0x7fcae4076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:38.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae4074540 0x7fcae4076a00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8005000 tx=0x7fcaf80195c0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 -- 172.21.15.44:0/2342778139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb0415e160 msgr2=0x7fcb0415e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:49:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb0415e160 0x7fcb0415e610 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fcb04070240 tx=0x7fcafc010f20 comp rx=0 tx=0).stop 2024-09-13T08:49:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 -- 172.21.15.44:0/2342778139 shutdown_connections 2024-09-13T08:49:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcae4074540 0x7fcae4076a00 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb0415e160 0x7fcb0415e610 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb0406ed40 0x7fcb0406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 --2- 172.21.15.44:0/2342778139 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcaec003a10 0x7fcb041f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:49:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 -- 172.21.15.44:0/2342778139 >> 172.21.15.44:0/2342778139 conn(0x7fcb0409aa10 msgr2=0x7fcb041597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:49:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 -- 172.21.15.44:0/2342778139 shutdown_connections 2024-09-13T08:49:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:49:38.627+0000 7fcb0a600640 1 -- 172.21.15.44:0/2342778139 wait complete. 2024-09-13T08:49:38.650 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/24f33b94-a680-4f9a-9f2a-22efda418d40 2024-09-13T08:49:38.650 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:49:38.650 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:38.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:38.659 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/24f33b94-a680-4f9a-9f2a-22efda418d40/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:49:48.127 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:49:48.127 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:49:48.127 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.40874 s, 114 MB/s 2024-09-13T08:49:48.128 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:49:48.129 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:48.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:48.138 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/24f33b94-a680-4f9a-9f2a-22efda418d40/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:49:57.864 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:49:57.864 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:49:57.864 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.67543 s, 111 MB/s 2024-09-13T08:49:57.865 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:49:57.865 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:49:57.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:49:57.874 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/24f33b94-a680-4f9a-9f2a-22efda418d40/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:50:02.692 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:50:02.697 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:50:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:50:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:50:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.21064 s, 117 MB/s 2024-09-13T08:50:07.136 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-13T08:50:07.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 -- 172.21.15.44:0/1106978137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef80d28e0 msgr2=0x7f9ef80d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:07.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 --2- 172.21.15.44:0/1106978137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef80d28e0 0x7f9ef80d0170 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80746b0 tx=0x7f9ee001c7d0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 -- 172.21.15.44:0/1106978137 shutdown_connections 2024-09-13T08:50:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 --2- 172.21.15.44:0/1106978137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef80d28e0 0x7f9ef80d0170 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 --2- 172.21.15.44:0/1106978137 >> v2:172.21.15.129:3300/0 conn(0x7f9ef815f390 0x7f9ef80cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 -- 172.21.15.44:0/1106978137 >> 172.21.15.44:0/1106978137 conn(0x7f9ef80d1dc0 msgr2=0x7f9ef80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 -- 172.21.15.44:0/1106978137 shutdown_connections 2024-09-13T08:50:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 -- 172.21.15.44:0/1106978137 wait complete. 2024-09-13T08:50:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 Processor -- start 2024-09-13T08:50:07.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 -- start start 2024-09-13T08:50:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ef80d06b0 0x7f9ef8154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ef80d28e0 0x7f9ef8159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef815f390 0x7f9ef814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef8168f80 con 0x7f9ef815f390 2024-09-13T08:50:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef8168e00 con 0x7f9ef80d06b0 2024-09-13T08:50:07.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.286+0000 7f9efe000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef8169100 con 0x7f9ef80d28e0 2024-09-13T08:50:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9ef7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef815f390 0x7f9ef814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9ef7400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ef80d06b0 0x7f9ef8154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9ef7400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ef80d06b0 0x7f9ef8154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:34088/0 (socket says 172.21.15.44:34088) 2024-09-13T08:50:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9ef7400640 1 -- 172.21.15.44:0/627641023 learned_addr learned my addr 172.21.15.44:0/627641023 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9eefe00640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ef80d28e0 0x7f9ef8159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9ef7e00640 1 -- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ef80d28e0 msgr2=0x7f9ef8159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9ef7e00640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ef80d28e0 0x7f9ef8159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9ef7e00640 1 -- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ef80d06b0 msgr2=0x7f9ef8154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9ef7e00640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ef80d06b0 0x7f9ef8154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9ef7e00640 1 -- 172.21.15.44:0/627641023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef80ce270 con 0x7f9ef815f390 2024-09-13T08:50:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9ef7e00640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef815f390 0x7f9ef814fe80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000cb10 tx=0x7f9ef000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9eef400640 1 -- 172.21.15.44:0/627641023 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0016020 con 0x7f9ef815f390 2024-09-13T08:50:07.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9eef400640 1 -- 172.21.15.44:0/627641023 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ef000ce50 con 0x7f9ef815f390 2024-09-13T08:50:07.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9eef400640 1 -- 172.21.15.44:0/627641023 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef000ae00 con 0x7f9ef815f390 2024-09-13T08:50:07.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9efe000640 1 -- 172.21.15.44:0/627641023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef80ce550 con 0x7f9ef815f390 2024-09-13T08:50:07.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9efe000640 1 -- 172.21.15.44:0/627641023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef80ceb10 con 0x7f9ef815f390 2024-09-13T08:50:07.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.290+0000 7f9efe000640 1 -- 172.21.15.44:0/627641023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef80b1350 con 0x7f9ef815f390 2024-09-13T08:50:07.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.294+0000 7f9eef400640 1 -- 172.21.15.44:0/627641023 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 42) ==== 99222+0+0 (secure 0 0 0) 0x7f9ef0019020 con 0x7f9ef815f390 2024-09-13T08:50:07.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.294+0000 7f9eef400640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ed4074660 0x7f9ed4076b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:07.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.294+0000 7f9eef400640 1 -- 172.21.15.44:0/627641023 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f9ef0092940 con 0x7f9ef815f390 2024-09-13T08:50:07.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.294+0000 7f9ef7400640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ed4074660 0x7f9ed4076b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:07.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.294+0000 7f9ef7400640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ed4074660 0x7f9ed4076b20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009c00 tx=0x7f9ee80091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:07.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.298+0000 7f9eef400640 1 -- 172.21.15.44:0/627641023 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9ef0020030 con 0x7f9ef815f390 2024-09-13T08:50:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.506+0000 7f9efe000640 1 -- 172.21.15.44:0/627641023 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f9ef8157900 con 0x7f9ed4074660 2024-09-13T08:50:07.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9eef400640 1 -- 172.21.15.44:0/627641023 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ef8157900 con 0x7f9ed4074660 2024-09-13T08:50:07.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 -- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ed4074660 msgr2=0x7f9ed4076b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:07.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ed4074660 0x7f9ed4076b20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009c00 tx=0x7f9ee80091e0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 -- 172.21.15.44:0/627641023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef815f390 msgr2=0x7f9ef814fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:07.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef815f390 0x7f9ef814fe80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000cb10 tx=0x7f9ef000cc00 comp rx=0 tx=0).stop 2024-09-13T08:50:07.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 -- 172.21.15.44:0/627641023 shutdown_connections 2024-09-13T08:50:07.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9ed4074660 0x7f9ed4076b20 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef815f390 0x7f9ef814fe80 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ef80d28e0 0x7f9ef8159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 --2- 172.21.15.44:0/627641023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ef80d06b0 0x7f9ef8154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 -- 172.21.15.44:0/627641023 >> 172.21.15.44:0/627641023 conn(0x7f9ef80d1dc0 msgr2=0x7f9ef80b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:07.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 -- 172.21.15.44:0/627641023 shutdown_connections 2024-09-13T08:50:07.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.518+0000 7f9efe000640 1 -- 172.21.15.44:0/627641023 wait complete. 2024-09-13T08:50:07.542 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:50:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 -- 172.21.15.44:0/2478640994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b940b1650 msgr2=0x7f9b940b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 --2- 172.21.15.44:0/2478640994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b940b1650 0x7f9b940b1a30 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009840 tx=0x7f9b7c01c820 comp rx=0 tx=0).stop 2024-09-13T08:50:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 -- 172.21.15.44:0/2478640994 shutdown_connections 2024-09-13T08:50:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 --2- 172.21.15.44:0/2478640994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b940b1650 0x7f9b940b1a30 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 -- 172.21.15.44:0/2478640994 >> 172.21.15.44:0/2478640994 conn(0x7f9b940d1dd0 msgr2=0x7f9b940d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 -- 172.21.15.44:0/2478640994 shutdown_connections 2024-09-13T08:50:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 -- 172.21.15.44:0/2478640994 wait complete. 2024-09-13T08:50:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 Processor -- start 2024-09-13T08:50:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 -- start start 2024-09-13T08:50:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b940afb10 0x7f9b94158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b940b1650 0x7f9b940b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9b941605d0 0x7f9b9415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9b940b1220 con 0x7f9b940b1650 2024-09-13T08:50:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b940b0ee0 con 0x7f9b940afb10 2024-09-13T08:50:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b9b200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9b940b13a0 con 0x7f9b941605d0 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b940b1650 0x7f9b940b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b940b1650 0x7f9b940b6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47306/0 (socket says 172.21.15.44:47306) 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b8be00640 1 -- 172.21.15.44:0/737779791 learned_addr learned my addr 172.21.15.44:0/737779791 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b93e00640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b940afb10 0x7f9b94158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.690+0000 7f9b99200640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9b941605d0 0x7f9b9415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b8be00640 1 -- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9b941605d0 msgr2=0x7f9b9415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b8be00640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9b941605d0 0x7f9b9415a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b8be00640 1 -- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b940afb10 msgr2=0x7f9b94158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b8be00640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b940afb10 0x7f9b94158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b8be00640 1 -- 172.21.15.44:0/737779791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b940b5da0 con 0x7f9b940b1650 2024-09-13T08:50:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b8be00640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b940b1650 0x7f9b940b6e40 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009790 tx=0x7f9b7c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:07.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b90c00640 1 -- 172.21.15.44:0/737779791 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c004430 con 0x7f9b940b1650 2024-09-13T08:50:07.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b90c00640 1 -- 172.21.15.44:0/737779791 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b7c005110 con 0x7f9b940b1650 2024-09-13T08:50:07.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b9b200640 1 -- 172.21.15.44:0/737779791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b940b6080 con 0x7f9b940b1650 2024-09-13T08:50:07.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b90c00640 1 -- 172.21.15.44:0/737779791 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c005560 con 0x7f9b940b1650 2024-09-13T08:50:07.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b9b200640 1 -- 172.21.15.44:0/737779791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b940b6640 con 0x7f9b940b1650 2024-09-13T08:50:07.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b9b200640 1 -- 172.21.15.44:0/737779791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b940affc0 con 0x7f9b940b1650 2024-09-13T08:50:07.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b90c00640 1 -- 172.21.15.44:0/737779791 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 42) ==== 99222+0+0 (secure 0 0 0) 0x7f9b7c0c1060 con 0x7f9b940b1650 2024-09-13T08:50:07.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b90c00640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b70074690 0x7f9b70076b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:07.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.694+0000 7f9b90c00640 1 -- 172.21.15.44:0/737779791 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f9b7c12cf70 con 0x7f9b940b1650 2024-09-13T08:50:07.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.698+0000 7f9b93e00640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b70074690 0x7f9b70076b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:07.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.698+0000 7f9b93e00640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b70074690 0x7f9b70076b50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9b84000f70 tx=0x7f9b84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:07.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.698+0000 7f9b90c00640 1 -- 172.21.15.44:0/737779791 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9b7c016410 con 0x7f9b940b1650 2024-09-13T08:50:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.902+0000 7f9b9b200640 1 -- 172.21.15.44:0/737779791 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f9b94166bd0 con 0x7f9b70074690 2024-09-13T08:50:07.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b90c00640 1 -- 172.21.15.44:0/737779791 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f9b94166bd0 con 0x7f9b70074690 2024-09-13T08:50:07.908 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/24f33b94-a680-4f9a-9f2a-22efda418d40 2024-09-13T08:50:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 -- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b70074690 msgr2=0x7f9b70076b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b70074690 0x7f9b70076b50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9b84000f70 tx=0x7f9b84009040 comp rx=0 tx=0).stop 2024-09-13T08:50:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 -- 172.21.15.44:0/737779791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b940b1650 msgr2=0x7f9b940b6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b940b1650 0x7f9b940b6e40 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009790 tx=0x7f9b7c004040 comp rx=0 tx=0).stop 2024-09-13T08:50:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 -- 172.21.15.44:0/737779791 shutdown_connections 2024-09-13T08:50:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9b70074690 0x7f9b70076b50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9b941605d0 0x7f9b9415a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b940b1650 0x7f9b940b6e40 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 --2- 172.21.15.44:0/737779791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b940afb10 0x7f9b94158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 -- 172.21.15.44:0/737779791 >> 172.21.15.44:0/737779791 conn(0x7f9b940d1dd0 msgr2=0x7f9b9415c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:07.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 -- 172.21.15.44:0/737779791 shutdown_connections 2024-09-13T08:50:07.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:07.906+0000 7f9b9b200640 1 -- 172.21.15.44:0/737779791 wait complete. 2024-09-13T08:50:07.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:07.930 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/24f33b94-a680-4f9a-9f2a-22efda418d40 2024-09-13T08:50:07.930 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:50:07.939 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/24f33b94-a680-4f9a-9f2a-22efda418d40 2024-09-13T08:50:07.939 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3055190016" 2024-09-13T08:50:07.940 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:07.940 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-13T08:50:09.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:09.941 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/24f33b94-a680-4f9a-9f2a-22efda418d40 2024-09-13T08:50:09.942 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:50:09.952 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/24f33b94-a680-4f9a-9f2a-22efda418d40 2024-09-13T08:50:09.953 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3221225472" 2024-09-13T08:50:09.953 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:09.954 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-13T08:50:09.954 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-13T08:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.154+0000 7f3d71000640 1 -- 172.21.15.44:0/511922685 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d6c0b25e0 msgr2=0x7f3d6c166f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.154+0000 7f3d71000640 1 --2- 172.21.15.44:0/511922685 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d6c0b25e0 0x7f3d6c166f00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009870 tx=0x7f3d6401c830 comp rx=0 tx=0).stop 2024-09-13T08:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.154+0000 7f3d71000640 1 -- 172.21.15.44:0/511922685 shutdown_connections 2024-09-13T08:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.154+0000 7f3d71000640 1 --2- 172.21.15.44:0/511922685 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d6c0b25e0 0x7f3d6c166f00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.154+0000 7f3d71000640 1 -- 172.21.15.44:0/511922685 >> 172.21.15.44:0/511922685 conn(0x7f3d6c0d1cf0 msgr2=0x7f3d6c0d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.154+0000 7f3d71000640 1 -- 172.21.15.44:0/511922685 shutdown_connections 2024-09-13T08:50:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 -- 172.21.15.44:0/511922685 wait complete. 2024-09-13T08:50:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 Processor -- start 2024-09-13T08:50:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 -- start start 2024-09-13T08:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d6c0b25e0 0x7f3d6c0c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d6c0cfd30 0x7f3d6c0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d6c0d2810 0x7f3d6c0b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c16a320 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c16a1a0 con 0x7f3d6c0d2810 2024-09-13T08:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c16a4a0 con 0x7f3d6c0b25e0 2024-09-13T08:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d6c0cfd30 0x7f3d6c0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d6c0cfd30 0x7f3d6c0b70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47310/0 (socket says 172.21.15.44:47310) 2024-09-13T08:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d62a00640 1 -- 172.21.15.44:0/3254516453 learned_addr learned my addr 172.21.15.44:0/3254516453 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d6b400640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d6c0b25e0 0x7f3d6c0c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d6be00640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d6c0d2810 0x7f3d6c0b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d62a00640 1 -- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d6c0b25e0 msgr2=0x7f3d6c0c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d62a00640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d6c0b25e0 0x7f3d6c0c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d62a00640 1 -- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d6c0d2810 msgr2=0x7f3d6c0b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d62a00640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d6c0d2810 0x7f3d6c0b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d62a00640 1 -- 172.21.15.44:0/3254516453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c1f9380 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d6b400640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d6c0b25e0 0x7f3d6c0c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d6be00640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d6c0d2810 0x7f3d6c0b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d62a00640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d6c0cfd30 0x7f3d6c0b70d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c430 tx=0x7f3d5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d63e00640 1 -- 172.21.15.44:0/3254516453 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54011790 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d63e00640 1 -- 172.21.15.44:0/3254516453 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d54017070 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d63e00640 1 -- 172.21.15.44:0/3254516453 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54011ce0 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 -- 172.21.15.44:0/3254516453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c1f9660 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.158+0000 7f3d71000640 1 -- 172.21.15.44:0/3254516453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c1f9c20 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.162+0000 7f3d71000640 1 -- 172.21.15.44:0/3254516453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c15fc40 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.162+0000 7f3d63e00640 1 -- 172.21.15.44:0/3254516453 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 42) ==== 99222+0+0 (secure 0 0 0) 0x7f3d5401c460 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.162+0000 7f3d63e00640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d480746c0 0x7f3d48076b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.162+0000 7f3d63e00640 1 -- 172.21.15.44:0/3254516453 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f3d54092920 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.162+0000 7f3d6b400640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d480746c0 0x7f3d48076b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.162+0000 7f3d6b400640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d480746c0 0x7f3d48076b80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c007fd0 tx=0x7f3d5c007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.166+0000 7f3d63e00640 1 -- 172.21.15.44:0/3254516453 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3d5405f710 con 0x7f3d6c0cfd30 2024-09-13T08:50:10.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.374+0000 7f3d71000640 1 -- 172.21.15.44:0/3254516453 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f3d6c0b4390 con 0x7f3d480746c0 2024-09-13T08:50:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d63e00640 1 -- 172.21.15.44:0/3254516453 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d6c0b4390 con 0x7f3d480746c0 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 -- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d480746c0 msgr2=0x7f3d48076b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d480746c0 0x7f3d48076b80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c007fd0 tx=0x7f3d5c007d60 comp rx=0 tx=0).stop 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 -- 172.21.15.44:0/3254516453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d6c0cfd30 msgr2=0x7f3d6c0b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d6c0cfd30 0x7f3d6c0b70d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c430 tx=0x7f3d5400c8f0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 -- 172.21.15.44:0/3254516453 shutdown_connections 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d480746c0 0x7f3d48076b80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d6c0d2810 0x7f3d6c0b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d6c0cfd30 0x7f3d6c0b70d0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 --2- 172.21.15.44:0/3254516453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d6c0b25e0 0x7f3d6c0c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 -- 172.21.15.44:0/3254516453 >> 172.21.15.44:0/3254516453 conn(0x7f3d6c0d1cf0 msgr2=0x7f3d6c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 -- 172.21.15.44:0/3254516453 shutdown_connections 2024-09-13T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.442+0000 7f3d71000640 1 -- 172.21.15.44:0/3254516453 wait complete. 2024-09-13T08:50:10.453 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:10.453+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:10.453 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:10.453+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:10.453 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:10.453+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:10.453 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:10.453+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:10.453 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:10.453+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:10.467 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-13T08:50:10.470 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:10.469+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:10.470 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:10.469+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:10.470 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:10.469+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:10.470 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:10.469+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:10.470 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:10.469+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 -- 172.21.15.44:0/592993858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3000cf1d0 msgr2=0x7ff30015fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 --2- 172.21.15.44:0/592993858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3000cf1d0 0x7ff30015fd80 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff3000b26f0 tx=0x7ff2e801c890 comp rx=0 tx=0).stop 2024-09-13T08:50:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 -- 172.21.15.44:0/592993858 shutdown_connections 2024-09-13T08:50:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 --2- 172.21.15.44:0/592993858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3000cf1d0 0x7ff30015fd80 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 --2- 172.21.15.44:0/592993858 >> v2:172.21.15.129:3300/0 conn(0x7ff30015f9a0 0x7ff30006e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 -- 172.21.15.44:0/592993858 >> 172.21.15.44:0/592993858 conn(0x7ff3000b18f0 msgr2=0x7ff3000b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:10.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 -- 172.21.15.44:0/592993858 shutdown_connections 2024-09-13T08:50:10.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 -- 172.21.15.44:0/592993858 wait complete. 2024-09-13T08:50:10.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 Processor -- start 2024-09-13T08:50:10.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 -- start start 2024-09-13T08:50:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3000cf1d0 0x7ff3000b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30015f9a0 0x7ff30015d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3001602c0 0x7ff300153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3001695a0 con 0x7ff30015f9a0 2024-09-13T08:50:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff300169720 con 0x7ff3000cf1d0 2024-09-13T08:50:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff306c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff3001698a0 con 0x7ff3001602c0 2024-09-13T08:50:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff2f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30015f9a0 0x7ff30015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff2f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30015f9a0 0x7ff30015d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47332/0 (socket says 172.21.15.44:47332) 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff2f7e00640 1 -- 172.21.15.44:0/1336926986 learned_addr learned my addr 172.21.15.44:0/1336926986 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff2ffe00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3000cf1d0 0x7ff3000b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.622+0000 7ff304c00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3001602c0 0x7ff300153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff2f7e00640 1 -- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3001602c0 msgr2=0x7ff300153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff2f7e00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3001602c0 0x7ff300153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff2f7e00640 1 -- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3000cf1d0 msgr2=0x7ff3000b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff2f7e00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3000cf1d0 0x7ff3000b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff2f7e00640 1 -- 172.21.15.44:0/1336926986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3000b4cd0 con 0x7ff30015f9a0 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff2f7e00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30015f9a0 0x7ff30015d480 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80b0040 tx=0x7ff2e8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff2fcc00640 1 -- 172.21.15.44:0/1336926986 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80b2040 con 0x7ff30015f9a0 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff2fcc00640 1 -- 172.21.15.44:0/1336926986 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2e8005070 con 0x7ff30015f9a0 2024-09-13T08:50:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff306c00640 1 -- 172.21.15.44:0/1336926986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3000b09c0 con 0x7ff30015f9a0 2024-09-13T08:50:10.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff2fcc00640 1 -- 172.21.15.44:0/1336926986 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80054c0 con 0x7ff30015f9a0 2024-09-13T08:50:10.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff306c00640 1 -- 172.21.15.44:0/1336926986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3000b0f80 con 0x7ff30015f9a0 2024-09-13T08:50:10.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff306c00640 1 -- 172.21.15.44:0/1336926986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff300069410 con 0x7ff30015f9a0 2024-09-13T08:50:10.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.626+0000 7ff2fcc00640 1 -- 172.21.15.44:0/1336926986 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 42) ==== 99222+0+0 (secure 0 0 0) 0x7ff2e80046d0 con 0x7ff30015f9a0 2024-09-13T08:50:10.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.630+0000 7ff2fcc00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff2dc074660 0x7ff2dc076b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:10.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.630+0000 7ff2fcc00640 1 -- 172.21.15.44:0/1336926986 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7ff2e812c750 con 0x7ff30015f9a0 2024-09-13T08:50:10.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.630+0000 7ff2ffe00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff2dc074660 0x7ff2dc076b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:10.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.630+0000 7ff2ffe00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff2dc074660 0x7ff2dc076b20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0004630 tx=0x7ff2f000b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:10.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.634+0000 7ff2fcc00640 1 -- 172.21.15.44:0/1336926986 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff2e80164b0 con 0x7ff30015f9a0 2024-09-13T08:50:10.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:10.846+0000 7ff306c00640 1 -- 172.21.15.44:0/1336926986 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7ff30015fd80 con 0x7ff2dc074660 2024-09-13T08:50:11.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.442+0000 7ff2fcc00640 1 -- 172.21.15.44:0/1336926986 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 43) ==== 99324+0+0 (secure 0 0 0) 0x7ff2e80f4f70 con 0x7ff30015f9a0 2024-09-13T08:50:11.453 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:11.453+0000 7f8a87e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:11.453 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:11.453+0000 7f8a87e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:11.453 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:11.453+0000 7f8a87e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:11.453 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:11.453+0000 7f8a87e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:11.453 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:11.453+0000 7f8a87e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:11.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff2fcc00640 1 -- 172.21.15.44:0/1336926986 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7ff30015fd80 con 0x7ff2dc074660 2024-09-13T08:50:11.483 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:50:11.483 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "in-progress", 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_report": { 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: "percentage cloned": "0.0%", 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: "amount cloned": "184/3.0G", 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: "files cloned": "3/4" 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:11.484 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:11.485 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:50:11.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 -- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff2dc074660 msgr2=0x7ff2dc076b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:11.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff2dc074660 0x7ff2dc076b20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0004630 tx=0x7ff2f000b390 comp rx=0 tx=0).stop 2024-09-13T08:50:11.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 -- 172.21.15.44:0/1336926986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30015f9a0 msgr2=0x7ff30015d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:11.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30015f9a0 0x7ff30015d480 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80b0040 tx=0x7ff2e8004040 comp rx=0 tx=0).stop 2024-09-13T08:50:11.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 -- 172.21.15.44:0/1336926986 shutdown_connections 2024-09-13T08:50:11.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff2dc074660 0x7ff2dc076b20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:11.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3001602c0 0x7ff300153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:11.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30015f9a0 0x7ff30015d480 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:11.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 --2- 172.21.15.44:0/1336926986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3000cf1d0 0x7ff3000b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:11.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 -- 172.21.15.44:0/1336926986 >> 172.21.15.44:0/1336926986 conn(0x7ff3000b18f0 msgr2=0x7ff3000cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:11.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 -- 172.21.15.44:0/1336926986 shutdown_connections 2024-09-13T08:50:11.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.482+0000 7ff306c00640 1 -- 172.21.15.44:0/1336926986 wait complete. 2024-09-13T08:50:11.507 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-13T08:50:11.508 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-13T08:50:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.726+0000 7f540f200640 1 -- 172.21.15.44:0/713355091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541015a740 msgr2=0x7f541015ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.726+0000 7f540f200640 1 --2- 172.21.15.44:0/713355091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541015a740 0x7f541015ab20 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f540000a950 tx=0x7f540001db30 comp rx=0 tx=0).stop 2024-09-13T08:50:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.726+0000 7f540f200640 1 -- 172.21.15.44:0/713355091 shutdown_connections 2024-09-13T08:50:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.726+0000 7f540f200640 1 --2- 172.21.15.44:0/713355091 >> v2:172.21.15.129:3300/0 conn(0x7f541015b060 0x7f541006ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.726+0000 7f540f200640 1 --2- 172.21.15.44:0/713355091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541015a740 0x7f541015ab20 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.726+0000 7f540f200640 1 -- 172.21.15.44:0/713355091 >> 172.21.15.44:0/713355091 conn(0x7f541006cc20 msgr2=0x7f541006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.726+0000 7f540f200640 1 -- 172.21.15.44:0/713355091 shutdown_connections 2024-09-13T08:50:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.726+0000 7f540f200640 1 -- 172.21.15.44:0/713355091 wait complete. 2024-09-13T08:50:11.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.730+0000 7f540f200640 1 Processor -- start 2024-09-13T08:50:11.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.730+0000 7f540f200640 1 -- start start 2024-09-13T08:50:11.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.730+0000 7f540f200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f541006f130 0x7f5410146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:11.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.730+0000 7f540f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541015a740 0x7f5410162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.730+0000 7f540f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f541015b060 0x7f54100c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.730+0000 7f540f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5410074500 con 0x7f541015a740 2024-09-13T08:50:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.730+0000 7f540f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5410074380 con 0x7f541015b060 2024-09-13T08:50:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.730+0000 7f540f200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5410074680 con 0x7f541006f130 2024-09-13T08:50:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5407e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541015a740 0x7f5410162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5407e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541015a740 0x7f5410162cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47350/0 (socket says 172.21.15.44:47350) 2024-09-13T08:50:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5407e00640 1 -- 172.21.15.44:0/3580360454 learned_addr learned my addr 172.21.15.44:0/3580360454 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f540de00640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f541006f130 0x7f5410146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5407e00640 1 -- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f541006f130 msgr2=0x7f5410146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5407e00640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f541006f130 0x7f5410146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:11.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5407e00640 1 -- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f541015b060 msgr2=0x7f54100c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:11.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5407e00640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f541015b060 0x7f54100c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:11.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5407e00640 1 -- 172.21.15.44:0/3580360454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54101637a0 con 0x7f541015a740 2024-09-13T08:50:11.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5407e00640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541015a740 0x7f5410162cf0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f53f800ba50 tx=0x7f53f800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:11.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5406000640 1 -- 172.21.15.44:0/3580360454 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8019070 con 0x7f541015a740 2024-09-13T08:50:11.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5406000640 1 -- 172.21.15.44:0/3580360454 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53f800ccf0 con 0x7f541015a740 2024-09-13T08:50:11.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f5406000640 1 -- 172.21.15.44:0/3580360454 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f80146a0 con 0x7f541015a740 2024-09-13T08:50:11.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f540f200640 1 -- 172.21.15.44:0/3580360454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5410163a80 con 0x7f541015a740 2024-09-13T08:50:11.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f540f200640 1 -- 172.21.15.44:0/3580360454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5410153eb0 con 0x7f541015a740 2024-09-13T08:50:11.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.734+0000 7f540f200640 1 -- 172.21.15.44:0/3580360454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541015fc60 con 0x7f541015a740 2024-09-13T08:50:11.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.738+0000 7f5406000640 1 -- 172.21.15.44:0/3580360454 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 43) ==== 99324+0+0 (secure 0 0 0) 0x7f53f801c070 con 0x7f541015a740 2024-09-13T08:50:11.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.738+0000 7f5406000640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53ec074780 0x7f53ec076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:11.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.738+0000 7f5406000640 1 -- 172.21.15.44:0/3580360454 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f53f8092d50 con 0x7f541015a740 2024-09-13T08:50:11.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.742+0000 7f540de00640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53ec074780 0x7f53ec076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.742+0000 7f540de00640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53ec074780 0x7f53ec076c40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f54000b0000 tx=0x7f540001c370 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:11.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:11.746+0000 7f5406000640 1 -- 172.21.15.44:0/3580360454 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f541015fc60 con 0x7f541015a740 2024-09-13T08:50:12.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.058+0000 7f540f200640 1 -- 172.21.15.44:0/3580360454 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f541015ab20 con 0x7f53ec074780 2024-09-13T08:50:12.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.118+0000 7f5406000640 1 -- 172.21.15.44:0/3580360454 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f541015ab20 con 0x7f53ec074780 2024-09-13T08:50:12.122 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:50:12.122 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:50:12.122 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "in-progress", 2024-09-13T08:50:12.122 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:50:12.122 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:50:12.122 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:50:12.122 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:50:12.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:12.123 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_report": { 2024-09-13T08:50:12.123 INFO:teuthology.orchestra.run.smithi044.stdout: "percentage cloned": "0.0%", 2024-09-13T08:50:12.123 INFO:teuthology.orchestra.run.smithi044.stdout: "amount cloned": "184/3.0G", 2024-09-13T08:50:12.123 INFO:teuthology.orchestra.run.smithi044.stdout: "files cloned": "4/4" 2024-09-13T08:50:12.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:12.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:12.123 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:50:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 -- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53ec074780 msgr2=0x7f53ec076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53ec074780 0x7f53ec076c40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f54000b0000 tx=0x7f540001c370 comp rx=0 tx=0).stop 2024-09-13T08:50:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 -- 172.21.15.44:0/3580360454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541015a740 msgr2=0x7f5410162cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541015a740 0x7f5410162cf0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f53f800ba50 tx=0x7f53f800bf10 comp rx=0 tx=0).stop 2024-09-13T08:50:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 -- 172.21.15.44:0/3580360454 shutdown_connections 2024-09-13T08:50:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53ec074780 0x7f53ec076c40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f541015b060 0x7f54100c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541015a740 0x7f5410162cf0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 --2- 172.21.15.44:0/3580360454 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f541006f130 0x7f5410146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 -- 172.21.15.44:0/3580360454 >> 172.21.15.44:0/3580360454 conn(0x7f541006cc20 msgr2=0x7f541015d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:12.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 -- 172.21.15.44:0/3580360454 shutdown_connections 2024-09-13T08:50:12.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:12.122+0000 7f540f200640 1 -- 172.21.15.44:0/3580360454 wait complete. 2024-09-13T08:50:12.154 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 0.0% 2024-09-13T08:50:13.155 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-13T08:50:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.358+0000 7fdc27a00640 1 -- 172.21.15.44:0/3045159164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc201614f0 msgr2=0x7fdc200d4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.358+0000 7fdc27a00640 1 --2- 172.21.15.44:0/3045159164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc201614f0 0x7fdc200d4ab0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fdc14005fd0 tx=0x7fdc1401c790 comp rx=0 tx=0).stop 2024-09-13T08:50:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.358+0000 7fdc27a00640 1 -- 172.21.15.44:0/3045159164 shutdown_connections 2024-09-13T08:50:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.358+0000 7fdc27a00640 1 --2- 172.21.15.44:0/3045159164 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc200d2570 0x7fdc200d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.358+0000 7fdc27a00640 1 --2- 172.21.15.44:0/3045159164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc201614f0 0x7fdc200d4ab0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.358+0000 7fdc27a00640 1 -- 172.21.15.44:0/3045159164 >> 172.21.15.44:0/3045159164 conn(0x7fdc200d3f90 msgr2=0x7fdc200d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.358+0000 7fdc27a00640 1 -- 172.21.15.44:0/3045159164 shutdown_connections 2024-09-13T08:50:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.358+0000 7fdc27a00640 1 -- 172.21.15.44:0/3045159164 wait complete. 2024-09-13T08:50:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc27a00640 1 Processor -- start 2024-09-13T08:50:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc27a00640 1 -- start start 2024-09-13T08:50:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc27a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc200d2570 0x7fdc20152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc27a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc201614f0 0x7fdc200b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc27a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc20162530 0x7fdc200b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc27a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2016b450 con 0x7fdc200d2570 2024-09-13T08:50:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc27a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2016b2d0 con 0x7fdc201614f0 2024-09-13T08:50:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc27a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2016b5d0 con 0x7fdc20162530 2024-09-13T08:50:13.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc25000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc200d2570 0x7fdc20152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:13.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc25000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc200d2570 0x7fdc20152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48546/0 (socket says 172.21.15.44:48546) 2024-09-13T08:50:13.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc25000640 1 -- 172.21.15.44:0/2061886613 learned_addr learned my addr 172.21.15.44:0/2061886613 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:13.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc25000640 1 -- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc20162530 msgr2=0x7fdc200b3e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:50:13.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc25000640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc20162530 0x7fdc200b3e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:13.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc25000640 1 -- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc201614f0 msgr2=0x7fdc200b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:13.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc25000640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc201614f0 0x7fdc200b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.362+0000 7fdc25000640 1 -- 172.21.15.44:0/2061886613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc200cb060 con 0x7fdc200d2570 2024-09-13T08:50:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.366+0000 7fdc25000640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc200d2570 0x7fdc20152fc0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009790 tx=0x7fdc14005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.366+0000 7fdc1cc00640 1 -- 172.21.15.44:0/2061886613 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc140bb020 con 0x7fdc200d2570 2024-09-13T08:50:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.366+0000 7fdc1cc00640 1 -- 172.21.15.44:0/2061886613 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc140049c0 con 0x7fdc200d2570 2024-09-13T08:50:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.366+0000 7fdc1cc00640 1 -- 172.21.15.44:0/2061886613 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14004db0 con 0x7fdc200d2570 2024-09-13T08:50:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.366+0000 7fdc27a00640 1 -- 172.21.15.44:0/2061886613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc200cb340 con 0x7fdc200d2570 2024-09-13T08:50:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.366+0000 7fdc27a00640 1 -- 172.21.15.44:0/2061886613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc200cb900 con 0x7fdc200d2570 2024-09-13T08:50:13.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.366+0000 7fdc27a00640 1 -- 172.21.15.44:0/2061886613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc200d2a00 con 0x7fdc200d2570 2024-09-13T08:50:13.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.370+0000 7fdc1cc00640 1 -- 172.21.15.44:0/2061886613 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 43) ==== 99324+0+0 (secure 0 0 0) 0x7fdc140be020 con 0x7fdc200d2570 2024-09-13T08:50:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.370+0000 7fdc1cc00640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc00074780 0x7fdc00076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:13.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.370+0000 7fdc1cc00640 1 -- 172.21.15.44:0/2061886613 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7fdc1412db90 con 0x7fdc200d2570 2024-09-13T08:50:13.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.374+0000 7fdc1cc00640 1 -- 172.21.15.44:0/2061886613 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdc140c5030 con 0x7fdc200d2570 2024-09-13T08:50:13.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.374+0000 7fdc1fe00640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc00074780 0x7fdc00076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:13.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.378+0000 7fdc1fe00640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc00074780 0x7fdc00076c40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdc200b71c0 tx=0x7fdc0c007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:13.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.458+0000 7fdc1cc00640 1 -- 172.21.15.44:0/2061886613 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7fdc140f64d0 con 0x7fdc200d2570 2024-09-13T08:50:13.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.610+0000 7fdc27a00640 1 -- 172.21.15.44:0/2061886613 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fdc200b9f10 con 0x7fdc00074780 2024-09-13T08:50:13.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.630+0000 7fdc1cc00640 1 -- 172.21.15.44:0/2061886613 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7fdc200b9f10 con 0x7fdc00074780 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "in-progress", 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_report": { 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout: "percentage cloned": "23.958%", 2024-09-13T08:50:13.635 INFO:teuthology.orchestra.run.smithi044.stdout: "amount cloned": "736M/3.0G", 2024-09-13T08:50:13.636 INFO:teuthology.orchestra.run.smithi044.stdout: "files cloned": "4/4" 2024-09-13T08:50:13.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:13.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:13.636 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:50:13.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 -- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc00074780 msgr2=0x7fdc00076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:13.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc00074780 0x7fdc00076c40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdc200b71c0 tx=0x7fdc0c007a40 comp rx=0 tx=0).stop 2024-09-13T08:50:13.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 -- 172.21.15.44:0/2061886613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc200d2570 msgr2=0x7fdc20152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:13.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc200d2570 0x7fdc20152fc0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009790 tx=0x7fdc14005ce0 comp rx=0 tx=0).stop 2024-09-13T08:50:13.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 -- 172.21.15.44:0/2061886613 shutdown_connections 2024-09-13T08:50:13.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc00074780 0x7fdc00076c40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:13.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc20162530 0x7fdc200b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:13.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc201614f0 0x7fdc200b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:13.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 --2- 172.21.15.44:0/2061886613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc200d2570 0x7fdc20152fc0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:13.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 -- 172.21.15.44:0/2061886613 >> 172.21.15.44:0/2061886613 conn(0x7fdc200d3f90 msgr2=0x7fdc200b7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:13.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 -- 172.21.15.44:0/2061886613 shutdown_connections 2024-09-13T08:50:13.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:13.634+0000 7fdc27a00640 1 -- 172.21.15.44:0/2061886613 wait complete. 2024-09-13T08:50:13.657 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 23.958% 2024-09-13T08:50:14.658 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-13T08:50:14.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.870+0000 7f919a000640 1 -- 172.21.15.44:0/560947232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9194162530 msgr2=0x7f91940b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:14.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.870+0000 7f919a000640 1 --2- 172.21.15.44:0/560947232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9194162530 0x7f91940b0ae0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f917c009900 tx=0x7f917c01c660 comp rx=0 tx=0).stop 2024-09-13T08:50:14.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.870+0000 7f919a000640 1 -- 172.21.15.44:0/560947232 shutdown_connections 2024-09-13T08:50:14.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.870+0000 7f919a000640 1 --2- 172.21.15.44:0/560947232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9194162530 0x7f91940b0ae0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:14.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.870+0000 7f919a000640 1 --2- 172.21.15.44:0/560947232 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91941614f0 0x7f91940d4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:14.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.870+0000 7f919a000640 1 -- 172.21.15.44:0/560947232 >> 172.21.15.44:0/560947232 conn(0x7f91940d3f90 msgr2=0x7f91940d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:14.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.870+0000 7f919a000640 1 -- 172.21.15.44:0/560947232 shutdown_connections 2024-09-13T08:50:14.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.870+0000 7f919a000640 1 -- 172.21.15.44:0/560947232 wait complete. 2024-09-13T08:50:14.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.870+0000 7f919a000640 1 Processor -- start 2024-09-13T08:50:14.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.870+0000 7f919a000640 1 -- start start 2024-09-13T08:50:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f919a000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91940d2570 0x7f91941f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f919a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91941614f0 0x7f9194157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f919a000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9194162530 0x7f919406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f919a000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f919416ba60 con 0x7f91941614f0 2024-09-13T08:50:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f919a000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f919416b8e0 con 0x7f9194162530 2024-09-13T08:50:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f919a000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f919416bbe0 con 0x7f91940d2570 2024-09-13T08:50:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91941614f0 0x7f9194157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91941614f0 0x7f9194157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48564/0 (socket says 172.21.15.44:48564) 2024-09-13T08:50:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918aa00640 1 -- 172.21.15.44:0/1365138657 learned_addr learned my addr 172.21.15.44:0/1365138657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:14.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f9193e00640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9194162530 0x7f919406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f9193400640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91940d2570 0x7f91941f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918aa00640 1 -- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91940d2570 msgr2=0x7f91941f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918aa00640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91940d2570 0x7f91941f8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918aa00640 1 -- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9194162530 msgr2=0x7f919406d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918aa00640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9194162530 0x7f919406d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918aa00640 1 -- 172.21.15.44:0/1365138657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91940cd970 con 0x7f91941614f0 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918aa00640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91941614f0 0x7f9194157d30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f917c0b0040 tx=0x7f917c0a6b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918be00640 1 -- 172.21.15.44:0/1365138657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c0bb020 con 0x7f91941614f0 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918be00640 1 -- 172.21.15.44:0/1365138657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f917c004330 con 0x7f91941614f0 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f919a000640 1 -- 172.21.15.44:0/1365138657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91940cdc50 con 0x7f91941614f0 2024-09-13T08:50:14.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f918be00640 1 -- 172.21.15.44:0/1365138657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c003860 con 0x7f91941614f0 2024-09-13T08:50:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.874+0000 7f919a000640 1 -- 172.21.15.44:0/1365138657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91940ce210 con 0x7f91941614f0 2024-09-13T08:50:14.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.878+0000 7f918be00640 1 -- 172.21.15.44:0/1365138657 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7f917c0be020 con 0x7f91941614f0 2024-09-13T08:50:14.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.878+0000 7f919a000640 1 -- 172.21.15.44:0/1365138657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91940b0ae0 con 0x7f91941614f0 2024-09-13T08:50:14.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.878+0000 7f918be00640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9170074810 0x7f9170076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:14.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.878+0000 7f918be00640 1 -- 172.21.15.44:0/1365138657 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f917c12d7a0 con 0x7f91941614f0 2024-09-13T08:50:14.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.878+0000 7f9193400640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9170074810 0x7f9170076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:14.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.882+0000 7f9193400640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9170074810 0x7f9170076cd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9184006fd0 tx=0x7f9184008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:14.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:14.886+0000 7f918be00640 1 -- 172.21.15.44:0/1365138657 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f917c0c5030 con 0x7f91941614f0 2024-09-13T08:50:15.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.178+0000 7f919a000640 1 -- 172.21.15.44:0/1365138657 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f91940b9f10 con 0x7f9170074810 2024-09-13T08:50:15.263 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "in-progress", 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_report": { 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "percentage cloned": "57.292%", 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "amount cloned": "1.7G/3.0G", 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: "files cloned": "5/4" 2024-09-13T08:50:15.264 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:15.265 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.258+0000 7f918be00640 1 -- 172.21.15.44:0/1365138657 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f91940b9f10 con 0x7f9170074810 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 -- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9170074810 msgr2=0x7f9170076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9170074810 0x7f9170076cd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9184006fd0 tx=0x7f9184008040 comp rx=0 tx=0).stop 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 -- 172.21.15.44:0/1365138657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91941614f0 msgr2=0x7f9194157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91941614f0 0x7f9194157d30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f917c0b0040 tx=0x7f917c0a6b70 comp rx=0 tx=0).stop 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 -- 172.21.15.44:0/1365138657 shutdown_connections 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9170074810 0x7f9170076cd0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9194162530 0x7f919406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91941614f0 0x7f9194157d30 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 --2- 172.21.15.44:0/1365138657 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91940d2570 0x7f91941f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 -- 172.21.15.44:0/1365138657 >> 172.21.15.44:0/1365138657 conn(0x7f91940d3f90 msgr2=0x7f919406e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 -- 172.21.15.44:0/1365138657 shutdown_connections 2024-09-13T08:50:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:15.262+0000 7f919a000640 1 -- 172.21.15.44:0/1365138657 wait complete. 2024-09-13T08:50:15.287 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 57.292% 2024-09-13T08:50:16.289 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-13T08:50:16.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.514+0000 7f6964800640 1 -- 172.21.15.44:0/2551878721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695c0b1650 msgr2=0x7f695c0b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:16.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.514+0000 7f6964800640 1 --2- 172.21.15.44:0/2551878721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695c0b1650 0x7f695c0b1a30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f695c073810 tx=0x7f694c01c8c0 comp rx=0 tx=0).stop 2024-09-13T08:50:16.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.514+0000 7f6964800640 1 -- 172.21.15.44:0/2551878721 shutdown_connections 2024-09-13T08:50:16.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.514+0000 7f6964800640 1 --2- 172.21.15.44:0/2551878721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695c0b1650 0x7f695c0b1a30 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:16.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.514+0000 7f6964800640 1 --2- 172.21.15.44:0/2551878721 >> v2:172.21.15.129:3300/0 conn(0x7f695c1605d0 0x7f695c0d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:16.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.514+0000 7f6964800640 1 -- 172.21.15.44:0/2551878721 >> 172.21.15.44:0/2551878721 conn(0x7f695c0d1dd0 msgr2=0x7f695c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:16.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.514+0000 7f6964800640 1 -- 172.21.15.44:0/2551878721 shutdown_connections 2024-09-13T08:50:16.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.514+0000 7f6964800640 1 -- 172.21.15.44:0/2551878721 wait complete. 2024-09-13T08:50:16.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.518+0000 7f6964800640 1 Processor -- start 2024-09-13T08:50:16.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.518+0000 7f6964800640 1 -- start start 2024-09-13T08:50:16.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.518+0000 7f6964800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f695c0aeb00 0x7f695c0b7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:16.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.518+0000 7f6964800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695c0b1650 0x7f695c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:16.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.518+0000 7f6964800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f695c1605d0 0x7f695c071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:16.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.518+0000 7f6964800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f695c0b1220 con 0x7f695c0b1650 2024-09-13T08:50:16.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.518+0000 7f6964800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f695c0b0ee0 con 0x7f695c1605d0 2024-09-13T08:50:16.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.518+0000 7f6964800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f695c0b13a0 con 0x7f695c0aeb00 2024-09-13T08:50:16.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.522+0000 7f6961400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695c0b1650 0x7f695c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:16.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.522+0000 7f6961400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695c0b1650 0x7f695c06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48584/0 (socket says 172.21.15.44:48584) 2024-09-13T08:50:16.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.522+0000 7f6961400640 1 -- 172.21.15.44:0/1154551924 learned_addr learned my addr 172.21.15.44:0/1154551924 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:16.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.522+0000 7f6961400640 1 -- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f695c0aeb00 msgr2=0x7f695c0b7690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:50:16.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.522+0000 7f6961400640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f695c0aeb00 0x7f695c0b7690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:16.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.522+0000 7f6961400640 1 -- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f695c1605d0 msgr2=0x7f695c071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:16.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.522+0000 7f6961400640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f695c1605d0 0x7f695c071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:16.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.522+0000 7f6961400640 1 -- 172.21.15.44:0/1154551924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c06e270 con 0x7f695c0b1650 2024-09-13T08:50:16.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.522+0000 7f6961400640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695c0b1650 0x7f695c06dd30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f694c009850 tx=0x7f694c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:16.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.526+0000 7f6956000640 1 -- 172.21.15.44:0/1154551924 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c0b2040 con 0x7f695c0b1650 2024-09-13T08:50:16.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.526+0000 7f6956000640 1 -- 172.21.15.44:0/1154551924 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f694c0044b0 con 0x7f695c0b1650 2024-09-13T08:50:16.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.526+0000 7f6964800640 1 -- 172.21.15.44:0/1154551924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695c15bd00 con 0x7f695c0b1650 2024-09-13T08:50:16.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.526+0000 7f6956000640 1 -- 172.21.15.44:0/1154551924 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c003860 con 0x7f695c0b1650 2024-09-13T08:50:16.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.526+0000 7f6964800640 1 -- 172.21.15.44:0/1154551924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695c15c2c0 con 0x7f695c0b1650 2024-09-13T08:50:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.526+0000 7f6964800640 1 -- 172.21.15.44:0/1154551924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f695c0b1a30 con 0x7f695c0b1650 2024-09-13T08:50:16.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.526+0000 7f6956000640 1 -- 172.21.15.44:0/1154551924 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7f694c0a56f0 con 0x7f695c0b1650 2024-09-13T08:50:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.530+0000 7f6956000640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6940074810 0x7f6940076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.530+0000 7f6956000640 1 -- 172.21.15.44:0/1154551924 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5793+0+0 (secure 0 0 0) 0x7f694c12c9e0 con 0x7f695c0b1650 2024-09-13T08:50:16.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.530+0000 7f6961e00640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6940074810 0x7f6940076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:16.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.534+0000 7f6956000640 1 -- 172.21.15.44:0/1154551924 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f694c0164b0 con 0x7f695c0b1650 2024-09-13T08:50:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.542+0000 7f6961e00640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6940074810 0x7f6940076cd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6950009670 tx=0x7f6950006b90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:16.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:16.858+0000 7f6964800640 1 -- 172.21.15.44:0/1154551924 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f695c1f7bc0 con 0x7f6940074810 2024-09-13T08:50:17.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.002+0000 7f6956000640 1 -- 172.21.15.44:0/1154551924 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f6950009280 con 0x7f6940074810 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "in-progress", 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_report": { 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: "percentage cloned": "77.083%", 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: "amount cloned": "2.3G/3.0G", 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: "files cloned": "6/4" 2024-09-13T08:50:17.004 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:17.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:17.005 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:50:17.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.002+0000 7f6964800640 1 -- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6940074810 msgr2=0x7f6940076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:17.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.002+0000 7f6964800640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6940074810 0x7f6940076cd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6950009670 tx=0x7f6950006b90 comp rx=0 tx=0).stop 2024-09-13T08:50:17.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.002+0000 7f6964800640 1 -- 172.21.15.44:0/1154551924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695c0b1650 msgr2=0x7f695c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:17.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.002+0000 7f6964800640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695c0b1650 0x7f695c06dd30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f694c009850 tx=0x7f694c005770 comp rx=0 tx=0).stop 2024-09-13T08:50:17.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.006+0000 7f6964800640 1 -- 172.21.15.44:0/1154551924 shutdown_connections 2024-09-13T08:50:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.006+0000 7f6964800640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6940074810 0x7f6940076cd0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.006+0000 7f6964800640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f695c1605d0 0x7f695c071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.006+0000 7f6964800640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695c0b1650 0x7f695c06dd30 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.006+0000 7f6964800640 1 --2- 172.21.15.44:0/1154551924 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f695c0aeb00 0x7f695c0b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.006+0000 7f6964800640 1 -- 172.21.15.44:0/1154551924 >> 172.21.15.44:0/1154551924 conn(0x7f695c0d1dd0 msgr2=0x7f695c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.006+0000 7f6964800640 1 -- 172.21.15.44:0/1154551924 shutdown_connections 2024-09-13T08:50:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:17.006+0000 7f6964800640 1 -- 172.21.15.44:0/1154551924 wait complete. 2024-09-13T08:50:17.054 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 77.083% 2024-09-13T08:50:18.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-13T08:50:18.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.206+0000 7f8ea2600640 1 -- 172.21.15.44:0/3210884733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e80003f80 msgr2=0x7f8e80004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.206+0000 7f8ea2600640 1 --2- 172.21.15.44:0/3210884733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e80003f80 0x7f8e80004810 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8e840017c0 tx=0x7f8e8401f030 comp rx=0 tx=0).stop 2024-09-13T08:50:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.206+0000 7f8e9be00640 1 -- 172.21.15.44:0/3210884733 reap_dead start 2024-09-13T08:50:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.206+0000 7f8ea2600640 1 -- 172.21.15.44:0/3210884733 shutdown_connections 2024-09-13T08:50:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.206+0000 7f8ea2600640 1 -- 172.21.15.44:0/3210884733 >> 172.21.15.44:0/3210884733 conn(0x7f8e9c0d1da0 msgr2=0x7f8e9c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.206+0000 7f8ea2600640 1 -- 172.21.15.44:0/3210884733 shutdown_connections 2024-09-13T08:50:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.206+0000 7f8ea2600640 1 -- 172.21.15.44:0/3210884733 wait complete. 2024-09-13T08:50:18.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 Processor -- start 2024-09-13T08:50:18.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 -- start start 2024-09-13T08:50:18.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e80003f80 0x7f8e9c1f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:18.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e9c09c720 0x7f8e9c09dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:18.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e9c09bd70 0x7f8e9c159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:18.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c0b34e0 con 0x7f8e9c09bd70 2024-09-13T08:50:18.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c0b3360 con 0x7f8e9c09c720 2024-09-13T08:50:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8e9c0d06f0 con 0x7f8e80003f80 2024-09-13T08:50:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e9c09bd70 0x7f8e9c159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e80003f80 0x7f8e9c1f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e9c09c720 0x7f8e9c09dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e80003f80 0x7f8e9c1f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:43426/0 (socket says 172.21.15.44:43426) 2024-09-13T08:50:18.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9b400640 1 -- 172.21.15.44:0/4123900860 learned_addr learned my addr 172.21.15.44:0/4123900860 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:18.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9b400640 1 -- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e9c09c720 msgr2=0x7f8e9c09dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:18.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9b400640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e9c09c720 0x7f8e9c09dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:18.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9b400640 1 -- 172.21.15.44:0/4123900860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e9c09bd70 msgr2=0x7f8e9c159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:18.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9b400640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e9c09bd70 0x7f8e9c159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:18.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9b400640 1 -- 172.21.15.44:0/4123900860 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c1f5620 con 0x7f8e80003f80 2024-09-13T08:50:18.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9be00640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e9c09bd70 0x7f8e9c159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:50:18.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e9b400640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e80003f80 0x7f8e9c1f50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0064a0 tx=0x7f8e8c0039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:18.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e93400640 1 -- 172.21.15.44:0/4123900860 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c015070 con 0x7f8e80003f80 2024-09-13T08:50:18.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e93400640 1 -- 172.21.15.44:0/4123900860 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e8c011920 con 0x7f8e80003f80 2024-09-13T08:50:18.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 -- 172.21.15.44:0/4123900860 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c09e130 con 0x7f8e80003f80 2024-09-13T08:50:18.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8e93400640 1 -- 172.21.15.44:0/4123900860 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c011d70 con 0x7f8e80003f80 2024-09-13T08:50:18.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 -- 172.21.15.44:0/4123900860 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c161390 con 0x7f8e80003f80 2024-09-13T08:50:18.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.210+0000 7f8ea2600640 1 -- 172.21.15.44:0/4123900860 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9c0ae990 con 0x7f8e80003f80 2024-09-13T08:50:18.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.214+0000 7f8e93400640 1 -- 172.21.15.44:0/4123900860 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7f8e8c018070 con 0x7f8e80003f80 2024-09-13T08:50:18.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.214+0000 7f8e93400640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e78074810 0x7f8e78076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:18.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.214+0000 7f8e93400640 1 -- 172.21.15.44:0/4123900860 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7f8e8c093bf0 con 0x7f8e80003f80 2024-09-13T08:50:18.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.214+0000 7f8e93e00640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e78074810 0x7f8e78076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:18.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.214+0000 7f8e93e00640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e78074810 0x7f8e78076cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8e800057f0 tx=0x7f8e8401bdd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:18.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.218+0000 7f8e93400640 1 -- 172.21.15.44:0/4123900860 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8e9c1f5620 con 0x7f8e80003f80 2024-09-13T08:50:18.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.426+0000 7f8ea2600640 1 -- 172.21.15.44:0/4123900860 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f8e9c15d930 con 0x7f8e78074810 2024-09-13T08:50:18.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.834+0000 7f8e93400640 1 -- 172.21.15.44:0/4123900860 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f8e9c15d930 con 0x7f8e78074810 2024-09-13T08:50:18.838 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:50:18.839 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:50:18.839 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "complete" 2024-09-13T08:50:18.839 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:18.839 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:50:18.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 -- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e78074810 msgr2=0x7f8e78076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:18.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e78074810 0x7f8e78076cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8e800057f0 tx=0x7f8e8401bdd0 comp rx=0 tx=0).stop 2024-09-13T08:50:18.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 -- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e80003f80 msgr2=0x7f8e9c1f50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:18.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e80003f80 0x7f8e9c1f50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0064a0 tx=0x7f8e8c0039d0 comp rx=0 tx=0).stop 2024-09-13T08:50:18.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 -- 172.21.15.44:0/4123900860 shutdown_connections 2024-09-13T08:50:18.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8e78074810 0x7f8e78076cd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:18.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e9c09bd70 0x7f8e9c159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:18.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e9c09c720 0x7f8e9c09dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:18.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 --2- 172.21.15.44:0/4123900860 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8e80003f80 0x7f8e9c1f50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:18.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 -- 172.21.15.44:0/4123900860 >> 172.21.15.44:0/4123900860 conn(0x7f8e9c0d1da0 msgr2=0x7f8e9c0d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:18.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 -- 172.21.15.44:0/4123900860 shutdown_connections 2024-09-13T08:50:18.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:18.838+0000 7f8ea2600640 1 -- 172.21.15.44:0/4123900860 wait complete. 2024-09-13T08:50:18.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-13T08:50:19.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.010+0000 7fa6a6e00640 1 -- 172.21.15.44:0/41913101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa688003a90 msgr2=0x7fa688003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.010+0000 7fa6a6e00640 1 --2- 172.21.15.44:0/41913101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa688003a90 0x7fa688003eb0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fa694006f60 tx=0x7fa69401ceb0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.010+0000 7fa6a6400640 1 -- 172.21.15.44:0/41913101 reap_dead start 2024-09-13T08:50:19.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.010+0000 7fa6a6e00640 1 -- 172.21.15.44:0/41913101 shutdown_connections 2024-09-13T08:50:19.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.010+0000 7fa6a6e00640 1 -- 172.21.15.44:0/41913101 >> 172.21.15.44:0/41913101 conn(0x7fa6a009d0c0 msgr2=0x7fa6a00d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:19.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.010+0000 7fa6a6e00640 1 -- 172.21.15.44:0/41913101 shutdown_connections 2024-09-13T08:50:19.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.010+0000 7fa6a6e00640 1 -- 172.21.15.44:0/41913101 wait complete. 2024-09-13T08:50:19.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 Processor -- start 2024-09-13T08:50:19.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 -- start start 2024-09-13T08:50:19.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa688003a90 0x7fa6a00b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6a015ca20 0x7fa6a01f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6a00caed0 0x7fa6a015df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a00d16b0 con 0x7fa6a00caed0 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a00d1530 con 0x7fa688003a90 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a0169d50 con 0x7fa6a015ca20 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6a00caed0 0x7fa6a015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6a00caed0 0x7fa6a015df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48628/0 (socket says 172.21.15.44:48628) 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6400640 1 -- 172.21.15.44:0/225857610 learned_addr learned my addr 172.21.15.44:0/225857610 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a5000640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6a015ca20 0x7fa6a01f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6400640 1 -- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6a015ca20 msgr2=0x7fa6a01f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6400640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6a015ca20 0x7fa6a01f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6400640 1 -- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa688003a90 msgr2=0x7fa6a00b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6400640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa688003a90 0x7fa6a00b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6400640 1 -- 172.21.15.44:0/225857610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a0155bf0 con 0x7fa6a00caed0 2024-09-13T08:50:19.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6400640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6a00caed0 0x7fa6a015df40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0162a90 tx=0x7fa69800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:19.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa69d600640 1 -- 172.21.15.44:0/225857610 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698004b30 con 0x7fa6a00caed0 2024-09-13T08:50:19.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa69d600640 1 -- 172.21.15.44:0/225857610 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa698004cd0 con 0x7fa6a00caed0 2024-09-13T08:50:19.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa69d600640 1 -- 172.21.15.44:0/225857610 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698003be0 con 0x7fa6a00caed0 2024-09-13T08:50:19.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 -- 172.21.15.44:0/225857610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a01f8620 con 0x7fa6a00caed0 2024-09-13T08:50:19.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 -- 172.21.15.44:0/225857610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a015d180 con 0x7fa6a00caed0 2024-09-13T08:50:19.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.014+0000 7fa6a6e00640 1 -- 172.21.15.44:0/225857610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a00d2240 con 0x7fa6a00caed0 2024-09-13T08:50:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.018+0000 7fa69d600640 1 -- 172.21.15.44:0/225857610 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7fa698004030 con 0x7fa6a00caed0 2024-09-13T08:50:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.018+0000 7fa69d600640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa680074810 0x7fa680076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.018+0000 7fa69d600640 1 -- 172.21.15.44:0/225857610 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7fa698093410 con 0x7fa6a00caed0 2024-09-13T08:50:19.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.018+0000 7fa6a5a00640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa680074810 0x7fa680076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:19.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.022+0000 7fa6a5a00640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa680074810 0x7fa680076cd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa6940ab000 tx=0x7fa688005840 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:19.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.022+0000 7fa69d600640 1 -- 172.21.15.44:0/225857610 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa6a01f8620 con 0x7fa6a00caed0 2024-09-13T08:50:19.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.234+0000 7fa6a6e00640 1 -- 172.21.15.44:0/225857610 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa6a008d540 con 0x7fa680074810 2024-09-13T08:50:19.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa69d600640 1 -- 172.21.15.44:0/225857610 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fa6a008d540 con 0x7fa680074810 2024-09-13T08:50:19.244 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:50:19.245 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:50:19.245 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "complete" 2024-09-13T08:50:19.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:19.245 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:50:19.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa6a6e00640 1 -- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa680074810 msgr2=0x7fa680076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa6a6e00640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa680074810 0x7fa680076cd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa6940ab000 tx=0x7fa688005840 comp rx=0 tx=0).stop 2024-09-13T08:50:19.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa6a6e00640 1 -- 172.21.15.44:0/225857610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6a00caed0 msgr2=0x7fa6a015df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa6a6e00640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6a00caed0 0x7fa6a015df40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0162a90 tx=0x7fa69800f3c0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa6a6e00640 1 -- 172.21.15.44:0/225857610 shutdown_connections 2024-09-13T08:50:19.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa6a6e00640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa680074810 0x7fa680076cd0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa6a6e00640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6a00caed0 0x7fa6a015df40 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa6a6e00640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa6a015ca20 0x7fa6a01f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa6a6e00640 1 --2- 172.21.15.44:0/225857610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa688003a90 0x7fa6a00b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.242+0000 7fa6a6e00640 1 -- 172.21.15.44:0/225857610 >> 172.21.15.44:0/225857610 conn(0x7fa6a009d0c0 msgr2=0x7fa6a00ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:19.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.246+0000 7fa6a6e00640 1 -- 172.21.15.44:0/225857610 shutdown_connections 2024-09-13T08:50:19.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.246+0000 7fa6a6e00640 1 -- 172.21.15.44:0/225857610 wait complete. 2024-09-13T08:50:19.267 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-13T08:50:19.267 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:50:19.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.410+0000 7f823b000640 1 -- 172.21.15.44:0/871352003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f823415af60 msgr2=0x7f82340cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.410+0000 7f823b000640 1 --2- 172.21.15.44:0/871352003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f823415af60 0x7f82340cffc0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8224009840 tx=0x7f822401c820 comp rx=0 tx=0).stop 2024-09-13T08:50:19.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.410+0000 7f823b000640 1 -- 172.21.15.44:0/871352003 shutdown_connections 2024-09-13T08:50:19.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.410+0000 7f823b000640 1 --2- 172.21.15.44:0/871352003 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f82340737c0 0x7f8234073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.410+0000 7f823b000640 1 --2- 172.21.15.44:0/871352003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f823415af60 0x7f82340cffc0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.410+0000 7f823b000640 1 -- 172.21.15.44:0/871352003 >> 172.21.15.44:0/871352003 conn(0x7f82340d0c90 msgr2=0x7f82340d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:19.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.410+0000 7f823b000640 1 -- 172.21.15.44:0/871352003 shutdown_connections 2024-09-13T08:50:19.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.410+0000 7f823b000640 1 -- 172.21.15.44:0/871352003 wait complete. 2024-09-13T08:50:19.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f823b000640 1 Processor -- start 2024-09-13T08:50:19.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f823b000640 1 -- start start 2024-09-13T08:50:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f823b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82340737c0 0x7f823415df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f823b000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8234074160 0x7f82340c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f823b000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f823415af60 0x7f823406cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f823b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8234167b40 con 0x7f82340737c0 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f823b000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8234070b70 con 0x7f823415af60 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f823b000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8234167cc0 con 0x7f8234074160 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8233e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82340737c0 0x7f823415df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8233e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82340737c0 0x7f823415df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48650/0 (socket says 172.21.15.44:48650) 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8233e00640 1 -- 172.21.15.44:0/3565776381 learned_addr learned my addr 172.21.15.44:0/3565776381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8233e00640 1 -- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8234074160 msgr2=0x7f82340c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8233e00640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8234074160 0x7f82340c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8233e00640 1 -- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f823415af60 msgr2=0x7f823406cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8233e00640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f823415af60 0x7f823406cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8233e00640 1 -- 172.21.15.44:0/3565776381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823415e490 con 0x7f82340737c0 2024-09-13T08:50:19.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8233e00640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82340737c0 0x7f823415df50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8224009790 tx=0x7f82240a67d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:19.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8230c00640 1 -- 172.21.15.44:0/3565776381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82240043c0 con 0x7f82340737c0 2024-09-13T08:50:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8230c00640 1 -- 172.21.15.44:0/3565776381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8224005030 con 0x7f82340737c0 2024-09-13T08:50:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f8230c00640 1 -- 172.21.15.44:0/3565776381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224005480 con 0x7f82340737c0 2024-09-13T08:50:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f823b000640 1 -- 172.21.15.44:0/3565776381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82340c9ff0 con 0x7f82340737c0 2024-09-13T08:50:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.414+0000 7f823b000640 1 -- 172.21.15.44:0/3565776381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82340ca5b0 con 0x7f82340737c0 2024-09-13T08:50:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.418+0000 7f823b000640 1 -- 172.21.15.44:0/3565776381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8234073c20 con 0x7f82340737c0 2024-09-13T08:50:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.418+0000 7f8230c00640 1 -- 172.21.15.44:0/3565776381 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7f82240046d0 con 0x7f82340737c0 2024-09-13T08:50:19.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.418+0000 7f8230c00640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8210074810 0x7f8210076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.418+0000 7f8230c00640 1 -- 172.21.15.44:0/3565776381 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7f822412d3a0 con 0x7f82340737c0 2024-09-13T08:50:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.418+0000 7f822be00640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8210074810 0x7f8210076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:19.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.422+0000 7f822be00640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8210074810 0x7f8210076cd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f82340b0fa0 tx=0x7f821c005ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.422+0000 7f8230c00640 1 -- 172.21.15.44:0/3565776381 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8224016410 con 0x7f82340737c0 2024-09-13T08:50:19.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.622+0000 7f823b000640 1 -- 172.21.15.44:0/3565776381 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f823406da00 con 0x7f8210074810 2024-09-13T08:50:19.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f8230c00640 1 -- 172.21.15.44:0/3565776381 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f823406da00 con 0x7f8210074810 2024-09-13T08:50:19.628 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:50:19.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:19.628 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:50:19.628 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:19.628 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:50:19.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 -- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8210074810 msgr2=0x7f8210076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8210074810 0x7f8210076cd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f82340b0fa0 tx=0x7f821c005ca0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 -- 172.21.15.44:0/3565776381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82340737c0 msgr2=0x7f823415df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82340737c0 0x7f823415df50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8224009790 tx=0x7f82240a67d0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 -- 172.21.15.44:0/3565776381 shutdown_connections 2024-09-13T08:50:19.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8210074810 0x7f8210076cd0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f823415af60 0x7f823406cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8234074160 0x7f82340c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 --2- 172.21.15.44:0/3565776381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82340737c0 0x7f823415df50 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 -- 172.21.15.44:0/3565776381 >> 172.21.15.44:0/3565776381 conn(0x7f82340d0c90 msgr2=0x7f8234162c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:19.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 -- 172.21.15.44:0/3565776381 shutdown_connections 2024-09-13T08:50:19.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.626+0000 7f823b000640 1 -- 172.21.15.44:0/3565776381 wait complete. 2024-09-13T08:50:19.650 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-13T08:50:19.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.798+0000 7f5afc200640 1 -- 172.21.15.44:0/882647194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5adc004d50 msgr2=0x7f5adc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.798+0000 7f5afc200640 1 --2- 172.21.15.44:0/882647194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5adc004d50 0x7f5adc0051d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c670 tx=0x7f5aec01d300 comp rx=0 tx=0).stop 2024-09-13T08:50:19.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.798+0000 7f5afa200640 1 -- 172.21.15.44:0/882647194 reap_dead start 2024-09-13T08:50:19.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.798+0000 7f5afc200640 1 -- 172.21.15.44:0/882647194 shutdown_connections 2024-09-13T08:50:19.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.798+0000 7f5afc200640 1 -- 172.21.15.44:0/882647194 >> 172.21.15.44:0/882647194 conn(0x7f5af40d04f0 msgr2=0x7f5af40d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:19.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.798+0000 7f5afc200640 1 -- 172.21.15.44:0/882647194 shutdown_connections 2024-09-13T08:50:19.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.798+0000 7f5afc200640 1 -- 172.21.15.44:0/882647194 wait complete. 2024-09-13T08:50:19.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.798+0000 7f5afc200640 1 Processor -- start 2024-09-13T08:50:19.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.798+0000 7f5afc200640 1 -- start start 2024-09-13T08:50:19.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5afc200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5adc004d50 0x7f5af4150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5afc200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5af41f6700 0x7f5af40c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5afc200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5af41f2400 0x7f5af40aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5afc200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5af415f610 con 0x7f5adc004d50 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5afc200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5af415f490 con 0x7f5af41f6700 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5afc200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5af40cb2e0 con 0x7f5af41f2400 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5adc004d50 0x7f5af4150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5adc004d50 0x7f5af4150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48666/0 (socket says 172.21.15.44:48666) 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af9800640 1 -- 172.21.15.44:0/3347242549 learned_addr learned my addr 172.21.15.44:0/3347242549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af8e00640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5af41f6700 0x7f5af40c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5afa200640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5af41f2400 0x7f5af40aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:19.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af9800640 1 -- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5af41f2400 msgr2=0x7f5af40aae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af9800640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5af41f2400 0x7f5af40aae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af9800640 1 -- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5af41f6700 msgr2=0x7f5af40c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:19.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af9800640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5af41f6700 0x7f5af40c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:19.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af9800640 1 -- 172.21.15.44:0/3347242549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af41f33f0 con 0x7f5adc004d50 2024-09-13T08:50:19.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af8e00640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5af41f6700 0x7f5af40c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:19.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af9800640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5adc004d50 0x7f5af4150a80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8011ae0 tx=0x7f5ae8009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af1600640 1 -- 172.21.15.44:0/3347242549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae801e070 con 0x7f5adc004d50 2024-09-13T08:50:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af1600640 1 -- 172.21.15.44:0/3347242549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ae8004460 con 0x7f5adc004d50 2024-09-13T08:50:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5af1600640 1 -- 172.21.15.44:0/3347242549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae800fc30 con 0x7f5adc004d50 2024-09-13T08:50:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5afc200640 1 -- 172.21.15.44:0/3347242549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af41f2e30 con 0x7f5adc004d50 2024-09-13T08:50:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5afc200640 1 -- 172.21.15.44:0/3347242549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af40a95b0 con 0x7f5adc004d50 2024-09-13T08:50:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.802+0000 7f5afc200640 1 -- 172.21.15.44:0/3347242549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af415e7b0 con 0x7f5adc004d50 2024-09-13T08:50:19.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.806+0000 7f5af1600640 1 -- 172.21.15.44:0/3347242549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7f5ae8021070 con 0x7f5adc004d50 2024-09-13T08:50:19.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.806+0000 7f5af1600640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ad4074840 0x7f5ad4076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:19.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.806+0000 7f5af1600640 1 -- 172.21.15.44:0/3347242549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7f5ae8093640 con 0x7f5adc004d50 2024-09-13T08:50:19.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.806+0000 7f5af8e00640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ad4074840 0x7f5ad4076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:19.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.810+0000 7f5af8e00640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ad4074840 0x7f5ad4076d00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5af40b6140 tx=0x7f5ae00091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:19.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:19.810+0000 7f5af1600640 1 -- 172.21.15.44:0/3347242549 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5ae805fd60 con 0x7f5adc004d50 2024-09-13T08:50:20.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.010+0000 7f5afc200640 1 -- 172.21.15.44:0/3347242549 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5af4068e40 con 0x7f5ad4074840 2024-09-13T08:50:20.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.014+0000 7f5af1600640 1 -- 172.21.15.44:0/3347242549 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f5af4068e40 con 0x7f5ad4074840 2024-09-13T08:50:20.018 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:20.019 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:50:20.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:20.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sv1" 2024-09-13T08:50:20.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:20.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:20.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1clone1" 2024-09-13T08:50:20.019 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:20.019 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:50:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 -- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ad4074840 msgr2=0x7f5ad4076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ad4074840 0x7f5ad4076d00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5af40b6140 tx=0x7f5ae00091e0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 -- 172.21.15.44:0/3347242549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5adc004d50 msgr2=0x7f5af4150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5adc004d50 0x7f5af4150a80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8011ae0 tx=0x7f5ae8009ea0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 -- 172.21.15.44:0/3347242549 shutdown_connections 2024-09-13T08:50:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5ad4074840 0x7f5ad4076d00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5af41f2400 0x7f5af40aae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5af41f6700 0x7f5af40c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 --2- 172.21.15.44:0/3347242549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5adc004d50 0x7f5af4150a80 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 -- 172.21.15.44:0/3347242549 >> 172.21.15.44:0/3347242549 conn(0x7f5af40d04f0 msgr2=0x7f5af40b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:20.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 -- 172.21.15.44:0/3347242549 shutdown_connections 2024-09-13T08:50:20.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.018+0000 7f5afc200640 1 -- 172.21.15.44:0/3347242549 wait complete. 2024-09-13T08:50:20.040 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-13T08:50:20.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.186+0000 7ff8fc400640 1 -- 172.21.15.44:0/1585541796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8f40b0b60 msgr2=0x7ff8f40cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.186+0000 7ff8fc400640 1 --2- 172.21.15.44:0/1585541796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8f40b0b60 0x7ff8f40cf180 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4074060 tx=0x7ff8ec01f580 comp rx=0 tx=0).stop 2024-09-13T08:50:20.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.186+0000 7ff8fc400640 1 -- 172.21.15.44:0/1585541796 shutdown_connections 2024-09-13T08:50:20.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.186+0000 7ff8fc400640 1 --2- 172.21.15.44:0/1585541796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8f40b0b60 0x7ff8f40cf180 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.186+0000 7ff8fc400640 1 --2- 172.21.15.44:0/1585541796 >> v2:172.21.15.129:3300/0 conn(0x7ff8f415c1f0 0x7ff8f40cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.186+0000 7ff8fc400640 1 --2- 172.21.15.44:0/1585541796 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8f415be10 0x7ff8f40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.186+0000 7ff8fc400640 1 -- 172.21.15.44:0/1585541796 >> 172.21.15.44:0/1585541796 conn(0x7ff8f40d1da0 msgr2=0x7ff8f40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:20.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.186+0000 7ff8fc400640 1 -- 172.21.15.44:0/1585541796 shutdown_connections 2024-09-13T08:50:20.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.186+0000 7ff8fc400640 1 -- 172.21.15.44:0/1585541796 wait complete. 2024-09-13T08:50:20.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8fc400640 1 Processor -- start 2024-09-13T08:50:20.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8fc400640 1 -- start start 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8fc400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8f40b0b60 0x7ff8f40cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8fc400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8f415be10 0x7ff8f406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8fc400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8f415c1f0 0x7ff8f406f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8fc400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41600a0 con 0x7ff8f415be10 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8fc400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f415ff20 con 0x7ff8f40b0b60 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8fc400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f4160220 con 0x7ff8f415c1f0 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8f415be10 0x7ff8f406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8f415be10 0x7ff8f406dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48678/0 (socket says 172.21.15.44:48678) 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f9000640 1 -- 172.21.15.44:0/137908027 learned_addr learned my addr 172.21.15.44:0/137908027 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f9a00640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8f40b0b60 0x7ff8f40cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:20.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8fa400640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8f415c1f0 0x7ff8f406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:20.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f9000640 1 -- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8f415c1f0 msgr2=0x7ff8f406f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f9000640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8f415c1f0 0x7ff8f406f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f9000640 1 -- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8f40b0b60 msgr2=0x7ff8f40cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f9000640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8f40b0b60 0x7ff8f40cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f9000640 1 -- 172.21.15.44:0/137908027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f40c6ac0 con 0x7ff8f415be10 2024-09-13T08:50:20.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f9000640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8f415be10 0x7ff8f406dd30 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000b410 tx=0x7ff8e000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:20.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f1600640 1 -- 172.21.15.44:0/137908027 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0015020 con 0x7ff8f415be10 2024-09-13T08:50:20.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f1600640 1 -- 172.21.15.44:0/137908027 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8e0011920 con 0x7ff8f415be10 2024-09-13T08:50:20.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8f1600640 1 -- 172.21.15.44:0/137908027 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0011d70 con 0x7ff8f415be10 2024-09-13T08:50:20.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.190+0000 7ff8fc400640 1 -- 172.21.15.44:0/137908027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f41572a0 con 0x7ff8f415be10 2024-09-13T08:50:20.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.194+0000 7ff8fc400640 1 -- 172.21.15.44:0/137908027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f40c7040 con 0x7ff8f415be10 2024-09-13T08:50:20.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.194+0000 7ff8fc400640 1 -- 172.21.15.44:0/137908027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f41606f0 con 0x7ff8f415be10 2024-09-13T08:50:20.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.194+0000 7ff8f1600640 1 -- 172.21.15.44:0/137908027 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7ff8e0018020 con 0x7ff8f415be10 2024-09-13T08:50:20.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.194+0000 7ff8f1600640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8d4074810 0x7ff8d4076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:20.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.194+0000 7ff8f1600640 1 -- 172.21.15.44:0/137908027 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7ff8e0092a60 con 0x7ff8f415be10 2024-09-13T08:50:20.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.194+0000 7ff8f9a00640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8d4074810 0x7ff8d4076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:20.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.198+0000 7ff8f9a00640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8d4074810 0x7ff8d4076cd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009990 tx=0x7ff8e8009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:20.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.198+0000 7ff8f1600640 1 -- 172.21.15.44:0/137908027 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff8e001f020 con 0x7ff8f415be10 2024-09-13T08:50:20.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.406+0000 7ff8fc400640 1 -- 172.21.15.44:0/137908027 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8f40c55e0 con 0x7ff8d4074810 2024-09-13T08:50:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.418+0000 7ff8f1600640 1 -- 172.21.15.44:0/137908027 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff8f40c55e0 con 0x7ff8d4074810 2024-09-13T08:50:20.422 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:20.422 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:50:20.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:20.423 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1" 2024-09-13T08:50:20.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:20.423 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:50:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 -- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8d4074810 msgr2=0x7ff8d4076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8d4074810 0x7ff8d4076cd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009990 tx=0x7ff8e8009110 comp rx=0 tx=0).stop 2024-09-13T08:50:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 -- 172.21.15.44:0/137908027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8f415be10 msgr2=0x7ff8f406dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8f415be10 0x7ff8f406dd30 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000b410 tx=0x7ff8e000b8d0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 -- 172.21.15.44:0/137908027 shutdown_connections 2024-09-13T08:50:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8d4074810 0x7ff8d4076cd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8f415c1f0 0x7ff8f406f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8f415be10 0x7ff8f406dd30 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 --2- 172.21.15.44:0/137908027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8f40b0b60 0x7ff8f40cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 -- 172.21.15.44:0/137908027 >> 172.21.15.44:0/137908027 conn(0x7ff8f40d1da0 msgr2=0x7ff8f40b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 -- 172.21.15.44:0/137908027 shutdown_connections 2024-09-13T08:50:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.422+0000 7ff8fc400640 1 -- 172.21.15.44:0/137908027 wait complete. 2024-09-13T08:50:20.444 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-13T08:50:20.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.594+0000 7f55fde00640 1 -- 172.21.15.44:0/2563983462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f81613c0 msgr2=0x7f55f8161bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.594+0000 7f55fde00640 1 --2- 172.21.15.44:0/2563983462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f81613c0 0x7f55f8161bf0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009900 tx=0x7f55e801c790 comp rx=0 tx=0).stop 2024-09-13T08:50:20.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.594+0000 7f55fde00640 1 -- 172.21.15.44:0/2563983462 shutdown_connections 2024-09-13T08:50:20.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.594+0000 7f55fde00640 1 --2- 172.21.15.44:0/2563983462 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55f80b3190 0x7f55f80b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.594+0000 7f55fde00640 1 --2- 172.21.15.44:0/2563983462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f81613c0 0x7f55f8161bf0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.594+0000 7f55fde00640 1 -- 172.21.15.44:0/2563983462 >> 172.21.15.44:0/2563983462 conn(0x7f55f80d1dd0 msgr2=0x7f55f80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.594+0000 7f55fde00640 1 -- 172.21.15.44:0/2563983462 shutdown_connections 2024-09-13T08:50:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.594+0000 7f55fde00640 1 -- 172.21.15.44:0/2563983462 wait complete. 2024-09-13T08:50:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.594+0000 7f55fde00640 1 Processor -- start 2024-09-13T08:50:20.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.594+0000 7f55fde00640 1 -- start start 2024-09-13T08:50:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55fde00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55f80b3190 0x7f55f81fb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55fde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f81613c0 0x7f55f80cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55fde00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f81617a0 0x7f55f8157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55fde00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55f816b0b0 con 0x7f55f81613c0 2024-09-13T08:50:20.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55fde00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55f816af30 con 0x7f55f81617a0 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55fde00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f55f816b230 con 0x7f55f80b3190 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f81613c0 0x7f55f80cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f81613c0 0x7f55f80cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48696/0 (socket says 172.21.15.44:48696) 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f6000640 1 -- 172.21.15.44:0/3426499550 learned_addr learned my addr 172.21.15.44:0/3426499550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f6a00640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55f80b3190 0x7f55f81fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f7400640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f81617a0 0x7f55f8157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f6000640 1 -- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55f80b3190 msgr2=0x7f55f81fb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f6000640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55f80b3190 0x7f55f81fb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f6000640 1 -- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f81617a0 msgr2=0x7f55f8157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f6000640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f81617a0 0x7f55f8157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f6000640 1 -- 172.21.15.44:0/3426499550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f81fbfa0 con 0x7f55f81613c0 2024-09-13T08:50:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55f6000640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f81613c0 0x7f55f80cce80 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f55e400b410 tx=0x7f55e400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55ef400640 1 -- 172.21.15.44:0/3426499550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4015030 con 0x7f55f81613c0 2024-09-13T08:50:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55ef400640 1 -- 172.21.15.44:0/3426499550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55e4004020 con 0x7f55f81613c0 2024-09-13T08:50:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55fde00640 1 -- 172.21.15.44:0/3426499550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f81fb700 con 0x7f55f81613c0 2024-09-13T08:50:20.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55ef400640 1 -- 172.21.15.44:0/3426499550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e40044b0 con 0x7f55f81613c0 2024-09-13T08:50:20.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55fde00640 1 -- 172.21.15.44:0/3426499550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f81fbcc0 con 0x7f55f81613c0 2024-09-13T08:50:20.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.598+0000 7f55fde00640 1 -- 172.21.15.44:0/3426499550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f80b1450 con 0x7f55f81613c0 2024-09-13T08:50:20.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.602+0000 7f55ef400640 1 -- 172.21.15.44:0/3426499550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7f55e4018070 con 0x7f55f81613c0 2024-09-13T08:50:20.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.602+0000 7f55ef400640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55d8074810 0x7f55d8076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:20.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.602+0000 7f55ef400640 1 -- 172.21.15.44:0/3426499550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7f55e4092b90 con 0x7f55f81613c0 2024-09-13T08:50:20.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.602+0000 7f55f6a00640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55d8074810 0x7f55d8076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:20.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.602+0000 7f55f6a00640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55d8074810 0x7f55d8076cd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f55e80af000 tx=0x7f55e801afd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:20.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.606+0000 7f55ef400640 1 -- 172.21.15.44:0/3426499550 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f55e405f8c0 con 0x7f55f81613c0 2024-09-13T08:50:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.814+0000 7f55fde00640 1 -- 172.21.15.44:0/3426499550 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55f815e3d0 con 0x7f55d8074810 2024-09-13T08:50:20.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55ef400640 1 -- 172.21.15.44:0/3426499550 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f55f815e3d0 con 0x7f55d8074810 2024-09-13T08:50:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:20.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 -- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55d8074810 msgr2=0x7f55d8076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55d8074810 0x7f55d8076cd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f55e80af000 tx=0x7f55e801afd0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 -- 172.21.15.44:0/3426499550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f81613c0 msgr2=0x7f55f80cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f81613c0 0x7f55f80cce80 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f55e400b410 tx=0x7f55e400b8d0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 -- 172.21.15.44:0/3426499550 shutdown_connections 2024-09-13T08:50:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f55d8074810 0x7f55d8076cd0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f81617a0 0x7f55f8157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f81613c0 0x7f55f80cce80 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 --2- 172.21.15.44:0/3426499550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f55f80b3190 0x7f55f81fb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 -- 172.21.15.44:0/3426499550 >> 172.21.15.44:0/3426499550 conn(0x7f55f80d1dd0 msgr2=0x7f55f80b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 -- 172.21.15.44:0/3426499550 shutdown_connections 2024-09-13T08:50:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:20.842+0000 7f55fde00640 1 -- 172.21.15.44:0/3426499550 wait complete. 2024-09-13T08:50:20.866 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-13T08:50:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 -- 172.21.15.44:0/3586395530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1cc0b0950 msgr2=0x7ff1cc1606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 --2- 172.21.15.44:0/3586395530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1cc0b0950 0x7ff1cc1606d0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0c56f0 tx=0x7ff1c401cc90 comp rx=0 tx=0).stop 2024-09-13T08:50:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 -- 172.21.15.44:0/3586395530 shutdown_connections 2024-09-13T08:50:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 --2- 172.21.15.44:0/3586395530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1cc0b0950 0x7ff1cc1606d0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 --2- 172.21.15.44:0/3586395530 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1cc0d0800 0x7ff1cc160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 --2- 172.21.15.44:0/3586395530 >> v2:172.21.15.129:3300/0 conn(0x7ff1cc0cfe50 0x7ff1cc15fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 -- 172.21.15.44:0/3586395530 >> 172.21.15.44:0/3586395530 conn(0x7ff1cc0d1bf0 msgr2=0x7ff1cc0d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 -- 172.21.15.44:0/3586395530 shutdown_connections 2024-09-13T08:50:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 -- 172.21.15.44:0/3586395530 wait complete. 2024-09-13T08:50:21.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 Processor -- start 2024-09-13T08:50:21.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 -- start start 2024-09-13T08:50:21.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1cc0b0950 0x7ff1cc0c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:21.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1cc0cfe50 0x7ff1cc0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:21.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1cc0d0800 0x7ff1cc1f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:21.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc0af9a0 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc0af820 con 0x7ff1cc0d0800 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1cb200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc0afb20 con 0x7ff1cc0b0950 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.014+0000 7ff1c1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1cc0cfe50 0x7ff1cc0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1cc0cfe50 0x7ff1cc0b70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48716/0 (socket says 172.21.15.44:48716) 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c1400640 1 -- 172.21.15.44:0/1401916871 learned_addr learned my addr 172.21.15.44:0/1401916871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c9e00640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1cc0b0950 0x7ff1cc0c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1ca800640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1cc0d0800 0x7ff1cc1f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c1400640 1 -- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1cc0b0950 msgr2=0x7ff1cc0c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c1400640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1cc0b0950 0x7ff1cc0c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c1400640 1 -- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1cc0d0800 msgr2=0x7ff1cc1f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:21.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c1400640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1cc0d0800 0x7ff1cc1f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c1400640 1 -- 172.21.15.44:0/1401916871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc0c7c10 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c1400640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1cc0cfe50 0x7ff1cc0b70d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c9f0 tx=0x7ff1b400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:21.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c2a00640 1 -- 172.21.15.44:0/1401916871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4007b50 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c2a00640 1 -- 172.21.15.44:0/1401916871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1b4007de0 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c2a00640 1 -- 172.21.15.44:0/1401916871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b400f6b0 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1cb200640 1 -- 172.21.15.44:0/1401916871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc0c7ef0 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1cb200640 1 -- 172.21.15.44:0/1401916871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc1f7c90 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1cb200640 1 -- 172.21.15.44:0/1401916871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc0b0040 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.018+0000 7ff1c2a00640 1 -- 172.21.15.44:0/1401916871 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7ff1b400f850 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.022+0000 7ff1c2a00640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a8074840 0x7ff1a8076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:21.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.022+0000 7ff1c2a00640 1 -- 172.21.15.44:0/1401916871 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7ff1b4093220 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.022+0000 7ff1c9e00640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a8074840 0x7ff1a8076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:21.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.022+0000 7ff1c9e00640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a8074840 0x7ff1a8076d00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc005fd0 tx=0x7ff1bc00a2d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:21.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.026+0000 7ff1c2a00640 1 -- 172.21.15.44:0/1401916871 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff1b4014030 con 0x7ff1cc0cfe50 2024-09-13T08:50:21.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.226+0000 7ff1cb200640 1 -- 172.21.15.44:0/1401916871 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff1cc0cf7c0 con 0x7ff1a8074840 2024-09-13T08:50:21.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1c2a00640 1 -- 172.21.15.44:0/1401916871 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff1cc0cf7c0 con 0x7ff1a8074840 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 -- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a8074840 msgr2=0x7ff1a8076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a8074840 0x7ff1a8076d00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc005fd0 tx=0x7ff1bc00a2d0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 -- 172.21.15.44:0/1401916871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1cc0cfe50 msgr2=0x7ff1cc0b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1cc0cfe50 0x7ff1cc0b70d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c9f0 tx=0x7ff1b400ceb0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 -- 172.21.15.44:0/1401916871 shutdown_connections 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff1a8074840 0x7ff1a8076d00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1cc0d0800 0x7ff1cc1f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1cc0cfe50 0x7ff1cc0b70d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 --2- 172.21.15.44:0/1401916871 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff1cc0b0950 0x7ff1cc0c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 -- 172.21.15.44:0/1401916871 >> 172.21.15.44:0/1401916871 conn(0x7ff1cc0d1bf0 msgr2=0x7ff1cc0cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:21.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 -- 172.21.15.44:0/1401916871 shutdown_connections 2024-09-13T08:50:21.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.250+0000 7ff1cb200640 1 -- 172.21.15.44:0/1401916871 wait complete. 2024-09-13T08:50:21.266 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:21.265+0000 7f8ac7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:21.266 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:21.265+0000 7f8ac7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:21.266 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:21.265+0000 7f8ac7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:21.266 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:21.265+0000 7f8ac7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:21.266 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:21.265+0000 7f8ac7a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:21.275 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-13T08:50:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 -- 172.21.15.44:0/636987112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a140b0b60 msgr2=0x7f6a140ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 --2- 172.21.15.44:0/636987112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a140b0b60 0x7f6a140ccbd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f6a140746b0 tx=0x7f6a0c01f370 comp rx=0 tx=0).stop 2024-09-13T08:50:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 -- 172.21.15.44:0/636987112 shutdown_connections 2024-09-13T08:50:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 --2- 172.21.15.44:0/636987112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a140b0b60 0x7f6a140ccbd0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 --2- 172.21.15.44:0/636987112 >> v2:172.21.15.129:3300/0 conn(0x7f6a1415c1f0 0x7f6a140cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 -- 172.21.15.44:0/636987112 >> 172.21.15.44:0/636987112 conn(0x7f6a140d1da0 msgr2=0x7f6a140d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 -- 172.21.15.44:0/636987112 shutdown_connections 2024-09-13T08:50:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 -- 172.21.15.44:0/636987112 wait complete. 2024-09-13T08:50:21.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 Processor -- start 2024-09-13T08:50:21.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 -- start start 2024-09-13T08:50:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a140b0b60 0x7f6a141567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a1415be10 0x7f6a140c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a1415c1f0 0x7f6a140afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a140cd4a0 con 0x7f6a1415be10 2024-09-13T08:50:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a140cd320 con 0x7f6a1415c1f0 2024-09-13T08:50:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a1b800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6a140cd620 con 0x7f6a140b0b60 2024-09-13T08:50:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a0be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a1415be10 0x7f6a140c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a0be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a1415be10 0x7f6a140c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48738/0 (socket says 172.21.15.44:48738) 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a0be00640 1 -- 172.21.15.44:0/1567722659 learned_addr learned my addr 172.21.15.44:0/1567722659 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a18e00640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a140b0b60 0x7f6a141567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.426+0000 7f6a19800640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a1415c1f0 0x7f6a140afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a0be00640 1 -- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a140b0b60 msgr2=0x7f6a141567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a0be00640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a140b0b60 0x7f6a141567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a0be00640 1 -- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a1415c1f0 msgr2=0x7f6a140afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a0be00640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a1415c1f0 0x7f6a140afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a0be00640 1 -- 172.21.15.44:0/1567722659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a14156d00 con 0x7f6a1415be10 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a0be00640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a1415be10 0x7f6a140c7f00 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00b440 tx=0x7f69fc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a11600640 1 -- 172.21.15.44:0/1567722659 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc015020 con 0x7f6a1415be10 2024-09-13T08:50:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a11600640 1 -- 172.21.15.44:0/1567722659 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69fc0119a0 con 0x7f6a1415be10 2024-09-13T08:50:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a11600640 1 -- 172.21.15.44:0/1567722659 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc011df0 con 0x7f6a1415be10 2024-09-13T08:50:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a1b800640 1 -- 172.21.15.44:0/1567722659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1415ca60 con 0x7f6a1415be10 2024-09-13T08:50:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a1b800640 1 -- 172.21.15.44:0/1567722659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a1415d020 con 0x7f6a1415be10 2024-09-13T08:50:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a1b800640 1 -- 172.21.15.44:0/1567722659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a140cdb40 con 0x7f6a1415be10 2024-09-13T08:50:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a11600640 1 -- 172.21.15.44:0/1567722659 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7f69fc018020 con 0x7f6a1415be10 2024-09-13T08:50:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.430+0000 7f6a11600640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69f0074840 0x7f69f0076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.434+0000 7f6a11600640 1 -- 172.21.15.44:0/1567722659 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7f69fc092b30 con 0x7f6a1415be10 2024-09-13T08:50:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.434+0000 7f6a18e00640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69f0074840 0x7f69f0076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.434+0000 7f6a18e00640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69f0074840 0x7f69f0076d00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6a04001910 tx=0x7f6a04001420 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:21.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.434+0000 7f6a11600640 1 -- 172.21.15.44:0/1567722659 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f69fc01f020 con 0x7f6a1415be10 2024-09-13T08:50:21.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.642+0000 7f6a1b800640 1 -- 172.21.15.44:0/1567722659 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f6a14070350 con 0x7f69f0074840 2024-09-13T08:50:21.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a11600640 1 -- 172.21.15.44:0/1567722659 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a14070350 con 0x7f69f0074840 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 -- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69f0074840 msgr2=0x7f69f0076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69f0074840 0x7f69f0076d00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6a04001910 tx=0x7f6a04001420 comp rx=0 tx=0).stop 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 -- 172.21.15.44:0/1567722659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a1415be10 msgr2=0x7f6a140c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a1415be10 0x7f6a140c7f00 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00b440 tx=0x7f69fc00b900 comp rx=0 tx=0).stop 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 -- 172.21.15.44:0/1567722659 shutdown_connections 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f69f0074840 0x7f69f0076d00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a1415c1f0 0x7f6a140afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a1415be10 0x7f6a140c7f00 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 --2- 172.21.15.44:0/1567722659 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a140b0b60 0x7f6a141567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 -- 172.21.15.44:0/1567722659 >> 172.21.15.44:0/1567722659 conn(0x7f6a140d1da0 msgr2=0x7f6a140b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:21.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 -- 172.21.15.44:0/1567722659 shutdown_connections 2024-09-13T08:50:21.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21.662+0000 7f6a1b800640 1 -- 172.21.15.44:0/1567722659 wait complete. 2024-09-13T08:50:21.676 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:21.677+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:21.676 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:21.677+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:21.676 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:21.677+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:21.676 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:21.677+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:21.676 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:21.677+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:21.688 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-13T08:50:21.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:50:21.997 INFO:teuthology.orchestra.run.smithi044.stdout:2 2024-09-13T08:50:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:50:22.004 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:50:22.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.150+0000 7f9a6ca00640 1 -- 172.21.15.44:0/2947847799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6815dd70 msgr2=0x7f9a680ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:22.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.150+0000 7f9a6ca00640 1 --2- 172.21.15.44:0/2947847799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6815dd70 0x7f9a680ccd60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f9a680d1c00 tx=0x7f9a5801c890 comp rx=0 tx=0).stop 2024-09-13T08:50:22.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.150+0000 7f9a6ca00640 1 -- 172.21.15.44:0/2947847799 shutdown_connections 2024-09-13T08:50:22.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.150+0000 7f9a6ca00640 1 --2- 172.21.15.44:0/2947847799 >> v2:172.21.15.129:3300/0 conn(0x7f9a6806d330 0x7f9a68073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.150+0000 7f9a6ca00640 1 --2- 172.21.15.44:0/2947847799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6815dd70 0x7f9a680ccd60 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.150+0000 7f9a6ca00640 1 -- 172.21.15.44:0/2947847799 >> 172.21.15.44:0/2947847799 conn(0x7f9a6806dd30 msgr2=0x7f9a6806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:22.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.150+0000 7f9a6ca00640 1 -- 172.21.15.44:0/2947847799 shutdown_connections 2024-09-13T08:50:22.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.150+0000 7f9a6ca00640 1 -- 172.21.15.44:0/2947847799 wait complete. 2024-09-13T08:50:22.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a6ca00640 1 Processor -- start 2024-09-13T08:50:22.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a6ca00640 1 -- start start 2024-09-13T08:50:22.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a6ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9a6806d330 0x7f9a680c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:22.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a6ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a680cc650 0x7f9a681fb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a6ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6815dd70 0x7f9a681fb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a6ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6816b1f0 con 0x7f9a6815dd70 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a6ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6816b070 con 0x7f9a680cc650 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a6ca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6816b370 con 0x7f9a6806d330 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a67e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6815dd70 0x7f9a681fb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a67e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6815dd70 0x7f9a681fb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39018/0 (socket says 172.21.15.44:39018) 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a67e00640 1 -- 172.21.15.44:0/1457064621 learned_addr learned my addr 172.21.15.44:0/1457064621 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a5ea00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a680cc650 0x7f9a681fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a67400640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9a6806d330 0x7f9a680c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a67e00640 1 -- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9a6806d330 msgr2=0x7f9a680c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a67e00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9a6806d330 0x7f9a680c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a67e00640 1 -- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a680cc650 msgr2=0x7f9a681fb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a67e00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a680cc650 0x7f9a681fb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a67e00640 1 -- 172.21.15.44:0/1457064621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a680a1cc0 con 0x7f9a6815dd70 2024-09-13T08:50:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a67e00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6815dd70 0x7f9a681fb5d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000ba10 tx=0x7f9a6000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a5fe00640 1 -- 172.21.15.44:0/1457064621 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60019070 con 0x7f9a6815dd70 2024-09-13T08:50:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a5fe00640 1 -- 172.21.15.44:0/1457064621 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a6000cc30 con 0x7f9a6815dd70 2024-09-13T08:50:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a6ca00640 1 -- 172.21.15.44:0/1457064621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6809ee20 con 0x7f9a6815dd70 2024-09-13T08:50:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a5fe00640 1 -- 172.21.15.44:0/1457064621 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a600146a0 con 0x7f9a6815dd70 2024-09-13T08:50:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.154+0000 7f9a6ca00640 1 -- 172.21.15.44:0/1457064621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a68156350 con 0x7f9a6815dd70 2024-09-13T08:50:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.158+0000 7f9a6ca00640 1 -- 172.21.15.44:0/1457064621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a6809f210 con 0x7f9a6815dd70 2024-09-13T08:50:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.158+0000 7f9a5fe00640 1 -- 172.21.15.44:0/1457064621 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7f9a6001c070 con 0x7f9a6815dd70 2024-09-13T08:50:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.158+0000 7f9a5fe00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9a44074810 0x7f9a44076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.158+0000 7f9a5fe00640 1 -- 172.21.15.44:0/1457064621 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7f9a600928f0 con 0x7f9a6815dd70 2024-09-13T08:50:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.158+0000 7f9a67400640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9a44074810 0x7f9a44076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.158+0000 7f9a67400640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9a44074810 0x7f9a44076cd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9a580af000 tx=0x7f9a5801b0d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:22.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.162+0000 7f9a5fe00640 1 -- 172.21.15.44:0/1457064621 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9a6000ce20 con 0x7f9a6815dd70 2024-09-13T08:50:22.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.342+0000 7f9a6ca00640 1 -- 172.21.15.44:0/1457064621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f9a68073c00 con 0x7f9a6815dd70 2024-09-13T08:50:22.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.342+0000 7f9a5fe00640 1 -- 172.21.15.44:0/1457064621 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7f9a6005ef50 con 0x7f9a6815dd70 2024-09-13T08:50:22.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 -- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9a44074810 msgr2=0x7f9a44076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:22.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9a44074810 0x7f9a44076cd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9a580af000 tx=0x7f9a5801b0d0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 -- 172.21.15.44:0/1457064621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6815dd70 msgr2=0x7f9a681fb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6815dd70 0x7f9a681fb5d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000ba10 tx=0x7f9a6000bed0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 -- 172.21.15.44:0/1457064621 shutdown_connections 2024-09-13T08:50:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9a44074810 0x7f9a44076cd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6815dd70 0x7f9a681fb5d0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a680cc650 0x7f9a681fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 --2- 172.21.15.44:0/1457064621 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9a6806d330 0x7f9a680c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 -- 172.21.15.44:0/1457064621 >> 172.21.15.44:0/1457064621 conn(0x7f9a6806dd30 msgr2=0x7f9a68161ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 -- 172.21.15.44:0/1457064621 shutdown_connections 2024-09-13T08:50:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.346+0000 7f9a6ca00640 1 -- 172.21.15.44:0/1457064621 wait complete. 2024-09-13T08:50:22.366 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:50:22.366 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:50:22.454 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:50:22.545 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:50:22.546 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:50:22.546 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:50:22.546 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:22.555 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:50:22.555 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:22.607 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:50:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.798+0000 7fe143200640 1 -- 172.21.15.44:0/3959969493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe13c0cd8c0 msgr2=0x7fe13c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.798+0000 7fe143200640 1 --2- 172.21.15.44:0/3959969493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe13c0cd8c0 0x7fe13c0cea20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe134009bc0 tx=0x7fe13401ca30 comp rx=0 tx=0).stop 2024-09-13T08:50:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.798+0000 7fe143200640 1 -- 172.21.15.44:0/3959969493 shutdown_connections 2024-09-13T08:50:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.798+0000 7fe143200640 1 --2- 172.21.15.44:0/3959969493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe13c0cd8c0 0x7fe13c0cea20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.798+0000 7fe143200640 1 --2- 172.21.15.44:0/3959969493 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe13c15be80 0x7fe13c0ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.798+0000 7fe143200640 1 -- 172.21.15.44:0/3959969493 >> 172.21.15.44:0/3959969493 conn(0x7fe13c0d1da0 msgr2=0x7fe13c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.798+0000 7fe143200640 1 -- 172.21.15.44:0/3959969493 shutdown_connections 2024-09-13T08:50:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.798+0000 7fe143200640 1 -- 172.21.15.44:0/3959969493 wait complete. 2024-09-13T08:50:22.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe143200640 1 Processor -- start 2024-09-13T08:50:22.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe143200640 1 -- start start 2024-09-13T08:50:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe143200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe13c0cd8c0 0x7fe13c1f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe143200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe13c0d28c0 0x7fe13c0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe143200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe13c15be80 0x7fe13c0c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe143200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c0b5e90 con 0x7fe13c0d28c0 2024-09-13T08:50:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe143200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c0b5d10 con 0x7fe13c15be80 2024-09-13T08:50:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe143200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe13c0b6010 con 0x7fe13c0cd8c0 2024-09-13T08:50:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe133e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe13c0d28c0 0x7fe13c0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe133e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe13c0d28c0 0x7fe13c0c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39030/0 (socket says 172.21.15.44:39030) 2024-09-13T08:50:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe133e00640 1 -- 172.21.15.44:0/1069022905 learned_addr learned my addr 172.21.15.44:0/1069022905 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:22.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe141200640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe13c15be80 0x7fe13c0c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:22.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe13be00640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe13c0cd8c0 0x7fe13c1f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:22.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe133e00640 1 -- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe13c0cd8c0 msgr2=0x7fe13c1f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:22.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe133e00640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe13c0cd8c0 0x7fe13c1f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe133e00640 1 -- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe13c15be80 msgr2=0x7fe13c0c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:22.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe133e00640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe13c15be80 0x7fe13c0c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:22.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe133e00640 1 -- 172.21.15.44:0/1069022905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c0c99e0 con 0x7fe13c0d28c0 2024-09-13T08:50:22.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe133e00640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe13c0d28c0 0x7fe13c0c8e70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe12400faa0 tx=0x7fe12400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe138c00640 1 -- 172.21.15.44:0/1069022905 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12401a070 con 0x7fe13c0d28c0 2024-09-13T08:50:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe138c00640 1 -- 172.21.15.44:0/1069022905 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe12400d730 con 0x7fe13c0d28c0 2024-09-13T08:50:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe138c00640 1 -- 172.21.15.44:0/1069022905 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12400db80 con 0x7fe13c0d28c0 2024-09-13T08:50:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe143200640 1 -- 172.21.15.44:0/1069022905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13c0c9cc0 con 0x7fe13c0d28c0 2024-09-13T08:50:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.802+0000 7fe143200640 1 -- 172.21.15.44:0/1069022905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13c0ca280 con 0x7fe13c0d28c0 2024-09-13T08:50:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.806+0000 7fe143200640 1 -- 172.21.15.44:0/1069022905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe13c0b6530 con 0x7fe13c0d28c0 2024-09-13T08:50:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.806+0000 7fe138c00640 1 -- 172.21.15.44:0/1069022905 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7fe12401d070 con 0x7fe13c0d28c0 2024-09-13T08:50:22.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.806+0000 7fe138c00640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe118074810 0x7fe118076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:22.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.806+0000 7fe138c00640 1 -- 172.21.15.44:0/1069022905 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7fe124095540 con 0x7fe13c0d28c0 2024-09-13T08:50:22.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.806+0000 7fe13be00640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe118074810 0x7fe118076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:22.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.806+0000 7fe13be00640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe118074810 0x7fe118076cd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009ad0 tx=0x7fe12c009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.810+0000 7fe138c00640 1 -- 172.21.15.44:0/1069022905 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe13c0d0350 con 0x7fe13c0d28c0 2024-09-13T08:50:23.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.998+0000 7fe143200640 1 -- 172.21.15.44:0/1069022905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe13c0b4590 con 0x7fe13c0d28c0 2024-09-13T08:50:23.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:22.998+0000 7fe138c00640 1 -- 172.21.15.44:0/1069022905 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v104) ==== 80+0+0 (secure 0 0 0) 0x7fe1240621f0 con 0x7fe13c0d28c0 2024-09-13T08:50:23.001 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:50:23.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 -- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe118074810 msgr2=0x7fe118076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:23.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe118074810 0x7fe118076cd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009ad0 tx=0x7fe12c009260 comp rx=0 tx=0).stop 2024-09-13T08:50:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 -- 172.21.15.44:0/1069022905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe13c0d28c0 msgr2=0x7fe13c0c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe13c0d28c0 0x7fe13c0c8e70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe12400faa0 tx=0x7fe12400ff60 comp rx=0 tx=0).stop 2024-09-13T08:50:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 -- 172.21.15.44:0/1069022905 shutdown_connections 2024-09-13T08:50:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe118074810 0x7fe118076cd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe13c15be80 0x7fe13c0c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe13c0d28c0 0x7fe13c0c8e70 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 --2- 172.21.15.44:0/1069022905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe13c0cd8c0 0x7fe13c1f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 -- 172.21.15.44:0/1069022905 >> 172.21.15.44:0/1069022905 conn(0x7fe13c0d1da0 msgr2=0x7fe13c0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 -- 172.21.15.44:0/1069022905 shutdown_connections 2024-09-13T08:50:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.002+0000 7fe143200640 1 -- 172.21.15.44:0/1069022905 wait complete. 2024-09-13T08:50:23.023 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T08:50:23.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:23.023 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T08:50:23.079 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:50:23.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:23.079 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T08:50:23.097 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:50:23.097 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:50:23.098 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:23.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:50:23.109 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:50:23.109 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:50:23.109 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:23.151 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:50:23.151 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:50:23.151 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:50:23.151 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:23.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:50:23.160 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:50:23.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.338+0000 7fe74c800640 1 -- 172.21.15.44:0/1755624931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7440cf4e0 msgr2=0x7fe7440cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:23.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.338+0000 7fe74c800640 1 --2- 172.21.15.44:0/1755624931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7440cf4e0 0x7fe7440cd950 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fe744073a20 tx=0x7fe74001cc60 comp rx=0 tx=0).stop 2024-09-13T08:50:23.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.338+0000 7fe74c800640 1 -- 172.21.15.44:0/1755624931 shutdown_connections 2024-09-13T08:50:23.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.338+0000 7fe74c800640 1 --2- 172.21.15.44:0/1755624931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7440cf4e0 0x7fe7440cd950 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.338+0000 7fe74c800640 1 --2- 172.21.15.44:0/1755624931 >> v2:172.21.15.129:3300/0 conn(0x7fe74415bd50 0x7fe7440ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.338+0000 7fe74c800640 1 -- 172.21.15.44:0/1755624931 >> 172.21.15.44:0/1755624931 conn(0x7fe7440d1ea0 msgr2=0x7fe7440d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:23.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.338+0000 7fe74c800640 1 -- 172.21.15.44:0/1755624931 shutdown_connections 2024-09-13T08:50:23.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe74c800640 1 -- 172.21.15.44:0/1755624931 wait complete. 2024-09-13T08:50:23.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe74c800640 1 Processor -- start 2024-09-13T08:50:23.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe74c800640 1 -- start start 2024-09-13T08:50:23.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe74c800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7440cf4e0 0x7fe744149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:23.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe74c800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe74415bd50 0x7fe7440c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:23.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe74c800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe74415c130 0x7fe7440c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:23.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe74c800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe744168ea0 con 0x7fe7440cf4e0 2024-09-13T08:50:23.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe74c800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe744168d20 con 0x7fe74415c130 2024-09-13T08:50:23.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe74c800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe744169020 con 0x7fe74415bd50 2024-09-13T08:50:23.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe749e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7440cf4e0 0x7fe744149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe749e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7440cf4e0 0x7fe744149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39062/0 (socket says 172.21.15.44:39062) 2024-09-13T08:50:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe749e00640 1 -- 172.21.15.44:0/2331812384 learned_addr learned my addr 172.21.15.44:0/2331812384 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe74a800640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe74415c130 0x7fe7440c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe749400640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe74415bd50 0x7fe7440c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe749e00640 1 -- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe74415bd50 msgr2=0x7fe7440c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe749e00640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe74415bd50 0x7fe7440c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe749e00640 1 -- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe74415c130 msgr2=0x7fe7440c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe749e00640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe74415c130 0x7fe7440c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe749e00640 1 -- 172.21.15.44:0/2331812384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7440c3980 con 0x7fe7440cf4e0 2024-09-13T08:50:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.342+0000 7fe749e00640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7440cf4e0 0x7fe744149f60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe73800e940 tx=0x7fe73800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:23.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.346+0000 7fe73e000640 1 -- 172.21.15.44:0/2331812384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73800cd80 con 0x7fe7440cf4e0 2024-09-13T08:50:23.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.346+0000 7fe73e000640 1 -- 172.21.15.44:0/2331812384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe738005020 con 0x7fe7440cf4e0 2024-09-13T08:50:23.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.346+0000 7fe73e000640 1 -- 172.21.15.44:0/2331812384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738005450 con 0x7fe7440cf4e0 2024-09-13T08:50:23.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.346+0000 7fe74c800640 1 -- 172.21.15.44:0/2331812384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7440c3c60 con 0x7fe7440cf4e0 2024-09-13T08:50:23.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.346+0000 7fe74c800640 1 -- 172.21.15.44:0/2331812384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe744159c60 con 0x7fe7440cf4e0 2024-09-13T08:50:23.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.346+0000 7fe74c800640 1 -- 172.21.15.44:0/2331812384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7440ce5e0 con 0x7fe7440cf4e0 2024-09-13T08:50:23.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.350+0000 7fe73e000640 1 -- 172.21.15.44:0/2331812384 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 99375+0+0 (secure 0 0 0) 0x7fe7380047b0 con 0x7fe7440cf4e0 2024-09-13T08:50:23.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.350+0000 7fe73e000640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe728074810 0x7fe728076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:23.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.350+0000 7fe73e000640 1 -- 172.21.15.44:0/2331812384 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7fe738093740 con 0x7fe7440cf4e0 2024-09-13T08:50:23.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.350+0000 7fe749400640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe728074810 0x7fe728076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:23.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.350+0000 7fe749400640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe728074810 0x7fe728076cd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe744071170 tx=0x7fe734009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:23.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.358+0000 7fe73e000640 1 -- 172.21.15.44:0/2331812384 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe7380603f0 con 0x7fe7440cf4e0 2024-09-13T08:50:23.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.690+0000 7fe73e000640 1 -- 172.21.15.44:0/2331812384 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fe73805bf60 con 0x7fe7440cf4e0 2024-09-13T08:50:23.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.742+0000 7fe74c800640 1 -- 172.21.15.44:0/2331812384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7440718f0 con 0x7fe7440cf4e0 2024-09-13T08:50:23.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe73e000640 1 -- 172.21.15.44:0/2331812384 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5225 (secure 0 0 0) 0x7fe73806c6e0 con 0x7fe7440cf4e0 2024-09-13T08:50:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:23.747 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":66,"btime":"2024-09-13T08:49:36:320618+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1987308515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1987308515},{"type":"v1","addr":"172.21.15.129:6837","nonce":1987308515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1224713128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1224713128},{"type":"v1","addr":"172.21.15.129:6836","nonce":1224713128}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7755,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/380853907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":380853907},{"type":"v1","addr":"172.21.15.44:6833","nonce":380853907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:49:27.697092+0000","modified":"2024-09-13T08:49:30.701664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/409896091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":409896091},{"type":"v1","addr":"172.21.15.44:6835","nonce":409896091}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7758,"qdb_cluster":[7758]},"id":7}]} 2024-09-13T08:50:23.748 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 66 2024-09-13T08:50:23.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 -- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe728074810 msgr2=0x7fe728076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe728074810 0x7fe728076cd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe744071170 tx=0x7fe734009040 comp rx=0 tx=0).stop 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 -- 172.21.15.44:0/2331812384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7440cf4e0 msgr2=0x7fe744149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7440cf4e0 0x7fe744149f60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe73800e940 tx=0x7fe73800ee00 comp rx=0 tx=0).stop 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 -- 172.21.15.44:0/2331812384 shutdown_connections 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe728074810 0x7fe728076cd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe74415c130 0x7fe7440c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe74415bd50 0x7fe7440c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 --2- 172.21.15.44:0/2331812384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7440cf4e0 0x7fe744149f60 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 -- 172.21.15.44:0/2331812384 >> 172.21.15.44:0/2331812384 conn(0x7fe7440d1ea0 msgr2=0x7fe7441593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 -- 172.21.15.44:0/2331812384 shutdown_connections 2024-09-13T08:50:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.746+0000 7fe74c800640 1 -- 172.21.15.44:0/2331812384 wait complete. 2024-09-13T08:50:23.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:50:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.914+0000 7f424f200640 1 -- 172.21.15.44:0/3105672163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42480b24b0 msgr2=0x7f42480ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.914+0000 7f424f200640 1 --2- 172.21.15.44:0/3105672163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42480b24b0 0x7f42480ce4e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4238009840 tx=0x7f423801c6d0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.914+0000 7f424f200640 1 -- 172.21.15.44:0/3105672163 shutdown_connections 2024-09-13T08:50:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.914+0000 7f424f200640 1 --2- 172.21.15.44:0/3105672163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42480b24b0 0x7f42480ce4e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.914+0000 7f424f200640 1 -- 172.21.15.44:0/3105672163 >> 172.21.15.44:0/3105672163 conn(0x7f424809aa10 msgr2=0x7f424809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.914+0000 7f424f200640 1 -- 172.21.15.44:0/3105672163 shutdown_connections 2024-09-13T08:50:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.914+0000 7f424f200640 1 -- 172.21.15.44:0/3105672163 wait complete. 2024-09-13T08:50:23.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 Processor -- start 2024-09-13T08:50:23.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 -- start start 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42480b24b0 0x7f42480d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42480cea20 0x7f4248150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42480d1b60 0x7f4248152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f424816c930 con 0x7f42480b24b0 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f424816c7b0 con 0x7f42480d1b60 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f424816cab0 con 0x7f42480cea20 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42480b24b0 0x7f42480d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42480b24b0 0x7f42480d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39070/0 (socket says 172.21.15.44:39070) 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424de00640 1 -- 172.21.15.44:0/619542346 learned_addr learned my addr 172.21.15.44:0/619542346 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424d400640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42480cea20 0x7f4248150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424e800640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42480d1b60 0x7f4248152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424de00640 1 -- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42480cea20 msgr2=0x7f4248150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424de00640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42480cea20 0x7f4248150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424de00640 1 -- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42480d1b60 msgr2=0x7f4248152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424de00640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42480d1b60 0x7f4248152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424de00640 1 -- 172.21.15.44:0/619542346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42480aec10 con 0x7f42480b24b0 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424de00640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42480b24b0 0x7f42480d13e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c450 tx=0x7f423c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f4242000640 1 -- 172.21.15.44:0/619542346 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c0052d0 con 0x7f42480b24b0 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f4242000640 1 -- 172.21.15.44:0/619542346 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f423c005470 con 0x7f42480b24b0 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 -- 172.21.15.44:0/619542346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42480aeef0 con 0x7f42480b24b0 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f4242000640 1 -- 172.21.15.44:0/619542346 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c012870 con 0x7f42480b24b0 2024-09-13T08:50:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 -- 172.21.15.44:0/619542346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42480af4b0 con 0x7f42480b24b0 2024-09-13T08:50:23.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.918+0000 7f424f200640 1 -- 172.21.15.44:0/619542346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42480ce4e0 con 0x7f42480b24b0 2024-09-13T08:50:23.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.922+0000 7f4242000640 1 -- 172.21.15.44:0/619542346 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f423c003600 con 0x7f42480b24b0 2024-09-13T08:50:23.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.922+0000 7f4242000640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f422c074960 0x7f422c076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:23.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.922+0000 7f4242000640 1 -- 172.21.15.44:0/619542346 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7f423c092670 con 0x7f42480b24b0 2024-09-13T08:50:23.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.922+0000 7f424d400640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f422c074960 0x7f422c076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.922+0000 7f424d400640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f422c074960 0x7f422c076e20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f42380af000 tx=0x7f423801af10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:23.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:23.926+0000 7f4242000640 1 -- 172.21.15.44:0/619542346 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f423c05ecd0 con 0x7f42480b24b0 2024-09-13T08:50:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.138+0000 7f424f200640 1 -- 172.21.15.44:0/619542346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42480c8a10 con 0x7f42480b24b0 2024-09-13T08:50:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.138+0000 7f4242000640 1 -- 172.21.15.44:0/619542346 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5225 (secure 0 0 0) 0x7f423c063700 con 0x7f42480b24b0 2024-09-13T08:50:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:24.143 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":66,"btime":"2024-09-13T08:49:36:320618+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1987308515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1987308515},{"type":"v1","addr":"172.21.15.129:6837","nonce":1987308515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1224713128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1224713128},{"type":"v1","addr":"172.21.15.129:6836","nonce":1224713128}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7755,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/380853907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":380853907},{"type":"v1","addr":"172.21.15.44:6833","nonce":380853907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:49:27.697092+0000","modified":"2024-09-13T08:49:30.701664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/409896091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":409896091},{"type":"v1","addr":"172.21.15.44:6835","nonce":409896091}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7758,"qdb_cluster":[7758]},"id":7}]} 2024-09-13T08:50:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 66 2024-09-13T08:50:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 -- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f422c074960 msgr2=0x7f422c076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f422c074960 0x7f422c076e20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f42380af000 tx=0x7f423801af10 comp rx=0 tx=0).stop 2024-09-13T08:50:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 -- 172.21.15.44:0/619542346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42480b24b0 msgr2=0x7f42480d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42480b24b0 0x7f42480d13e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c450 tx=0x7f423c00c910 comp rx=0 tx=0).stop 2024-09-13T08:50:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 -- 172.21.15.44:0/619542346 shutdown_connections 2024-09-13T08:50:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f422c074960 0x7f422c076e20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42480d1b60 0x7f4248152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f42480cea20 0x7f4248150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 --2- 172.21.15.44:0/619542346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42480b24b0 0x7f42480d13e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 -- 172.21.15.44:0/619542346 >> 172.21.15.44:0/619542346 conn(0x7f424809aa10 msgr2=0x7f42480d1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 -- 172.21.15.44:0/619542346 shutdown_connections 2024-09-13T08:50:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.142+0000 7f424f200640 1 -- 172.21.15.44:0/619542346 wait complete. 2024-09-13T08:50:24.165 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:50:24.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.310+0000 7f78fd000640 1 -- 172.21.15.44:0/4049085182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78f8155080 msgr2=0x7f78f80cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:24.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.310+0000 7f78fd000640 1 --2- 172.21.15.44:0/4049085182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78f8155080 0x7f78f80cd510 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009840 tx=0x7f78e801c7c0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.310+0000 7f78fd000640 1 -- 172.21.15.44:0/4049085182 shutdown_connections 2024-09-13T08:50:24.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.310+0000 7f78fd000640 1 --2- 172.21.15.44:0/4049085182 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f78f80cba70 0x7f78f80cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.310+0000 7f78fd000640 1 --2- 172.21.15.44:0/4049085182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78f8155080 0x7f78f80cd510 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.310+0000 7f78fd000640 1 -- 172.21.15.44:0/4049085182 >> 172.21.15.44:0/4049085182 conn(0x7f78f806dd30 msgr2=0x7f78f806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.310+0000 7f78fd000640 1 -- 172.21.15.44:0/4049085182 shutdown_connections 2024-09-13T08:50:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.310+0000 7f78fd000640 1 -- 172.21.15.44:0/4049085182 wait complete. 2024-09-13T08:50:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78fd000640 1 Processor -- start 2024-09-13T08:50:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78fd000640 1 -- start start 2024-09-13T08:50:24.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78fd000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78f80cba70 0x7f78f80c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:24.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78fd000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f78f80cda50 0x7f78f80c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:24.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78fd000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f8155080 0x7f78f8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:24.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78fd000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8167ae0 con 0x7f78f80cba70 2024-09-13T08:50:24.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78fd000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8167960 con 0x7f78f8155080 2024-09-13T08:50:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78fd000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f78f8167c60 con 0x7f78f80cda50 2024-09-13T08:50:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78f80cba70 0x7f78f80c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78f80cba70 0x7f78f80c36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39084/0 (socket says 172.21.15.44:39084) 2024-09-13T08:50:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7400640 1 -- 172.21.15.44:0/3259501079 learned_addr learned my addr 172.21.15.44:0/3259501079 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78efe00640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f78f80cda50 0x7f78f80c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7e00640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f8155080 0x7f78f8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7400640 1 -- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f78f80cda50 msgr2=0x7f78f80c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7400640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f78f80cda50 0x7f78f80c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7400640 1 -- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f8155080 msgr2=0x7f78f8070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7400640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f8155080 0x7f78f8070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7400640 1 -- 172.21.15.44:0/3259501079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f80c6d00 con 0x7f78f80cba70 2024-09-13T08:50:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7e00640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f8155080 0x7f78f8070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78f7400640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78f80cba70 0x7f78f80c36c0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009790 tx=0x7f78e80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78ef400640 1 -- 172.21.15.44:0/3259501079 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e80b2040 con 0x7f78f80cba70 2024-09-13T08:50:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78ef400640 1 -- 172.21.15.44:0/3259501079 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78e8005020 con 0x7f78f80cba70 2024-09-13T08:50:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78fd000640 1 -- 172.21.15.44:0/3259501079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f80c7560 con 0x7f78f80cba70 2024-09-13T08:50:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78ef400640 1 -- 172.21.15.44:0/3259501079 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8005450 con 0x7f78f80cba70 2024-09-13T08:50:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.314+0000 7f78fd000640 1 -- 172.21.15.44:0/3259501079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f80c7280 con 0x7f78f80cba70 2024-09-13T08:50:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.318+0000 7f78fd000640 1 -- 172.21.15.44:0/3259501079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f80cea20 con 0x7f78f80cba70 2024-09-13T08:50:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.318+0000 7f78ef400640 1 -- 172.21.15.44:0/3259501079 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f78e80be060 con 0x7f78f80cba70 2024-09-13T08:50:24.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.318+0000 7f78ef400640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f78d4074960 0x7f78d4076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.318+0000 7f78ef400640 1 -- 172.21.15.44:0/3259501079 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5851+0+0 (secure 0 0 0) 0x7f78e812c4b0 con 0x7f78f80cba70 2024-09-13T08:50:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.318+0000 7f78efe00640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f78d4074960 0x7f78d4076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.322+0000 7f78efe00640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f78d4074960 0x7f78d4076e20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f78f8159280 tx=0x7f78e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.322+0000 7f78ef400640 1 -- 172.21.15.44:0/3259501079 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f78e8016410 con 0x7f78f80cba70 2024-09-13T08:50:24.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.642+0000 7f78fd000640 1 -- 172.21.15.44:0/3259501079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78f80b0340 con 0x7f78f80cba70 2024-09-13T08:50:24.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.646+0000 7f78ef400640 1 -- 172.21.15.44:0/3259501079 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16921 (secure 0 0 0) 0x7f78e80f9200 con 0x7f78f80cba70 2024-09-13T08:50:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:24.648 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":104,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:50:17.567938+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:49:25.408514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-13T08:49:26.657487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":5,"to":0}]},{"pgid":"14.1e","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:50:24.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 -- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f78d4074960 msgr2=0x7f78d4076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:24.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f78d4074960 0x7f78d4076e20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f78f8159280 tx=0x7f78e0009040 comp rx=0 tx=0).stop 2024-09-13T08:50:24.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 -- 172.21.15.44:0/3259501079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78f80cba70 msgr2=0x7f78f80c36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:24.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78f80cba70 0x7f78f80c36c0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009790 tx=0x7f78e80040c0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 -- 172.21.15.44:0/3259501079 shutdown_connections 2024-09-13T08:50:24.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f78d4074960 0x7f78d4076e20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f8155080 0x7f78f8070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f78f80cda50 0x7f78f80c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 --2- 172.21.15.44:0/3259501079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78f80cba70 0x7f78f80c36c0 secure :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009790 tx=0x7f78e80040c0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 -- 172.21.15.44:0/3259501079 >> 172.21.15.44:0/3259501079 conn(0x7f78f806dd30 msgr2=0x7f78f815a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:24.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 -- 172.21.15.44:0/3259501079 shutdown_connections 2024-09-13T08:50:24.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.650+0000 7f78fd000640 1 -- 172.21.15.44:0/3259501079 wait complete. 2024-09-13T08:50:24.685 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:50:24.686 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:50:24.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.846+0000 7fecae800640 1 -- 172.21.15.44:0/1518911598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feca80b11a0 msgr2=0x7feca8165670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.846+0000 7fecae800640 1 --2- 172.21.15.44:0/1518911598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feca80b11a0 0x7feca8165670 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7feca0009bc0 tx=0x7feca001cac0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.846+0000 7fecae800640 1 -- 172.21.15.44:0/1518911598 shutdown_connections 2024-09-13T08:50:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.846+0000 7fecae800640 1 --2- 172.21.15.44:0/1518911598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feca80b11a0 0x7feca8165670 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.846+0000 7fecae800640 1 --2- 172.21.15.44:0/1518911598 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feca80b0ce0 0x7feca80b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.846+0000 7fecae800640 1 -- 172.21.15.44:0/1518911598 >> 172.21.15.44:0/1518911598 conn(0x7feca809aa10 msgr2=0x7feca809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.846+0000 7fecae800640 1 -- 172.21.15.44:0/1518911598 shutdown_connections 2024-09-13T08:50:24.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.846+0000 7fecae800640 1 -- 172.21.15.44:0/1518911598 wait complete. 2024-09-13T08:50:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.846+0000 7fecae800640 1 Processor -- start 2024-09-13T08:50:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecae800640 1 -- start start 2024-09-13T08:50:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecae800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca80b0ce0 0x7feca81f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecae800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feca80b11a0 0x7feca8147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecae800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feca80b4d80 0x7feca80b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecae800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feca8168970 con 0x7feca80b0ce0 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecae800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feca81687f0 con 0x7feca80b11a0 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecae800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7feca8168af0 con 0x7feca80b4d80 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca80b0ce0 0x7feca81f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca80b0ce0 0x7feca81f53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39094/0 (socket says 172.21.15.44:39094) 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecad400640 1 -- 172.21.15.44:0/1952866099 learned_addr learned my addr 172.21.15.44:0/1952866099 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecaca00640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feca80b11a0 0x7feca8147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecade00640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feca80b4d80 0x7feca80b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecad400640 1 -- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feca80b4d80 msgr2=0x7feca80b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecad400640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feca80b4d80 0x7feca80b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecad400640 1 -- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feca80b11a0 msgr2=0x7feca8147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:24.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecad400640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feca80b11a0 0x7feca8147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:24.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecad400640 1 -- 172.21.15.44:0/1952866099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca8148b00 con 0x7feca80b0ce0 2024-09-13T08:50:24.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecad400640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca80b0ce0 0x7feca81f53c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00c6e0 tx=0x7fec9c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:24.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7feca5600640 1 -- 172.21.15.44:0/1952866099 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c016020 con 0x7feca80b0ce0 2024-09-13T08:50:24.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7feca5600640 1 -- 172.21.15.44:0/1952866099 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec9c00a930 con 0x7feca80b0ce0 2024-09-13T08:50:24.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecae800640 1 -- 172.21.15.44:0/1952866099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca8148de0 con 0x7feca80b0ce0 2024-09-13T08:50:24.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7feca5600640 1 -- 172.21.15.44:0/1952866099 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c00ad80 con 0x7feca80b0ce0 2024-09-13T08:50:24.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecae800640 1 -- 172.21.15.44:0/1952866099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca81493a0 con 0x7feca80b0ce0 2024-09-13T08:50:24.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.850+0000 7fecae800640 1 -- 172.21.15.44:0/1952866099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca80b5160 con 0x7feca80b0ce0 2024-09-13T08:50:24.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.854+0000 7feca5600640 1 -- 172.21.15.44:0/1952866099 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fec9c019020 con 0x7feca80b0ce0 2024-09-13T08:50:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.854+0000 7feca5600640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec88074960 0x7fec88076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.854+0000 7feca5600640 1 -- 172.21.15.44:0/1952866099 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5907+0+0 (secure 0 0 0) 0x7fec9c092c70 con 0x7feca80b0ce0 2024-09-13T08:50:24.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.854+0000 7fecaca00640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec88074960 0x7fec88076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:24.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.854+0000 7fecaca00640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec88074960 0x7fec88076e20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7feca80b36a0 tx=0x7fec940091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:24.858+0000 7feca5600640 1 -- 172.21.15.44:0/1952866099 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fec9c020020 con 0x7feca80b0ce0 2024-09-13T08:50:25.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 -- 172.21.15.44:0/1952866099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feca80ce0c0 con 0x7feca80b0ce0 2024-09-13T08:50:25.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7feca5600640 1 -- 172.21.15.44:0/1952866099 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5225 (secure 0 0 0) 0x7fec9c05f2d0 con 0x7feca80b0ce0 2024-09-13T08:50:25.072 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:25.072 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":66,"btime":"2024-09-13T08:49:36:320618+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1987308515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1987308515},{"type":"v1","addr":"172.21.15.129:6837","nonce":1987308515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1224713128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1224713128},{"type":"v1","addr":"172.21.15.129:6836","nonce":1224713128}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7755,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/380853907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":380853907},{"type":"v1","addr":"172.21.15.44:6833","nonce":380853907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:49:27.697092+0000","modified":"2024-09-13T08:49:30.701664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/409896091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":409896091},{"type":"v1","addr":"172.21.15.44:6835","nonce":409896091}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7758,"qdb_cluster":[7758]},"id":7}]} 2024-09-13T08:50:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 66 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 -- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec88074960 msgr2=0x7fec88076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec88074960 0x7fec88076e20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7feca80b36a0 tx=0x7fec940091e0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 -- 172.21.15.44:0/1952866099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca80b0ce0 msgr2=0x7feca81f53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca80b0ce0 0x7feca81f53c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00c6e0 tx=0x7fec9c00cba0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 -- 172.21.15.44:0/1952866099 shutdown_connections 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec88074960 0x7fec88076e20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feca80b4d80 0x7feca80b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feca80b11a0 0x7feca8147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 --2- 172.21.15.44:0/1952866099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca80b0ce0 0x7feca81f53c0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 -- 172.21.15.44:0/1952866099 >> 172.21.15.44:0/1952866099 conn(0x7feca809aa10 msgr2=0x7feca815afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 -- 172.21.15.44:0/1952866099 shutdown_connections 2024-09-13T08:50:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.070+0000 7fecae800640 1 -- 172.21.15.44:0/1952866099 wait complete. 2024-09-13T08:50:25.095 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:50:25.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.238+0000 7fcac1c00640 1 -- 172.21.15.44:0/2196440114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcabc0cdb30 msgr2=0x7fcabc0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.238+0000 7fcac1c00640 1 --2- 172.21.15.44:0/2196440114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcabc0cdb30 0x7fcabc0cebb0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0745d0 tx=0x7fcaa401c770 comp rx=0 tx=0).stop 2024-09-13T08:50:25.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.238+0000 7fcac1c00640 1 -- 172.21.15.44:0/2196440114 shutdown_connections 2024-09-13T08:50:25.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.238+0000 7fcac1c00640 1 --2- 172.21.15.44:0/2196440114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcabc0cdb30 0x7fcabc0cebb0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.238+0000 7fcac1c00640 1 --2- 172.21.15.44:0/2196440114 >> v2:172.21.15.129:3300/0 conn(0x7fcabc15bea0 0x7fcabc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.238+0000 7fcac1c00640 1 -- 172.21.15.44:0/2196440114 >> 172.21.15.44:0/2196440114 conn(0x7fcabc0d1dc0 msgr2=0x7fcabc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:25.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.238+0000 7fcac1c00640 1 -- 172.21.15.44:0/2196440114 shutdown_connections 2024-09-13T08:50:25.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.238+0000 7fcac1c00640 1 -- 172.21.15.44:0/2196440114 wait complete. 2024-09-13T08:50:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 Processor -- start 2024-09-13T08:50:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 -- start start 2024-09-13T08:50:25.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcabc0cdb30 0x7fcabc14e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:25.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcabc0d28e0 0x7fcabc0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc15bea0 0x7fcabc165360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc168fc0 con 0x7fcabc0cdb30 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc168e40 con 0x7fcabc15bea0 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcabc169140 con 0x7fcabc0d28e0 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcabaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcabc0cdb30 0x7fcabc14e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcabaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcabc0cdb30 0x7fcabc14e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39108/0 (socket says 172.21.15.44:39108) 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcabaa00640 1 -- 172.21.15.44:0/1046755652 learned_addr learned my addr 172.21.15.44:0/1046755652 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcabb400640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc15bea0 0x7fcabc165360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcab3e00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcabc0d28e0 0x7fcabc0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcabaa00640 1 -- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcabc0d28e0 msgr2=0x7fcabc0c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcabaa00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcabc0d28e0 0x7fcabc0c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcabaa00640 1 -- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc15bea0 msgr2=0x7fcabc165360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcabaa00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc15bea0 0x7fcabc165360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcabaa00640 1 -- 172.21.15.44:0/1046755652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc0c7cf0 con 0x7fcabc0cdb30 2024-09-13T08:50:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcabaa00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcabc0cdb30 0x7fcabc14e180 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00d950 tx=0x7fcaac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcab2a00640 1 -- 172.21.15.44:0/1046755652 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac017070 con 0x7fcabc0cdb30 2024-09-13T08:50:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcab2a00640 1 -- 172.21.15.44:0/1046755652 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaac009d20 con 0x7fcabc0cdb30 2024-09-13T08:50:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 -- 172.21.15.44:0/1046755652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc0b5c40 con 0x7fcabc0cdb30 2024-09-13T08:50:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcab2a00640 1 -- 172.21.15.44:0/1046755652 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac0106c0 con 0x7fcabc0cdb30 2024-09-13T08:50:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 -- 172.21.15.44:0/1046755652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc0b6200 con 0x7fcabc0cdb30 2024-09-13T08:50:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.242+0000 7fcac1c00640 1 -- 172.21.15.44:0/1046755652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcabc0cc9d0 con 0x7fcabc0cdb30 2024-09-13T08:50:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.246+0000 7fcab2a00640 1 -- 172.21.15.44:0/1046755652 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fcaac01a070 con 0x7fcabc0cdb30 2024-09-13T08:50:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.246+0000 7fcab2a00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca98074930 0x7fca98076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.246+0000 7fcab2a00640 1 -- 172.21.15.44:0/1046755652 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5907+0+0 (secure 0 0 0) 0x7fcaac092dc0 con 0x7fcabc0cdb30 2024-09-13T08:50:25.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.246+0000 7fcab3e00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca98074930 0x7fca98076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:25.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.246+0000 7fcab3e00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca98074930 0x7fca98076df0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40af000 tx=0x7fcaa401afb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:25.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.250+0000 7fcab2a00640 1 -- 172.21.15.44:0/1046755652 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcaac05f420 con 0x7fcabc0cdb30 2024-09-13T08:50:25.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.462+0000 7fcac1c00640 1 -- 172.21.15.44:0/1046755652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcabc06f800 con 0x7fcabc0cdb30 2024-09-13T08:50:25.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.462+0000 7fcab2a00640 1 -- 172.21.15.44:0/1046755652 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5225 (secure 0 0 0) 0x7fcaac063e50 con 0x7fcabc0cdb30 2024-09-13T08:50:25.466 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:25.466 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":66,"btime":"2024-09-13T08:49:36:320618+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1987308515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1987308515},{"type":"v1","addr":"172.21.15.129:6837","nonce":1987308515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1224713128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1224713128},{"type":"v1","addr":"172.21.15.129:6836","nonce":1224713128}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7755,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/380853907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":380853907},{"type":"v1","addr":"172.21.15.44:6833","nonce":380853907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:49:27.697092+0000","modified":"2024-09-13T08:49:30.701664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/409896091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":409896091},{"type":"v1","addr":"172.21.15.44:6835","nonce":409896091}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7758,"qdb_cluster":[7758]},"id":7}]} 2024-09-13T08:50:25.466 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 66 2024-09-13T08:50:25.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 -- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca98074930 msgr2=0x7fca98076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca98074930 0x7fca98076df0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40af000 tx=0x7fcaa401afb0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 -- 172.21.15.44:0/1046755652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcabc0cdb30 msgr2=0x7fcabc14e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcabc0cdb30 0x7fcabc14e180 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00d950 tx=0x7fcaac00de10 comp rx=0 tx=0).stop 2024-09-13T08:50:25.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 -- 172.21.15.44:0/1046755652 shutdown_connections 2024-09-13T08:50:25.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca98074930 0x7fca98076df0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc15bea0 0x7fcabc165360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcabc0d28e0 0x7fcabc0c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 --2- 172.21.15.44:0/1046755652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcabc0cdb30 0x7fcabc14e180 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 -- 172.21.15.44:0/1046755652 >> 172.21.15.44:0/1046755652 conn(0x7fcabc0d1dc0 msgr2=0x7fcabc164190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:25.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 -- 172.21.15.44:0/1046755652 shutdown_connections 2024-09-13T08:50:25.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.466+0000 7fcac1c00640 1 -- 172.21.15.44:0/1046755652 wait complete. 2024-09-13T08:50:25.488 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:50:25.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3580282835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9e00ced80 msgr2=0x7ff9e00ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 --2- 172.21.15.44:0/3580282835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9e00ced80 0x7ff9e00ae9b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009840 tx=0x7ff9cc01c820 comp rx=0 tx=0).stop 2024-09-13T08:50:25.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3580282835 shutdown_connections 2024-09-13T08:50:25.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 --2- 172.21.15.44:0/3580282835 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9e00cdb30 0x7ff9e00aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 --2- 172.21.15.44:0/3580282835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9e00ced80 0x7ff9e00ae9b0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3580282835 >> 172.21.15.44:0/3580282835 conn(0x7ff9e00d1dc0 msgr2=0x7ff9e00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:25.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3580282835 shutdown_connections 2024-09-13T08:50:25.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3580282835 wait complete. 2024-09-13T08:50:25.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 Processor -- start 2024-09-13T08:50:25.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 -- start start 2024-09-13T08:50:25.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9e00cdb30 0x7ff9e0150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:25.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9e00ced80 0x7ff9e00c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:25.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9e015bea0 0x7ff9e00b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:25.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0169290 con 0x7ff9e00ced80 2024-09-13T08:50:25.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0169110 con 0x7ff9e015bea0 2024-09-13T08:50:25.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9e4e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e0169410 con 0x7ff9e00cdb30 2024-09-13T08:50:25.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9dd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9e00ced80 0x7ff9e00c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:25.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9dd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9e00ced80 0x7ff9e00c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39124/0 (socket says 172.21.15.44:39124) 2024-09-13T08:50:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9dd400640 1 -- 172.21.15.44:0/3769783323 learned_addr learned my addr 172.21.15.44:0/3769783323 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9dde00640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9e00cdb30 0x7ff9e0150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9de800640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9e015bea0 0x7ff9e00b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9dd400640 1 -- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9e00cdb30 msgr2=0x7ff9e0150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9dd400640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9e00cdb30 0x7ff9e0150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9dd400640 1 -- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9e015bea0 msgr2=0x7ff9e00b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9dd400640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9e015bea0 0x7ff9e00b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9dd400640 1 -- 172.21.15.44:0/3769783323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e00b3010 con 0x7ff9e00ced80 2024-09-13T08:50:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.634+0000 7ff9dd400640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9e00ced80 0x7ff9e00c8fb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc01cd00 tx=0x7ff9cc005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.638+0000 7ff9d6000640 1 -- 172.21.15.44:0/3769783323 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0ba020 con 0x7ff9e00ced80 2024-09-13T08:50:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.638+0000 7ff9d6000640 1 -- 172.21.15.44:0/3769783323 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9cc004330 con 0x7ff9e00ced80 2024-09-13T08:50:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.638+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3769783323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e00b4e40 con 0x7ff9e00ced80 2024-09-13T08:50:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.638+0000 7ff9d6000640 1 -- 172.21.15.44:0/3769783323 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0b2040 con 0x7ff9e00ced80 2024-09-13T08:50:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.638+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3769783323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e00b5400 con 0x7ff9e00ced80 2024-09-13T08:50:25.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.638+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3769783323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e00ce670 con 0x7ff9e00ced80 2024-09-13T08:50:25.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.638+0000 7ff9d6000640 1 -- 172.21.15.44:0/3769783323 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7ff9cc0bd070 con 0x7ff9e00ced80 2024-09-13T08:50:25.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.638+0000 7ff9d6000640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff9c0074930 0x7ff9c0076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:25.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.638+0000 7ff9d6000640 1 -- 172.21.15.44:0/3769783323 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5907+0+0 (secure 0 0 0) 0x7ff9cc12c070 con 0x7ff9e00ced80 2024-09-13T08:50:25.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.638+0000 7ff9dde00640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff9c0074930 0x7ff9c0076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:25.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.642+0000 7ff9dde00640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff9c0074930 0x7ff9c0076df0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000f70 tx=0x7ff9d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:25.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.642+0000 7ff9d6000640 1 -- 172.21.15.44:0/3769783323 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff9cc016410 con 0x7ff9e00ced80 2024-09-13T08:50:25.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.822+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3769783323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9e00725f0 con 0x7ff9e00ced80 2024-09-13T08:50:25.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.822+0000 7ff9d6000640 1 -- 172.21.15.44:0/3769783323 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+17033 (secure 0 0 0) 0x7ff9cc0f8d20 con 0x7ff9e00ced80 2024-09-13T08:50:25.826 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:25.827 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:50:24.685846+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:49:25.408514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-13T08:49:26.657487+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":5,"to":0}]},{"pgid":"14.1e","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:50:25.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff9c0074930 msgr2=0x7ff9c0076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff9c0074930 0x7ff9c0076df0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000f70 tx=0x7ff9d0009040 comp rx=0 tx=0).stop 2024-09-13T08:50:25.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3769783323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9e00ced80 msgr2=0x7ff9e00c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:25.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9e00ced80 0x7ff9e00c8fb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc01cd00 tx=0x7ff9cc005d20 comp rx=0 tx=0).stop 2024-09-13T08:50:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3769783323 shutdown_connections 2024-09-13T08:50:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff9c0074930 0x7ff9c0076df0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9e015bea0 0x7ff9e00b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9e00ced80 0x7ff9e00c8fb0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 --2- 172.21.15.44:0/3769783323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9e00cdb30 0x7ff9e0150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3769783323 >> 172.21.15.44:0/3769783323 conn(0x7ff9e00d1dc0 msgr2=0x7ff9e006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3769783323 shutdown_connections 2024-09-13T08:50:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.826+0000 7ff9e4e00640 1 -- 172.21.15.44:0/3769783323 wait complete. 2024-09-13T08:50:25.850 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T08:50:26.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.994+0000 7fe6ade00640 1 -- 172.21.15.44:0/115209222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe68c004d50 msgr2=0x7fe68c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:26.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.994+0000 7fe6ade00640 1 --2- 172.21.15.44:0/115209222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe68c004d50 0x7fe68c0051d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009e00 tx=0x7fe6a001d130 comp rx=0 tx=0).stop 2024-09-13T08:50:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.994+0000 7fe6a7400640 1 -- 172.21.15.44:0/115209222 reap_dead start 2024-09-13T08:50:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.994+0000 7fe6ade00640 1 -- 172.21.15.44:0/115209222 shutdown_connections 2024-09-13T08:50:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.994+0000 7fe6ade00640 1 -- 172.21.15.44:0/115209222 >> 172.21.15.44:0/115209222 conn(0x7fe6a80d1dc0 msgr2=0x7fe6a80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.994+0000 7fe6ade00640 1 -- 172.21.15.44:0/115209222 shutdown_connections 2024-09-13T08:50:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.994+0000 7fe6ade00640 1 -- 172.21.15.44:0/115209222 wait complete. 2024-09-13T08:50:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6ade00640 1 Processor -- start 2024-09-13T08:50:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6ade00640 1 -- start start 2024-09-13T08:50:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6ade00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe68c004d50 0x7fe6a80b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6ade00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6a81f89d0 0x7fe6a80b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6ade00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a81589e0 0x7fe6a81e3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6ade00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a80d2a60 con 0x7fe6a81589e0 2024-09-13T08:50:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6ade00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a80d28e0 con 0x7fe6a81f89d0 2024-09-13T08:50:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6ade00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a80d2be0 con 0x7fe68c004d50 2024-09-13T08:50:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6a7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a81589e0 0x7fe6a81e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6a7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a81589e0 0x7fe6a81e3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39138/0 (socket says 172.21.15.44:39138) 2024-09-13T08:50:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6a7400640 1 -- 172.21.15.44:0/753963982 learned_addr learned my addr 172.21.15.44:0/753963982 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6a6a00640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe68c004d50 0x7fe6a80b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe69fe00640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6a81f89d0 0x7fe6a80b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6a7400640 1 -- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe68c004d50 msgr2=0x7fe6a80b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6a7400640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe68c004d50 0x7fe6a80b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6a7400640 1 -- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6a81f89d0 msgr2=0x7fe6a80b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6a7400640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6a81f89d0 0x7fe6a80b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6a7400640 1 -- 172.21.15.44:0/753963982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a80b2800 con 0x7fe6a81589e0 2024-09-13T08:50:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6a7400640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a81589e0 0x7fe6a81e3510 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0002240 tx=0x7fe6a00adde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe69f400640 1 -- 172.21.15.44:0/753963982 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00b1040 con 0x7fe6a81589e0 2024-09-13T08:50:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe69f400640 1 -- 172.21.15.44:0/753963982 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6a00ade50 con 0x7fe6a81589e0 2024-09-13T08:50:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6ade00640 1 -- 172.21.15.44:0/753963982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a81f7ef0 con 0x7fe6a81589e0 2024-09-13T08:50:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe69f400640 1 -- 172.21.15.44:0/753963982 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00ac400 con 0x7fe6a81589e0 2024-09-13T08:50:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:25.998+0000 7fe6ade00640 1 -- 172.21.15.44:0/753963982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a80b2510 con 0x7fe6a81589e0 2024-09-13T08:50:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.002+0000 7fe6ade00640 1 -- 172.21.15.44:0/753963982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a80ae9b0 con 0x7fe6a81589e0 2024-09-13T08:50:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.002+0000 7fe69f400640 1 -- 172.21.15.44:0/753963982 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fe6a00be020 con 0x7fe6a81589e0 2024-09-13T08:50:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.002+0000 7fe69f400640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe684074990 0x7fe684076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.002+0000 7fe69f400640 1 -- 172.21.15.44:0/753963982 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5907+0+0 (secure 0 0 0) 0x7fe6a012d030 con 0x7fe6a81589e0 2024-09-13T08:50:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.002+0000 7fe6a6a00640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe684074990 0x7fe684076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.006+0000 7fe6a6a00640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe684074990 0x7fe684076e50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe698000fc0 tx=0x7fe698001ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:26.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.006+0000 7fe69f400640 1 -- 172.21.15.44:0/753963982 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe6a00c5020 con 0x7fe6a81589e0 2024-09-13T08:50:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.218+0000 7fe6ade00640 1 -- 172.21.15.44:0/753963982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe6a81e5200 con 0x7fe6a81589e0 2024-09-13T08:50:26.223 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T08:50:26.218+0000 7f03b3c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T08:50:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.222+0000 7fe69f400640 1 -- 172.21.15.44:0/753963982 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) ==== 161+0+0 (secure 0 0 0) 0x7fe6a00a5e20 con 0x7fe6a81589e0 2024-09-13T08:50:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 -- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe684074990 msgr2=0x7fe684076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe684074990 0x7fe684076e50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe698000fc0 tx=0x7fe698001ce0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 -- 172.21.15.44:0/753963982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a81589e0 msgr2=0x7fe6a81e3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a81589e0 0x7fe6a81e3510 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0002240 tx=0x7fe6a00adde0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 -- 172.21.15.44:0/753963982 shutdown_connections 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe684074990 0x7fe684076e50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6a81589e0 0x7fe6a81e3510 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6a81f89d0 0x7fe6a80b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 --2- 172.21.15.44:0/753963982 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe68c004d50 0x7fe6a80b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 -- 172.21.15.44:0/753963982 >> 172.21.15.44:0/753963982 conn(0x7fe6a80d1dc0 msgr2=0x7fe6a80b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 -- 172.21.15.44:0/753963982 shutdown_connections 2024-09-13T08:50:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.226+0000 7fe6ade00640 1 -- 172.21.15.44:0/753963982 wait complete. 2024-09-13T08:50:26.250 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T08:50:26.296 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:50:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.402+0000 7f0d6fe00640 1 -- 172.21.15.44:0/712472093 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d680d0990 msgr2=0x7f0d680d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.402+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/712472093 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d680d0990 0x7f0d680d0d70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009930 tx=0x7f0d5c01c820 comp rx=0 tx=0).stop 2024-09-13T08:50:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.402+0000 7f0d6fe00640 1 -- 172.21.15.44:0/712472093 shutdown_connections 2024-09-13T08:50:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.402+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/712472093 >> v2:172.21.15.129:3300/0 conn(0x7f0d680d12b0 0x7f0d68161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.402+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/712472093 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d680d0990 0x7f0d680d0d70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.402+0000 7f0d6fe00640 1 -- 172.21.15.44:0/712472093 >> 172.21.15.44:0/712472093 conn(0x7f0d680d3ff0 msgr2=0x7f0d680d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.402+0000 7f0d6fe00640 1 -- 172.21.15.44:0/712472093 shutdown_connections 2024-09-13T08:50:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.402+0000 7f0d6fe00640 1 -- 172.21.15.44:0/712472093 wait complete. 2024-09-13T08:50:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 Processor -- start 2024-09-13T08:50:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 -- start start 2024-09-13T08:50:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d680d0990 0x7f0d6815a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d680d12b0 0x7f0d680b7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d68162520 0x7f0d680b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6816b6a0 con 0x7f0d680d12b0 2024-09-13T08:50:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6816b520 con 0x7f0d680d0990 2024-09-13T08:50:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0d6816b820 con 0x7f0d68162520 2024-09-13T08:50:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d680d12b0 0x7f0d680b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d680d12b0 0x7f0d680b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39150/0 (socket says 172.21.15.44:39150) 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6ca00640 1 -- 172.21.15.44:0/1782801114 learned_addr learned my addr 172.21.15.44:0/1782801114 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6de00640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d68162520 0x7f0d680b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6d400640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d680d0990 0x7f0d6815a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6ca00640 1 -- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d68162520 msgr2=0x7f0d680b7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6ca00640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d68162520 0x7f0d680b7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6ca00640 1 -- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d680d0990 msgr2=0x7f0d6815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6ca00640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d680d0990 0x7f0d6815a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6ca00640 1 -- 172.21.15.44:0/1782801114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d68159630 con 0x7f0d680d12b0 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6d400640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d680d0990 0x7f0d6815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6ca00640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d680d12b0 0x7f0d680b7960 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400e9f0 tx=0x7f0d5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d65600640 1 -- 172.21.15.44:0/1782801114 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400cd80 con 0x7f0d680d12b0 2024-09-13T08:50:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d65600640 1 -- 172.21.15.44:0/1782801114 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d54010430 con 0x7f0d680d12b0 2024-09-13T08:50:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d65600640 1 -- 172.21.15.44:0/1782801114 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54010860 con 0x7f0d680d12b0 2024-09-13T08:50:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 -- 172.21.15.44:0/1782801114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d68159910 con 0x7f0d680d12b0 2024-09-13T08:50:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 -- 172.21.15.44:0/1782801114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d68159ed0 con 0x7f0d680d12b0 2024-09-13T08:50:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.406+0000 7f0d6fe00640 1 -- 172.21.15.44:0/1782801114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d680d0d70 con 0x7f0d680d12b0 2024-09-13T08:50:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.410+0000 7f0d65600640 1 -- 172.21.15.44:0/1782801114 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f0d54010a00 con 0x7f0d680d12b0 2024-09-13T08:50:26.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.410+0000 7f0d65600640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d48074930 0x7f0d48076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:26.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.410+0000 7f0d65600640 1 -- 172.21.15.44:0/1782801114 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5965+0+0 (secure 0 0 0) 0x7f0d540937d0 con 0x7f0d680d12b0 2024-09-13T08:50:26.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.410+0000 7f0d6d400640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d48074930 0x7f0d48076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:26.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.410+0000 7f0d6d400640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d48074930 0x7f0d48076df0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0af000 tx=0x7f0d5c01b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:26.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.414+0000 7f0d65600640 1 -- 172.21.15.44:0/1782801114 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0d5405fef0 con 0x7f0d680d12b0 2024-09-13T08:50:26.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:26.626+0000 7f0d6fe00640 1 -- 172.21.15.44:0/1782801114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0d68153af0 con 0x7f0d680d12b0 2024-09-13T08:50:27.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d65600640 1 -- 172.21.15.44:0/1782801114 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) ==== 104+0+0 (secure 0 0 0) 0x7f0d54064920 con 0x7f0d680d12b0 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 -- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d48074930 msgr2=0x7f0d48076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d48074930 0x7f0d48076df0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0af000 tx=0x7f0d5c01b060 comp rx=0 tx=0).stop 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 -- 172.21.15.44:0/1782801114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d680d12b0 msgr2=0x7f0d680b7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d680d12b0 0x7f0d680b7960 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400e9f0 tx=0x7f0d5400eeb0 comp rx=0 tx=0).stop 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 -- 172.21.15.44:0/1782801114 shutdown_connections 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0d48074930 0x7f0d48076df0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d68162520 0x7f0d680b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d680d12b0 0x7f0d680b7960 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/1782801114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d680d0990 0x7f0d6815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 -- 172.21.15.44:0/1782801114 >> 172.21.15.44:0/1782801114 conn(0x7f0d680d3ff0 msgr2=0x7f0d6806dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:27.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 -- 172.21.15.44:0/1782801114 shutdown_connections 2024-09-13T08:50:27.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.226+0000 7f0d6fe00640 1 -- 172.21.15.44:0/1782801114 wait complete. 2024-09-13T08:50:27.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T08:50:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 -- 172.21.15.44:0/3568533364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f622c0734f0 msgr2=0x7f622c15a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 --2- 172.21.15.44:0/3568533364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f622c0734f0 0x7f622c15a9d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f621c009840 tx=0x7f621c01c820 comp rx=0 tx=0).stop 2024-09-13T08:50:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 -- 172.21.15.44:0/3568533364 shutdown_connections 2024-09-13T08:50:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 --2- 172.21.15.44:0/3568533364 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f622c0cf440 0x7f622c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 --2- 172.21.15.44:0/3568533364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f622c0734f0 0x7f622c15a9d0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 -- 172.21.15.44:0/3568533364 >> 172.21.15.44:0/3568533364 conn(0x7f622c0cfbc0 msgr2=0x7f622c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 -- 172.21.15.44:0/3568533364 shutdown_connections 2024-09-13T08:50:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 -- 172.21.15.44:0/3568533364 wait complete. 2024-09-13T08:50:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 Processor -- start 2024-09-13T08:50:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 -- start start 2024-09-13T08:50:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f622c0734f0 0x7f622c139660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f622c0ce890 0x7f622c0b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f622c0cf440 0x7f622c0b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f622c167bb0 con 0x7f622c0734f0 2024-09-13T08:50:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f622c167a30 con 0x7f622c0cf440 2024-09-13T08:50:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f6232800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f622c167d30 con 0x7f622c0ce890 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f622b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f622c0734f0 0x7f622c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f622b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f622c0734f0 0x7f622c139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39168/0 (socket says 172.21.15.44:39168) 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.458+0000 7f622b400640 1 -- 172.21.15.44:0/3678408782 learned_addr learned my addr 172.21.15.44:0/3678408782 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f622be00640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f622c0cf440 0x7f622c0b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f6223e00640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f622c0ce890 0x7f622c0b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f622b400640 1 -- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f622c0ce890 msgr2=0x7f622c0b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f622b400640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f622c0ce890 0x7f622c0b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f622b400640 1 -- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f622c0cf440 msgr2=0x7f622c0b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f622b400640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f622c0cf440 0x7f622c0b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f622b400640 1 -- 172.21.15.44:0/3678408782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622c153be0 con 0x7f622c0734f0 2024-09-13T08:50:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f622b400640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f622c0734f0 0x7f622c139660 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f621c009790 tx=0x7f621c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:27.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f6223400640 1 -- 172.21.15.44:0/3678408782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c0bb070 con 0x7f622c0734f0 2024-09-13T08:50:27.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f6223400640 1 -- 172.21.15.44:0/3678408782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f621c005030 con 0x7f622c0734f0 2024-09-13T08:50:27.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f6232800640 1 -- 172.21.15.44:0/3678408782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622c153ec0 con 0x7f622c0734f0 2024-09-13T08:50:27.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f6223400640 1 -- 172.21.15.44:0/3678408782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c0b2040 con 0x7f622c0734f0 2024-09-13T08:50:27.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f6232800640 1 -- 172.21.15.44:0/3678408782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f622c154480 con 0x7f622c0734f0 2024-09-13T08:50:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f6232800640 1 -- 172.21.15.44:0/3678408782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f622c069410 con 0x7f622c0734f0 2024-09-13T08:50:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f6223400640 1 -- 172.21.15.44:0/3678408782 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f621c0be070 con 0x7f622c0734f0 2024-09-13T08:50:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.462+0000 7f6223400640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6208074960 0x7f6208076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.466+0000 7f6223400640 1 -- 172.21.15.44:0/3678408782 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5965+0+0 (secure 0 0 0) 0x7f621c12c5d0 con 0x7f622c0734f0 2024-09-13T08:50:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.466+0000 7f6223e00640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6208074960 0x7f6208076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:27.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.466+0000 7f6223e00640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6208074960 0x7f6208076e20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6214005fd0 tx=0x7f6214005d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:27.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.470+0000 7f6223400640 1 -- 172.21.15.44:0/3678408782 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f621c016410 con 0x7f622c0734f0 2024-09-13T08:50:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:27.646+0000 7f6232800640 1 -- 172.21.15.44:0/3678408782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f622c15da20 con 0x7f622c0734f0 2024-09-13T08:50:28.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6223400640 1 -- 172.21.15.44:0/3678408782 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) ==== 181+0+0 (secure 0 0 0) 0x7f621c0f9280 con 0x7f622c0734f0 2024-09-13T08:50:28.230 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T08:50:28.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 -- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6208074960 msgr2=0x7f6208076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:28.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6208074960 0x7f6208076e20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6214005fd0 tx=0x7f6214005d10 comp rx=0 tx=0).stop 2024-09-13T08:50:28.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 -- 172.21.15.44:0/3678408782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f622c0734f0 msgr2=0x7f622c139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:28.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f622c0734f0 0x7f622c139660 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f621c009790 tx=0x7f621c0040c0 comp rx=0 tx=0).stop 2024-09-13T08:50:28.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 -- 172.21.15.44:0/3678408782 shutdown_connections 2024-09-13T08:50:28.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6208074960 0x7f6208076e20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:28.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f622c0cf440 0x7f622c0b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:28.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f622c0ce890 0x7f622c0b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:28.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 --2- 172.21.15.44:0/3678408782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f622c0734f0 0x7f622c139660 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:28.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 -- 172.21.15.44:0/3678408782 >> 172.21.15.44:0/3678408782 conn(0x7f622c0cfbc0 msgr2=0x7f622c0cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:28.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 -- 172.21.15.44:0/3678408782 shutdown_connections 2024-09-13T08:50:28.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.230+0000 7f6232800640 1 -- 172.21.15.44:0/3678408782 wait complete. 2024-09-13T08:50:28.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T08:50:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.410+0000 7f0b71200640 1 -- 172.21.15.44:0/696277467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c15be10 msgr2=0x7f0b6c0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.410+0000 7f0b71200640 1 --2- 172.21.15.44:0/696277467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c15be10 0x7f0b6c0ce700 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009900 tx=0x7f0b5c01f0d0 comp rx=0 tx=0).stop 2024-09-13T08:50:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.410+0000 7f0b71200640 1 -- 172.21.15.44:0/696277467 shutdown_connections 2024-09-13T08:50:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.410+0000 7f0b71200640 1 --2- 172.21.15.44:0/696277467 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c15c1f0 0x7f0b6c0cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.410+0000 7f0b71200640 1 --2- 172.21.15.44:0/696277467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c15be10 0x7f0b6c0ce700 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.410+0000 7f0b71200640 1 -- 172.21.15.44:0/696277467 >> 172.21.15.44:0/696277467 conn(0x7f0b6c0d1da0 msgr2=0x7f0b6c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.410+0000 7f0b71200640 1 -- 172.21.15.44:0/696277467 shutdown_connections 2024-09-13T08:50:28.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.410+0000 7f0b71200640 1 -- 172.21.15.44:0/696277467 wait complete. 2024-09-13T08:50:28.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b71200640 1 Processor -- start 2024-09-13T08:50:28.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b71200640 1 -- start start 2024-09-13T08:50:28.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b71200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0b0b60 0x7f0b6c0c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:28.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b71200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c15be10 0x7f0b6c0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b71200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15c1f0 0x7f0b6c0b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b71200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c0cf510 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b71200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c0cf390 con 0x7f0b6c15c1f0 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b71200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c160290 con 0x7f0b6c15be10 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b69e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0b0b60 0x7f0b6c0c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b69e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0b0b60 0x7f0b6c0c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39188/0 (socket says 172.21.15.44:39188) 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b69e00640 1 -- 172.21.15.44:0/2649186070 learned_addr learned my addr 172.21.15.44:0/2649186070 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b63e00640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c15be10 0x7f0b6c0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b6a800640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15c1f0 0x7f0b6c0b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b69e00640 1 -- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c15be10 msgr2=0x7f0b6c0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:28.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b69e00640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c15be10 0x7f0b6c0cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b69e00640 1 -- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15c1f0 msgr2=0x7f0b6c0b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b69e00640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15c1f0 0x7f0b6c0b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b69e00640 1 -- 172.21.15.44:0/2649186070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c0c5a00 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b6a800640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15c1f0 0x7f0b6c0b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:50:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b69e00640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0b0b60 0x7f0b6c0c4c00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009850 tx=0x7f0b5c0a8d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b62000640 1 -- 172.21.15.44:0/2649186070 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0b5040 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b62000640 1 -- 172.21.15.44:0/2649186070 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b5c005320 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b62000640 1 -- 172.21.15.44:0/2649186070 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c003700 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b71200640 1 -- 172.21.15.44:0/2649186070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c0c5ce0 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.414+0000 7f0b71200640 1 -- 172.21.15.44:0/2649186070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c0cac50 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.418+0000 7f0b71200640 1 -- 172.21.15.44:0/2649186070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6c160780 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.418+0000 7f0b62000640 1 -- 172.21.15.44:0/2649186070 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f0b5c0038c0 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.418+0000 7f0b62000640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48074930 0x7f0b48076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:28.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.418+0000 7f0b62000640 1 -- 172.21.15.44:0/2649186070 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5469+0+0 (secure 0 0 0) 0x7f0b5c1304c0 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.418+0000 7f0b63e00640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48074930 0x7f0b48076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:28.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.418+0000 7f0b63e00640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48074930 0x7f0b48076df0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0b54005fd0 tx=0x7f0b54005d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:28.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.422+0000 7f0b62000640 1 -- 172.21.15.44:0/2649186070 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0b5c018c30 con 0x7f0b6c0b0b60 2024-09-13T08:50:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:28.598+0000 7f0b71200640 1 -- 172.21.15.44:0/2649186070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0b6c070350 con 0x7f0b6c0b0b60 2024-09-13T08:50:29.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.234+0000 7f0b62000640 1 -- 172.21.15.44:0/2649186070 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) ==== 169+0+0 (secure 0 0 0) 0x7f0b5c0fd170 con 0x7f0b6c0b0b60 2024-09-13T08:50:29.237 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 -- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48074930 msgr2=0x7f0b48076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48074930 0x7f0b48076df0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0b54005fd0 tx=0x7f0b54005d10 comp rx=0 tx=0).stop 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 -- 172.21.15.44:0/2649186070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0b0b60 msgr2=0x7f0b6c0c4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0b0b60 0x7f0b6c0c4c00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009850 tx=0x7f0b5c0a8d70 comp rx=0 tx=0).stop 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 -- 172.21.15.44:0/2649186070 shutdown_connections 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0b48074930 0x7f0b48076df0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b6c15c1f0 0x7f0b6c0b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b6c15be10 0x7f0b6c0cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 --2- 172.21.15.44:0/2649186070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c0b0b60 0x7f0b6c0c4c00 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 -- 172.21.15.44:0/2649186070 >> 172.21.15.44:0/2649186070 conn(0x7f0b6c0d1da0 msgr2=0x7f0b6c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 -- 172.21.15.44:0/2649186070 shutdown_connections 2024-09-13T08:50:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.238+0000 7f0b71200640 1 -- 172.21.15.44:0/2649186070 wait complete. 2024-09-13T08:50:29.264 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-13T08:50:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.410+0000 7fa993e00640 1 -- 172.21.15.44:0/1827666955 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa98c0b11d0 msgr2=0x7fa98c06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.410+0000 7fa993e00640 1 --2- 172.21.15.44:0/1827666955 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa98c0b11d0 0x7fa98c06ec80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa984009c80 tx=0x7fa98401cd10 comp rx=0 tx=0).stop 2024-09-13T08:50:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.410+0000 7fa993e00640 1 -- 172.21.15.44:0/1827666955 shutdown_connections 2024-09-13T08:50:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.410+0000 7fa993e00640 1 --2- 172.21.15.44:0/1827666955 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa98c0b11d0 0x7fa98c06ec80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.410+0000 7fa993e00640 1 --2- 172.21.15.44:0/1827666955 >> v2:172.21.15.129:3300/0 conn(0x7fa98c15df50 0x7fa98c06e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.410+0000 7fa993e00640 1 -- 172.21.15.44:0/1827666955 >> 172.21.15.44:0/1827666955 conn(0x7fa98c0d1dc0 msgr2=0x7fa98c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.410+0000 7fa993e00640 1 -- 172.21.15.44:0/1827666955 shutdown_connections 2024-09-13T08:50:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.410+0000 7fa993e00640 1 -- 172.21.15.44:0/1827666955 wait complete. 2024-09-13T08:50:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 Processor -- start 2024-09-13T08:50:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 -- start start 2024-09-13T08:50:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa98c0b11d0 0x7fa98c14aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa98c15df50 0x7fa98c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa98c15e330 0x7fa98c0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c0d0830 con 0x7fa98c15e330 2024-09-13T08:50:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c0d06b0 con 0x7fa98c15df50 2024-09-13T08:50:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa98c0d09b0 con 0x7fa98c0b11d0 2024-09-13T08:50:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa991e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa98c15e330 0x7fa98c0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa991e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa98c15e330 0x7fa98c0b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39200/0 (socket says 172.21.15.44:39200) 2024-09-13T08:50:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa991e00640 1 -- 172.21.15.44:0/2381029026 learned_addr learned my addr 172.21.15.44:0/2381029026 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa991400640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa98c0b11d0 0x7fa98c14aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa990a00640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa98c15df50 0x7fa98c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa991e00640 1 -- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa98c0b11d0 msgr2=0x7fa98c14aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa991e00640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa98c0b11d0 0x7fa98c14aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa991e00640 1 -- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa98c15df50 msgr2=0x7fa98c157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa991e00640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa98c15df50 0x7fa98c157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa991e00640 1 -- 172.21.15.44:0/2381029026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c0ce700 con 0x7fa98c15e330 2024-09-13T08:50:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa991e00640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa98c15e330 0x7fa98c0b2760 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa98401c780 tx=0x7fa984005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa989600640 1 -- 172.21.15.44:0/2381029026 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9840b2040 con 0x7fa98c15e330 2024-09-13T08:50:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa989600640 1 -- 172.21.15.44:0/2381029026 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa984004040 con 0x7fa98c15e330 2024-09-13T08:50:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa989600640 1 -- 172.21.15.44:0/2381029026 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9840043f0 con 0x7fa98c15e330 2024-09-13T08:50:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 -- 172.21.15.44:0/2381029026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c0ce9e0 con 0x7fa98c15e330 2024-09-13T08:50:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 -- 172.21.15.44:0/2381029026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c14b3f0 con 0x7fa98c15e330 2024-09-13T08:50:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.414+0000 7fa993e00640 1 -- 172.21.15.44:0/2381029026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98c0b5600 con 0x7fa98c15e330 2024-09-13T08:50:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.418+0000 7fa989600640 1 -- 172.21.15.44:0/2381029026 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fa9840bd030 con 0x7fa98c15e330 2024-09-13T08:50:29.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.418+0000 7fa989600640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa96c074930 0x7fa96c076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:29.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.418+0000 7fa989600640 1 -- 172.21.15.44:0/2381029026 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7fa98412cbb0 con 0x7fa98c15e330 2024-09-13T08:50:29.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.418+0000 7fa991400640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa96c074930 0x7fa96c076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:29.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.418+0000 7fa991400640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa96c074930 0x7fa96c076df0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa980009600 tx=0x7fa9800091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:29.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.422+0000 7fa989600640 1 -- 172.21.15.44:0/2381029026 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa9840c4060 con 0x7fa98c15e330 2024-09-13T08:50:29.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.602+0000 7fa993e00640 1 -- 172.21.15.44:0/2381029026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fa98c0afcd0 con 0x7fa98c15e330 2024-09-13T08:50:29.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa989600640 1 -- 172.21.15.44:0/2381029026 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7fa984016990 con 0x7fa98c15e330 2024-09-13T08:50:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 -- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa96c074930 msgr2=0x7fa96c076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa96c074930 0x7fa96c076df0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa980009600 tx=0x7fa9800091e0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 -- 172.21.15.44:0/2381029026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa98c15e330 msgr2=0x7fa98c0b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa98c15e330 0x7fa98c0b2760 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa98401c780 tx=0x7fa984005a60 comp rx=0 tx=0).stop 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 -- 172.21.15.44:0/2381029026 shutdown_connections 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa96c074930 0x7fa96c076df0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa98c15e330 0x7fa98c0b2760 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa98c15df50 0x7fa98c157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 --2- 172.21.15.44:0/2381029026 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa98c0b11d0 0x7fa98c14aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 -- 172.21.15.44:0/2381029026 >> 172.21.15.44:0/2381029026 conn(0x7fa98c0d1dc0 msgr2=0x7fa98c06e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 -- 172.21.15.44:0/2381029026 shutdown_connections 2024-09-13T08:50:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.606+0000 7fa993e00640 1 -- 172.21.15.44:0/2381029026 wait complete. 2024-09-13T08:50:29.631 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-13T08:50:29.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.774+0000 7f257a600640 1 -- 172.21.15.44:0/616654810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25740b3dc0 msgr2=0x7f25740b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:29.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.774+0000 7f257a600640 1 --2- 172.21.15.44:0/616654810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25740b3dc0 0x7f25740b46a0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f25740b0760 tx=0x7f255c01c600 comp rx=0 tx=0).stop 2024-09-13T08:50:29.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.774+0000 7f257a600640 1 -- 172.21.15.44:0/616654810 shutdown_connections 2024-09-13T08:50:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.774+0000 7f257a600640 1 --2- 172.21.15.44:0/616654810 >> v2:172.21.15.129:3300/0 conn(0x7f25740b4be0 0x7f25740738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.774+0000 7f257a600640 1 --2- 172.21.15.44:0/616654810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25740b3dc0 0x7f25740b46a0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.774+0000 7f257a600640 1 --2- 172.21.15.44:0/616654810 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f257415a980 0x7f25740d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.774+0000 7f257a600640 1 -- 172.21.15.44:0/616654810 >> 172.21.15.44:0/616654810 conn(0x7f25740b2890 msgr2=0x7f25740b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 -- 172.21.15.44:0/616654810 shutdown_connections 2024-09-13T08:50:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 -- 172.21.15.44:0/616654810 wait complete. 2024-09-13T08:50:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 Processor -- start 2024-09-13T08:50:29.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 -- start start 2024-09-13T08:50:29.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25740b3dc0 0x7f257415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:29.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25740b4be0 0x7f2574156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:29.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f257415a980 0x7f2574157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:29.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2574070870 con 0x7f25740b4be0 2024-09-13T08:50:29.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25740706f0 con 0x7f257415a980 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f25740709f0 con 0x7f25740b3dc0 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25740b4be0 0x7f2574156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25740b4be0 0x7f2574156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39222/0 (socket says 172.21.15.44:39222) 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256aa00640 1 -- 172.21.15.44:0/4073325337 learned_addr learned my addr 172.21.15.44:0/4073325337 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f2573400640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25740b3dc0 0x7f257415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f2573e00640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f257415a980 0x7f2574157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256aa00640 1 -- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25740b3dc0 msgr2=0x7f257415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256aa00640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25740b3dc0 0x7f257415f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256aa00640 1 -- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f257415a980 msgr2=0x7f2574157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256aa00640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f257415a980 0x7f2574157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256aa00640 1 -- 172.21.15.44:0/4073325337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2574150a60 con 0x7f25740b4be0 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f2573e00640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f257415a980 0x7f2574157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256aa00640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25740b4be0 0x7f2574156260 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f255c01acb0 tx=0x7f255c0a9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:29.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256be00640 1 -- 172.21.15.44:0/4073325337 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c0ad040 con 0x7f25740b4be0 2024-09-13T08:50:29.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256be00640 1 -- 172.21.15.44:0/4073325337 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f255c0aad60 con 0x7f25740b4be0 2024-09-13T08:50:29.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f256be00640 1 -- 172.21.15.44:0/4073325337 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c005290 con 0x7f25740b4be0 2024-09-13T08:50:29.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.778+0000 7f257a600640 1 -- 172.21.15.44:0/4073325337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2574150d40 con 0x7f25740b4be0 2024-09-13T08:50:29.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.782+0000 7f257a600640 1 -- 172.21.15.44:0/4073325337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2574151300 con 0x7f25740b4be0 2024-09-13T08:50:29.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.782+0000 7f257a600640 1 -- 172.21.15.44:0/4073325337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25741507d0 con 0x7f25740b4be0 2024-09-13T08:50:29.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.782+0000 7f256be00640 1 -- 172.21.15.44:0/4073325337 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f255c0b0070 con 0x7f25740b4be0 2024-09-13T08:50:29.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.782+0000 7f256be00640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2550074930 0x7f2550076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:29.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.782+0000 7f256be00640 1 -- 172.21.15.44:0/4073325337 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f255c12d2e0 con 0x7f25740b4be0 2024-09-13T08:50:29.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.782+0000 7f2573400640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2550074930 0x7f2550076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:29.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.786+0000 7f2573400640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2550074930 0x7f2550076df0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f25640045b0 tx=0x7f256400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:29.786+0000 7f256be00640 1 -- 172.21.15.44:0/4073325337 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f255c0c3030 con 0x7f25740b4be0 2024-09-13T08:50:30.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.026+0000 7f257a600640 1 -- 172.21.15.44:0/4073325337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7f25740738f0 con 0x7f25740b4be0 2024-09-13T08:50:30.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f256be00640 1 -- 172.21.15.44:0/4073325337 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v582) ==== 178+0+0 (secure 0 0 0) 0x7f255c0164b0 con 0x7f25740b4be0 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 -- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2550074930 msgr2=0x7f2550076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2550074930 0x7f2550076df0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f25640045b0 tx=0x7f256400b2e0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 -- 172.21.15.44:0/4073325337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25740b4be0 msgr2=0x7f2574156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25740b4be0 0x7f2574156260 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f255c01acb0 tx=0x7f255c0a9500 comp rx=0 tx=0).stop 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 -- 172.21.15.44:0/4073325337 shutdown_connections 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2550074930 0x7f2550076df0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f257415a980 0x7f2574157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25740b4be0 0x7f2574156260 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 --2- 172.21.15.44:0/4073325337 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f25740b3dc0 0x7f257415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 -- 172.21.15.44:0/4073325337 >> 172.21.15.44:0/4073325337 conn(0x7f25740b2890 msgr2=0x7f25741596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:30.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 -- 172.21.15.44:0/4073325337 shutdown_connections 2024-09-13T08:50:30.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.238+0000 7f257a600640 1 -- 172.21.15.44:0/4073325337 wait complete. 2024-09-13T08:50:30.262 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-09-13T08:50:30.263 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress bars, that printed for 3 ongoing clone jobs, are 2024-09-13T08:50:30.263 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-13T08:50:30.263 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:50:30.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.406+0000 7f399ee00640 1 -- 172.21.15.44:0/1872710844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39980cdb30 msgr2=0x7f39980cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.406+0000 7f399ee00640 1 --2- 172.21.15.44:0/1872710844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39980cdb30 0x7f39980cebb0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f3980009900 tx=0x7f398001c8e0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.406+0000 7f399ee00640 1 -- 172.21.15.44:0/1872710844 shutdown_connections 2024-09-13T08:50:30.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.406+0000 7f399ee00640 1 --2- 172.21.15.44:0/1872710844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39980cdb30 0x7f39980cebb0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.406+0000 7f399ee00640 1 --2- 172.21.15.44:0/1872710844 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f399815bea0 0x7f39980ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.406+0000 7f399ee00640 1 -- 172.21.15.44:0/1872710844 >> 172.21.15.44:0/1872710844 conn(0x7f39980d1dc0 msgr2=0x7f39980d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:30.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.406+0000 7f399ee00640 1 -- 172.21.15.44:0/1872710844 shutdown_connections 2024-09-13T08:50:30.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.406+0000 7f399ee00640 1 -- 172.21.15.44:0/1872710844 wait complete. 2024-09-13T08:50:30.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.406+0000 7f399ee00640 1 Processor -- start 2024-09-13T08:50:30.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.406+0000 7f399ee00640 1 -- start start 2024-09-13T08:50:30.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39980cdb30 0x7f3998153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:30.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ee00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39980d28e0 0x7f3998151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:30.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f399815bea0 0x7f39980b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:30.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3998169050 con 0x7f399815bea0 2024-09-13T08:50:30.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3998168ed0 con 0x7f39980cdb30 2024-09-13T08:50:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ee00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f39981691d0 con 0x7f39980d28e0 2024-09-13T08:50:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f399815bea0 0x7f39980b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f399815bea0 0x7f39980b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39252/0 (socket says 172.21.15.44:39252) 2024-09-13T08:50:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ce00640 1 -- 172.21.15.44:0/793528182 learned_addr learned my addr 172.21.15.44:0/793528182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f3997e00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39980cdb30 0x7f3998153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f398fe00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39980d28e0 0x7f3998151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ce00640 1 -- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39980d28e0 msgr2=0x7f3998151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ce00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39980d28e0 0x7f3998151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ce00640 1 -- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39980cdb30 msgr2=0x7f3998153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ce00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39980cdb30 0x7f3998153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ce00640 1 -- 172.21.15.44:0/793528182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3998151cd0 con 0x7f399815bea0 2024-09-13T08:50:30.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ce00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f399815bea0 0x7f39980b3040 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f399000c7b0 tx=0x7f399000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:30.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f3994c00640 1 -- 172.21.15.44:0/793528182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990017070 con 0x7f399815bea0 2024-09-13T08:50:30.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f3994c00640 1 -- 172.21.15.44:0/793528182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3990005070 con 0x7f399815bea0 2024-09-13T08:50:30.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f3994c00640 1 -- 172.21.15.44:0/793528182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39900054c0 con 0x7f399815bea0 2024-09-13T08:50:30.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ee00640 1 -- 172.21.15.44:0/793528182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3998152b70 con 0x7f399815bea0 2024-09-13T08:50:30.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ee00640 1 -- 172.21.15.44:0/793528182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3998153130 con 0x7f399815bea0 2024-09-13T08:50:30.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.410+0000 7f399ee00640 1 -- 172.21.15.44:0/793528182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39980cc9d0 con 0x7f399815bea0 2024-09-13T08:50:30.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.414+0000 7f3994c00640 1 -- 172.21.15.44:0/793528182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f399001a070 con 0x7f399815bea0 2024-09-13T08:50:30.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.414+0000 7f3994c00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3974074930 0x7f3974076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:30.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.414+0000 7f3994c00640 1 -- 172.21.15.44:0/793528182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f3990092280 con 0x7f399815bea0 2024-09-13T08:50:30.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.414+0000 7f3997e00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3974074930 0x7f3974076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:30.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.414+0000 7f3997e00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3974074930 0x7f3974076df0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f39980b2730 tx=0x7f3988006f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:30.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.422+0000 7f3994c00640 1 -- 172.21.15.44:0/793528182 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f399005e980 con 0x7f399815bea0 2024-09-13T08:50:30.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.602+0000 7f399ee00640 1 -- 172.21.15.44:0/793528182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3998159820 con 0x7f399815bea0 2024-09-13T08:50:30.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.602+0000 7f3994c00640 1 -- 172.21.15.44:0/793528182 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+13229 (secure 0 0 0) 0x7f39900633b0 con 0x7f399815bea0 2024-09-13T08:50:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:30.602 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":108,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:50:29.233309+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/409896091":"2024-09-14T08:50:26.226239+0000","172.21.15.44:6834/409896091":"2024-09-14T08:50:26.226239+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:50:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 -- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3974074930 msgr2=0x7f3974076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3974074930 0x7f3974076df0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f39980b2730 tx=0x7f3988006f90 comp rx=0 tx=0).stop 2024-09-13T08:50:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 -- 172.21.15.44:0/793528182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f399815bea0 msgr2=0x7f39980b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f399815bea0 0x7f39980b3040 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f399000c7b0 tx=0x7f399000cc70 comp rx=0 tx=0).stop 2024-09-13T08:50:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 -- 172.21.15.44:0/793528182 shutdown_connections 2024-09-13T08:50:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3974074930 0x7f3974076df0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f399815bea0 0x7f39980b3040 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39980d28e0 0x7f3998151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 --2- 172.21.15.44:0/793528182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39980cdb30 0x7f3998153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 -- 172.21.15.44:0/793528182 >> 172.21.15.44:0/793528182 conn(0x7f39980d1dc0 msgr2=0x7f39980b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 -- 172.21.15.44:0/793528182 shutdown_connections 2024-09-13T08:50:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.606+0000 7f399ee00640 1 -- 172.21.15.44:0/793528182 wait complete. 2024-09-13T08:50:30.624 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T08:37:24.563344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T08:50:30.625 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T08:50:30.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.774+0000 7f2b4e400640 1 -- 172.21.15.44:0/3346768323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b480b1650 msgr2=0x7f2b480b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.774+0000 7f2b4e400640 1 --2- 172.21.15.44:0/3346768323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b480b1650 0x7f2b480b1a30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009900 tx=0x7f2b3001c8e0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.774+0000 7f2b4e400640 1 -- 172.21.15.44:0/3346768323 shutdown_connections 2024-09-13T08:50:30.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.774+0000 7f2b4e400640 1 --2- 172.21.15.44:0/3346768323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b480b1650 0x7f2b480b1a30 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.774+0000 7f2b4e400640 1 --2- 172.21.15.44:0/3346768323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b481605d0 0x7f2b480d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.774+0000 7f2b4e400640 1 -- 172.21.15.44:0/3346768323 >> 172.21.15.44:0/3346768323 conn(0x7f2b480d1dd0 msgr2=0x7f2b480d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:30.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.774+0000 7f2b4e400640 1 -- 172.21.15.44:0/3346768323 shutdown_connections 2024-09-13T08:50:30.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.774+0000 7f2b4e400640 1 -- 172.21.15.44:0/3346768323 wait complete. 2024-09-13T08:50:30.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b4e400640 1 Processor -- start 2024-09-13T08:50:30.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b4e400640 1 -- start start 2024-09-13T08:50:30.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b4e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b480aeb00 0x7f2b481570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:30.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b4e400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b480b1650 0x7f2b481f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:30.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b4e400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b481605d0 0x7f2b4806f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:30.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b4e400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b480b1220 con 0x7f2b480aeb00 2024-09-13T08:50:30.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b4e400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b480b0ee0 con 0x7f2b481605d0 2024-09-13T08:50:30.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b4e400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2b480b13a0 con 0x7f2b480b1650 2024-09-13T08:50:30.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b47400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b480aeb00 0x7f2b481570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:30.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b47400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b480aeb00 0x7f2b481570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39272/0 (socket says 172.21.15.44:39272) 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b47400640 1 -- 172.21.15.44:0/1553293784 learned_addr learned my addr 172.21.15.44:0/1553293784 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b3ea00640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b480b1650 0x7f2b481f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b47e00640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b481605d0 0x7f2b4806f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b47400640 1 -- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b480b1650 msgr2=0x7f2b481f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b47400640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b480b1650 0x7f2b481f8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b47400640 1 -- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b481605d0 msgr2=0x7f2b4806f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b47400640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b481605d0 0x7f2b4806f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b47400640 1 -- 172.21.15.44:0/1553293784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4815ec10 con 0x7f2b480aeb00 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b47400640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b480aeb00 0x7f2b481570c0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c940 tx=0x7f2b3800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b3fe00640 1 -- 172.21.15.44:0/1553293784 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38004220 con 0x7f2b480aeb00 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b3fe00640 1 -- 172.21.15.44:0/1553293784 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b380079b0 con 0x7f2b480aeb00 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b4e400640 1 -- 172.21.15.44:0/1553293784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4815eeb0 con 0x7f2b480aeb00 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b3fe00640 1 -- 172.21.15.44:0/1553293784 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380044b0 con 0x7f2b480aeb00 2024-09-13T08:50:30.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.778+0000 7f2b4e400640 1 -- 172.21.15.44:0/1553293784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4815f430 con 0x7f2b480aeb00 2024-09-13T08:50:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.782+0000 7f2b4e400640 1 -- 172.21.15.44:0/1553293784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b480b1a30 con 0x7f2b480aeb00 2024-09-13T08:50:30.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.782+0000 7f2b3fe00640 1 -- 172.21.15.44:0/1553293784 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f2b38003600 con 0x7f2b480aeb00 2024-09-13T08:50:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.782+0000 7f2b3fe00640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b24074930 0x7f2b24076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.782+0000 7f2b3fe00640 1 -- 172.21.15.44:0/1553293784 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f2b380931f0 con 0x7f2b480aeb00 2024-09-13T08:50:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.782+0000 7f2b3ea00640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b24074930 0x7f2b24076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:30.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.782+0000 7f2b3ea00640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b24074930 0x7f2b24076df0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2b300af000 tx=0x7f2b3001b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:30.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.786+0000 7f2b3fe00640 1 -- 172.21.15.44:0/1553293784 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2b38014030 con 0x7f2b480aeb00 2024-09-13T08:50:30.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.962+0000 7f2b4e400640 1 -- 172.21.15.44:0/1553293784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b480ce6e0 con 0x7f2b480aeb00 2024-09-13T08:50:30.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.962+0000 7f2b3fe00640 1 -- 172.21.15.44:0/1553293784 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7f2b3805fea0 con 0x7f2b480aeb00 2024-09-13T08:50:30.962 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 -- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b24074930 msgr2=0x7f2b24076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b24074930 0x7f2b24076df0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2b300af000 tx=0x7f2b3001b120 comp rx=0 tx=0).stop 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 -- 172.21.15.44:0/1553293784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b480aeb00 msgr2=0x7f2b481570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b480aeb00 0x7f2b481570c0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c940 tx=0x7f2b3800ce00 comp rx=0 tx=0).stop 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 -- 172.21.15.44:0/1553293784 shutdown_connections 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b24074930 0x7f2b24076df0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b481605d0 0x7f2b4806f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b480b1650 0x7f2b481f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 --2- 172.21.15.44:0/1553293784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b480aeb00 0x7f2b481570c0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 -- 172.21.15.44:0/1553293784 >> 172.21.15.44:0/1553293784 conn(0x7f2b480d1dd0 msgr2=0x7f2b480cf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:30.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 -- 172.21.15.44:0/1553293784 shutdown_connections 2024-09-13T08:50:30.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:30.966+0000 7f2b4e400640 1 -- 172.21.15.44:0/1553293784 wait complete. 2024-09-13T08:50:30.983 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-13T08:50:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.130+0000 7f53ece00640 1 -- 172.21.15.44:0/1919724721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e80cd100 msgr2=0x7f53e80acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.130+0000 7f53ece00640 1 --2- 172.21.15.44:0/1919724721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e80cd100 0x7f53e80acd40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009900 tx=0x7f53d401c820 comp rx=0 tx=0).stop 2024-09-13T08:50:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.130+0000 7f53ece00640 1 -- 172.21.15.44:0/1919724721 shutdown_connections 2024-09-13T08:50:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.130+0000 7f53ece00640 1 --2- 172.21.15.44:0/1919724721 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e815db50 0x7f53e80b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.130+0000 7f53ece00640 1 --2- 172.21.15.44:0/1919724721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e80cd100 0x7f53e80acd40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.130+0000 7f53ece00640 1 -- 172.21.15.44:0/1919724721 >> 172.21.15.44:0/1919724721 conn(0x7f53e80b18f0 msgr2=0x7f53e80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.130+0000 7f53ece00640 1 -- 172.21.15.44:0/1919724721 shutdown_connections 2024-09-13T08:50:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.130+0000 7f53ece00640 1 -- 172.21.15.44:0/1919724721 wait complete. 2024-09-13T08:50:31.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 Processor -- start 2024-09-13T08:50:31.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 -- start start 2024-09-13T08:50:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e806f8b0 0x7f53e814ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e80cd100 0x7f53e814b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e815db50 0x7f53e814cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f53e80cf4d0 con 0x7f53e80cd100 2024-09-13T08:50:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f53e80cf350 con 0x7f53e806f8b0 2024-09-13T08:50:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f53e8168130 con 0x7f53e815db50 2024-09-13T08:50:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e80cd100 0x7f53e814b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e80cd100 0x7f53e814b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39276/0 (socket says 172.21.15.44:39276) 2024-09-13T08:50:31.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e5400640 1 -- 172.21.15.44:0/3776351630 learned_addr learned my addr 172.21.15.44:0/3776351630 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:31.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e5e00640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e806f8b0 0x7f53e814ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:31.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e6800640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e815db50 0x7f53e814cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:31.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e5400640 1 -- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e815db50 msgr2=0x7f53e814cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:31.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e5400640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e815db50 0x7f53e814cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:31.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e5400640 1 -- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e806f8b0 msgr2=0x7f53e814ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:31.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e5400640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e806f8b0 0x7f53e814ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:31.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e5400640 1 -- 172.21.15.44:0/3776351630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e814f1d0 con 0x7f53e80cd100 2024-09-13T08:50:31.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53e5400640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e80cd100 0x7f53e814b5f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f53d40b0040 tx=0x7f53d40a5810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53de000640 1 -- 172.21.15.44:0/3776351630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d40b2040 con 0x7f53e80cd100 2024-09-13T08:50:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53de000640 1 -- 172.21.15.44:0/3776351630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53d4004480 con 0x7f53e80cd100 2024-09-13T08:50:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 -- 172.21.15.44:0/3776351630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e814d9e0 con 0x7f53e80cd100 2024-09-13T08:50:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53de000640 1 -- 172.21.15.44:0/3776351630 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d40052b0 con 0x7f53e80cd100 2024-09-13T08:50:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 -- 172.21.15.44:0/3776351630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e814e490 con 0x7f53e80cd100 2024-09-13T08:50:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.134+0000 7f53ece00640 1 -- 172.21.15.44:0/3776351630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e8069410 con 0x7f53e80cd100 2024-09-13T08:50:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.138+0000 7f53de000640 1 -- 172.21.15.44:0/3776351630 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f53d4005450 con 0x7f53e80cd100 2024-09-13T08:50:31.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.138+0000 7f53de000640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c8074930 0x7f53c8076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:31.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.138+0000 7f53de000640 1 -- 172.21.15.44:0/3776351630 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f53d412c380 con 0x7f53e80cd100 2024-09-13T08:50:31.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.138+0000 7f53e5e00640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c8074930 0x7f53c8076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:31.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.138+0000 7f53e5e00640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c8074930 0x7f53c8076df0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f53d8007fd0 tx=0x7f53d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.142+0000 7f53de000640 1 -- 172.21.15.44:0/3776351630 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f53d40164d0 con 0x7f53e80cd100 2024-09-13T08:50:31.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:31.382+0000 7f53ece00640 1 -- 172.21.15.44:0/3776351630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7f53e80acd40 con 0x7f53e80cd100 2024-09-13T08:50:32.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.250+0000 7f53de000640 1 -- 172.21.15.44:0/3776351630 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v584) ==== 188+0+0 (secure 0 0 0) 0x7f53d40f8960 con 0x7f53e80cd100 2024-09-13T08:50:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 -- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c8074930 msgr2=0x7f53c8076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c8074930 0x7f53c8076df0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f53d8007fd0 tx=0x7f53d8009040 comp rx=0 tx=0).stop 2024-09-13T08:50:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 -- 172.21.15.44:0/3776351630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e80cd100 msgr2=0x7f53e814b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e80cd100 0x7f53e814b5f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f53d40b0040 tx=0x7f53d40a5810 comp rx=0 tx=0).stop 2024-09-13T08:50:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 -- 172.21.15.44:0/3776351630 shutdown_connections 2024-09-13T08:50:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f53c8074930 0x7f53c8076df0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f53e815db50 0x7f53e814cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53e80cd100 0x7f53e814b5f0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 --2- 172.21.15.44:0/3776351630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53e806f8b0 0x7f53e814ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 -- 172.21.15.44:0/3776351630 >> 172.21.15.44:0/3776351630 conn(0x7f53e80b18f0 msgr2=0x7f53e8073b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 -- 172.21.15.44:0/3776351630 shutdown_connections 2024-09-13T08:50:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.254+0000 7f53ece00640 1 -- 172.21.15.44:0/3776351630 wait complete. 2024-09-13T08:50:32.276 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-13T08:50:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.430+0000 7f5acfe00640 1 -- 172.21.15.44:0/2397622565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5ac80cdb30 msgr2=0x7f5ac80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.430+0000 7f5acfe00640 1 --2- 172.21.15.44:0/2397622565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5ac80cdb30 0x7f5ac80cebb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009840 tx=0x7f5ab401c820 comp rx=0 tx=0).stop 2024-09-13T08:50:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.430+0000 7f5acfe00640 1 -- 172.21.15.44:0/2397622565 shutdown_connections 2024-09-13T08:50:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.430+0000 7f5acfe00640 1 --2- 172.21.15.44:0/2397622565 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5ac80cdb30 0x7f5ac80cebb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.430+0000 7f5acfe00640 1 --2- 172.21.15.44:0/2397622565 >> v2:172.21.15.129:3300/0 conn(0x7f5ac815bea0 0x7f5ac80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.430+0000 7f5acfe00640 1 -- 172.21.15.44:0/2397622565 >> 172.21.15.44:0/2397622565 conn(0x7f5ac80d1dc0 msgr2=0x7f5ac80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.430+0000 7f5acfe00640 1 -- 172.21.15.44:0/2397622565 shutdown_connections 2024-09-13T08:50:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.430+0000 7f5acfe00640 1 -- 172.21.15.44:0/2397622565 wait complete. 2024-09-13T08:50:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.430+0000 7f5acfe00640 1 Processor -- start 2024-09-13T08:50:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.430+0000 7f5acfe00640 1 -- start start 2024-09-13T08:50:32.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acfe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ac80cdb30 0x7f5ac80c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:32.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acfe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5ac80d28e0 0x7f5ac814f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:32.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac815bea0 0x7f5ac814f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:32.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8168fc0 con 0x7f5ac815bea0 2024-09-13T08:50:32.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acfe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8168e40 con 0x7f5ac80cdb30 2024-09-13T08:50:32.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acfe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac8169140 con 0x7f5ac80d28e0 2024-09-13T08:50:32.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac815bea0 0x7f5ac814f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:32.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac815bea0 0x7f5ac814f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41232/0 (socket says 172.21.15.44:41232) 2024-09-13T08:50:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acde00640 1 -- 172.21.15.44:0/3043517878 learned_addr learned my addr 172.21.15.44:0/3043517878 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acd400640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ac80cdb30 0x7f5ac80c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acca00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5ac80d28e0 0x7f5ac814f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acde00640 1 -- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5ac80d28e0 msgr2=0x7f5ac814f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acde00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5ac80d28e0 0x7f5ac814f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acde00640 1 -- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ac80cdb30 msgr2=0x7f5ac80c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acde00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ac80cdb30 0x7f5ac80c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acde00640 1 -- 172.21.15.44:0/3043517878 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac80b3e80 con 0x7f5ac815bea0 2024-09-13T08:50:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acca00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5ac80d28e0 0x7f5ac814f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acde00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac815bea0 0x7f5ac814f910 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000cbc0 tx=0x7f5ac000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5ac5600640 1 -- 172.21.15.44:0/3043517878 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0016020 con 0x7f5ac815bea0 2024-09-13T08:50:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5ac5600640 1 -- 172.21.15.44:0/3043517878 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ac000ab00 con 0x7f5ac815bea0 2024-09-13T08:50:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5ac5600640 1 -- 172.21.15.44:0/3043517878 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0012420 con 0x7f5ac815bea0 2024-09-13T08:50:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acfe00640 1 -- 172.21.15.44:0/3043517878 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac80b4160 con 0x7f5ac815bea0 2024-09-13T08:50:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acfe00640 1 -- 172.21.15.44:0/3043517878 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac80b1570 con 0x7f5ac815bea0 2024-09-13T08:50:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.434+0000 7f5acfe00640 1 -- 172.21.15.44:0/3043517878 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac80cc9d0 con 0x7f5ac815bea0 2024-09-13T08:50:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.438+0000 7f5ac5600640 1 -- 172.21.15.44:0/3043517878 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f5ac0019020 con 0x7f5ac815bea0 2024-09-13T08:50:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.438+0000 7f5ac5600640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8074930 0x7f5aa8076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.438+0000 7f5ac5600640 1 -- 172.21.15.44:0/3043517878 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f5ac0092fc0 con 0x7f5ac815bea0 2024-09-13T08:50:32.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.438+0000 7f5acd400640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8074930 0x7f5aa8076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:32.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.438+0000 7f5acd400640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8074930 0x7f5aa8076df0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009ad0 tx=0x7f5abc0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:32.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.442+0000 7f5ac5600640 1 -- 172.21.15.44:0/3043517878 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5ac0020020 con 0x7f5ac815bea0 2024-09-13T08:50:32.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.618+0000 7f5acfe00640 1 -- 172.21.15.44:0/3043517878 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f5ac8071dd0 con 0x7f5ac815bea0 2024-09-13T08:50:32.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.618+0000 7f5ac5600640 1 -- 172.21.15.44:0/3043517878 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+1998 (secure 0 0 0) 0x7f5ac005fc70 con 0x7f5ac815bea0 2024-09-13T08:50:32.617 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-13T08:50:32.617 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-09-13T08:50:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-13T08:50:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-13T08:50:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4094.620852 2024-09-13T08:50:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3964.014083 2024-09-13T08:50:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3978.263337 2024-09-13T08:50:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4002.298162 2024-09-13T08:50:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4667.000623 2024-09-13T08:50:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4664.046790 2024-09-13T08:50:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6403.016405 2024-09-13T08:50:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4696.908546 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 2/2 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-09-13T08:50:32.619 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-09-13T08:50:32.620 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-09-13T08:50:32.620 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-09-13T08:50:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.618+0000 7f5acfe00640 1 -- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8074930 msgr2=0x7f5aa8076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.618+0000 7f5acfe00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8074930 0x7f5aa8076df0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009ad0 tx=0x7f5abc0091e0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.618+0000 7f5acfe00640 1 -- 172.21.15.44:0/3043517878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac815bea0 msgr2=0x7f5ac814f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.618+0000 7f5acfe00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac815bea0 0x7f5ac814f910 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000cbc0 tx=0x7f5ac000ccb0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.622+0000 7f5acfe00640 1 -- 172.21.15.44:0/3043517878 shutdown_connections 2024-09-13T08:50:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.622+0000 7f5acfe00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8074930 0x7f5aa8076df0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.622+0000 7f5acfe00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac815bea0 0x7f5ac814f910 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.622+0000 7f5acfe00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5ac80d28e0 0x7f5ac814f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.622+0000 7f5acfe00640 1 --2- 172.21.15.44:0/3043517878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ac80cdb30 0x7f5ac80c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.622+0000 7f5acfe00640 1 -- 172.21.15.44:0/3043517878 >> 172.21.15.44:0/3043517878 conn(0x7f5ac80d1dc0 msgr2=0x7f5ac80c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.622+0000 7f5acfe00640 1 -- 172.21.15.44:0/3043517878 shutdown_connections 2024-09-13T08:50:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.622+0000 7f5acfe00640 1 -- 172.21.15.44:0/3043517878 wait complete. 2024-09-13T08:50:32.638 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:50:32.773 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:50:32.776 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:50:32.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.790+0000 7f648ea00640 1 -- 172.21.15.44:0/595781731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64880afb10 msgr2=0x7f64880affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.790+0000 7f648ea00640 1 --2- 172.21.15.44:0/595781731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64880afb10 0x7f64880affc0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f6480009bc0 tx=0x7f648001cba0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.790+0000 7f648ea00640 1 -- 172.21.15.44:0/595781731 shutdown_connections 2024-09-13T08:50:32.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.790+0000 7f648ea00640 1 --2- 172.21.15.44:0/595781731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64880afb10 0x7f64880affc0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.790+0000 7f648ea00640 1 --2- 172.21.15.44:0/595781731 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64880b1650 0x7f64880b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.790+0000 7f648ea00640 1 -- 172.21.15.44:0/595781731 >> 172.21.15.44:0/595781731 conn(0x7f64880d1dd0 msgr2=0x7f64880d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:32.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.790+0000 7f648ea00640 1 -- 172.21.15.44:0/595781731 shutdown_connections 2024-09-13T08:50:32.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.790+0000 7f648ea00640 1 -- 172.21.15.44:0/595781731 wait complete. 2024-09-13T08:50:32.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 Processor -- start 2024-09-13T08:50:32.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 -- start start 2024-09-13T08:50:32.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64880afb10 0x7f64881f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:32.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64880b1650 0x7f64881665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:32.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64881605d0 0x7f648806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:32.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6488169540 con 0x7f64881605d0 2024-09-13T08:50:32.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f64880b1430 con 0x7f64880b1650 2024-09-13T08:50:32.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f64881696c0 con 0x7f64880afb10 2024-09-13T08:50:32.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64881605d0 0x7f648806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64881605d0 0x7f648806dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41250/0 (socket says 172.21.15.44:41250) 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ca00640 1 -- 172.21.15.44:0/3431330477 learned_addr learned my addr 172.21.15.44:0/3431330477 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f6487e00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64880afb10 0x7f64881f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f647fe00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64880b1650 0x7f64881665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ca00640 1 -- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64880afb10 msgr2=0x7f64881f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ca00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64880afb10 0x7f64881f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ca00640 1 -- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64880b1650 msgr2=0x7f64881665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ca00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64880b1650 0x7f64881665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ca00640 1 -- 172.21.15.44:0/3431330477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6488160140 con 0x7f64881605d0 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ca00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64881605d0 0x7f648806dd30 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f648001a7d0 tx=0x7f6480003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f6484c00640 1 -- 172.21.15.44:0/3431330477 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64800b2040 con 0x7f64881605d0 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f6484c00640 1 -- 172.21.15.44:0/3431330477 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6480004d40 con 0x7f64881605d0 2024-09-13T08:50:32.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 -- 172.21.15.44:0/3431330477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64881f8fe0 con 0x7f64881605d0 2024-09-13T08:50:32.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f6484c00640 1 -- 172.21.15.44:0/3431330477 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480004210 con 0x7f64881605d0 2024-09-13T08:50:32.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 -- 172.21.15.44:0/3431330477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64881f8a20 con 0x7f64881605d0 2024-09-13T08:50:32.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.794+0000 7f648ea00640 1 -- 172.21.15.44:0/3431330477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64880affc0 con 0x7f64881605d0 2024-09-13T08:50:32.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.798+0000 7f6484c00640 1 -- 172.21.15.44:0/3431330477 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f64800be060 con 0x7f64881605d0 2024-09-13T08:50:32.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.798+0000 7f6484c00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6464074930 0x7f6464076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:32.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.798+0000 7f6484c00640 1 -- 172.21.15.44:0/3431330477 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f648012c9a0 con 0x7f64881605d0 2024-09-13T08:50:32.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.798+0000 7f6487e00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6464074930 0x7f6464076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:32.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.798+0000 7f6487e00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6464074930 0x7f6464076df0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6478001870 tx=0x7f6478001380 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:32.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.802+0000 7f6484c00640 1 -- 172.21.15.44:0/3431330477 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6480016790 con 0x7f64881605d0 2024-09-13T08:50:32.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.978+0000 7f648ea00640 1 -- 172.21.15.44:0/3431330477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f64880ccab0 con 0x7f64881605d0 2024-09-13T08:50:32.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.982+0000 7f6484c00640 1 -- 172.21.15.44:0/3431330477 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f64800f8f80 con 0x7f64881605d0 2024-09-13T08:50:32.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 -- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6464074930 msgr2=0x7f6464076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6464074930 0x7f6464076df0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6478001870 tx=0x7f6478001380 comp rx=0 tx=0).stop 2024-09-13T08:50:32.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 -- 172.21.15.44:0/3431330477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64881605d0 msgr2=0x7f648806dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:32.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64881605d0 0x7f648806dd30 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f648001a7d0 tx=0x7f6480003f60 comp rx=0 tx=0).stop 2024-09-13T08:50:32.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 -- 172.21.15.44:0/3431330477 shutdown_connections 2024-09-13T08:50:32.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6464074930 0x7f6464076df0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64881605d0 0x7f648806dd30 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64880b1650 0x7f64881665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 --2- 172.21.15.44:0/3431330477 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f64880afb10 0x7f64881f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:32.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 -- 172.21.15.44:0/3431330477 >> 172.21.15.44:0/3431330477 conn(0x7f64880d1dd0 msgr2=0x7f6488070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:32.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 -- 172.21.15.44:0/3431330477 shutdown_connections 2024-09-13T08:50:32.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:32.986+0000 7f648ea00640 1 -- 172.21.15.44:0/3431330477 wait complete. 2024-09-13T08:50:33.003 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:50:33.003 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:50:33.090 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:50:33.181 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:50:33.182 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:50:33.182 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:33.190 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:50:33.191 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:50:33.191 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:33.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:50:33.200 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:50:33.200 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:33.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:50:33.247 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:50:33.247 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:33.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:50:33.257 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T08:50:33.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:50:33.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 -- 172.21.15.44:0/2777106187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53415bd50 msgr2=0x7fe5340ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:33.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 --2- 172.21.15.44:0/2777106187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53415bd50 0x7fe5340ce5e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe524009900 tx=0x7fe52401c7c0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 -- 172.21.15.44:0/2777106187 shutdown_connections 2024-09-13T08:50:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 --2- 172.21.15.44:0/2777106187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe53415c130 0x7fe5340ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 --2- 172.21.15.44:0/2777106187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53415bd50 0x7fe5340ce5e0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 -- 172.21.15.44:0/2777106187 >> 172.21.15.44:0/2777106187 conn(0x7fe5340d1ea0 msgr2=0x7fe5340d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 -- 172.21.15.44:0/2777106187 shutdown_connections 2024-09-13T08:50:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 -- 172.21.15.44:0/2777106187 wait complete. 2024-09-13T08:50:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 Processor -- start 2024-09-13T08:50:33.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 -- start start 2024-09-13T08:50:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5340cf4e0 0x7fe534156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53415bd50 0x7fe5340cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe53415c130 0x7fe534070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe534168e50 con 0x7fe53415bd50 2024-09-13T08:50:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe534168cd0 con 0x7fe5340cf4e0 2024-09-13T08:50:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.442+0000 7fe539e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe534168fd0 con 0x7fe53415c130 2024-09-13T08:50:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53415bd50 0x7fe5340cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:33.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe532a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5340cf4e0 0x7fe534156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:33.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53415bd50 0x7fe5340cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41262/0 (socket says 172.21.15.44:41262) 2024-09-13T08:50:33.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52be00640 1 -- 172.21.15.44:0/1951456932 learned_addr learned my addr 172.21.15.44:0/1951456932 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:33.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe533400640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe53415c130 0x7fe534070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52be00640 1 -- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe53415c130 msgr2=0x7fe534070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52be00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe53415c130 0x7fe534070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52be00640 1 -- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5340cf4e0 msgr2=0x7fe534156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52be00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5340cf4e0 0x7fe534156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52be00640 1 -- 172.21.15.44:0/1951456932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5340cb560 con 0x7fe53415bd50 2024-09-13T08:50:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52be00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53415bd50 0x7fe5340cab80 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00d720 tx=0x7fe51c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52b400640 1 -- 172.21.15.44:0/1951456932 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c017070 con 0x7fe53415bd50 2024-09-13T08:50:33.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52b400640 1 -- 172.21.15.44:0/1951456932 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe51c005310 con 0x7fe53415bd50 2024-09-13T08:50:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe539e00640 1 -- 172.21.15.44:0/1951456932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5341fac00 con 0x7fe53415bd50 2024-09-13T08:50:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52b400640 1 -- 172.21.15.44:0/1951456932 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c009820 con 0x7fe53415bd50 2024-09-13T08:50:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe539e00640 1 -- 172.21.15.44:0/1951456932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5341575e0 con 0x7fe53415bd50 2024-09-13T08:50:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe539e00640 1 -- 172.21.15.44:0/1951456932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5340c5dd0 con 0x7fe53415bd50 2024-09-13T08:50:33.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.446+0000 7fe52b400640 1 -- 172.21.15.44:0/1951456932 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fe51c0099c0 con 0x7fe53415bd50 2024-09-13T08:50:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.450+0000 7fe52b400640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe510074930 0x7fe510076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.450+0000 7fe52b400640 1 -- 172.21.15.44:0/1951456932 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7fe51c092c90 con 0x7fe53415bd50 2024-09-13T08:50:33.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.450+0000 7fe532a00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe510074930 0x7fe510076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:33.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.450+0000 7fe532a00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe510074930 0x7fe510076df0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe5240af000 tx=0x7fe52401b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:33.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.454+0000 7fe52b400640 1 -- 172.21.15.44:0/1951456932 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe51c05f390 con 0x7fe53415bd50 2024-09-13T08:50:33.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.662+0000 7fe539e00640 1 -- 172.21.15.44:0/1951456932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe534076360 con 0x7fe53415bd50 2024-09-13T08:50:33.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe52b400640 1 -- 172.21.15.44:0/1951456932 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+3914 (secure 0 0 0) 0x7fe51c063dc0 con 0x7fe53415bd50 2024-09-13T08:50:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:33.664 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":68,"btime":"2024-09-13T08:50:27:228705+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1987308515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1987308515},{"type":"v1","addr":"172.21.15.129:6837","nonce":1987308515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/1224713128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1224713128},{"type":"v1","addr":"172.21.15.129:6836","nonce":1224713128}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7755,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/380853907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":380853907},{"type":"v1","addr":"172.21.15.44:6833","nonce":380853907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":8172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1273941089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1273941089},{"type":"v1","addr":"172.21.15.44:6835","nonce":1273941089}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[]} 2024-09-13T08:50:33.664 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 68 2024-09-13T08:50:33.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 -- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe510074930 msgr2=0x7fe510076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe510074930 0x7fe510076df0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe5240af000 tx=0x7fe52401b000 comp rx=0 tx=0).stop 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 -- 172.21.15.44:0/1951456932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53415bd50 msgr2=0x7fe5340cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53415bd50 0x7fe5340cab80 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00d720 tx=0x7fe51c00dbe0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 -- 172.21.15.44:0/1951456932 shutdown_connections 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe510074930 0x7fe510076df0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe53415c130 0x7fe534070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe53415bd50 0x7fe5340cab80 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 --2- 172.21.15.44:0/1951456932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5340cf4e0 0x7fe534156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 -- 172.21.15.44:0/1951456932 >> 172.21.15.44:0/1951456932 conn(0x7fe5340d1ea0 msgr2=0x7fe5340c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 -- 172.21.15.44:0/1951456932 shutdown_connections 2024-09-13T08:50:33.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.666+0000 7fe539e00640 1 -- 172.21.15.44:0/1951456932 wait complete. 2024-09-13T08:50:33.685 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T08:50:33.685 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T08:50:33.685 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T08:50:33.685 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:50:33.685 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T08:50:33.685 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T08:50:33.686 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T08:50:33.686 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:50:33.686 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T08:50:33.686 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T08:50:33.686 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T08:50:33.686 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:50:33.686 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T08:50:33.687 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T08:50:33.687 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T08:50:33.687 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:50:33.697 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T08:50:33.697 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T08:50:33.698 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T08:50:33.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T08:50:33.701 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T08:50:33.701 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T08:50:33.703 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T08:50:33.703 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T08:50:33.744 INFO:tasks.ceph.mds.a:Started 2024-09-13T08:50:33.745 DEBUG:teuthology.parallel:result is None 2024-09-13T08:50:33.747 INFO:tasks.ceph.mds.c:Started 2024-09-13T08:50:33.747 DEBUG:teuthology.parallel:result is None 2024-09-13T08:50:33.748 INFO:tasks.ceph.mds.b:Started 2024-09-13T08:50:33.749 DEBUG:teuthology.parallel:result is None 2024-09-13T08:50:33.751 INFO:tasks.ceph.mds.d:Started 2024-09-13T08:50:33.751 DEBUG:teuthology.parallel:result is None 2024-09-13T08:50:33.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T08:50:33.833 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:50:33.833 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:50:33.839 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:50:33.839 INFO:tasks.ceph.mds.b.smithi129.stdout:starting mds.b at 2024-09-13T08:50:33.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 -- 172.21.15.44:0/1071388484 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffac80b2590 msgr2=0x7ffac80b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:33.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 --2- 172.21.15.44:0/1071388484 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffac80b2590 0x7ffac80b29e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009bc0 tx=0x7ffac001cb10 comp rx=0 tx=0).stop 2024-09-13T08:50:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 -- 172.21.15.44:0/1071388484 shutdown_connections 2024-09-13T08:50:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 --2- 172.21.15.44:0/1071388484 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffac80b2590 0x7ffac80b29e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 --2- 172.21.15.44:0/1071388484 >> v2:172.21.15.44:3300/0 conn(0x7ffac815f000 0x7ffac80d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 -- 172.21.15.44:0/1071388484 >> 172.21.15.44:0/1071388484 conn(0x7ffac809aa10 msgr2=0x7ffac809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 -- 172.21.15.44:0/1071388484 shutdown_connections 2024-09-13T08:50:33.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 -- 172.21.15.44:0/1071388484 wait complete. 2024-09-13T08:50:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 Processor -- start 2024-09-13T08:50:33.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 -- start start 2024-09-13T08:50:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffac80b2590 0x7ffac80c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffac80cefb0 0x7ffac8149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac815f000 0x7ffac815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:33.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8168f10 con 0x7ffac815f000 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8168d90 con 0x7ffac80b2590 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffacca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ffac8169090 con 0x7ffac80cefb0 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffac7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac815f000 0x7ffac815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffac7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac815f000 0x7ffac815d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41286/0 (socket says 172.21.15.44:41286) 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.906+0000 7ffac7200640 1 -- 172.21.15.44:0/2823486532 learned_addr learned my addr 172.21.15.44:0/2823486532 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffac6800640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffac80b2590 0x7ffac80c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffabde00640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffac80cefb0 0x7ffac8149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffac7200640 1 -- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffac80cefb0 msgr2=0x7ffac8149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffac7200640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffac80cefb0 0x7ffac8149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffac7200640 1 -- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffac80b2590 msgr2=0x7ffac80c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffac7200640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffac80b2590 0x7ffac80c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffac7200640 1 -- 172.21.15.44:0/2823486532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac80c5850 con 0x7ffac815f000 2024-09-13T08:50:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffac7200640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac815f000 0x7ffac815d430 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009b10 tx=0x7ffac0009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffabf400640 1 -- 172.21.15.44:0/2823486532 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac00ba020 con 0x7ffac815f000 2024-09-13T08:50:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffabf400640 1 -- 172.21.15.44:0/2823486532 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffac0004d40 con 0x7ffac815f000 2024-09-13T08:50:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffacca00640 1 -- 172.21.15.44:0/2823486532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac80c5b30 con 0x7ffac815f000 2024-09-13T08:50:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffabf400640 1 -- 172.21.15.44:0/2823486532 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac00b2040 con 0x7ffac815f000 2024-09-13T08:50:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffacca00640 1 -- 172.21.15.44:0/2823486532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac81597d0 con 0x7ffac815f000 2024-09-13T08:50:33.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffacca00640 1 -- 172.21.15.44:0/2823486532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac80b29e0 con 0x7ffac815f000 2024-09-13T08:50:33.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.910+0000 7ffabf400640 1 -- 172.21.15.44:0/2823486532 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7ffac00a5980 con 0x7ffac815f000 2024-09-13T08:50:33.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.914+0000 7ffabf400640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffaa4074930 0x7ffaa4076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:33.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.914+0000 7ffabf400640 1 -- 172.21.15.44:0/2823486532 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7ffac012d200 con 0x7ffac815f000 2024-09-13T08:50:33.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.914+0000 7ffac6800640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffaa4074930 0x7ffaa4076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:33.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.914+0000 7ffac6800640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffaa4074930 0x7ffaa4076df0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ffab80096c0 tx=0x7ffab8009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:33.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:33.918+0000 7ffabf400640 1 -- 172.21.15.44:0/2823486532 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ffac00c4020 con 0x7ffac815f000 2024-09-13T08:50:34.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.102+0000 7ffacca00640 1 -- 172.21.15.44:0/2823486532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ffac80d0450 con 0x7ffac815f000 2024-09-13T08:50:34.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffabf400640 1 -- 172.21.15.44:0/2823486532 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v110) ==== 97+0+0 (secure 0 0 0) 0x7ffac0016790 con 0x7ffac815f000 2024-09-13T08:50:34.273 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-09-13T08:50:34.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 -- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffaa4074930 msgr2=0x7ffaa4076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:34.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffaa4074930 0x7ffaa4076df0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ffab80096c0 tx=0x7ffab8009260 comp rx=0 tx=0).stop 2024-09-13T08:50:34.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 -- 172.21.15.44:0/2823486532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac815f000 msgr2=0x7ffac815d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac815f000 0x7ffac815d430 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009b10 tx=0x7ffac0009300 comp rx=0 tx=0).stop 2024-09-13T08:50:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 -- 172.21.15.44:0/2823486532 shutdown_connections 2024-09-13T08:50:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ffaa4074930 0x7ffaa4076df0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac815f000 0x7ffac815d430 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ffac80cefb0 0x7ffac8149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 --2- 172.21.15.44:0/2823486532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffac80b2590 0x7ffac80c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 -- 172.21.15.44:0/2823486532 >> 172.21.15.44:0/2823486532 conn(0x7ffac809aa10 msgr2=0x7ffac8075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 -- 172.21.15.44:0/2823486532 shutdown_connections 2024-09-13T08:50:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.274+0000 7ffacca00640 1 -- 172.21.15.44:0/2823486532 wait complete. 2024-09-13T08:50:34.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T08:50:34.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.446+0000 7ff3d3800640 1 -- 172.21.15.44:0/3115387880 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3cc0b49e0 msgr2=0x7ff3cc06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:34.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.446+0000 7ff3d3800640 1 --2- 172.21.15.44:0/3115387880 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3cc0b49e0 0x7ff3cc06f7b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400ad00 tx=0x7ff3c401dda0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.446+0000 7ff3d3800640 1 -- 172.21.15.44:0/3115387880 shutdown_connections 2024-09-13T08:50:34.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.446+0000 7ff3d3800640 1 --2- 172.21.15.44:0/3115387880 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3cc0b49e0 0x7ff3cc06f7b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.446+0000 7ff3d3800640 1 -- 172.21.15.44:0/3115387880 >> 172.21.15.44:0/3115387880 conn(0x7ff3cc0b18f0 msgr2=0x7ff3cc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:34.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.446+0000 7ff3d3800640 1 -- 172.21.15.44:0/3115387880 shutdown_connections 2024-09-13T08:50:34.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.446+0000 7ff3d3800640 1 -- 172.21.15.44:0/3115387880 wait complete. 2024-09-13T08:50:34.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.446+0000 7ff3d3800640 1 Processor -- start 2024-09-13T08:50:34.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.446+0000 7ff3d3800640 1 -- start start 2024-09-13T08:50:34.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d3800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3cc0b49e0 0x7ff3cc1458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:34.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d3800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3cc0cd100 0x7ff3cc0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:34.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d3800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3cc15dae0 0x7ff3cc0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:34.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d3800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1685d0 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d3800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc168450 con 0x7ff3cc0cd100 2024-09-13T08:50:34.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d3800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff3cc168750 con 0x7ff3cc15dae0 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3cc0b49e0 0x7ff3cc1458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3cc0b49e0 0x7ff3cc1458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41292/0 (socket says 172.21.15.44:41292) 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d0e00640 1 -- 172.21.15.44:0/1209217007 learned_addr learned my addr 172.21.15.44:0/1209217007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d1800640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3cc15dae0 0x7ff3cc0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3cbe00640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3cc0cd100 0x7ff3cc0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d0e00640 1 -- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3cc15dae0 msgr2=0x7ff3cc0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d0e00640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3cc15dae0 0x7ff3cc0aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d0e00640 1 -- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3cc0cd100 msgr2=0x7ff3cc0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d0e00640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3cc0cd100 0x7ff3cc0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d0e00640 1 -- 172.21.15.44:0/1209217007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc0af430 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d0e00640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3cc0b49e0 0x7ff3cc1458d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0001230 tx=0x7ff3c000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:34.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3c8c00640 1 -- 172.21.15.44:0/1209217007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0016020 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3c8c00640 1 -- 172.21.15.44:0/1209217007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3c000a930 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d3800640 1 -- 172.21.15.44:0/1209217007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc0af700 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3c8c00640 1 -- 172.21.15.44:0/1209217007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000ad80 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d3800640 1 -- 172.21.15.44:0/1209217007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc0afcc0 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3d3800640 1 -- 172.21.15.44:0/1209217007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3cc069410 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.450+0000 7ff3c8c00640 1 -- 172.21.15.44:0/1209217007 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7ff3c0019020 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.454+0000 7ff3c8c00640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3ac074960 0x7ff3ac076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:34.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.454+0000 7ff3c8c00640 1 -- 172.21.15.44:0/1209217007 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4969+0+0 (secure 0 0 0) 0x7ff3c0092c60 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.454+0000 7ff3cbe00640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3ac074960 0x7ff3ac076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:34.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.454+0000 7ff3cbe00640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3ac074960 0x7ff3ac076e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8007a80 tx=0x7ff3b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:34.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.458+0000 7ff3c8c00640 1 -- 172.21.15.44:0/1209217007 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff3c0020020 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 -- 172.21.15.44:0/1209217007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff3cc0ce810 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3c8c00640 1 -- 172.21.15.44:0/1209217007 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7ff3c005f9b0 con 0x7ff3cc0b49e0 2024-09-13T08:50:34.692 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw==", 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.693 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ==", 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA==", 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:50:34.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw==", 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg==", 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg==", 2024-09-13T08:50:34.696 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw==", 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig==", 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-09-13T08:50:34.698 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ==", 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g==", 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.699 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg==", 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg==", 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA==", 2024-09-13T08:50:34.701 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w==", 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:50:34.702 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA==", 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-09-13T08:50:34.703 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w==", 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmDD2LJxAA3s0jEb2EfPsLcu0w145G3Q==", 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:50:34.704 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ==", 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg==", 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T08:50:34.706 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:50:34.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 -- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3ac074960 msgr2=0x7ff3ac076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:34.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3ac074960 0x7ff3ac076e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8007a80 tx=0x7ff3b8009040 comp rx=0 tx=0).stop 2024-09-13T08:50:34.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 -- 172.21.15.44:0/1209217007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3cc0b49e0 msgr2=0x7ff3cc1458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3cc0b49e0 0x7ff3cc1458d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0001230 tx=0x7ff3c000cb70 comp rx=0 tx=0).stop 2024-09-13T08:50:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 -- 172.21.15.44:0/1209217007 shutdown_connections 2024-09-13T08:50:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff3ac074960 0x7ff3ac076e20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff3cc15dae0 0x7ff3cc0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3cc0cd100 0x7ff3cc0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 --2- 172.21.15.44:0/1209217007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3cc0b49e0 0x7ff3cc1458d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 -- 172.21.15.44:0/1209217007 >> 172.21.15.44:0/1209217007 conn(0x7ff3cc0b18f0 msgr2=0x7ff3cc06fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 -- 172.21.15.44:0/1209217007 shutdown_connections 2024-09-13T08:50:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.694+0000 7ff3d3800640 1 -- 172.21.15.44:0/1209217007 wait complete. 2024-09-13T08:50:34.714 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T08:50:34.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.862+0000 7eff85c00640 1 -- 172.21.15.44:0/2097296464 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff800d28e0 msgr2=0x7eff800cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:34.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.862+0000 7eff85c00640 1 --2- 172.21.15.44:0/2097296464 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff800d28e0 0x7eff800cebb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff78009c80 tx=0x7eff7801cda0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.862+0000 7eff85c00640 1 -- 172.21.15.44:0/2097296464 shutdown_connections 2024-09-13T08:50:34.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.862+0000 7eff85c00640 1 --2- 172.21.15.44:0/2097296464 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff800d28e0 0x7eff800cebb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.862+0000 7eff85c00640 1 --2- 172.21.15.44:0/2097296464 >> v2:172.21.15.129:3300/0 conn(0x7eff8015bea0 0x7eff800ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.862+0000 7eff85c00640 1 -- 172.21.15.44:0/2097296464 >> 172.21.15.44:0/2097296464 conn(0x7eff800d1dc0 msgr2=0x7eff800d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:34.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff85c00640 1 -- 172.21.15.44:0/2097296464 shutdown_connections 2024-09-13T08:50:34.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff85c00640 1 -- 172.21.15.44:0/2097296464 wait complete. 2024-09-13T08:50:34.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff85c00640 1 Processor -- start 2024-09-13T08:50:34.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff85c00640 1 -- start start 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff85c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800cdb30 0x7eff8015a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff85c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff800d28e0 0x7eff80154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff85c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff8015bea0 0x7eff80153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff85c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff80169050 con 0x7eff800cdb30 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff85c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7eff80168ed0 con 0x7eff8015bea0 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff85c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7eff801691d0 con 0x7eff800d28e0 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff7ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800cdb30 0x7eff8015a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff7ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800cdb30 0x7eff8015a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41306/0 (socket says 172.21.15.44:41306) 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff7ea00640 1 -- 172.21.15.44:0/1949943636 learned_addr learned my addr 172.21.15.44:0/1949943636 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff7f400640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff8015bea0 0x7eff80153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff77e00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff800d28e0 0x7eff80154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff7ea00640 1 -- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff800d28e0 msgr2=0x7eff80154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff7ea00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff800d28e0 0x7eff80154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff7ea00640 1 -- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff8015bea0 msgr2=0x7eff80153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff7ea00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff8015bea0 0x7eff80153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff7ea00640 1 -- 172.21.15.44:0/1949943636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8015af80 con 0x7eff800cdb30 2024-09-13T08:50:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.866+0000 7eff7ea00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800cdb30 0x7eff8015a330 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7eff7000e7a0 tx=0x7eff7000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.870+0000 7eff76a00640 1 -- 172.21.15.44:0/1949943636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7000ccc0 con 0x7eff800cdb30 2024-09-13T08:50:34.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.870+0000 7eff76a00640 1 -- 172.21.15.44:0/1949943636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff70010430 con 0x7eff800cdb30 2024-09-13T08:50:34.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.870+0000 7eff76a00640 1 -- 172.21.15.44:0/1949943636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70010860 con 0x7eff800cdb30 2024-09-13T08:50:34.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.870+0000 7eff85c00640 1 -- 172.21.15.44:0/1949943636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff801552f0 con 0x7eff800cdb30 2024-09-13T08:50:34.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.870+0000 7eff85c00640 1 -- 172.21.15.44:0/1949943636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8015a9c0 con 0x7eff800cdb30 2024-09-13T08:50:34.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.870+0000 7eff85c00640 1 -- 172.21.15.44:0/1949943636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff800cc9d0 con 0x7eff800cdb30 2024-09-13T08:50:34.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.870+0000 7eff76a00640 1 -- 172.21.15.44:0/1949943636 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7eff70010a00 con 0x7eff800cdb30 2024-09-13T08:50:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.870+0000 7eff76a00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff5c074930 0x7eff5c076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.870+0000 7eff76a00640 1 -- 172.21.15.44:0/1949943636 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4969+0+0 (secure 0 0 0) 0x7eff700936a0 con 0x7eff800cdb30 2024-09-13T08:50:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.874+0000 7eff77e00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff5c074930 0x7eff5c076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.874+0000 7eff77e00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff5c074930 0x7eff5c076df0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7eff80159a30 tx=0x7eff68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:34.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:34.878+0000 7eff76a00640 1 -- 172.21.15.44:0/1949943636 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7eff7005fd00 con 0x7eff800cdb30 2024-09-13T08:50:35.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.054+0000 7eff85c00640 1 -- 172.21.15.44:0/1949943636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7eff800d2550 con 0x7eff800cdb30 2024-09-13T08:50:35.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.054+0000 7eff76a00640 1 -- 172.21.15.44:0/1949943636 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v112) ==== 84+0+65 (secure 0 0 0) 0x7eff70064730 con 0x7eff800cdb30 2024-09-13T08:50:35.055 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:35.055 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:50:35.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:35.055 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T08:50:35.055 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T08:50:35.055 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:35.055 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:50:35.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 -- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff5c074930 msgr2=0x7eff5c076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:35.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff5c074930 0x7eff5c076df0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7eff80159a30 tx=0x7eff68009040 comp rx=0 tx=0).stop 2024-09-13T08:50:35.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 -- 172.21.15.44:0/1949943636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800cdb30 msgr2=0x7eff8015a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:35.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800cdb30 0x7eff8015a330 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7eff7000e7a0 tx=0x7eff7000ec60 comp rx=0 tx=0).stop 2024-09-13T08:50:35.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 -- 172.21.15.44:0/1949943636 shutdown_connections 2024-09-13T08:50:35.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7eff5c074930 0x7eff5c076df0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:35.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff8015bea0 0x7eff80153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:35.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7eff800d28e0 0x7eff80154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:35.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 --2- 172.21.15.44:0/1949943636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800cdb30 0x7eff8015a330 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:35.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 -- 172.21.15.44:0/1949943636 >> 172.21.15.44:0/1949943636 conn(0x7eff800d1dc0 msgr2=0x7eff800b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:35.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 -- 172.21.15.44:0/1949943636 shutdown_connections 2024-09-13T08:50:35.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.058+0000 7eff85c00640 1 -- 172.21.15.44:0/1949943636 wait complete. 2024-09-13T08:50:35.077 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T08:50:35.077 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T08:50:35.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 -- 172.21.15.44:0/3756119612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bfc15c1f0 msgr2=0x7f4bfc0cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:35.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 --2- 172.21.15.44:0/3756119612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bfc15c1f0 0x7f4bfc0cee10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0b0fb0 tx=0x7f4be401f0b0 comp rx=0 tx=0).stop 2024-09-13T08:50:35.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 -- 172.21.15.44:0/3756119612 shutdown_connections 2024-09-13T08:50:35.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 --2- 172.21.15.44:0/3756119612 >> v2:172.21.15.129:3300/0 conn(0x7f4bfc0b0b60 0x7f4bfc0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:35.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 --2- 172.21.15.44:0/3756119612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bfc15c1f0 0x7f4bfc0cee10 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:35.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 -- 172.21.15.44:0/3756119612 >> 172.21.15.44:0/3756119612 conn(0x7f4bfc0d1da0 msgr2=0x7f4bfc0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:35.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 -- 172.21.15.44:0/3756119612 shutdown_connections 2024-09-13T08:50:35.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 -- 172.21.15.44:0/3756119612 wait complete. 2024-09-13T08:50:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 Processor -- start 2024-09-13T08:50:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 -- start start 2024-09-13T08:50:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4bfc0b0b60 0x7f4bfc1f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bfc15be10 0x7f4bfc0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bfc15c1f0 0x7f4bfc14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc0cd4a0 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc0cd320 con 0x7f4bfc15be10 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.230+0000 7f4c02600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4bfc0cd620 con 0x7f4bfc0b0b60 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bfbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bfc15c1f0 0x7f4bfc14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bf3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bfc15be10 0x7f4bfc0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bfbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bfc15c1f0 0x7f4bfc14e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41320/0 (socket says 172.21.15.44:41320) 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bfb400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4bfc0b0b60 0x7f4bfc1f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bfbe00640 1 -- 172.21.15.44:0/3352392338 learned_addr learned my addr 172.21.15.44:0/3352392338 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bfbe00640 1 -- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4bfc0b0b60 msgr2=0x7f4bfc1f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bfbe00640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4bfc0b0b60 0x7f4bfc1f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bfbe00640 1 -- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bfc15be10 msgr2=0x7f4bfc0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bfbe00640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bfc15be10 0x7f4bfc0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bfbe00640 1 -- 172.21.15.44:0/3352392338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bfc1f3630 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bf3e00640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bfc15be10 0x7f4bfc0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:50:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bfbe00640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bfc15c1f0 0x7f4bfc14e1f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400ea20 tx=0x7f4bf400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bf3400640 1 -- 172.21.15.44:0/3352392338 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf400cde0 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bf3400640 1 -- 172.21.15.44:0/3352392338 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bf4004c40 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bf3400640 1 -- 172.21.15.44:0/3352392338 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4005280 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4c02600640 1 -- 172.21.15.44:0/3352392338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bfc0c8e70 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4c02600640 1 -- 172.21.15.44:0/3352392338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bfc0c96d0 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4c02600640 1 -- 172.21.15.44:0/3352392338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bfc0c9170 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.234+0000 7f4bf3400640 1 -- 172.21.15.44:0/3352392338 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f4bf4005420 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.238+0000 7f4bf3400640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4bd8074930 0x7f4bd8076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.238+0000 7f4bf3400640 1 -- 172.21.15.44:0/3352392338 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4969+0+0 (secure 0 0 0) 0x7f4bf4093680 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.238+0000 7f4bfb400640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4bd8074930 0x7f4bd8076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:35.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.238+0000 7f4bfb400640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4bd8074930 0x7f4bd8076df0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4bec007fd0 tx=0x7f4bec007e40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:35.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.242+0000 7f4bf3400640 1 -- 172.21.15.44:0/3352392338 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4bfc0c9170 con 0x7f4bfc15c1f0 2024-09-13T08:50:35.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:35.418+0000 7f4c02600640 1 -- 172.21.15.44:0/3352392338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4bfc0c55e0 con 0x7f4bfc15c1f0 2024-09-13T08:50:36.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.286+0000 7f4bf3400640 1 -- 172.21.15.44:0/3352392338 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v113) ==== 138+0+0 (secure 0 0 0) 0x7f4bfc1f3630 con 0x7f4bfc15c1f0 2024-09-13T08:50:36.286 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T08:50:36.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 -- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4bd8074930 msgr2=0x7f4bd8076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:36.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4bd8074930 0x7f4bd8076df0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4bec007fd0 tx=0x7f4bec007e40 comp rx=0 tx=0).stop 2024-09-13T08:50:36.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 -- 172.21.15.44:0/3352392338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bfc15c1f0 msgr2=0x7f4bfc14e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:36.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bfc15c1f0 0x7f4bfc14e1f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400ea20 tx=0x7f4bf400eee0 comp rx=0 tx=0).stop 2024-09-13T08:50:36.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 -- 172.21.15.44:0/3352392338 shutdown_connections 2024-09-13T08:50:36.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4bd8074930 0x7f4bd8076df0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:36.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bfc15c1f0 0x7f4bfc14e1f0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:36.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bfc15be10 0x7f4bfc0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:36.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 --2- 172.21.15.44:0/3352392338 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4bfc0b0b60 0x7f4bfc1f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:36.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 -- 172.21.15.44:0/3352392338 >> 172.21.15.44:0/3352392338 conn(0x7f4bfc0d1da0 msgr2=0x7f4bfc0b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:36.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 -- 172.21.15.44:0/3352392338 shutdown_connections 2024-09-13T08:50:36.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.290+0000 7f4c02600640 1 -- 172.21.15.44:0/3352392338 wait complete. 2024-09-13T08:50:36.317 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T08:50:36.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.478+0000 7fa274400640 1 -- 172.21.15.44:0/118447815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa254004810 msgr2=0x7fa254004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:36.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.478+0000 7fa274400640 1 --2- 172.21.15.44:0/118447815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa254004810 0x7fa254004c90 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fa258006fc0 tx=0x7fa25801cec0 comp rx=0 tx=0).stop 2024-09-13T08:50:36.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.478+0000 7fa272400640 1 -- 172.21.15.44:0/118447815 reap_dead start 2024-09-13T08:50:36.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.478+0000 7fa274400640 1 -- 172.21.15.44:0/118447815 shutdown_connections 2024-09-13T08:50:36.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.478+0000 7fa274400640 1 -- 172.21.15.44:0/118447815 >> 172.21.15.44:0/118447815 conn(0x7fa26c0cd740 msgr2=0x7fa26c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:36.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.478+0000 7fa274400640 1 -- 172.21.15.44:0/118447815 shutdown_connections 2024-09-13T08:50:36.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.478+0000 7fa274400640 1 -- 172.21.15.44:0/118447815 wait complete. 2024-09-13T08:50:36.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.478+0000 7fa274400640 1 Processor -- start 2024-09-13T08:50:36.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.478+0000 7fa274400640 1 -- start start 2024-09-13T08:50:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa274400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa254004810 0x7fa26c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa274400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa26c0b66e0 0x7fa26c0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa274400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa26c1f1e40 0x7fa26c15d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa274400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c070000 con 0x7fa26c1f1e40 2024-09-13T08:50:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa274400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c0cf590 con 0x7fa26c0b66e0 2024-09-13T08:50:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa274400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa26c070180 con 0x7fa254004810 2024-09-13T08:50:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa272400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa26c1f1e40 0x7fa26c15d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa272400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa26c1f1e40 0x7fa26c15d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41336/0 (socket says 172.21.15.44:41336) 2024-09-13T08:50:36.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa272400640 1 -- 172.21.15.44:0/4091304348 learned_addr learned my addr 172.21.15.44:0/4091304348 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:36.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa271000640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa26c0b66e0 0x7fa26c0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:36.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa271a00640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa254004810 0x7fa26c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:36.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa272400640 1 -- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa254004810 msgr2=0x7fa26c0b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:36.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa272400640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa254004810 0x7fa26c0b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:36.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa272400640 1 -- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa26c0b66e0 msgr2=0x7fa26c0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:36.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa272400640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa26c0b66e0 0x7fa26c0b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:36.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa272400640 1 -- 172.21.15.44:0/4091304348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26c0b5640 con 0x7fa26c1f1e40 2024-09-13T08:50:36.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa272400640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa26c1f1e40 0x7fa26c15d410 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0cd020 tx=0x7fa26400e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:36.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa269600640 1 -- 172.21.15.44:0/4091304348 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264003b80 con 0x7fa26c1f1e40 2024-09-13T08:50:36.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa269600640 1 -- 172.21.15.44:0/4091304348 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa264003e30 con 0x7fa26c1f1e40 2024-09-13T08:50:36.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa269600640 1 -- 172.21.15.44:0/4091304348 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264006350 con 0x7fa26c1f1e40 2024-09-13T08:50:36.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa274400640 1 -- 172.21.15.44:0/4091304348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26c0b5920 con 0x7fa26c1f1e40 2024-09-13T08:50:36.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa274400640 1 -- 172.21.15.44:0/4091304348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26c0b5ee0 con 0x7fa26c1f1e40 2024-09-13T08:50:36.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.482+0000 7fa274400640 1 -- 172.21.15.44:0/4091304348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26c0706e0 con 0x7fa26c1f1e40 2024-09-13T08:50:36.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.486+0000 7fa269600640 1 -- 172.21.15.44:0/4091304348 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fa26400d070 con 0x7fa26c1f1e40 2024-09-13T08:50:36.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.486+0000 7fa269600640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa24c074960 0x7fa24c076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:36.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.486+0000 7fa269600640 1 -- 172.21.15.44:0/4091304348 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5339+0+0 (secure 0 0 0) 0x7fa264096c40 con 0x7fa26c1f1e40 2024-09-13T08:50:36.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.486+0000 7fa271a00640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa24c074960 0x7fa24c076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:36.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.486+0000 7fa271a00640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa24c074960 0x7fa24c076e20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa260009a80 tx=0x7fa2600091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.490+0000 7fa269600640 1 -- 172.21.15.44:0/4091304348 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa264063320 con 0x7fa26c1f1e40 2024-09-13T08:50:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:36.666+0000 7fa274400640 1 -- 172.21.15.44:0/4091304348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa26c0c4590 con 0x7fa26c1f1e40 2024-09-13T08:50:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.294+0000 7fa269600640 1 -- 172.21.15.44:0/4091304348 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v114) ==== 170+0+0 (secure 0 0 0) 0x7fa264067d50 con 0x7fa26c1f1e40 2024-09-13T08:50:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T08:50:37.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 -- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa24c074960 msgr2=0x7fa24c076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:37.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa24c074960 0x7fa24c076e20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa260009a80 tx=0x7fa2600091e0 comp rx=0 tx=0).stop 2024-09-13T08:50:37.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 -- 172.21.15.44:0/4091304348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa26c1f1e40 msgr2=0x7fa26c15d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:37.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa26c1f1e40 0x7fa26c15d410 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0cd020 tx=0x7fa26400e6b0 comp rx=0 tx=0).stop 2024-09-13T08:50:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 -- 172.21.15.44:0/4091304348 shutdown_connections 2024-09-13T08:50:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa24c074960 0x7fa24c076e20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa26c1f1e40 0x7fa26c15d410 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa26c0b66e0 0x7fa26c0b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 --2- 172.21.15.44:0/4091304348 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa254004810 0x7fa26c0b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 -- 172.21.15.44:0/4091304348 >> 172.21.15.44:0/4091304348 conn(0x7fa26c0cd740 msgr2=0x7fa26c0b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 -- 172.21.15.44:0/4091304348 shutdown_connections 2024-09-13T08:50:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.298+0000 7fa274400640 1 -- 172.21.15.44:0/4091304348 wait complete. 2024-09-13T08:50:37.337 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T08:50:37.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.490+0000 7f146d800640 1 -- 172.21.15.44:0/1546153913 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f146815dde0 msgr2=0x7f1468161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:37.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.490+0000 7f146d800640 1 --2- 172.21.15.44:0/1546153913 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f146815dde0 0x7f1468161b70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1458009bf0 tx=0x7f145801cb20 comp rx=0 tx=0).stop 2024-09-13T08:50:37.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 -- 172.21.15.44:0/1546153913 shutdown_connections 2024-09-13T08:50:37.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 --2- 172.21.15.44:0/1546153913 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f146815dde0 0x7f1468161b70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:37.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 -- 172.21.15.44:0/1546153913 >> 172.21.15.44:0/1546153913 conn(0x7f14680d0cd0 msgr2=0x7f14680d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:37.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 -- 172.21.15.44:0/1546153913 shutdown_connections 2024-09-13T08:50:37.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 -- 172.21.15.44:0/1546153913 wait complete. 2024-09-13T08:50:37.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 Processor -- start 2024-09-13T08:50:37.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 -- start start 2024-09-13T08:50:37.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14680d02e0 0x7f1468159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:37.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f146815dde0 0x7f1468156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:37.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f146815e1c0 0x7f1468156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:37.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14680cb6b0 con 0x7f146815e1c0 2024-09-13T08:50:37.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f14680cb530 con 0x7f14680d02e0 2024-09-13T08:50:37.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f146d800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f14680cb830 con 0x7f146815dde0 2024-09-13T08:50:37.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f1467400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f146815e1c0 0x7f1468156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:37.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f1466a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14680d02e0 0x7f1468159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:37.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f1467400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f146815e1c0 0x7f1468156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41348/0 (socket says 172.21.15.44:41348) 2024-09-13T08:50:37.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f1467400640 1 -- 172.21.15.44:0/609494162 learned_addr learned my addr 172.21.15.44:0/609494162 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:37.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f145e000640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f146815dde0 0x7f1468156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f1466a00640 1 -- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f146815dde0 msgr2=0x7f1468156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f1466a00640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f146815dde0 0x7f1468156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f1466a00640 1 -- 172.21.15.44:0/609494162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f146815e1c0 msgr2=0x7f1468156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f1466a00640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f146815e1c0 0x7f1468156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f1466a00640 1 -- 172.21.15.44:0/609494162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1468160370 con 0x7f14680d02e0 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.494+0000 7f1466a00640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14680d02e0 0x7f1468159070 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f14580ac000 tx=0x7f14580092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.498+0000 7f145f400640 1 -- 172.21.15.44:0/609494162 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14580bb070 con 0x7f14680d02e0 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.498+0000 7f145f400640 1 -- 172.21.15.44:0/609494162 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1458004e20 con 0x7f14680d02e0 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.498+0000 7f146d800640 1 -- 172.21.15.44:0/609494162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1468160650 con 0x7f14680d02e0 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.498+0000 7f145f400640 1 -- 172.21.15.44:0/609494162 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14580b2040 con 0x7f14680d02e0 2024-09-13T08:50:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.498+0000 7f146d800640 1 -- 172.21.15.44:0/609494162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1468160c10 con 0x7f14680d02e0 2024-09-13T08:50:37.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.498+0000 7f146d800640 1 -- 172.21.15.44:0/609494162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1468161b70 con 0x7f14680d02e0 2024-09-13T08:50:37.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.498+0000 7f145f400640 1 -- 172.21.15.44:0/609494162 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f14580a5aa0 con 0x7f14680d02e0 2024-09-13T08:50:37.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.498+0000 7f145f400640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1444074960 0x7f1444076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:37.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.498+0000 7f145f400640 1 -- 172.21.15.44:0/609494162 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5721+0+0 (secure 0 0 0) 0x7f145812cb10 con 0x7f14680d02e0 2024-09-13T08:50:37.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.498+0000 7f145e000640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1444074960 0x7f1444076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:37.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.502+0000 7f145e000640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1444074960 0x7f1444076e20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f14680cc000 tx=0x7f1450009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:37.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.506+0000 7f145f400640 1 -- 172.21.15.44:0/609494162 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f14580167d0 con 0x7f14680d02e0 2024-09-13T08:50:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:37.718+0000 7f146d800640 1 -- 172.21.15.44:0/609494162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f14680cc030 con 0x7f14680d02e0 2024-09-13T08:50:38.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.302+0000 7f145f400640 1 -- 172.21.15.44:0/609494162 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v71) ==== 174+0+0 (secure 0 0 0) 0x7f14580f90f0 con 0x7f14680d02e0 2024-09-13T08:50:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 16 and data pool 17 2024-09-13T08:50:38.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.322+0000 7f146d800640 1 -- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1444074960 msgr2=0x7f1444076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:38.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.322+0000 7f146d800640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1444074960 0x7f1444076e20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f14680cc000 tx=0x7f1450009040 comp rx=0 tx=0).stop 2024-09-13T08:50:38.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.322+0000 7f146d800640 1 -- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14680d02e0 msgr2=0x7f1468159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:38.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.322+0000 7f146d800640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14680d02e0 0x7f1468159070 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f14580ac000 tx=0x7f14580092f0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.322+0000 7f146d800640 1 -- 172.21.15.44:0/609494162 shutdown_connections 2024-09-13T08:50:38.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.322+0000 7f146d800640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1444074960 0x7f1444076e20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.322+0000 7f146d800640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f146815e1c0 0x7f1468156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.322+0000 7f146d800640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f146815dde0 0x7f1468156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.322+0000 7f146d800640 1 --2- 172.21.15.44:0/609494162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14680d02e0 0x7f1468159070 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.322+0000 7f146d800640 1 -- 172.21.15.44:0/609494162 >> 172.21.15.44:0/609494162 conn(0x7f14680d0cd0 msgr2=0x7f14680aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:38.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.326+0000 7f146d800640 1 -- 172.21.15.44:0/609494162 shutdown_connections 2024-09-13T08:50:38.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.326+0000 7f146d800640 1 -- 172.21.15.44:0/609494162 wait complete. 2024-09-13T08:50:38.357 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:50:38.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.506+0000 7f15a8600640 1 -- 172.21.15.44:0/3745390400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15a00cebf0 msgr2=0x7f15a00cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:38.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.506+0000 7f15a8600640 1 --2- 172.21.15.44:0/3745390400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15a00cebf0 0x7f15a00cc8b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f158c009900 tx=0x7f158c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.506+0000 7f15a8600640 1 -- 172.21.15.44:0/3745390400 shutdown_connections 2024-09-13T08:50:38.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.506+0000 7f15a8600640 1 --2- 172.21.15.44:0/3745390400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15a00cebf0 0x7f15a00cc8b0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.506+0000 7f15a8600640 1 -- 172.21.15.44:0/3745390400 >> 172.21.15.44:0/3745390400 conn(0x7f15a00d1da0 msgr2=0x7f15a00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:38.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.506+0000 7f15a8600640 1 -- 172.21.15.44:0/3745390400 shutdown_connections 2024-09-13T08:50:38.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.506+0000 7f15a8600640 1 -- 172.21.15.44:0/3745390400 wait complete. 2024-09-13T08:50:38.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a8600640 1 Processor -- start 2024-09-13T08:50:38.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a8600640 1 -- start start 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a8600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f15a00b6400 0x7f15a01f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a8600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15a00cebf0 0x7f15a014bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a8600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15a015be80 0x7f15a00c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a8600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15a016a360 con 0x7f15a00cebf0 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a8600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15a016a1e0 con 0x7f15a015be80 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a8600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f15a016a4e0 con 0x7f15a00b6400 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15a00cebf0 0x7f15a014bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15a00cebf0 0x7f15a014bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41374/0 (socket says 172.21.15.44:41374) 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a5200640 1 -- 172.21.15.44:0/1962335437 learned_addr learned my addr 172.21.15.44:0/1962335437 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a5c00640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f15a00b6400 0x7f15a01f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:38.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a6600640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15a015be80 0x7f15a00c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:38.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a5200640 1 -- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f15a00b6400 msgr2=0x7f15a01f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:38.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a5200640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f15a00b6400 0x7f15a01f8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a5200640 1 -- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15a015be80 msgr2=0x7f15a00c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:38.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a5200640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15a015be80 0x7f15a00c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a5200640 1 -- 172.21.15.44:0/1962335437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a00c94d0 con 0x7f15a00cebf0 2024-09-13T08:50:38.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a5200640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15a00cebf0 0x7f15a014bea0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f158c009850 tx=0x7f158c0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f159d600640 1 -- 172.21.15.44:0/1962335437 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c0b2040 con 0x7f15a00cebf0 2024-09-13T08:50:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f159d600640 1 -- 172.21.15.44:0/1962335437 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f158c005020 con 0x7f15a00cebf0 2024-09-13T08:50:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a8600640 1 -- 172.21.15.44:0/1962335437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a00c97b0 con 0x7f15a00cebf0 2024-09-13T08:50:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f159d600640 1 -- 172.21.15.44:0/1962335437 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c005470 con 0x7f15a00cebf0 2024-09-13T08:50:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.510+0000 7f15a8600640 1 -- 172.21.15.44:0/1962335437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a014c3e0 con 0x7f15a00cebf0 2024-09-13T08:50:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.514+0000 7f15a8600640 1 -- 172.21.15.44:0/1962335437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a0072490 con 0x7f15a00cebf0 2024-09-13T08:50:38.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.514+0000 7f159d600640 1 -- 172.21.15.44:0/1962335437 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f158c0046d0 con 0x7f15a00cebf0 2024-09-13T08:50:38.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.514+0000 7f159d600640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1580074960 0x7f1580076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:38.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.514+0000 7f159d600640 1 -- 172.21.15.44:0/1962335437 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5821+0+0 (secure 0 0 0) 0x7f158c12caf0 con 0x7f15a00cebf0 2024-09-13T08:50:38.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.514+0000 7f15a5c00640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1580074960 0x7f1580076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:38.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.518+0000 7f15a5c00640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1580074960 0x7f1580076e20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1594000f70 tx=0x7f1594009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:38.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.518+0000 7f159d600640 1 -- 172.21.15.44:0/1962335437 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f158c0164d0 con 0x7f15a00cebf0 2024-09-13T08:50:38.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.694+0000 7f15a8600640 1 -- 172.21.15.44:0/1962335437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15a00cdfb0 con 0x7f15a00cebf0 2024-09-13T08:50:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f159d600640 1 -- 172.21.15.44:0/1962335437 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16945 (secure 0 0 0) 0x7f158c0f9820 con 0x7f15a00cebf0 2024-09-13T08:50:38.696 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:38.696 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":115,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:50:37.721681+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:50:35.432294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-13T08:50:36.678893+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:50:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 -- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1580074960 msgr2=0x7f1580076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1580074960 0x7f1580076e20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1594000f70 tx=0x7f1594009040 comp rx=0 tx=0).stop 2024-09-13T08:50:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 -- 172.21.15.44:0/1962335437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15a00cebf0 msgr2=0x7f15a014bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15a00cebf0 0x7f15a014bea0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f158c009850 tx=0x7f158c0040e0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 -- 172.21.15.44:0/1962335437 shutdown_connections 2024-09-13T08:50:38.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1580074960 0x7f1580076e20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15a015be80 0x7f15a00c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15a00cebf0 0x7f15a014bea0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 --2- 172.21.15.44:0/1962335437 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f15a00b6400 0x7f15a01f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 -- 172.21.15.44:0/1962335437 >> 172.21.15.44:0/1962335437 conn(0x7f15a00d1da0 msgr2=0x7f15a00d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 -- 172.21.15.44:0/1962335437 shutdown_connections 2024-09-13T08:50:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.698+0000 7f15a8600640 1 -- 172.21.15.44:0/1962335437 wait complete. 2024-09-13T08:50:38.718 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:50:38.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.866+0000 7f9df5400640 1 -- 172.21.15.44:0/4086063481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd4003c20 msgr2=0x7f9dd4004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:38.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.866+0000 7f9df5400640 1 --2- 172.21.15.44:0/4086063481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd4003c20 0x7f9dd4004020 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009b90 tx=0x7f9de001d240 comp rx=0 tx=0).stop 2024-09-13T08:50:38.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.866+0000 7f9def200640 1 -- 172.21.15.44:0/4086063481 reap_dead start 2024-09-13T08:50:38.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.866+0000 7f9df5400640 1 -- 172.21.15.44:0/4086063481 shutdown_connections 2024-09-13T08:50:38.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.866+0000 7f9df5400640 1 -- 172.21.15.44:0/4086063481 >> 172.21.15.44:0/4086063481 conn(0x7f9df00d1e30 msgr2=0x7f9df00d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:38.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.866+0000 7f9df5400640 1 -- 172.21.15.44:0/4086063481 shutdown_connections 2024-09-13T08:50:38.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.866+0000 7f9df5400640 1 -- 172.21.15.44:0/4086063481 wait complete. 2024-09-13T08:50:38.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 Processor -- start 2024-09-13T08:50:38.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 -- start start 2024-09-13T08:50:38.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd4003c20 0x7f9df014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:38.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9df00b5de0 0x7f9df00b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:38.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9df014e7c0 0x7f9df014ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9df00b2290 con 0x7f9dd4003c20 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9df00b2110 con 0x7f9df00b5de0 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9df0168490 con 0x7f9df014e7c0 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9dee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd4003c20 0x7f9df014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9dee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd4003c20 0x7f9df014be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41394/0 (socket says 172.21.15.44:41394) 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9dee800640 1 -- 172.21.15.44:0/4253475508 learned_addr learned my addr 172.21.15.44:0/4253475508 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9def200640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9df014e7c0 0x7f9df014ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9de7e00640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9df00b5de0 0x7f9df00b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9dee800640 1 -- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9df014e7c0 msgr2=0x7f9df014ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9dee800640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9df014e7c0 0x7f9df014ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9dee800640 1 -- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9df00b5de0 msgr2=0x7f9df00b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9dee800640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9df00b5de0 0x7f9df00b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:38.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9dee800640 1 -- 172.21.15.44:0/4253475508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df00b1a50 con 0x7f9dd4003c20 2024-09-13T08:50:38.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9def200640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9df014e7c0 0x7f9df014ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:38.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9dee800640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd4003c20 0x7f9df014be80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f9de001d070 tx=0x7f9de0005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:38.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9de6a00640 1 -- 172.21.15.44:0/4253475508 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de00b1040 con 0x7f9dd4003c20 2024-09-13T08:50:38.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9de6a00640 1 -- 172.21.15.44:0/4253475508 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9de00ac400 con 0x7f9dd4003c20 2024-09-13T08:50:38.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9de6a00640 1 -- 172.21.15.44:0/4253475508 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0009730 con 0x7f9dd4003c20 2024-09-13T08:50:38.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 -- 172.21.15.44:0/4253475508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df01f83e0 con 0x7f9dd4003c20 2024-09-13T08:50:38.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 -- 172.21.15.44:0/4253475508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df014f610 con 0x7f9dd4003c20 2024-09-13T08:50:38.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.870+0000 7f9df5400640 1 -- 172.21.15.44:0/4253475508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df00b0130 con 0x7f9dd4003c20 2024-09-13T08:50:38.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.874+0000 7f9de6a00640 1 -- 172.21.15.44:0/4253475508 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f9de00bd020 con 0x7f9dd4003c20 2024-09-13T08:50:38.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.874+0000 7f9de6a00640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9dcc074960 0x7f9dcc076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:38.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.874+0000 7f9de6a00640 1 -- 172.21.15.44:0/4253475508 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5821+0+0 (secure 0 0 0) 0x7f9de012d340 con 0x7f9dd4003c20 2024-09-13T08:50:38.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.874+0000 7f9de7e00640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9dcc074960 0x7f9dcc076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:38.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.874+0000 7f9de7e00640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9dcc074960 0x7f9dcc076e20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8006800 tx=0x7f9dd8006790 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:38.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:38.878+0000 7f9de6a00640 1 -- 172.21.15.44:0/4253475508 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9de00c4020 con 0x7f9dd4003c20 2024-09-13T08:50:39.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 -- 172.21.15.44:0/4253475508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9df006ee50 con 0x7f9dd4003c20 2024-09-13T08:50:39.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9de6a00640 1 -- 172.21.15.44:0/4253475508 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16945 (secure 0 0 0) 0x7f9de00a5e20 con 0x7f9dd4003c20 2024-09-13T08:50:39.052 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:39.053 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":115,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:50:37.721681+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:50:35.432294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-13T08:50:36.678893+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 -- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9dcc074960 msgr2=0x7f9dcc076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9dcc074960 0x7f9dcc076e20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8006800 tx=0x7f9dd8006790 comp rx=0 tx=0).stop 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 -- 172.21.15.44:0/4253475508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd4003c20 msgr2=0x7f9df014be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd4003c20 0x7f9df014be80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f9de001d070 tx=0x7f9de0005590 comp rx=0 tx=0).stop 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 -- 172.21.15.44:0/4253475508 shutdown_connections 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9dcc074960 0x7f9dcc076e20 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9df014e7c0 0x7f9df014ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9df00b5de0 0x7f9df00b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 --2- 172.21.15.44:0/4253475508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd4003c20 0x7f9df014be80 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 -- 172.21.15.44:0/4253475508 >> 172.21.15.44:0/4253475508 conn(0x7f9df00d1e30 msgr2=0x7f9df0166d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 -- 172.21.15.44:0/4253475508 shutdown_connections 2024-09-13T08:50:39.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.054+0000 7f9df5400640 1 -- 172.21.15.44:0/4253475508 wait complete. 2024-09-13T08:50:39.075 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T08:50:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.226+0000 7fca97400640 1 -- 172.21.15.44:0/1062578760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9015ac70 msgr2=0x7fca900cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.226+0000 7fca97400640 1 --2- 172.21.15.44:0/1062578760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9015ac70 0x7fca900cf2a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fca84009930 tx=0x7fca8401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:50:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.226+0000 7fca97400640 1 -- 172.21.15.44:0/1062578760 shutdown_connections 2024-09-13T08:50:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.226+0000 7fca97400640 1 --2- 172.21.15.44:0/1062578760 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca900cbdb0 0x7fca900b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.226+0000 7fca97400640 1 --2- 172.21.15.44:0/1062578760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9015ac70 0x7fca900cf2a0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.226+0000 7fca97400640 1 -- 172.21.15.44:0/1062578760 >> 172.21.15.44:0/1062578760 conn(0x7fca900cd740 msgr2=0x7fca900cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.226+0000 7fca97400640 1 -- 172.21.15.44:0/1062578760 shutdown_connections 2024-09-13T08:50:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.226+0000 7fca97400640 1 -- 172.21.15.44:0/1062578760 wait complete. 2024-09-13T08:50:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca97400640 1 Processor -- start 2024-09-13T08:50:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca97400640 1 -- start start 2024-09-13T08:50:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca97400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca900b2c80 0x7fca90162930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca97400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca900cbdb0 0x7fca901fa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca97400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9015ac70 0x7fca90154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca97400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fca90167c60 con 0x7fca9015ac70 2024-09-13T08:50:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca97400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fca90167ae0 con 0x7fca900b2c80 2024-09-13T08:50:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca97400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fca90167de0 con 0x7fca900cbdb0 2024-09-13T08:50:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca95400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9015ac70 0x7fca90154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca95400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9015ac70 0x7fca90154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41408/0 (socket says 172.21.15.44:41408) 2024-09-13T08:50:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca95400640 1 -- 172.21.15.44:0/3657472277 learned_addr learned my addr 172.21.15.44:0/3657472277 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca94a00640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca900b2c80 0x7fca90162930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca8fe00640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca900cbdb0 0x7fca901fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:39.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca95400640 1 -- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca900cbdb0 msgr2=0x7fca901fa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:39.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca95400640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca900cbdb0 0x7fca901fa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca95400640 1 -- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca900b2c80 msgr2=0x7fca90162930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca95400640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca900b2c80 0x7fca90162930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca95400640 1 -- 172.21.15.44:0/3657472277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca90153620 con 0x7fca9015ac70 2024-09-13T08:50:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca95400640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9015ac70 0x7fca90154e20 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fca8800c700 tx=0x7fca8800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca8cc00640 1 -- 172.21.15.44:0/3657472277 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88003b80 con 0x7fca9015ac70 2024-09-13T08:50:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca8cc00640 1 -- 172.21.15.44:0/3657472277 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca88003d20 con 0x7fca9015ac70 2024-09-13T08:50:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca8cc00640 1 -- 172.21.15.44:0/3657472277 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8800aab0 con 0x7fca9015ac70 2024-09-13T08:50:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca97400640 1 -- 172.21.15.44:0/3657472277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca901fad60 con 0x7fca9015ac70 2024-09-13T08:50:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.230+0000 7fca97400640 1 -- 172.21.15.44:0/3657472277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca90153ba0 con 0x7fca9015ac70 2024-09-13T08:50:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.234+0000 7fca97400640 1 -- 172.21.15.44:0/3657472277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca900b2740 con 0x7fca9015ac70 2024-09-13T08:50:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.234+0000 7fca8cc00640 1 -- 172.21.15.44:0/3657472277 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fca8800ac50 con 0x7fca9015ac70 2024-09-13T08:50:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.234+0000 7fca8cc00640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca70074960 0x7fca70076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.234+0000 7fca8cc00640 1 -- 172.21.15.44:0/3657472277 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5821+0+0 (secure 0 0 0) 0x7fca880929a0 con 0x7fca9015ac70 2024-09-13T08:50:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.234+0000 7fca94a00640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca70074960 0x7fca70076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:39.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.238+0000 7fca94a00640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca70074960 0x7fca70076e20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fca840af000 tx=0x7fca8401b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.238+0000 7fca8cc00640 1 -- 172.21.15.44:0/3657472277 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fca8805f000 con 0x7fca9015ac70 2024-09-13T08:50:39.271 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-09-13T08:50:39.274+0000 7fcf15200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:50:39.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:39.454+0000 7fca97400640 1 -- 172.21.15.44:0/3657472277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fca900c4590 con 0x7fca9015ac70 2024-09-13T08:50:40.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca8cc00640 1 -- 172.21.15.44:0/3657472277 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) ==== 118+0+0 (secure 0 0 0) 0x7fca88063a30 con 0x7fca9015ac70 2024-09-13T08:50:40.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 -- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca70074960 msgr2=0x7fca70076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:40.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca70074960 0x7fca70076e20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fca840af000 tx=0x7fca8401b120 comp rx=0 tx=0).stop 2024-09-13T08:50:40.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 -- 172.21.15.44:0/3657472277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9015ac70 msgr2=0x7fca90154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:40.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9015ac70 0x7fca90154e20 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fca8800c700 tx=0x7fca8800cbc0 comp rx=0 tx=0).stop 2024-09-13T08:50:40.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 -- 172.21.15.44:0/3657472277 shutdown_connections 2024-09-13T08:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fca70074960 0x7fca70076e20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca9015ac70 0x7fca90154e20 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fca900cbdb0 0x7fca901fa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 --2- 172.21.15.44:0/3657472277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca900b2c80 0x7fca90162930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 -- 172.21.15.44:0/3657472277 >> 172.21.15.44:0/3657472277 conn(0x7fca900cd740 msgr2=0x7fca90157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 -- 172.21.15.44:0/3657472277 shutdown_connections 2024-09-13T08:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.334+0000 7fca97400640 1 -- 172.21.15.44:0/3657472277 wait complete. 2024-09-13T08:50:40.358 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T08:50:40.358 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T08:50:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.514+0000 7fd142e00640 1 -- 172.21.15.44:0/3284198283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c0b4bd0 msgr2=0x7fd13c0b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.514+0000 7fd142e00640 1 --2- 172.21.15.44:0/3284198283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c0b4bd0 0x7fd13c0b2bf0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd128009900 tx=0x7fd12801c8e0 comp rx=0 tx=0).stop 2024-09-13T08:50:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.514+0000 7fd142e00640 1 -- 172.21.15.44:0/3284198283 shutdown_connections 2024-09-13T08:50:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.514+0000 7fd142e00640 1 --2- 172.21.15.44:0/3284198283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c0b4bd0 0x7fd13c0b2bf0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.514+0000 7fd142e00640 1 --2- 172.21.15.44:0/3284198283 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd13c15b580 0x7fd13c15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.514+0000 7fd142e00640 1 -- 172.21.15.44:0/3284198283 >> 172.21.15.44:0/3284198283 conn(0x7fd13c0b18f0 msgr2=0x7fd13c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.514+0000 7fd142e00640 1 -- 172.21.15.44:0/3284198283 shutdown_connections 2024-09-13T08:50:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.514+0000 7fd142e00640 1 -- 172.21.15.44:0/3284198283 wait complete. 2024-09-13T08:50:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.514+0000 7fd142e00640 1 Processor -- start 2024-09-13T08:50:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.514+0000 7fd142e00640 1 -- start start 2024-09-13T08:50:40.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd142e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13c0b4bd0 0x7fd13c154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd142e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c15b580 0x7fd13c1f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd142e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd13c15bea0 0x7fd13c141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd142e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c168000 con 0x7fd13c15b580 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd142e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c167e80 con 0x7fd13c0b4bd0 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd142e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd13c168180 con 0x7fd13c15bea0 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c15b580 0x7fd13c1f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c15b580 0x7fd13c1f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41426/0 (socket says 172.21.15.44:41426) 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13b400640 1 -- 172.21.15.44:0/3675044712 learned_addr learned my addr 172.21.15.44:0/3675044712 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd140e00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd13c15bea0 0x7fd13c141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13be00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13c0b4bd0 0x7fd13c154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13b400640 1 -- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd13c15bea0 msgr2=0x7fd13c141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13b400640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd13c15bea0 0x7fd13c141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13b400640 1 -- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13c0b4bd0 msgr2=0x7fd13c154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13b400640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13c0b4bd0 0x7fd13c154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13b400640 1 -- 172.21.15.44:0/3675044712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c1f62f0 con 0x7fd13c15b580 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd140e00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd13c15bea0 0x7fd13c141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13be00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13c0b4bd0 0x7fd13c154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd13b400640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c15b580 0x7fd13c1f5b80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fd128009850 tx=0x7fd1280a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd133e00640 1 -- 172.21.15.44:0/3675044712 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1280b2040 con 0x7fd13c15b580 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd133e00640 1 -- 172.21.15.44:0/3675044712 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1280a6d90 con 0x7fd13c15b580 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd133e00640 1 -- 172.21.15.44:0/3675044712 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128005280 con 0x7fd13c15b580 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd142e00640 1 -- 172.21.15.44:0/3675044712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c1f65d0 con 0x7fd13c15b580 2024-09-13T08:50:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd142e00640 1 -- 172.21.15.44:0/3675044712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c1524a0 con 0x7fd13c15b580 2024-09-13T08:50:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd142e00640 1 -- 172.21.15.44:0/3675044712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13c069410 con 0x7fd13c15b580 2024-09-13T08:50:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.518+0000 7fd133e00640 1 -- 172.21.15.44:0/3675044712 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fd128005420 con 0x7fd13c15b580 2024-09-13T08:50:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.522+0000 7fd133e00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd11c074930 0x7fd11c076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:40.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.522+0000 7fd133e00640 1 -- 172.21.15.44:0/3675044712 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fd12812c740 con 0x7fd13c15b580 2024-09-13T08:50:40.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.522+0000 7fd13be00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd11c074930 0x7fd11c076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:40.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.522+0000 7fd13be00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd11c074930 0x7fd11c076df0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005fd0 tx=0x7fd12c005e60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:40.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.526+0000 7fd133e00640 1 -- 172.21.15.44:0/3675044712 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd1280164d0 con 0x7fd13c15b580 2024-09-13T08:50:40.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:40.738+0000 7fd142e00640 1 -- 172.21.15.44:0/3675044712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd13c0b2bf0 con 0x7fd13c15b580 2024-09-13T08:50:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.338+0000 7fd133e00640 1 -- 172.21.15.44:0/3675044712 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) ==== 122+0+0 (secure 0 0 0) 0x7fd1280f93f0 con 0x7fd13c15b580 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 -- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd11c074930 msgr2=0x7fd11c076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd11c074930 0x7fd11c076df0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005fd0 tx=0x7fd12c005e60 comp rx=0 tx=0).stop 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 -- 172.21.15.44:0/3675044712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c15b580 msgr2=0x7fd13c1f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c15b580 0x7fd13c1f5b80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fd128009850 tx=0x7fd1280a55a0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 -- 172.21.15.44:0/3675044712 shutdown_connections 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd11c074930 0x7fd11c076df0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd13c15bea0 0x7fd13c141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c15b580 0x7fd13c1f5b80 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 --2- 172.21.15.44:0/3675044712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13c0b4bd0 0x7fd13c154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 -- 172.21.15.44:0/3675044712 >> 172.21.15.44:0/3675044712 conn(0x7fd13c0b18f0 msgr2=0x7fd13c1509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 -- 172.21.15.44:0/3675044712 shutdown_connections 2024-09-13T08:50:41.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.342+0000 7fd142e00640 1 -- 172.21.15.44:0/3675044712 wait complete. 2024-09-13T08:50:41.364 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T08:50:41.364 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:50:41.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 -- 172.21.15.44:0/3438632617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a80b4a50 msgr2=0x7f60a80ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:41.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 --2- 172.21.15.44:0/3438632617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a80b4a50 0x7f60a80ce810 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f60a8069950 tx=0x7f60a001ccf0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 -- 172.21.15.44:0/3438632617 shutdown_connections 2024-09-13T08:50:41.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 --2- 172.21.15.44:0/3438632617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a80b4a50 0x7f60a80ce810 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 --2- 172.21.15.44:0/3438632617 >> v2:172.21.15.129:3300/0 conn(0x7f60a815db50 0x7f60a80acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 -- 172.21.15.44:0/3438632617 >> 172.21.15.44:0/3438632617 conn(0x7f60a80b18f0 msgr2=0x7f60a80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:41.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 -- 172.21.15.44:0/3438632617 shutdown_connections 2024-09-13T08:50:41.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 -- 172.21.15.44:0/3438632617 wait complete. 2024-09-13T08:50:41.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 Processor -- start 2024-09-13T08:50:41.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 -- start start 2024-09-13T08:50:41.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f60a80b4a50 0x7f60a8148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:41.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60a80cd100 0x7f60a81f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:41.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a815db50 0x7f60a81fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:41.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8167120 con 0x7f60a815db50 2024-09-13T08:50:41.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f60a80cf5a0 con 0x7f60a80cd100 2024-09-13T08:50:41.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60ae600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f60a81672a0 con 0x7f60a80b4a50 2024-09-13T08:50:41.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a815db50 0x7f60a81fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:41.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a815db50 0x7f60a81fb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41440/0 (socket says 172.21.15.44:41440) 2024-09-13T08:50:41.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60a7e00640 1 -- 172.21.15.44:0/1651271646 learned_addr learned my addr 172.21.15.44:0/1651271646 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:41.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f60a7400640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f60a80b4a50 0x7f60a8148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:41.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.518+0000 7f609fe00640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60a80cd100 0x7f60a81f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:41.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f60a7e00640 1 -- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f60a80b4a50 msgr2=0x7f60a8148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:41.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f60a7e00640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f60a80b4a50 0x7f60a8148220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f60a7e00640 1 -- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60a80cd100 msgr2=0x7f60a81f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:41.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f60a7e00640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60a80cd100 0x7f60a81f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f60a7e00640 1 -- 172.21.15.44:0/1651271646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a80c4c00 con 0x7f60a815db50 2024-09-13T08:50:41.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f60a7e00640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a815db50 0x7f60a81fb2e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f60a001b3a0 tx=0x7f60a0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:41.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f609f400640 1 -- 172.21.15.44:0/1651271646 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00bb070 con 0x7f60a815db50 2024-09-13T08:50:41.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f609f400640 1 -- 172.21.15.44:0/1651271646 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60a0004e00 con 0x7f60a815db50 2024-09-13T08:50:41.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f609f400640 1 -- 172.21.15.44:0/1651271646 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00b2040 con 0x7f60a815db50 2024-09-13T08:50:41.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f60ae600640 1 -- 172.21.15.44:0/1651271646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a80c4ee0 con 0x7f60a815db50 2024-09-13T08:50:41.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f60ae600640 1 -- 172.21.15.44:0/1651271646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a80af450 con 0x7f60a815db50 2024-09-13T08:50:41.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f60ae600640 1 -- 172.21.15.44:0/1651271646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a8069410 con 0x7f60a815db50 2024-09-13T08:50:41.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.522+0000 7f609f400640 1 -- 172.21.15.44:0/1651271646 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f60a00a5b90 con 0x7f60a815db50 2024-09-13T08:50:41.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.526+0000 7f609f400640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6084074930 0x7f6084076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:41.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.526+0000 7f609f400640 1 -- 172.21.15.44:0/1651271646 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f60a012cb80 con 0x7f60a815db50 2024-09-13T08:50:41.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.526+0000 7f60a7400640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6084074930 0x7f6084076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:41.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.526+0000 7f60a7400640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6084074930 0x7f6084076df0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f60980058d0 tx=0x7f6098005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:41.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.526+0000 7f609f400640 1 -- 172.21.15.44:0/1651271646 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f60a0016970 con 0x7f60a815db50 2024-09-13T08:50:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.742+0000 7f60ae600640 1 -- 172.21.15.44:0/1651271646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60a80b16d0 con 0x7f60a815db50 2024-09-13T08:50:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.742+0000 7f609f400640 1 -- 172.21.15.44:0/1651271646 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5231 (secure 0 0 0) 0x7f60a00f9830 con 0x7f60a815db50 2024-09-13T08:50:41.741 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:41.741 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":74,"btime":"2024-09-13T08:50:41:334888+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6854,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3108776730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3108776730},{"type":"v1","addr":"172.21.15.129:6836","nonce":3108776730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1076017330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1076017330},{"type":"v1","addr":"172.21.15.129:6837","nonce":1076017330}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4098004813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4098004813},{"type":"v1","addr":"172.21.15.44:6835","nonce":4098004813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:50:37.721609+0000","modified":"2024-09-13T08:50:40.739478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/1590451084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1590451084},{"type":"v1","addr":"172.21.15.44:6834","nonce":1590451084}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":8}]} 2024-09-13T08:50:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 74 2024-09-13T08:50:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 -- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6084074930 msgr2=0x7f6084076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6084074930 0x7f6084076df0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f60980058d0 tx=0x7f6098005860 comp rx=0 tx=0).stop 2024-09-13T08:50:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 -- 172.21.15.44:0/1651271646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a815db50 msgr2=0x7f60a81fb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a815db50 0x7f60a81fb2e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f60a001b3a0 tx=0x7f60a0003f60 comp rx=0 tx=0).stop 2024-09-13T08:50:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 -- 172.21.15.44:0/1651271646 shutdown_connections 2024-09-13T08:50:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6084074930 0x7f6084076df0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a815db50 0x7f60a81fb2e0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60a80cd100 0x7f60a81f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 --2- 172.21.15.44:0/1651271646 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f60a80b4a50 0x7f60a8148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 -- 172.21.15.44:0/1651271646 >> 172.21.15.44:0/1651271646 conn(0x7f60a80b18f0 msgr2=0x7f60a815bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 -- 172.21.15.44:0/1651271646 shutdown_connections 2024-09-13T08:50:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.746+0000 7f60ae600640 1 -- 172.21.15.44:0/1651271646 wait complete. 2024-09-13T08:50:41.763 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:50:41.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.910+0000 7f0b1ac00640 1 -- 172.21.15.44:0/2712349825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af8004810 msgr2=0x7f0af8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:41.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.910+0000 7f0b1ac00640 1 --2- 172.21.15.44:0/2712349825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af8004810 0x7f0af8004c90 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009b90 tx=0x7f0afc01d2b0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.910+0000 7f0b18c00640 1 -- 172.21.15.44:0/2712349825 reap_dead start 2024-09-13T08:50:41.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.910+0000 7f0b1ac00640 1 -- 172.21.15.44:0/2712349825 shutdown_connections 2024-09-13T08:50:41.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.910+0000 7f0b1ac00640 1 -- 172.21.15.44:0/2712349825 >> 172.21.15.44:0/2712349825 conn(0x7f0b140d1dc0 msgr2=0x7f0b140d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:41.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.910+0000 7f0b1ac00640 1 -- 172.21.15.44:0/2712349825 shutdown_connections 2024-09-13T08:50:41.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.910+0000 7f0b1ac00640 1 -- 172.21.15.44:0/2712349825 wait complete. 2024-09-13T08:50:41.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b1ac00640 1 Processor -- start 2024-09-13T08:50:41.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b1ac00640 1 -- start start 2024-09-13T08:50:41.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b1ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af8004810 0x7f0b1406f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:41.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b1ac00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b140cb2c0 0x7f0b14154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:41.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b1ac00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b140cb740 0x7f0b14155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:41.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b1ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b140cfb30 con 0x7f0af8004810 2024-09-13T08:50:41.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b1ac00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1415c490 con 0x7f0b140cb2c0 2024-09-13T08:50:41.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b1ac00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0b140ce670 con 0x7f0b140cb740 2024-09-13T08:50:41.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af8004810 0x7f0b1406f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:41.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af8004810 0x7f0b1406f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41464/0 (socket says 172.21.15.44:41464) 2024-09-13T08:50:41.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b13e00640 1 -- 172.21.15.44:0/242898551 learned_addr learned my addr 172.21.15.44:0/242898551 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:41.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b13e00640 1 -- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b140cb740 msgr2=0x7f0b14155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:41.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b13e00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b140cb740 0x7f0b14155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b13e00640 1 -- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b140cb2c0 msgr2=0x7f0b14154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:41.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b13e00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b140cb2c0 0x7f0b14154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:41.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b13e00640 1 -- 172.21.15.44:0/242898551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b141f85b0 con 0x7f0af8004810 2024-09-13T08:50:41.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b13e00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af8004810 0x7f0b1406f690 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0b04011ae0 tx=0x7f0b04009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:41.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b10c00640 1 -- 172.21.15.44:0/242898551 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0401e070 con 0x7f0af8004810 2024-09-13T08:50:41.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b10c00640 1 -- 172.21.15.44:0/242898551 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b04004460 con 0x7f0af8004810 2024-09-13T08:50:41.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b10c00640 1 -- 172.21.15.44:0/242898551 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400fc10 con 0x7f0af8004810 2024-09-13T08:50:41.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b1ac00640 1 -- 172.21.15.44:0/242898551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b141f8890 con 0x7f0af8004810 2024-09-13T08:50:41.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.914+0000 7f0b1ac00640 1 -- 172.21.15.44:0/242898551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b141f8e50 con 0x7f0af8004810 2024-09-13T08:50:41.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.918+0000 7f0b1ac00640 1 -- 172.21.15.44:0/242898551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b140ceb90 con 0x7f0af8004810 2024-09-13T08:50:41.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.918+0000 7f0b10c00640 1 -- 172.21.15.44:0/242898551 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f0b04021070 con 0x7f0af8004810 2024-09-13T08:50:41.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.918+0000 7f0b10c00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0af0074930 0x7f0af0076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:41.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.918+0000 7f0b10c00640 1 -- 172.21.15.44:0/242898551 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f0b040935a0 con 0x7f0af8004810 2024-09-13T08:50:41.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.918+0000 7f0b0be00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0af0074930 0x7f0af0076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:41.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.918+0000 7f0b0be00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0af0074930 0x7f0af0076df0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0ae000 tx=0x7f0af8005840 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:41.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:41.922+0000 7f0b10c00640 1 -- 172.21.15.44:0/242898551 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0b0405fd60 con 0x7f0af8004810 2024-09-13T08:50:42.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.098+0000 7f0b1ac00640 1 -- 172.21.15.44:0/242898551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b1406d080 con 0x7f0af8004810 2024-09-13T08:50:42.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b10c00640 1 -- 172.21.15.44:0/242898551 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16932 (secure 0 0 0) 0x7f0b04064790 con 0x7f0af8004810 2024-09-13T08:50:42.100 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:42.100 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":116,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:50:39.298622+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:50:35.432294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-13T08:50:36.678893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:50:42.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 -- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0af0074930 msgr2=0x7f0af0076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0af0074930 0x7f0af0076df0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0ae000 tx=0x7f0af8005840 comp rx=0 tx=0).stop 2024-09-13T08:50:42.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 -- 172.21.15.44:0/242898551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af8004810 msgr2=0x7f0b1406f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af8004810 0x7f0b1406f690 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0b04011ae0 tx=0x7f0b04009ea0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 -- 172.21.15.44:0/242898551 shutdown_connections 2024-09-13T08:50:42.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0af0074930 0x7f0af0076df0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0b140cb740 0x7f0b14155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b140cb2c0 0x7f0b14154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 --2- 172.21.15.44:0/242898551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af8004810 0x7f0b1406f690 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 -- 172.21.15.44:0/242898551 >> 172.21.15.44:0/242898551 conn(0x7f0b140d1dc0 msgr2=0x7f0b140b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:42.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 -- 172.21.15.44:0/242898551 shutdown_connections 2024-09-13T08:50:42.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.102+0000 7f0b1ac00640 1 -- 172.21.15.44:0/242898551 wait complete. 2024-09-13T08:50:42.122 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:50:42.123 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:50:42.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.270+0000 7f0677e00640 1 -- 172.21.15.44:0/3623689858 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06780b11a0 msgr2=0x7f0678165670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.270+0000 7f0677e00640 1 --2- 172.21.15.44:0/3623689858 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06780b11a0 0x7f0678165670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0670009c80 tx=0x7f067001cda0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.270+0000 7f0677e00640 1 -- 172.21.15.44:0/3623689858 shutdown_connections 2024-09-13T08:50:42.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.270+0000 7f0677e00640 1 --2- 172.21.15.44:0/3623689858 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06780b11a0 0x7f0678165670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.270+0000 7f0677e00640 1 --2- 172.21.15.44:0/3623689858 >> v2:172.21.15.129:3300/0 conn(0x7f06780b0ce0 0x7f06780b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.270+0000 7f0677e00640 1 -- 172.21.15.44:0/3623689858 >> 172.21.15.44:0/3623689858 conn(0x7f067809aa10 msgr2=0x7f067809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:42.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.270+0000 7f0677e00640 1 -- 172.21.15.44:0/3623689858 shutdown_connections 2024-09-13T08:50:42.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.270+0000 7f0677e00640 1 -- 172.21.15.44:0/3623689858 wait complete. 2024-09-13T08:50:42.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 Processor -- start 2024-09-13T08:50:42.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 -- start start 2024-09-13T08:50:42.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06780b0ce0 0x7f06781f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:42.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06780b11a0 0x7f0678151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:42.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06780b4d80 0x7f0678157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:42.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06781689b0 con 0x7f06780b11a0 2024-09-13T08:50:42.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0678168830 con 0x7f06780b4d80 2024-09-13T08:50:42.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0678168b30 con 0x7f06780b0ce0 2024-09-13T08:50:42.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0676000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06780b11a0 0x7f0678151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:42.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0676000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06780b11a0 0x7f0678151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45798/0 (socket says 172.21.15.44:45798) 2024-09-13T08:50:42.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0676000640 1 -- 172.21.15.44:0/3578873667 learned_addr learned my addr 172.21.15.44:0/3578873667 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:42.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0676a00640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06780b0ce0 0x7f06781f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677400640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06780b4d80 0x7f0678157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0676000640 1 -- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06780b0ce0 msgr2=0x7f06781f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0676000640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06780b0ce0 0x7f06781f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0676000640 1 -- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06780b4d80 msgr2=0x7f0678157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0676000640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06780b4d80 0x7f0678157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0676000640 1 -- 172.21.15.44:0/3578873667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0678151ce0 con 0x7f06780b11a0 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0676000640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06780b11a0 0x7f0678151120 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f066400d980 tx=0x7f066400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f066ea00640 1 -- 172.21.15.44:0/3578873667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664017030 con 0x7f06780b11a0 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f066ea00640 1 -- 172.21.15.44:0/3578873667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0664009b20 con 0x7f06780b11a0 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 -- 172.21.15.44:0/3578873667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06780b2a50 con 0x7f06780b11a0 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f066ea00640 1 -- 172.21.15.44:0/3578873667 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664004020 con 0x7f06780b11a0 2024-09-13T08:50:42.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 -- 172.21.15.44:0/3578873667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06780b3010 con 0x7f06780b11a0 2024-09-13T08:50:42.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.274+0000 7f0677e00640 1 -- 172.21.15.44:0/3578873667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06780b5160 con 0x7f06780b11a0 2024-09-13T08:50:42.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.278+0000 7f066ea00640 1 -- 172.21.15.44:0/3578873667 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f066401a070 con 0x7f06780b11a0 2024-09-13T08:50:42.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.278+0000 7f066ea00640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658074930 0x7f0658076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:42.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.278+0000 7f066ea00640 1 -- 172.21.15.44:0/3578873667 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f0664092ee0 con 0x7f06780b11a0 2024-09-13T08:50:42.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.278+0000 7f0676a00640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658074930 0x7f0658076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:42.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.278+0000 7f0676a00640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658074930 0x7f0658076df0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0668007fd0 tx=0x7f0668009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.282+0000 7f066ea00640 1 -- 172.21.15.44:0/3578873667 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f066405f4c0 con 0x7f06780b11a0 2024-09-13T08:50:42.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.458+0000 7f0677e00640 1 -- 172.21.15.44:0/3578873667 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f067806fd20 con 0x7f0658074930 2024-09-13T08:50:42.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f066ea00640 1 -- 172.21.15.44:0/3578873667 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252478 (secure 0 0 0) 0x7f067806fd20 con 0x7f0658074930 2024-09-13T08:50:42.466 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:42.467 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:50:42.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f0677e00640 1 -- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658074930 msgr2=0x7f0658076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f0677e00640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658074930 0x7f0658076df0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0668007fd0 tx=0x7f0668009040 comp rx=0 tx=0).stop 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f0677e00640 1 -- 172.21.15.44:0/3578873667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06780b11a0 msgr2=0x7f0678151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f0677e00640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06780b11a0 0x7f0678151120 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f066400d980 tx=0x7f066400de40 comp rx=0 tx=0).stop 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f0677e00640 1 -- 172.21.15.44:0/3578873667 shutdown_connections 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f0677e00640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658074930 0x7f0658076df0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f0677e00640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06780b4d80 0x7f0678157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f0677e00640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06780b11a0 0x7f0678151120 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f0677e00640 1 --2- 172.21.15.44:0/3578873667 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f06780b0ce0 0x7f06781f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.466+0000 7f0677e00640 1 -- 172.21.15.44:0/3578873667 >> 172.21.15.44:0/3578873667 conn(0x7f067809aa10 msgr2=0x7f067809adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.470+0000 7f0677e00640 1 -- 172.21.15.44:0/3578873667 shutdown_connections 2024-09-13T08:50:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.470+0000 7f0677e00640 1 -- 172.21.15.44:0/3578873667 wait complete. 2024-09-13T08:50:42.486 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-13T08:50:40.792992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":999896,"kb_used_data":12316,"kb_used_omap":12,"kb_used_meta":987379,"kb_avail":753974824,"statfs":{"total":773094113280,"available":772070219776,"internally_reserved":0,"allocated":12611584,"data_stored":6156293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1011076657},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.501343"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.336631+0000","last_change":"2024-09-13T08:50:38.334028+0000","last_active":"2024-09-13T08:50:38.336631+0000","last_peered":"2024-09-13T08:50:38.336631+0000","last_clean":"2024-09-13T08:50:38.336631+0000","last_became_active":"2024-09-13T08:50:38.333740+0000","last_became_peered":"2024-09-13T08:50:38.333740+0000","last_unstale":"2024-09-13T08:50:38.336631+0000","last_undegraded":"2024-09-13T08:50:38.336631+0000","last_fullsized":"2024-09-13T08:50:38.336631+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:52.676771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303418+0000","last_change":"2024-09-13T08:50:38.316173+0000","last_active":"2024-09-13T08:50:39.303418+0000","last_peered":"2024-09-13T08:50:39.303418+0000","last_clean":"2024-09-13T08:50:39.303418+0000","last_became_active":"2024-09-13T08:50:38.313776+0000","last_became_peered":"2024-09-13T08:50:38.313776+0000","last_unstale":"2024-09-13T08:50:39.303418+0000","last_undegraded":"2024-09-13T08:50:39.303418+0000","last_fullsized":"2024-09-13T08:50:39.303418+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:06:44.650320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.328007+0000","last_change":"2024-09-13T08:50:38.323267+0000","last_active":"2024-09-13T08:50:38.328007+0000","last_peered":"2024-09-13T08:50:38.328007+0000","last_clean":"2024-09-13T08:50:38.328007+0000","last_became_active":"2024-09-13T08:50:38.321135+0000","last_became_peered":"2024-09-13T08:50:38.321135+0000","last_unstale":"2024-09-13T08:50:38.328007+0000","last_undegraded":"2024-09-13T08:50:38.328007+0000","last_fullsized":"2024-09-13T08:50:38.328007+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:40.486106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307004+0000","last_change":"2024-09-13T08:50:38.322244+0000","last_active":"2024-09-13T08:50:39.307004+0000","last_peered":"2024-09-13T08:50:39.307004+0000","last_clean":"2024-09-13T08:50:39.307004+0000","last_became_active":"2024-09-13T08:50:38.321968+0000","last_became_peered":"2024-09-13T08:50:38.321968+0000","last_unstale":"2024-09-13T08:50:39.307004+0000","last_undegraded":"2024-09-13T08:50:39.307004+0000","last_fullsized":"2024-09-13T08:50:39.307004+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:53.666147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.331161+0000","last_change":"2024-09-13T08:50:38.329370+0000","last_active":"2024-09-13T08:50:38.331161+0000","last_peered":"2024-09-13T08:50:38.331161+0000","last_clean":"2024-09-13T08:50:38.331161+0000","last_became_active":"2024-09-13T08:50:38.329086+0000","last_became_peered":"2024-09-13T08:50:38.329086+0000","last_unstale":"2024-09-13T08:50:38.331161+0000","last_undegraded":"2024-09-13T08:50:38.331161+0000","last_fullsized":"2024-09-13T08:50:38.331161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:04.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.305854+0000","last_change":"2024-09-13T08:50:38.309511+0000","last_active":"2024-09-13T08:50:39.305854+0000","last_peered":"2024-09-13T08:50:39.305854+0000","last_clean":"2024-09-13T08:50:39.305854+0000","last_became_active":"2024-09-13T08:50:38.309218+0000","last_became_peered":"2024-09-13T08:50:38.309218+0000","last_unstale":"2024-09-13T08:50:39.305854+0000","last_undegraded":"2024-09-13T08:50:39.305854+0000","last_fullsized":"2024-09-13T08:50:39.305854+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:56.872021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.335916+0000","last_change":"2024-09-13T08:50:38.334809+0000","last_active":"2024-09-13T08:50:38.335916+0000","last_peered":"2024-09-13T08:50:38.335916+0000","last_clean":"2024-09-13T08:50:38.335916+0000","last_became_active":"2024-09-13T08:50:38.334473+0000","last_became_peered":"2024-09-13T08:50:38.334473+0000","last_unstale":"2024-09-13T08:50:38.335916+0000","last_undegraded":"2024-09-13T08:50:38.335916+0000","last_fullsized":"2024-09-13T08:50:38.335916+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:01.134140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.322159+0000","last_change":"2024-09-13T08:50:38.321030+0000","last_active":"2024-09-13T08:50:38.322159+0000","last_peered":"2024-09-13T08:50:38.322159+0000","last_clean":"2024-09-13T08:50:38.322159+0000","last_became_active":"2024-09-13T08:50:38.320674+0000","last_became_peered":"2024-09-13T08:50:38.320674+0000","last_unstale":"2024-09-13T08:50:38.322159+0000","last_undegraded":"2024-09-13T08:50:38.322159+0000","last_fullsized":"2024-09-13T08:50:38.322159+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:18.789226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338558+0000","last_change":"2024-09-13T08:50:38.335980+0000","last_active":"2024-09-13T08:50:38.338558+0000","last_peered":"2024-09-13T08:50:38.338558+0000","last_clean":"2024-09-13T08:50:38.338558+0000","last_became_active":"2024-09-13T08:50:38.334985+0000","last_became_peered":"2024-09-13T08:50:38.334985+0000","last_unstale":"2024-09-13T08:50:38.338558+0000","last_undegraded":"2024-09-13T08:50:38.338558+0000","last_fullsized":"2024-09-13T08:50:38.338558+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:17:08.888187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.322473+0000","last_change":"2024-09-13T08:50:38.317440+0000","last_active":"2024-09-13T08:50:38.322473+0000","last_peered":"2024-09-13T08:50:38.322473+0000","last_clean":"2024-09-13T08:50:38.322473+0000","last_became_active":"2024-09-13T08:50:38.317184+0000","last_became_peered":"2024-09-13T08:50:38.317184+0000","last_unstale":"2024-09-13T08:50:38.322473+0000","last_undegraded":"2024-09-13T08:50:38.322473+0000","last_fullsized":"2024-09-13T08:50:38.322473+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:17:44.236802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338981+0000","last_change":"2024-09-13T08:50:38.337477+0000","last_active":"2024-09-13T08:50:38.338981+0000","last_peered":"2024-09-13T08:50:38.338981+0000","last_clean":"2024-09-13T08:50:38.338981+0000","last_became_active":"2024-09-13T08:50:38.337170+0000","last_became_peered":"2024-09-13T08:50:38.337170+0000","last_unstale":"2024-09-13T08:50:38.338981+0000","last_undegraded":"2024-09-13T08:50:38.338981+0000","last_fullsized":"2024-09-13T08:50:38.338981+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:00:37.477533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.330384+0000","last_change":"2024-09-13T08:50:38.325102+0000","last_active":"2024-09-13T08:50:38.330384+0000","last_peered":"2024-09-13T08:50:38.330384+0000","last_clean":"2024-09-13T08:50:38.330384+0000","last_became_active":"2024-09-13T08:50:38.324823+0000","last_became_peered":"2024-09-13T08:50:38.324823+0000","last_unstale":"2024-09-13T08:50:38.330384+0000","last_undegraded":"2024-09-13T08:50:38.330384+0000","last_fullsized":"2024-09-13T08:50:38.330384+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:53.169076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303414+0000","last_change":"2024-09-13T08:50:38.318664+0000","last_active":"2024-09-13T08:50:39.303414+0000","last_peered":"2024-09-13T08:50:39.303414+0000","last_clean":"2024-09-13T08:50:39.303414+0000","last_became_active":"2024-09-13T08:50:38.318366+0000","last_became_peered":"2024-09-13T08:50:38.318366+0000","last_unstale":"2024-09-13T08:50:39.303414+0000","last_undegraded":"2024-09-13T08:50:39.303414+0000","last_fullsized":"2024-09-13T08:50:39.303414+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:06.940407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.336493+0000","last_change":"2024-09-13T08:50:38.325782+0000","last_active":"2024-09-13T08:50:38.336493+0000","last_peered":"2024-09-13T08:50:38.336493+0000","last_clean":"2024-09-13T08:50:38.336493+0000","last_became_active":"2024-09-13T08:50:38.325389+0000","last_became_peered":"2024-09-13T08:50:38.325389+0000","last_unstale":"2024-09-13T08:50:38.336493+0000","last_undegraded":"2024-09-13T08:50:38.336493+0000","last_fullsized":"2024-09-13T08:50:38.336493+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:14.143320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.330767+0000","last_change":"2024-09-13T08:50:38.324173+0000","last_active":"2024-09-13T08:50:38.330767+0000","last_peered":"2024-09-13T08:50:38.330767+0000","last_clean":"2024-09-13T08:50:38.330767+0000","last_became_active":"2024-09-13T08:50:38.323881+0000","last_became_peered":"2024-09-13T08:50:38.323881+0000","last_unstale":"2024-09-13T08:50:38.330767+0000","last_undegraded":"2024-09-13T08:50:38.330767+0000","last_fullsized":"2024-09-13T08:50:38.330767+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:48:08.598625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.330070+0000","last_change":"2024-09-13T08:50:38.325056+0000","last_active":"2024-09-13T08:50:38.330070+0000","last_peered":"2024-09-13T08:50:38.330070+0000","last_clean":"2024-09-13T08:50:38.330070+0000","last_became_active":"2024-09-13T08:50:38.324754+0000","last_became_peered":"2024-09-13T08:50:38.324754+0000","last_unstale":"2024-09-13T08:50:38.330070+0000","last_undegraded":"2024-09-13T08:50:38.330070+0000","last_fullsized":"2024-09-13T08:50:38.330070+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:05.928958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.334597+0000","last_change":"2024-09-13T08:50:38.332537+0000","last_active":"2024-09-13T08:50:38.334597+0000","last_peered":"2024-09-13T08:50:38.334597+0000","last_clean":"2024-09-13T08:50:38.334597+0000","last_became_active":"2024-09-13T08:50:38.332265+0000","last_became_peered":"2024-09-13T08:50:38.332265+0000","last_unstale":"2024-09-13T08:50:38.334597+0000","last_undegraded":"2024-09-13T08:50:38.334597+0000","last_fullsized":"2024-09-13T08:50:38.334597+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:53.169076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.326386+0000","last_change":"2024-09-13T08:50:38.325185+0000","last_active":"2024-09-13T08:50:38.326386+0000","last_peered":"2024-09-13T08:50:38.326386+0000","last_clean":"2024-09-13T08:50:38.326386+0000","last_became_active":"2024-09-13T08:50:38.324843+0000","last_became_peered":"2024-09-13T08:50:38.324843+0000","last_unstale":"2024-09-13T08:50:38.326386+0000","last_undegraded":"2024-09-13T08:50:38.326386+0000","last_fullsized":"2024-09-13T08:50:38.326386+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:40.286583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.328035+0000","last_change":"2024-09-13T08:50:38.324361+0000","last_active":"2024-09-13T08:50:38.328035+0000","last_peered":"2024-09-13T08:50:38.328035+0000","last_clean":"2024-09-13T08:50:38.328035+0000","last_became_active":"2024-09-13T08:50:38.324038+0000","last_became_peered":"2024-09-13T08:50:38.324038+0000","last_unstale":"2024-09-13T08:50:38.328035+0000","last_undegraded":"2024-09-13T08:50:38.328035+0000","last_fullsized":"2024-09-13T08:50:38.328035+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:53:51.725566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.327933+0000","last_change":"2024-09-13T08:50:38.325011+0000","last_active":"2024-09-13T08:50:38.327933+0000","last_peered":"2024-09-13T08:50:38.327933+0000","last_clean":"2024-09-13T08:50:38.327933+0000","last_became_active":"2024-09-13T08:50:38.322278+0000","last_became_peered":"2024-09-13T08:50:38.322278+0000","last_unstale":"2024-09-13T08:50:38.327933+0000","last_undegraded":"2024-09-13T08:50:38.327933+0000","last_fullsized":"2024-09-13T08:50:38.327933+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:59.701322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.339715+0000","last_change":"2024-09-13T08:50:38.338541+0000","last_active":"2024-09-13T08:50:38.339715+0000","last_peered":"2024-09-13T08:50:38.339715+0000","last_clean":"2024-09-13T08:50:38.339715+0000","last_became_active":"2024-09-13T08:50:38.338253+0000","last_became_peered":"2024-09-13T08:50:38.338253+0000","last_unstale":"2024-09-13T08:50:38.339715+0000","last_undegraded":"2024-09-13T08:50:38.339715+0000","last_fullsized":"2024-09-13T08:50:38.339715+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:28:45.309065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306077+0000","last_change":"2024-09-13T08:50:38.317333+0000","last_active":"2024-09-13T08:50:39.306077+0000","last_peered":"2024-09-13T08:50:39.306077+0000","last_clean":"2024-09-13T08:50:39.306077+0000","last_became_active":"2024-09-13T08:50:38.317055+0000","last_became_peered":"2024-09-13T08:50:38.317055+0000","last_unstale":"2024-09-13T08:50:39.306077+0000","last_undegraded":"2024-09-13T08:50:39.306077+0000","last_fullsized":"2024-09-13T08:50:39.306077+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:04:08.686524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337859+0000","last_change":"2024-09-13T08:50:38.330994+0000","last_active":"2024-09-13T08:50:38.337859+0000","last_peered":"2024-09-13T08:50:38.337859+0000","last_clean":"2024-09-13T08:50:38.337859+0000","last_became_active":"2024-09-13T08:50:38.330719+0000","last_became_peered":"2024-09-13T08:50:38.330719+0000","last_unstale":"2024-09-13T08:50:38.337859+0000","last_undegraded":"2024-09-13T08:50:38.337859+0000","last_fullsized":"2024-09-13T08:50:38.337859+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:53.055415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306959+0000","last_change":"2024-09-13T08:50:38.322148+0000","last_active":"2024-09-13T08:50:39.306959+0000","last_peered":"2024-09-13T08:50:39.306959+0000","last_clean":"2024-09-13T08:50:39.306959+0000","last_became_active":"2024-09-13T08:50:38.321067+0000","last_became_peered":"2024-09-13T08:50:38.321067+0000","last_unstale":"2024-09-13T08:50:39.306959+0000","last_undegraded":"2024-09-13T08:50:39.306959+0000","last_fullsized":"2024-09-13T08:50:39.306959+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:14.586163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.324823+0000","last_change":"2024-09-13T08:50:38.317856+0000","last_active":"2024-09-13T08:50:38.324823+0000","last_peered":"2024-09-13T08:50:38.324823+0000","last_clean":"2024-09-13T08:50:38.324823+0000","last_became_active":"2024-09-13T08:50:38.317549+0000","last_became_peered":"2024-09-13T08:50:38.317549+0000","last_unstale":"2024-09-13T08:50:38.324823+0000","last_undegraded":"2024-09-13T08:50:38.324823+0000","last_fullsized":"2024-09-13T08:50:38.324823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:28:49.287669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306816+0000","last_change":"2024-09-13T08:50:38.315689+0000","last_active":"2024-09-13T08:50:39.306816+0000","last_peered":"2024-09-13T08:50:39.306816+0000","last_clean":"2024-09-13T08:50:39.306816+0000","last_became_active":"2024-09-13T08:50:38.315294+0000","last_became_peered":"2024-09-13T08:50:38.315294+0000","last_unstale":"2024-09-13T08:50:39.306816+0000","last_undegraded":"2024-09-13T08:50:39.306816+0000","last_fullsized":"2024-09-13T08:50:39.306816+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:42.048259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307259+0000","last_change":"2024-09-13T08:50:38.340073+0000","last_active":"2024-09-13T08:50:39.307259+0000","last_peered":"2024-09-13T08:50:39.307259+0000","last_clean":"2024-09-13T08:50:39.307259+0000","last_became_active":"2024-09-13T08:50:38.336726+0000","last_became_peered":"2024-09-13T08:50:38.336726+0000","last_unstale":"2024-09-13T08:50:39.307259+0000","last_undegraded":"2024-09-13T08:50:39.307259+0000","last_fullsized":"2024-09-13T08:50:39.307259+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:17.977219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306447+0000","last_change":"2024-09-13T08:50:38.311132+0000","last_active":"2024-09-13T08:50:39.306447+0000","last_peered":"2024-09-13T08:50:39.306447+0000","last_clean":"2024-09-13T08:50:39.306447+0000","last_became_active":"2024-09-13T08:50:38.310859+0000","last_became_peered":"2024-09-13T08:50:38.310859+0000","last_unstale":"2024-09-13T08:50:39.306447+0000","last_undegraded":"2024-09-13T08:50:39.306447+0000","last_fullsized":"2024-09-13T08:50:39.306447+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:36.355940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338051+0000","last_change":"2024-09-13T08:50:38.332611+0000","last_active":"2024-09-13T08:50:38.338051+0000","last_peered":"2024-09-13T08:50:38.338051+0000","last_clean":"2024-09-13T08:50:38.338051+0000","last_became_active":"2024-09-13T08:50:38.332349+0000","last_became_peered":"2024-09-13T08:50:38.332349+0000","last_unstale":"2024-09-13T08:50:38.338051+0000","last_undegraded":"2024-09-13T08:50:38.338051+0000","last_fullsized":"2024-09-13T08:50:38.338051+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.539115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.326516+0000","last_change":"2024-09-13T08:50:38.322123+0000","last_active":"2024-09-13T08:50:38.326516+0000","last_peered":"2024-09-13T08:50:38.326516+0000","last_clean":"2024-09-13T08:50:38.326516+0000","last_became_active":"2024-09-13T08:50:38.321781+0000","last_became_peered":"2024-09-13T08:50:38.321781+0000","last_unstale":"2024-09-13T08:50:38.326516+0000","last_undegraded":"2024-09-13T08:50:38.326516+0000","last_fullsized":"2024-09-13T08:50:38.326516+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:16:32.298192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.341272+0000","last_change":"2024-09-13T08:50:38.340302+0000","last_active":"2024-09-13T08:50:38.341272+0000","last_peered":"2024-09-13T08:50:38.341272+0000","last_clean":"2024-09-13T08:50:38.341272+0000","last_became_active":"2024-09-13T08:50:38.339943+0000","last_became_peered":"2024-09-13T08:50:38.339943+0000","last_unstale":"2024-09-13T08:50:38.341272+0000","last_undegraded":"2024-09-13T08:50:38.341272+0000","last_fullsized":"2024-09-13T08:50:38.341272+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:39.856764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.336587+0000","last_change":"2024-09-13T08:50:38.331844+0000","last_active":"2024-09-13T08:50:38.336587+0000","last_peered":"2024-09-13T08:50:38.336587+0000","last_clean":"2024-09-13T08:50:38.336587+0000","last_became_active":"2024-09-13T08:50:38.331551+0000","last_became_peered":"2024-09-13T08:50:38.331551+0000","last_unstale":"2024-09-13T08:50:38.336587+0000","last_undegraded":"2024-09-13T08:50:38.336587+0000","last_fullsized":"2024-09-13T08:50:38.336587+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:58.280641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.339946+0000","last_change":"2024-09-13T08:50:38.336289+0000","last_active":"2024-09-13T08:50:38.339946+0000","last_peered":"2024-09-13T08:50:38.339946+0000","last_clean":"2024-09-13T08:50:38.339946+0000","last_became_active":"2024-09-13T08:50:38.335973+0000","last_became_peered":"2024-09-13T08:50:38.335973+0000","last_unstale":"2024-09-13T08:50:38.339946+0000","last_undegraded":"2024-09-13T08:50:38.339946+0000","last_fullsized":"2024-09-13T08:50:38.339946+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:30.031056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"115'4","reported_seq":17,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.340130+0000","last_change":"2024-09-13T08:50:37.319445+0000","last_active":"2024-09-13T08:50:38.340130+0000","last_peered":"2024-09-13T08:50:38.340130+0000","last_clean":"2024-09-13T08:50:38.340130+0000","last_became_active":"2024-09-13T08:50:37.319150+0000","last_became_peered":"2024-09-13T08:50:37.319150+0000","last_unstale":"2024-09-13T08:50:38.340130+0000","last_undegraded":"2024-09-13T08:50:38.340130+0000","last_fullsized":"2024-09-13T08:50:38.340130+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:04.165354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.331403+0000","last_change":"2024-09-13T08:50:38.327199+0000","last_active":"2024-09-13T08:50:38.331403+0000","last_peered":"2024-09-13T08:50:38.331403+0000","last_clean":"2024-09-13T08:50:38.331403+0000","last_became_active":"2024-09-13T08:50:38.326908+0000","last_became_peered":"2024-09-13T08:50:38.326908+0000","last_unstale":"2024-09-13T08:50:38.331403+0000","last_undegraded":"2024-09-13T08:50:38.331403+0000","last_fullsized":"2024-09-13T08:50:38.331403+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:42.880926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303538+0000","last_change":"2024-09-13T08:50:37.315116+0000","last_active":"2024-09-13T08:50:39.303538+0000","last_peered":"2024-09-13T08:50:39.303538+0000","last_clean":"2024-09-13T08:50:39.303538+0000","last_became_active":"2024-09-13T08:50:37.314822+0000","last_became_peered":"2024-09-13T08:50:37.314822+0000","last_unstale":"2024-09-13T08:50:39.303538+0000","last_undegraded":"2024-09-13T08:50:39.303538+0000","last_fullsized":"2024-09-13T08:50:39.303538+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:39.912232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306687+0000","last_change":"2024-09-13T08:50:38.320450+0000","last_active":"2024-09-13T08:50:39.306687+0000","last_peered":"2024-09-13T08:50:39.306687+0000","last_clean":"2024-09-13T08:50:39.306687+0000","last_became_active":"2024-09-13T08:50:38.320197+0000","last_became_peered":"2024-09-13T08:50:38.320197+0000","last_unstale":"2024-09-13T08:50:39.306687+0000","last_undegraded":"2024-09-13T08:50:39.306687+0000","last_fullsized":"2024-09-13T08:50:39.306687+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:54.118418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307192+0000","last_change":"2024-09-13T08:50:37.325301+0000","last_active":"2024-09-13T08:50:39.307192+0000","last_peered":"2024-09-13T08:50:39.307192+0000","last_clean":"2024-09-13T08:50:39.307192+0000","last_became_active":"2024-09-13T08:50:37.322891+0000","last_became_peered":"2024-09-13T08:50:37.322891+0000","last_unstale":"2024-09-13T08:50:39.307192+0000","last_undegraded":"2024-09-13T08:50:39.307192+0000","last_fullsized":"2024-09-13T08:50:39.307192+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:54.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.328058+0000","last_change":"2024-09-13T08:50:38.325674+0000","last_active":"2024-09-13T08:50:38.328058+0000","last_peered":"2024-09-13T08:50:38.328058+0000","last_clean":"2024-09-13T08:50:38.328058+0000","last_became_active":"2024-09-13T08:50:38.325394+0000","last_became_peered":"2024-09-13T08:50:38.325394+0000","last_unstale":"2024-09-13T08:50:38.328058+0000","last_undegraded":"2024-09-13T08:50:38.328058+0000","last_fullsized":"2024-09-13T08:50:38.328058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:05.928958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306900+0000","last_change":"2024-09-13T08:50:37.322773+0000","last_active":"2024-09-13T08:50:39.306900+0000","last_peered":"2024-09-13T08:50:39.306900+0000","last_clean":"2024-09-13T08:50:39.306900+0000","last_became_active":"2024-09-13T08:50:37.322486+0000","last_became_peered":"2024-09-13T08:50:37.322486+0000","last_unstale":"2024-09-13T08:50:39.306900+0000","last_undegraded":"2024-09-13T08:50:39.306900+0000","last_fullsized":"2024-09-13T08:50:39.306900+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:06:20.382509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306042+0000","last_change":"2024-09-13T08:50:38.335349+0000","last_active":"2024-09-13T08:50:39.306042+0000","last_peered":"2024-09-13T08:50:39.306042+0000","last_clean":"2024-09-13T08:50:39.306042+0000","last_became_active":"2024-09-13T08:50:38.335075+0000","last_became_peered":"2024-09-13T08:50:38.335075+0000","last_unstale":"2024-09-13T08:50:39.306042+0000","last_undegraded":"2024-09-13T08:50:39.306042+0000","last_fullsized":"2024-09-13T08:50:39.306042+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:24:17.428198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"115'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.322785+0000","last_change":"2024-09-13T08:50:37.317886+0000","last_active":"2024-09-13T08:50:38.322785+0000","last_peered":"2024-09-13T08:50:38.322785+0000","last_clean":"2024-09-13T08:50:38.322785+0000","last_became_active":"2024-09-13T08:50:37.317476+0000","last_became_peered":"2024-09-13T08:50:37.317476+0000","last_unstale":"2024-09-13T08:50:38.322785+0000","last_undegraded":"2024-09-13T08:50:38.322785+0000","last_fullsized":"2024-09-13T08:50:38.322785+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:57.250836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.324686+0000","last_change":"2024-09-13T08:50:38.321557+0000","last_active":"2024-09-13T08:50:38.324686+0000","last_peered":"2024-09-13T08:50:38.324686+0000","last_clean":"2024-09-13T08:50:38.324686+0000","last_became_active":"2024-09-13T08:50:38.315841+0000","last_became_peered":"2024-09-13T08:50:38.315841+0000","last_unstale":"2024-09-13T08:50:38.324686+0000","last_undegraded":"2024-09-13T08:50:38.324686+0000","last_fullsized":"2024-09-13T08:50:38.324686+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:00.505183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"115'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338051+0000","last_change":"2024-09-13T08:50:37.316630+0000","last_active":"2024-09-13T08:50:38.338051+0000","last_peered":"2024-09-13T08:50:38.338051+0000","last_clean":"2024-09-13T08:50:38.338051+0000","last_became_active":"2024-09-13T08:50:37.316264+0000","last_became_peered":"2024-09-13T08:50:37.316264+0000","last_unstale":"2024-09-13T08:50:38.338051+0000","last_undegraded":"2024-09-13T08:50:38.338051+0000","last_fullsized":"2024-09-13T08:50:38.338051+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:53:20.702317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337570+0000","last_change":"2024-09-13T08:50:38.335589+0000","last_active":"2024-09-13T08:50:38.337570+0000","last_peered":"2024-09-13T08:50:38.337570+0000","last_clean":"2024-09-13T08:50:38.337570+0000","last_became_active":"2024-09-13T08:50:38.332260+0000","last_became_peered":"2024-09-13T08:50:38.332260+0000","last_unstale":"2024-09-13T08:50:38.337570+0000","last_undegraded":"2024-09-13T08:50:38.337570+0000","last_fullsized":"2024-09-13T08:50:38.337570+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:14.143320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.310426+0000","last_change":"2024-09-13T08:50:37.319791+0000","last_active":"2024-09-13T08:50:38.310426+0000","last_peered":"2024-09-13T08:50:38.310426+0000","last_clean":"2024-09-13T08:50:38.310426+0000","last_became_active":"2024-09-13T08:50:37.319298+0000","last_became_peered":"2024-09-13T08:50:37.319298+0000","last_unstale":"2024-09-13T08:50:38.310426+0000","last_undegraded":"2024-09-13T08:50:38.310426+0000","last_fullsized":"2024-09-13T08:50:38.310426+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:56:17.126526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.321253+0000","last_change":"2024-09-13T08:50:38.316759+0000","last_active":"2024-09-13T08:50:38.321253+0000","last_peered":"2024-09-13T08:50:38.321253+0000","last_clean":"2024-09-13T08:50:38.321253+0000","last_became_active":"2024-09-13T08:50:38.315442+0000","last_became_peered":"2024-09-13T08:50:38.315442+0000","last_unstale":"2024-09-13T08:50:38.321253+0000","last_undegraded":"2024-09-13T08:50:38.321253+0000","last_fullsized":"2024-09-13T08:50:38.321253+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:17.664286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.312500+0000","last_change":"2024-09-13T08:50:37.322876+0000","last_active":"2024-09-13T08:50:38.312500+0000","last_peered":"2024-09-13T08:50:38.312500+0000","last_clean":"2024-09-13T08:50:38.312500+0000","last_became_active":"2024-09-13T08:50:37.322617+0000","last_became_peered":"2024-09-13T08:50:37.322617+0000","last_unstale":"2024-09-13T08:50:38.312500+0000","last_undegraded":"2024-09-13T08:50:38.312500+0000","last_fullsized":"2024-09-13T08:50:38.312500+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:37.054021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303488+0000","last_change":"2024-09-13T08:50:38.326482+0000","last_active":"2024-09-13T08:50:39.303488+0000","last_peered":"2024-09-13T08:50:39.303488+0000","last_clean":"2024-09-13T08:50:39.303488+0000","last_became_active":"2024-09-13T08:50:38.326133+0000","last_became_peered":"2024-09-13T08:50:38.326133+0000","last_unstale":"2024-09-13T08:50:39.303488+0000","last_undegraded":"2024-09-13T08:50:39.303488+0000","last_fullsized":"2024-09-13T08:50:39.303488+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:25.779337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307284+0000","last_change":"2024-09-13T08:50:37.322055+0000","last_active":"2024-09-13T08:50:39.307284+0000","last_peered":"2024-09-13T08:50:39.307284+0000","last_clean":"2024-09-13T08:50:39.307284+0000","last_became_active":"2024-09-13T08:50:37.321782+0000","last_became_peered":"2024-09-13T08:50:37.321782+0000","last_unstale":"2024-09-13T08:50:39.307284+0000","last_undegraded":"2024-09-13T08:50:39.307284+0000","last_fullsized":"2024-09-13T08:50:39.307284+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:00:19.615104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303673+0000","last_change":"2024-09-13T08:50:38.320109+0000","last_active":"2024-09-13T08:50:39.303673+0000","last_peered":"2024-09-13T08:50:39.303673+0000","last_clean":"2024-09-13T08:50:39.303673+0000","last_became_active":"2024-09-13T08:50:38.319663+0000","last_became_peered":"2024-09-13T08:50:38.319663+0000","last_unstale":"2024-09-13T08:50:39.303673+0000","last_undegraded":"2024-09-13T08:50:39.303673+0000","last_fullsized":"2024-09-13T08:50:39.303673+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:17.490590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.312515+0000","last_change":"2024-09-13T08:50:37.326617+0000","last_active":"2024-09-13T08:50:38.312515+0000","last_peered":"2024-09-13T08:50:38.312515+0000","last_clean":"2024-09-13T08:50:38.312515+0000","last_became_active":"2024-09-13T08:50:37.326359+0000","last_became_peered":"2024-09-13T08:50:37.326359+0000","last_unstale":"2024-09-13T08:50:38.312515+0000","last_undegraded":"2024-09-13T08:50:38.312515+0000","last_fullsized":"2024-09-13T08:50:38.312515+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:39.483343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306909+0000","last_change":"2024-09-13T08:50:38.310299+0000","last_active":"2024-09-13T08:50:39.306909+0000","last_peered":"2024-09-13T08:50:39.306909+0000","last_clean":"2024-09-13T08:50:39.306909+0000","last_became_active":"2024-09-13T08:50:38.309985+0000","last_became_peered":"2024-09-13T08:50:38.309985+0000","last_unstale":"2024-09-13T08:50:39.306909+0000","last_undegraded":"2024-09-13T08:50:39.306909+0000","last_fullsized":"2024-09-13T08:50:39.306909+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:02.413353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"115'3","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338283+0000","last_change":"2024-09-13T08:50:37.327554+0000","last_active":"2024-09-13T08:50:38.338283+0000","last_peered":"2024-09-13T08:50:38.338283+0000","last_clean":"2024-09-13T08:50:38.338283+0000","last_became_active":"2024-09-13T08:50:37.327147+0000","last_became_peered":"2024-09-13T08:50:37.327147+0000","last_unstale":"2024-09-13T08:50:38.338283+0000","last_undegraded":"2024-09-13T08:50:38.338283+0000","last_fullsized":"2024-09-13T08:50:38.338283+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:26.512453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338956+0000","last_change":"2024-09-13T08:50:38.336814+0000","last_active":"2024-09-13T08:50:38.338956+0000","last_peered":"2024-09-13T08:50:38.338956+0000","last_clean":"2024-09-13T08:50:38.338956+0000","last_became_active":"2024-09-13T08:50:38.336526+0000","last_became_peered":"2024-09-13T08:50:38.336526+0000","last_unstale":"2024-09-13T08:50:38.338956+0000","last_undegraded":"2024-09-13T08:50:38.338956+0000","last_fullsized":"2024-09-13T08:50:38.338956+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:45:28.830478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"115'6","reported_seq":21,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307024+0000","last_change":"2024-09-13T08:50:37.323675+0000","last_active":"2024-09-13T08:50:39.307024+0000","last_peered":"2024-09-13T08:50:39.307024+0000","last_clean":"2024-09-13T08:50:39.307024+0000","last_became_active":"2024-09-13T08:50:37.323361+0000","last_became_peered":"2024-09-13T08:50:37.323361+0000","last_unstale":"2024-09-13T08:50:39.307024+0000","last_undegraded":"2024-09-13T08:50:39.307024+0000","last_fullsized":"2024-09-13T08:50:39.307024+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:26.311178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.329798+0000","last_change":"2024-09-13T08:50:38.322069+0000","last_active":"2024-09-13T08:50:38.329798+0000","last_peered":"2024-09-13T08:50:38.329798+0000","last_clean":"2024-09-13T08:50:38.329798+0000","last_became_active":"2024-09-13T08:50:38.321719+0000","last_became_peered":"2024-09-13T08:50:38.321719+0000","last_unstale":"2024-09-13T08:50:38.329798+0000","last_undegraded":"2024-09-13T08:50:38.329798+0000","last_fullsized":"2024-09-13T08:50:38.329798+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:53:51.725566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"115'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337892+0000","last_change":"2024-09-13T08:50:37.326630+0000","last_active":"2024-09-13T08:50:38.337892+0000","last_peered":"2024-09-13T08:50:38.337892+0000","last_clean":"2024-09-13T08:50:38.337892+0000","last_became_active":"2024-09-13T08:50:37.326372+0000","last_became_peered":"2024-09-13T08:50:37.326372+0000","last_unstale":"2024-09-13T08:50:38.337892+0000","last_undegraded":"2024-09-13T08:50:38.337892+0000","last_fullsized":"2024-09-13T08:50:38.337892+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:44.524564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306178+0000","last_change":"2024-09-13T08:50:38.332277+0000","last_active":"2024-09-13T08:50:39.306178+0000","last_peered":"2024-09-13T08:50:39.306178+0000","last_clean":"2024-09-13T08:50:39.306178+0000","last_became_active":"2024-09-13T08:50:38.331981+0000","last_became_peered":"2024-09-13T08:50:38.331981+0000","last_unstale":"2024-09-13T08:50:39.306178+0000","last_undegraded":"2024-09-13T08:50:39.306178+0000","last_fullsized":"2024-09-13T08:50:39.306178+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:49.901443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.310093+0000","last_change":"2024-09-13T08:50:37.317785+0000","last_active":"2024-09-13T08:50:38.310093+0000","last_peered":"2024-09-13T08:50:38.310093+0000","last_clean":"2024-09-13T08:50:38.310093+0000","last_became_active":"2024-09-13T08:50:37.317224+0000","last_became_peered":"2024-09-13T08:50:37.317224+0000","last_unstale":"2024-09-13T08:50:38.310093+0000","last_undegraded":"2024-09-13T08:50:38.310093+0000","last_fullsized":"2024-09-13T08:50:38.310093+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:17.140354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306812+0000","last_change":"2024-09-13T08:50:38.317099+0000","last_active":"2024-09-13T08:50:39.306812+0000","last_peered":"2024-09-13T08:50:39.306812+0000","last_clean":"2024-09-13T08:50:39.306812+0000","last_became_active":"2024-09-13T08:50:38.316854+0000","last_became_peered":"2024-09-13T08:50:38.316854+0000","last_unstale":"2024-09-13T08:50:39.306812+0000","last_undegraded":"2024-09-13T08:50:39.306812+0000","last_fullsized":"2024-09-13T08:50:39.306812+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:40.914994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"115'8","reported_seq":21,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.342488+0000","last_change":"2024-09-13T08:50:37.319716+0000","last_active":"2024-09-13T08:50:38.342488+0000","last_peered":"2024-09-13T08:50:38.342488+0000","last_clean":"2024-09-13T08:50:38.342488+0000","last_became_active":"2024-09-13T08:50:37.319434+0000","last_became_peered":"2024-09-13T08:50:37.319434+0000","last_unstale":"2024-09-13T08:50:38.342488+0000","last_undegraded":"2024-09-13T08:50:38.342488+0000","last_fullsized":"2024-09-13T08:50:38.342488+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:26.087733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.328510+0000","last_change":"2024-09-13T08:50:38.323770+0000","last_active":"2024-09-13T08:50:38.328510+0000","last_peered":"2024-09-13T08:50:38.328510+0000","last_clean":"2024-09-13T08:50:38.328510+0000","last_became_active":"2024-09-13T08:50:38.323308+0000","last_became_peered":"2024-09-13T08:50:38.323308+0000","last_unstale":"2024-09-13T08:50:38.328510+0000","last_undegraded":"2024-09-13T08:50:38.328510+0000","last_fullsized":"2024-09-13T08:50:38.328510+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:02.348561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306560+0000","last_change":"2024-09-13T08:50:37.308701+0000","last_active":"2024-09-13T08:50:39.306560+0000","last_peered":"2024-09-13T08:50:39.306560+0000","last_clean":"2024-09-13T08:50:39.306560+0000","last_became_active":"2024-09-13T08:50:37.308438+0000","last_became_peered":"2024-09-13T08:50:37.308438+0000","last_unstale":"2024-09-13T08:50:39.306560+0000","last_undegraded":"2024-09-13T08:50:39.306560+0000","last_fullsized":"2024-09-13T08:50:39.306560+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:58.560875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338535+0000","last_change":"2024-09-13T08:50:38.335691+0000","last_active":"2024-09-13T08:50:38.338535+0000","last_peered":"2024-09-13T08:50:38.338535+0000","last_clean":"2024-09-13T08:50:38.338535+0000","last_became_active":"2024-09-13T08:50:38.334670+0000","last_became_peered":"2024-09-13T08:50:38.334670+0000","last_unstale":"2024-09-13T08:50:38.338535+0000","last_undegraded":"2024-09-13T08:50:38.338535+0000","last_fullsized":"2024-09-13T08:50:38.338535+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:01:31.175248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"115'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337682+0000","last_change":"2024-09-13T08:50:37.328125+0000","last_active":"2024-09-13T08:50:38.337682+0000","last_peered":"2024-09-13T08:50:38.337682+0000","last_clean":"2024-09-13T08:50:38.337682+0000","last_became_active":"2024-09-13T08:50:37.327792+0000","last_became_peered":"2024-09-13T08:50:37.327792+0000","last_unstale":"2024-09-13T08:50:38.337682+0000","last_undegraded":"2024-09-13T08:50:38.337682+0000","last_fullsized":"2024-09-13T08:50:38.337682+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:57.250836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337389+0000","last_change":"2024-09-13T08:50:38.335289+0000","last_active":"2024-09-13T08:50:38.337389+0000","last_peered":"2024-09-13T08:50:38.337389+0000","last_clean":"2024-09-13T08:50:38.337389+0000","last_became_active":"2024-09-13T08:50:38.334666+0000","last_became_peered":"2024-09-13T08:50:38.334666+0000","last_unstale":"2024-09-13T08:50:38.337389+0000","last_undegraded":"2024-09-13T08:50:38.337389+0000","last_fullsized":"2024-09-13T08:50:38.337389+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:20.974465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"115'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337789+0000","last_change":"2024-09-13T08:50:37.318774+0000","last_active":"2024-09-13T08:50:38.337789+0000","last_peered":"2024-09-13T08:50:38.337789+0000","last_clean":"2024-09-13T08:50:38.337789+0000","last_became_active":"2024-09-13T08:50:37.318303+0000","last_became_peered":"2024-09-13T08:50:37.318303+0000","last_unstale":"2024-09-13T08:50:38.337789+0000","last_undegraded":"2024-09-13T08:50:38.337789+0000","last_fullsized":"2024-09-13T08:50:38.337789+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:57.277878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337715+0000","last_change":"2024-09-13T08:50:38.329617+0000","last_active":"2024-09-13T08:50:38.337715+0000","last_peered":"2024-09-13T08:50:38.337715+0000","last_clean":"2024-09-13T08:50:38.337715+0000","last_became_active":"2024-09-13T08:50:38.329322+0000","last_became_peered":"2024-09-13T08:50:38.329322+0000","last_unstale":"2024-09-13T08:50:38.337715+0000","last_undegraded":"2024-09-13T08:50:38.337715+0000","last_fullsized":"2024-09-13T08:50:38.337715+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:55:24.117213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.324188+0000","last_change":"2024-09-13T08:50:37.316491+0000","last_active":"2024-09-13T08:50:38.324188+0000","last_peered":"2024-09-13T08:50:38.324188+0000","last_clean":"2024-09-13T08:50:38.324188+0000","last_became_active":"2024-09-13T08:50:37.316205+0000","last_became_peered":"2024-09-13T08:50:37.316205+0000","last_unstale":"2024-09-13T08:50:38.324188+0000","last_undegraded":"2024-09-13T08:50:38.324188+0000","last_fullsized":"2024-09-13T08:50:38.324188+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:03.353011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.327763+0000","last_change":"2024-09-13T08:50:38.323493+0000","last_active":"2024-09-13T08:50:38.327763+0000","last_peered":"2024-09-13T08:50:38.327763+0000","last_clean":"2024-09-13T08:50:38.327763+0000","last_became_active":"2024-09-13T08:50:38.320918+0000","last_became_peered":"2024-09-13T08:50:38.320918+0000","last_unstale":"2024-09-13T08:50:38.327763+0000","last_undegraded":"2024-09-13T08:50:38.327763+0000","last_fullsized":"2024-09-13T08:50:38.327763+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:17:44.236802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"115'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.336257+0000","last_change":"2024-09-13T08:50:37.325999+0000","last_active":"2024-09-13T08:50:38.336257+0000","last_peered":"2024-09-13T08:50:38.336257+0000","last_clean":"2024-09-13T08:50:38.336257+0000","last_became_active":"2024-09-13T08:50:37.325727+0000","last_became_peered":"2024-09-13T08:50:37.325727+0000","last_unstale":"2024-09-13T08:50:38.336257+0000","last_undegraded":"2024-09-13T08:50:38.336257+0000","last_fullsized":"2024-09-13T08:50:38.336257+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:17.140354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.329519+0000","last_change":"2024-09-13T08:50:38.326178+0000","last_active":"2024-09-13T08:50:38.329519+0000","last_peered":"2024-09-13T08:50:38.329519+0000","last_clean":"2024-09-13T08:50:38.329519+0000","last_became_active":"2024-09-13T08:50:38.325902+0000","last_became_peered":"2024-09-13T08:50:38.325902+0000","last_unstale":"2024-09-13T08:50:38.329519+0000","last_undegraded":"2024-09-13T08:50:38.329519+0000","last_fullsized":"2024-09-13T08:50:38.329519+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:02.477067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.317867+0000","last_change":"2024-09-13T08:50:37.317783+0000","last_active":"2024-09-13T08:50:38.317867+0000","last_peered":"2024-09-13T08:50:38.317867+0000","last_clean":"2024-09-13T08:50:38.317867+0000","last_became_active":"2024-09-13T08:50:37.317501+0000","last_became_peered":"2024-09-13T08:50:37.317501+0000","last_unstale":"2024-09-13T08:50:38.317867+0000","last_undegraded":"2024-09-13T08:50:38.317867+0000","last_fullsized":"2024-09-13T08:50:38.317867+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:02.423257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338646+0000","last_change":"2024-09-13T08:50:38.333743+0000","last_active":"2024-09-13T08:50:38.338646+0000","last_peered":"2024-09-13T08:50:38.338646+0000","last_clean":"2024-09-13T08:50:38.338646+0000","last_became_active":"2024-09-13T08:50:38.331776+0000","last_became_peered":"2024-09-13T08:50:38.331776+0000","last_unstale":"2024-09-13T08:50:38.338646+0000","last_undegraded":"2024-09-13T08:50:38.338646+0000","last_fullsized":"2024-09-13T08:50:38.338646+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:27.647503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.305702+0000","last_change":"2024-09-13T08:50:37.312699+0000","last_active":"2024-09-13T08:50:39.305702+0000","last_peered":"2024-09-13T08:50:39.305702+0000","last_clean":"2024-09-13T08:50:39.305702+0000","last_became_active":"2024-09-13T08:50:37.311131+0000","last_became_peered":"2024-09-13T08:50:37.311131+0000","last_unstale":"2024-09-13T08:50:39.305702+0000","last_undegraded":"2024-09-13T08:50:39.305702+0000","last_fullsized":"2024-09-13T08:50:39.305702+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:34.755785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.329969+0000","last_change":"2024-09-13T08:50:38.322564+0000","last_active":"2024-09-13T08:50:38.329969+0000","last_peered":"2024-09-13T08:50:38.329969+0000","last_clean":"2024-09-13T08:50:38.329969+0000","last_became_active":"2024-09-13T08:50:38.322271+0000","last_became_peered":"2024-09-13T08:50:38.322271+0000","last_unstale":"2024-09-13T08:50:38.329969+0000","last_undegraded":"2024-09-13T08:50:38.329969+0000","last_fullsized":"2024-09-13T08:50:38.329969+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:59.701322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.305857+0000","last_change":"2024-09-13T08:50:37.324754+0000","last_active":"2024-09-13T08:50:39.305857+0000","last_peered":"2024-09-13T08:50:39.305857+0000","last_clean":"2024-09-13T08:50:39.305857+0000","last_became_active":"2024-09-13T08:50:37.324439+0000","last_became_peered":"2024-09-13T08:50:37.324439+0000","last_unstale":"2024-09-13T08:50:39.305857+0000","last_undegraded":"2024-09-13T08:50:39.305857+0000","last_fullsized":"2024-09-13T08:50:39.305857+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:25.366534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.336573+0000","last_change":"2024-09-13T08:50:38.329272+0000","last_active":"2024-09-13T08:50:38.336573+0000","last_peered":"2024-09-13T08:50:38.336573+0000","last_clean":"2024-09-13T08:50:38.336573+0000","last_became_active":"2024-09-13T08:50:38.328998+0000","last_became_peered":"2024-09-13T08:50:38.328998+0000","last_unstale":"2024-09-13T08:50:38.336573+0000","last_undegraded":"2024-09-13T08:50:38.336573+0000","last_fullsized":"2024-09-13T08:50:38.336573+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:00:20.617866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307351+0000","last_change":"2024-09-13T08:50:37.321939+0000","last_active":"2024-09-13T08:50:39.307351+0000","last_peered":"2024-09-13T08:50:39.307351+0000","last_clean":"2024-09-13T08:50:39.307351+0000","last_became_active":"2024-09-13T08:50:37.321652+0000","last_became_peered":"2024-09-13T08:50:37.321652+0000","last_unstale":"2024-09-13T08:50:39.307351+0000","last_undegraded":"2024-09-13T08:50:39.307351+0000","last_fullsized":"2024-09-13T08:50:39.307351+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:46:53.223486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306422+0000","last_change":"2024-09-13T08:50:38.324223+0000","last_active":"2024-09-13T08:50:39.306422+0000","last_peered":"2024-09-13T08:50:39.306422+0000","last_clean":"2024-09-13T08:50:39.306422+0000","last_became_active":"2024-09-13T08:50:38.323340+0000","last_became_peered":"2024-09-13T08:50:38.323340+0000","last_unstale":"2024-09-13T08:50:39.306422+0000","last_undegraded":"2024-09-13T08:50:39.306422+0000","last_fullsized":"2024-09-13T08:50:39.306422+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:50:34.481083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.310148+0000","last_change":"2024-09-13T08:50:37.316168+0000","last_active":"2024-09-13T08:50:38.310148+0000","last_peered":"2024-09-13T08:50:38.310148+0000","last_clean":"2024-09-13T08:50:38.310148+0000","last_became_active":"2024-09-13T08:50:37.314936+0000","last_became_peered":"2024-09-13T08:50:37.314936+0000","last_unstale":"2024-09-13T08:50:38.310148+0000","last_undegraded":"2024-09-13T08:50:38.310148+0000","last_fullsized":"2024-09-13T08:50:38.310148+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:55:03.174486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.329578+0000","last_change":"2024-09-13T08:50:38.327362+0000","last_active":"2024-09-13T08:50:38.329578+0000","last_peered":"2024-09-13T08:50:38.329578+0000","last_clean":"2024-09-13T08:50:38.329578+0000","last_became_active":"2024-09-13T08:50:38.325797+0000","last_became_peered":"2024-09-13T08:50:38.325797+0000","last_unstale":"2024-09-13T08:50:38.329578+0000","last_undegraded":"2024-09-13T08:50:38.329578+0000","last_fullsized":"2024-09-13T08:50:38.329578+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:56:34.257936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307289+0000","last_change":"2024-09-13T08:50:37.325729+0000","last_active":"2024-09-13T08:50:39.307289+0000","last_peered":"2024-09-13T08:50:39.307289+0000","last_clean":"2024-09-13T08:50:39.307289+0000","last_became_active":"2024-09-13T08:50:37.325418+0000","last_became_peered":"2024-09-13T08:50:37.325418+0000","last_unstale":"2024-09-13T08:50:39.307289+0000","last_undegraded":"2024-09-13T08:50:39.307289+0000","last_fullsized":"2024-09-13T08:50:39.307289+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:50.116112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.331852+0000","last_change":"2024-09-13T08:50:38.328815+0000","last_active":"2024-09-13T08:50:38.331852+0000","last_peered":"2024-09-13T08:50:38.331852+0000","last_clean":"2024-09-13T08:50:38.331852+0000","last_became_active":"2024-09-13T08:50:38.324616+0000","last_became_peered":"2024-09-13T08:50:38.324616+0000","last_unstale":"2024-09-13T08:50:38.331852+0000","last_undegraded":"2024-09-13T08:50:38.331852+0000","last_fullsized":"2024-09-13T08:50:38.331852+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:22:13.811984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307132+0000","last_change":"2024-09-13T08:50:37.308690+0000","last_active":"2024-09-13T08:50:39.307132+0000","last_peered":"2024-09-13T08:50:39.307132+0000","last_clean":"2024-09-13T08:50:39.307132+0000","last_became_active":"2024-09-13T08:50:37.308301+0000","last_became_peered":"2024-09-13T08:50:37.308301+0000","last_unstale":"2024-09-13T08:50:39.307132+0000","last_undegraded":"2024-09-13T08:50:39.307132+0000","last_fullsized":"2024-09-13T08:50:39.307132+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:22.809438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.320013+0000","last_change":"2024-09-13T08:50:38.313852+0000","last_active":"2024-09-13T08:50:38.320013+0000","last_peered":"2024-09-13T08:50:38.320013+0000","last_clean":"2024-09-13T08:50:38.320013+0000","last_became_active":"2024-09-13T08:50:38.313579+0000","last_became_peered":"2024-09-13T08:50:38.313579+0000","last_unstale":"2024-09-13T08:50:38.320013+0000","last_undegraded":"2024-09-13T08:50:38.320013+0000","last_fullsized":"2024-09-13T08:50:38.320013+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:38.056783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.311282+0000","last_change":"2024-09-13T08:50:37.317076+0000","last_active":"2024-09-13T08:50:38.311282+0000","last_peered":"2024-09-13T08:50:38.311282+0000","last_clean":"2024-09-13T08:50:38.311282+0000","last_became_active":"2024-09-13T08:50:37.316787+0000","last_became_peered":"2024-09-13T08:50:37.316787+0000","last_unstale":"2024-09-13T08:50:38.311282+0000","last_undegraded":"2024-09-13T08:50:38.311282+0000","last_fullsized":"2024-09-13T08:50:38.311282+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:59.805355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338021+0000","last_change":"2024-09-13T08:50:38.330146+0000","last_active":"2024-09-13T08:50:38.338021+0000","last_peered":"2024-09-13T08:50:38.338021+0000","last_clean":"2024-09-13T08:50:38.338021+0000","last_became_active":"2024-09-13T08:50:38.329943+0000","last_became_peered":"2024-09-13T08:50:38.329943+0000","last_unstale":"2024-09-13T08:50:38.338021+0000","last_undegraded":"2024-09-13T08:50:38.338021+0000","last_fullsized":"2024-09-13T08:50:38.338021+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:16.864797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.323389+0000","last_change":"2024-09-13T08:50:37.317813+0000","last_active":"2024-09-13T08:50:38.323389+0000","last_peered":"2024-09-13T08:50:38.323389+0000","last_clean":"2024-09-13T08:50:38.323389+0000","last_became_active":"2024-09-13T08:50:37.317303+0000","last_became_peered":"2024-09-13T08:50:37.317303+0000","last_unstale":"2024-09-13T08:50:38.323389+0000","last_undegraded":"2024-09-13T08:50:38.323389+0000","last_fullsized":"2024-09-13T08:50:38.323389+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:52.166314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.333579+0000","last_change":"2024-09-13T08:50:38.328528+0000","last_active":"2024-09-13T08:50:38.333579+0000","last_peered":"2024-09-13T08:50:38.333579+0000","last_clean":"2024-09-13T08:50:38.333579+0000","last_became_active":"2024-09-13T08:50:38.327185+0000","last_became_peered":"2024-09-13T08:50:38.327185+0000","last_unstale":"2024-09-13T08:50:38.333579+0000","last_undegraded":"2024-09-13T08:50:38.333579+0000","last_fullsized":"2024-09-13T08:50:38.333579+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:49:13.372489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.313671+0000","last_change":"2024-09-13T08:50:37.323741+0000","last_active":"2024-09-13T08:50:38.313671+0000","last_peered":"2024-09-13T08:50:38.313671+0000","last_clean":"2024-09-13T08:50:38.313671+0000","last_became_active":"2024-09-13T08:50:37.323415+0000","last_became_peered":"2024-09-13T08:50:37.323415+0000","last_unstale":"2024-09-13T08:50:38.313671+0000","last_undegraded":"2024-09-13T08:50:38.313671+0000","last_fullsized":"2024-09-13T08:50:38.313671+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:49.680947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.305977+0000","last_change":"2024-09-13T08:50:38.320648+0000","last_active":"2024-09-13T08:50:39.305977+0000","last_peered":"2024-09-13T08:50:39.305977+0000","last_clean":"2024-09-13T08:50:39.305977+0000","last_became_active":"2024-09-13T08:50:38.318904+0000","last_became_peered":"2024-09-13T08:50:38.318904+0000","last_unstale":"2024-09-13T08:50:39.305977+0000","last_undegraded":"2024-09-13T08:50:39.305977+0000","last_fullsized":"2024-09-13T08:50:39.305977+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:04.152832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306420+0000","last_change":"2024-09-13T08:50:37.307262+0000","last_active":"2024-09-13T08:50:39.306420+0000","last_peered":"2024-09-13T08:50:39.306420+0000","last_clean":"2024-09-13T08:50:39.306420+0000","last_became_active":"2024-09-13T08:50:37.306985+0000","last_became_peered":"2024-09-13T08:50:37.306985+0000","last_unstale":"2024-09-13T08:50:39.306420+0000","last_undegraded":"2024-09-13T08:50:39.306420+0000","last_fullsized":"2024-09-13T08:50:39.306420+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:01.050855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.305734+0000","last_change":"2024-09-13T08:50:38.322432+0000","last_active":"2024-09-13T08:50:39.305734+0000","last_peered":"2024-09-13T08:50:39.305734+0000","last_clean":"2024-09-13T08:50:39.305734+0000","last_became_active":"2024-09-13T08:50:38.321841+0000","last_became_peered":"2024-09-13T08:50:38.321841+0000","last_unstale":"2024-09-13T08:50:39.305734+0000","last_undegraded":"2024-09-13T08:50:39.305734+0000","last_fullsized":"2024-09-13T08:50:39.305734+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:46.287852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303090+0000","last_change":"2024-09-13T08:50:37.307307+0000","last_active":"2024-09-13T08:50:39.303090+0000","last_peered":"2024-09-13T08:50:39.303090+0000","last_clean":"2024-09-13T08:50:39.303090+0000","last_became_active":"2024-09-13T08:50:37.306979+0000","last_became_peered":"2024-09-13T08:50:37.306979+0000","last_unstale":"2024-09-13T08:50:39.303090+0000","last_undegraded":"2024-09-13T08:50:39.303090+0000","last_fullsized":"2024-09-13T08:50:39.303090+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:48.898680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":290,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.309742+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:50:38.309742+0000","last_peered":"2024-09-13T08:50:38.309742+0000","last_clean":"2024-09-13T08:50:38.309742+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:50:38.309742+0000","last_undegraded":"2024-09-13T08:50:38.309742+0000","last_fullsized":"2024-09-13T08:50:38.309742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509601,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82516,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289324,"statfs":{"total":96636764160,"available":96552267776,"internally_reserved":0,"allocated":1441792,"data_stored":623939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509601,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127504,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244336,"statfs":{"total":96636764160,"available":96506200064,"internally_reserved":0,"allocated":2023424,"data_stored":1212663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509602,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144976,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226864,"statfs":{"total":96636764160,"available":96488308736,"internally_reserved":0,"allocated":1437696,"data_stored":623905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509602,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126900,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244940,"statfs":{"total":96636764160,"available":96506818560,"internally_reserved":0,"allocated":1409024,"data_stored":615316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509601,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122772,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249068,"statfs":{"total":96636764160,"available":96511045632,"internally_reserved":0,"allocated":1433600,"data_stored":622841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509603,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141388,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230452,"statfs":{"total":96636764160,"available":96491982848,"internally_reserved":0,"allocated":2015232,"data_stored":1212571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509601,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104720,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267120,"statfs":{"total":96636764160,"available":96529530880,"internally_reserved":0,"allocated":1429504,"data_stored":622855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542307,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149120,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222720,"statfs":{"total":96636764160,"available":96484065280,"internally_reserved":0,"allocated":1421312,"data_stored":622203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:50:42.491 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:50:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.682+0000 7fe223400640 1 -- 172.21.15.44:0/176718400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c06f9a0 msgr2=0x7fe21c06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.682+0000 7fe223400640 1 --2- 172.21.15.44:0/176718400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c06f9a0 0x7fe21c06fe30 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe21c0d1b50 tx=0x7fe21401cb50 comp rx=0 tx=0).stop 2024-09-13T08:50:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.682+0000 7fe223400640 1 -- 172.21.15.44:0/176718400 shutdown_connections 2024-09-13T08:50:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.682+0000 7fe223400640 1 --2- 172.21.15.44:0/176718400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c06f9a0 0x7fe21c06fe30 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.682+0000 7fe223400640 1 --2- 172.21.15.44:0/176718400 >> v2:172.21.15.129:3300/0 conn(0x7fe21c0cffc0 0x7fe21c073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.682+0000 7fe223400640 1 -- 172.21.15.44:0/176718400 >> 172.21.15.44:0/176718400 conn(0x7fe21c0d0c90 msgr2=0x7fe21c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.682+0000 7fe223400640 1 -- 172.21.15.44:0/176718400 shutdown_connections 2024-09-13T08:50:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.682+0000 7fe223400640 1 -- 172.21.15.44:0/176718400 wait complete. 2024-09-13T08:50:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.682+0000 7fe223400640 1 Processor -- start 2024-09-13T08:50:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.682+0000 7fe223400640 1 -- start start 2024-09-13T08:50:42.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe223400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe21c06f9a0 0x7fe21c1f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:42.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe223400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c0cffc0 0x7fe21c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:42.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe223400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c15af60 0x7fe21c1f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe223400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c167cc0 con 0x7fe21c0cffc0 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe223400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c167b40 con 0x7fe21c15af60 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe223400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe21c167e40 con 0x7fe21c06f9a0 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe21be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c0cffc0 0x7fe21c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe21be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c0cffc0 0x7fe21c154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45816/0 (socket says 172.21.15.44:45816) 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe21be00640 1 -- 172.21.15.44:0/2128835437 learned_addr learned my addr 172.21.15.44:0/2128835437 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe220a00640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe21c06f9a0 0x7fe21c1f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe221400640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c15af60 0x7fe21c1f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe21be00640 1 -- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe21c06f9a0 msgr2=0x7fe21c1f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe21be00640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe21c06f9a0 0x7fe21c1f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe21be00640 1 -- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c15af60 msgr2=0x7fe21c1f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe21be00640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c15af60 0x7fe21c1f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe21be00640 1 -- 172.21.15.44:0/2128835437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21c1533e0 con 0x7fe21c0cffc0 2024-09-13T08:50:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe221400640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c15af60 0x7fe21c1f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe21be00640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c0cffc0 0x7fe21c154330 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe20800c970 tx=0x7fe20800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe218c00640 1 -- 172.21.15.44:0/2128835437 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208007c70 con 0x7fe21c0cffc0 2024-09-13T08:50:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe218c00640 1 -- 172.21.15.44:0/2128835437 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe20800f450 con 0x7fe21c0cffc0 2024-09-13T08:50:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe218c00640 1 -- 172.21.15.44:0/2128835437 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20800f8a0 con 0x7fe21c0cffc0 2024-09-13T08:50:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe223400640 1 -- 172.21.15.44:0/2128835437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21c154a30 con 0x7fe21c0cffc0 2024-09-13T08:50:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe223400640 1 -- 172.21.15.44:0/2128835437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe21c154ff0 con 0x7fe21c0cffc0 2024-09-13T08:50:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe223400640 1 -- 172.21.15.44:0/2128835437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe21c06fe30 con 0x7fe21c0cffc0 2024-09-13T08:50:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.686+0000 7fe218c00640 1 -- 172.21.15.44:0/2128835437 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fe20800fa40 con 0x7fe21c0cffc0 2024-09-13T08:50:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.690+0000 7fe218c00640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1fc074930 0x7fe1fc076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.690+0000 7fe218c00640 1 -- 172.21.15.44:0/2128835437 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fe2080934c0 con 0x7fe21c0cffc0 2024-09-13T08:50:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.690+0000 7fe220a00640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1fc074930 0x7fe1fc076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.690+0000 7fe220a00640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1fc074930 0x7fe1fc076df0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe210007fd0 tx=0x7fe210009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.694+0000 7fe218c00640 1 -- 172.21.15.44:0/2128835437 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe21c1533e0 con 0x7fe21c0cffc0 2024-09-13T08:50:42.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.862+0000 7fe223400640 1 -- 172.21.15.44:0/2128835437 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe21c069940 con 0x7fe1fc074930 2024-09-13T08:50:42.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.870+0000 7fe218c00640 1 -- 172.21.15.44:0/2128835437 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252478 (secure 0 0 0) 0x7fe21c069940 con 0x7fe1fc074930 2024-09-13T08:50:42.870 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:42.872 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:50:42.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 -- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1fc074930 msgr2=0x7fe1fc076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1fc074930 0x7fe1fc076df0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe210007fd0 tx=0x7fe210009040 comp rx=0 tx=0).stop 2024-09-13T08:50:42.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 -- 172.21.15.44:0/2128835437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c0cffc0 msgr2=0x7fe21c154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:42.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c0cffc0 0x7fe21c154330 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe20800c970 tx=0x7fe20800ce30 comp rx=0 tx=0).stop 2024-09-13T08:50:42.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 -- 172.21.15.44:0/2128835437 shutdown_connections 2024-09-13T08:50:42.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe1fc074930 0x7fe1fc076df0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c15af60 0x7fe21c1f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c0cffc0 0x7fe21c154330 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 --2- 172.21.15.44:0/2128835437 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe21c06f9a0 0x7fe21c1f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 -- 172.21.15.44:0/2128835437 >> 172.21.15.44:0/2128835437 conn(0x7fe21c0d0c90 msgr2=0x7fe21c070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 -- 172.21.15.44:0/2128835437 shutdown_connections 2024-09-13T08:50:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:42.874+0000 7fe223400640 1 -- 172.21.15.44:0/2128835437 wait complete. 2024-09-13T08:50:42.893 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-13T08:50:42.793537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":999896,"kb_used_data":12316,"kb_used_omap":12,"kb_used_meta":987379,"kb_avail":753974824,"statfs":{"total":773094113280,"available":772070219776,"internally_reserved":0,"allocated":12611584,"data_stored":6156293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1011076657},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001420"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.336631+0000","last_change":"2024-09-13T08:50:38.334028+0000","last_active":"2024-09-13T08:50:38.336631+0000","last_peered":"2024-09-13T08:50:38.336631+0000","last_clean":"2024-09-13T08:50:38.336631+0000","last_became_active":"2024-09-13T08:50:38.333740+0000","last_became_peered":"2024-09-13T08:50:38.333740+0000","last_unstale":"2024-09-13T08:50:38.336631+0000","last_undegraded":"2024-09-13T08:50:38.336631+0000","last_fullsized":"2024-09-13T08:50:38.336631+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:52.676771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303418+0000","last_change":"2024-09-13T08:50:38.316173+0000","last_active":"2024-09-13T08:50:39.303418+0000","last_peered":"2024-09-13T08:50:39.303418+0000","last_clean":"2024-09-13T08:50:39.303418+0000","last_became_active":"2024-09-13T08:50:38.313776+0000","last_became_peered":"2024-09-13T08:50:38.313776+0000","last_unstale":"2024-09-13T08:50:39.303418+0000","last_undegraded":"2024-09-13T08:50:39.303418+0000","last_fullsized":"2024-09-13T08:50:39.303418+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:06:44.650320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.328007+0000","last_change":"2024-09-13T08:50:38.323267+0000","last_active":"2024-09-13T08:50:38.328007+0000","last_peered":"2024-09-13T08:50:38.328007+0000","last_clean":"2024-09-13T08:50:38.328007+0000","last_became_active":"2024-09-13T08:50:38.321135+0000","last_became_peered":"2024-09-13T08:50:38.321135+0000","last_unstale":"2024-09-13T08:50:38.328007+0000","last_undegraded":"2024-09-13T08:50:38.328007+0000","last_fullsized":"2024-09-13T08:50:38.328007+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:40.486106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307004+0000","last_change":"2024-09-13T08:50:38.322244+0000","last_active":"2024-09-13T08:50:39.307004+0000","last_peered":"2024-09-13T08:50:39.307004+0000","last_clean":"2024-09-13T08:50:39.307004+0000","last_became_active":"2024-09-13T08:50:38.321968+0000","last_became_peered":"2024-09-13T08:50:38.321968+0000","last_unstale":"2024-09-13T08:50:39.307004+0000","last_undegraded":"2024-09-13T08:50:39.307004+0000","last_fullsized":"2024-09-13T08:50:39.307004+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:53.666147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.331161+0000","last_change":"2024-09-13T08:50:38.329370+0000","last_active":"2024-09-13T08:50:38.331161+0000","last_peered":"2024-09-13T08:50:38.331161+0000","last_clean":"2024-09-13T08:50:38.331161+0000","last_became_active":"2024-09-13T08:50:38.329086+0000","last_became_peered":"2024-09-13T08:50:38.329086+0000","last_unstale":"2024-09-13T08:50:38.331161+0000","last_undegraded":"2024-09-13T08:50:38.331161+0000","last_fullsized":"2024-09-13T08:50:38.331161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:04.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.305854+0000","last_change":"2024-09-13T08:50:38.309511+0000","last_active":"2024-09-13T08:50:39.305854+0000","last_peered":"2024-09-13T08:50:39.305854+0000","last_clean":"2024-09-13T08:50:39.305854+0000","last_became_active":"2024-09-13T08:50:38.309218+0000","last_became_peered":"2024-09-13T08:50:38.309218+0000","last_unstale":"2024-09-13T08:50:39.305854+0000","last_undegraded":"2024-09-13T08:50:39.305854+0000","last_fullsized":"2024-09-13T08:50:39.305854+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:56.872021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.335916+0000","last_change":"2024-09-13T08:50:38.334809+0000","last_active":"2024-09-13T08:50:38.335916+0000","last_peered":"2024-09-13T08:50:38.335916+0000","last_clean":"2024-09-13T08:50:38.335916+0000","last_became_active":"2024-09-13T08:50:38.334473+0000","last_became_peered":"2024-09-13T08:50:38.334473+0000","last_unstale":"2024-09-13T08:50:38.335916+0000","last_undegraded":"2024-09-13T08:50:38.335916+0000","last_fullsized":"2024-09-13T08:50:38.335916+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:01.134140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.322159+0000","last_change":"2024-09-13T08:50:38.321030+0000","last_active":"2024-09-13T08:50:38.322159+0000","last_peered":"2024-09-13T08:50:38.322159+0000","last_clean":"2024-09-13T08:50:38.322159+0000","last_became_active":"2024-09-13T08:50:38.320674+0000","last_became_peered":"2024-09-13T08:50:38.320674+0000","last_unstale":"2024-09-13T08:50:38.322159+0000","last_undegraded":"2024-09-13T08:50:38.322159+0000","last_fullsized":"2024-09-13T08:50:38.322159+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:18.789226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338558+0000","last_change":"2024-09-13T08:50:38.335980+0000","last_active":"2024-09-13T08:50:38.338558+0000","last_peered":"2024-09-13T08:50:38.338558+0000","last_clean":"2024-09-13T08:50:38.338558+0000","last_became_active":"2024-09-13T08:50:38.334985+0000","last_became_peered":"2024-09-13T08:50:38.334985+0000","last_unstale":"2024-09-13T08:50:38.338558+0000","last_undegraded":"2024-09-13T08:50:38.338558+0000","last_fullsized":"2024-09-13T08:50:38.338558+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:17:08.888187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.322473+0000","last_change":"2024-09-13T08:50:38.317440+0000","last_active":"2024-09-13T08:50:38.322473+0000","last_peered":"2024-09-13T08:50:38.322473+0000","last_clean":"2024-09-13T08:50:38.322473+0000","last_became_active":"2024-09-13T08:50:38.317184+0000","last_became_peered":"2024-09-13T08:50:38.317184+0000","last_unstale":"2024-09-13T08:50:38.322473+0000","last_undegraded":"2024-09-13T08:50:38.322473+0000","last_fullsized":"2024-09-13T08:50:38.322473+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:17:44.236802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338981+0000","last_change":"2024-09-13T08:50:38.337477+0000","last_active":"2024-09-13T08:50:38.338981+0000","last_peered":"2024-09-13T08:50:38.338981+0000","last_clean":"2024-09-13T08:50:38.338981+0000","last_became_active":"2024-09-13T08:50:38.337170+0000","last_became_peered":"2024-09-13T08:50:38.337170+0000","last_unstale":"2024-09-13T08:50:38.338981+0000","last_undegraded":"2024-09-13T08:50:38.338981+0000","last_fullsized":"2024-09-13T08:50:38.338981+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:00:37.477533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.330384+0000","last_change":"2024-09-13T08:50:38.325102+0000","last_active":"2024-09-13T08:50:38.330384+0000","last_peered":"2024-09-13T08:50:38.330384+0000","last_clean":"2024-09-13T08:50:38.330384+0000","last_became_active":"2024-09-13T08:50:38.324823+0000","last_became_peered":"2024-09-13T08:50:38.324823+0000","last_unstale":"2024-09-13T08:50:38.330384+0000","last_undegraded":"2024-09-13T08:50:38.330384+0000","last_fullsized":"2024-09-13T08:50:38.330384+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:53.169076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303414+0000","last_change":"2024-09-13T08:50:38.318664+0000","last_active":"2024-09-13T08:50:39.303414+0000","last_peered":"2024-09-13T08:50:39.303414+0000","last_clean":"2024-09-13T08:50:39.303414+0000","last_became_active":"2024-09-13T08:50:38.318366+0000","last_became_peered":"2024-09-13T08:50:38.318366+0000","last_unstale":"2024-09-13T08:50:39.303414+0000","last_undegraded":"2024-09-13T08:50:39.303414+0000","last_fullsized":"2024-09-13T08:50:39.303414+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:06.940407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.336493+0000","last_change":"2024-09-13T08:50:38.325782+0000","last_active":"2024-09-13T08:50:38.336493+0000","last_peered":"2024-09-13T08:50:38.336493+0000","last_clean":"2024-09-13T08:50:38.336493+0000","last_became_active":"2024-09-13T08:50:38.325389+0000","last_became_peered":"2024-09-13T08:50:38.325389+0000","last_unstale":"2024-09-13T08:50:38.336493+0000","last_undegraded":"2024-09-13T08:50:38.336493+0000","last_fullsized":"2024-09-13T08:50:38.336493+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:14.143320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.330767+0000","last_change":"2024-09-13T08:50:38.324173+0000","last_active":"2024-09-13T08:50:38.330767+0000","last_peered":"2024-09-13T08:50:38.330767+0000","last_clean":"2024-09-13T08:50:38.330767+0000","last_became_active":"2024-09-13T08:50:38.323881+0000","last_became_peered":"2024-09-13T08:50:38.323881+0000","last_unstale":"2024-09-13T08:50:38.330767+0000","last_undegraded":"2024-09-13T08:50:38.330767+0000","last_fullsized":"2024-09-13T08:50:38.330767+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:48:08.598625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.330070+0000","last_change":"2024-09-13T08:50:38.325056+0000","last_active":"2024-09-13T08:50:38.330070+0000","last_peered":"2024-09-13T08:50:38.330070+0000","last_clean":"2024-09-13T08:50:38.330070+0000","last_became_active":"2024-09-13T08:50:38.324754+0000","last_became_peered":"2024-09-13T08:50:38.324754+0000","last_unstale":"2024-09-13T08:50:38.330070+0000","last_undegraded":"2024-09-13T08:50:38.330070+0000","last_fullsized":"2024-09-13T08:50:38.330070+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:05.928958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.334597+0000","last_change":"2024-09-13T08:50:38.332537+0000","last_active":"2024-09-13T08:50:38.334597+0000","last_peered":"2024-09-13T08:50:38.334597+0000","last_clean":"2024-09-13T08:50:38.334597+0000","last_became_active":"2024-09-13T08:50:38.332265+0000","last_became_peered":"2024-09-13T08:50:38.332265+0000","last_unstale":"2024-09-13T08:50:38.334597+0000","last_undegraded":"2024-09-13T08:50:38.334597+0000","last_fullsized":"2024-09-13T08:50:38.334597+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:53.169076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.326386+0000","last_change":"2024-09-13T08:50:38.325185+0000","last_active":"2024-09-13T08:50:38.326386+0000","last_peered":"2024-09-13T08:50:38.326386+0000","last_clean":"2024-09-13T08:50:38.326386+0000","last_became_active":"2024-09-13T08:50:38.324843+0000","last_became_peered":"2024-09-13T08:50:38.324843+0000","last_unstale":"2024-09-13T08:50:38.326386+0000","last_undegraded":"2024-09-13T08:50:38.326386+0000","last_fullsized":"2024-09-13T08:50:38.326386+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:40.286583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.328035+0000","last_change":"2024-09-13T08:50:38.324361+0000","last_active":"2024-09-13T08:50:38.328035+0000","last_peered":"2024-09-13T08:50:38.328035+0000","last_clean":"2024-09-13T08:50:38.328035+0000","last_became_active":"2024-09-13T08:50:38.324038+0000","last_became_peered":"2024-09-13T08:50:38.324038+0000","last_unstale":"2024-09-13T08:50:38.328035+0000","last_undegraded":"2024-09-13T08:50:38.328035+0000","last_fullsized":"2024-09-13T08:50:38.328035+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:53:51.725566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.327933+0000","last_change":"2024-09-13T08:50:38.325011+0000","last_active":"2024-09-13T08:50:38.327933+0000","last_peered":"2024-09-13T08:50:38.327933+0000","last_clean":"2024-09-13T08:50:38.327933+0000","last_became_active":"2024-09-13T08:50:38.322278+0000","last_became_peered":"2024-09-13T08:50:38.322278+0000","last_unstale":"2024-09-13T08:50:38.327933+0000","last_undegraded":"2024-09-13T08:50:38.327933+0000","last_fullsized":"2024-09-13T08:50:38.327933+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:59.701322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.339715+0000","last_change":"2024-09-13T08:50:38.338541+0000","last_active":"2024-09-13T08:50:38.339715+0000","last_peered":"2024-09-13T08:50:38.339715+0000","last_clean":"2024-09-13T08:50:38.339715+0000","last_became_active":"2024-09-13T08:50:38.338253+0000","last_became_peered":"2024-09-13T08:50:38.338253+0000","last_unstale":"2024-09-13T08:50:38.339715+0000","last_undegraded":"2024-09-13T08:50:38.339715+0000","last_fullsized":"2024-09-13T08:50:38.339715+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:28:45.309065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306077+0000","last_change":"2024-09-13T08:50:38.317333+0000","last_active":"2024-09-13T08:50:39.306077+0000","last_peered":"2024-09-13T08:50:39.306077+0000","last_clean":"2024-09-13T08:50:39.306077+0000","last_became_active":"2024-09-13T08:50:38.317055+0000","last_became_peered":"2024-09-13T08:50:38.317055+0000","last_unstale":"2024-09-13T08:50:39.306077+0000","last_undegraded":"2024-09-13T08:50:39.306077+0000","last_fullsized":"2024-09-13T08:50:39.306077+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:04:08.686524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337859+0000","last_change":"2024-09-13T08:50:38.330994+0000","last_active":"2024-09-13T08:50:38.337859+0000","last_peered":"2024-09-13T08:50:38.337859+0000","last_clean":"2024-09-13T08:50:38.337859+0000","last_became_active":"2024-09-13T08:50:38.330719+0000","last_became_peered":"2024-09-13T08:50:38.330719+0000","last_unstale":"2024-09-13T08:50:38.337859+0000","last_undegraded":"2024-09-13T08:50:38.337859+0000","last_fullsized":"2024-09-13T08:50:38.337859+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:53.055415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306959+0000","last_change":"2024-09-13T08:50:38.322148+0000","last_active":"2024-09-13T08:50:39.306959+0000","last_peered":"2024-09-13T08:50:39.306959+0000","last_clean":"2024-09-13T08:50:39.306959+0000","last_became_active":"2024-09-13T08:50:38.321067+0000","last_became_peered":"2024-09-13T08:50:38.321067+0000","last_unstale":"2024-09-13T08:50:39.306959+0000","last_undegraded":"2024-09-13T08:50:39.306959+0000","last_fullsized":"2024-09-13T08:50:39.306959+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:14.586163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.324823+0000","last_change":"2024-09-13T08:50:38.317856+0000","last_active":"2024-09-13T08:50:38.324823+0000","last_peered":"2024-09-13T08:50:38.324823+0000","last_clean":"2024-09-13T08:50:38.324823+0000","last_became_active":"2024-09-13T08:50:38.317549+0000","last_became_peered":"2024-09-13T08:50:38.317549+0000","last_unstale":"2024-09-13T08:50:38.324823+0000","last_undegraded":"2024-09-13T08:50:38.324823+0000","last_fullsized":"2024-09-13T08:50:38.324823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:28:49.287669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306816+0000","last_change":"2024-09-13T08:50:38.315689+0000","last_active":"2024-09-13T08:50:39.306816+0000","last_peered":"2024-09-13T08:50:39.306816+0000","last_clean":"2024-09-13T08:50:39.306816+0000","last_became_active":"2024-09-13T08:50:38.315294+0000","last_became_peered":"2024-09-13T08:50:38.315294+0000","last_unstale":"2024-09-13T08:50:39.306816+0000","last_undegraded":"2024-09-13T08:50:39.306816+0000","last_fullsized":"2024-09-13T08:50:39.306816+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:42.048259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307259+0000","last_change":"2024-09-13T08:50:38.340073+0000","last_active":"2024-09-13T08:50:39.307259+0000","last_peered":"2024-09-13T08:50:39.307259+0000","last_clean":"2024-09-13T08:50:39.307259+0000","last_became_active":"2024-09-13T08:50:38.336726+0000","last_became_peered":"2024-09-13T08:50:38.336726+0000","last_unstale":"2024-09-13T08:50:39.307259+0000","last_undegraded":"2024-09-13T08:50:39.307259+0000","last_fullsized":"2024-09-13T08:50:39.307259+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:17.977219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306447+0000","last_change":"2024-09-13T08:50:38.311132+0000","last_active":"2024-09-13T08:50:39.306447+0000","last_peered":"2024-09-13T08:50:39.306447+0000","last_clean":"2024-09-13T08:50:39.306447+0000","last_became_active":"2024-09-13T08:50:38.310859+0000","last_became_peered":"2024-09-13T08:50:38.310859+0000","last_unstale":"2024-09-13T08:50:39.306447+0000","last_undegraded":"2024-09-13T08:50:39.306447+0000","last_fullsized":"2024-09-13T08:50:39.306447+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:36.355940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338051+0000","last_change":"2024-09-13T08:50:38.332611+0000","last_active":"2024-09-13T08:50:38.338051+0000","last_peered":"2024-09-13T08:50:38.338051+0000","last_clean":"2024-09-13T08:50:38.338051+0000","last_became_active":"2024-09-13T08:50:38.332349+0000","last_became_peered":"2024-09-13T08:50:38.332349+0000","last_unstale":"2024-09-13T08:50:38.338051+0000","last_undegraded":"2024-09-13T08:50:38.338051+0000","last_fullsized":"2024-09-13T08:50:38.338051+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.539115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.326516+0000","last_change":"2024-09-13T08:50:38.322123+0000","last_active":"2024-09-13T08:50:38.326516+0000","last_peered":"2024-09-13T08:50:38.326516+0000","last_clean":"2024-09-13T08:50:38.326516+0000","last_became_active":"2024-09-13T08:50:38.321781+0000","last_became_peered":"2024-09-13T08:50:38.321781+0000","last_unstale":"2024-09-13T08:50:38.326516+0000","last_undegraded":"2024-09-13T08:50:38.326516+0000","last_fullsized":"2024-09-13T08:50:38.326516+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:16:32.298192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.341272+0000","last_change":"2024-09-13T08:50:38.340302+0000","last_active":"2024-09-13T08:50:38.341272+0000","last_peered":"2024-09-13T08:50:38.341272+0000","last_clean":"2024-09-13T08:50:38.341272+0000","last_became_active":"2024-09-13T08:50:38.339943+0000","last_became_peered":"2024-09-13T08:50:38.339943+0000","last_unstale":"2024-09-13T08:50:38.341272+0000","last_undegraded":"2024-09-13T08:50:38.341272+0000","last_fullsized":"2024-09-13T08:50:38.341272+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:39.856764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.336587+0000","last_change":"2024-09-13T08:50:38.331844+0000","last_active":"2024-09-13T08:50:38.336587+0000","last_peered":"2024-09-13T08:50:38.336587+0000","last_clean":"2024-09-13T08:50:38.336587+0000","last_became_active":"2024-09-13T08:50:38.331551+0000","last_became_peered":"2024-09-13T08:50:38.331551+0000","last_unstale":"2024-09-13T08:50:38.336587+0000","last_undegraded":"2024-09-13T08:50:38.336587+0000","last_fullsized":"2024-09-13T08:50:38.336587+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:58.280641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.339946+0000","last_change":"2024-09-13T08:50:38.336289+0000","last_active":"2024-09-13T08:50:38.339946+0000","last_peered":"2024-09-13T08:50:38.339946+0000","last_clean":"2024-09-13T08:50:38.339946+0000","last_became_active":"2024-09-13T08:50:38.335973+0000","last_became_peered":"2024-09-13T08:50:38.335973+0000","last_unstale":"2024-09-13T08:50:38.339946+0000","last_undegraded":"2024-09-13T08:50:38.339946+0000","last_fullsized":"2024-09-13T08:50:38.339946+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:30.031056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"115'4","reported_seq":17,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.340130+0000","last_change":"2024-09-13T08:50:37.319445+0000","last_active":"2024-09-13T08:50:38.340130+0000","last_peered":"2024-09-13T08:50:38.340130+0000","last_clean":"2024-09-13T08:50:38.340130+0000","last_became_active":"2024-09-13T08:50:37.319150+0000","last_became_peered":"2024-09-13T08:50:37.319150+0000","last_unstale":"2024-09-13T08:50:38.340130+0000","last_undegraded":"2024-09-13T08:50:38.340130+0000","last_fullsized":"2024-09-13T08:50:38.340130+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:04.165354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.331403+0000","last_change":"2024-09-13T08:50:38.327199+0000","last_active":"2024-09-13T08:50:38.331403+0000","last_peered":"2024-09-13T08:50:38.331403+0000","last_clean":"2024-09-13T08:50:38.331403+0000","last_became_active":"2024-09-13T08:50:38.326908+0000","last_became_peered":"2024-09-13T08:50:38.326908+0000","last_unstale":"2024-09-13T08:50:38.331403+0000","last_undegraded":"2024-09-13T08:50:38.331403+0000","last_fullsized":"2024-09-13T08:50:38.331403+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:42.880926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303538+0000","last_change":"2024-09-13T08:50:37.315116+0000","last_active":"2024-09-13T08:50:39.303538+0000","last_peered":"2024-09-13T08:50:39.303538+0000","last_clean":"2024-09-13T08:50:39.303538+0000","last_became_active":"2024-09-13T08:50:37.314822+0000","last_became_peered":"2024-09-13T08:50:37.314822+0000","last_unstale":"2024-09-13T08:50:39.303538+0000","last_undegraded":"2024-09-13T08:50:39.303538+0000","last_fullsized":"2024-09-13T08:50:39.303538+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:39.912232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306687+0000","last_change":"2024-09-13T08:50:38.320450+0000","last_active":"2024-09-13T08:50:39.306687+0000","last_peered":"2024-09-13T08:50:39.306687+0000","last_clean":"2024-09-13T08:50:39.306687+0000","last_became_active":"2024-09-13T08:50:38.320197+0000","last_became_peered":"2024-09-13T08:50:38.320197+0000","last_unstale":"2024-09-13T08:50:39.306687+0000","last_undegraded":"2024-09-13T08:50:39.306687+0000","last_fullsized":"2024-09-13T08:50:39.306687+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:54.118418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307192+0000","last_change":"2024-09-13T08:50:37.325301+0000","last_active":"2024-09-13T08:50:39.307192+0000","last_peered":"2024-09-13T08:50:39.307192+0000","last_clean":"2024-09-13T08:50:39.307192+0000","last_became_active":"2024-09-13T08:50:37.322891+0000","last_became_peered":"2024-09-13T08:50:37.322891+0000","last_unstale":"2024-09-13T08:50:39.307192+0000","last_undegraded":"2024-09-13T08:50:39.307192+0000","last_fullsized":"2024-09-13T08:50:39.307192+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:54.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.328058+0000","last_change":"2024-09-13T08:50:38.325674+0000","last_active":"2024-09-13T08:50:38.328058+0000","last_peered":"2024-09-13T08:50:38.328058+0000","last_clean":"2024-09-13T08:50:38.328058+0000","last_became_active":"2024-09-13T08:50:38.325394+0000","last_became_peered":"2024-09-13T08:50:38.325394+0000","last_unstale":"2024-09-13T08:50:38.328058+0000","last_undegraded":"2024-09-13T08:50:38.328058+0000","last_fullsized":"2024-09-13T08:50:38.328058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:05.928958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306900+0000","last_change":"2024-09-13T08:50:37.322773+0000","last_active":"2024-09-13T08:50:39.306900+0000","last_peered":"2024-09-13T08:50:39.306900+0000","last_clean":"2024-09-13T08:50:39.306900+0000","last_became_active":"2024-09-13T08:50:37.322486+0000","last_became_peered":"2024-09-13T08:50:37.322486+0000","last_unstale":"2024-09-13T08:50:39.306900+0000","last_undegraded":"2024-09-13T08:50:39.306900+0000","last_fullsized":"2024-09-13T08:50:39.306900+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:06:20.382509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306042+0000","last_change":"2024-09-13T08:50:38.335349+0000","last_active":"2024-09-13T08:50:39.306042+0000","last_peered":"2024-09-13T08:50:39.306042+0000","last_clean":"2024-09-13T08:50:39.306042+0000","last_became_active":"2024-09-13T08:50:38.335075+0000","last_became_peered":"2024-09-13T08:50:38.335075+0000","last_unstale":"2024-09-13T08:50:39.306042+0000","last_undegraded":"2024-09-13T08:50:39.306042+0000","last_fullsized":"2024-09-13T08:50:39.306042+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:24:17.428198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"115'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.322785+0000","last_change":"2024-09-13T08:50:37.317886+0000","last_active":"2024-09-13T08:50:38.322785+0000","last_peered":"2024-09-13T08:50:38.322785+0000","last_clean":"2024-09-13T08:50:38.322785+0000","last_became_active":"2024-09-13T08:50:37.317476+0000","last_became_peered":"2024-09-13T08:50:37.317476+0000","last_unstale":"2024-09-13T08:50:38.322785+0000","last_undegraded":"2024-09-13T08:50:38.322785+0000","last_fullsized":"2024-09-13T08:50:38.322785+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:57.250836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.324686+0000","last_change":"2024-09-13T08:50:38.321557+0000","last_active":"2024-09-13T08:50:38.324686+0000","last_peered":"2024-09-13T08:50:38.324686+0000","last_clean":"2024-09-13T08:50:38.324686+0000","last_became_active":"2024-09-13T08:50:38.315841+0000","last_became_peered":"2024-09-13T08:50:38.315841+0000","last_unstale":"2024-09-13T08:50:38.324686+0000","last_undegraded":"2024-09-13T08:50:38.324686+0000","last_fullsized":"2024-09-13T08:50:38.324686+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:00.505183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"115'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338051+0000","last_change":"2024-09-13T08:50:37.316630+0000","last_active":"2024-09-13T08:50:38.338051+0000","last_peered":"2024-09-13T08:50:38.338051+0000","last_clean":"2024-09-13T08:50:38.338051+0000","last_became_active":"2024-09-13T08:50:37.316264+0000","last_became_peered":"2024-09-13T08:50:37.316264+0000","last_unstale":"2024-09-13T08:50:38.338051+0000","last_undegraded":"2024-09-13T08:50:38.338051+0000","last_fullsized":"2024-09-13T08:50:38.338051+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:53:20.702317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337570+0000","last_change":"2024-09-13T08:50:38.335589+0000","last_active":"2024-09-13T08:50:38.337570+0000","last_peered":"2024-09-13T08:50:38.337570+0000","last_clean":"2024-09-13T08:50:38.337570+0000","last_became_active":"2024-09-13T08:50:38.332260+0000","last_became_peered":"2024-09-13T08:50:38.332260+0000","last_unstale":"2024-09-13T08:50:38.337570+0000","last_undegraded":"2024-09-13T08:50:38.337570+0000","last_fullsized":"2024-09-13T08:50:38.337570+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:14.143320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.310426+0000","last_change":"2024-09-13T08:50:37.319791+0000","last_active":"2024-09-13T08:50:38.310426+0000","last_peered":"2024-09-13T08:50:38.310426+0000","last_clean":"2024-09-13T08:50:38.310426+0000","last_became_active":"2024-09-13T08:50:37.319298+0000","last_became_peered":"2024-09-13T08:50:37.319298+0000","last_unstale":"2024-09-13T08:50:38.310426+0000","last_undegraded":"2024-09-13T08:50:38.310426+0000","last_fullsized":"2024-09-13T08:50:38.310426+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:56:17.126526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.321253+0000","last_change":"2024-09-13T08:50:38.316759+0000","last_active":"2024-09-13T08:50:38.321253+0000","last_peered":"2024-09-13T08:50:38.321253+0000","last_clean":"2024-09-13T08:50:38.321253+0000","last_became_active":"2024-09-13T08:50:38.315442+0000","last_became_peered":"2024-09-13T08:50:38.315442+0000","last_unstale":"2024-09-13T08:50:38.321253+0000","last_undegraded":"2024-09-13T08:50:38.321253+0000","last_fullsized":"2024-09-13T08:50:38.321253+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:17.664286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.312500+0000","last_change":"2024-09-13T08:50:37.322876+0000","last_active":"2024-09-13T08:50:38.312500+0000","last_peered":"2024-09-13T08:50:38.312500+0000","last_clean":"2024-09-13T08:50:38.312500+0000","last_became_active":"2024-09-13T08:50:37.322617+0000","last_became_peered":"2024-09-13T08:50:37.322617+0000","last_unstale":"2024-09-13T08:50:38.312500+0000","last_undegraded":"2024-09-13T08:50:38.312500+0000","last_fullsized":"2024-09-13T08:50:38.312500+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:37.054021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303488+0000","last_change":"2024-09-13T08:50:38.326482+0000","last_active":"2024-09-13T08:50:39.303488+0000","last_peered":"2024-09-13T08:50:39.303488+0000","last_clean":"2024-09-13T08:50:39.303488+0000","last_became_active":"2024-09-13T08:50:38.326133+0000","last_became_peered":"2024-09-13T08:50:38.326133+0000","last_unstale":"2024-09-13T08:50:39.303488+0000","last_undegraded":"2024-09-13T08:50:39.303488+0000","last_fullsized":"2024-09-13T08:50:39.303488+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:25.779337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307284+0000","last_change":"2024-09-13T08:50:37.322055+0000","last_active":"2024-09-13T08:50:39.307284+0000","last_peered":"2024-09-13T08:50:39.307284+0000","last_clean":"2024-09-13T08:50:39.307284+0000","last_became_active":"2024-09-13T08:50:37.321782+0000","last_became_peered":"2024-09-13T08:50:37.321782+0000","last_unstale":"2024-09-13T08:50:39.307284+0000","last_undegraded":"2024-09-13T08:50:39.307284+0000","last_fullsized":"2024-09-13T08:50:39.307284+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:00:19.615104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303673+0000","last_change":"2024-09-13T08:50:38.320109+0000","last_active":"2024-09-13T08:50:39.303673+0000","last_peered":"2024-09-13T08:50:39.303673+0000","last_clean":"2024-09-13T08:50:39.303673+0000","last_became_active":"2024-09-13T08:50:38.319663+0000","last_became_peered":"2024-09-13T08:50:38.319663+0000","last_unstale":"2024-09-13T08:50:39.303673+0000","last_undegraded":"2024-09-13T08:50:39.303673+0000","last_fullsized":"2024-09-13T08:50:39.303673+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:17.490590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.312515+0000","last_change":"2024-09-13T08:50:37.326617+0000","last_active":"2024-09-13T08:50:38.312515+0000","last_peered":"2024-09-13T08:50:38.312515+0000","last_clean":"2024-09-13T08:50:38.312515+0000","last_became_active":"2024-09-13T08:50:37.326359+0000","last_became_peered":"2024-09-13T08:50:37.326359+0000","last_unstale":"2024-09-13T08:50:38.312515+0000","last_undegraded":"2024-09-13T08:50:38.312515+0000","last_fullsized":"2024-09-13T08:50:38.312515+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:39.483343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306909+0000","last_change":"2024-09-13T08:50:38.310299+0000","last_active":"2024-09-13T08:50:39.306909+0000","last_peered":"2024-09-13T08:50:39.306909+0000","last_clean":"2024-09-13T08:50:39.306909+0000","last_became_active":"2024-09-13T08:50:38.309985+0000","last_became_peered":"2024-09-13T08:50:38.309985+0000","last_unstale":"2024-09-13T08:50:39.306909+0000","last_undegraded":"2024-09-13T08:50:39.306909+0000","last_fullsized":"2024-09-13T08:50:39.306909+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:02.413353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"115'3","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338283+0000","last_change":"2024-09-13T08:50:37.327554+0000","last_active":"2024-09-13T08:50:38.338283+0000","last_peered":"2024-09-13T08:50:38.338283+0000","last_clean":"2024-09-13T08:50:38.338283+0000","last_became_active":"2024-09-13T08:50:37.327147+0000","last_became_peered":"2024-09-13T08:50:37.327147+0000","last_unstale":"2024-09-13T08:50:38.338283+0000","last_undegraded":"2024-09-13T08:50:38.338283+0000","last_fullsized":"2024-09-13T08:50:38.338283+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:26.512453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338956+0000","last_change":"2024-09-13T08:50:38.336814+0000","last_active":"2024-09-13T08:50:38.338956+0000","last_peered":"2024-09-13T08:50:38.338956+0000","last_clean":"2024-09-13T08:50:38.338956+0000","last_became_active":"2024-09-13T08:50:38.336526+0000","last_became_peered":"2024-09-13T08:50:38.336526+0000","last_unstale":"2024-09-13T08:50:38.338956+0000","last_undegraded":"2024-09-13T08:50:38.338956+0000","last_fullsized":"2024-09-13T08:50:38.338956+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:45:28.830478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"115'6","reported_seq":21,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307024+0000","last_change":"2024-09-13T08:50:37.323675+0000","last_active":"2024-09-13T08:50:39.307024+0000","last_peered":"2024-09-13T08:50:39.307024+0000","last_clean":"2024-09-13T08:50:39.307024+0000","last_became_active":"2024-09-13T08:50:37.323361+0000","last_became_peered":"2024-09-13T08:50:37.323361+0000","last_unstale":"2024-09-13T08:50:39.307024+0000","last_undegraded":"2024-09-13T08:50:39.307024+0000","last_fullsized":"2024-09-13T08:50:39.307024+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:26.311178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.329798+0000","last_change":"2024-09-13T08:50:38.322069+0000","last_active":"2024-09-13T08:50:38.329798+0000","last_peered":"2024-09-13T08:50:38.329798+0000","last_clean":"2024-09-13T08:50:38.329798+0000","last_became_active":"2024-09-13T08:50:38.321719+0000","last_became_peered":"2024-09-13T08:50:38.321719+0000","last_unstale":"2024-09-13T08:50:38.329798+0000","last_undegraded":"2024-09-13T08:50:38.329798+0000","last_fullsized":"2024-09-13T08:50:38.329798+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:53:51.725566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"115'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337892+0000","last_change":"2024-09-13T08:50:37.326630+0000","last_active":"2024-09-13T08:50:38.337892+0000","last_peered":"2024-09-13T08:50:38.337892+0000","last_clean":"2024-09-13T08:50:38.337892+0000","last_became_active":"2024-09-13T08:50:37.326372+0000","last_became_peered":"2024-09-13T08:50:37.326372+0000","last_unstale":"2024-09-13T08:50:38.337892+0000","last_undegraded":"2024-09-13T08:50:38.337892+0000","last_fullsized":"2024-09-13T08:50:38.337892+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:44.524564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306178+0000","last_change":"2024-09-13T08:50:38.332277+0000","last_active":"2024-09-13T08:50:39.306178+0000","last_peered":"2024-09-13T08:50:39.306178+0000","last_clean":"2024-09-13T08:50:39.306178+0000","last_became_active":"2024-09-13T08:50:38.331981+0000","last_became_peered":"2024-09-13T08:50:38.331981+0000","last_unstale":"2024-09-13T08:50:39.306178+0000","last_undegraded":"2024-09-13T08:50:39.306178+0000","last_fullsized":"2024-09-13T08:50:39.306178+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:49.901443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.310093+0000","last_change":"2024-09-13T08:50:37.317785+0000","last_active":"2024-09-13T08:50:38.310093+0000","last_peered":"2024-09-13T08:50:38.310093+0000","last_clean":"2024-09-13T08:50:38.310093+0000","last_became_active":"2024-09-13T08:50:37.317224+0000","last_became_peered":"2024-09-13T08:50:37.317224+0000","last_unstale":"2024-09-13T08:50:38.310093+0000","last_undegraded":"2024-09-13T08:50:38.310093+0000","last_fullsized":"2024-09-13T08:50:38.310093+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:17.140354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306812+0000","last_change":"2024-09-13T08:50:38.317099+0000","last_active":"2024-09-13T08:50:39.306812+0000","last_peered":"2024-09-13T08:50:39.306812+0000","last_clean":"2024-09-13T08:50:39.306812+0000","last_became_active":"2024-09-13T08:50:38.316854+0000","last_became_peered":"2024-09-13T08:50:38.316854+0000","last_unstale":"2024-09-13T08:50:39.306812+0000","last_undegraded":"2024-09-13T08:50:39.306812+0000","last_fullsized":"2024-09-13T08:50:39.306812+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:40.914994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"115'8","reported_seq":21,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.342488+0000","last_change":"2024-09-13T08:50:37.319716+0000","last_active":"2024-09-13T08:50:38.342488+0000","last_peered":"2024-09-13T08:50:38.342488+0000","last_clean":"2024-09-13T08:50:38.342488+0000","last_became_active":"2024-09-13T08:50:37.319434+0000","last_became_peered":"2024-09-13T08:50:37.319434+0000","last_unstale":"2024-09-13T08:50:38.342488+0000","last_undegraded":"2024-09-13T08:50:38.342488+0000","last_fullsized":"2024-09-13T08:50:38.342488+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:26.087733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.328510+0000","last_change":"2024-09-13T08:50:38.323770+0000","last_active":"2024-09-13T08:50:38.328510+0000","last_peered":"2024-09-13T08:50:38.328510+0000","last_clean":"2024-09-13T08:50:38.328510+0000","last_became_active":"2024-09-13T08:50:38.323308+0000","last_became_peered":"2024-09-13T08:50:38.323308+0000","last_unstale":"2024-09-13T08:50:38.328510+0000","last_undegraded":"2024-09-13T08:50:38.328510+0000","last_fullsized":"2024-09-13T08:50:38.328510+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:02.348561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306560+0000","last_change":"2024-09-13T08:50:37.308701+0000","last_active":"2024-09-13T08:50:39.306560+0000","last_peered":"2024-09-13T08:50:39.306560+0000","last_clean":"2024-09-13T08:50:39.306560+0000","last_became_active":"2024-09-13T08:50:37.308438+0000","last_became_peered":"2024-09-13T08:50:37.308438+0000","last_unstale":"2024-09-13T08:50:39.306560+0000","last_undegraded":"2024-09-13T08:50:39.306560+0000","last_fullsized":"2024-09-13T08:50:39.306560+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:58.560875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338535+0000","last_change":"2024-09-13T08:50:38.335691+0000","last_active":"2024-09-13T08:50:38.338535+0000","last_peered":"2024-09-13T08:50:38.338535+0000","last_clean":"2024-09-13T08:50:38.338535+0000","last_became_active":"2024-09-13T08:50:38.334670+0000","last_became_peered":"2024-09-13T08:50:38.334670+0000","last_unstale":"2024-09-13T08:50:38.338535+0000","last_undegraded":"2024-09-13T08:50:38.338535+0000","last_fullsized":"2024-09-13T08:50:38.338535+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:01:31.175248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"115'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337682+0000","last_change":"2024-09-13T08:50:37.328125+0000","last_active":"2024-09-13T08:50:38.337682+0000","last_peered":"2024-09-13T08:50:38.337682+0000","last_clean":"2024-09-13T08:50:38.337682+0000","last_became_active":"2024-09-13T08:50:37.327792+0000","last_became_peered":"2024-09-13T08:50:37.327792+0000","last_unstale":"2024-09-13T08:50:38.337682+0000","last_undegraded":"2024-09-13T08:50:38.337682+0000","last_fullsized":"2024-09-13T08:50:38.337682+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:57.250836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337389+0000","last_change":"2024-09-13T08:50:38.335289+0000","last_active":"2024-09-13T08:50:38.337389+0000","last_peered":"2024-09-13T08:50:38.337389+0000","last_clean":"2024-09-13T08:50:38.337389+0000","last_became_active":"2024-09-13T08:50:38.334666+0000","last_became_peered":"2024-09-13T08:50:38.334666+0000","last_unstale":"2024-09-13T08:50:38.337389+0000","last_undegraded":"2024-09-13T08:50:38.337389+0000","last_fullsized":"2024-09-13T08:50:38.337389+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:20.974465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"115'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337789+0000","last_change":"2024-09-13T08:50:37.318774+0000","last_active":"2024-09-13T08:50:38.337789+0000","last_peered":"2024-09-13T08:50:38.337789+0000","last_clean":"2024-09-13T08:50:38.337789+0000","last_became_active":"2024-09-13T08:50:37.318303+0000","last_became_peered":"2024-09-13T08:50:37.318303+0000","last_unstale":"2024-09-13T08:50:38.337789+0000","last_undegraded":"2024-09-13T08:50:38.337789+0000","last_fullsized":"2024-09-13T08:50:38.337789+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:57.277878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.337715+0000","last_change":"2024-09-13T08:50:38.329617+0000","last_active":"2024-09-13T08:50:38.337715+0000","last_peered":"2024-09-13T08:50:38.337715+0000","last_clean":"2024-09-13T08:50:38.337715+0000","last_became_active":"2024-09-13T08:50:38.329322+0000","last_became_peered":"2024-09-13T08:50:38.329322+0000","last_unstale":"2024-09-13T08:50:38.337715+0000","last_undegraded":"2024-09-13T08:50:38.337715+0000","last_fullsized":"2024-09-13T08:50:38.337715+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:55:24.117213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.324188+0000","last_change":"2024-09-13T08:50:37.316491+0000","last_active":"2024-09-13T08:50:38.324188+0000","last_peered":"2024-09-13T08:50:38.324188+0000","last_clean":"2024-09-13T08:50:38.324188+0000","last_became_active":"2024-09-13T08:50:37.316205+0000","last_became_peered":"2024-09-13T08:50:37.316205+0000","last_unstale":"2024-09-13T08:50:38.324188+0000","last_undegraded":"2024-09-13T08:50:38.324188+0000","last_fullsized":"2024-09-13T08:50:38.324188+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:03.353011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.327763+0000","last_change":"2024-09-13T08:50:38.323493+0000","last_active":"2024-09-13T08:50:38.327763+0000","last_peered":"2024-09-13T08:50:38.327763+0000","last_clean":"2024-09-13T08:50:38.327763+0000","last_became_active":"2024-09-13T08:50:38.320918+0000","last_became_peered":"2024-09-13T08:50:38.320918+0000","last_unstale":"2024-09-13T08:50:38.327763+0000","last_undegraded":"2024-09-13T08:50:38.327763+0000","last_fullsized":"2024-09-13T08:50:38.327763+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:17:44.236802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"115'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.336257+0000","last_change":"2024-09-13T08:50:37.325999+0000","last_active":"2024-09-13T08:50:38.336257+0000","last_peered":"2024-09-13T08:50:38.336257+0000","last_clean":"2024-09-13T08:50:38.336257+0000","last_became_active":"2024-09-13T08:50:37.325727+0000","last_became_peered":"2024-09-13T08:50:37.325727+0000","last_unstale":"2024-09-13T08:50:38.336257+0000","last_undegraded":"2024-09-13T08:50:38.336257+0000","last_fullsized":"2024-09-13T08:50:38.336257+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:17.140354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.329519+0000","last_change":"2024-09-13T08:50:38.326178+0000","last_active":"2024-09-13T08:50:38.329519+0000","last_peered":"2024-09-13T08:50:38.329519+0000","last_clean":"2024-09-13T08:50:38.329519+0000","last_became_active":"2024-09-13T08:50:38.325902+0000","last_became_peered":"2024-09-13T08:50:38.325902+0000","last_unstale":"2024-09-13T08:50:38.329519+0000","last_undegraded":"2024-09-13T08:50:38.329519+0000","last_fullsized":"2024-09-13T08:50:38.329519+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:02.477067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.317867+0000","last_change":"2024-09-13T08:50:37.317783+0000","last_active":"2024-09-13T08:50:38.317867+0000","last_peered":"2024-09-13T08:50:38.317867+0000","last_clean":"2024-09-13T08:50:38.317867+0000","last_became_active":"2024-09-13T08:50:37.317501+0000","last_became_peered":"2024-09-13T08:50:37.317501+0000","last_unstale":"2024-09-13T08:50:38.317867+0000","last_undegraded":"2024-09-13T08:50:38.317867+0000","last_fullsized":"2024-09-13T08:50:38.317867+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:02.423257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338646+0000","last_change":"2024-09-13T08:50:38.333743+0000","last_active":"2024-09-13T08:50:38.338646+0000","last_peered":"2024-09-13T08:50:38.338646+0000","last_clean":"2024-09-13T08:50:38.338646+0000","last_became_active":"2024-09-13T08:50:38.331776+0000","last_became_peered":"2024-09-13T08:50:38.331776+0000","last_unstale":"2024-09-13T08:50:38.338646+0000","last_undegraded":"2024-09-13T08:50:38.338646+0000","last_fullsized":"2024-09-13T08:50:38.338646+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:27.647503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.305702+0000","last_change":"2024-09-13T08:50:37.312699+0000","last_active":"2024-09-13T08:50:39.305702+0000","last_peered":"2024-09-13T08:50:39.305702+0000","last_clean":"2024-09-13T08:50:39.305702+0000","last_became_active":"2024-09-13T08:50:37.311131+0000","last_became_peered":"2024-09-13T08:50:37.311131+0000","last_unstale":"2024-09-13T08:50:39.305702+0000","last_undegraded":"2024-09-13T08:50:39.305702+0000","last_fullsized":"2024-09-13T08:50:39.305702+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:34.755785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.329969+0000","last_change":"2024-09-13T08:50:38.322564+0000","last_active":"2024-09-13T08:50:38.329969+0000","last_peered":"2024-09-13T08:50:38.329969+0000","last_clean":"2024-09-13T08:50:38.329969+0000","last_became_active":"2024-09-13T08:50:38.322271+0000","last_became_peered":"2024-09-13T08:50:38.322271+0000","last_unstale":"2024-09-13T08:50:38.329969+0000","last_undegraded":"2024-09-13T08:50:38.329969+0000","last_fullsized":"2024-09-13T08:50:38.329969+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:59.701322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.305857+0000","last_change":"2024-09-13T08:50:37.324754+0000","last_active":"2024-09-13T08:50:39.305857+0000","last_peered":"2024-09-13T08:50:39.305857+0000","last_clean":"2024-09-13T08:50:39.305857+0000","last_became_active":"2024-09-13T08:50:37.324439+0000","last_became_peered":"2024-09-13T08:50:37.324439+0000","last_unstale":"2024-09-13T08:50:39.305857+0000","last_undegraded":"2024-09-13T08:50:39.305857+0000","last_fullsized":"2024-09-13T08:50:39.305857+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:25.366534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.336573+0000","last_change":"2024-09-13T08:50:38.329272+0000","last_active":"2024-09-13T08:50:38.336573+0000","last_peered":"2024-09-13T08:50:38.336573+0000","last_clean":"2024-09-13T08:50:38.336573+0000","last_became_active":"2024-09-13T08:50:38.328998+0000","last_became_peered":"2024-09-13T08:50:38.328998+0000","last_unstale":"2024-09-13T08:50:38.336573+0000","last_undegraded":"2024-09-13T08:50:38.336573+0000","last_fullsized":"2024-09-13T08:50:38.336573+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:00:20.617866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307351+0000","last_change":"2024-09-13T08:50:37.321939+0000","last_active":"2024-09-13T08:50:39.307351+0000","last_peered":"2024-09-13T08:50:39.307351+0000","last_clean":"2024-09-13T08:50:39.307351+0000","last_became_active":"2024-09-13T08:50:37.321652+0000","last_became_peered":"2024-09-13T08:50:37.321652+0000","last_unstale":"2024-09-13T08:50:39.307351+0000","last_undegraded":"2024-09-13T08:50:39.307351+0000","last_fullsized":"2024-09-13T08:50:39.307351+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:46:53.223486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306422+0000","last_change":"2024-09-13T08:50:38.324223+0000","last_active":"2024-09-13T08:50:39.306422+0000","last_peered":"2024-09-13T08:50:39.306422+0000","last_clean":"2024-09-13T08:50:39.306422+0000","last_became_active":"2024-09-13T08:50:38.323340+0000","last_became_peered":"2024-09-13T08:50:38.323340+0000","last_unstale":"2024-09-13T08:50:39.306422+0000","last_undegraded":"2024-09-13T08:50:39.306422+0000","last_fullsized":"2024-09-13T08:50:39.306422+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:50:34.481083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.310148+0000","last_change":"2024-09-13T08:50:37.316168+0000","last_active":"2024-09-13T08:50:38.310148+0000","last_peered":"2024-09-13T08:50:38.310148+0000","last_clean":"2024-09-13T08:50:38.310148+0000","last_became_active":"2024-09-13T08:50:37.314936+0000","last_became_peered":"2024-09-13T08:50:37.314936+0000","last_unstale":"2024-09-13T08:50:38.310148+0000","last_undegraded":"2024-09-13T08:50:38.310148+0000","last_fullsized":"2024-09-13T08:50:38.310148+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:55:03.174486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.329578+0000","last_change":"2024-09-13T08:50:38.327362+0000","last_active":"2024-09-13T08:50:38.329578+0000","last_peered":"2024-09-13T08:50:38.329578+0000","last_clean":"2024-09-13T08:50:38.329578+0000","last_became_active":"2024-09-13T08:50:38.325797+0000","last_became_peered":"2024-09-13T08:50:38.325797+0000","last_unstale":"2024-09-13T08:50:38.329578+0000","last_undegraded":"2024-09-13T08:50:38.329578+0000","last_fullsized":"2024-09-13T08:50:38.329578+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:56:34.257936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307289+0000","last_change":"2024-09-13T08:50:37.325729+0000","last_active":"2024-09-13T08:50:39.307289+0000","last_peered":"2024-09-13T08:50:39.307289+0000","last_clean":"2024-09-13T08:50:39.307289+0000","last_became_active":"2024-09-13T08:50:37.325418+0000","last_became_peered":"2024-09-13T08:50:37.325418+0000","last_unstale":"2024-09-13T08:50:39.307289+0000","last_undegraded":"2024-09-13T08:50:39.307289+0000","last_fullsized":"2024-09-13T08:50:39.307289+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:50.116112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.331852+0000","last_change":"2024-09-13T08:50:38.328815+0000","last_active":"2024-09-13T08:50:38.331852+0000","last_peered":"2024-09-13T08:50:38.331852+0000","last_clean":"2024-09-13T08:50:38.331852+0000","last_became_active":"2024-09-13T08:50:38.324616+0000","last_became_peered":"2024-09-13T08:50:38.324616+0000","last_unstale":"2024-09-13T08:50:38.331852+0000","last_undegraded":"2024-09-13T08:50:38.331852+0000","last_fullsized":"2024-09-13T08:50:38.331852+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:22:13.811984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.307132+0000","last_change":"2024-09-13T08:50:37.308690+0000","last_active":"2024-09-13T08:50:39.307132+0000","last_peered":"2024-09-13T08:50:39.307132+0000","last_clean":"2024-09-13T08:50:39.307132+0000","last_became_active":"2024-09-13T08:50:37.308301+0000","last_became_peered":"2024-09-13T08:50:37.308301+0000","last_unstale":"2024-09-13T08:50:39.307132+0000","last_undegraded":"2024-09-13T08:50:39.307132+0000","last_fullsized":"2024-09-13T08:50:39.307132+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:22.809438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.320013+0000","last_change":"2024-09-13T08:50:38.313852+0000","last_active":"2024-09-13T08:50:38.320013+0000","last_peered":"2024-09-13T08:50:38.320013+0000","last_clean":"2024-09-13T08:50:38.320013+0000","last_became_active":"2024-09-13T08:50:38.313579+0000","last_became_peered":"2024-09-13T08:50:38.313579+0000","last_unstale":"2024-09-13T08:50:38.320013+0000","last_undegraded":"2024-09-13T08:50:38.320013+0000","last_fullsized":"2024-09-13T08:50:38.320013+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:38.056783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.311282+0000","last_change":"2024-09-13T08:50:37.317076+0000","last_active":"2024-09-13T08:50:38.311282+0000","last_peered":"2024-09-13T08:50:38.311282+0000","last_clean":"2024-09-13T08:50:38.311282+0000","last_became_active":"2024-09-13T08:50:37.316787+0000","last_became_peered":"2024-09-13T08:50:37.316787+0000","last_unstale":"2024-09-13T08:50:38.311282+0000","last_undegraded":"2024-09-13T08:50:38.311282+0000","last_fullsized":"2024-09-13T08:50:38.311282+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:59.805355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.338021+0000","last_change":"2024-09-13T08:50:38.330146+0000","last_active":"2024-09-13T08:50:38.338021+0000","last_peered":"2024-09-13T08:50:38.338021+0000","last_clean":"2024-09-13T08:50:38.338021+0000","last_became_active":"2024-09-13T08:50:38.329943+0000","last_became_peered":"2024-09-13T08:50:38.329943+0000","last_unstale":"2024-09-13T08:50:38.338021+0000","last_undegraded":"2024-09-13T08:50:38.338021+0000","last_fullsized":"2024-09-13T08:50:38.338021+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:16.864797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.323389+0000","last_change":"2024-09-13T08:50:37.317813+0000","last_active":"2024-09-13T08:50:38.323389+0000","last_peered":"2024-09-13T08:50:38.323389+0000","last_clean":"2024-09-13T08:50:38.323389+0000","last_became_active":"2024-09-13T08:50:37.317303+0000","last_became_peered":"2024-09-13T08:50:37.317303+0000","last_unstale":"2024-09-13T08:50:38.323389+0000","last_undegraded":"2024-09-13T08:50:38.323389+0000","last_fullsized":"2024-09-13T08:50:38.323389+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:52.166314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.333579+0000","last_change":"2024-09-13T08:50:38.328528+0000","last_active":"2024-09-13T08:50:38.333579+0000","last_peered":"2024-09-13T08:50:38.333579+0000","last_clean":"2024-09-13T08:50:38.333579+0000","last_became_active":"2024-09-13T08:50:38.327185+0000","last_became_peered":"2024-09-13T08:50:38.327185+0000","last_unstale":"2024-09-13T08:50:38.333579+0000","last_undegraded":"2024-09-13T08:50:38.333579+0000","last_fullsized":"2024-09-13T08:50:38.333579+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:49:13.372489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.313671+0000","last_change":"2024-09-13T08:50:37.323741+0000","last_active":"2024-09-13T08:50:38.313671+0000","last_peered":"2024-09-13T08:50:38.313671+0000","last_clean":"2024-09-13T08:50:38.313671+0000","last_became_active":"2024-09-13T08:50:37.323415+0000","last_became_peered":"2024-09-13T08:50:37.323415+0000","last_unstale":"2024-09-13T08:50:38.313671+0000","last_undegraded":"2024-09-13T08:50:38.313671+0000","last_fullsized":"2024-09-13T08:50:38.313671+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:49.680947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.305977+0000","last_change":"2024-09-13T08:50:38.320648+0000","last_active":"2024-09-13T08:50:39.305977+0000","last_peered":"2024-09-13T08:50:39.305977+0000","last_clean":"2024-09-13T08:50:39.305977+0000","last_became_active":"2024-09-13T08:50:38.318904+0000","last_became_peered":"2024-09-13T08:50:38.318904+0000","last_unstale":"2024-09-13T08:50:39.305977+0000","last_undegraded":"2024-09-13T08:50:39.305977+0000","last_fullsized":"2024-09-13T08:50:39.305977+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:56:04.152832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.306420+0000","last_change":"2024-09-13T08:50:37.307262+0000","last_active":"2024-09-13T08:50:39.306420+0000","last_peered":"2024-09-13T08:50:39.306420+0000","last_clean":"2024-09-13T08:50:39.306420+0000","last_became_active":"2024-09-13T08:50:37.306985+0000","last_became_peered":"2024-09-13T08:50:37.306985+0000","last_unstale":"2024-09-13T08:50:39.306420+0000","last_undegraded":"2024-09-13T08:50:39.306420+0000","last_fullsized":"2024-09-13T08:50:39.306420+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:01.050855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.305734+0000","last_change":"2024-09-13T08:50:38.322432+0000","last_active":"2024-09-13T08:50:39.305734+0000","last_peered":"2024-09-13T08:50:39.305734+0000","last_clean":"2024-09-13T08:50:39.305734+0000","last_became_active":"2024-09-13T08:50:38.321841+0000","last_became_peered":"2024-09-13T08:50:38.321841+0000","last_unstale":"2024-09-13T08:50:39.305734+0000","last_undegraded":"2024-09-13T08:50:39.305734+0000","last_fullsized":"2024-09-13T08:50:39.305734+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:37.289378+0000","last_clean_scrub_stamp":"2024-09-13T08:50:37.289378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:46.287852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T08:50:39.303090+0000","last_change":"2024-09-13T08:50:37.307307+0000","last_active":"2024-09-13T08:50:39.303090+0000","last_peered":"2024-09-13T08:50:39.303090+0000","last_clean":"2024-09-13T08:50:39.303090+0000","last_became_active":"2024-09-13T08:50:37.306979+0000","last_became_peered":"2024-09-13T08:50:37.306979+0000","last_unstale":"2024-09-13T08:50:39.303090+0000","last_undegraded":"2024-09-13T08:50:39.303090+0000","last_fullsized":"2024-09-13T08:50:39.303090+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:50:36.286616+0000","last_clean_scrub_stamp":"2024-09-13T08:50:36.286616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:48.898680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":290,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T08:50:38.309742+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:50:38.309742+0000","last_peered":"2024-09-13T08:50:38.309742+0000","last_clean":"2024-09-13T08:50:38.309742+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:50:38.309742+0000","last_undegraded":"2024-09-13T08:50:38.309742+0000","last_fullsized":"2024-09-13T08:50:38.309742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509601,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82516,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289324,"statfs":{"total":96636764160,"available":96552267776,"internally_reserved":0,"allocated":1441792,"data_stored":623939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509601,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127504,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244336,"statfs":{"total":96636764160,"available":96506200064,"internally_reserved":0,"allocated":2023424,"data_stored":1212663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509602,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144976,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226864,"statfs":{"total":96636764160,"available":96488308736,"internally_reserved":0,"allocated":1437696,"data_stored":623905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509602,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126900,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244940,"statfs":{"total":96636764160,"available":96506818560,"internally_reserved":0,"allocated":1409024,"data_stored":615316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509601,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122772,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249068,"statfs":{"total":96636764160,"available":96511045632,"internally_reserved":0,"allocated":1433600,"data_stored":622841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509603,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141388,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230452,"statfs":{"total":96636764160,"available":96491982848,"internally_reserved":0,"allocated":2015232,"data_stored":1212571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509601,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104720,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267120,"statfs":{"total":96636764160,"available":96529530880,"internally_reserved":0,"allocated":1429504,"data_stored":622855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542307,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149120,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222720,"statfs":{"total":96636764160,"available":96484065280,"internally_reserved":0,"allocated":1421312,"data_stored":622203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:50:42.897 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:50:42.897 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T08:50:43.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.090+0000 7f1520000640 1 -- 172.21.15.44:0/2694927930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1500003fe0 msgr2=0x7f1500004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:43.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.090+0000 7f1520000640 1 --2- 172.21.15.44:0/2694927930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1500003fe0 0x7f1500004810 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f150400c290 tx=0x7f150401f6c0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.090+0000 7f151e000640 1 -- 172.21.15.44:0/2694927930 reap_dead start 2024-09-13T08:50:43.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.090+0000 7f1520000640 1 -- 172.21.15.44:0/2694927930 shutdown_connections 2024-09-13T08:50:43.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.090+0000 7f1520000640 1 -- 172.21.15.44:0/2694927930 >> 172.21.15.44:0/2694927930 conn(0x7f15180b18f0 msgr2=0x7f15180b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:43.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.090+0000 7f1520000640 1 -- 172.21.15.44:0/2694927930 shutdown_connections 2024-09-13T08:50:43.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.090+0000 7f1520000640 1 -- 172.21.15.44:0/2694927930 wait complete. 2024-09-13T08:50:43.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.090+0000 7f1520000640 1 Processor -- start 2024-09-13T08:50:43.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.090+0000 7f1520000640 1 -- start start 2024-09-13T08:50:43.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1520000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1500003fe0 0x7f151814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1520000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f151809d350 0x7f151809d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1520000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15180b0320 0x7f15180b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1520000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f151815bd20 con 0x7f15180b0320 2024-09-13T08:50:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1520000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f151815bba0 con 0x7f151809d350 2024-09-13T08:50:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1520000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f151815bea0 con 0x7f1500003fe0 2024-09-13T08:50:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15180b0320 0x7f15180b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15180b0320 0x7f15180b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45840/0 (socket says 172.21.15.44:45840) 2024-09-13T08:50:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151e000640 1 -- 172.21.15.44:0/2709199746 learned_addr learned my addr 172.21.15.44:0/2709199746 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151d600640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1500003fe0 0x7f151814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151cc00640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f151809d350 0x7f151809d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:43.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151e000640 1 -- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1500003fe0 msgr2=0x7f151814b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:43.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151e000640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1500003fe0 0x7f151814b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151e000640 1 -- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f151809d350 msgr2=0x7f151809d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:43.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151e000640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f151809d350 0x7f151809d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151e000640 1 -- 172.21.15.44:0/2709199746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15181f4920 con 0x7f15180b0320 2024-09-13T08:50:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f151e000640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15180b0320 0x7f15180b07a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f15180cf880 tx=0x7f1510010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1515600640 1 -- 172.21.15.44:0/2709199746 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151000ee10 con 0x7f15180b0320 2024-09-13T08:50:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1515600640 1 -- 172.21.15.44:0/2709199746 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1510015070 con 0x7f15180b0320 2024-09-13T08:50:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1515600640 1 -- 172.21.15.44:0/2709199746 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510003d10 con 0x7f15180b0320 2024-09-13T08:50:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1520000640 1 -- 172.21.15.44:0/2709199746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15180acd40 con 0x7f15180b0320 2024-09-13T08:50:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1520000640 1 -- 172.21.15.44:0/2709199746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15181f4bc0 con 0x7f15180b0320 2024-09-13T08:50:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.094+0000 7f1520000640 1 -- 172.21.15.44:0/2709199746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15180cc550 con 0x7f15180b0320 2024-09-13T08:50:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.098+0000 7f1515600640 1 -- 172.21.15.44:0/2709199746 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f1510004030 con 0x7f15180b0320 2024-09-13T08:50:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.098+0000 7f1515600640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f14f8074960 0x7f14f8076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.098+0000 7f1515600640 1 -- 172.21.15.44:0/2709199746 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f1510093740 con 0x7f15180b0320 2024-09-13T08:50:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.098+0000 7f151d600640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f14f8074960 0x7f14f8076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.098+0000 7f151d600640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f14f8074960 0x7f14f8076e20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f150c009a80 tx=0x7f150c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:43.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.102+0000 7f1515600640 1 -- 172.21.15.44:0/2709199746 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f151005fda0 con 0x7f15180b0320 2024-09-13T08:50:43.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.346+0000 7f1520000640 1 -- 172.21.15.44:0/2709199746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f15180b1ea0 con 0x7f15180b0320 2024-09-13T08:50:43.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1515600640 1 -- 172.21.15.44:0/2709199746 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f15100647d0 con 0x7f15180b0320 2024-09-13T08:50:43.353 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T08:50:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:50:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-09-13T08:50:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-09-13T08:50:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-13T08:50:43.353 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-09-13T08:50:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1520000640 1 -- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f14f8074960 msgr2=0x7f14f8076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1520000640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f14f8074960 0x7f14f8076e20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f150c009a80 tx=0x7f150c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1520000640 1 -- 172.21.15.44:0/2709199746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15180b0320 msgr2=0x7f15180b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1520000640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15180b0320 0x7f15180b07a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f15180cf880 tx=0x7f1510010e90 comp rx=0 tx=0).stop 2024-09-13T08:50:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1520000640 1 -- 172.21.15.44:0/2709199746 shutdown_connections 2024-09-13T08:50:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1520000640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f14f8074960 0x7f14f8076e20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1520000640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15180b0320 0x7f15180b07a0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1520000640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f151809d350 0x7f151809d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1520000640 1 --2- 172.21.15.44:0/2709199746 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1500003fe0 0x7f151814b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.354+0000 7f1520000640 1 -- 172.21.15.44:0/2709199746 >> 172.21.15.44:0/2709199746 conn(0x7f15180b18f0 msgr2=0x7f1518154d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:43.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.358+0000 7f1520000640 1 -- 172.21.15.44:0/2709199746 shutdown_connections 2024-09-13T08:50:43.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.358+0000 7f1520000640 1 -- 172.21.15.44:0/2709199746 wait complete. 2024-09-13T08:50:43.374 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:50:43.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 -- 172.21.15.44:0/3901401648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d40cf4e0 msgr2=0x7f07d40c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:43.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 --2- 172.21.15.44:0/3901401648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d40cf4e0 0x7f07d40c5dd0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f07d40d0a60 tx=0x7f07cc01cc40 comp rx=0 tx=0).stop 2024-09-13T08:50:43.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 -- 172.21.15.44:0/3901401648 shutdown_connections 2024-09-13T08:50:43.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 --2- 172.21.15.44:0/3901401648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d40cf4e0 0x7f07d40c5dd0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 --2- 172.21.15.44:0/3901401648 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07d415c130 0x7f07d40ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 --2- 172.21.15.44:0/3901401648 >> v2:172.21.15.129:3300/0 conn(0x7f07d415bd50 0x7f07d40ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 -- 172.21.15.44:0/3901401648 >> 172.21.15.44:0/3901401648 conn(0x7f07d40d1ea0 msgr2=0x7f07d40d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:43.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 -- 172.21.15.44:0/3901401648 shutdown_connections 2024-09-13T08:50:43.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 -- 172.21.15.44:0/3901401648 wait complete. 2024-09-13T08:50:43.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 Processor -- start 2024-09-13T08:50:43.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.522+0000 7f07daa00640 1 -- start start 2024-09-13T08:50:43.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07daa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07d40cf4e0 0x7f07d4156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:43.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07daa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d415bd50 0x7f07d41f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:43.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07daa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07d415c130 0x7f07d4151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:43.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07daa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07d40ccac0 con 0x7f07d415bd50 2024-09-13T08:50:43.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07daa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07d40cc940 con 0x7f07d415c130 2024-09-13T08:50:43.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07daa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f07d40ccc40 con 0x7f07d40cf4e0 2024-09-13T08:50:43.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d415bd50 0x7f07d41f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:43.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d415bd50 0x7f07d41f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45858/0 (socket says 172.21.15.44:45858) 2024-09-13T08:50:43.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07cbe00640 1 -- 172.21.15.44:0/439012301 learned_addr learned my addr 172.21.15.44:0/439012301 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:43.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07d3e00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07d40cf4e0 0x7f07d4156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:43.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07d8a00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07d415c130 0x7f07d4151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:43.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07cbe00640 1 -- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07d40cf4e0 msgr2=0x7f07d4156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:43.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07cbe00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07d40cf4e0 0x7f07d4156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07cbe00640 1 -- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07d415c130 msgr2=0x7f07d4151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:43.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07cbe00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07d415c130 0x7f07d4151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07cbe00640 1 -- 172.21.15.44:0/439012301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d41562e0 con 0x7f07d415bd50 2024-09-13T08:50:43.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07cbe00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d415bd50 0x7f07d41f4c90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00d720 tx=0x7f07bc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:43.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07d0c00640 1 -- 172.21.15.44:0/439012301 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc017070 con 0x7f07d415bd50 2024-09-13T08:50:43.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07d0c00640 1 -- 172.21.15.44:0/439012301 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07bc009d20 con 0x7f07d415bd50 2024-09-13T08:50:43.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07d0c00640 1 -- 172.21.15.44:0/439012301 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0106c0 con 0x7f07d415bd50 2024-09-13T08:50:43.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07daa00640 1 -- 172.21.15.44:0/439012301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d40cbe70 con 0x7f07d415bd50 2024-09-13T08:50:43.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07daa00640 1 -- 172.21.15.44:0/439012301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d41f5950 con 0x7f07d415bd50 2024-09-13T08:50:43.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.526+0000 7f07daa00640 1 -- 172.21.15.44:0/439012301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d40c5dd0 con 0x7f07d415bd50 2024-09-13T08:50:43.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.530+0000 7f07d0c00640 1 -- 172.21.15.44:0/439012301 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f07bc01a070 con 0x7f07d415bd50 2024-09-13T08:50:43.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.530+0000 7f07d0c00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07b0074930 0x7f07b0076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:43.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.530+0000 7f07d0c00640 1 -- 172.21.15.44:0/439012301 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f07bc092e50 con 0x7f07d415bd50 2024-09-13T08:50:43.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.530+0000 7f07d3e00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07b0074930 0x7f07b0076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:43.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.530+0000 7f07d3e00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07b0074930 0x7f07b0076df0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f07d41575a0 tx=0x7f07c400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:43.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.534+0000 7f07d0c00640 1 -- 172.21.15.44:0/439012301 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f07d41562e0 con 0x7f07d415bd50 2024-09-13T08:50:43.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.746+0000 7f07daa00640 1 -- 172.21.15.44:0/439012301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07d41f5b50 con 0x7f07d415bd50 2024-09-13T08:50:43.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.746+0000 7f07d0c00640 1 -- 172.21.15.44:0/439012301 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5231 (secure 0 0 0) 0x7f07bc05f4b0 con 0x7f07d415bd50 2024-09-13T08:50:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:50:43.747 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":74,"btime":"2024-09-13T08:50:41:334888+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6854,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3108776730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3108776730},{"type":"v1","addr":"172.21.15.129:6836","nonce":3108776730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1076017330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1076017330},{"type":"v1","addr":"172.21.15.129:6837","nonce":1076017330}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4098004813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4098004813},{"type":"v1","addr":"172.21.15.44:6835","nonce":4098004813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:50:37.721609+0000","modified":"2024-09-13T08:50:40.739478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/1590451084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1590451084},{"type":"v1","addr":"172.21.15.44:6834","nonce":1590451084}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":8}]} 2024-09-13T08:50:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 74 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 -- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07b0074930 msgr2=0x7f07b0076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07b0074930 0x7f07b0076df0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f07d41575a0 tx=0x7f07c400b2e0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 -- 172.21.15.44:0/439012301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d415bd50 msgr2=0x7f07d41f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d415bd50 0x7f07d41f4c90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00d720 tx=0x7f07bc00dbe0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 -- 172.21.15.44:0/439012301 shutdown_connections 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f07b0074930 0x7f07b0076df0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07d415c130 0x7f07d4151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d415bd50 0x7f07d41f4c90 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 --2- 172.21.15.44:0/439012301 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07d40cf4e0 0x7f07d4156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 -- 172.21.15.44:0/439012301 >> 172.21.15.44:0/439012301 conn(0x7f07d40d1ea0 msgr2=0x7f07d4071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 -- 172.21.15.44:0/439012301 shutdown_connections 2024-09-13T08:50:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:43.750+0000 7f07daa00640 1 -- 172.21.15.44:0/439012301 wait complete. 2024-09-13T08:50:44.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T08:50:37.721609+0000', 'modified': '2024-09-13T08:50:40.739478+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8253}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8253': {'gid': 8253, 'name': 'c', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6834/1590451084', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 1590451084}, {'type': 'v1', 'addr': '172.21.15.44:6834', 'nonce': 1590451084}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8253, 'qdb_cluster': [8253]} 2024-09-13T08:50:44.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T08:50:44.173 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T08:50:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.326+0000 7f3ff4e00640 1 -- 172.21.15.44:0/2616450080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff015c1c0 msgr2=0x7f3ff006d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.326+0000 7f3ff4e00640 1 --2- 172.21.15.44:0/2616450080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff015c1c0 0x7f3ff006d000 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00b3820 tx=0x7f3fd801c890 comp rx=0 tx=0).stop 2024-09-13T08:50:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.326+0000 7f3ff4e00640 1 -- 172.21.15.44:0/2616450080 shutdown_connections 2024-09-13T08:50:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.326+0000 7f3ff4e00640 1 --2- 172.21.15.44:0/2616450080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3ff015f7b0 0x7f3ff015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.326+0000 7f3ff4e00640 1 --2- 172.21.15.44:0/2616450080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff015c1c0 0x7f3ff006d000 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.326+0000 7f3ff4e00640 1 --2- 172.21.15.44:0/2616450080 >> v2:172.21.15.129:3300/0 conn(0x7f3ff006cc20 0x7f3ff00737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.326+0000 7f3ff4e00640 1 -- 172.21.15.44:0/2616450080 >> 172.21.15.44:0/2616450080 conn(0x7f3ff00d0c90 msgr2=0x7f3ff00d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.326+0000 7f3ff4e00640 1 -- 172.21.15.44:0/2616450080 shutdown_connections 2024-09-13T08:50:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.326+0000 7f3ff4e00640 1 -- 172.21.15.44:0/2616450080 wait complete. 2024-09-13T08:50:44.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3ff4e00640 1 Processor -- start 2024-09-13T08:50:44.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3ff4e00640 1 -- start start 2024-09-13T08:50:44.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3ff4e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ff006cc20 0x7f3ff00b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3ff4e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3ff015c1c0 0x7f3ff00c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3ff4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff015f7b0 0x7f3ff00c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3ff4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff00719c0 con 0x7f3ff015f7b0 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3ff4e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0071840 con 0x7f3ff006cc20 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3ff4e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff0071b40 con 0x7f3ff015c1c0 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff015f7b0 0x7f3ff00c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff015f7b0 0x7f3ff00c5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45880/0 (socket says 172.21.15.44:45880) 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fee800640 1 -- 172.21.15.44:0/846026784 learned_addr learned my addr 172.21.15.44:0/846026784 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fede00640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ff006cc20 0x7f3ff00b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fe7e00640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3ff015c1c0 0x7f3ff00c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fee800640 1 -- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3ff015c1c0 msgr2=0x7f3ff00c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fee800640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3ff015c1c0 0x7f3ff00c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fee800640 1 -- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ff006cc20 msgr2=0x7f3ff00b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fee800640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ff006cc20 0x7f3ff00b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fee800640 1 -- 172.21.15.44:0/846026784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff00b3ac0 con 0x7f3ff015f7b0 2024-09-13T08:50:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fee800640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff015f7b0 0x7f3ff00c5380 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e970 tx=0x7f3fe800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fe6000640 1 -- 172.21.15.44:0/846026784 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8004d30 con 0x7f3ff015f7b0 2024-09-13T08:50:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fe6000640 1 -- 172.21.15.44:0/846026784 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fe800cb20 con 0x7f3ff015f7b0 2024-09-13T08:50:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3ff4e00640 1 -- 172.21.15.44:0/846026784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff00b3da0 con 0x7f3ff015f7b0 2024-09-13T08:50:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3fe6000640 1 -- 172.21.15.44:0/846026784 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8005020 con 0x7f3ff015f7b0 2024-09-13T08:50:44.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.330+0000 7f3ff4e00640 1 -- 172.21.15.44:0/846026784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff0155300 con 0x7f3ff015f7b0 2024-09-13T08:50:44.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.334+0000 7f3fe6000640 1 -- 172.21.15.44:0/846026784 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f3fe80051c0 con 0x7f3ff015f7b0 2024-09-13T08:50:44.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.334+0000 7f3ff8162000 1 Processor -- start 2024-09-13T08:50:44.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.334+0000 7f3fe6000640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fcc074930 0x7f3fcc076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.334+0000 7f3ff8162000 1 -- start start 2024-09-13T08:50:44.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.334+0000 7f3fe6000640 1 -- 172.21.15.44:0/846026784 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f3fe8093690 con 0x7f3ff015f7b0 2024-09-13T08:50:44.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.334+0000 7f3fe7e00640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fcc074930 0x7f3fcc076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.334+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 0x565441af2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441af32f0 0x565441aa68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fe7e00640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fcc074930 0x7f3fcc076df0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80af000 tx=0x7f3fd801b0d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441aa6e10 0x565441aa71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5654419c73f0 con 0x565441aa6e10 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565441904b20 con 0x565441af29d0 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x5654418f3f50 con 0x565441af32f0 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441aa6e10 0x565441aa71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441aa6e10 0x565441aa71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45892/0 (socket says 172.21.15.44:45892) 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fee800640 1 -- 172.21.15.44:0/336466269 learned_addr learned my addr 172.21.15.44:0/336466269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fe7e00640 1 --2- 172.21.15.44:0/336466269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441af32f0 0x565441aa68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fede00640 1 --2- 172.21.15.44:0/336466269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 0x565441af2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fee800640 1 -- 172.21.15.44:0/336466269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441af32f0 msgr2=0x565441aa68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:50:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fee800640 1 --2- 172.21.15.44:0/336466269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441af32f0 0x565441aa68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fee800640 1 -- 172.21.15.44:0/336466269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 msgr2=0x565441af2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:50:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fee800640 1 --2- 172.21.15.44:0/336466269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 0x565441af2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fee800640 1 -- 172.21.15.44:0/336466269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654418bb590 con 0x565441aa6e10 2024-09-13T08:50:44.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fee800640 1 --2- 172.21.15.44:0/336466269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441aa6e10 0x565441aa71f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f3fe805fbf0 tx=0x7f3fe805f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68c6f0c216f0c80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fdea00640 1 -- 172.21.15.44:0/336466269 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8085020 con 0x565441aa6e10 2024-09-13T08:50:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fdea00640 1 -- 172.21.15.44:0/336466269 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fe80851c0 con 0x565441aa6e10 2024-09-13T08:50:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3fdea00640 1 -- 172.21.15.44:0/336466269 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe801f030 con 0x565441aa6e10 2024-09-13T08:50:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 -- 172.21.15.44:0/336466269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441aa6e10 msgr2=0x565441aa71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 --2- 172.21.15.44:0/336466269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441aa6e10 0x565441aa71f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f3fe805fbf0 tx=0x7f3fe805f4d0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 -- 172.21.15.44:0/336466269 shutdown_connections 2024-09-13T08:50:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 --2- 172.21.15.44:0/336466269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441aa6e10 0x565441aa71f0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 --2- 172.21.15.44:0/336466269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441af32f0 0x565441aa68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 --2- 172.21.15.44:0/336466269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 0x565441af2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 -- 172.21.15.44:0/336466269 >> 172.21.15.44:0/336466269 conn(0x565441990130 msgr2=0x565441ad5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 -- 172.21.15.44:0/336466269 shutdown_connections 2024-09-13T08:50:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 -- 172.21.15.44:0/336466269 wait complete. 2024-09-13T08:50:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 -- start start 2024-09-13T08:50:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 Processor -- start 2024-09-13T08:50:44.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441aa6e10 0x565441ad36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 0x565441ad3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.338+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441ad43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56544188e5d0 con 0x565441af32f0 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5654418f7d60 con 0x565441af29d0 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x56544189a500 con 0x565441aa6e10 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 10 client.0 _ll_register_callbacks cb 0x56544189e810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441ad43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441ad43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45900/0 (socket says 172.21.15.44:45900) 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fede00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441aa6e10 0x565441ad36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fee800640 1 -- 172.21.15.44:0/2328505947 learned_addr learned my addr 172.21.15.44:0/2328505947 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fe7e00640 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 0x565441ad3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fee800640 1 -- 172.21.15.44:0/2328505947 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441aa6e10 msgr2=0x565441ad36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:50:44.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fee800640 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441aa6e10 0x565441ad36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fee800640 1 -- 172.21.15.44:0/2328505947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 msgr2=0x565441ad3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fee800640 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 0x565441ad3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fee800640 1 -- 172.21.15.44:0/2328505947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565441892170 con 0x565441af32f0 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fe7e00640 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 0x565441ad3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fee800640 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441ad43b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f3fe805fbf0 tx=0x7f3fe804d3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2229bde93c96f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fd3400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fd3400640 1 -- 172.21.15.44:0/2328505947 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe807e060 con 0x565441af32f0 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fd3400640 1 -- 172.21.15.44:0/2328505947 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f3fe805eec0 con 0x565441af32f0 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- 172.21.15.44:0/2328505947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56544188c0f0 con 0x565441af32f0 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fd3400640 1 -- 172.21.15.44:0/2328505947 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f3fe801f040 con 0x565441af32f0 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 10 client.8376 fetch_fsmap learned FSMap version 74 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- 172.21.15.44:0/2328505947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x565441978ef0 con 0x565441af32f0 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fd3400640 1 -- 172.21.15.44:0/2328505947 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56544188c0f0 con 0x565441af32f0 2024-09-13T08:50:44.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fd3400640 1 -- 172.21.15.44:0/2328505947 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f3fe800f040 con 0x565441af32f0 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 10 client.8376 fetch_fsmap finished waiting for FSMap version 74 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 10 client.8376 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 20 client.8376 populate_metadata read hostname 'smithi044' 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.44:6832/1590451084,v1:172.21.15.44:6834/1590451084] conn(0x565441ad50f0 0x565441b04c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 4 client.8376 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- 172.21.15.44:0/2328505947 --> [v2:172.21.15.44:6832/1590451084,v1:172.21.15.44:6834/1590451084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565441906a70 con 0x565441ad50f0 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fede00640 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.44:6832/1590451084,v1:172.21.15.44:6834/1590451084] conn(0x565441ad50f0 0x565441b04c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fede00640 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.44:6832/1590451084,v1:172.21.15.44:6834/1590451084] conn(0x565441ad50f0 0x565441b04c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb1abc6d4454e58 server_cookie=5cc28a25dceec6bb in_seq=0 out_seq=0 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fd3400640 10 client.8376 ms_handle_connect on v2:172.21.15.44:6832/1590451084 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fd3400640 1 -- 172.21.15.44:0/2328505947 <== mds.0 v2:172.21.15.44:6832/1590451084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565441906a70 con 0x565441ad50f0 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fd3400640 10 client.8376 handle_command_reply: tid=0 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 client.8376 shutdown 2024-09-13T08:50:44.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- 172.21.15.44:0/2328505947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 msgr2=0x565441ad43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441ad43b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f3fe805fbf0 tx=0x7f3fe804d3f0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- 172.21.15.44:0/2328505947 shutdown_connections 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.44:6832/1590451084,v1:172.21.15.44:6834/1590451084] conn(0x565441ad50f0 0x565441b04c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441ad43b0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af29d0 0x565441ad3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3fd3400640 0 client.8376 ms_handle_reset on v2:172.21.15.44:6832/1590451084 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 --2- 172.21.15.44:0/2328505947 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441aa6e10 0x565441ad36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- 172.21.15.44:0/2328505947 >> 172.21.15.44:0/2328505947 conn(0x565441990130 msgr2=0x565441aa5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- 172.21.15.44:0/2328505947 shutdown_connections 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 1 -- 172.21.15.44:0/2328505947 wait complete. 2024-09-13T08:50:44.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.342+0000 7f3ff8162000 20 client.8376 trim_cache size 0 max 16384 2024-09-13T08:50:44.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3ff8162000 1 Processor -- start 2024-09-13T08:50:44.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3ff8162000 1 -- start start 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441aa6e10 0x565441af1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af29d0 0x565441af2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af32f0 0x565441aa50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3ff8162000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0203910 con 0x565441af29d0 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3ff8162000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff00b1540 con 0x565441af32f0 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3ff8162000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff00b6320 con 0x565441aa6e10 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fe7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af29d0 0x565441af2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fe7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af29d0 0x565441af2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45902/0 (socket says 172.21.15.44:45902) 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fe7e00640 1 -- 172.21.15.44:0/2253075417 learned_addr learned my addr 172.21.15.44:0/2253075417 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fede00640 1 --2- 172.21.15.44:0/2253075417 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441aa6e10 0x565441af1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fee800640 1 --2- 172.21.15.44:0/2253075417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af32f0 0x565441aa50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fe7e00640 1 -- 172.21.15.44:0/2253075417 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441aa6e10 msgr2=0x565441af1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:50:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fe7e00640 1 --2- 172.21.15.44:0/2253075417 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441aa6e10 0x565441af1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fe7e00640 1 -- 172.21.15.44:0/2253075417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af32f0 msgr2=0x565441aa50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:50:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fe7e00640 1 --2- 172.21.15.44:0/2253075417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af32f0 0x565441aa50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fe7e00640 1 -- 172.21.15.44:0/2253075417 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544194fd90 con 0x565441af29d0 2024-09-13T08:50:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fe7e00640 1 --2- 172.21.15.44:0/2253075417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af29d0 0x565441af2200 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80096c0 tx=0x7f3fd80b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186e164f653d7c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:44.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fde000640 1 -- 172.21.15.44:0/2253075417 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80b2040 con 0x565441af29d0 2024-09-13T08:50:44.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fde000640 1 -- 172.21.15.44:0/2253075417 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fd8005240 con 0x565441af29d0 2024-09-13T08:50:44.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3fde000640 1 -- 172.21.15.44:0/2253075417 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80a5710 con 0x565441af29d0 2024-09-13T08:50:44.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.366+0000 7f3ff8162000 1 -- 172.21.15.44:0/2253075417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af29d0 msgr2=0x565441af2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 --2- 172.21.15.44:0/2253075417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af29d0 0x565441af2200 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80096c0 tx=0x7f3fd80b4000 comp rx=0 tx=0).stop 2024-09-13T08:50:44.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- 172.21.15.44:0/2253075417 shutdown_connections 2024-09-13T08:50:44.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 --2- 172.21.15.44:0/2253075417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441af32f0 0x565441aa50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 --2- 172.21.15.44:0/2253075417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af29d0 0x565441af2200 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 --2- 172.21.15.44:0/2253075417 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441aa6e10 0x565441af1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- 172.21.15.44:0/2253075417 >> 172.21.15.44:0/2253075417 conn(0x565441990130 msgr2=0x565441af15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:44.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- 172.21.15.44:0/2253075417 shutdown_connections 2024-09-13T08:50:44.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- 172.21.15.44:0/2253075417 wait complete. 2024-09-13T08:50:44.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- start start 2024-09-13T08:50:44.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 Processor -- start 2024-09-13T08:50:44.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441aa6e10 0x565441b68c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441af29d0 0x565441b69140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441b69680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56544188e5d0 con 0x565441af32f0 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5654418bfec0 con 0x565441aa6e10 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x5654418f7d60 con 0x565441af29d0 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 10 client.0 _ll_register_callbacks cb 0x7f3fe0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441b69680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441b69680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45904/0 (socket says 172.21.15.44:45904) 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fee800640 1 -- 172.21.15.44:0/3990801816 learned_addr learned my addr 172.21.15.44:0/3990801816 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fee800640 1 -- 172.21.15.44:0/3990801816 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441af29d0 msgr2=0x565441b69140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fee800640 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441af29d0 0x565441b69140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fee800640 1 -- 172.21.15.44:0/3990801816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441aa6e10 msgr2=0x565441b68c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fee800640 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441aa6e10 0x565441b68c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fee800640 1 -- 172.21.15.44:0/3990801816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654418baa90 con 0x565441af32f0 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fee800640 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441b69680 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f3fe805ced0 tx=0x7f3fe805f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92509afc0a9e2278 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fd3400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fd3400640 1 -- 172.21.15.44:0/3990801816 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8069620 con 0x565441af32f0 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fd3400640 1 -- 172.21.15.44:0/3990801816 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f3fe805e930 con 0x565441af32f0 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- 172.21.15.44:0/3990801816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fd8005240 con 0x565441af32f0 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fd3400640 1 -- 172.21.15.44:0/3990801816 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f3fe805eb30 con 0x565441af32f0 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 10 client.8382 fetch_fsmap learned FSMap version 74 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- 172.21.15.44:0/3990801816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x565441899050 con 0x565441af32f0 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fd3400640 1 -- 172.21.15.44:0/3990801816 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fd8005240 con 0x565441af32f0 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fd3400640 1 -- 172.21.15.44:0/3990801816 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f3fe8065820 con 0x565441af32f0 2024-09-13T08:50:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 10 client.8382 fetch_fsmap finished waiting for FSMap version 74 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 10 client.8382 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 20 client.8382 populate_metadata read hostname 'smithi044' 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.44:6832/1590451084,v1:172.21.15.44:6834/1590451084] conn(0x565441b07070 0x565441b07450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 4 client.8382 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3ff8162000 1 -- 172.21.15.44:0/3990801816 --> [v2:172.21.15.44:6832/1590451084,v1:172.21.15.44:6834/1590451084] -- command(tid 0: {"prefix": "status"}) -- 0x565441902520 con 0x565441b07070 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.370+0000 7f3fede00640 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.44:6832/1590451084,v1:172.21.15.44:6834/1590451084] conn(0x565441b07070 0x565441b07450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3fede00640 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.44:6832/1590451084,v1:172.21.15.44:6834/1590451084] conn(0x565441b07070 0x565441b07450 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d43623930d8fd1 server_cookie=de68fee404cbce88 in_seq=0 out_seq=0 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3fd3400640 10 client.8382 ms_handle_connect on v2:172.21.15.44:6832/1590451084 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3fd3400640 1 -- 172.21.15.44:0/3990801816 <== mds.0 v2:172.21.15.44:6832/1590451084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x565441902520 con 0x565441b07070 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3fd3400640 10 client.8382 handle_command_reply: tid=0 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 client.8382 shutdown 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 -- 172.21.15.44:0/3990801816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 msgr2=0x565441b69680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441b69680 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f3fe805ced0 tx=0x7f3fe805f0f0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 -- 172.21.15.44:0/3990801816 shutdown_connections 2024-09-13T08:50:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.44:6832/1590451084,v1:172.21.15.44:6834/1590451084] conn(0x565441b07070 0x565441b07450 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565441af32f0 0x565441b69680 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x565441af29d0 0x565441b69140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 --2- 172.21.15.44:0/3990801816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565441aa6e10 0x565441b68c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3fd3400640 0 client.8382 ms_handle_reset on v2:172.21.15.44:6832/1590451084 2024-09-13T08:50:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 -- 172.21.15.44:0/3990801816 >> 172.21.15.44:0/3990801816 conn(0x565441990130 msgr2=0x565441aa50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 -- 172.21.15.44:0/3990801816 shutdown_connections 2024-09-13T08:50:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 1 -- 172.21.15.44:0/3990801816 wait complete. 2024-09-13T08:50:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff8162000 20 client.8382 trim_cache size 0 max 16384 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 8253, 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.075391497, 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 74, 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 116, 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 116, 2024-09-13T08:50:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 10.534969565999999 2024-09-13T08:50:44.378 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:50:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 -- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fcc074930 msgr2=0x7f3fcc076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fcc074930 0x7f3fcc076df0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80af000 tx=0x7f3fd801b0d0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 -- 172.21.15.44:0/846026784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff015f7b0 msgr2=0x7f3ff00c5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff015f7b0 0x7f3ff00c5380 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e970 tx=0x7f3fe800ee30 comp rx=0 tx=0).stop 2024-09-13T08:50:44.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 -- 172.21.15.44:0/846026784 shutdown_connections 2024-09-13T08:50:44.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fcc074930 0x7f3fcc076df0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff015f7b0 0x7f3ff00c5380 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3ff015c1c0 0x7f3ff00c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 --2- 172.21.15.44:0/846026784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ff006cc20 0x7f3ff00b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 -- 172.21.15.44:0/846026784 >> 172.21.15.44:0/846026784 conn(0x7f3ff00d0c90 msgr2=0x7f3ff00c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:44.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 -- 172.21.15.44:0/846026784 shutdown_connections 2024-09-13T08:50:44.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.374+0000 7f3ff4e00640 1 -- 172.21.15.44:0/846026784 wait complete. 2024-09-13T08:50:44.392 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:50:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.538+0000 7f7566800640 1 -- 172.21.15.44:0/3038021661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75600cdb30 msgr2=0x7f75600aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.538+0000 7f7566800640 1 --2- 172.21.15.44:0/3038021661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75600cdb30 0x7f75600aeef0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f7558009c80 tx=0x7f755801cda0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.538+0000 7f7566800640 1 -- 172.21.15.44:0/3038021661 shutdown_connections 2024-09-13T08:50:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.538+0000 7f7566800640 1 --2- 172.21.15.44:0/3038021661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75600cdb30 0x7f75600aeef0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.538+0000 7f7566800640 1 --2- 172.21.15.44:0/3038021661 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75600ced80 0x7f75600ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.538+0000 7f7566800640 1 -- 172.21.15.44:0/3038021661 >> 172.21.15.44:0/3038021661 conn(0x7f75600d1dc0 msgr2=0x7f75600d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.538+0000 7f7566800640 1 -- 172.21.15.44:0/3038021661 shutdown_connections 2024-09-13T08:50:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.538+0000 7f7566800640 1 -- 172.21.15.44:0/3038021661 wait complete. 2024-09-13T08:50:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7566800640 1 Processor -- start 2024-09-13T08:50:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7566800640 1 -- start start 2024-09-13T08:50:44.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7566800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75600cdb30 0x7f7560072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7566800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75600ced80 0x7f75600cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7566800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756015bea0 0x7f75601564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7566800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7560169340 con 0x7f756015bea0 2024-09-13T08:50:44.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7566800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75601691c0 con 0x7f75600cdb30 2024-09-13T08:50:44.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7566800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f75601694c0 con 0x7f75600ced80 2024-09-13T08:50:44.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f755fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756015bea0 0x7f75601564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f755fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756015bea0 0x7f75601564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45918/0 (socket says 172.21.15.44:45918) 2024-09-13T08:50:44.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f755fe00640 1 -- 172.21.15.44:0/1564611593 learned_addr learned my addr 172.21.15.44:0/1564611593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7557e00640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75600ced80 0x7f75600cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f755f400640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75600cdb30 0x7f7560072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f755fe00640 1 -- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75600ced80 msgr2=0x7f75600cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f755fe00640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75600ced80 0x7f75600cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f755fe00640 1 -- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75600cdb30 msgr2=0x7f7560072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f755fe00640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75600cdb30 0x7f7560072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f755fe00640 1 -- 172.21.15.44:0/1564611593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75601fb2b0 con 0x7f756015bea0 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f755fe00640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756015bea0 0x7f75601564a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7558009bd0 tx=0x7f7558009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7557400640 1 -- 172.21.15.44:0/1564611593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580bb070 con 0x7f756015bea0 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7557400640 1 -- 172.21.15.44:0/1564611593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7558004e00 con 0x7f756015bea0 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7557400640 1 -- 172.21.15.44:0/1564611593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580b2040 con 0x7f756015bea0 2024-09-13T08:50:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7566800640 1 -- 172.21.15.44:0/1564611593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75601fb590 con 0x7f756015bea0 2024-09-13T08:50:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.542+0000 7f7566800640 1 -- 172.21.15.44:0/1564611593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75601fbb50 con 0x7f756015bea0 2024-09-13T08:50:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.546+0000 7f7566800640 1 -- 172.21.15.44:0/1564611593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75600ce670 con 0x7f756015bea0 2024-09-13T08:50:44.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.546+0000 7f7557400640 1 -- 172.21.15.44:0/1564611593 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f75580be070 con 0x7f756015bea0 2024-09-13T08:50:44.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.546+0000 7f7557400640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f753c074930 0x7f753c076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:44.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.546+0000 7f7557400640 1 -- 172.21.15.44:0/1564611593 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f755812c900 con 0x7f756015bea0 2024-09-13T08:50:44.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.546+0000 7f755f400640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f753c074930 0x7f753c076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:44.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.546+0000 7f755f400640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f753c074930 0x7f753c076df0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7550001650 tx=0x7f7550009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:44.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.550+0000 7f7557400640 1 -- 172.21.15.44:0/1564611593 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7558016990 con 0x7f756015bea0 2024-09-13T08:50:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.766+0000 7f7566800640 1 -- 172.21.15.44:0/1564611593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f75600b7c20 con 0x7f756015bea0 2024-09-13T08:50:44.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.766+0000 7f7557400640 1 -- 172.21.15.44:0/1564611593 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f75580f95b0 con 0x7f756015bea0 2024-09-13T08:50:44.766 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:50:44.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 -- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f753c074930 msgr2=0x7f753c076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f753c074930 0x7f753c076df0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7550001650 tx=0x7f7550009040 comp rx=0 tx=0).stop 2024-09-13T08:50:44.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 -- 172.21.15.44:0/1564611593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756015bea0 msgr2=0x7f75601564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:44.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756015bea0 0x7f75601564a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7558009bd0 tx=0x7f7558009390 comp rx=0 tx=0).stop 2024-09-13T08:50:44.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 -- 172.21.15.44:0/1564611593 shutdown_connections 2024-09-13T08:50:44.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f753c074930 0x7f753c076df0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756015bea0 0x7f75601564a0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75600ced80 0x7f75600cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 --2- 172.21.15.44:0/1564611593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75600cdb30 0x7f7560072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:44.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 -- 172.21.15.44:0/1564611593 >> 172.21.15.44:0/1564611593 conn(0x7f75600d1dc0 msgr2=0x7f75600b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:44.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 -- 172.21.15.44:0/1564611593 shutdown_connections 2024-09-13T08:50:44.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:44.770+0000 7f7566800640 1 -- 172.21.15.44:0/1564611593 wait complete. 2024-09-13T08:50:44.788 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T08:50:44.788 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:50:44.788 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:50:44.788 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T08:50:44.788 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:50:44.788 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T08:50:44.788 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T08:50:44.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:50:44.789 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:50:44.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:44.789 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:50:44.798 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:50:44.799 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41638sec preferred_lft 41638sec 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:50:44.800 INFO:teuthology.orchestra.run.smithi044.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T08:50:44.801 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:50:44.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:44.802 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:50:44.861 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:50:44.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:44.863 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:50:44.916 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:50:44.916 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:50:44.917 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:50:44.917 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:50:44.917 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:50:44.917 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:50:44.917 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:50:44.917 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:50:44.918 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:50:44.918 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:50:44.918 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:50:44.918 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:50:45.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:50:45.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:50:45.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:45.056 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:50:45.065 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:50:45.065 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T08:50:45.066 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:50:45.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:45.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T08:50:45.126 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:50:45.126 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T08:50:45.183 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:50:45.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:50:45.183 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:50:45.235 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:50:45.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:45.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:50:45.328 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:50:45.329 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:50:45.329 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:50:45.344 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:50:45.344 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:50:45.344 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:50:45.344 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:50:45.344 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:50:45.362 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:50:45.363 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:50:45.363 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:50:45.363 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:50:45.363 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:50:45.363 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:50:45.363 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:50:45.363 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:50:45.366 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:50:45.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:45.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:50:45.387 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:50:45.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:50:45.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:50:45.495 INFO:teuthology.orchestra.run.smithi044.stdout:client7801 2024-09-13T08:50:45.496 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:50:45.496 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:45.506 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-13T08:50:45.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.698+0000 7fc717c00640 1 -- 172.21.15.44:0/2838387225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7100b0b60 msgr2=0x7fc7100ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:45.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.698+0000 7fc717c00640 1 --2- 172.21.15.44:0/2838387225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7100b0b60 0x7fc7100ccbd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc708009bc0 tx=0x7fc70801f3b0 comp rx=0 tx=0).stop 2024-09-13T08:50:45.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.698+0000 7fc717c00640 1 -- 172.21.15.44:0/2838387225 shutdown_connections 2024-09-13T08:50:45.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.698+0000 7fc717c00640 1 --2- 172.21.15.44:0/2838387225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7100b0b60 0x7fc7100ccbd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:45.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.698+0000 7fc717c00640 1 --2- 172.21.15.44:0/2838387225 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc71015c1f0 0x7fc7100cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:45.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.698+0000 7fc717c00640 1 -- 172.21.15.44:0/2838387225 >> 172.21.15.44:0/2838387225 conn(0x7fc7100d1da0 msgr2=0x7fc7100d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:45.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.698+0000 7fc717c00640 1 -- 172.21.15.44:0/2838387225 shutdown_connections 2024-09-13T08:50:45.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.698+0000 7fc717c00640 1 -- 172.21.15.44:0/2838387225 wait complete. 2024-09-13T08:50:45.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 Processor -- start 2024-09-13T08:50:45.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 -- start start 2024-09-13T08:50:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7100b0b60 0x7fc7100b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc71015be10 0x7fc71006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71015c1f0 0x7fc7101587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc7100cd320 con 0x7fc71015c1f0 2024-09-13T08:50:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc7100cf590 con 0x7fc7100b0b60 2024-09-13T08:50:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc7100cd4a0 con 0x7fc71015be10 2024-09-13T08:50:45.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc715c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71015c1f0 0x7fc7101587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc715c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71015c1f0 0x7fc7101587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45920/0 (socket says 172.21.15.44:45920) 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc715200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7100b0b60 0x7fc7100b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc715c00640 1 -- 172.21.15.44:0/4214557006 learned_addr learned my addr 172.21.15.44:0/4214557006 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc707e00640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc71015be10 0x7fc71006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc715c00640 1 -- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc71015be10 msgr2=0x7fc71006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc715c00640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc71015be10 0x7fc71006dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc715c00640 1 -- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7100b0b60 msgr2=0x7fc7100b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc715c00640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7100b0b60 0x7fc7100b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc715c00640 1 -- 172.21.15.44:0/4214557006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71006e6f0 con 0x7fc71015c1f0 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc715c00640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71015c1f0 0x7fc7101587f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc70801cfe0 tx=0x7fc708009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc70d600640 1 -- 172.21.15.44:0/4214557006 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080b4040 con 0x7fc71015c1f0 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc70d600640 1 -- 172.21.15.44:0/4214557006 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc708004b40 con 0x7fc71015c1f0 2024-09-13T08:50:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc70d600640 1 -- 172.21.15.44:0/4214557006 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080af8a0 con 0x7fc71015c1f0 2024-09-13T08:50:45.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 -- 172.21.15.44:0/4214557006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71006e9d0 con 0x7fc71015c1f0 2024-09-13T08:50:45.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 -- 172.21.15.44:0/4214557006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71006ef90 con 0x7fc71015c1f0 2024-09-13T08:50:45.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.702+0000 7fc717c00640 1 -- 172.21.15.44:0/4214557006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7100ccbd0 con 0x7fc71015c1f0 2024-09-13T08:50:45.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.706+0000 7fc70d600640 1 -- 172.21.15.44:0/4214557006 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fc7080c1020 con 0x7fc71015c1f0 2024-09-13T08:50:45.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.706+0000 7fc70d600640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6ec074930 0x7fc6ec076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:45.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.706+0000 7fc70d600640 1 -- 172.21.15.44:0/4214557006 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fc7081306a0 con 0x7fc71015c1f0 2024-09-13T08:50:45.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.706+0000 7fc715200640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6ec074930 0x7fc6ec076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:45.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.706+0000 7fc715200640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6ec074930 0x7fc6ec076df0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc700007fd0 tx=0x7fc700007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:45.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.710+0000 7fc70d600640 1 -- 172.21.15.44:0/4214557006 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc708139070 con 0x7fc71015c1f0 2024-09-13T08:50:45.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:45.922+0000 7fc717c00640 1 -- 172.21.15.44:0/4214557006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fc710070350 con 0x7fc71015c1f0 2024-09-13T08:50:46.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc70d600640 1 -- 172.21.15.44:0/4214557006 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v75) ==== 136+0+0 (secure 0 0 0) 0x7fc708018f10 con 0x7fc71015c1f0 2024-09-13T08:50:46.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc717c00640 1 -- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6ec074930 msgr2=0x7fc6ec076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:46.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc717c00640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6ec074930 0x7fc6ec076df0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc700007fd0 tx=0x7fc700007ea0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc717c00640 1 -- 172.21.15.44:0/4214557006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71015c1f0 msgr2=0x7fc7101587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:46.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc717c00640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71015c1f0 0x7fc7101587f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc70801cfe0 tx=0x7fc708009300 comp rx=0 tx=0).stop 2024-09-13T08:50:46.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc717c00640 1 -- 172.21.15.44:0/4214557006 shutdown_connections 2024-09-13T08:50:46.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc717c00640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc6ec074930 0x7fc6ec076df0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc717c00640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc71015c1f0 0x7fc7101587f0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc717c00640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc71015be10 0x7fc71006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc717c00640 1 --2- 172.21.15.44:0/4214557006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7100b0b60 0x7fc7100b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.358+0000 7fc717c00640 1 -- 172.21.15.44:0/4214557006 >> 172.21.15.44:0/4214557006 conn(0x7fc7100d1da0 msgr2=0x7fc7100cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:46.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.362+0000 7fc717c00640 1 -- 172.21.15.44:0/4214557006 shutdown_connections 2024-09-13T08:50:46.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.362+0000 7fc717c00640 1 -- 172.21.15.44:0/4214557006 wait complete. 2024-09-13T08:50:46.381 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:50:46.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.530+0000 7f2a13e00640 1 -- 172.21.15.44:0/78330460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a140d01b0 msgr2=0x7f2a140cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:46.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.530+0000 7f2a13e00640 1 --2- 172.21.15.44:0/78330460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a140d01b0 0x7f2a140cefb0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2a14073000 tx=0x7f29fc01c830 comp rx=0 tx=0).stop 2024-09-13T08:50:46.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.530+0000 7f2a13e00640 1 -- 172.21.15.44:0/78330460 shutdown_connections 2024-09-13T08:50:46.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.530+0000 7f2a13e00640 1 --2- 172.21.15.44:0/78330460 >> v2:172.21.15.129:3300/0 conn(0x7f2a140b2590 0x7f2a140b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.530+0000 7f2a13e00640 1 --2- 172.21.15.44:0/78330460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a140d01b0 0x7f2a140cefb0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.530+0000 7f2a13e00640 1 -- 172.21.15.44:0/78330460 >> 172.21.15.44:0/78330460 conn(0x7f2a1409aa10 msgr2=0x7f2a1409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:46.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.530+0000 7f2a13e00640 1 -- 172.21.15.44:0/78330460 shutdown_connections 2024-09-13T08:50:46.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 -- 172.21.15.44:0/78330460 wait complete. 2024-09-13T08:50:46.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 Processor -- start 2024-09-13T08:50:46.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 -- start start 2024-09-13T08:50:46.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a140b2590 0x7f2a141f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:46.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a140d01b0 0x7f2a140c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:46.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2a1415f000 0x7f2a14156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:46.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14168e20 con 0x7f2a140d01b0 2024-09-13T08:50:46.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14168ca0 con 0x7f2a140b2590 2024-09-13T08:50:46.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2a14168fa0 con 0x7f2a1415f000 2024-09-13T08:50:46.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a140d01b0 0x7f2a140c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:46.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a140d01b0 0x7f2a140c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45932/0 (socket says 172.21.15.44:45932) 2024-09-13T08:50:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0be00640 1 -- 172.21.15.44:0/2865848983 learned_addr learned my addr 172.21.15.44:0/2865848983 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0be00640 1 -- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2a1415f000 msgr2=0x7f2a14156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0be00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2a1415f000 0x7f2a14156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0be00640 1 -- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a140b2590 msgr2=0x7f2a141f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0be00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a140b2590 0x7f2a141f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0be00640 1 -- 172.21.15.44:0/2865848983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a141f5c30 con 0x7f2a140d01b0 2024-09-13T08:50:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0be00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a140d01b0 0x7f2a140c6460 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f29fc019440 tx=0x7f29fc005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0aa00640 1 -- 172.21.15.44:0/2865848983 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0bb020 con 0x7f2a140d01b0 2024-09-13T08:50:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0aa00640 1 -- 172.21.15.44:0/2865848983 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29fc0042b0 con 0x7f2a140d01b0 2024-09-13T08:50:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a0aa00640 1 -- 172.21.15.44:0/2865848983 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc005290 con 0x7f2a140d01b0 2024-09-13T08:50:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 -- 172.21.15.44:0/2865848983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a141f5f00 con 0x7f2a140d01b0 2024-09-13T08:50:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.534+0000 7f2a13e00640 1 -- 172.21.15.44:0/2865848983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a141f64c0 con 0x7f2a140d01b0 2024-09-13T08:50:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.538+0000 7f2a13e00640 1 -- 172.21.15.44:0/2865848983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a140b29e0 con 0x7f2a140d01b0 2024-09-13T08:50:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.538+0000 7f2a0aa00640 1 -- 172.21.15.44:0/2865848983 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f29fc0be020 con 0x7f2a140d01b0 2024-09-13T08:50:46.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.538+0000 7f2a0aa00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f29f0074930 0x7f29f0076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.538+0000 7f2a0aa00640 1 -- 172.21.15.44:0/2865848983 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f29fc12cfc0 con 0x7f2a140d01b0 2024-09-13T08:50:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.538+0000 7f2a12a00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f29f0074930 0x7f29f0076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.542+0000 7f2a12a00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f29f0074930 0x7f29f0076df0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2a04007fd0 tx=0x7f2a04007940 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.542+0000 7f2a0aa00640 1 -- 172.21.15.44:0/2865848983 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f29fc0c6020 con 0x7f2a140d01b0 2024-09-13T08:50:46.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.742+0000 7f2a13e00640 1 -- 172.21.15.44:0/2865848983 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f2a140c8940 con 0x7f29f0074930 2024-09-13T08:50:46.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.746+0000 7f2a0aa00640 1 -- 172.21.15.44:0/2865848983 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f2a140c8940 con 0x7f29f0074930 2024-09-13T08:50:46.745 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:50:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:50:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:50:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:50:46.745 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:50:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.746+0000 7f2a13e00640 1 -- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f29f0074930 msgr2=0x7f29f0076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.746+0000 7f2a13e00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f29f0074930 0x7f29f0076df0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2a04007fd0 tx=0x7f2a04007940 comp rx=0 tx=0).stop 2024-09-13T08:50:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.746+0000 7f2a13e00640 1 -- 172.21.15.44:0/2865848983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a140d01b0 msgr2=0x7f2a140c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.746+0000 7f2a13e00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a140d01b0 0x7f2a140c6460 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f29fc019440 tx=0x7f29fc005760 comp rx=0 tx=0).stop 2024-09-13T08:50:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.750+0000 7f2a13e00640 1 -- 172.21.15.44:0/2865848983 shutdown_connections 2024-09-13T08:50:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.750+0000 7f2a13e00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f29f0074930 0x7f29f0076df0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.750+0000 7f2a13e00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2a1415f000 0x7f2a14156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.750+0000 7f2a13e00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a140d01b0 0x7f2a140c6460 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.750+0000 7f2a13e00640 1 --2- 172.21.15.44:0/2865848983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a140b2590 0x7f2a141f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.750+0000 7f2a13e00640 1 -- 172.21.15.44:0/2865848983 >> 172.21.15.44:0/2865848983 conn(0x7f2a1409aa10 msgr2=0x7f2a140ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.750+0000 7f2a13e00640 1 -- 172.21.15.44:0/2865848983 shutdown_connections 2024-09-13T08:50:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.750+0000 7f2a13e00640 1 -- 172.21.15.44:0/2865848983 wait complete. 2024-09-13T08:50:46.767 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:50:46.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 -- 172.21.15.44:0/2773756967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48080d20f0 msgr2=0x7f48080ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:46.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 --2- 172.21.15.44:0/2773756967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48080d20f0 0x7f48080ce670 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f48080cf7e0 tx=0x7f480001ca60 comp rx=0 tx=0).stop 2024-09-13T08:50:46.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 -- 172.21.15.44:0/2773756967 shutdown_connections 2024-09-13T08:50:46.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 --2- 172.21.15.44:0/2773756967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48080d20f0 0x7f48080ce670 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 --2- 172.21.15.44:0/2773756967 >> v2:172.21.15.129:3300/0 conn(0x7f48080b0d10 0x7f48080d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 --2- 172.21.15.44:0/2773756967 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f480815bd40 0x7f48080d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 -- 172.21.15.44:0/2773756967 >> 172.21.15.44:0/2773756967 conn(0x7f480809aa10 msgr2=0x7f480809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:46.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 -- 172.21.15.44:0/2773756967 shutdown_connections 2024-09-13T08:50:46.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 -- 172.21.15.44:0/2773756967 wait complete. 2024-09-13T08:50:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 Processor -- start 2024-09-13T08:50:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.918+0000 7f480e600640 1 -- start start 2024-09-13T08:50:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48080b0d10 0x7f4808159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:46.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480e600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48080d20f0 0x7f48080c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:46.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480e600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f480815bd40 0x7f48080b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:46.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48080cce60 con 0x7f48080b0d10 2024-09-13T08:50:46.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480e600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48080ccce0 con 0x7f480815bd40 2024-09-13T08:50:46.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480e600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f48080ccfe0 con 0x7f48080d20f0 2024-09-13T08:50:46.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48080b0d10 0x7f4808159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48080b0d10 0x7f4808159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45950/0 (socket says 172.21.15.44:45950) 2024-09-13T08:50:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480d200640 1 -- 172.21.15.44:0/921473643 learned_addr learned my addr 172.21.15.44:0/921473643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f4807e00640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48080d20f0 0x7f48080c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480dc00640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f480815bd40 0x7f48080b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480d200640 1 -- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48080d20f0 msgr2=0x7f48080c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:46.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480d200640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48080d20f0 0x7f48080c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480d200640 1 -- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f480815bd40 msgr2=0x7f48080b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:46.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480d200640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f480815bd40 0x7f48080b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:46.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480d200640 1 -- 172.21.15.44:0/921473643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480815a740 con 0x7f48080b0d10 2024-09-13T08:50:46.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480d200640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48080b0d10 0x7f4808159d90 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00b410 tx=0x7f47fc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:46.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f4804c00640 1 -- 172.21.15.44:0/921473643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc015020 con 0x7f48080b0d10 2024-09-13T08:50:46.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f4804c00640 1 -- 172.21.15.44:0/921473643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47fc011920 con 0x7f48080b0d10 2024-09-13T08:50:46.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f4804c00640 1 -- 172.21.15.44:0/921473643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc011d70 con 0x7f48080b0d10 2024-09-13T08:50:46.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480e600640 1 -- 172.21.15.44:0/921473643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480806e910 con 0x7f48080b0d10 2024-09-13T08:50:46.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480e600640 1 -- 172.21.15.44:0/921473643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480815c920 con 0x7f48080b0d10 2024-09-13T08:50:46.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f480e600640 1 -- 172.21.15.44:0/921473643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48080cdcc0 con 0x7f48080b0d10 2024-09-13T08:50:46.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.922+0000 7f4804c00640 1 -- 172.21.15.44:0/921473643 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f47fc018020 con 0x7f48080b0d10 2024-09-13T08:50:46.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.926+0000 7f4804c00640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47e8074930 0x7f47e8076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:46.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.926+0000 7f4804c00640 1 -- 172.21.15.44:0/921473643 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f47fc092ad0 con 0x7f48080b0d10 2024-09-13T08:50:46.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.926+0000 7f4807e00640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47e8074930 0x7f47e8076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:46.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.926+0000 7f4807e00640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47e8074930 0x7f47e8076df0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f48081fab90 tx=0x7f47f40091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:46.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:46.930+0000 7f4804c00640 1 -- 172.21.15.44:0/921473643 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f47fc01f020 con 0x7f48080b0d10 2024-09-13T08:50:47.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.102+0000 7f480e600640 1 -- 172.21.15.44:0/921473643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f48080b4d00 con 0x7f48080b0d10 2024-09-13T08:50:47.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.102+0000 7f4804c00640 1 -- 172.21.15.44:0/921473643 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f47fc05f130 con 0x7f48080b0d10 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 -- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47e8074930 msgr2=0x7f47e8076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47e8074930 0x7f47e8076df0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f48081fab90 tx=0x7f47f40091e0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 -- 172.21.15.44:0/921473643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48080b0d10 msgr2=0x7f4808159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48080b0d10 0x7f4808159d90 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00b410 tx=0x7f47fc00b8d0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 -- 172.21.15.44:0/921473643 shutdown_connections 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f47e8074930 0x7f47e8076df0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f480815bd40 0x7f48080b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48080d20f0 0x7f48080c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 --2- 172.21.15.44:0/921473643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48080b0d10 0x7f4808159d90 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 -- 172.21.15.44:0/921473643 >> 172.21.15.44:0/921473643 conn(0x7f480809aa10 msgr2=0x7f48080b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 -- 172.21.15.44:0/921473643 shutdown_connections 2024-09-13T08:50:47.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.106+0000 7f480e600640 1 -- 172.21.15.44:0/921473643 wait complete. 2024-09-13T08:50:47.125 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-13T08:50:47.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.274+0000 7f3d86a00640 1 -- 172.21.15.44:0/3697872252 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d800d29b0 msgr2=0x7f3d800b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.274+0000 7f3d86a00640 1 --2- 172.21.15.44:0/3697872252 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d800d29b0 0x7f3d800b1cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009840 tx=0x7f3d6801ef30 comp rx=0 tx=0).stop 2024-09-13T08:50:47.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 -- 172.21.15.44:0/3697872252 shutdown_connections 2024-09-13T08:50:47.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 --2- 172.21.15.44:0/3697872252 >> v2:172.21.15.129:3300/0 conn(0x7f3d800d04d0 0x7f3d800af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 --2- 172.21.15.44:0/3697872252 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d800d29b0 0x7f3d800b1cf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 -- 172.21.15.44:0/3697872252 >> 172.21.15.44:0/3697872252 conn(0x7f3d800d1e90 msgr2=0x7f3d800d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:47.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 -- 172.21.15.44:0/3697872252 shutdown_connections 2024-09-13T08:50:47.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 -- 172.21.15.44:0/3697872252 wait complete. 2024-09-13T08:50:47.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 Processor -- start 2024-09-13T08:50:47.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 -- start start 2024-09-13T08:50:47.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d800d00f0 0x7f3d8014d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:47.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d800d04d0 0x7f3d800c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d800d29b0 0x7f3d801f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3d800b0ed0 con 0x7f3d800d04d0 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3d800b0d50 con 0x7f3d800d29b0 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d86a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3d800af740 con 0x7f3d800d00f0 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d77e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d800d04d0 0x7f3d800c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d77e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d800d04d0 0x7f3d800c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45960/0 (socket says 172.21.15.44:45960) 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d77e00640 1 -- 172.21.15.44:0/20719552 learned_addr learned my addr 172.21.15.44:0/20719552 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d7fe00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d800d00f0 0x7f3d8014d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d84a00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d800d29b0 0x7f3d801f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d77e00640 1 -- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d800d00f0 msgr2=0x7f3d8014d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d77e00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d800d00f0 0x7f3d8014d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d77e00640 1 -- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d800d29b0 msgr2=0x7f3d801f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d77e00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d800d29b0 0x7f3d801f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d77e00640 1 -- 172.21.15.44:0/20719552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d800b5030 con 0x7f3d800d04d0 2024-09-13T08:50:47.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d77e00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d800d04d0 0x7f3d800c63c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f3d6801cb60 tx=0x7f3d68005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.278+0000 7f3d7cc00640 1 -- 172.21.15.44:0/20719552 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d680b5040 con 0x7f3d800d04d0 2024-09-13T08:50:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.282+0000 7f3d7cc00640 1 -- 172.21.15.44:0/20719552 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d680041e0 con 0x7f3d800d04d0 2024-09-13T08:50:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.282+0000 7f3d7cc00640 1 -- 172.21.15.44:0/20719552 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d680046d0 con 0x7f3d800d04d0 2024-09-13T08:50:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.282+0000 7f3d86a00640 1 -- 172.21.15.44:0/20719552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d800c5c10 con 0x7f3d800d04d0 2024-09-13T08:50:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.282+0000 7f3d86a00640 1 -- 172.21.15.44:0/20719552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d800b4cc0 con 0x7f3d800d04d0 2024-09-13T08:50:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.282+0000 7f3d86a00640 1 -- 172.21.15.44:0/20719552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d800af200 con 0x7f3d800d04d0 2024-09-13T08:50:47.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.282+0000 7f3d7cc00640 1 -- 172.21.15.44:0/20719552 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f3d680c0030 con 0x7f3d800d04d0 2024-09-13T08:50:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.282+0000 7f3d7cc00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d5c074930 0x7f3d5c076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.282+0000 7f3d7cc00640 1 -- 172.21.15.44:0/20719552 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f3d68130000 con 0x7f3d800d04d0 2024-09-13T08:50:47.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.282+0000 7f3d7fe00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d5c074930 0x7f3d5c076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:47.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.286+0000 7f3d7fe00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d5c074930 0x7f3d5c076df0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3d70007970 tx=0x7f3d70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:47.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.286+0000 7f3d7cc00640 1 -- 172.21.15.44:0/20719552 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3d680c7060 con 0x7f3d800d04d0 2024-09-13T08:50:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.466+0000 7f3d86a00640 1 -- 172.21.15.44:0/20719552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f3d80071620 con 0x7f3d800d04d0 2024-09-13T08:50:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.466+0000 7f3d7cc00640 1 -- 172.21.15.44:0/20719552 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v48) ==== 116+0+2 (secure 0 0 0) 0x7f3d68018a90 con 0x7f3d800d04d0 2024-09-13T08:50:47.465 INFO:teuthology.orchestra.run.smithi044.stdout:4 2024-09-13T08:50:47.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 -- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d5c074930 msgr2=0x7f3d5c076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d5c074930 0x7f3d5c076df0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3d70007970 tx=0x7f3d70009040 comp rx=0 tx=0).stop 2024-09-13T08:50:47.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 -- 172.21.15.44:0/20719552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d800d04d0 msgr2=0x7f3d800c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d800d04d0 0x7f3d800c63c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f3d6801cb60 tx=0x7f3d68005620 comp rx=0 tx=0).stop 2024-09-13T08:50:47.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 -- 172.21.15.44:0/20719552 shutdown_connections 2024-09-13T08:50:47.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3d5c074930 0x7f3d5c076df0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d800d29b0 0x7f3d801f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d800d04d0 0x7f3d800c63c0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 --2- 172.21.15.44:0/20719552 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3d800d00f0 0x7f3d8014d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 -- 172.21.15.44:0/20719552 >> 172.21.15.44:0/20719552 conn(0x7f3d800d1e90 msgr2=0x7f3d8015c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:47.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 -- 172.21.15.44:0/20719552 shutdown_connections 2024-09-13T08:50:47.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.470+0000 7f3d86a00640 1 -- 172.21.15.44:0/20719552 wait complete. 2024-09-13T08:50:47.489 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:50:47.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 -- 172.21.15.44:0/3722093246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf406cc20 msgr2=0x7feaf406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 --2- 172.21.15.44:0/3722093246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf406cc20 0x7feaf406d0d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7feaec009c80 tx=0x7feaec01cda0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 -- 172.21.15.44:0/3722093246 shutdown_connections 2024-09-13T08:50:47.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 --2- 172.21.15.44:0/3722093246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf406cc20 0x7feaf406d0d0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 --2- 172.21.15.44:0/3722093246 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feaf415be10 0x7feaf41602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 -- 172.21.15.44:0/3722093246 >> 172.21.15.44:0/3722093246 conn(0x7feaf40cd740 msgr2=0x7feaf40cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:47.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 -- 172.21.15.44:0/3722093246 shutdown_connections 2024-09-13T08:50:47.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 -- 172.21.15.44:0/3722093246 wait complete. 2024-09-13T08:50:47.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 Processor -- start 2024-09-13T08:50:47.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 -- start start 2024-09-13T08:50:47.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaf406cc20 0x7feaf40c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf40cbeb0 0x7feaf41f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feaf415be10 0x7feaf4165e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feaf4168f40 con 0x7feaf40cbeb0 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feaf4168dc0 con 0x7feaf406cc20 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feafb000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7feaf41690c0 con 0x7feaf415be10 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feaf3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf40cbeb0 0x7feaf41f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feaf3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf40cbeb0 0x7feaf41f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45976/0 (socket says 172.21.15.44:45976) 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feaf3400640 1 -- 172.21.15.44:0/3864988837 learned_addr learned my addr 172.21.15.44:0/3864988837 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feaf3e00640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaf406cc20 0x7feaf40c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feaf9000640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feaf415be10 0x7feaf4165e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.642+0000 7feaf3400640 1 -- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feaf415be10 msgr2=0x7feaf4165e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaf3400640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feaf415be10 0x7feaf4165e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaf3400640 1 -- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaf406cc20 msgr2=0x7feaf40c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaf3400640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaf406cc20 0x7feaf40c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaf3400640 1 -- 172.21.15.44:0/3864988837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf4158530 con 0x7feaf40cbeb0 2024-09-13T08:50:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaf3400640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf40cbeb0 0x7feaf41f7f90 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7feae000c430 tx=0x7feae000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaebe00640 1 -- 172.21.15.44:0/3864988837 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0017070 con 0x7feaf40cbeb0 2024-09-13T08:50:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaebe00640 1 -- 172.21.15.44:0/3864988837 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feae00053e0 con 0x7feaf40cbeb0 2024-09-13T08:50:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaebe00640 1 -- 172.21.15.44:0/3864988837 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0003860 con 0x7feaf40cbeb0 2024-09-13T08:50:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feafb000640 1 -- 172.21.15.44:0/3864988837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf41f7790 con 0x7feaf40cbeb0 2024-09-13T08:50:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feafb000640 1 -- 172.21.15.44:0/3864988837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf41f84d0 con 0x7feaf40cbeb0 2024-09-13T08:50:47.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feafb000640 1 -- 172.21.15.44:0/3864988837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf406d0d0 con 0x7feaf40cbeb0 2024-09-13T08:50:47.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaebe00640 1 -- 172.21.15.44:0/3864988837 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7feae001a070 con 0x7feaf40cbeb0 2024-09-13T08:50:47.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaebe00640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fead4074930 0x7fead4076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:47.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.646+0000 7feaebe00640 1 -- 172.21.15.44:0/3864988837 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7feae0092020 con 0x7feaf40cbeb0 2024-09-13T08:50:47.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.650+0000 7feaf3e00640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fead4074930 0x7fead4076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:47.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.650+0000 7feaf3e00640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fead4074930 0x7fead4076df0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7feaf41568a0 tx=0x7feae400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:47.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.650+0000 7feaebe00640 1 -- 172.21.15.44:0/3864988837 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7feae0001130 con 0x7feaf40cbeb0 2024-09-13T08:50:47.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.826+0000 7feafb000640 1 -- 172.21.15.44:0/3864988837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7feaf4073a40 con 0x7feaf40cbeb0 2024-09-13T08:50:47.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.830+0000 7feaebe00640 1 -- 172.21.15.44:0/3864988837 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v49)=0 v49) ==== 131+0+0 (secure 0 0 0) 0x7feae005e680 con 0x7feaf40cbeb0 2024-09-13T08:50:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 -- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fead4074930 msgr2=0x7fead4076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fead4074930 0x7fead4076df0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7feaf41568a0 tx=0x7feae400b2e0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 -- 172.21.15.44:0/3864988837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf40cbeb0 msgr2=0x7feaf41f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf40cbeb0 0x7feaf41f7f90 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7feae000c430 tx=0x7feae000c8f0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 -- 172.21.15.44:0/3864988837 shutdown_connections 2024-09-13T08:50:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fead4074930 0x7fead4076df0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feaf415be10 0x7feaf4165e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf40cbeb0 0x7feaf41f7f90 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 --2- 172.21.15.44:0/3864988837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaf406cc20 0x7feaf40c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 -- 172.21.15.44:0/3864988837 >> 172.21.15.44:0/3864988837 conn(0x7feaf40cd740 msgr2=0x7feaf40c71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:47.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 -- 172.21.15.44:0/3864988837 shutdown_connections 2024-09-13T08:50:47.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:47.834+0000 7feafb000640 1 -- 172.21.15.44:0/3864988837 wait complete. 2024-09-13T08:50:47.852 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-13T08:50:47.853 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:47.852+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:47.853 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:47.852+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:47.853 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:47.852+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:47.853 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:47.852+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:47.853 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:47.852+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 -- 172.21.15.44:0/230660443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b815e360 msgr2=0x7f39b80adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 --2- 172.21.15.44:0/230660443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b815e360 0x7f39b80adf50 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009900 tx=0x7f39a801c850 comp rx=0 tx=0).stop 2024-09-13T08:50:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 -- 172.21.15.44:0/230660443 shutdown_connections 2024-09-13T08:50:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 --2- 172.21.15.44:0/230660443 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39b815e740 0x7f39b815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 --2- 172.21.15.44:0/230660443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b815e360 0x7f39b80adf50 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 -- 172.21.15.44:0/230660443 >> 172.21.15.44:0/230660443 conn(0x7f39b80cfbc0 msgr2=0x7f39b80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 -- 172.21.15.44:0/230660443 shutdown_connections 2024-09-13T08:50:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 -- 172.21.15.44:0/230660443 wait complete. 2024-09-13T08:50:48.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 Processor -- start 2024-09-13T08:50:48.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 -- start start 2024-09-13T08:50:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39b80aef60 0x7f39b81f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b815e360 0x7f39b814d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39b815e740 0x7f39b81fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81680b0 con 0x7f39b815e360 2024-09-13T08:50:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8167f30 con 0x7f39b80aef60 2024-09-13T08:50:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bec00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f39b8168230 con 0x7f39b815e740 2024-09-13T08:50:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39af400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b815e360 0x7f39b814d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39af400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b815e360 0x7f39b814d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46002/0 (socket says 172.21.15.44:46002) 2024-09-13T08:50:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39af400640 1 -- 172.21.15.44:0/1133430935 learned_addr learned my addr 172.21.15.44:0/1133430935 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39b7e00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39b80aef60 0x7f39b81f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39bcc00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39b815e740 0x7f39b81fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39af400640 1 -- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39b815e740 msgr2=0x7f39b81fbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39af400640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39b815e740 0x7f39b81fbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.006+0000 7f39af400640 1 -- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39b80aef60 msgr2=0x7f39b81f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39af400640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39b80aef60 0x7f39b81f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39af400640 1 -- 172.21.15.44:0/1133430935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b81fc3b0 con 0x7f39b815e360 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39bcc00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39b815e740 0x7f39b81fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39af400640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b815e360 0x7f39b814d500 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f39a000c530 tx=0x7f39a000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39b4c00640 1 -- 172.21.15.44:0/1133430935 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0017070 con 0x7f39b815e360 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39b4c00640 1 -- 172.21.15.44:0/1133430935 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39a0003600 con 0x7f39b815e360 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39bec00640 1 -- 172.21.15.44:0/1133430935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b81f8450 con 0x7f39b815e360 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39b4c00640 1 -- 172.21.15.44:0/1133430935 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0003a50 con 0x7f39b815e360 2024-09-13T08:50:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39bec00640 1 -- 172.21.15.44:0/1133430935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b81f8a10 con 0x7f39b815e360 2024-09-13T08:50:48.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39bec00640 1 -- 172.21.15.44:0/1133430935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b8069410 con 0x7f39b815e360 2024-09-13T08:50:48.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39b4c00640 1 -- 172.21.15.44:0/1133430935 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f39a001a070 con 0x7f39b815e360 2024-09-13T08:50:48.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39b4c00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3994074930 0x7f3994076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.010+0000 7f39b4c00640 1 -- 172.21.15.44:0/1133430935 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f39a0091fd0 con 0x7f39b815e360 2024-09-13T08:50:48.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.014+0000 7f39b7e00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3994074930 0x7f3994076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.014+0000 7f39b7e00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3994074930 0x7f3994076df0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f39a80af000 tx=0x7f39a801b090 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.014+0000 7f39b4c00640 1 -- 172.21.15.44:0/1133430935 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f39b81fc3b0 con 0x7f39b815e360 2024-09-13T08:50:48.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.218+0000 7f39bec00640 1 -- 172.21.15.44:0/1133430935 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f39b80d0850 con 0x7f3994074930 2024-09-13T08:50:48.226 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:48.224+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:48.226 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:48.224+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:48.226 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:48.224+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:48.226 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:48.224+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:48.226 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:50:48.224+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:50:48.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39b4c00640 1 -- 172.21.15.44:0/1133430935 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f39b80d0850 con 0x7f3994074930 2024-09-13T08:50:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 -- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3994074930 msgr2=0x7f3994076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3994074930 0x7f3994076df0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f39a80af000 tx=0x7f39a801b090 comp rx=0 tx=0).stop 2024-09-13T08:50:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 -- 172.21.15.44:0/1133430935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b815e360 msgr2=0x7f39b814d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b815e360 0x7f39b814d500 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f39a000c530 tx=0x7f39a000c9f0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 -- 172.21.15.44:0/1133430935 shutdown_connections 2024-09-13T08:50:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3994074930 0x7f3994076df0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f39b815e740 0x7f39b81fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b815e360 0x7f39b814d500 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 --2- 172.21.15.44:0/1133430935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39b80aef60 0x7f39b81f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 -- 172.21.15.44:0/1133430935 >> 172.21.15.44:0/1133430935 conn(0x7f39b80cfbc0 msgr2=0x7f39b8157000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:48.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 -- 172.21.15.44:0/1133430935 shutdown_connections 2024-09-13T08:50:48.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.274+0000 7f39bec00640 1 -- 172.21.15.44:0/1133430935 wait complete. 2024-09-13T08:50:48.292 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:50:48.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.442+0000 7f85bf400640 1 -- 172.21.15.44:0/3114594283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b80b1650 msgr2=0x7f85b80b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.442+0000 7f85bf400640 1 --2- 172.21.15.44:0/3114594283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b80b1650 0x7f85b80b1a30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009840 tx=0x7f85a401c820 comp rx=0 tx=0).stop 2024-09-13T08:50:48.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.442+0000 7f85bf400640 1 -- 172.21.15.44:0/3114594283 shutdown_connections 2024-09-13T08:50:48.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.442+0000 7f85bf400640 1 --2- 172.21.15.44:0/3114594283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b80b1650 0x7f85b80b1a30 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.442+0000 7f85bf400640 1 -- 172.21.15.44:0/3114594283 >> 172.21.15.44:0/3114594283 conn(0x7f85b80d1dd0 msgr2=0x7f85b80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:48.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.442+0000 7f85bf400640 1 -- 172.21.15.44:0/3114594283 shutdown_connections 2024-09-13T08:50:48.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.442+0000 7f85bf400640 1 -- 172.21.15.44:0/3114594283 wait complete. 2024-09-13T08:50:48.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bf400640 1 Processor -- start 2024-09-13T08:50:48.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bf400640 1 -- start start 2024-09-13T08:50:48.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bf400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b80afb10 0x7f85b8152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bf400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85b80b1650 0x7f85b8151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b81605d0 0x7f85b8165710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bf400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f85b80b13a0 con 0x7f85b81605d0 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bf400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f85b80b1220 con 0x7f85b80afb10 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bf400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f85b816a550 con 0x7f85b80b1650 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b81605d0 0x7f85b8165710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b81605d0 0x7f85b8165710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46028/0 (socket says 172.21.15.44:46028) 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bd400640 1 -- 172.21.15.44:0/337091108 learned_addr learned my addr 172.21.15.44:0/337091108 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85b7e00640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85b80b1650 0x7f85b8151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bca00640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b80afb10 0x7f85b8152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bd400640 1 -- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85b80b1650 msgr2=0x7f85b8151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bd400640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85b80b1650 0x7f85b8151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bd400640 1 -- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b80afb10 msgr2=0x7f85b8152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bd400640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b80afb10 0x7f85b8152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bd400640 1 -- 172.21.15.44:0/337091108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b8154cb0 con 0x7f85b81605d0 2024-09-13T08:50:48.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bca00640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b80afb10 0x7f85b8152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:48.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85b7e00640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85b80b1650 0x7f85b8151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:48.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bd400640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b81605d0 0x7f85b8165710 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f85b000c780 tx=0x7f85b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:48.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85b4c00640 1 -- 172.21.15.44:0/337091108 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0004b70 con 0x7f85b81605d0 2024-09-13T08:50:48.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85b4c00640 1 -- 172.21.15.44:0/337091108 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85b0004e00 con 0x7f85b81605d0 2024-09-13T08:50:48.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bf400640 1 -- 172.21.15.44:0/337091108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b8151c70 con 0x7f85b81605d0 2024-09-13T08:50:48.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85b4c00640 1 -- 172.21.15.44:0/337091108 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b000aa90 con 0x7f85b81605d0 2024-09-13T08:50:48.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.446+0000 7f85bf400640 1 -- 172.21.15.44:0/337091108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b81541d0 con 0x7f85b81605d0 2024-09-13T08:50:48.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.450+0000 7f85bf400640 1 -- 172.21.15.44:0/337091108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b80affc0 con 0x7f85b81605d0 2024-09-13T08:50:48.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.450+0000 7f85b4c00640 1 -- 172.21.15.44:0/337091108 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7f85b000ac30 con 0x7f85b81605d0 2024-09-13T08:50:48.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.450+0000 7f85b4c00640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8598074960 0x7f8598076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.450+0000 7f85b4c00640 1 -- 172.21.15.44:0/337091108 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f85b0093030 con 0x7f85b81605d0 2024-09-13T08:50:48.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.450+0000 7f85bca00640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8598074960 0x7f8598076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.450+0000 7f85bca00640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8598074960 0x7f8598076e20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f85ac000f70 tx=0x7f85ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:48.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.454+0000 7f85b4c00640 1 -- 172.21.15.44:0/337091108 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f85b0020070 con 0x7f85b81605d0 2024-09-13T08:50:48.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.654+0000 7f85bf400640 1 -- 172.21.15.44:0/337091108 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f85b80d1080 con 0x7f8598074960 2024-09-13T08:50:48.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.662+0000 7f85b4c00640 1 -- 172.21.15.44:0/337091108 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f85b80d1080 con 0x7f8598074960 2024-09-13T08:50:48.662 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881 2024-09-13T08:50:48.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 -- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8598074960 msgr2=0x7f8598076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8598074960 0x7f8598076e20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f85ac000f70 tx=0x7f85ac009040 comp rx=0 tx=0).stop 2024-09-13T08:50:48.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 -- 172.21.15.44:0/337091108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b81605d0 msgr2=0x7f85b8165710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b81605d0 0x7f85b8165710 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f85b000c780 tx=0x7f85b000cc40 comp rx=0 tx=0).stop 2024-09-13T08:50:48.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 -- 172.21.15.44:0/337091108 shutdown_connections 2024-09-13T08:50:48.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8598074960 0x7f8598076e20 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b81605d0 0x7f85b8165710 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f85b80b1650 0x7f85b8151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 --2- 172.21.15.44:0/337091108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b80afb10 0x7f85b8152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 -- 172.21.15.44:0/337091108 >> 172.21.15.44:0/337091108 conn(0x7f85b80d1dd0 msgr2=0x7f85b8071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:48.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 -- 172.21.15.44:0/337091108 shutdown_connections 2024-09-13T08:50:48.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.666+0000 7f85bf400640 1 -- 172.21.15.44:0/337091108 wait complete. 2024-09-13T08:50:48.683 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:50:48.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.834+0000 7fad73a00640 1 -- 172.21.15.44:0/1111074642 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad6c0cf2a0 msgr2=0x7fad6c0b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.834+0000 7fad73a00640 1 --2- 172.21.15.44:0/1111074642 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad6c0cf2a0 0x7fad6c0b2740 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fad580098a0 tx=0x7fad5801c840 comp rx=0 tx=0).stop 2024-09-13T08:50:48.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.834+0000 7fad73a00640 1 -- 172.21.15.44:0/1111074642 shutdown_connections 2024-09-13T08:50:48.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.834+0000 7fad73a00640 1 --2- 172.21.15.44:0/1111074642 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad6c0cf2a0 0x7fad6c0b2740 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.834+0000 7fad73a00640 1 -- 172.21.15.44:0/1111074642 >> 172.21.15.44:0/1111074642 conn(0x7fad6c0cd740 msgr2=0x7fad6c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:48.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.834+0000 7fad73a00640 1 -- 172.21.15.44:0/1111074642 shutdown_connections 2024-09-13T08:50:48.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.834+0000 7fad73a00640 1 -- 172.21.15.44:0/1111074642 wait complete. 2024-09-13T08:50:48.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 Processor -- start 2024-09-13T08:50:48.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 -- start start 2024-09-13T08:50:48.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad6c0b2c80 0x7fad6c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c0cf2a0 0x7fad6c1f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c15ac70 0x7fad6c0afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1690d0 con 0x7fad6c0cf2a0 2024-09-13T08:50:48.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c168f50 con 0x7fad6c15ac70 2024-09-13T08:50:48.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fad6c169250 con 0x7fad6c0b2c80 2024-09-13T08:50:48.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad6be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c0cf2a0 0x7fad6c1f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad6be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c0cf2a0 0x7fad6c1f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46030/0 (socket says 172.21.15.44:46030) 2024-09-13T08:50:48.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad6be00640 1 -- 172.21.15.44:0/3306013837 learned_addr learned my addr 172.21.15.44:0/3306013837 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad71a00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c15ac70 0x7fad6c0afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad71000640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad6c0b2c80 0x7fad6c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad6be00640 1 -- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad6c0b2c80 msgr2=0x7fad6c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad6be00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad6c0b2c80 0x7fad6c15c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad6be00640 1 -- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c15ac70 msgr2=0x7fad6c0afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad6be00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c15ac70 0x7fad6c0afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad6be00640 1 -- 172.21.15.44:0/3306013837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c0affa0 con 0x7fad6c0cf2a0 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad71a00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c15ac70 0x7fad6c0afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad6be00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c0cf2a0 0x7fad6c1f8440 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fad5801a470 tx=0x7fad58005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad68c00640 1 -- 172.21.15.44:0/3306013837 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad580b2040 con 0x7fad6c0cf2a0 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad68c00640 1 -- 172.21.15.44:0/3306013837 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad580043e0 con 0x7fad6c0cf2a0 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad68c00640 1 -- 172.21.15.44:0/3306013837 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad580052d0 con 0x7fad6c0cf2a0 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 -- 172.21.15.44:0/3306013837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c1f7980 con 0x7fad6c0cf2a0 2024-09-13T08:50:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 -- 172.21.15.44:0/3306013837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c06d640 con 0x7fad6c0cf2a0 2024-09-13T08:50:48.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.838+0000 7fad73a00640 1 -- 172.21.15.44:0/3306013837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6c0b2740 con 0x7fad6c0cf2a0 2024-09-13T08:50:48.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.842+0000 7fad68c00640 1 -- 172.21.15.44:0/3306013837 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 99477+0+0 (secure 0 0 0) 0x7fad580a5770 con 0x7fad6c0cf2a0 2024-09-13T08:50:48.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.842+0000 7fad68c00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad4c074930 0x7fad4c076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:50:48.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.842+0000 7fad68c00640 1 -- 172.21.15.44:0/3306013837 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fad5812c690 con 0x7fad6c0cf2a0 2024-09-13T08:50:48.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.842+0000 7fad71000640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad4c074930 0x7fad4c076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:50:48.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.842+0000 7fad71000640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad4c074930 0x7fad4c076df0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fad60001130 tx=0x7fad60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:50:48.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:48.846+0000 7fad68c00640 1 -- 172.21.15.44:0/3306013837 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fad58135070 con 0x7fad6c0cf2a0 2024-09-13T08:50:49.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.046+0000 7fad73a00640 1 -- 172.21.15.44:0/3306013837 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fad6c1f8980 con 0x7fad4c074930 2024-09-13T08:50:49.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad68c00640 1 -- 172.21.15.44:0/3306013837 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fad60001c70 con 0x7fad4c074930 2024-09-13T08:50:49.052 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881 2024-09-13T08:50:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 -- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad4c074930 msgr2=0x7fad4c076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad4c074930 0x7fad4c076df0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fad60001130 tx=0x7fad60009040 comp rx=0 tx=0).stop 2024-09-13T08:50:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 -- 172.21.15.44:0/3306013837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c0cf2a0 msgr2=0x7fad6c1f8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:50:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c0cf2a0 0x7fad6c1f8440 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fad5801a470 tx=0x7fad58005dd0 comp rx=0 tx=0).stop 2024-09-13T08:50:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 -- 172.21.15.44:0/3306013837 shutdown_connections 2024-09-13T08:50:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad4c074930 0x7fad4c076df0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad6c15ac70 0x7fad6c0afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c0cf2a0 0x7fad6c1f8440 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 --2- 172.21.15.44:0/3306013837 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad6c0b2c80 0x7fad6c15c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:50:49.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 -- 172.21.15.44:0/3306013837 >> 172.21.15.44:0/3306013837 conn(0x7fad6c0cd740 msgr2=0x7fad6c0b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:50:49.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 -- 172.21.15.44:0/3306013837 shutdown_connections 2024-09-13T08:50:49.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:50:49.054+0000 7fad73a00640 1 -- 172.21.15.44:0/3306013837 wait complete. 2024-09-13T08:50:49.073 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881 2024-09-13T08:50:49.073 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:50:49.073 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:49.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:49.082 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:50:58.742 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:50:58.742 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:50:58.742 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.59852 s, 112 MB/s 2024-09-13T08:50:58.743 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:50:58.743 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:50:58.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:50:58.752 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:51:02.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:51:02.837 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:51:07.724 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:51:07.724 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:51:07.724 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.92031 s, 120 MB/s 2024-09-13T08:51:07.725 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:51:07.725 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:51:07.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:51:07.734 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:51:17.284 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:51:17.284 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:51:17.284 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.49622 s, 113 MB/s 2024-09-13T08:51:17.285 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-13T08:51:17.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.438+0000 7fd463600640 1 -- 172.21.15.44:0/2231734094 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd45c06d330 msgr2=0x7fd45c15fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:17.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.438+0000 7fd463600640 1 --2- 172.21.15.44:0/2231734094 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd45c06d330 0x7fd45c15fc60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd454009cb0 tx=0x7fd45401cd70 comp rx=0 tx=0).stop 2024-09-13T08:51:17.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.438+0000 7fd463600640 1 -- 172.21.15.44:0/2231734094 shutdown_connections 2024-09-13T08:51:17.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.438+0000 7fd463600640 1 --2- 172.21.15.44:0/2231734094 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd45c06d330 0x7fd45c15fc60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.438+0000 7fd463600640 1 --2- 172.21.15.44:0/2231734094 >> v2:172.21.15.129:3300/0 conn(0x7fd45c15bcb0 0x7fd45c0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.438+0000 7fd463600640 1 -- 172.21.15.44:0/2231734094 >> 172.21.15.44:0/2231734094 conn(0x7fd45c06dd30 msgr2=0x7fd45c06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:17.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.438+0000 7fd463600640 1 -- 172.21.15.44:0/2231734094 shutdown_connections 2024-09-13T08:51:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.438+0000 7fd463600640 1 -- 172.21.15.44:0/2231734094 wait complete. 2024-09-13T08:51:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd463600640 1 Processor -- start 2024-09-13T08:51:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd463600640 1 -- start start 2024-09-13T08:51:17.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd463600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd45c06d330 0x7fd45c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd463600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c15bcb0 0x7fd45c1487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd463600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd45c15f7f0 0x7fd45c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd463600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c168d10 con 0x7fd45c15bcb0 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd463600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c168b90 con 0x7fd45c06d330 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd463600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd45c168e90 con 0x7fd45c15f7f0 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd45be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c15bcb0 0x7fd45c1487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd45be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c15bcb0 0x7fd45c1487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56580/0 (socket says 172.21.15.44:56580) 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd45be00640 1 -- 172.21.15.44:0/1276587449 learned_addr learned my addr 172.21.15.44:0/1276587449 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd460c00640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd45c06d330 0x7fd45c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd461600640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd45c15f7f0 0x7fd45c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd45be00640 1 -- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd45c15f7f0 msgr2=0x7fd45c1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd45be00640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd45c15f7f0 0x7fd45c1572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd45be00640 1 -- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd45c06d330 msgr2=0x7fd45c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd45be00640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd45c06d330 0x7fd45c159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd45be00640 1 -- 172.21.15.44:0/1276587449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c15a1a0 con 0x7fd45c15bcb0 2024-09-13T08:51:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd45be00640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c15bcb0 0x7fd45c1487d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fd44800c3c0 tx=0x7fd44800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd458c00640 1 -- 172.21.15.44:0/1276587449 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448017070 con 0x7fd45c15bcb0 2024-09-13T08:51:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd458c00640 1 -- 172.21.15.44:0/1276587449 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4480053c0 con 0x7fd45c15bcb0 2024-09-13T08:51:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd463600640 1 -- 172.21.15.44:0/1276587449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45c15a4e0 con 0x7fd45c15bcb0 2024-09-13T08:51:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.442+0000 7fd458c00640 1 -- 172.21.15.44:0/1276587449 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448003860 con 0x7fd45c15bcb0 2024-09-13T08:51:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.446+0000 7fd463600640 1 -- 172.21.15.44:0/1276587449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45c15aaa0 con 0x7fd45c15bcb0 2024-09-13T08:51:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.446+0000 7fd463600640 1 -- 172.21.15.44:0/1276587449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45c15fc60 con 0x7fd45c15bcb0 2024-09-13T08:51:17.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.446+0000 7fd458c00640 1 -- 172.21.15.44:0/1276587449 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 46) ==== 99579+0+0 (secure 0 0 0) 0x7fd44801a070 con 0x7fd45c15bcb0 2024-09-13T08:51:17.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.446+0000 7fd458c00640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd43c074a50 0x7fd43c076f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:17.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.446+0000 7fd458c00640 1 -- 172.21.15.44:0/1276587449 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fd448091fc0 con 0x7fd45c15bcb0 2024-09-13T08:51:17.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.446+0000 7fd460c00640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd43c074a50 0x7fd43c076f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:17.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.450+0000 7fd460c00640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd43c074a50 0x7fd43c076f10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd450009be0 tx=0x7fd4500091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:17.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.450+0000 7fd458c00640 1 -- 172.21.15.44:0/1276587449 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd44805e620 con 0x7fd45c15bcb0 2024-09-13T08:51:17.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.662+0000 7fd463600640 1 -- 172.21.15.44:0/1276587449 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fd45c15e5d0 con 0x7fd43c074a50 2024-09-13T08:51:17.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.674+0000 7fd458c00640 1 -- 172.21.15.44:0/1276587449 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd45c15e5d0 con 0x7fd43c074a50 2024-09-13T08:51:17.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 -- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd43c074a50 msgr2=0x7fd43c076f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:17.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd43c074a50 0x7fd43c076f10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd450009be0 tx=0x7fd4500091e0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 -- 172.21.15.44:0/1276587449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c15bcb0 msgr2=0x7fd45c1487d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:17.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c15bcb0 0x7fd45c1487d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fd44800c3c0 tx=0x7fd44800c880 comp rx=0 tx=0).stop 2024-09-13T08:51:17.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 -- 172.21.15.44:0/1276587449 shutdown_connections 2024-09-13T08:51:17.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd43c074a50 0x7fd43c076f10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd45c15f7f0 0x7fd45c1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c15bcb0 0x7fd45c1487d0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 --2- 172.21.15.44:0/1276587449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd45c06d330 0x7fd45c159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 -- 172.21.15.44:0/1276587449 >> 172.21.15.44:0/1276587449 conn(0x7fd45c06dd30 msgr2=0x7fd45c0708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:17.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 -- 172.21.15.44:0/1276587449 shutdown_connections 2024-09-13T08:51:17.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.678+0000 7fd463600640 1 -- 172.21.15.44:0/1276587449 wait complete. 2024-09-13T08:51:17.695 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:51:17.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.846+0000 7fad46400640 1 -- 172.21.15.44:0/3387570727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad400b4bd0 msgr2=0x7fad400b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:17.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.846+0000 7fad46400640 1 --2- 172.21.15.44:0/3387570727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad400b4bd0 0x7fad400b2bf0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fad4006db60 tx=0x7fad2801c8c0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.846+0000 7fad46400640 1 -- 172.21.15.44:0/3387570727 shutdown_connections 2024-09-13T08:51:17.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.846+0000 7fad46400640 1 --2- 172.21.15.44:0/3387570727 >> v2:172.21.15.129:3300/0 conn(0x7fad400b3130 0x7fad400cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.846+0000 7fad46400640 1 --2- 172.21.15.44:0/3387570727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad400b4bd0 0x7fad400b2bf0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.846+0000 7fad46400640 1 -- 172.21.15.44:0/3387570727 >> 172.21.15.44:0/3387570727 conn(0x7fad400b18f0 msgr2=0x7fad400b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:17.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.846+0000 7fad46400640 1 -- 172.21.15.44:0/3387570727 shutdown_connections 2024-09-13T08:51:17.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.846+0000 7fad46400640 1 -- 172.21.15.44:0/3387570727 wait complete. 2024-09-13T08:51:17.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.846+0000 7fad46400640 1 Processor -- start 2024-09-13T08:51:17.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.846+0000 7fad46400640 1 -- start start 2024-09-13T08:51:17.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad46400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad400b3130 0x7fad401f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:17.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad46400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad400b4bd0 0x7fad401fa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:17.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad46400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad4015b580 0x7fad400cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad46400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad40167ae0 con 0x7fad400b3130 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad46400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad40167960 con 0x7fad4015b580 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad46400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fad40167c60 con 0x7fad400b4bd0 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad400b3130 0x7fad401f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad400b3130 0x7fad401f8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56592/0 (socket says 172.21.15.44:56592) 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3f400640 1 -- 172.21.15.44:0/2409664132 learned_addr learned my addr 172.21.15.44:0/2409664132 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad37e00640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad400b4bd0 0x7fad401fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3fe00640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad4015b580 0x7fad400cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3f400640 1 -- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad400b4bd0 msgr2=0x7fad401fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3f400640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad400b4bd0 0x7fad401fa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3f400640 1 -- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad4015b580 msgr2=0x7fad400cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3f400640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad4015b580 0x7fad400cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3f400640 1 -- 172.21.15.44:0/2409664132 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad401fafc0 con 0x7fad400b3130 2024-09-13T08:51:17.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad37e00640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad400b4bd0 0x7fad401fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:51:17.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3fe00640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad4015b580 0x7fad400cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:51:17.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad3f400640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad400b3130 0x7fad401f8940 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fad3000c3c0 tx=0x7fad3000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:17.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad37400640 1 -- 172.21.15.44:0/2409664132 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30017070 con 0x7fad400b3130 2024-09-13T08:51:17.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad37400640 1 -- 172.21.15.44:0/2409664132 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad300053c0 con 0x7fad400b3130 2024-09-13T08:51:17.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad37400640 1 -- 172.21.15.44:0/2409664132 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30003860 con 0x7fad400b3130 2024-09-13T08:51:17.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad46400640 1 -- 172.21.15.44:0/2409664132 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad401fb2a0 con 0x7fad400b3130 2024-09-13T08:51:17.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad46400640 1 -- 172.21.15.44:0/2409664132 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad401fb860 con 0x7fad400b3130 2024-09-13T08:51:17.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad46400640 1 -- 172.21.15.44:0/2409664132 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad40069410 con 0x7fad400b3130 2024-09-13T08:51:17.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.850+0000 7fad37400640 1 -- 172.21.15.44:0/2409664132 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 46) ==== 99579+0+0 (secure 0 0 0) 0x7fad3001a070 con 0x7fad400b3130 2024-09-13T08:51:17.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.854+0000 7fad37400640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad1c074a50 0x7fad1c076f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:17.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.854+0000 7fad37400640 1 -- 172.21.15.44:0/2409664132 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5908+0+0 (secure 0 0 0) 0x7fad300920f0 con 0x7fad400b3130 2024-09-13T08:51:17.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.854+0000 7fad37e00640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad1c074a50 0x7fad1c076f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:17.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.854+0000 7fad37e00640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad1c074a50 0x7fad1c076f10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fad280af000 tx=0x7fad2801b100 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:17.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:17.858+0000 7fad37400640 1 -- 172.21.15.44:0/2409664132 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fad3005eda0 con 0x7fad400b3130 2024-09-13T08:51:18.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.054+0000 7fad46400640 1 -- 172.21.15.44:0/2409664132 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fad400cc550 con 0x7fad1c074a50 2024-09-13T08:51:18.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad37400640 1 -- 172.21.15.44:0/2409664132 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fad400cc550 con 0x7fad1c074a50 2024-09-13T08:51:18.059 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881 2024-09-13T08:51:18.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 -- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad1c074a50 msgr2=0x7fad1c076f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad1c074a50 0x7fad1c076f10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fad280af000 tx=0x7fad2801b100 comp rx=0 tx=0).stop 2024-09-13T08:51:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 -- 172.21.15.44:0/2409664132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad400b3130 msgr2=0x7fad401f8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad400b3130 0x7fad401f8940 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fad3000c3c0 tx=0x7fad3000c880 comp rx=0 tx=0).stop 2024-09-13T08:51:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 -- 172.21.15.44:0/2409664132 shutdown_connections 2024-09-13T08:51:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad1c074a50 0x7fad1c076f10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad4015b580 0x7fad400cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad400b4bd0 0x7fad401fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 --2- 172.21.15.44:0/2409664132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad400b3130 0x7fad401f8940 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 -- 172.21.15.44:0/2409664132 >> 172.21.15.44:0/2409664132 conn(0x7fad400b18f0 msgr2=0x7fad400b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 -- 172.21.15.44:0/2409664132 shutdown_connections 2024-09-13T08:51:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:18.062+0000 7fad46400640 1 -- 172.21.15.44:0/2409664132 wait complete. 2024-09-13T08:51:18.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:51:18.080 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881 2024-09-13T08:51:18.080 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:51:18.090 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881 2024-09-13T08:51:18.090 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3055190016" 2024-09-13T08:51:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:18.091 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-13T08:51:20.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:51:20.092 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881 2024-09-13T08:51:20.092 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:51:20.103 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/a394d3b5-bc51-4d71-a0ac-2956ea0a5881 2024-09-13T08:51:20.103 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3221225472" 2024-09-13T08:51:20.104 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:20.104 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-13T08:51:20.105 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone45030 2024-09-13T08:51:20.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.310+0000 7f4012800640 1 -- 172.21.15.44:0/2957479256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400c0b27a0 msgr2=0x7f400c0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:20.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.310+0000 7f4012800640 1 --2- 172.21.15.44:0/2957479256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400c0b27a0 0x7f400c0ce0e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f400c0cc410 tx=0x7f3ff401c7a0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.310+0000 7f4012800640 1 -- 172.21.15.44:0/2957479256 shutdown_connections 2024-09-13T08:51:20.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.310+0000 7f4012800640 1 --2- 172.21.15.44:0/2957479256 >> v2:172.21.15.129:3300/0 conn(0x7f400c0ce620 0x7f400c0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.310+0000 7f4012800640 1 --2- 172.21.15.44:0/2957479256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400c0b27a0 0x7f400c0ce0e0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.310+0000 7f4012800640 1 -- 172.21.15.44:0/2957479256 >> 172.21.15.44:0/2957479256 conn(0x7f400c0d1e30 msgr2=0x7f400c0d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:20.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.310+0000 7f4012800640 1 -- 172.21.15.44:0/2957479256 shutdown_connections 2024-09-13T08:51:20.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.310+0000 7f4012800640 1 -- 172.21.15.44:0/2957479256 wait complete. 2024-09-13T08:51:20.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4012800640 1 Processor -- start 2024-09-13T08:51:20.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4012800640 1 -- start start 2024-09-13T08:51:20.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4012800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400c0b27a0 0x7f400c1558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:20.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4012800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f400c0ce620 0x7f400c1563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:20.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4012800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f400c15f640 0x7f400c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:20.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4012800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f400c0b1010 con 0x7f400c0b27a0 2024-09-13T08:51:20.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4012800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f400c0b0e90 con 0x7f400c0ce620 2024-09-13T08:51:20.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4012800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f400c169390 con 0x7f400c15f640 2024-09-13T08:51:20.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400c0b27a0 0x7f400c1558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:20.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400c0b27a0 0x7f400c1558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56612/0 (socket says 172.21.15.44:56612) 2024-09-13T08:51:20.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400b400640 1 -- 172.21.15.44:0/1200344434 learned_addr learned my addr 172.21.15.44:0/1200344434 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:20.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4003e00640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f400c0ce620 0x7f400c1563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400be00640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f400c15f640 0x7f400c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400b400640 1 -- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f400c15f640 msgr2=0x7f400c1568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400b400640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f400c15f640 0x7f400c1568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400b400640 1 -- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f400c0ce620 msgr2=0x7f400c1563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400b400640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f400c0ce620 0x7f400c1563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400b400640 1 -- 172.21.15.44:0/1200344434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400c0711f0 con 0x7f400c0b27a0 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400be00640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f400c15f640 0x7f400c1568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f400b400640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400c0b27a0 0x7f400c1558b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00c440 tx=0x7f3ffc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4003400640 1 -- 172.21.15.44:0/1200344434 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc017070 con 0x7f400c0b27a0 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4003400640 1 -- 172.21.15.44:0/1200344434 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ffc003600 con 0x7f400c0b27a0 2024-09-13T08:51:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4003400640 1 -- 172.21.15.44:0/1200344434 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc003a30 con 0x7f400c0b27a0 2024-09-13T08:51:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4012800640 1 -- 172.21.15.44:0/1200344434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400c14dde0 con 0x7f400c0b27a0 2024-09-13T08:51:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.314+0000 7f4012800640 1 -- 172.21.15.44:0/1200344434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f400c1f59a0 con 0x7f400c0b27a0 2024-09-13T08:51:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.318+0000 7f4012800640 1 -- 172.21.15.44:0/1200344434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f400c0ce0e0 con 0x7f400c0b27a0 2024-09-13T08:51:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.318+0000 7f4003400640 1 -- 172.21.15.44:0/1200344434 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 46) ==== 99579+0+0 (secure 0 0 0) 0x7f3ffc01a070 con 0x7f400c0b27a0 2024-09-13T08:51:20.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.318+0000 7f4003400640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fe8074a50 0x7f3fe8076f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:20.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.318+0000 7f4003400640 1 -- 172.21.15.44:0/1200344434 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f3ffc091fe0 con 0x7f400c0b27a0 2024-09-13T08:51:20.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.318+0000 7f4003e00640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fe8074a50 0x7f3fe8076f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:20.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.318+0000 7f4003e00640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fe8074a50 0x7f3fe8076f10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40af000 tx=0x7f3ff401afe0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:20.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.322+0000 7f4003400640 1 -- 172.21.15.44:0/1200344434 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3ffc09b070 con 0x7f400c0b27a0 2024-09-13T08:51:20.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.530+0000 7f4012800640 1 -- 172.21.15.44:0/1200344434 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone45030", "target": ["mon-mgr", ""]}) -- 0x7f400c15ddf0 con 0x7f3fe8074a50 2024-09-13T08:51:20.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4003400640 1 -- 172.21.15.44:0/1200344434 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f400c15ddf0 con 0x7f3fe8074a50 2024-09-13T08:51:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 -- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fe8074a50 msgr2=0x7f3fe8076f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fe8074a50 0x7f3fe8076f10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40af000 tx=0x7f3ff401afe0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 -- 172.21.15.44:0/1200344434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400c0b27a0 msgr2=0x7f400c1558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:20.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400c0b27a0 0x7f400c1558b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00c440 tx=0x7f3ffc00c900 comp rx=0 tx=0).stop 2024-09-13T08:51:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 -- 172.21.15.44:0/1200344434 shutdown_connections 2024-09-13T08:51:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3fe8074a50 0x7f3fe8076f10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f400c15f640 0x7f400c1568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f400c0ce620 0x7f400c1563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 --2- 172.21.15.44:0/1200344434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400c0b27a0 0x7f400c1558b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 -- 172.21.15.44:0/1200344434 >> 172.21.15.44:0/1200344434 conn(0x7f400c0d1e30 msgr2=0x7f400c15e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 -- 172.21.15.44:0/1200344434 shutdown_connections 2024-09-13T08:51:20.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.602+0000 7f4012800640 1 -- 172.21.15.44:0/1200344434 wait complete. 2024-09-13T08:51:20.603 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:20.607+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:20.603 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:20.607+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:20.603 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:20.607+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:20.603 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:20.607+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:20.603 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:20.607+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:20.619 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:20.623+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:20.619 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:20.623+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:20.620 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:20.623+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:20.620 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:20.623+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:20.620 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:20.623+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:20.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15351 2024-09-13T08:51:20.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.782+0000 7f4d2d800640 1 -- 172.21.15.44:0/3455703006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d280b2a00 msgr2=0x7f4d280b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.782+0000 7f4d2d800640 1 --2- 172.21.15.44:0/3455703006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d280b2a00 0x7f4d280b39e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009900 tx=0x7f4d1001c8b0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.782+0000 7f4d2d800640 1 -- 172.21.15.44:0/3455703006 shutdown_connections 2024-09-13T08:51:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.782+0000 7f4d2d800640 1 --2- 172.21.15.44:0/3455703006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d280b2a00 0x7f4d280b39e0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.782+0000 7f4d2d800640 1 --2- 172.21.15.44:0/3455703006 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d2815f340 0x7f4d280b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.782+0000 7f4d2d800640 1 -- 172.21.15.44:0/3455703006 >> 172.21.15.44:0/3455703006 conn(0x7f4d280d1e00 msgr2=0x7f4d280d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.782+0000 7f4d2d800640 1 -- 172.21.15.44:0/3455703006 shutdown_connections 2024-09-13T08:51:20.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.782+0000 7f4d2d800640 1 -- 172.21.15.44:0/3455703006 wait complete. 2024-09-13T08:51:20.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.782+0000 7f4d2d800640 1 Processor -- start 2024-09-13T08:51:20.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.782+0000 7f4d2d800640 1 -- start start 2024-09-13T08:51:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d2d800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d280b2a00 0x7f4d281664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d2d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d280ccf10 0x7f4d2815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:20.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d2d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d2815f340 0x7f4d2806f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d2d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28168fc0 con 0x7f4d2815f340 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d2d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28168e40 con 0x7f4d280ccf10 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d2d800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4d28169140 con 0x7f4d280b2a00 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d27400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d2815f340 0x7f4d2806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d27400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d2815f340 0x7f4d2806f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56628/0 (socket says 172.21.15.44:56628) 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d27400640 1 -- 172.21.15.44:0/507994900 learned_addr learned my addr 172.21.15.44:0/507994900 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d26a00640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d280b2a00 0x7f4d281664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d1e000640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d280ccf10 0x7f4d2815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d27400640 1 -- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d280b2a00 msgr2=0x7f4d281664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d27400640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d280b2a00 0x7f4d281664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d27400640 1 -- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d280ccf10 msgr2=0x7f4d2815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d27400640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d280ccf10 0x7f4d2815e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d27400640 1 -- 172.21.15.44:0/507994900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2815b050 con 0x7f4d2815f340 2024-09-13T08:51:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d27400640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d2815f340 0x7f4d2806f680 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000c7f0 tx=0x7f4d2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d1f400640 1 -- 172.21.15.44:0/507994900 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20003b80 con 0x7f4d2815f340 2024-09-13T08:51:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d1f400640 1 -- 172.21.15.44:0/507994900 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d20003d20 con 0x7f4d2815f340 2024-09-13T08:51:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d1f400640 1 -- 172.21.15.44:0/507994900 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000aab0 con 0x7f4d2815f340 2024-09-13T08:51:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d2d800640 1 -- 172.21.15.44:0/507994900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d28166280 con 0x7f4d2815f340 2024-09-13T08:51:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d2d800640 1 -- 172.21.15.44:0/507994900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2815b5b0 con 0x7f4d2815f340 2024-09-13T08:51:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d2d800640 1 -- 172.21.15.44:0/507994900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d2806ef50 con 0x7f4d2815f340 2024-09-13T08:51:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.786+0000 7f4d1f400640 1 -- 172.21.15.44:0/507994900 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 46) ==== 99579+0+0 (secure 0 0 0) 0x7f4d2000ac50 con 0x7f4d2815f340 2024-09-13T08:51:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.790+0000 7f4d1f400640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d04074a50 0x7f4d04076f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:20.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.790+0000 7f4d1f400640 1 -- 172.21.15.44:0/507994900 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f4d20092b80 con 0x7f4d2815f340 2024-09-13T08:51:20.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.790+0000 7f4d26a00640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d04074a50 0x7f4d04076f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:20.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.790+0000 7f4d26a00640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d04074a50 0x7f4d04076f10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009ad0 tx=0x7f4d180091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:20.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:20.794+0000 7f4d1f400640 1 -- 172.21.15.44:0/507994900 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4d2815b050 con 0x7f4d2815f340 2024-09-13T08:51:21.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.006+0000 7f4d2d800640 1 -- 172.21.15.44:0/507994900 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15351", "target": ["mon-mgr", ""]}) -- 0x7f4d28159c10 con 0x7f4d04074a50 2024-09-13T08:51:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.498+0000 7f4d1f400640 1 -- 172.21.15.44:0/507994900 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 47) ==== 99681+0+0 (secure 0 0 0) 0x7f4d2005b4f0 con 0x7f4d2815f340 2024-09-13T08:51:21.603 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:21.607+0000 7f8a58600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:21.603 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:21.607+0000 7f8a58600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:21.603 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:21.607+0000 7f8a58600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:21.603 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:21.607+0000 7f8a58600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:21.603 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:21.607+0000 7f8a58600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:21.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d1f400640 1 -- 172.21.15.44:0/507994900 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d18009720 con 0x7f4d04074a50 2024-09-13T08:51:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 -- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d04074a50 msgr2=0x7f4d04076f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d04074a50 0x7f4d04076f10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009ad0 tx=0x7f4d180091e0 comp rx=0 tx=0).stop 2024-09-13T08:51:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 -- 172.21.15.44:0/507994900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d2815f340 msgr2=0x7f4d2806f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d2815f340 0x7f4d2806f680 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000c7f0 tx=0x7f4d2000ccb0 comp rx=0 tx=0).stop 2024-09-13T08:51:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 -- 172.21.15.44:0/507994900 shutdown_connections 2024-09-13T08:51:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d04074a50 0x7f4d04076f10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d2815f340 0x7f4d2806f680 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d280ccf10 0x7f4d2815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 --2- 172.21.15.44:0/507994900 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d280b2a00 0x7f4d281664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 -- 172.21.15.44:0/507994900 >> 172.21.15.44:0/507994900 conn(0x7f4d280d1e00 msgr2=0x7f4d280ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 -- 172.21.15.44:0/507994900 shutdown_connections 2024-09-13T08:51:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.694+0000 7f4d2d800640 1 -- 172.21.15.44:0/507994900 wait complete. 2024-09-13T08:51:21.726 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone47392 2024-09-13T08:51:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.966+0000 7f1cbea00640 1 -- 172.21.15.44:0/3201410614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cb80b3190 msgr2=0x7f1cb80b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.966+0000 7f1cbea00640 1 --2- 172.21.15.44:0/3201410614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cb80b3190 0x7f1cb80b1450 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8074d80 tx=0x7f1cb001ca30 comp rx=0 tx=0).stop 2024-09-13T08:51:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.966+0000 7f1cbea00640 1 -- 172.21.15.44:0/3201410614 shutdown_connections 2024-09-13T08:51:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.966+0000 7f1cbea00640 1 --2- 172.21.15.44:0/3201410614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cb80b3190 0x7f1cb80b1450 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.966+0000 7f1cbea00640 1 --2- 172.21.15.44:0/3201410614 >> v2:172.21.15.129:3300/0 conn(0x7f1cb81617a0 0x7f1cb8162300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.966+0000 7f1cbea00640 1 -- 172.21.15.44:0/3201410614 >> 172.21.15.44:0/3201410614 conn(0x7f1cb80d1dd0 msgr2=0x7f1cb80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:21.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.966+0000 7f1cbea00640 1 -- 172.21.15.44:0/3201410614 shutdown_connections 2024-09-13T08:51:21.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.966+0000 7f1cbea00640 1 -- 172.21.15.44:0/3201410614 wait complete. 2024-09-13T08:51:21.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.966+0000 7f1cbea00640 1 Processor -- start 2024-09-13T08:51:21.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.966+0000 7f1cbea00640 1 -- start start 2024-09-13T08:51:21.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb80b3190 0x7f1cb8208550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:21.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cb81613c0 0x7f1cb8159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cb81617a0 0x7f1cb80c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cbea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb816b0b0 con 0x7f1cb81613c0 2024-09-13T08:51:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cbea00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb816af30 con 0x7f1cb80b3190 2024-09-13T08:51:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cbea00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb816b230 con 0x7f1cb81617a0 2024-09-13T08:51:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cb7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cb81613c0 0x7f1cb8159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cb7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cb81613c0 0x7f1cb8159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56634/0 (socket says 172.21.15.44:56634) 2024-09-13T08:51:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cb7400640 1 -- 172.21.15.44:0/2487667840 learned_addr learned my addr 172.21.15.44:0/2487667840 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cbca00640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cb81617a0 0x7f1cb80c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cb7400640 1 -- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cb81617a0 msgr2=0x7f1cb80c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cb7400640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cb81617a0 0x7f1cb80c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cb7400640 1 -- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb80b3190 msgr2=0x7f1cb8208550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cb7400640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb80b3190 0x7f1cb8208550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cb7400640 1 -- 172.21.15.44:0/2487667840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb8204bd0 con 0x7f1cb81613c0 2024-09-13T08:51:21.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cb7400640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cb81613c0 0x7f1cb8159530 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400b550 tx=0x7f1ca400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:21.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cafe00640 1 -- 172.21.15.44:0/2487667840 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4014070 con 0x7f1cb81613c0 2024-09-13T08:51:21.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cafe00640 1 -- 172.21.15.44:0/2487667840 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ca4010980 con 0x7f1cb81613c0 2024-09-13T08:51:21.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cafe00640 1 -- 172.21.15.44:0/2487667840 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4010dd0 con 0x7f1cb81613c0 2024-09-13T08:51:21.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cbea00640 1 -- 172.21.15.44:0/2487667840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb8204eb0 con 0x7f1cb81613c0 2024-09-13T08:51:21.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.970+0000 7f1cbea00640 1 -- 172.21.15.44:0/2487667840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb8205470 con 0x7f1cb81613c0 2024-09-13T08:51:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.974+0000 7f1cafe00640 1 -- 172.21.15.44:0/2487667840 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 99681+0+0 (secure 0 0 0) 0x7f1ca4017070 con 0x7f1cb81613c0 2024-09-13T08:51:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.978+0000 7f1cbea00640 1 -- 172.21.15.44:0/2487667840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb80b1450 con 0x7f1cb81613c0 2024-09-13T08:51:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.978+0000 7f1cafe00640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1c98074bc0 0x7f1c98077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.978+0000 7f1cafe00640 1 -- 172.21.15.44:0/2487667840 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f1ca4092c10 con 0x7f1cb81613c0 2024-09-13T08:51:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.978+0000 7f1cb7e00640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1c98074bc0 0x7f1c98077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.978+0000 7f1cb7e00640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1c98074bc0 0x7f1c98077080 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80015d0 tx=0x7f1ca8007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:21.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:21.982+0000 7f1cafe00640 1 -- 172.21.15.44:0/2487667840 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1ca405f940 con 0x7f1cb81613c0 2024-09-13T08:51:22.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:22.334+0000 7f1cbea00640 1 -- 172.21.15.44:0/2487667840 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone47392", "target": ["mon-mgr", ""]}) -- 0x7f1cb815b010 con 0x7f1c98074bc0 2024-09-13T08:51:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cafe00640 1 -- 172.21.15.44:0/2487667840 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1cb815b010 con 0x7f1c98074bc0 2024-09-13T08:51:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 -- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1c98074bc0 msgr2=0x7f1c98077080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1c98074bc0 0x7f1c98077080 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80015d0 tx=0x7f1ca8007a40 comp rx=0 tx=0).stop 2024-09-13T08:51:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 -- 172.21.15.44:0/2487667840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cb81613c0 msgr2=0x7f1cb8159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cb81613c0 0x7f1cb8159530 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400b550 tx=0x7f1ca400ba10 comp rx=0 tx=0).stop 2024-09-13T08:51:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 -- 172.21.15.44:0/2487667840 shutdown_connections 2024-09-13T08:51:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1c98074bc0 0x7f1c98077080 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cb81617a0 0x7f1cb80c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cb81613c0 0x7f1cb8159530 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 --2- 172.21.15.44:0/2487667840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb80b3190 0x7f1cb8208550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 -- 172.21.15.44:0/2487667840 >> 172.21.15.44:0/2487667840 conn(0x7f1cb80d1dd0 msgr2=0x7f1cb8161b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.069+0000 7f1cbea00640 1 -- 172.21.15.44:0/2487667840 shutdown_connections 2024-09-13T08:51:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:23.073+0000 7f1cbea00640 1 -- 172.21.15.44:0/2487667840 wait complete. 2024-09-13T08:51:23.085 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:23.087+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:23.085 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:23.087+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:23.085 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:23.087+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:23.085 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:23.087+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:23.085 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:23.087+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:24.112 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone45030 2024-09-13T08:51:24.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.357+0000 7fc023600640 1 -- 172.21.15.44:0/3273385973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc004003ab0 msgr2=0x7fc004003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:24.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.357+0000 7fc023600640 1 --2- 172.21.15.44:0/3273385973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc004003ab0 0x7fc004003ed0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fc010001ec0 tx=0x7fc01001f5d0 comp rx=0 tx=0).stop 2024-09-13T08:51:24.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.361+0000 7fc023600640 1 -- 172.21.15.44:0/3273385973 shutdown_connections 2024-09-13T08:51:24.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.361+0000 7fc023600640 1 --2- 172.21.15.44:0/3273385973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc004003ab0 0x7fc004003ed0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:24.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.361+0000 7fc023600640 1 -- 172.21.15.44:0/3273385973 >> 172.21.15.44:0/3273385973 conn(0x7fc01c0b18f0 msgr2=0x7fc01c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:24.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.361+0000 7fc023600640 1 -- 172.21.15.44:0/3273385973 shutdown_connections 2024-09-13T08:51:24.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.361+0000 7fc023600640 1 -- 172.21.15.44:0/3273385973 wait complete. 2024-09-13T08:51:24.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc023600640 1 Processor -- start 2024-09-13T08:51:24.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc023600640 1 -- start start 2024-09-13T08:51:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc023600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc004003ab0 0x7fc01c0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc023600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc01c14f090 0x7fc01c14e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc023600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc01c1543a0 0x7fc01c154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc023600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c0cd390 con 0x7fc004003ab0 2024-09-13T08:51:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc023600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c0cd210 con 0x7fc01c14f090 2024-09-13T08:51:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc023600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c15c7c0 con 0x7fc01c1543a0 2024-09-13T08:51:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc020c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc004003ab0 0x7fc01c0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc020c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc004003ab0 0x7fc01c0b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55570/0 (socket says 172.21.15.44:55570) 2024-09-13T08:51:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc020c00640 1 -- 172.21.15.44:0/3811667614 learned_addr learned my addr 172.21.15.44:0/3811667614 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:24.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc020c00640 1 -- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc01c1543a0 msgr2=0x7fc01c154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:24.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc020c00640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc01c1543a0 0x7fc01c154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:24.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc020c00640 1 -- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc01c14f090 msgr2=0x7fc01c14e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:24.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc020c00640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc01c14f090 0x7fc01c14e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:24.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.365+0000 7fc020c00640 1 -- 172.21.15.44:0/3811667614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0b4a80 con 0x7fc004003ab0 2024-09-13T08:51:24.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.369+0000 7fc020c00640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc004003ab0 0x7fc01c0b41e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fc01001dcd0 tx=0x7fc010003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:24.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.369+0000 7fc018c00640 1 -- 172.21.15.44:0/3811667614 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100bf070 con 0x7fc004003ab0 2024-09-13T08:51:24.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.369+0000 7fc018c00640 1 -- 172.21.15.44:0/3811667614 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc010003da0 con 0x7fc004003ab0 2024-09-13T08:51:24.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.369+0000 7fc018c00640 1 -- 172.21.15.44:0/3811667614 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010006bb0 con 0x7fc004003ab0 2024-09-13T08:51:24.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.369+0000 7fc023600640 1 -- 172.21.15.44:0/3811667614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c0b4d60 con 0x7fc004003ab0 2024-09-13T08:51:24.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.369+0000 7fc023600640 1 -- 172.21.15.44:0/3811667614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c0b39e0 con 0x7fc004003ab0 2024-09-13T08:51:24.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.369+0000 7fc023600640 1 -- 172.21.15.44:0/3811667614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c069410 con 0x7fc004003ab0 2024-09-13T08:51:24.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.373+0000 7fc018c00640 1 -- 172.21.15.44:0/3811667614 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 48) ==== 99732+0+0 (secure 0 0 0) 0x7fc0100b8070 con 0x7fc004003ab0 2024-09-13T08:51:24.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.373+0000 7fc018c00640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbffc074c00 0x7fbffc0770c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:24.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.373+0000 7fc018c00640 1 -- 172.21.15.44:0/3811667614 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7fc01012ff90 con 0x7fc004003ab0 2024-09-13T08:51:24.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.381+0000 7fc018c00640 1 -- 172.21.15.44:0/3811667614 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc0100c8080 con 0x7fc004003ab0 2024-09-13T08:51:24.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.381+0000 7fc01be00640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbffc074c00 0x7fbffc0770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:24.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.433+0000 7fc01be00640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbffc074c00 0x7fbffc0770c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc008005d10 tx=0x7fc008005c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:24.757+0000 7fc023600640 1 -- 172.21.15.44:0/3811667614 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone45030", "target": ["mon-mgr", ""]}) -- 0x7fc01c0b4720 con 0x7fbffc074c00 2024-09-13T08:51:24.836 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:24.839+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:24.836 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:24.839+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:24.836 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:24.839+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:24.836 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:24.839+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:24.836 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:24.839+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:25.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:25.661+0000 7fc018c00640 1 -- 172.21.15.44:0/3811667614 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 49) ==== 99783+0+0 (secure 0 0 0) 0x7fc0100f8930 con 0x7fc004003ab0 2024-09-13T08:51:26.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.489+0000 7fc018c00640 1 -- 172.21.15.44:0/3811667614 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc008004020 con 0x7fbffc074c00 2024-09-13T08:51:26.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 -- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbffc074c00 msgr2=0x7fbffc0770c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbffc074c00 0x7fbffc0770c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc008005d10 tx=0x7fc008005c60 comp rx=0 tx=0).stop 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 -- 172.21.15.44:0/3811667614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc004003ab0 msgr2=0x7fc01c0b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc004003ab0 0x7fc01c0b41e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fc01001dcd0 tx=0x7fc010003bb0 comp rx=0 tx=0).stop 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 -- 172.21.15.44:0/3811667614 shutdown_connections 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbffc074c00 0x7fbffc0770c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc01c1543a0 0x7fc01c154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc01c14f090 0x7fc01c14e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 --2- 172.21.15.44:0/3811667614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc004003ab0 0x7fc01c0b41e0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 -- 172.21.15.44:0/3811667614 >> 172.21.15.44:0/3811667614 conn(0x7fc01c0b18f0 msgr2=0x7fc01c0cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 -- 172.21.15.44:0/3811667614 shutdown_connections 2024-09-13T08:51:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.493+0000 7fc023600640 1 -- 172.21.15.44:0/3811667614 wait complete. 2024-09-13T08:51:26.520 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15351 2024-09-13T08:51:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.761+0000 7f2b9d200640 1 -- 172.21.15.44:0/2709926468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b980d2570 msgr2=0x7f2b980d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.761+0000 7f2b9d200640 1 --2- 172.21.15.44:0/2709926468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b980d2570 0x7f2b980d2a00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009c80 tx=0x7f2b9001cd30 comp rx=0 tx=0).stop 2024-09-13T08:51:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.761+0000 7f2b9d200640 1 -- 172.21.15.44:0/2709926468 shutdown_connections 2024-09-13T08:51:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.761+0000 7f2b9d200640 1 --2- 172.21.15.44:0/2709926468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b980d2570 0x7f2b980d2a00 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.761+0000 7f2b9d200640 1 --2- 172.21.15.44:0/2709926468 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b980d4ab0 0x7f2b980b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.761+0000 7f2b9d200640 1 -- 172.21.15.44:0/2709926468 >> 172.21.15.44:0/2709926468 conn(0x7f2b980d3f90 msgr2=0x7f2b980d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.765+0000 7f2b9d200640 1 -- 172.21.15.44:0/2709926468 shutdown_connections 2024-09-13T08:51:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.765+0000 7f2b9d200640 1 -- 172.21.15.44:0/2709926468 wait complete. 2024-09-13T08:51:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.769+0000 7f2b9d200640 1 Processor -- start 2024-09-13T08:51:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.769+0000 7f2b9d200640 1 -- start start 2024-09-13T08:51:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.769+0000 7f2b9d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b980d2570 0x7f2b980b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.769+0000 7f2b9d200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b980d4ab0 0x7f2b980b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.769+0000 7f2b9d200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b981614f0 0x7f2b980cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.769+0000 7f2b9d200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9816b630 con 0x7f2b980d2570 2024-09-13T08:51:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.769+0000 7f2b9d200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9816b4b0 con 0x7f2b980d4ab0 2024-09-13T08:51:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.769+0000 7f2b9d200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9816b7b0 con 0x7f2b981614f0 2024-09-13T08:51:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b95e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b980d2570 0x7f2b980b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b95e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b980d2570 0x7f2b980b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55590/0 (socket says 172.21.15.44:55590) 2024-09-13T08:51:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b95e00640 1 -- 172.21.15.44:0/1047404820 learned_addr learned my addr 172.21.15.44:0/1047404820 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b95400640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b980d4ab0 0x7f2b980b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b95e00640 1 -- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b981614f0 msgr2=0x7f2b980cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b95e00640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b981614f0 0x7f2b980cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b95e00640 1 -- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b980d4ab0 msgr2=0x7f2b980b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b95e00640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b980d4ab0 0x7f2b980b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b95e00640 1 -- 172.21.15.44:0/1047404820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b980b54d0 con 0x7f2b980d2570 2024-09-13T08:51:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b95e00640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b980d2570 0x7f2b980b4020 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800c700 tx=0x7f2b8800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b8e000640 1 -- 172.21.15.44:0/1047404820 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88017070 con 0x7f2b980d2570 2024-09-13T08:51:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b8e000640 1 -- 172.21.15.44:0/1047404820 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b88005070 con 0x7f2b980d2570 2024-09-13T08:51:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b8e000640 1 -- 172.21.15.44:0/1047404820 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800a640 con 0x7f2b980d2570 2024-09-13T08:51:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b9d200640 1 -- 172.21.15.44:0/1047404820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b980b57b0 con 0x7f2b980d2570 2024-09-13T08:51:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.773+0000 7f2b9d200640 1 -- 172.21.15.44:0/1047404820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b980b51c0 con 0x7f2b980d2570 2024-09-13T08:51:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.777+0000 7f2b9d200640 1 -- 172.21.15.44:0/1047404820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b981f8480 con 0x7f2b980d2570 2024-09-13T08:51:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.777+0000 7f2b8e000640 1 -- 172.21.15.44:0/1047404820 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 49) ==== 99783+0+0 (secure 0 0 0) 0x7f2b8801a070 con 0x7f2b980d2570 2024-09-13T08:51:26.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.781+0000 7f2b8e000640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b78074c90 0x7f2b78077150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:26.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.781+0000 7f2b8e000640 1 -- 172.21.15.44:0/1047404820 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f2b88092360 con 0x7f2b980d2570 2024-09-13T08:51:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.781+0000 7f2b8e000640 1 -- 172.21.15.44:0/1047404820 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2b88021020 con 0x7f2b980d2570 2024-09-13T08:51:26.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.789+0000 7f2b95400640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b78074c90 0x7f2b78077150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:26.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:26.793+0000 7f2b95400640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b78074c90 0x7f2b78077150 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2b980ce570 tx=0x7f2b840091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:27.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.141+0000 7f2b9d200640 1 -- 172.21.15.44:0/1047404820 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15351", "target": ["mon-mgr", ""]}) -- 0x7f2b980b0cf0 con 0x7f2b78074c90 2024-09-13T08:51:27.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.545+0000 7f2b8e000640 1 -- 172.21.15.44:0/1047404820 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2b980b0cf0 con 0x7f2b78074c90 2024-09-13T08:51:27.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 -- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b78074c90 msgr2=0x7f2b78077150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b78074c90 0x7f2b78077150 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2b980ce570 tx=0x7f2b840091e0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 -- 172.21.15.44:0/1047404820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b980d2570 msgr2=0x7f2b980b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b980d2570 0x7f2b980b4020 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800c700 tx=0x7f2b8800cbc0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 -- 172.21.15.44:0/1047404820 shutdown_connections 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2b78074c90 0x7f2b78077150 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2b981614f0 0x7f2b980cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b980d4ab0 0x7f2b980b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 --2- 172.21.15.44:0/1047404820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b980d2570 0x7f2b980b4020 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 -- 172.21.15.44:0/1047404820 >> 172.21.15.44:0/1047404820 conn(0x7f2b980d3f90 msgr2=0x7f2b980b6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 -- 172.21.15.44:0/1047404820 shutdown_connections 2024-09-13T08:51:27.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.549+0000 7f2b9d200640 1 -- 172.21.15.44:0/1047404820 wait complete. 2024-09-13T08:51:27.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone47392 2024-09-13T08:51:27.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 -- 172.21.15.44:0/1619326120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2810162750 msgr2=0x7f28100d21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:27.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 --2- 172.21.15.44:0/1619326120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2810162750 0x7f28100d21c0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f28100d1bb0 tx=0x7f280001c830 comp rx=0 tx=0).stop 2024-09-13T08:51:27.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 -- 172.21.15.44:0/1619326120 shutdown_connections 2024-09-13T08:51:27.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 --2- 172.21.15.44:0/1619326120 >> v2:172.21.15.129:3300/0 conn(0x7f28100d2700 0x7f28100b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 --2- 172.21.15.44:0/1619326120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2810162750 0x7f28100d21c0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 --2- 172.21.15.44:0/1619326120 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2810162370 0x7f28100b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 -- 172.21.15.44:0/1619326120 >> 172.21.15.44:0/1619326120 conn(0x7f281009d0c0 msgr2=0x7f28100d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:27.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 -- 172.21.15.44:0/1619326120 shutdown_connections 2024-09-13T08:51:27.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 -- 172.21.15.44:0/1619326120 wait complete. 2024-09-13T08:51:27.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 Processor -- start 2024-09-13T08:51:27.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.789+0000 7f2817200640 1 -- start start 2024-09-13T08:51:27.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2817200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28100d2700 0x7f2810151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:27.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2817200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2810162370 0x7f281015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:27.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2817200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2810162750 0x7f281015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:27.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2817200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28100b3af0 con 0x7f2810162750 2024-09-13T08:51:27.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2817200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f28100b3970 con 0x7f2810162370 2024-09-13T08:51:27.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2817200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f28100b3c70 con 0x7f28100d2700 2024-09-13T08:51:27.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2816800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2810162750 0x7f281015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:27.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2816800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2810162750 0x7f281015b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55618/0 (socket says 172.21.15.44:55618) 2024-09-13T08:51:27.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2816800640 1 -- 172.21.15.44:0/3915861513 learned_addr learned my addr 172.21.15.44:0/3915861513 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:27.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2815400640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2810162370 0x7f281015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:27.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2815e00640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28100d2700 0x7f2810151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:27.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2816800640 1 -- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28100d2700 msgr2=0x7f2810151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:27.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2816800640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28100d2700 0x7f2810151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2816800640 1 -- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2810162370 msgr2=0x7f281015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:27.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2816800640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2810162370 0x7f281015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:27.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2816800640 1 -- 172.21.15.44:0/3915861513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2810151d80 con 0x7f2810162750 2024-09-13T08:51:27.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2815400640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2810162370 0x7f281015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:27.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2816800640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2810162750 0x7f281015b0b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f280c00d780 tx=0x7f280c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:27.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f280a000640 1 -- 172.21.15.44:0/3915861513 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c017070 con 0x7f2810162750 2024-09-13T08:51:27.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f280a000640 1 -- 172.21.15.44:0/3915861513 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f280c00de90 con 0x7f2810162750 2024-09-13T08:51:27.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2817200640 1 -- 172.21.15.44:0/3915861513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281015f0f0 con 0x7f2810162750 2024-09-13T08:51:27.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f280a000640 1 -- 172.21.15.44:0/3915861513 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c005020 con 0x7f2810162750 2024-09-13T08:51:27.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2817200640 1 -- 172.21.15.44:0/3915861513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281015f690 con 0x7f2810162750 2024-09-13T08:51:27.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.793+0000 7f2817200640 1 -- 172.21.15.44:0/3915861513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28100b4cc0 con 0x7f2810162750 2024-09-13T08:51:27.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.797+0000 7f280a000640 1 -- 172.21.15.44:0/3915861513 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7f280c0051c0 con 0x7f2810162750 2024-09-13T08:51:27.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.797+0000 7f280a000640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f27f4074d50 0x7f27f4077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:27.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.797+0000 7f280a000640 1 -- 172.21.15.44:0/3915861513 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f280c092f80 con 0x7f2810162750 2024-09-13T08:51:27.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.801+0000 7f2815e00640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f27f4074d50 0x7f27f4077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:27.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.801+0000 7f2815e00640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f27f4074d50 0x7f27f4077210 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2804009f20 tx=0x7f28040094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:27.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:27.805+0000 7f280a000640 1 -- 172.21.15.44:0/3915861513 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f280c05fc30 con 0x7f2810162750 2024-09-13T08:51:28.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.157+0000 7f2817200640 1 -- 172.21.15.44:0/3915861513 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone47392", "target": ["mon-mgr", ""]}) -- 0x7f281008d540 con 0x7f27f4074d50 2024-09-13T08:51:28.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f280a000640 1 -- 172.21.15.44:0/3915861513 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f281008d540 con 0x7f27f4074d50 2024-09-13T08:51:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 -- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f27f4074d50 msgr2=0x7f27f4077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f27f4074d50 0x7f27f4077210 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2804009f20 tx=0x7f28040094f0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 -- 172.21.15.44:0/3915861513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2810162750 msgr2=0x7f281015b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2810162750 0x7f281015b0b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f280c00d780 tx=0x7f280c00dc40 comp rx=0 tx=0).stop 2024-09-13T08:51:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 -- 172.21.15.44:0/3915861513 shutdown_connections 2024-09-13T08:51:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f27f4074d50 0x7f27f4077210 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2810162750 0x7f281015b0b0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2810162370 0x7f281015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 --2- 172.21.15.44:0/3915861513 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f28100d2700 0x7f2810151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 -- 172.21.15.44:0/3915861513 >> 172.21.15.44:0/3915861513 conn(0x7f281009d0c0 msgr2=0x7f28100d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:28.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 -- 172.21.15.44:0/3915861513 shutdown_connections 2024-09-13T08:51:28.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.293+0000 7f2817200640 1 -- 172.21.15.44:0/3915861513 wait complete. 2024-09-13T08:51:28.314 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone45030 2024-09-13T08:51:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.465+0000 7febf1000640 1 -- 172.21.15.44:0/18840536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec0cbeb0 msgr2=0x7febec073c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.465+0000 7febf1000640 1 --2- 172.21.15.44:0/18840536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec0cbeb0 0x7febec073c80 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7febd4009900 tx=0x7febd401c8b0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.465+0000 7febf1000640 1 -- 172.21.15.44:0/18840536 shutdown_connections 2024-09-13T08:51:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.465+0000 7febf1000640 1 --2- 172.21.15.44:0/18840536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec0cbeb0 0x7febec073c80 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.465+0000 7febf1000640 1 --2- 172.21.15.44:0/18840536 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7febec15be10 0x7febec1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.465+0000 7febf1000640 1 -- 172.21.15.44:0/18840536 >> 172.21.15.44:0/18840536 conn(0x7febec0cd740 msgr2=0x7febec0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.465+0000 7febf1000640 1 -- 172.21.15.44:0/18840536 shutdown_connections 2024-09-13T08:51:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.465+0000 7febf1000640 1 -- 172.21.15.44:0/18840536 wait complete. 2024-09-13T08:51:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febf1000640 1 Processor -- start 2024-09-13T08:51:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febf1000640 1 -- start start 2024-09-13T08:51:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febf1000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febec06cc20 0x7febec0b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febf1000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7febec0cbeb0 0x7febec0c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febf1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec15be10 0x7febec0c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febf1000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7febec1697a0 con 0x7febec15be10 2024-09-13T08:51:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febf1000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7febec169620 con 0x7febec06cc20 2024-09-13T08:51:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febf1000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7febec169920 con 0x7febec0cbeb0 2024-09-13T08:51:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febea800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec15be10 0x7febec0c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febea800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec15be10 0x7febec0c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55640/0 (socket says 172.21.15.44:55640) 2024-09-13T08:51:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febea800640 1 -- 172.21.15.44:0/3642417061 learned_addr learned my addr 172.21.15.44:0/3642417061 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febe9e00640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febec06cc20 0x7febec0b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febe1400640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7febec0cbeb0 0x7febec0c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febea800640 1 -- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7febec0cbeb0 msgr2=0x7febec0c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febea800640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7febec0cbeb0 0x7febec0c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febea800640 1 -- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febec06cc20 msgr2=0x7febec0b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febea800640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febec06cc20 0x7febec0b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febea800640 1 -- 172.21.15.44:0/3642417061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec0c8f70 con 0x7febec15be10 2024-09-13T08:51:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febea800640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec15be10 0x7febec0c8600 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7febe400c700 tx=0x7febe400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febe2a00640 1 -- 172.21.15.44:0/3642417061 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4003b80 con 0x7febec15be10 2024-09-13T08:51:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febe2a00640 1 -- 172.21.15.44:0/3642417061 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febe4003d20 con 0x7febec15be10 2024-09-13T08:51:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febf1000640 1 -- 172.21.15.44:0/3642417061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec0c9250 con 0x7febec15be10 2024-09-13T08:51:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febe2a00640 1 -- 172.21.15.44:0/3642417061 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe400aab0 con 0x7febec15be10 2024-09-13T08:51:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.469+0000 7febf1000640 1 -- 172.21.15.44:0/3642417061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec0b4f50 con 0x7febec15be10 2024-09-13T08:51:28.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.473+0000 7febf1000640 1 -- 172.21.15.44:0/3642417061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febec073c80 con 0x7febec15be10 2024-09-13T08:51:28.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.473+0000 7febe2a00640 1 -- 172.21.15.44:0/3642417061 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7febe400ac50 con 0x7febec15be10 2024-09-13T08:51:28.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.473+0000 7febe2a00640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7febc8074d50 0x7febc8077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:28.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.473+0000 7febe2a00640 1 -- 172.21.15.44:0/3642417061 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7febe4092af0 con 0x7febec15be10 2024-09-13T08:51:28.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.473+0000 7febe9e00640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7febc8074d50 0x7febc8077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:28.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.477+0000 7febe9e00640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7febc8074d50 0x7febc8077210 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7febdc00bb50 tx=0x7febdc00b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:28.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.477+0000 7febe2a00640 1 -- 172.21.15.44:0/3642417061 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7febe405f150 con 0x7febec15be10 2024-09-13T08:51:28.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.689+0000 7febf1000640 1 -- 172.21.15.44:0/3642417061 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone45030", "target": ["mon-mgr", ""]}) -- 0x7febec0c9420 con 0x7febc8074d50 2024-09-13T08:51:28.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.853+0000 7febe2a00640 1 -- 172.21.15.44:0/3642417061 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7febec0c9420 con 0x7febc8074d50 2024-09-13T08:51:28.855 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:51:28.855 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:51:28.855 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:51:28.855 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:51:28.855 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:51:28.855 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:51:28.856 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:51:28.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:28.856 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:51:28.856 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:51:28.856 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:51:28.856 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:28.856 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:28.856 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:51:28.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 -- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7febc8074d50 msgr2=0x7febc8077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7febc8074d50 0x7febc8077210 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7febdc00bb50 tx=0x7febdc00b2e0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 -- 172.21.15.44:0/3642417061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec15be10 msgr2=0x7febec0c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec15be10 0x7febec0c8600 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7febe400c700 tx=0x7febe400cbc0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 -- 172.21.15.44:0/3642417061 shutdown_connections 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7febc8074d50 0x7febc8077210 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec15be10 0x7febec0c8600 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7febec0cbeb0 0x7febec0c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 --2- 172.21.15.44:0/3642417061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febec06cc20 0x7febec0b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 -- 172.21.15.44:0/3642417061 >> 172.21.15.44:0/3642417061 conn(0x7febec0cd740 msgr2=0x7febec0cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 -- 172.21.15.44:0/3642417061 shutdown_connections 2024-09-13T08:51:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:28.857+0000 7febf1000640 1 -- 172.21.15.44:0/3642417061 wait complete. 2024-09-13T08:51:28.877 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:51:28.877 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15351 2024-09-13T08:51:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 -- 172.21.15.44:0/1616228421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fe4003a60 msgr2=0x7f0fe4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 --2- 172.21.15.44:0/1616228421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fe4003a60 0x7f0fe4003e80 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0006fb0 tx=0x7f0ff001ce40 comp rx=0 tx=0).stop 2024-09-13T08:51:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002200640 1 -- 172.21.15.44:0/1616228421 reap_dead start 2024-09-13T08:51:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 -- 172.21.15.44:0/1616228421 shutdown_connections 2024-09-13T08:51:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 -- 172.21.15.44:0/1616228421 >> 172.21.15.44:0/1616228421 conn(0x7f0ffc09aa10 msgr2=0x7f0ffc09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 -- 172.21.15.44:0/1616228421 shutdown_connections 2024-09-13T08:51:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 -- 172.21.15.44:0/1616228421 wait complete. 2024-09-13T08:51:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 Processor -- start 2024-09-13T08:51:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 -- start start 2024-09-13T08:51:29.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fe4003a60 0x7f0ffc0b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:29.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ffc1f7a50 0x7f0ffc06f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:29.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc0b6410 0x7f0ffc156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc166b90 con 0x7f0ffc0b6410 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc166a10 con 0x7f0ffc1f7a50 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc166d10 con 0x7f0fe4003a60 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1001800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fe4003a60 0x7f0ffc0b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.029+0000 7f1002200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc0b6410 0x7f0ffc156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1001800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fe4003a60 0x7f0ffc0b4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:52230/0 (socket says 172.21.15.44:52230) 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1001800640 1 -- 172.21.15.44:0/2650900177 learned_addr learned my addr 172.21.15.44:0/2650900177 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1000e00640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ffc1f7a50 0x7f0ffc06f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1001800640 1 -- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ffc1f7a50 msgr2=0x7f0ffc06f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1001800640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ffc1f7a50 0x7f0ffc06f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:29.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1001800640 1 -- 172.21.15.44:0/2650900177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc0b6410 msgr2=0x7f0ffc156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1001800640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc0b6410 0x7f0ffc156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1001800640 1 -- 172.21.15.44:0/2650900177 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc1f7f60 con 0x7f0fe4003a60 2024-09-13T08:51:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1000e00640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ffc1f7a50 0x7f0ffc06f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1002200640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc0b6410 0x7f0ffc156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:51:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1001800640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fe4003a60 0x7f0ffc0b4a90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0ff001b540 tx=0x7f0ff0001c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:29.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f0ff9600640 1 -- 172.21.15.44:0/2650900177 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00013e0 con 0x7f0fe4003a60 2024-09-13T08:51:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1002c00640 1 -- 172.21.15.44:0/2650900177 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc1f8240 con 0x7f0fe4003a60 2024-09-13T08:51:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f0ff9600640 1 -- 172.21.15.44:0/2650900177 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ff00015a0 con 0x7f0fe4003a60 2024-09-13T08:51:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f0ff9600640 1 -- 172.21.15.44:0/2650900177 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0006330 con 0x7f0fe4003a60 2024-09-13T08:51:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1002c00640 1 -- 172.21.15.44:0/2650900177 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1f8800 con 0x7f0fe4003a60 2024-09-13T08:51:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f1002c00640 1 -- 172.21.15.44:0/2650900177 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ffc1f8a40 con 0x7f0fe4003a60 2024-09-13T08:51:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.033+0000 7f0ff9600640 1 -- 172.21.15.44:0/2650900177 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7f0ff00a5b90 con 0x7f0fe4003a60 2024-09-13T08:51:29.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.037+0000 7f0ff9600640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fdc074d50 0x7f0fdc077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:29.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.037+0000 7f0ff9600640 1 -- 172.21.15.44:0/2650900177 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f0ff012c7e0 con 0x7f0fe4003a60 2024-09-13T08:51:29.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.037+0000 7f1000e00640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fdc074d50 0x7f0fdc077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:29.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.037+0000 7f1000e00640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fdc074d50 0x7f0fdc077210 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0b5dc0 tx=0x7f0fe800b260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:29.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.037+0000 7f0ff9600640 1 -- 172.21.15.44:0/2650900177 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0ff0016bf0 con 0x7f0fe4003a60 2024-09-13T08:51:29.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:29.249+0000 7f1002c00640 1 -- 172.21.15.44:0/2650900177 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15351", "target": ["mon-mgr", ""]}) -- 0x7f0ffc0d1bb0 con 0x7f0fdc074d50 2024-09-13T08:51:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f0ff9600640 1 -- 172.21.15.44:0/2650900177 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f0ffc0d1bb0 con 0x7f0fdc074d50 2024-09-13T08:51:32.216 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:51:32.216 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:51:32.216 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:51:32.216 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:51:32.216 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:51:32.216 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:51:32.216 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:51:32.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:32.216 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:51:32.217 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:51:32.217 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:51:32.217 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:32.217 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:32.217 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:51:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 -- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fdc074d50 msgr2=0x7f0fdc077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fdc074d50 0x7f0fdc077210 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0b5dc0 tx=0x7f0fe800b260 comp rx=0 tx=0).stop 2024-09-13T08:51:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 -- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fe4003a60 msgr2=0x7f0ffc0b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fe4003a60 0x7f0ffc0b4a90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0ff001b540 tx=0x7f0ff0001c80 comp rx=0 tx=0).stop 2024-09-13T08:51:32.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 -- 172.21.15.44:0/2650900177 shutdown_connections 2024-09-13T08:51:32.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0fdc074d50 0x7f0fdc077210 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:32.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc0b6410 0x7f0ffc156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:32.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ffc1f7a50 0x7f0ffc06f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:32.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 --2- 172.21.15.44:0/2650900177 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0fe4003a60 0x7f0ffc0b4a90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:32.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 -- 172.21.15.44:0/2650900177 >> 172.21.15.44:0/2650900177 conn(0x7f0ffc09aa10 msgr2=0x7f0ffc09adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:32.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 -- 172.21.15.44:0/2650900177 shutdown_connections 2024-09-13T08:51:32.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.217+0000 7f1002c00640 1 -- 172.21.15.44:0/2650900177 wait complete. 2024-09-13T08:51:32.242 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:51:32.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone47392 2024-09-13T08:51:32.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.397+0000 7f2cc0200640 1 -- 172.21.15.44:0/3612809150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cb815ab80 msgr2=0x7f2cb80ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:32.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.397+0000 7f2cc0200640 1 --2- 172.21.15.44:0/3612809150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cb815ab80 0x7f2cb80ce8a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009900 tx=0x7f2cac01c820 comp rx=0 tx=0).stop 2024-09-13T08:51:32.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.397+0000 7f2cc0200640 1 -- 172.21.15.44:0/3612809150 shutdown_connections 2024-09-13T08:51:32.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.397+0000 7f2cc0200640 1 --2- 172.21.15.44:0/3612809150 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2cb8073860 0x7f2cb8073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:32.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.397+0000 7f2cc0200640 1 --2- 172.21.15.44:0/3612809150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cb815ab80 0x7f2cb80ce8a0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:32.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.397+0000 7f2cc0200640 1 -- 172.21.15.44:0/3612809150 >> 172.21.15.44:0/3612809150 conn(0x7f2cb80d0cd0 msgr2=0x7f2cb80d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:32.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.397+0000 7f2cc0200640 1 -- 172.21.15.44:0/3612809150 shutdown_connections 2024-09-13T08:51:32.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.397+0000 7f2cc0200640 1 -- 172.21.15.44:0/3612809150 wait complete. 2024-09-13T08:51:32.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 Processor -- start 2024-09-13T08:51:32.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 -- start start 2024-09-13T08:51:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cb8073860 0x7f2cb814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2cb80d01a0 0x7f2cb814b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cb815ab80 0x7f2cb81f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb81678c0 con 0x7f2cb8073860 2024-09-13T08:51:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb8071a70 con 0x7f2cb815ab80 2024-09-13T08:51:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb8167a40 con 0x7f2cb80d01a0 2024-09-13T08:51:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cbd800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cb8073860 0x7f2cb814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cbd800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cb8073860 0x7f2cb814ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59190/0 (socket says 172.21.15.44:59190) 2024-09-13T08:51:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cbd800640 1 -- 172.21.15.44:0/3228216798 learned_addr learned my addr 172.21.15.44:0/3228216798 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cbd800640 1 -- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2cb80d01a0 msgr2=0x7f2cb814b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cbd800640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2cb80d01a0 0x7f2cb814b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cbd800640 1 -- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cb815ab80 msgr2=0x7f2cb81f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cbd800640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cb815ab80 0x7f2cb81f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cbd800640 1 -- 172.21.15.44:0/3228216798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb8158b10 con 0x7f2cb8073860 2024-09-13T08:51:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cbd800640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cb8073860 0x7f2cb814ebb0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2cac01a450 tx=0x7f2cac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cb5600640 1 -- 172.21.15.44:0/3228216798 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac0b2040 con 0x7f2cb8073860 2024-09-13T08:51:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cb5600640 1 -- 172.21.15.44:0/3228216798 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cac005030 con 0x7f2cb8073860 2024-09-13T08:51:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 -- 172.21.15.44:0/3228216798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb8158db0 con 0x7f2cb8073860 2024-09-13T08:51:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cb5600640 1 -- 172.21.15.44:0/3228216798 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac0044f0 con 0x7f2cb8073860 2024-09-13T08:51:32.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 -- 172.21.15.44:0/3228216798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb8159370 con 0x7f2cb8073860 2024-09-13T08:51:32.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.401+0000 7f2cc0200640 1 -- 172.21.15.44:0/3228216798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb8073cc0 con 0x7f2cb8073860 2024-09-13T08:51:32.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.405+0000 7f2cb5600640 1 -- 172.21.15.44:0/3228216798 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7f2cac005400 con 0x7f2cb8073860 2024-09-13T08:51:32.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.405+0000 7f2cb5600640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c98074d50 0x7f2c98077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:32.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.405+0000 7f2cb5600640 1 -- 172.21.15.44:0/3228216798 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f2cac12c950 con 0x7f2cb8073860 2024-09-13T08:51:32.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.405+0000 7f2cbce00640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c98074d50 0x7f2c98077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:32.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.405+0000 7f2cbce00640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c98074d50 0x7f2c98077210 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4000f70 tx=0x7f2ca40079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:32.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.409+0000 7f2cb5600640 1 -- 172.21.15.44:0/3228216798 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2cac0164d0 con 0x7f2cb8073860 2024-09-13T08:51:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:32.621+0000 7f2cc0200640 1 -- 172.21.15.44:0/3228216798 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone47392", "target": ["mon-mgr", ""]}) -- 0x7f2cb815d4f0 con 0x7f2c98074d50 2024-09-13T08:51:32.894 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:51:32.897 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:51:33.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cb5600640 1 -- 172.21.15.44:0/3228216798 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f2cb815d4f0 con 0x7f2c98074d50 2024-09-13T08:51:33.852 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:51:33.852 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:51:33.852 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:51:33.852 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:51:33.852 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:51:33.852 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:51:33.852 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:51:33.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:33.852 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:51:33.852 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:51:33.853 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:51:33.853 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:33.853 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:33.853 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:51:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 -- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c98074d50 msgr2=0x7f2c98077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c98074d50 0x7f2c98077210 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4000f70 tx=0x7f2ca40079c0 comp rx=0 tx=0).stop 2024-09-13T08:51:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 -- 172.21.15.44:0/3228216798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cb8073860 msgr2=0x7f2cb814ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cb8073860 0x7f2cb814ebb0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2cac01a450 tx=0x7f2cac004040 comp rx=0 tx=0).stop 2024-09-13T08:51:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 -- 172.21.15.44:0/3228216798 shutdown_connections 2024-09-13T08:51:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2c98074d50 0x7f2c98077210 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cb815ab80 0x7f2cb81f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2cb80d01a0 0x7f2cb814b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 --2- 172.21.15.44:0/3228216798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cb8073860 0x7f2cb814ebb0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 -- 172.21.15.44:0/3228216798 >> 172.21.15.44:0/3228216798 conn(0x7f2cb80d0cd0 msgr2=0x7f2cb814f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 -- 172.21.15.44:0/3228216798 shutdown_connections 2024-09-13T08:51:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:33.853+0000 7f2cc0200640 1 -- 172.21.15.44:0/3228216798 wait complete. 2024-09-13T08:51:33.879 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:51:33.879 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:51:34.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.029+0000 7fa422200640 1 -- 172.21.15.44:0/3155950502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa41c0cfe80 msgr2=0x7fa41c0d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.029+0000 7fa422200640 1 --2- 172.21.15.44:0/3155950502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa41c0cfe80 0x7fa41c0d0260 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0b33d0 tx=0x7fa40c01c630 comp rx=0 tx=0).stop 2024-09-13T08:51:34.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.029+0000 7fa422200640 1 -- 172.21.15.44:0/3155950502 shutdown_connections 2024-09-13T08:51:34.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.029+0000 7fa422200640 1 --2- 172.21.15.44:0/3155950502 >> v2:172.21.15.129:3300/0 conn(0x7fa41c0b2760 0x7fa41c1605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.029+0000 7fa422200640 1 --2- 172.21.15.44:0/3155950502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa41c0cfe80 0x7fa41c0d0260 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.029+0000 7fa422200640 1 -- 172.21.15.44:0/3155950502 >> 172.21.15.44:0/3155950502 conn(0x7fa41c0d1e40 msgr2=0x7fa41c0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:34.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.029+0000 7fa422200640 1 -- 172.21.15.44:0/3155950502 shutdown_connections 2024-09-13T08:51:34.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.029+0000 7fa422200640 1 -- 172.21.15.44:0/3155950502 wait complete. 2024-09-13T08:51:34.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 Processor -- start 2024-09-13T08:51:34.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 -- start start 2024-09-13T08:51:34.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa41c0b2760 0x7fa41c157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c0b2d00 0x7fa41c0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa41c0cfe80 0x7fa41c0c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c0b0940 con 0x7fa41c0cfe80 2024-09-13T08:51:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c0b07c0 con 0x7fa41c0b2d00 2024-09-13T08:51:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa41c0b0ac0 con 0x7fa41c0b2760 2024-09-13T08:51:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa41be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa41c0cfe80 0x7fa41c0c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa41be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa41c0cfe80 0x7fa41c0c68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59208/0 (socket says 172.21.15.44:59208) 2024-09-13T08:51:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa41be00640 1 -- 172.21.15.44:0/728498694 learned_addr learned my addr 172.21.15.44:0/728498694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa41b400640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa41c0b2760 0x7fa41c157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa413e00640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c0b2d00 0x7fa41c0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa41be00640 1 -- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa41c0b2760 msgr2=0x7fa41c157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa41be00640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa41c0b2760 0x7fa41c157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa41be00640 1 -- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c0b2d00 msgr2=0x7fa41c0c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa41be00640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c0b2d00 0x7fa41c0c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa41be00640 1 -- 172.21.15.44:0/728498694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c1581f0 con 0x7fa41c0cfe80 2024-09-13T08:51:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa413e00640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c0b2d00 0x7fa41c0c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa41be00640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa41c0cfe80 0x7fa41c0c68a0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa41400cce0 tx=0x7fa41400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa413400640 1 -- 172.21.15.44:0/728498694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414003980 con 0x7fa41c0cfe80 2024-09-13T08:51:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa413400640 1 -- 172.21.15.44:0/728498694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa414003c10 con 0x7fa41c0cfe80 2024-09-13T08:51:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 -- 172.21.15.44:0/728498694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41c1fb0e0 con 0x7fa41c0cfe80 2024-09-13T08:51:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa413400640 1 -- 172.21.15.44:0/728498694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414004030 con 0x7fa41c0cfe80 2024-09-13T08:51:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 -- 172.21.15.44:0/728498694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41c1fb6a0 con 0x7fa41c0cfe80 2024-09-13T08:51:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.033+0000 7fa422200640 1 -- 172.21.15.44:0/728498694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41c0d0260 con 0x7fa41c0cfe80 2024-09-13T08:51:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.037+0000 7fa413400640 1 -- 172.21.15.44:0/728498694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7fa4140041d0 con 0x7fa41c0cfe80 2024-09-13T08:51:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.037+0000 7fa413400640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3f8074d20 0x7fa3f80771e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.037+0000 7fa413400640 1 -- 172.21.15.44:0/728498694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7fa414093660 con 0x7fa41c0cfe80 2024-09-13T08:51:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.037+0000 7fa41b400640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3f8074d20 0x7fa3f80771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.037+0000 7fa41b400640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3f8074d20 0x7fa3f80771e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0ae000 tx=0x7fa40c01ae70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:34.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.041+0000 7fa413400640 1 -- 172.21.15.44:0/728498694 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa414014070 con 0x7fa41c0cfe80 2024-09-13T08:51:34.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.277+0000 7fa422200640 1 -- 172.21.15.44:0/728498694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa41c0b7bc0 con 0x7fa41c0cfe80 2024-09-13T08:51:34.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.277+0000 7fa413400640 1 -- 172.21.15.44:0/728498694 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1921 (secure 0 0 0) 0x7fa41405fcc0 con 0x7fa41c0cfe80 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:51:34.278 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 865, 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:34.279 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 118, 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:51:34.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 1813, 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 7450393978, 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 16010534912, 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 757083578368, 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:51:34.281 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 357746292, 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 352204054, 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 87, 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 99 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 75, 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:50:46:358102+0000", 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 8, 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:51:34.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:34.284 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 8, 2024-09-13T08:51:34.284 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:51:34.284 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:51:34.284 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:51:34.284 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 8253 2024-09-13T08:51:34.284 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:34.284 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:51:34.284 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:51:34.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 19, 2024-09-13T08:51:34.285 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:50:26.790034+0000", 2024-09-13T08:51:34.286 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:51:34.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:34.286 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": {} 2024-09-13T08:51:34.286 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 -- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3f8074d20 msgr2=0x7fa3f80771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3f8074d20 0x7fa3f80771e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0ae000 tx=0x7fa40c01ae70 comp rx=0 tx=0).stop 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 -- 172.21.15.44:0/728498694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa41c0cfe80 msgr2=0x7fa41c0c68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa41c0cfe80 0x7fa41c0c68a0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa41400cce0 tx=0x7fa41400a620 comp rx=0 tx=0).stop 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 -- 172.21.15.44:0/728498694 shutdown_connections 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa3f8074d20 0x7fa3f80771e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa41c0cfe80 0x7fa41c0c68a0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c0b2d00 0x7fa41c0c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 --2- 172.21.15.44:0/728498694 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa41c0b2760 0x7fa41c157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 -- 172.21.15.44:0/728498694 >> 172.21.15.44:0/728498694 conn(0x7fa41c0d1e40 msgr2=0x7fa41c158b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 -- 172.21.15.44:0/728498694 shutdown_connections 2024-09-13T08:51:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.281+0000 7fa422200640 1 -- 172.21.15.44:0/728498694 wait complete. 2024-09-13T08:51:34.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:51:34.299 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone45030/a394d3b5-bc51-4d71-a0ac-2956ea0a5881 2024-09-13T08:51:34.299 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:51:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone45030/a394d3b5-bc51-4d71-a0ac-2956ea0a5881': No such file or directory 2024-09-13T08:51:34.311 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-13T08:51:34.311 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:51:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.501+0000 7f34a5e00640 1 -- 172.21.15.44:0/3276407254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a00cecf0 msgr2=0x7f34a00cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.501+0000 7f34a5e00640 1 --2- 172.21.15.44:0/3276407254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a00cecf0 0x7f34a00cc940 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3488009900 tx=0x7f348801c7c0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 -- 172.21.15.44:0/3276407254 shutdown_connections 2024-09-13T08:51:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 --2- 172.21.15.44:0/3276407254 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a015c130 0x7f34a00cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 --2- 172.21.15.44:0/3276407254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a00cecf0 0x7f34a00cc940 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 -- 172.21.15.44:0/3276407254 >> 172.21.15.44:0/3276407254 conn(0x7f34a00d1ea0 msgr2=0x7f34a00d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:34.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 -- 172.21.15.44:0/3276407254 shutdown_connections 2024-09-13T08:51:34.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 -- 172.21.15.44:0/3276407254 wait complete. 2024-09-13T08:51:34.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 Processor -- start 2024-09-13T08:51:34.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 -- start start 2024-09-13T08:51:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a00cecf0 0x7f34a0156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a015bd50 0x7f34a01fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a015c130 0x7f34a0157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01694f0 con 0x7f34a00cecf0 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0169370 con 0x7f34a015bd50 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f34a0169670 con 0x7f34a015c130 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f349ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a00cecf0 0x7f34a0156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f349ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a00cecf0 0x7f34a0156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59214/0 (socket says 172.21.15.44:59214) 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f349ea00640 1 -- 172.21.15.44:0/1146752201 learned_addr learned my addr 172.21.15.44:0/1146752201 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f3497e00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a015bd50 0x7f34a01fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f349f400640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a015c130 0x7f34a0157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f349ea00640 1 -- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a015c130 msgr2=0x7f34a0157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f349ea00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a015c130 0x7f34a0157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f349ea00640 1 -- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a015bd50 msgr2=0x7f34a01fb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f349ea00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a015bd50 0x7f34a01fb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f349ea00640 1 -- 172.21.15.44:0/1146752201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a00b7a20 con 0x7f34a00cecf0 2024-09-13T08:51:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f3497e00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a015bd50 0x7f34a01fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f349ea00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a00cecf0 0x7f34a0156950 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f349000c500 tx=0x7f349000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f3497400640 1 -- 172.21.15.44:0/1146752201 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490017070 con 0x7f34a00cecf0 2024-09-13T08:51:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f3497400640 1 -- 172.21.15.44:0/1146752201 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34900053c0 con 0x7f34a00cecf0 2024-09-13T08:51:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f34a5e00640 1 -- 172.21.15.44:0/1146752201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a00b7460 con 0x7f34a00cecf0 2024-09-13T08:51:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.505+0000 7f3497400640 1 -- 172.21.15.44:0/1146752201 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490012890 con 0x7f34a00cecf0 2024-09-13T08:51:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.509+0000 7f34a5e00640 1 -- 172.21.15.44:0/1146752201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01fb130 con 0x7f34a00cecf0 2024-09-13T08:51:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.509+0000 7f34a5e00640 1 -- 172.21.15.44:0/1146752201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a00ce5e0 con 0x7f34a00cecf0 2024-09-13T08:51:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.509+0000 7f3497400640 1 -- 172.21.15.44:0/1146752201 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7f349001a070 con 0x7f34a00cecf0 2024-09-13T08:51:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.509+0000 7f3497400640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f347c074d20 0x7f347c0771e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.509+0000 7f3497400640 1 -- 172.21.15.44:0/1146752201 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f3490092210 con 0x7f34a00cecf0 2024-09-13T08:51:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.509+0000 7f3497e00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f347c074d20 0x7f347c0771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.513+0000 7f3497e00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f347c074d20 0x7f347c0771e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f34880af000 tx=0x7f348801b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:34.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.513+0000 7f3497400640 1 -- 172.21.15.44:0/1146752201 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f349005ee40 con 0x7f34a00cecf0 2024-09-13T08:51:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.713+0000 7f34a5e00640 1 -- 172.21.15.44:0/1146752201 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f34a0076360 con 0x7f347c074d20 2024-09-13T08:51:34.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f3497400640 1 -- 172.21.15.44:0/1146752201 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f34a0076360 con 0x7f347c074d20 2024-09-13T08:51:34.716 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:51:34.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:34.716 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:51:34.716 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:34.716 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:51:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 -- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f347c074d20 msgr2=0x7f347c0771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f347c074d20 0x7f347c0771e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f34880af000 tx=0x7f348801b000 comp rx=0 tx=0).stop 2024-09-13T08:51:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 -- 172.21.15.44:0/1146752201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a00cecf0 msgr2=0x7f34a0156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a00cecf0 0x7f34a0156950 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f349000c500 tx=0x7f349000c9c0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 -- 172.21.15.44:0/1146752201 shutdown_connections 2024-09-13T08:51:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f347c074d20 0x7f347c0771e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f34a015c130 0x7f34a0157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a015bd50 0x7f34a01fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 --2- 172.21.15.44:0/1146752201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34a00cecf0 0x7f34a0156950 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 -- 172.21.15.44:0/1146752201 >> 172.21.15.44:0/1146752201 conn(0x7f34a00d1ea0 msgr2=0x7f34a00d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 -- 172.21.15.44:0/1146752201 shutdown_connections 2024-09-13T08:51:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.717+0000 7f34a5e00640 1 -- 172.21.15.44:0/1146752201 wait complete. 2024-09-13T08:51:34.738 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-13T08:51:34.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 -- 172.21.15.44:0/1352194100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e500ce980 msgr2=0x7f7e500cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 --2- 172.21.15.44:0/1352194100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e500ce980 0x7f7e500cf9c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009c80 tx=0x7f7e4801cd70 comp rx=0 tx=0).stop 2024-09-13T08:51:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 -- 172.21.15.44:0/1352194100 shutdown_connections 2024-09-13T08:51:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 --2- 172.21.15.44:0/1352194100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e500ce980 0x7f7e500cf9c0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 --2- 172.21.15.44:0/1352194100 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e501603f0 0x7f7e500b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 -- 172.21.15.44:0/1352194100 >> 172.21.15.44:0/1352194100 conn(0x7f7e500d1df0 msgr2=0x7f7e500d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 -- 172.21.15.44:0/1352194100 shutdown_connections 2024-09-13T08:51:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 -- 172.21.15.44:0/1352194100 wait complete. 2024-09-13T08:51:34.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 Processor -- start 2024-09-13T08:51:34.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 -- start start 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500b28d0 0x7f7e5015de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e500ce980 0x7f7e50159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e501603f0 0x7f7e501599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50169670 con 0x7f7e501603f0 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501694f0 con 0x7f7e500ce980 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e56800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7e501697f0 con 0x7f7e500b28d0 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e4fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e501603f0 0x7f7e501599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e4fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e501603f0 0x7f7e501599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59220/0 (socket says 172.21.15.44:59220) 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e4fe00640 1 -- 172.21.15.44:0/2864543502 learned_addr learned my addr 172.21.15.44:0/2864543502 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e47e00640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e500ce980 0x7f7e50159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e4f400640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500b28d0 0x7f7e5015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e4fe00640 1 -- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500b28d0 msgr2=0x7f7e5015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e4fe00640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500b28d0 0x7f7e5015de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e4fe00640 1 -- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e500ce980 msgr2=0x7f7e50159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e4fe00640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e500ce980 0x7f7e50159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e4fe00640 1 -- 172.21.15.44:0/2864543502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e500aeaa0 con 0x7f7e501603f0 2024-09-13T08:51:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.889+0000 7f7e4f400640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500b28d0 0x7f7e5015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.893+0000 7f7e4fe00640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e501603f0 0x7f7e501599d0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f7e4801cba0 tx=0x7f7e480a5ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:34.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.893+0000 7f7e47400640 1 -- 172.21.15.44:0/2864543502 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480bb070 con 0x7f7e501603f0 2024-09-13T08:51:34.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.893+0000 7f7e47400640 1 -- 172.21.15.44:0/2864543502 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e48004d60 con 0x7f7e501603f0 2024-09-13T08:51:34.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.893+0000 7f7e47400640 1 -- 172.21.15.44:0/2864543502 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480b2040 con 0x7f7e501603f0 2024-09-13T08:51:34.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.893+0000 7f7e56800640 1 -- 172.21.15.44:0/2864543502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5015ebf0 con 0x7f7e501603f0 2024-09-13T08:51:34.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.893+0000 7f7e56800640 1 -- 172.21.15.44:0/2864543502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5015eed0 con 0x7f7e501603f0 2024-09-13T08:51:34.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.893+0000 7f7e56800640 1 -- 172.21.15.44:0/2864543502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e500cf9c0 con 0x7f7e501603f0 2024-09-13T08:51:34.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.893+0000 7f7e47400640 1 -- 172.21.15.44:0/2864543502 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7f7e48004320 con 0x7f7e501603f0 2024-09-13T08:51:34.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.893+0000 7f7e47400640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e2c074d20 0x7f7e2c0771e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:34.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.893+0000 7f7e47400640 1 -- 172.21.15.44:0/2864543502 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f7e4812cee0 con 0x7f7e501603f0 2024-09-13T08:51:34.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.897+0000 7f7e4f400640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e2c074d20 0x7f7e2c0771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:34.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.897+0000 7f7e4f400640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e2c074d20 0x7f7e2c0771e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7e40005fd0 tx=0x7f7e4000a540 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:34.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:34.897+0000 7f7e47400640 1 -- 172.21.15.44:0/2864543502 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7e48016990 con 0x7f7e501603f0 2024-09-13T08:51:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.097+0000 7f7e56800640 1 -- 172.21.15.44:0/2864543502 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e500b7d70 con 0x7f7e2c074d20 2024-09-13T08:51:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e47400640 1 -- 172.21.15.44:0/2864543502 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+185 (secure 0 0 0) 0x7f7e500b7d70 con 0x7f7e2c074d20 2024-09-13T08:51:35.107 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:35.107 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:51:35.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:35.107 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone45030" 2024-09-13T08:51:35.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:35.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sv1" 2024-09-13T08:51:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone47392" 2024-09-13T08:51:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone15351" 2024-09-13T08:51:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:35.108 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:51:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 -- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e2c074d20 msgr2=0x7f7e2c0771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e2c074d20 0x7f7e2c0771e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7e40005fd0 tx=0x7f7e4000a540 comp rx=0 tx=0).stop 2024-09-13T08:51:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 -- 172.21.15.44:0/2864543502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e501603f0 msgr2=0x7f7e501599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e501603f0 0x7f7e501599d0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f7e4801cba0 tx=0x7f7e480a5ba0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 -- 172.21.15.44:0/2864543502 shutdown_connections 2024-09-13T08:51:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7e2c074d20 0x7f7e2c0771e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e501603f0 0x7f7e501599d0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e500ce980 0x7f7e50159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 --2- 172.21.15.44:0/2864543502 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7e500b28d0 0x7f7e5015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 -- 172.21.15.44:0/2864543502 >> 172.21.15.44:0/2864543502 conn(0x7f7e500d1df0 msgr2=0x7f7e500b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 -- 172.21.15.44:0/2864543502 shutdown_connections 2024-09-13T08:51:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.109+0000 7f7e56800640 1 -- 172.21.15.44:0/2864543502 wait complete. 2024-09-13T08:51:35.129 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone45030 2024-09-13T08:51:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 -- 172.21.15.44:0/3183709232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9d80cf440 msgr2=0x7fd9d80ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 --2- 172.21.15.44:0/3183709232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9d80cf440 0x7fd9d80ce350 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009840 tx=0x7fd9c001c820 comp rx=0 tx=0).stop 2024-09-13T08:51:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 -- 172.21.15.44:0/3183709232 shutdown_connections 2024-09-13T08:51:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 --2- 172.21.15.44:0/3183709232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9d80cf440 0x7fd9d80ce350 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 --2- 172.21.15.44:0/3183709232 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9d80734f0 0x7fd9d815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 -- 172.21.15.44:0/3183709232 >> 172.21.15.44:0/3183709232 conn(0x7fd9d80cfbc0 msgr2=0x7fd9d80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:35.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 -- 172.21.15.44:0/3183709232 shutdown_connections 2024-09-13T08:51:35.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 -- 172.21.15.44:0/3183709232 wait complete. 2024-09-13T08:51:35.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 Processor -- start 2024-09-13T08:51:35.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 -- start start 2024-09-13T08:51:35.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9d80734f0 0x7fd9d8164b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:35.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9d80ce890 0x7fd9d81face0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:35.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9d80cf440 0x7fd9d8159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:35.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d8167360 con 0x7fd9d80cf440 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d815edd0 con 0x7fd9d80ce890 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.281+0000 7fd9dd800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d81674e0 con 0x7fd9d80734f0 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9d80cf440 0x7fd9d8159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cfe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9d80ce890 0x7fd9d81face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9d6a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9d80734f0 0x7fd9d8164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9d80cf440 0x7fd9d8159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59236/0 (socket says 172.21.15.44:59236) 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9d7400640 1 -- 172.21.15.44:0/1522572885 learned_addr learned my addr 172.21.15.44:0/1522572885 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cfe00640 1 -- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9d80734f0 msgr2=0x7fd9d8164b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cfe00640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9d80734f0 0x7fd9d8164b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cfe00640 1 -- 172.21.15.44:0/1522572885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9d80cf440 msgr2=0x7fd9d8159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cfe00640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9d80cf440 0x7fd9d8159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cfe00640 1 -- 172.21.15.44:0/1522572885 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d8165070 con 0x7fd9d80ce890 2024-09-13T08:51:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9d7400640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9d80cf440 0x7fd9d8159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:51:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cfe00640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9d80ce890 0x7fd9d81face0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00ab000 tx=0x7fd9c00a7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cea00640 1 -- 172.21.15.44:0/1522572885 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00ac070 con 0x7fd9d80ce890 2024-09-13T08:51:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cea00640 1 -- 172.21.15.44:0/1522572885 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9c0004320 con 0x7fd9d80ce890 2024-09-13T08:51:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cea00640 1 -- 172.21.15.44:0/1522572885 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00052d0 con 0x7fd9d80ce890 2024-09-13T08:51:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9dd800640 1 -- 172.21.15.44:0/1522572885 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d81f9c40 con 0x7fd9d80ce890 2024-09-13T08:51:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9dd800640 1 -- 172.21.15.44:0/1522572885 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d81fa200 con 0x7fd9d80ce890 2024-09-13T08:51:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9dd800640 1 -- 172.21.15.44:0/1522572885 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d8069410 con 0x7fd9d80ce890 2024-09-13T08:51:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.285+0000 7fd9cea00640 1 -- 172.21.15.44:0/1522572885 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7fd9c00af070 con 0x7fd9d80ce890 2024-09-13T08:51:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.289+0000 7fd9cea00640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9b4074d50 0x7fd9b4077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.289+0000 7fd9cea00640 1 -- 172.21.15.44:0/1522572885 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7fd9c012d330 con 0x7fd9d80ce890 2024-09-13T08:51:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.289+0000 7fd9d6a00640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9b4074d50 0x7fd9b4077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.289+0000 7fd9d6a00640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9b4074d50 0x7fd9b4077210 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8075030 tx=0x7fd9c8010040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.293+0000 7fd9cea00640 1 -- 172.21.15.44:0/1522572885 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd9c00b6070 con 0x7fd9d80ce890 2024-09-13T08:51:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.493+0000 7fd9dd800640 1 -- 172.21.15.44:0/1522572885 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone45030", "target": ["mon-mgr", ""]}) -- 0x7fd9d80d0850 con 0x7fd9b4074d50 2024-09-13T08:51:35.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.513+0000 7fd9cea00640 1 -- 172.21.15.44:0/1522572885 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd9d80d0850 con 0x7fd9b4074d50 2024-09-13T08:51:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 -- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9b4074d50 msgr2=0x7fd9b4077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9b4074d50 0x7fd9b4077210 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8075030 tx=0x7fd9c8010040 comp rx=0 tx=0).stop 2024-09-13T08:51:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 -- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9d80ce890 msgr2=0x7fd9d81face0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9d80ce890 0x7fd9d81face0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00ab000 tx=0x7fd9c00a7000 comp rx=0 tx=0).stop 2024-09-13T08:51:35.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 -- 172.21.15.44:0/1522572885 shutdown_connections 2024-09-13T08:51:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd9b4074d50 0x7fd9b4077210 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9d80cf440 0x7fd9d8159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9d80ce890 0x7fd9d81face0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 --2- 172.21.15.44:0/1522572885 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd9d80734f0 0x7fd9d8164b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 -- 172.21.15.44:0/1522572885 >> 172.21.15.44:0/1522572885 conn(0x7fd9d80cfbc0 msgr2=0x7fd9d81546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 -- 172.21.15.44:0/1522572885 shutdown_connections 2024-09-13T08:51:35.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.517+0000 7fd9dd800640 1 -- 172.21.15.44:0/1522572885 wait complete. 2024-09-13T08:51:35.528 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:35.531+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:35.528 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:35.531+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:35.528 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:35.531+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:35.528 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:35.531+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:35.528 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:51:35.531+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:51:35.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-13T08:51:35.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 -- 172.21.15.44:0/2840203046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db415a8c0 msgr2=0x7f7db415f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 --2- 172.21.15.44:0/2840203046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db415a8c0 0x7f7db415f100 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009900 tx=0x7f7da801c8e0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 -- 172.21.15.44:0/2840203046 shutdown_connections 2024-09-13T08:51:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 --2- 172.21.15.44:0/2840203046 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7db4073830 0x7f7db406fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 --2- 172.21.15.44:0/2840203046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db415a8c0 0x7f7db415f100 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 -- 172.21.15.44:0/2840203046 >> 172.21.15.44:0/2840203046 conn(0x7f7db40b2140 msgr2=0x7f7db40b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 -- 172.21.15.44:0/2840203046 shutdown_connections 2024-09-13T08:51:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 -- 172.21.15.44:0/2840203046 wait complete. 2024-09-13T08:51:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 Processor -- start 2024-09-13T08:51:35.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 -- start start 2024-09-13T08:51:35.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.697+0000 7f7dbc200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db4073830 0x7f7db41f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:35.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7dbc200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db415a8c0 0x7f7db40cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7dbc200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7db415aca0 0x7f7db40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7dbc200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7db4167a90 con 0x7f7db415a8c0 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7dbc200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7db4070dd0 con 0x7f7db4073830 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7dbc200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7db4167c10 con 0x7f7db415aca0 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db415a8c0 0x7f7db40cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db415a8c0 0x7f7db40cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59260/0 (socket says 172.21.15.44:59260) 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db8e00640 1 -- 172.21.15.44:0/201155368 learned_addr learned my addr 172.21.15.44:0/201155368 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db9800640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db4073830 0x7f7db41f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7dba200640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7db415aca0 0x7f7db40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db8e00640 1 -- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7db415aca0 msgr2=0x7f7db40b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db8e00640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7db415aca0 0x7f7db40b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db8e00640 1 -- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db4073830 msgr2=0x7f7db41f8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db8e00640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db4073830 0x7f7db41f8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db8e00640 1 -- 172.21.15.44:0/201155368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db4159f80 con 0x7f7db415a8c0 2024-09-13T08:51:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db8e00640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db415a8c0 0x7f7db40cd150 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f7da000c3c0 tx=0x7f7da000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db1600640 1 -- 172.21.15.44:0/201155368 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0017070 con 0x7f7db415a8c0 2024-09-13T08:51:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db1600640 1 -- 172.21.15.44:0/201155368 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7da00053c0 con 0x7f7db415a8c0 2024-09-13T08:51:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db1600640 1 -- 172.21.15.44:0/201155368 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0003860 con 0x7f7db415a8c0 2024-09-13T08:51:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7dbc200640 1 -- 172.21.15.44:0/201155368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db415c660 con 0x7f7db415a8c0 2024-09-13T08:51:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7dbc200640 1 -- 172.21.15.44:0/201155368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db415cc20 con 0x7f7db415a8c0 2024-09-13T08:51:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7dbc200640 1 -- 172.21.15.44:0/201155368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db406fae0 con 0x7f7db415a8c0 2024-09-13T08:51:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.701+0000 7f7db1600640 1 -- 172.21.15.44:0/201155368 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7f7da001a070 con 0x7f7db415a8c0 2024-09-13T08:51:35.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.705+0000 7f7db1600640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d94074d50 0x7f7d94077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:35.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.705+0000 7f7db1600640 1 -- 172.21.15.44:0/201155368 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f7da0092070 con 0x7f7db415a8c0 2024-09-13T08:51:35.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.705+0000 7f7db9800640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d94074d50 0x7f7d94077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:35.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.705+0000 7f7db9800640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d94074d50 0x7f7d94077210 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7da80af000 tx=0x7f7da801b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:35.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.709+0000 7f7db1600640 1 -- 172.21.15.44:0/201155368 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7db415f640 con 0x7f7db415a8c0 2024-09-13T08:51:35.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.917+0000 7f7dbc200640 1 -- 172.21.15.44:0/201155368 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7db41f8bd0 con 0x7f7d94074d50 2024-09-13T08:51:35.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7db1600640 1 -- 172.21.15.44:0/201155368 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7db41f8bd0 con 0x7f7d94074d50 2024-09-13T08:51:35.932 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:35.933 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:51:35.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:35.933 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1" 2024-09-13T08:51:35.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:35.933 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 -- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d94074d50 msgr2=0x7f7d94077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d94074d50 0x7f7d94077210 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7da80af000 tx=0x7f7da801b120 comp rx=0 tx=0).stop 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 -- 172.21.15.44:0/201155368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db415a8c0 msgr2=0x7f7db40cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db415a8c0 0x7f7db40cd150 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f7da000c3c0 tx=0x7f7da000c880 comp rx=0 tx=0).stop 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 -- 172.21.15.44:0/201155368 shutdown_connections 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7d94074d50 0x7f7d94077210 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7db415aca0 0x7f7db40b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db415a8c0 0x7f7db40cd150 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 --2- 172.21.15.44:0/201155368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db4073830 0x7f7db41f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 -- 172.21.15.44:0/201155368 >> 172.21.15.44:0/201155368 conn(0x7f7db40b2140 msgr2=0x7f7db40cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:35.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 -- 172.21.15.44:0/201155368 shutdown_connections 2024-09-13T08:51:35.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:35.933+0000 7f7dbc200640 1 -- 172.21.15.44:0/201155368 wait complete. 2024-09-13T08:51:35.954 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-13T08:51:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.105+0000 7f8db0600640 1 -- 172.21.15.44:0/372622751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8da80afb10 msgr2=0x7f8da80affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.105+0000 7f8db0600640 1 --2- 172.21.15.44:0/372622751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8da80afb10 0x7f8da80affc0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8da8073810 tx=0x7f8da001cd80 comp rx=0 tx=0).stop 2024-09-13T08:51:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.105+0000 7f8db0600640 1 -- 172.21.15.44:0/372622751 shutdown_connections 2024-09-13T08:51:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.105+0000 7f8db0600640 1 --2- 172.21.15.44:0/372622751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8da80afb10 0x7f8da80affc0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.105+0000 7f8db0600640 1 --2- 172.21.15.44:0/372622751 >> v2:172.21.15.129:3300/0 conn(0x7f8da81605d0 0x7f8da80d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.105+0000 7f8db0600640 1 -- 172.21.15.44:0/372622751 >> 172.21.15.44:0/372622751 conn(0x7f8da80d1dd0 msgr2=0x7f8da80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.105+0000 7f8db0600640 1 -- 172.21.15.44:0/372622751 shutdown_connections 2024-09-13T08:51:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.105+0000 7f8db0600640 1 -- 172.21.15.44:0/372622751 wait complete. 2024-09-13T08:51:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8db0600640 1 Processor -- start 2024-09-13T08:51:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8db0600640 1 -- start start 2024-09-13T08:51:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8db0600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8da80afb10 0x7f8da8155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8db0600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8da80b1650 0x7f8da80ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8db0600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8da81605d0 0x7f8da80ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8db0600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8169540 con 0x7f8da80b1650 2024-09-13T08:51:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8db0600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8da80b1430 con 0x7f8da80afb10 2024-09-13T08:51:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8db0600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8da81696c0 con 0x7f8da81605d0 2024-09-13T08:51:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8dad200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8da80b1650 0x7f8da80ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8dad200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8da80b1650 0x7f8da80ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59268/0 (socket says 172.21.15.44:59268) 2024-09-13T08:51:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8dad200640 1 -- 172.21.15.44:0/686164504 learned_addr learned my addr 172.21.15.44:0/686164504 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8dadc00640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8da80afb10 0x7f8da8155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8dad200640 1 -- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8da81605d0 msgr2=0x7f8da80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8dad200640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8da81605d0 0x7f8da80ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8dad200640 1 -- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8da80afb10 msgr2=0x7f8da8155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8dad200640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8da80afb10 0x7f8da8155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8dad200640 1 -- 172.21.15.44:0/686164504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da81559b0 con 0x7f8da80b1650 2024-09-13T08:51:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8dad200640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8da80b1650 0x7f8da80ce220 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400c490 tx=0x7f8d94001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8da5600640 1 -- 172.21.15.44:0/686164504 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94017070 con 0x7f8da80b1650 2024-09-13T08:51:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8da5600640 1 -- 172.21.15.44:0/686164504 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d94003600 con 0x7f8da80b1650 2024-09-13T08:51:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8db0600640 1 -- 172.21.15.44:0/686164504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da8155c90 con 0x7f8da80b1650 2024-09-13T08:51:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8da5600640 1 -- 172.21.15.44:0/686164504 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94003a90 con 0x7f8da80b1650 2024-09-13T08:51:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.109+0000 7f8db0600640 1 -- 172.21.15.44:0/686164504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da8156250 con 0x7f8da80b1650 2024-09-13T08:51:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.113+0000 7f8db0600640 1 -- 172.21.15.44:0/686164504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da80affc0 con 0x7f8da80b1650 2024-09-13T08:51:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.113+0000 7f8da5600640 1 -- 172.21.15.44:0/686164504 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7f8d9401a070 con 0x7f8da80b1650 2024-09-13T08:51:36.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.113+0000 7f8da5600640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d88074d20 0x7f8d880771e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.113+0000 7f8da5600640 1 -- 172.21.15.44:0/686164504 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f8d940920d0 con 0x7f8da80b1650 2024-09-13T08:51:36.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.117+0000 7f8dadc00640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d88074d20 0x7f8d880771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:36.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.117+0000 7f8da5600640 1 -- 172.21.15.44:0/686164504 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8d9405e7d0 con 0x7f8da80b1650 2024-09-13T08:51:36.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.121+0000 7f8dadc00640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d88074d20 0x7f8d880771e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8da8166cf0 tx=0x7f8d9c006f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:36.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.325+0000 7f8db0600640 1 -- 172.21.15.44:0/686164504 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8da80d1080 con 0x7f8d88074d20 2024-09-13T08:51:36.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.353+0000 7f8da5600640 1 -- 172.21.15.44:0/686164504 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8da80d1080 con 0x7f8d88074d20 2024-09-13T08:51:36.354 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:36.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 -- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d88074d20 msgr2=0x7f8d880771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d88074d20 0x7f8d880771e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8da8166cf0 tx=0x7f8d9c006f90 comp rx=0 tx=0).stop 2024-09-13T08:51:36.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 -- 172.21.15.44:0/686164504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8da80b1650 msgr2=0x7f8da80ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8da80b1650 0x7f8da80ce220 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400c490 tx=0x7f8d94001e80 comp rx=0 tx=0).stop 2024-09-13T08:51:36.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 -- 172.21.15.44:0/686164504 shutdown_connections 2024-09-13T08:51:36.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d88074d20 0x7f8d880771e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8da81605d0 0x7f8da80ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8da80b1650 0x7f8da80ce220 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 --2- 172.21.15.44:0/686164504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8da80afb10 0x7f8da8155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 -- 172.21.15.44:0/686164504 >> 172.21.15.44:0/686164504 conn(0x7f8da80d1dd0 msgr2=0x7f8da815a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:36.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 -- 172.21.15.44:0/686164504 shutdown_connections 2024-09-13T08:51:36.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.357+0000 7f8db0600640 1 -- 172.21.15.44:0/686164504 wait complete. 2024-09-13T08:51:36.376 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-13T08:51:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.525+0000 7fad34000640 1 -- 172.21.15.44:0/535128365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2c0b2cf0 msgr2=0x7fad2c160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.525+0000 7fad34000640 1 --2- 172.21.15.44:0/535128365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2c0b2cf0 0x7fad2c160330 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fad24009c80 tx=0x7fad2401f5a0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.525+0000 7fad34000640 1 -- 172.21.15.44:0/535128365 shutdown_connections 2024-09-13T08:51:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.525+0000 7fad34000640 1 --2- 172.21.15.44:0/535128365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2c0b2cf0 0x7fad2c160330 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.525+0000 7fad34000640 1 --2- 172.21.15.44:0/535128365 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad2c0cfe90 0x7fad2c0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.525+0000 7fad34000640 1 -- 172.21.15.44:0/535128365 >> 172.21.15.44:0/535128365 conn(0x7fad2c0d1dc0 msgr2=0x7fad2c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.525+0000 7fad34000640 1 -- 172.21.15.44:0/535128365 shutdown_connections 2024-09-13T08:51:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.525+0000 7fad34000640 1 -- 172.21.15.44:0/535128365 wait complete. 2024-09-13T08:51:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 Processor -- start 2024-09-13T08:51:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 -- start start 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2c0b2cf0 0x7fad2c14f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad2c0cfe90 0x7fad2c14c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad2c0d07b0 0x7fad2c0b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c0afb20 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c0af9a0 con 0x7fad2c0d07b0 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fad2c0afca0 con 0x7fad2c0cfe90 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad31600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2c0b2cf0 0x7fad2c14f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad31600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2c0b2cf0 0x7fad2c14f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59298/0 (socket says 172.21.15.44:59298) 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad31600640 1 -- 172.21.15.44:0/301325668 learned_addr learned my addr 172.21.15.44:0/301325668 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad30c00640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad2c0cfe90 0x7fad2c14c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad31600640 1 -- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad2c0cfe90 msgr2=0x7fad2c14c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad31600640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad2c0cfe90 0x7fad2c14c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad31600640 1 -- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad2c0d07b0 msgr2=0x7fad2c0b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad31600640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad2c0d07b0 0x7fad2c0b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad31600640 1 -- 172.21.15.44:0/301325668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2c14fd50 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad31600640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2c0b2cf0 0x7fad2c14f810 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fad2000b550 tx=0x7fad2000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad29600640 1 -- 172.21.15.44:0/301325668 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20014070 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad29600640 1 -- 172.21.15.44:0/301325668 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad20010980 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad29600640 1 -- 172.21.15.44:0/301325668 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20010dd0 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 -- 172.21.15.44:0/301325668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2c0b5fd0 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 -- 172.21.15.44:0/301325668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2c0b6590 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.529+0000 7fad34000640 1 -- 172.21.15.44:0/301325668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad2c0b6ca0 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.533+0000 7fad29600640 1 -- 172.21.15.44:0/301325668 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7fad20017070 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.533+0000 7fad29600640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad0c074d20 0x7fad0c0771e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.533+0000 7fad29600640 1 -- 172.21.15.44:0/301325668 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7fad20092c40 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.533+0000 7fad30c00640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad0c074d20 0x7fad0c0771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.533+0000 7fad30c00640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad0c074d20 0x7fad0c0771e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fad2c1f9000 tx=0x7fad18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:36.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.537+0000 7fad29600640 1 -- 172.21.15.44:0/301325668 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fad2c14fd50 con 0x7fad2c0b2cf0 2024-09-13T08:51:36.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.737+0000 7fad34000640 1 -- 172.21.15.44:0/301325668 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fad2c0d0270 con 0x7fad0c074d20 2024-09-13T08:51:36.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.765+0000 7fad29600640 1 -- 172.21.15.44:0/301325668 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad2c0d0270 con 0x7fad0c074d20 2024-09-13T08:51:36.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 -- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad0c074d20 msgr2=0x7fad0c0771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad0c074d20 0x7fad0c0771e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fad2c1f9000 tx=0x7fad18009040 comp rx=0 tx=0).stop 2024-09-13T08:51:36.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 -- 172.21.15.44:0/301325668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2c0b2cf0 msgr2=0x7fad2c14f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2c0b2cf0 0x7fad2c14f810 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fad2000b550 tx=0x7fad2000ba10 comp rx=0 tx=0).stop 2024-09-13T08:51:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 -- 172.21.15.44:0/301325668 shutdown_connections 2024-09-13T08:51:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fad0c074d20 0x7fad0c0771e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad2c0d07b0 0x7fad2c0b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fad2c0cfe90 0x7fad2c14c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 --2- 172.21.15.44:0/301325668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2c0b2cf0 0x7fad2c14f810 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 -- 172.21.15.44:0/301325668 >> 172.21.15.44:0/301325668 conn(0x7fad2c0d1dc0 msgr2=0x7fad2c070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 -- 172.21.15.44:0/301325668 shutdown_connections 2024-09-13T08:51:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.769+0000 7fad34000640 1 -- 172.21.15.44:0/301325668 wait complete. 2024-09-13T08:51:36.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone47392 2024-09-13T08:51:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.941+0000 7f0df0a00640 1 -- 172.21.15.44:0/1247831161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dec15be10 msgr2=0x7f0dec0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.941+0000 7f0df0a00640 1 --2- 172.21.15.44:0/1247831161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dec15be10 0x7f0dec0ce700 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009900 tx=0x7f0ddc01f0d0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.941+0000 7f0df0a00640 1 -- 172.21.15.44:0/1247831161 shutdown_connections 2024-09-13T08:51:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.941+0000 7f0df0a00640 1 --2- 172.21.15.44:0/1247831161 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0dec15c1f0 0x7f0dec0cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.941+0000 7f0df0a00640 1 --2- 172.21.15.44:0/1247831161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dec15be10 0x7f0dec0ce700 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.941+0000 7f0df0a00640 1 -- 172.21.15.44:0/1247831161 >> 172.21.15.44:0/1247831161 conn(0x7f0dec0d1da0 msgr2=0x7f0dec0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:36.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.941+0000 7f0df0a00640 1 -- 172.21.15.44:0/1247831161 shutdown_connections 2024-09-13T08:51:36.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.941+0000 7f0df0a00640 1 -- 172.21.15.44:0/1247831161 wait complete. 2024-09-13T08:51:36.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.941+0000 7f0df0a00640 1 Processor -- start 2024-09-13T08:51:36.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.941+0000 7f0df0a00640 1 -- start start 2024-09-13T08:51:36.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0df0a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dec0b0b60 0x7f0dec0b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0df0a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0dec15be10 0x7f0dec1f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0df0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dec15c1f0 0x7f0dec1f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0df0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec0cf510 con 0x7f0dec15c1f0 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0df0a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec0cf390 con 0x7f0dec0b0b60 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0df0a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0dec160290 con 0x7f0dec15be10 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0dea800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dec15c1f0 0x7f0dec1f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0de9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dec0b0b60 0x7f0dec0b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0dea800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dec15c1f0 0x7f0dec1f41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59310/0 (socket says 172.21.15.44:59310) 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0dea800640 1 -- 172.21.15.44:0/3595705332 learned_addr learned my addr 172.21.15.44:0/3595705332 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0de9400640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0dec15be10 0x7f0dec1f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0dea800640 1 -- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0dec15be10 msgr2=0x7f0dec1f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0dea800640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0dec15be10 0x7f0dec1f3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0dea800640 1 -- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dec0b0b60 msgr2=0x7f0dec0b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0dea800640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dec0b0b60 0x7f0dec0b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:36.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0dea800640 1 -- 172.21.15.44:0/3595705332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dec0b5400 con 0x7f0dec15c1f0 2024-09-13T08:51:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0dea800640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dec15c1f0 0x7f0dec1f41d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c7a0 tx=0x7f0de400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0de2000640 1 -- 172.21.15.44:0/3595705332 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4003b80 con 0x7f0dec15c1f0 2024-09-13T08:51:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0de2000640 1 -- 172.21.15.44:0/3595705332 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0de4003d40 con 0x7f0dec15c1f0 2024-09-13T08:51:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0de2000640 1 -- 172.21.15.44:0/3595705332 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de400aab0 con 0x7f0dec15c1f0 2024-09-13T08:51:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0df0a00640 1 -- 172.21.15.44:0/3595705332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dec0c4df0 con 0x7f0dec15c1f0 2024-09-13T08:51:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0df0a00640 1 -- 172.21.15.44:0/3595705332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dec0b5120 con 0x7f0dec15c1f0 2024-09-13T08:51:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.945+0000 7f0df0a00640 1 -- 172.21.15.44:0/3595705332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dec160780 con 0x7f0dec15c1f0 2024-09-13T08:51:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.949+0000 7f0de2000640 1 -- 172.21.15.44:0/3595705332 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7f0de400ac50 con 0x7f0dec15c1f0 2024-09-13T08:51:36.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.949+0000 7f0de2000640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0dcc074d20 0x7f0dcc0771e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:36.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.949+0000 7f0de2000640 1 -- 172.21.15.44:0/3595705332 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f0de4092b20 con 0x7f0dec15c1f0 2024-09-13T08:51:36.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.949+0000 7f0de9e00640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0dcc074d20 0x7f0dcc0771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.949+0000 7f0de9e00640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0dcc074d20 0x7f0dcc0771e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0b2000 tx=0x7f0ddc0aa040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:36.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:36.953+0000 7f0de2000640 1 -- 172.21.15.44:0/3595705332 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0dec0b5400 con 0x7f0dec15c1f0 2024-09-13T08:51:37.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.157+0000 7f0df0a00640 1 -- 172.21.15.44:0/3595705332 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone47392", "target": ["mon-mgr", ""]}) -- 0x7f0dec1f4710 con 0x7f0dcc074d20 2024-09-13T08:51:37.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.541+0000 7f0de2000640 1 -- 172.21.15.44:0/3595705332 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ddc004140 con 0x7f0dcc074d20 2024-09-13T08:51:37.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 -- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0dcc074d20 msgr2=0x7f0dcc0771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:37.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0dcc074d20 0x7f0dcc0771e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0b2000 tx=0x7f0ddc0aa040 comp rx=0 tx=0).stop 2024-09-13T08:51:37.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 -- 172.21.15.44:0/3595705332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dec15c1f0 msgr2=0x7f0dec1f41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:37.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dec15c1f0 0x7f0dec1f41d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c7a0 tx=0x7f0de400cc60 comp rx=0 tx=0).stop 2024-09-13T08:51:37.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 -- 172.21.15.44:0/3595705332 shutdown_connections 2024-09-13T08:51:37.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0dcc074d20 0x7f0dcc0771e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dec15c1f0 0x7f0dec1f41d0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0dec15be10 0x7f0dec1f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 --2- 172.21.15.44:0/3595705332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dec0b0b60 0x7f0dec0b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 -- 172.21.15.44:0/3595705332 >> 172.21.15.44:0/3595705332 conn(0x7f0dec0d1da0 msgr2=0x7f0dec071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:37.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 -- 172.21.15.44:0/3595705332 shutdown_connections 2024-09-13T08:51:37.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.545+0000 7f0df0a00640 1 -- 172.21.15.44:0/3595705332 wait complete. 2024-09-13T08:51:37.565 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15351 2024-09-13T08:51:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.717+0000 7fcfe2400640 1 -- 172.21.15.44:0/2282585 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcfdc15be10 msgr2=0x7fcfdc0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.717+0000 7fcfe2400640 1 --2- 172.21.15.44:0/2282585 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcfdc15be10 0x7fcfdc0ce700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009840 tx=0x7fcfcc01ef30 comp rx=0 tx=0).stop 2024-09-13T08:51:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.717+0000 7fcfe2400640 1 -- 172.21.15.44:0/2282585 shutdown_connections 2024-09-13T08:51:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.717+0000 7fcfe2400640 1 --2- 172.21.15.44:0/2282585 >> v2:172.21.15.129:3300/0 conn(0x7fcfdc0b0b60 0x7fcfdc0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.717+0000 7fcfe2400640 1 --2- 172.21.15.44:0/2282585 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcfdc15be10 0x7fcfdc0ce700 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.717+0000 7fcfe2400640 1 -- 172.21.15.44:0/2282585 >> 172.21.15.44:0/2282585 conn(0x7fcfdc0d1da0 msgr2=0x7fcfdc0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.717+0000 7fcfe2400640 1 -- 172.21.15.44:0/2282585 shutdown_connections 2024-09-13T08:51:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.717+0000 7fcfe2400640 1 -- 172.21.15.44:0/2282585 wait complete. 2024-09-13T08:51:37.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfe2400640 1 Processor -- start 2024-09-13T08:51:37.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfe2400640 1 -- start start 2024-09-13T08:51:37.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfe2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfdc0b0b60 0x7fcfdc15ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:37.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfe2400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcfdc15be10 0x7fcfdc14aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:37.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfe2400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfdc15c1f0 0x7fcfdc14b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfe2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc0cd320 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfe2400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc0cf590 con 0x7fcfdc15c1f0 2024-09-13T08:51:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfe2400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcfdc0cd4a0 con 0x7fcfdc15be10 2024-09-13T08:51:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfdc0b0b60 0x7fcfdc15ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfdc0b0b60 0x7fcfdc15ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59314/0 (socket says 172.21.15.44:59314) 2024-09-13T08:51:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdb400640 1 -- 172.21.15.44:0/3428105340 learned_addr learned my addr 172.21.15.44:0/3428105340 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfd3e00640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcfdc15be10 0x7fcfdc14aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdbe00640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfdc15c1f0 0x7fcfdc14b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdb400640 1 -- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcfdc15be10 msgr2=0x7fcfdc14aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdb400640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcfdc15be10 0x7fcfdc14aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdb400640 1 -- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfdc15c1f0 msgr2=0x7fcfdc14b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdb400640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfdc15c1f0 0x7fcfdc14b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdb400640 1 -- 172.21.15.44:0/3428105340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc0c7580 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdbe00640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfdc15c1f0 0x7fcfdc14b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:51:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfd3e00640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcfdc15be10 0x7fcfdc14aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:51:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfdb400640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfdc0b0b60 0x7fcfdc15ab70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc01d5e0 tx=0x7fcfcc005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfd3400640 1 -- 172.21.15.44:0/3428105340 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0b5040 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfd3400640 1 -- 172.21.15.44:0/3428105340 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfcc005310 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfd3400640 1 -- 172.21.15.44:0/3428105340 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0037c0 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.721+0000 7fcfe2400640 1 -- 172.21.15.44:0/3428105340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc0c7860 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.725+0000 7fcfe2400640 1 -- 172.21.15.44:0/3428105340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc0c7e20 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.725+0000 7fcfe2400640 1 -- 172.21.15.44:0/3428105340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc0ccbd0 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.725+0000 7fcfd3400640 1 -- 172.21.15.44:0/3428105340 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 99834+0+0 (secure 0 0 0) 0x7fcfcc003960 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.725+0000 7fcfd3400640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfb8074d20 0x7fcfb80771e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:37.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.725+0000 7fcfd3400640 1 -- 172.21.15.44:0/3428105340 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7fcfcc12fa90 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.725+0000 7fcfd3e00640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfb8074d20 0x7fcfb80771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:37.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.729+0000 7fcfd3e00640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfb8074d20 0x7fcfb80771e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4000c10 tx=0x7fcfc4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:37.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.729+0000 7fcfd3400640 1 -- 172.21.15.44:0/3428105340 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcfcc018a90 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.889+0000 7fcfd3400640 1 -- 172.21.15.44:0/3428105340 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fcfcc0f8350 con 0x7fcfdc0b0b60 2024-09-13T08:51:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.933+0000 7fcfe2400640 1 -- 172.21.15.44:0/3428105340 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15351", "target": ["mon-mgr", ""]}) -- 0x7fcfdc0c7330 con 0x7fcfb8074d20 2024-09-13T08:51:37.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.949+0000 7fcfd3400640 1 -- 172.21.15.44:0/3428105340 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcfdc0c7330 con 0x7fcfb8074d20 2024-09-13T08:51:37.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 -- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfb8074d20 msgr2=0x7fcfb80771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:37.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfb8074d20 0x7fcfb80771e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4000c10 tx=0x7fcfc4009040 comp rx=0 tx=0).stop 2024-09-13T08:51:37.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 -- 172.21.15.44:0/3428105340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfdc0b0b60 msgr2=0x7fcfdc15ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:37.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfdc0b0b60 0x7fcfdc15ab70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc01d5e0 tx=0x7fcfcc005d20 comp rx=0 tx=0).stop 2024-09-13T08:51:37.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 -- 172.21.15.44:0/3428105340 shutdown_connections 2024-09-13T08:51:37.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcfb8074d20 0x7fcfb80771e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfdc15c1f0 0x7fcfdc14b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcfdc15be10 0x7fcfdc14aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 --2- 172.21.15.44:0/3428105340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfdc0b0b60 0x7fcfdc15ab70 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:37.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 -- 172.21.15.44:0/3428105340 >> 172.21.15.44:0/3428105340 conn(0x7fcfdc0d1da0 msgr2=0x7fcfdc070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:37.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 -- 172.21.15.44:0/3428105340 shutdown_connections 2024-09-13T08:51:37.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:37.953+0000 7fcfe2400640 1 -- 172.21.15.44:0/3428105340 wait complete. 2024-09-13T08:51:37.977 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-13T08:51:38.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:51:38.032 INFO:teuthology.orchestra.run.smithi044.stdout:2 2024-09-13T08:51:38.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:51:38.039 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:51:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.189+0000 7f3796200640 1 -- 172.21.15.44:0/916714793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3774005920 msgr2=0x7f3774005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.189+0000 7f3796200640 1 --2- 172.21.15.44:0/916714793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3774005920 0x7f3774005da0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f377800c310 tx=0x7f377801cf20 comp rx=0 tx=0).stop 2024-09-13T08:51:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.189+0000 7f3796200640 1 -- 172.21.15.44:0/916714793 shutdown_connections 2024-09-13T08:51:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.189+0000 7f3796200640 1 --2- 172.21.15.44:0/916714793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3774005920 0x7f3774005da0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.189+0000 7f3796200640 1 -- 172.21.15.44:0/916714793 >> 172.21.15.44:0/916714793 conn(0x7f37900b2060 msgr2=0x7f37900b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.189+0000 7f3796200640 1 -- 172.21.15.44:0/916714793 shutdown_connections 2024-09-13T08:51:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.189+0000 7f3796200640 1 -- 172.21.15.44:0/916714793 wait complete. 2024-09-13T08:51:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3796200640 1 Processor -- start 2024-09-13T08:51:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3796200640 1 -- start start 2024-09-13T08:51:38.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3796200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3774005920 0x7f37901526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:38.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3796200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3790159760 0x7f37900b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:38.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3796200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f379015cb00 0x7f379015cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:38.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3796200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3790073450 con 0x7f3774005920 2024-09-13T08:51:38.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3796200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f37900732d0 con 0x7f3790159760 2024-09-13T08:51:38.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3796200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f37900735d0 con 0x7f379015cb00 2024-09-13T08:51:38.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f378f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3774005920 0x7f37901526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f378f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3774005920 0x7f37901526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59334/0 (socket says 172.21.15.44:59334) 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f378f400640 1 -- 172.21.15.44:0/1557506100 learned_addr learned my addr 172.21.15.44:0/1557506100 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f378fe00640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f379015cb00 0x7f379015cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3787e00640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3790159760 0x7f37900b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f378f400640 1 -- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f379015cb00 msgr2=0x7f379015cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f378f400640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f379015cb00 0x7f379015cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f378f400640 1 -- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3790159760 msgr2=0x7f37900b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f378f400640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3790159760 0x7f37900b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f378f400640 1 -- 172.21.15.44:0/1557506100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3790152bf0 con 0x7f3774005920 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3787e00640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3790159760 0x7f37900b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:38.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f378f400640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3774005920 0x7f37901526b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f3780003f10 tx=0x7f3780001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:38.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3787400640 1 -- 172.21.15.44:0/1557506100 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780011640 con 0x7f3774005920 2024-09-13T08:51:38.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3787400640 1 -- 172.21.15.44:0/1557506100 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3780016020 con 0x7f3774005920 2024-09-13T08:51:38.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.193+0000 7f3787400640 1 -- 172.21.15.44:0/1557506100 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780011a80 con 0x7f3774005920 2024-09-13T08:51:38.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.197+0000 7f3796200640 1 -- 172.21.15.44:0/1557506100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37900b4670 con 0x7f3774005920 2024-09-13T08:51:38.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.197+0000 7f3796200640 1 -- 172.21.15.44:0/1557506100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37900b4c30 con 0x7f3774005920 2024-09-13T08:51:38.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.197+0000 7f3796200640 1 -- 172.21.15.44:0/1557506100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3790073b30 con 0x7f3774005920 2024-09-13T08:51:38.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.197+0000 7f3787400640 1 -- 172.21.15.44:0/1557506100 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f3780011c20 con 0x7f3774005920 2024-09-13T08:51:38.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.197+0000 7f3787400640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f376c074db0 0x7f376c077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:38.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.201+0000 7f3787400640 1 -- 172.21.15.44:0/1557506100 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f3780093ef0 con 0x7f3774005920 2024-09-13T08:51:38.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.201+0000 7f3787e00640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f376c074db0 0x7f376c077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:38.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.201+0000 7f3787e00640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f376c074db0 0x7f376c077270 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f37780ab000 tx=0x7f3778019d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:38.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.201+0000 7f3787400640 1 -- 172.21.15.44:0/1557506100 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3780020020 con 0x7f3774005920 2024-09-13T08:51:38.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.377+0000 7f3796200640 1 -- 172.21.15.44:0/1557506100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f37900cc0e0 con 0x7f3774005920 2024-09-13T08:51:38.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.377+0000 7f3787400640 1 -- 172.21.15.44:0/1557506100 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v49)=0 v49) ==== 131+0+0 (secure 0 0 0) 0x7f3780060c20 con 0x7f3774005920 2024-09-13T08:51:38.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 -- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f376c074db0 msgr2=0x7f376c077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:38.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f376c074db0 0x7f376c077270 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f37780ab000 tx=0x7f3778019d60 comp rx=0 tx=0).stop 2024-09-13T08:51:38.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 -- 172.21.15.44:0/1557506100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3774005920 msgr2=0x7f37901526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:38.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3774005920 0x7f37901526b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f3780003f10 tx=0x7f3780001640 comp rx=0 tx=0).stop 2024-09-13T08:51:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 -- 172.21.15.44:0/1557506100 shutdown_connections 2024-09-13T08:51:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f376c074db0 0x7f376c077270 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f379015cb00 0x7f379015cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3790159760 0x7f37900b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 --2- 172.21.15.44:0/1557506100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3774005920 0x7f37901526b0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 -- 172.21.15.44:0/1557506100 >> 172.21.15.44:0/1557506100 conn(0x7f37900b2060 msgr2=0x7f37900c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 -- 172.21.15.44:0/1557506100 shutdown_connections 2024-09-13T08:51:38.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.381+0000 7f3796200640 1 -- 172.21.15.44:0/1557506100 wait complete. 2024-09-13T08:51:38.399 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:51:38.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:51:38.493 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:51:38.601 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:51:38.601 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:51:38.602 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:51:38.602 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:51:38.610 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:51:38.611 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:51:38.663 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:51:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 -- 172.21.15.44:0/3752798148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b80ce800 msgr2=0x7f48b806cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 --2- 172.21.15.44:0/3752798148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b80ce800 0x7f48b806cc20 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f48b8068f50 tx=0x7f48a801c600 comp rx=0 tx=0).stop 2024-09-13T08:51:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 -- 172.21.15.44:0/3752798148 shutdown_connections 2024-09-13T08:51:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 --2- 172.21.15.44:0/3752798148 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48b8143fe0 0x7f48b815e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 --2- 172.21.15.44:0/3752798148 >> v2:172.21.15.129:3300/0 conn(0x7f48b8073170 0x7f48b806d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 --2- 172.21.15.44:0/3752798148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b80ce800 0x7f48b806cc20 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 -- 172.21.15.44:0/3752798148 >> 172.21.15.44:0/3752798148 conn(0x7f48b80b2240 msgr2=0x7f48b80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:38.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 -- 172.21.15.44:0/3752798148 shutdown_connections 2024-09-13T08:51:38.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 -- 172.21.15.44:0/3752798148 wait complete. 2024-09-13T08:51:38.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 Processor -- start 2024-09-13T08:51:38.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 -- start start 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48b8073170 0x7f48b813faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48b80ce800 0x7f48b813e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b8143fe0 0x7f48b813ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48b806fe10 con 0x7f48b8143fe0 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48b806fc90 con 0x7f48b80ce800 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48bd200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f48b806ff90 con 0x7f48b8073170 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48b6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b8143fe0 0x7f48b813ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48b6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b8143fe0 0x7f48b813ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59340/0 (socket says 172.21.15.44:59340) 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48b6800640 1 -- 172.21.15.44:0/1423306722 learned_addr learned my addr 172.21.15.44:0/1423306722 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.857+0000 7f48b5e00640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48b8073170 0x7f48b813faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48afe00640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48b80ce800 0x7f48b813e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:38.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48b6800640 1 -- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48b8073170 msgr2=0x7f48b813faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:38.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48b6800640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48b8073170 0x7f48b813faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48b6800640 1 -- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48b80ce800 msgr2=0x7f48b813e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48b6800640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48b80ce800 0x7f48b813e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48b6800640 1 -- 172.21.15.44:0/1423306722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b815d980 con 0x7f48b8143fe0 2024-09-13T08:51:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48b6800640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b8143fe0 0x7f48b813ed40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f48b000ba10 tx=0x7f48b000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48ae000640 1 -- 172.21.15.44:0/1423306722 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0019070 con 0x7f48b8143fe0 2024-09-13T08:51:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48ae000640 1 -- 172.21.15.44:0/1423306722 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48b000cbb0 con 0x7f48b8143fe0 2024-09-13T08:51:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48ae000640 1 -- 172.21.15.44:0/1423306722 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0014440 con 0x7f48b8143fe0 2024-09-13T08:51:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48bd200640 1 -- 172.21.15.44:0/1423306722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b815dc00 con 0x7f48b8143fe0 2024-09-13T08:51:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48bd200640 1 -- 172.21.15.44:0/1423306722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b813f5d0 con 0x7f48b8143fe0 2024-09-13T08:51:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48bd200640 1 -- 172.21.15.44:0/1423306722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b8070540 con 0x7f48b8143fe0 2024-09-13T08:51:38.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.861+0000 7f48ae000640 1 -- 172.21.15.44:0/1423306722 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f48b001c070 con 0x7f48b8143fe0 2024-09-13T08:51:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.865+0000 7f48ae000640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4894074db0 0x7f4894077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.865+0000 7f48ae000640 1 -- 172.21.15.44:0/1423306722 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5908+0+0 (secure 0 0 0) 0x7f48b0092a50 con 0x7f48b8143fe0 2024-09-13T08:51:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.865+0000 7f48b5e00640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4894074db0 0x7f4894077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:38.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.865+0000 7f48b5e00640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4894074db0 0x7f4894077270 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f48a80af000 tx=0x7f48a80b2040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:38.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:38.869+0000 7f48ae000640 1 -- 172.21.15.44:0/1423306722 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f48b815d980 con 0x7f48b8143fe0 2024-09-13T08:51:39.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 -- 172.21.15.44:0/1423306722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f48b813f7a0 con 0x7f48b8143fe0 2024-09-13T08:51:39.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48ae000640 1 -- 172.21.15.44:0/1423306722 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7f48b005f0b0 con 0x7f48b8143fe0 2024-09-13T08:51:39.061 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:51:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 -- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4894074db0 msgr2=0x7f4894077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4894074db0 0x7f4894077270 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f48a80af000 tx=0x7f48a80b2040 comp rx=0 tx=0).stop 2024-09-13T08:51:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 -- 172.21.15.44:0/1423306722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b8143fe0 msgr2=0x7f48b813ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b8143fe0 0x7f48b813ed40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f48b000ba10 tx=0x7f48b000be10 comp rx=0 tx=0).stop 2024-09-13T08:51:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 -- 172.21.15.44:0/1423306722 shutdown_connections 2024-09-13T08:51:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4894074db0 0x7f4894077270 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b8143fe0 0x7f48b813ed40 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48b80ce800 0x7f48b813e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 --2- 172.21.15.44:0/1423306722 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f48b8073170 0x7f48b813faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 -- 172.21.15.44:0/1423306722 >> 172.21.15.44:0/1423306722 conn(0x7f48b80b2240 msgr2=0x7f48b80b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.061+0000 7f48bd200640 1 -- 172.21.15.44:0/1423306722 shutdown_connections 2024-09-13T08:51:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.065+0000 7f48bd200640 1 -- 172.21.15.44:0/1423306722 wait complete. 2024-09-13T08:51:39.082 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T08:51:39.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:51:39.082 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T08:51:39.131 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:51:39.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:51:39.132 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T08:51:39.156 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:51:39.156 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:51:39.157 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:51:39.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:51:39.168 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:51:39.168 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:51:39.168 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:51:39.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:51:39.211 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:51:39.211 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:51:39.211 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:51:39.220 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:51:39.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:51:39.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.433+0000 7f6350e00640 1 -- 172.21.15.44:0/725391817 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f634c15ef00 msgr2=0x7f634c161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:39.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.433+0000 7f6350e00640 1 --2- 172.21.15.44:0/725391817 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f634c15ef00 0x7f634c161950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6338009900 tx=0x7f633801c8b0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.433+0000 7f6350e00640 1 -- 172.21.15.44:0/725391817 shutdown_connections 2024-09-13T08:51:39.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.433+0000 7f6350e00640 1 --2- 172.21.15.44:0/725391817 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f634c15ef00 0x7f634c161950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.433+0000 7f6350e00640 1 --2- 172.21.15.44:0/725391817 >> v2:172.21.15.129:3300/0 conn(0x7f634c15e5e0 0x7f634c15e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.433+0000 7f6350e00640 1 -- 172.21.15.44:0/725391817 >> 172.21.15.44:0/725391817 conn(0x7f634c074310 msgr2=0x7f634c0746f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:39.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.433+0000 7f6350e00640 1 -- 172.21.15.44:0/725391817 shutdown_connections 2024-09-13T08:51:39.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.433+0000 7f6350e00640 1 -- 172.21.15.44:0/725391817 wait complete. 2024-09-13T08:51:39.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.437+0000 7f6350e00640 1 Processor -- start 2024-09-13T08:51:39.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.437+0000 7f6350e00640 1 -- start start 2024-09-13T08:51:39.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.437+0000 7f6350e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f634c15e5e0 0x7f634c099600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:39.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.437+0000 7f6350e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f634c15ef00 0x7f634c14f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:39.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f6350e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634c161e90 0x7f634c14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:39.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f6350e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f634c0ad8d0 con 0x7f634c161e90 2024-09-13T08:51:39.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f6350e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f634c0ad750 con 0x7f634c15ef00 2024-09-13T08:51:39.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f6350e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f634c0ada50 con 0x7f634c15e5e0 2024-09-13T08:51:39.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f634c15e5e0 0x7f634c099600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:39.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634c161e90 0x7f634c14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:39.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634c161e90 0x7f634c14faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59344/0 (socket says 172.21.15.44:59344) 2024-09-13T08:51:39.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634be00640 1 -- 172.21.15.44:0/1721160216 learned_addr learned my addr 172.21.15.44:0/1721160216 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:39.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634aa00640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f634c15ef00 0x7f634c14f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:39.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634be00640 1 -- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f634c15e5e0 msgr2=0x7f634c099600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:39.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634be00640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f634c15e5e0 0x7f634c099600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634be00640 1 -- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f634c15ef00 msgr2=0x7f634c14f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:39.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634be00640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f634c15ef00 0x7f634c14f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634be00640 1 -- 172.21.15.44:0/1721160216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c156b50 con 0x7f634c161e90 2024-09-13T08:51:39.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f634be00640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634c161e90 0x7f634c14faa0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f634400c6e0 tx=0x7f634400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:39.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f6343400640 1 -- 172.21.15.44:0/1721160216 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344003b80 con 0x7f634c161e90 2024-09-13T08:51:39.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f6343400640 1 -- 172.21.15.44:0/1721160216 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6344003d40 con 0x7f634c161e90 2024-09-13T08:51:39.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f6350e00640 1 -- 172.21.15.44:0/1721160216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634c156df0 con 0x7f634c161e90 2024-09-13T08:51:39.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f6350e00640 1 -- 172.21.15.44:0/1721160216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634c154ae0 con 0x7f634c161e90 2024-09-13T08:51:39.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.441+0000 7f6343400640 1 -- 172.21.15.44:0/1721160216 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344005330 con 0x7f634c161e90 2024-09-13T08:51:39.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.445+0000 7f6350e00640 1 -- 172.21.15.44:0/1721160216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f634c157030 con 0x7f634c161e90 2024-09-13T08:51:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.445+0000 7f6343400640 1 -- 172.21.15.44:0/1721160216 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f63440054d0 con 0x7f634c161e90 2024-09-13T08:51:39.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.449+0000 7f6343400640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f632c07d5c0 0x7f632c07fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.449+0000 7f6343400640 1 -- 172.21.15.44:0/1721160216 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5964+0+0 (secure 0 0 0) 0x7f6344092ad0 con 0x7f634c161e90 2024-09-13T08:51:39.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.449+0000 7f634b400640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f632c07d5c0 0x7f632c07fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:39.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.453+0000 7f634b400640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f632c07d5c0 0x7f632c07fa80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f634c151660 tx=0x7f633c00b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:39.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.457+0000 7f6343400640 1 -- 172.21.15.44:0/1721160216 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f634405f150 con 0x7f634c161e90 2024-09-13T08:51:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.877+0000 7f6350e00640 1 -- 172.21.15.44:0/1721160216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f634c150830 con 0x7f634c161e90 2024-09-13T08:51:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.877+0000 7f6343400640 1 -- 172.21.15.44:0/1721160216 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5231 (secure 0 0 0) 0x7f6344063b80 con 0x7f634c161e90 2024-09-13T08:51:39.879 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:39.879 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"btime":"2024-09-13T08:50:46:358102+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6854,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3108776730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3108776730},{"type":"v1","addr":"172.21.15.129:6836","nonce":3108776730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1076017330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1076017330},{"type":"v1","addr":"172.21.15.129:6837","nonce":1076017330}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4098004813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4098004813},{"type":"v1","addr":"172.21.15.44:6835","nonce":4098004813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:50:37.721609+0000","modified":"2024-09-13T08:50:40.739478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/1590451084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1590451084},{"type":"v1","addr":"172.21.15.44:6834","nonce":1590451084}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":8}]} 2024-09-13T08:51:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 75 2024-09-13T08:51:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 -- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f632c07d5c0 msgr2=0x7f632c07fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f632c07d5c0 0x7f632c07fa80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f634c151660 tx=0x7f633c00b390 comp rx=0 tx=0).stop 2024-09-13T08:51:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 -- 172.21.15.44:0/1721160216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634c161e90 msgr2=0x7f634c14faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634c161e90 0x7f634c14faa0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f634400c6e0 tx=0x7f634400cba0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 -- 172.21.15.44:0/1721160216 shutdown_connections 2024-09-13T08:51:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f632c07d5c0 0x7f632c07fa80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634c161e90 0x7f634c14faa0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f634c15ef00 0x7f634c14f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 --2- 172.21.15.44:0/1721160216 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f634c15e5e0 0x7f634c099600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 -- 172.21.15.44:0/1721160216 >> 172.21.15.44:0/1721160216 conn(0x7f634c074310 msgr2=0x7f634c15c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 -- 172.21.15.44:0/1721160216 shutdown_connections 2024-09-13T08:51:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:39.881+0000 7f6350e00640 1 -- 172.21.15.44:0/1721160216 wait complete. 2024-09-13T08:51:39.907 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:51:40.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.053+0000 7f095d800640 1 -- 172.21.15.44:0/130948022 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f095815bea0 msgr2=0x7f09580ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.053+0000 7f095d800640 1 --2- 172.21.15.44:0/130948022 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f095815bea0 0x7f09580ce670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0948009870 tx=0x7f094801c860 comp rx=0 tx=0).stop 2024-09-13T08:51:40.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.053+0000 7f095d800640 1 -- 172.21.15.44:0/130948022 shutdown_connections 2024-09-13T08:51:40.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.053+0000 7f095d800640 1 --2- 172.21.15.44:0/130948022 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f095815bea0 0x7f09580ce670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.053+0000 7f095d800640 1 -- 172.21.15.44:0/130948022 >> 172.21.15.44:0/130948022 conn(0x7f09580d1dc0 msgr2=0x7f09580d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:40.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.053+0000 7f095d800640 1 -- 172.21.15.44:0/130948022 shutdown_connections 2024-09-13T08:51:40.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.053+0000 7f095d800640 1 -- 172.21.15.44:0/130948022 wait complete. 2024-09-13T08:51:40.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.053+0000 7f095d800640 1 Processor -- start 2024-09-13T08:51:40.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.053+0000 7f095d800640 1 -- start start 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f095d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09580cdb30 0x7f095809eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f095d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09580d28e0 0x7f095814c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f095d800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f095815bea0 0x7f095809c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f095d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f095816a930 con 0x7f09580d28e0 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f095d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f095816a7b0 con 0x7f09580cdb30 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f095d800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f095816aab0 con 0x7f095815bea0 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09580cdb30 0x7f095809eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09580d28e0 0x7f095814c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0957400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f095815bea0 0x7f095809c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09580d28e0 0x7f095814c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59348/0 (socket says 172.21.15.44:59348) 2024-09-13T08:51:40.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09580cdb30 0x7f095809eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:54474/0 (socket says 172.21.15.44:54474) 2024-09-13T08:51:40.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0957400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f095815bea0 0x7f095809c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:41022/0 (socket says 172.21.15.44:41022) 2024-09-13T08:51:40.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956000640 1 -- 172.21.15.44:0/2709462456 learned_addr learned my addr 172.21.15.44:0/2709462456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956000640 1 -- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f095815bea0 msgr2=0x7f095809c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956000640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f095815bea0 0x7f095809c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956000640 1 -- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09580cdb30 msgr2=0x7f095809eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956000640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09580cdb30 0x7f095809eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956000640 1 -- 172.21.15.44:0/2709462456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095809fa70 con 0x7f09580d28e0 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956a00640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09580cdb30 0x7f095809eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f0956000640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09580d28e0 0x7f095814c2f0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f094400c530 tx=0x7f094400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f094ea00640 1 -- 172.21.15.44:0/2709462456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944016020 con 0x7f09580d28e0 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f094ea00640 1 -- 172.21.15.44:0/2709462456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0944012940 con 0x7f09580d28e0 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f094ea00640 1 -- 172.21.15.44:0/2709462456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944012d90 con 0x7f09580d28e0 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f095d800640 1 -- 172.21.15.44:0/2709462456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095809f1d0 con 0x7f09580d28e0 2024-09-13T08:51:40.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f095d800640 1 -- 172.21.15.44:0/2709462456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095809f790 con 0x7f09580d28e0 2024-09-13T08:51:40.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.057+0000 7f095d800640 1 -- 172.21.15.44:0/2709462456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09580ce030 con 0x7f09580d28e0 2024-09-13T08:51:40.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.061+0000 7f094ea00640 1 -- 172.21.15.44:0/2709462456 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f0944019020 con 0x7f09580d28e0 2024-09-13T08:51:40.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.061+0000 7f094ea00640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0938074db0 0x7f0938077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.061+0000 7f094ea00640 1 -- 172.21.15.44:0/2709462456 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5964+0+0 (secure 0 0 0) 0x7f0944092ac0 con 0x7f09580d28e0 2024-09-13T08:51:40.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.061+0000 7f0956a00640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0938074db0 0x7f0938077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.061+0000 7f0956a00640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0938074db0 0x7f0938077270 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f09480af000 tx=0x7f094801b0a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:40.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.065+0000 7f094ea00640 1 -- 172.21.15.44:0/2709462456 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0944020030 con 0x7f09580d28e0 2024-09-13T08:51:40.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.277+0000 7f095d800640 1 -- 172.21.15.44:0/2709462456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09580b54a0 con 0x7f09580d28e0 2024-09-13T08:51:40.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.277+0000 7f094ea00640 1 -- 172.21.15.44:0/2709462456 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5231 (secure 0 0 0) 0x7f094405f0a0 con 0x7f09580d28e0 2024-09-13T08:51:40.277 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:40.277 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"btime":"2024-09-13T08:50:46:358102+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6854,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3108776730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3108776730},{"type":"v1","addr":"172.21.15.129:6836","nonce":3108776730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1076017330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1076017330},{"type":"v1","addr":"172.21.15.129:6837","nonce":1076017330}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4098004813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4098004813},{"type":"v1","addr":"172.21.15.44:6835","nonce":4098004813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:50:37.721609+0000","modified":"2024-09-13T08:50:40.739478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/1590451084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1590451084},{"type":"v1","addr":"172.21.15.44:6834","nonce":1590451084}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":8}]} 2024-09-13T08:51:40.277 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 75 2024-09-13T08:51:40.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 -- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0938074db0 msgr2=0x7f0938077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0938074db0 0x7f0938077270 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f09480af000 tx=0x7f094801b0a0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 -- 172.21.15.44:0/2709462456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09580d28e0 msgr2=0x7f095814c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09580d28e0 0x7f095814c2f0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f094400c530 tx=0x7f094400c9f0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 -- 172.21.15.44:0/2709462456 shutdown_connections 2024-09-13T08:51:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0938074db0 0x7f0938077270 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f095815bea0 0x7f095809c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09580d28e0 0x7f095814c2f0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 --2- 172.21.15.44:0/2709462456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09580cdb30 0x7f095809eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 -- 172.21.15.44:0/2709462456 >> 172.21.15.44:0/2709462456 conn(0x7f09580d1dc0 msgr2=0x7f095806f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 -- 172.21.15.44:0/2709462456 shutdown_connections 2024-09-13T08:51:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.281+0000 7f095d800640 1 -- 172.21.15.44:0/2709462456 wait complete. 2024-09-13T08:51:40.299 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:51:40.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.501+0000 7f81d0200640 1 -- 172.21.15.44:0/71603779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c80ced80 msgr2=0x7f81c80ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.501+0000 7f81d0200640 1 --2- 172.21.15.44:0/71603779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c80ced80 0x7f81c80ae9b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009840 tx=0x7f81b801c820 comp rx=0 tx=0).stop 2024-09-13T08:51:40.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.501+0000 7f81d0200640 1 -- 172.21.15.44:0/71603779 shutdown_connections 2024-09-13T08:51:40.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.501+0000 7f81d0200640 1 --2- 172.21.15.44:0/71603779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c80ced80 0x7f81c80ae9b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.501+0000 7f81d0200640 1 -- 172.21.15.44:0/71603779 >> 172.21.15.44:0/71603779 conn(0x7f81c80d1dc0 msgr2=0x7f81c80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:40.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.501+0000 7f81d0200640 1 -- 172.21.15.44:0/71603779 shutdown_connections 2024-09-13T08:51:40.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.501+0000 7f81d0200640 1 -- 172.21.15.44:0/71603779 wait complete. 2024-09-13T08:51:40.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.505+0000 7f81d0200640 1 Processor -- start 2024-09-13T08:51:40.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.505+0000 7f81d0200640 1 -- start start 2024-09-13T08:51:40.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.505+0000 7f81d0200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c80cdb30 0x7f81c8146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.505+0000 7f81d0200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c80ced80 0x7f81c8149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.505+0000 7f81d0200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81c815bea0 0x7f81c815abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.505+0000 7f81d0200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81c816a8a0 con 0x7f81c80cdb30 2024-09-13T08:51:40.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.505+0000 7f81d0200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f81c816a720 con 0x7f81c80ced80 2024-09-13T08:51:40.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.505+0000 7f81d0200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f81c816aa20 con 0x7f81c815bea0 2024-09-13T08:51:40.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81ce200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81c815bea0 0x7f81c815abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81ce200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81c815bea0 0x7f81c815abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:41032/0 (socket says 172.21.15.44:41032) 2024-09-13T08:51:40.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81ce200640 1 -- 172.21.15.44:0/2573606945 learned_addr learned my addr 172.21.15.44:0/2573606945 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:40.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81ce200640 1 -- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c80ced80 msgr2=0x7f81c8149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81cce00640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c80ced80 0x7f81c8149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81ce200640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c80ced80 0x7f81c8149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81ce200640 1 -- 172.21.15.44:0/2573606945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c80cdb30 msgr2=0x7f81c8146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81ce200640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c80cdb30 0x7f81c8146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81ce200640 1 -- 172.21.15.44:0/2573606945 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c80b2b50 con 0x7f81c815bea0 2024-09-13T08:51:40.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81cce00640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c80ced80 0x7f81c8149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:51:40.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81ce200640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81c815bea0 0x7f81c815abb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00c440 tx=0x7f81bc00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:40.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81c5600640 1 -- 172.21.15.44:0/2573606945 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0052d0 con 0x7f81c815bea0 2024-09-13T08:51:40.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81c5600640 1 -- 172.21.15.44:0/2573606945 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81bc005490 con 0x7f81c815bea0 2024-09-13T08:51:40.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81c5600640 1 -- 172.21.15.44:0/2573606945 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc012870 con 0x7f81c815bea0 2024-09-13T08:51:40.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.509+0000 7f81d0200640 1 -- 172.21.15.44:0/2573606945 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c80cbb00 con 0x7f81c815bea0 2024-09-13T08:51:40.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.513+0000 7f81d0200640 1 -- 172.21.15.44:0/2573606945 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c80b2df0 con 0x7f81c815bea0 2024-09-13T08:51:40.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.513+0000 7f81d0200640 1 -- 172.21.15.44:0/2573606945 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c80ce670 con 0x7f81c815bea0 2024-09-13T08:51:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.517+0000 7f81c5600640 1 -- 172.21.15.44:0/2573606945 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f81bc012a10 con 0x7f81c815bea0 2024-09-13T08:51:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.517+0000 7f81c5600640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f81ac074db0 0x7f81ac077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.517+0000 7f81c5600640 1 -- 172.21.15.44:0/2573606945 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(119..119 src has 1..119) ==== 5964+0+0 (secure 0 0 0) 0x7f81bc092960 con 0x7f81c815bea0 2024-09-13T08:51:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.521+0000 7f81cd800640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f81ac074db0 0x7f81ac077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.521+0000 7f81cd800640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f81ac074db0 0x7f81ac077270 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f81c815b710 tx=0x7f81c0007cc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:40.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.525+0000 7f81c5600640 1 -- 172.21.15.44:0/2573606945 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f81bc05f610 con 0x7f81c815bea0 2024-09-13T08:51:40.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.713+0000 7f81d0200640 1 -- 172.21.15.44:0/2573606945 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81c81468b0 con 0x7f81c815bea0 2024-09-13T08:51:40.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.713+0000 7f81c5600640 1 -- 172.21.15.44:0/2573606945 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+17189 (secure 0 0 0) 0x7f81bc05ef10 con 0x7f81c815bea0 2024-09-13T08:51:40.713 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:40.714 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":119,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:51:38.886927+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:50:35.432294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-13T08:50:36.678893+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"17.25","mappings":[{"from":4,"to":1}]},{"pgid":"17.28","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:51:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 -- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f81ac074db0 msgr2=0x7f81ac077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f81ac074db0 0x7f81ac077270 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f81c815b710 tx=0x7f81c0007cc0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 -- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81c815bea0 msgr2=0x7f81c815abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81c815bea0 0x7f81c815abb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00c440 tx=0x7f81bc00c900 comp rx=0 tx=0).stop 2024-09-13T08:51:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 -- 172.21.15.44:0/2573606945 shutdown_connections 2024-09-13T08:51:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f81ac074db0 0x7f81ac077270 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f81c815bea0 0x7f81c815abb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c80ced80 0x7f81c8149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 --2- 172.21.15.44:0/2573606945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c80cdb30 0x7f81c8146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 -- 172.21.15.44:0/2573606945 >> 172.21.15.44:0/2573606945 conn(0x7f81c80d1dc0 msgr2=0x7f81c80b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 -- 172.21.15.44:0/2573606945 shutdown_connections 2024-09-13T08:51:40.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.717+0000 7f81d0200640 1 -- 172.21.15.44:0/2573606945 wait complete. 2024-09-13T08:51:40.736 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:51:40.736 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:51:40.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.881+0000 7fb431800640 1 -- 172.21.15.44:0/3925421906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c0744c0 msgr2=0x7fb42c074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.881+0000 7fb431800640 1 --2- 172.21.15.44:0/3925421906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c0744c0 0x7fb42c074930 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0aea80 tx=0x7fb42401cd50 comp rx=0 tx=0).stop 2024-09-13T08:51:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.881+0000 7fb431800640 1 -- 172.21.15.44:0/3925421906 shutdown_connections 2024-09-13T08:51:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.881+0000 7fb431800640 1 --2- 172.21.15.44:0/3925421906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c0744c0 0x7fb42c074930 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.881+0000 7fb431800640 1 --2- 172.21.15.44:0/3925421906 >> v2:172.21.15.129:3300/0 conn(0x7fb42c072c90 0x7fb42c15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.881+0000 7fb431800640 1 --2- 172.21.15.44:0/3925421906 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb42c15c5e0 0x7fb42c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.881+0000 7fb431800640 1 -- 172.21.15.44:0/3925421906 >> 172.21.15.44:0/3925421906 conn(0x7fb42c0b1c20 msgr2=0x7fb42c0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.881+0000 7fb431800640 1 -- 172.21.15.44:0/3925421906 shutdown_connections 2024-09-13T08:51:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.881+0000 7fb431800640 1 -- 172.21.15.44:0/3925421906 wait complete. 2024-09-13T08:51:40.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb431800640 1 Processor -- start 2024-09-13T08:51:40.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb431800640 1 -- start start 2024-09-13T08:51:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb431800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c072c90 0x7fb42c1405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb431800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb42c0744c0 0x7fb42c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb431800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb42c15c5e0 0x7fb42c09cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb431800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c152260 con 0x7fb42c072c90 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb431800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c0af950 con 0x7fb42c0744c0 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb431800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb42c1523e0 con 0x7fb42c15c5e0 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb42aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c072c90 0x7fb42c1405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb42aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c072c90 0x7fb42c1405d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59380/0 (socket says 172.21.15.44:59380) 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb42aa00640 1 -- 172.21.15.44:0/1600592968 learned_addr learned my addr 172.21.15.44:0/1600592968 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb423e00640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb42c0744c0 0x7fb42c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb42b400640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb42c15c5e0 0x7fb42c09cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb42aa00640 1 -- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb42c15c5e0 msgr2=0x7fb42c09cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb42aa00640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb42c15c5e0 0x7fb42c09cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb42aa00640 1 -- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb42c0744c0 msgr2=0x7fb42c0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb42aa00640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb42c0744c0 0x7fb42c0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb42aa00640 1 -- 172.21.15.44:0/1600592968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c0cf1d0 con 0x7fb42c072c90 2024-09-13T08:51:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb42aa00640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c072c90 0x7fb42c1405d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b410 tx=0x7fb41c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb422a00640 1 -- 172.21.15.44:0/1600592968 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c015020 con 0x7fb42c072c90 2024-09-13T08:51:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb422a00640 1 -- 172.21.15.44:0/1600592968 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb41c011920 con 0x7fb42c072c90 2024-09-13T08:51:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb431800640 1 -- 172.21.15.44:0/1600592968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42c0b0fc0 con 0x7fb42c072c90 2024-09-13T08:51:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb422a00640 1 -- 172.21.15.44:0/1600592968 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c011d70 con 0x7fb42c072c90 2024-09-13T08:51:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.885+0000 7fb431800640 1 -- 172.21.15.44:0/1600592968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42c09bf90 con 0x7fb42c072c90 2024-09-13T08:51:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.889+0000 7fb431800640 1 -- 172.21.15.44:0/1600592968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb42c074930 con 0x7fb42c072c90 2024-09-13T08:51:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.889+0000 7fb422a00640 1 -- 172.21.15.44:0/1600592968 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fb41c018020 con 0x7fb42c072c90 2024-09-13T08:51:40.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.889+0000 7fb422a00640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb408074db0 0x7fb408077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:40.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.889+0000 7fb422a00640 1 -- 172.21.15.44:0/1600592968 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5964+0+0 (secure 0 0 0) 0x7fb41c092bf0 con 0x7fb42c072c90 2024-09-13T08:51:40.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.889+0000 7fb423e00640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb408074db0 0x7fb408077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.893+0000 7fb423e00640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb408074db0 0x7fb408077270 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb42c09dbd0 tx=0x7fb414009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:40.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:40.893+0000 7fb422a00640 1 -- 172.21.15.44:0/1600592968 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb41c01f020 con 0x7fb42c072c90 2024-09-13T08:51:41.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.105+0000 7fb431800640 1 -- 172.21.15.44:0/1600592968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb42c09ba60 con 0x7fb42c072c90 2024-09-13T08:51:41.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb422a00640 1 -- 172.21.15.44:0/1600592968 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5231 (secure 0 0 0) 0x7fb41c05f2f0 con 0x7fb42c072c90 2024-09-13T08:51:41.107 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:41.107 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"btime":"2024-09-13T08:50:46:358102+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6854,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3108776730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3108776730},{"type":"v1","addr":"172.21.15.129:6836","nonce":3108776730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1076017330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1076017330},{"type":"v1","addr":"172.21.15.129:6837","nonce":1076017330}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4098004813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4098004813},{"type":"v1","addr":"172.21.15.44:6835","nonce":4098004813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:50:37.721609+0000","modified":"2024-09-13T08:50:40.739478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/1590451084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1590451084},{"type":"v1","addr":"172.21.15.44:6834","nonce":1590451084}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":8}]} 2024-09-13T08:51:41.108 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 75 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 -- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb408074db0 msgr2=0x7fb408077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb408074db0 0x7fb408077270 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb42c09dbd0 tx=0x7fb414009110 comp rx=0 tx=0).stop 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 -- 172.21.15.44:0/1600592968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c072c90 msgr2=0x7fb42c1405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c072c90 0x7fb42c1405d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b410 tx=0x7fb41c00b8d0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 -- 172.21.15.44:0/1600592968 shutdown_connections 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb408074db0 0x7fb408077270 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb42c15c5e0 0x7fb42c09cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb42c0744c0 0x7fb42c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 --2- 172.21.15.44:0/1600592968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c072c90 0x7fb42c1405d0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 -- 172.21.15.44:0/1600592968 >> 172.21.15.44:0/1600592968 conn(0x7fb42c0b1c20 msgr2=0x7fb42c15b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 -- 172.21.15.44:0/1600592968 shutdown_connections 2024-09-13T08:51:41.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.109+0000 7fb431800640 1 -- 172.21.15.44:0/1600592968 wait complete. 2024-09-13T08:51:41.130 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:51:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 -- 172.21.15.44:0/142684162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f794415f000 msgr2=0x7f79440d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 --2- 172.21.15.44:0/142684162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f794415f000 0x7f79440d01b0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f79440c56a0 tx=0x7f793401c830 comp rx=0 tx=0).stop 2024-09-13T08:51:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 -- 172.21.15.44:0/142684162 shutdown_connections 2024-09-13T08:51:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 --2- 172.21.15.44:0/142684162 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79440ccaf0 0x7f79440b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 --2- 172.21.15.44:0/142684162 >> v2:172.21.15.129:3300/0 conn(0x7f79440cefb0 0x7f79440cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 --2- 172.21.15.44:0/142684162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f794415f000 0x7f79440d01b0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 -- 172.21.15.44:0/142684162 >> 172.21.15.44:0/142684162 conn(0x7f794409aa10 msgr2=0x7f794409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 -- 172.21.15.44:0/142684162 shutdown_connections 2024-09-13T08:51:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 -- 172.21.15.44:0/142684162 wait complete. 2024-09-13T08:51:41.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 Processor -- start 2024-09-13T08:51:41.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.277+0000 7f7943e00640 1 -- start start 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7943e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79440ccaf0 0x7f7944151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7943e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79440cefb0 0x7f7944156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7943e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f794415f000 0x7f79440ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7943e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f794415ff80 con 0x7f79440ccaf0 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7943e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f794415fe00 con 0x7f794415f000 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7943e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7944160100 con 0x7f79440cefb0 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79440ccaf0 0x7f7944151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79440ccaf0 0x7f7944151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59398/0 (socket says 172.21.15.44:59398) 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942a00640 1 -- 172.21.15.44:0/2182498966 learned_addr learned my addr 172.21.15.44:0/2182498966 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942000640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79440cefb0 0x7f7944156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942a00640 1 -- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79440cefb0 msgr2=0x7f7944156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942a00640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79440cefb0 0x7f7944156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942a00640 1 -- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f794415f000 msgr2=0x7f79440ca310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942a00640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f794415f000 0x7f79440ca310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942a00640 1 -- 172.21.15.44:0/2182498966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79440cac10 con 0x7f79440ccaf0 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942000640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79440cefb0 0x7f7944156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7942a00640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79440ccaf0 0x7f7944151950 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f79340b0040 tx=0x7f79340a6750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f793aa00640 1 -- 172.21.15.44:0/2182498966 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934004340 con 0x7f79440ccaf0 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f793aa00640 1 -- 172.21.15.44:0/2182498966 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7934005020 con 0x7f79440ccaf0 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f793aa00640 1 -- 172.21.15.44:0/2182498966 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934005470 con 0x7f79440ccaf0 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7943e00640 1 -- 172.21.15.44:0/2182498966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79440caef0 con 0x7f79440ccaf0 2024-09-13T08:51:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7943e00640 1 -- 172.21.15.44:0/2182498966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7944156240 con 0x7f79440ccaf0 2024-09-13T08:51:41.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.281+0000 7f7943e00640 1 -- 172.21.15.44:0/2182498966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79440cc5b0 con 0x7f79440ccaf0 2024-09-13T08:51:41.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.285+0000 7f793aa00640 1 -- 172.21.15.44:0/2182498966 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f79340be060 con 0x7f79440ccaf0 2024-09-13T08:51:41.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.285+0000 7f793aa00640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7924074db0 0x7f7924077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:41.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.285+0000 7f793aa00640 1 -- 172.21.15.44:0/2182498966 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5964+0+0 (secure 0 0 0) 0x7f793412cf20 con 0x7f79440ccaf0 2024-09-13T08:51:41.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.285+0000 7f7942000640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7924074db0 0x7f7924077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:41.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.285+0000 7f7942000640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7924074db0 0x7f7924077270 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7930005fd0 tx=0x7f7930005e60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:41.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.289+0000 7f793aa00640 1 -- 172.21.15.44:0/2182498966 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f79340164b0 con 0x7f79440ccaf0 2024-09-13T08:51:41.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 -- 172.21.15.44:0/2182498966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79440d02d0 con 0x7f79440ccaf0 2024-09-13T08:51:41.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f793aa00640 1 -- 172.21.15.44:0/2182498966 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5231 (secure 0 0 0) 0x7f79340f9bd0 con 0x7f79440ccaf0 2024-09-13T08:51:41.501 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:41.501 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"btime":"2024-09-13T08:50:46:358102+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6854,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3108776730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3108776730},{"type":"v1","addr":"172.21.15.129:6836","nonce":3108776730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1076017330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1076017330},{"type":"v1","addr":"172.21.15.129:6837","nonce":1076017330}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4098004813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4098004813},{"type":"v1","addr":"172.21.15.44:6835","nonce":4098004813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:50:37.721609+0000","modified":"2024-09-13T08:50:40.739478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/1590451084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1590451084},{"type":"v1","addr":"172.21.15.44:6834","nonce":1590451084}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":8}]} 2024-09-13T08:51:41.501 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 75 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 -- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7924074db0 msgr2=0x7f7924077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7924074db0 0x7f7924077270 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7930005fd0 tx=0x7f7930005e60 comp rx=0 tx=0).stop 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 -- 172.21.15.44:0/2182498966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79440ccaf0 msgr2=0x7f7944151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79440ccaf0 0x7f7944151950 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f79340b0040 tx=0x7f79340a6750 comp rx=0 tx=0).stop 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 -- 172.21.15.44:0/2182498966 shutdown_connections 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7924074db0 0x7f7924077270 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f794415f000 0x7f79440ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f79440cefb0 0x7f7944156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 --2- 172.21.15.44:0/2182498966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79440ccaf0 0x7f7944151950 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.501+0000 7f7943e00640 1 -- 172.21.15.44:0/2182498966 >> 172.21.15.44:0/2182498966 conn(0x7f794409aa10 msgr2=0x7f79440ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.505+0000 7f7943e00640 1 -- 172.21.15.44:0/2182498966 shutdown_connections 2024-09-13T08:51:41.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.505+0000 7f7943e00640 1 -- 172.21.15.44:0/2182498966 wait complete. 2024-09-13T08:51:41.523 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:51:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.669+0000 7f7f55200640 1 -- 172.21.15.44:0/888882569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f500cffc0 msgr2=0x7f7f50073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.669+0000 7f7f55200640 1 --2- 172.21.15.44:0/888882569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f500cffc0 0x7f7f50073ed0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7f50069410 tx=0x7f7f3801c890 comp rx=0 tx=0).stop 2024-09-13T08:51:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.669+0000 7f7f55200640 1 -- 172.21.15.44:0/888882569 shutdown_connections 2024-09-13T08:51:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.669+0000 7f7f55200640 1 --2- 172.21.15.44:0/888882569 >> v2:172.21.15.129:3300/0 conn(0x7f7f5006f9a0 0x7f7f5006fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.669+0000 7f7f55200640 1 --2- 172.21.15.44:0/888882569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f500cffc0 0x7f7f50073ed0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.669+0000 7f7f55200640 1 -- 172.21.15.44:0/888882569 >> 172.21.15.44:0/888882569 conn(0x7f7f500d0c90 msgr2=0x7f7f500d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:41.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f55200640 1 -- 172.21.15.44:0/888882569 shutdown_connections 2024-09-13T08:51:41.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f55200640 1 -- 172.21.15.44:0/888882569 wait complete. 2024-09-13T08:51:41.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f55200640 1 Processor -- start 2024-09-13T08:51:41.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f55200640 1 -- start start 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f55200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f5006f9a0 0x7f7f50157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f55200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f500cffc0 0x7f7f501fb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f55200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f5015af60 0x7f7f500b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f55200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50167cc0 con 0x7f7f5015af60 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f55200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50167b40 con 0x7f7f5006f9a0 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f55200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7f50167e40 con 0x7f7f500cffc0 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f5015af60 0x7f7f500b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f5015af60 0x7f7f500b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59424/0 (socket says 172.21.15.44:59424) 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4e800640 1 -- 172.21.15.44:0/1597701645 learned_addr learned my addr 172.21.15.44:0/1597701645 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f45400640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f500cffc0 0x7f7f501fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4de00640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f5006f9a0 0x7f7f50157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4e800640 1 -- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f500cffc0 msgr2=0x7f7f501fb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4e800640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f500cffc0 0x7f7f501fb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4e800640 1 -- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f5006f9a0 msgr2=0x7f7f50157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4e800640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f5006f9a0 0x7f7f50157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4e800640 1 -- 172.21.15.44:0/1597701645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f50071280 con 0x7f7f5015af60 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4de00640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f5006f9a0 0x7f7f50157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f45400640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f500cffc0 0x7f7f501fb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f4e800640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f5015af60 0x7f7f500b0c10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800b9a0 tx=0x7f7f4800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.673+0000 7f7f46a00640 1 -- 172.21.15.44:0/1597701645 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48019070 con 0x7f7f5015af60 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.677+0000 7f7f46a00640 1 -- 172.21.15.44:0/1597701645 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f4800ccb0 con 0x7f7f5015af60 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.677+0000 7f7f46a00640 1 -- 172.21.15.44:0/1597701645 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f480146a0 con 0x7f7f5015af60 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.677+0000 7f7f55200640 1 -- 172.21.15.44:0/1597701645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f501fb970 con 0x7f7f5015af60 2024-09-13T08:51:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.677+0000 7f7f55200640 1 -- 172.21.15.44:0/1597701645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f500b6850 con 0x7f7f5015af60 2024-09-13T08:51:41.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.677+0000 7f7f55200640 1 -- 172.21.15.44:0/1597701645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f5006fe30 con 0x7f7f5015af60 2024-09-13T08:51:41.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.677+0000 7f7f46a00640 1 -- 172.21.15.44:0/1597701645 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f7f4801c070 con 0x7f7f5015af60 2024-09-13T08:51:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.677+0000 7f7f46a00640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f2c074db0 0x7f7f2c077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.677+0000 7f7f46a00640 1 -- 172.21.15.44:0/1597701645 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5964+0+0 (secure 0 0 0) 0x7f7f48092c10 con 0x7f7f5015af60 2024-09-13T08:51:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.677+0000 7f7f4de00640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f2c074db0 0x7f7f2c077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:41.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.681+0000 7f7f4de00640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f2c074db0 0x7f7f2c077270 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009890 tx=0x7f7f40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:41.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.681+0000 7f7f46a00640 1 -- 172.21.15.44:0/1597701645 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7f4805f8c0 con 0x7f7f5015af60 2024-09-13T08:51:41.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 -- 172.21.15.44:0/1597701645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f5015a190 con 0x7f7f5015af60 2024-09-13T08:51:41.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f46a00640 1 -- 172.21.15.44:0/1597701645 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+17189 (secure 0 0 0) 0x7f7f480942a0 con 0x7f7f5015af60 2024-09-13T08:51:41.860 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:41.861 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":119,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:51:38.886927+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:50:35.432294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-13T08:50:36.678893+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"17.25","mappings":[{"from":4,"to":1}]},{"pgid":"17.28","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 -- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f2c074db0 msgr2=0x7f7f2c077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f2c074db0 0x7f7f2c077270 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009890 tx=0x7f7f40008040 comp rx=0 tx=0).stop 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 -- 172.21.15.44:0/1597701645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f5015af60 msgr2=0x7f7f500b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f5015af60 0x7f7f500b0c10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800b9a0 tx=0x7f7f4800be60 comp rx=0 tx=0).stop 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 -- 172.21.15.44:0/1597701645 shutdown_connections 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f2c074db0 0x7f7f2c077270 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f5015af60 0x7f7f500b0c10 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f500cffc0 0x7f7f501fb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 --2- 172.21.15.44:0/1597701645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f5006f9a0 0x7f7f50157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 -- 172.21.15.44:0/1597701645 >> 172.21.15.44:0/1597701645 conn(0x7f7f500d0c90 msgr2=0x7f7f500737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:41.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 -- 172.21.15.44:0/1597701645 shutdown_connections 2024-09-13T08:51:41.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:41.861+0000 7f7f55200640 1 -- 172.21.15.44:0/1597701645 wait complete. 2024-09-13T08:51:41.884 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T08:51:42.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 -- 172.21.15.44:0/827079601 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba1c0cdb30 msgr2=0x7fba1c0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:42.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 --2- 172.21.15.44:0/827079601 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba1c0cdb30 0x7fba1c0aeef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fba14009bc0 tx=0x7fba1401cba0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 -- 172.21.15.44:0/827079601 shutdown_connections 2024-09-13T08:51:42.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 --2- 172.21.15.44:0/827079601 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba1c0cdb30 0x7fba1c0aeef0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 --2- 172.21.15.44:0/827079601 >> v2:172.21.15.129:3300/0 conn(0x7fba1c0ced80 0x7fba1c0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 -- 172.21.15.44:0/827079601 >> 172.21.15.44:0/827079601 conn(0x7fba1c0d1dc0 msgr2=0x7fba1c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:42.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 -- 172.21.15.44:0/827079601 shutdown_connections 2024-09-13T08:51:42.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 -- 172.21.15.44:0/827079601 wait complete. 2024-09-13T08:51:42.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 Processor -- start 2024-09-13T08:51:42.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 -- start start 2024-09-13T08:51:42.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba1c0cdb30 0x7fba1c151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:42.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1c0ced80 0x7fba1c0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:42.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.033+0000 7fba21200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba1c15bea0 0x7fba1c0b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:42.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba21200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1692a0 con 0x7fba1c0ced80 2024-09-13T08:51:42.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba21200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c169120 con 0x7fba1c0cdb30 2024-09-13T08:51:42.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba21200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fba1c169420 con 0x7fba1c15bea0 2024-09-13T08:51:42.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba13e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1c0ced80 0x7fba1c0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:42.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba13e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1c0ced80 0x7fba1c0b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59426/0 (socket says 172.21.15.44:59426) 2024-09-13T08:51:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba13e00640 1 -- 172.21.15.44:0/1800646377 learned_addr learned my addr 172.21.15.44:0/1800646377 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba19e00640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba1c0cdb30 0x7fba1c151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba1a800640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba1c15bea0 0x7fba1c0b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:42.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba13e00640 1 -- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba1c15bea0 msgr2=0x7fba1c0b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:42.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba13e00640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba1c15bea0 0x7fba1c0b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba13e00640 1 -- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba1c0cdb30 msgr2=0x7fba1c151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:42.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba13e00640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba1c0cdb30 0x7fba1c151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba13e00640 1 -- 172.21.15.44:0/1800646377 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c0b35b0 con 0x7fba1c0ced80 2024-09-13T08:51:42.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba13e00640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1c0ced80 0x7fba1c0b21d0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fba0400ea10 tx=0x7fba0400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:42.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba12000640 1 -- 172.21.15.44:0/1800646377 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0400cde0 con 0x7fba1c0ced80 2024-09-13T08:51:42.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba12000640 1 -- 172.21.15.44:0/1800646377 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba04004020 con 0x7fba1c0ced80 2024-09-13T08:51:42.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba21200640 1 -- 172.21.15.44:0/1800646377 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c0b16f0 con 0x7fba1c0ced80 2024-09-13T08:51:42.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba12000640 1 -- 172.21.15.44:0/1800646377 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04004470 con 0x7fba1c0ced80 2024-09-13T08:51:42.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba21200640 1 -- 172.21.15.44:0/1800646377 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c151d40 con 0x7fba1c0ced80 2024-09-13T08:51:42.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba21200640 1 -- 172.21.15.44:0/1800646377 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1c0ce670 con 0x7fba1c0ced80 2024-09-13T08:51:42.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.037+0000 7fba12000640 1 -- 172.21.15.44:0/1800646377 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fba04003600 con 0x7fba1c0ced80 2024-09-13T08:51:42.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.041+0000 7fba12000640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9f8074db0 0x7fb9f8077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:42.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.041+0000 7fba12000640 1 -- 172.21.15.44:0/1800646377 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5964+0+0 (secure 0 0 0) 0x7fba04093780 con 0x7fba1c0ced80 2024-09-13T08:51:42.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.041+0000 7fba19e00640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9f8074db0 0x7fb9f8077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:42.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.041+0000 7fba19e00640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9f8074db0 0x7fb9f8077270 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fba0c001870 tx=0x7fba0c001380 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:42.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.045+0000 7fba12000640 1 -- 172.21.15.44:0/1800646377 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fba0405fd60 con 0x7fba1c0ced80 2024-09-13T08:51:42.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.253+0000 7fba21200640 1 -- 172.21.15.44:0/1800646377 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fba1c072680 con 0x7fba1c0ced80 2024-09-13T08:51:42.254 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T08:51:42.253+0000 7f03b3c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T08:51:42.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.257+0000 7fba12000640 1 -- 172.21.15.44:0/1800646377 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) ==== 161+0+0 (secure 0 0 0) 0x7fba04064790 con 0x7fba1c0ced80 2024-09-13T08:51:42.256 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T08:51:42.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 -- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9f8074db0 msgr2=0x7fb9f8077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:42.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9f8074db0 0x7fb9f8077270 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fba0c001870 tx=0x7fba0c001380 comp rx=0 tx=0).stop 2024-09-13T08:51:42.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 -- 172.21.15.44:0/1800646377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1c0ced80 msgr2=0x7fba1c0b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:42.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1c0ced80 0x7fba1c0b21d0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fba0400ea10 tx=0x7fba0400eed0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 -- 172.21.15.44:0/1800646377 shutdown_connections 2024-09-13T08:51:42.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9f8074db0 0x7fb9f8077270 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fba1c15bea0 0x7fba1c0b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1c0ced80 0x7fba1c0b21d0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 --2- 172.21.15.44:0/1800646377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba1c0cdb30 0x7fba1c151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 -- 172.21.15.44:0/1800646377 >> 172.21.15.44:0/1800646377 conn(0x7fba1c0d1dc0 msgr2=0x7fba1c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:42.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 -- 172.21.15.44:0/1800646377 shutdown_connections 2024-09-13T08:51:42.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.261+0000 7fba21200640 1 -- 172.21.15.44:0/1800646377 wait complete. 2024-09-13T08:51:42.281 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T08:51:42.328 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:51:42.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 -- 172.21.15.44:0/1857556538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc480ce8f0 msgr2=0x7fbc480ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:42.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 --2- 172.21.15.44:0/1857556538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc480ce8f0 0x7fbc480ca480 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fbc48069630 tx=0x7fbc4401ccf0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 -- 172.21.15.44:0/1857556538 shutdown_connections 2024-09-13T08:51:42.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 --2- 172.21.15.44:0/1857556538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc480ce8f0 0x7fbc480ca480 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 --2- 172.21.15.44:0/1857556538 >> v2:172.21.15.129:3300/0 conn(0x7fbc480cf710 0x7fbc480ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 --2- 172.21.15.44:0/1857556538 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc48151410 0x7fbc480cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 -- 172.21.15.44:0/1857556538 >> 172.21.15.44:0/1857556538 conn(0x7fbc480cfc20 msgr2=0x7fbc480d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:42.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 -- 172.21.15.44:0/1857556538 shutdown_connections 2024-09-13T08:51:42.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 -- 172.21.15.44:0/1857556538 wait complete. 2024-09-13T08:51:42.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 Processor -- start 2024-09-13T08:51:42.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 -- start start 2024-09-13T08:51:42.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc480ce8f0 0x7fbc48151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:42.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.437+0000 7fbc50800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc480cf710 0x7fbc480b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:42.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc50800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc48151410 0x7fbc48142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:42.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc50800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4815dec0 con 0x7fbc48151410 2024-09-13T08:51:42.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc50800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbc480cafd0 con 0x7fbc480ce8f0 2024-09-13T08:51:42.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc50800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4815e040 con 0x7fbc480cf710 2024-09-13T08:51:42.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc48151410 0x7fbc48142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc48151410 0x7fbc48142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34856/0 (socket says 172.21.15.44:34856) 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4e800640 1 -- 172.21.15.44:0/1889291349 learned_addr learned my addr 172.21.15.44:0/1889291349 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4d400640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc480cf710 0x7fbc480b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4de00640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc480ce8f0 0x7fbc48151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4e800640 1 -- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc480cf710 msgr2=0x7fbc480b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4e800640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc480cf710 0x7fbc480b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4e800640 1 -- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc480ce8f0 msgr2=0x7fbc48151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4e800640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc480ce8f0 0x7fbc48151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4e800640 1 -- 172.21.15.44:0/1889291349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc481f7950 con 0x7fbc48151410 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc4e800640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc48151410 0x7fbc48142530 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fbc4401b3a0 tx=0x7fbc44003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc42000640 1 -- 172.21.15.44:0/1889291349 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc440bb070 con 0x7fbc48151410 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc42000640 1 -- 172.21.15.44:0/1889291349 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc44004e00 con 0x7fbc48151410 2024-09-13T08:51:42.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc50800640 1 -- 172.21.15.44:0/1889291349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc48154690 con 0x7fbc48151410 2024-09-13T08:51:42.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc42000640 1 -- 172.21.15.44:0/1889291349 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc440b2040 con 0x7fbc48151410 2024-09-13T08:51:42.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc50800640 1 -- 172.21.15.44:0/1889291349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4815c5b0 con 0x7fbc48151410 2024-09-13T08:51:42.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc50800640 1 -- 172.21.15.44:0/1889291349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc4815c3e0 con 0x7fbc48151410 2024-09-13T08:51:42.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.441+0000 7fbc42000640 1 -- 172.21.15.44:0/1889291349 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fbc440be070 con 0x7fbc48151410 2024-09-13T08:51:42.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.445+0000 7fbc42000640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc2c074db0 0x7fbc2c077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:42.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.445+0000 7fbc42000640 1 -- 172.21.15.44:0/1889291349 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6022+0+0 (secure 0 0 0) 0x7fbc4412cc00 con 0x7fbc48151410 2024-09-13T08:51:42.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.445+0000 7fbc4de00640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc2c074db0 0x7fbc2c077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:42.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.445+0000 7fbc4de00640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc2c074db0 0x7fbc2c077270 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0058d0 tx=0x7fbc3c005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:42.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.449+0000 7fbc42000640 1 -- 172.21.15.44:0/1889291349 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fbc481f7950 con 0x7fbc48151410 2024-09-13T08:51:42.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:42.661+0000 7fbc50800640 1 -- 172.21.15.44:0/1889291349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbc480ce3b0 con 0x7fbc48151410 2024-09-13T08:51:43.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.265+0000 7fbc42000640 1 -- 172.21.15.44:0/1889291349 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) ==== 104+0+0 (secure 0 0 0) 0x7fbc440fee20 con 0x7fbc48151410 2024-09-13T08:51:43.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 -- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc2c074db0 msgr2=0x7fbc2c077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:43.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc2c074db0 0x7fbc2c077270 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0058d0 tx=0x7fbc3c005860 comp rx=0 tx=0).stop 2024-09-13T08:51:43.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 -- 172.21.15.44:0/1889291349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc48151410 msgr2=0x7fbc48142530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:43.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc48151410 0x7fbc48142530 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fbc4401b3a0 tx=0x7fbc44003f60 comp rx=0 tx=0).stop 2024-09-13T08:51:43.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 -- 172.21.15.44:0/1889291349 shutdown_connections 2024-09-13T08:51:43.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbc2c074db0 0x7fbc2c077270 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:43.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc48151410 0x7fbc48142530 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbc480cf710 0x7fbc480b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 --2- 172.21.15.44:0/1889291349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc480ce8f0 0x7fbc48151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 -- 172.21.15.44:0/1889291349 >> 172.21.15.44:0/1889291349 conn(0x7fbc480cfc20 msgr2=0x7fbc48150650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 -- 172.21.15.44:0/1889291349 shutdown_connections 2024-09-13T08:51:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.269+0000 7fbc50800640 1 -- 172.21.15.44:0/1889291349 wait complete. 2024-09-13T08:51:43.292 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T08:51:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.445+0000 7ff559200640 1 -- 172.21.15.44:0/1871005244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff55415de40 msgr2=0x7ff5540b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.445+0000 7ff559200640 1 --2- 172.21.15.44:0/1871005244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff55415de40 0x7ff5540b35a0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff5540d3f30 tx=0x7ff54401c890 comp rx=0 tx=0).stop 2024-09-13T08:51:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.445+0000 7ff559200640 1 -- 172.21.15.44:0/1871005244 shutdown_connections 2024-09-13T08:51:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.445+0000 7ff559200640 1 --2- 172.21.15.44:0/1871005244 >> v2:172.21.15.129:3300/0 conn(0x7ff55415e220 0x7ff5540d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.445+0000 7ff559200640 1 --2- 172.21.15.44:0/1871005244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff55415de40 0x7ff5540b35a0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.445+0000 7ff559200640 1 -- 172.21.15.44:0/1871005244 >> 172.21.15.44:0/1871005244 conn(0x7ff5540d3f90 msgr2=0x7ff5540d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.445+0000 7ff559200640 1 -- 172.21.15.44:0/1871005244 shutdown_connections 2024-09-13T08:51:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.445+0000 7ff559200640 1 -- 172.21.15.44:0/1871005244 wait complete. 2024-09-13T08:51:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 Processor -- start 2024-09-13T08:51:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 -- start start 2024-09-13T08:51:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5540d20a0 0x7ff554152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff55415de40 0x7ff5540b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff55415e220 0x7ff5540b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff55416b0e0 con 0x7ff55415de40 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff55416af60 con 0x7ff55415e220 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff55416b260 con 0x7ff5540d20a0 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff55415de40 0x7ff5540b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff55415de40 0x7ff5540b87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34868/0 (socket says 172.21.15.44:34868) 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54be00640 1 -- 172.21.15.44:0/1524726527 learned_addr learned my addr 172.21.15.44:0/1524726527 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff552800640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff55415e220 0x7ff5540b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff551e00640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5540d20a0 0x7ff554152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54be00640 1 -- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5540d20a0 msgr2=0x7ff554152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54be00640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5540d20a0 0x7ff554152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54be00640 1 -- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff55415e220 msgr2=0x7ff5540b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:43.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54be00640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff55415e220 0x7ff5540b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54be00640 1 -- 172.21.15.44:0/1524726527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5540b7750 con 0x7ff55415de40 2024-09-13T08:51:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54be00640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff55415de40 0x7ff5540b87f0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00d720 tx=0x7ff53c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54a000640 1 -- 172.21.15.44:0/1524726527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c017070 con 0x7ff55415de40 2024-09-13T08:51:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54a000640 1 -- 172.21.15.44:0/1524726527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff53c009d20 con 0x7ff55415de40 2024-09-13T08:51:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 -- 172.21.15.44:0/1524726527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5540b7a30 con 0x7ff55415de40 2024-09-13T08:51:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff54a000640 1 -- 172.21.15.44:0/1524726527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0106c0 con 0x7ff55415de40 2024-09-13T08:51:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 -- 172.21.15.44:0/1524726527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5540b7ff0 con 0x7ff55415de40 2024-09-13T08:51:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.449+0000 7ff559200640 1 -- 172.21.15.44:0/1524726527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5540b84d0 con 0x7ff55415de40 2024-09-13T08:51:43.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.453+0000 7ff54a000640 1 -- 172.21.15.44:0/1524726527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7ff53c01a070 con 0x7ff55415de40 2024-09-13T08:51:43.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.453+0000 7ff54a000640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff530074db0 0x7ff530077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:43.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.453+0000 7ff54a000640 1 -- 172.21.15.44:0/1524726527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6022+0+0 (secure 0 0 0) 0x7ff53c093000 con 0x7ff55415de40 2024-09-13T08:51:43.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.453+0000 7ff551e00640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff530074db0 0x7ff530077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:43.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.453+0000 7ff551e00640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff530074db0 0x7ff530077270 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff5440af000 tx=0x7ff54401b0d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:43.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.457+0000 7ff54a000640 1 -- 172.21.15.44:0/1524726527 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff5540b84d0 con 0x7ff55415de40 2024-09-13T08:51:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:43.633+0000 7ff559200640 1 -- 172.21.15.44:0/1524726527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff5540b5860 con 0x7ff55415de40 2024-09-13T08:51:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff54a000640 1 -- 172.21.15.44:0/1524726527 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) ==== 181+0+0 (secure 0 0 0) 0x7ff53c05fcb0 con 0x7ff55415de40 2024-09-13T08:51:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T08:51:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff559200640 1 -- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff530074db0 msgr2=0x7ff530077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff559200640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff530074db0 0x7ff530077270 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff5440af000 tx=0x7ff54401b0d0 comp rx=0 tx=0).stop 2024-09-13T08:51:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff559200640 1 -- 172.21.15.44:0/1524726527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff55415de40 msgr2=0x7ff5540b87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff559200640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff55415de40 0x7ff5540b87f0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00d720 tx=0x7ff53c00dbe0 comp rx=0 tx=0).stop 2024-09-13T08:51:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff559200640 1 -- 172.21.15.44:0/1524726527 shutdown_connections 2024-09-13T08:51:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff559200640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff530074db0 0x7ff530077270 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff559200640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff55415e220 0x7ff5540b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff559200640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff55415de40 0x7ff5540b87f0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff559200640 1 --2- 172.21.15.44:0/1524726527 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff5540d20a0 0x7ff554152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.197+0000 7ff559200640 1 -- 172.21.15.44:0/1524726527 >> 172.21.15.44:0/1524726527 conn(0x7ff5540d3f90 msgr2=0x7ff5540b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.201+0000 7ff559200640 1 -- 172.21.15.44:0/1524726527 shutdown_connections 2024-09-13T08:51:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.201+0000 7ff559200640 1 -- 172.21.15.44:0/1524726527 wait complete. 2024-09-13T08:51:44.226 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T08:51:44.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 -- 172.21.15.44:0/3140296327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb815e360 msgr2=0x7f8bb80adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:44.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 --2- 172.21.15.44:0/3140296327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb815e360 0x7f8bb80adf50 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009840 tx=0x7f8bac01c790 comp rx=0 tx=0).stop 2024-09-13T08:51:44.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 -- 172.21.15.44:0/3140296327 shutdown_connections 2024-09-13T08:51:44.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 --2- 172.21.15.44:0/3140296327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb815e360 0x7f8bb80adf50 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:44.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 -- 172.21.15.44:0/3140296327 >> 172.21.15.44:0/3140296327 conn(0x7f8bb80cfbc0 msgr2=0x7f8bb80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:44.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 -- 172.21.15.44:0/3140296327 shutdown_connections 2024-09-13T08:51:44.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 -- 172.21.15.44:0/3140296327 wait complete. 2024-09-13T08:51:44.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 Processor -- start 2024-09-13T08:51:44.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 -- start start 2024-09-13T08:51:44.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bb80aef60 0x7f8bb8166790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:44.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb815e360 0x7f8bb80b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:44.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8bb815e740 0x7f8bb80b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:44.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8169540 con 0x7f8bb815e360 2024-09-13T08:51:44.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb815f900 con 0x7f8bb80aef60 2024-09-13T08:51:44.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bc0600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb81696c0 con 0x7f8bb815e740 2024-09-13T08:51:44.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bbd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb815e360 0x7f8bb80b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:44.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bbd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb815e360 0x7f8bb80b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34894/0 (socket says 172.21.15.44:34894) 2024-09-13T08:51:44.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bbd200640 1 -- 172.21.15.44:0/1993494328 learned_addr learned my addr 172.21.15.44:0/1993494328 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:44.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bbdc00640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bb80aef60 0x7f8bb8166790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.381+0000 7f8bbe600640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8bb815e740 0x7f8bb80b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bbd200640 1 -- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8bb815e740 msgr2=0x7f8bb80b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bbd200640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8bb815e740 0x7f8bb80b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bbd200640 1 -- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bb80aef60 msgr2=0x7f8bb8166790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bbd200640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bb80aef60 0x7f8bb8166790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bbd200640 1 -- 172.21.15.44:0/1993494328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb813c190 con 0x7f8bb815e360 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bbe600640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8bb815e740 0x7f8bb80b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bbd200640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb815e360 0x7f8bb80b0510 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c4d0 tx=0x7f8ba400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bb5600640 1 -- 172.21.15.44:0/1993494328 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4016020 con 0x7f8bb815e360 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bb5600640 1 -- 172.21.15.44:0/1993494328 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ba4012920 con 0x7f8bb815e360 2024-09-13T08:51:44.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bc0600640 1 -- 172.21.15.44:0/1993494328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb813beb0 con 0x7f8bb815e360 2024-09-13T08:51:44.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bb5600640 1 -- 172.21.15.44:0/1993494328 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4012d50 con 0x7f8bb815e360 2024-09-13T08:51:44.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bc0600640 1 -- 172.21.15.44:0/1993494328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb81fb510 con 0x7f8bb815e360 2024-09-13T08:51:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bc0600640 1 -- 172.21.15.44:0/1993494328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb815ebd0 con 0x7f8bb815e360 2024-09-13T08:51:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bb5600640 1 -- 172.21.15.44:0/1993494328 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f8ba4019020 con 0x7f8bb815e360 2024-09-13T08:51:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bb5600640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b98074db0 0x7f8b98077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.385+0000 7f8bb5600640 1 -- 172.21.15.44:0/1993494328 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5555+0+0 (secure 0 0 0) 0x7f8ba4092b20 con 0x7f8bb815e360 2024-09-13T08:51:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.389+0000 7f8bbdc00640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b98074db0 0x7f8b98077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:44.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.389+0000 7f8bbdc00640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b98074db0 0x7f8b98077270 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0af000 tx=0x7f8bac01afd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:44.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.389+0000 7f8bb5600640 1 -- 172.21.15.44:0/1993494328 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8ba4020030 con 0x7f8bb815e360 2024-09-13T08:51:44.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:44.569+0000 7f8bc0600640 1 -- 172.21.15.44:0/1993494328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8bb80ce5a0 con 0x7f8bb815e360 2024-09-13T08:51:45.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.205+0000 7f8bb5600640 1 -- 172.21.15.44:0/1993494328 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) ==== 169+0+0 (secure 0 0 0) 0x7f8ba405f7d0 con 0x7f8bb815e360 2024-09-13T08:51:45.205 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T08:51:45.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 -- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b98074db0 msgr2=0x7f8b98077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:45.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b98074db0 0x7f8b98077270 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0af000 tx=0x7f8bac01afd0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 -- 172.21.15.44:0/1993494328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb815e360 msgr2=0x7f8bb80b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:45.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb815e360 0x7f8bb80b0510 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c4d0 tx=0x7f8ba400c990 comp rx=0 tx=0).stop 2024-09-13T08:51:45.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 -- 172.21.15.44:0/1993494328 shutdown_connections 2024-09-13T08:51:45.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b98074db0 0x7f8b98077270 secure :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0af000 tx=0x7f8bac01afd0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8bb815e740 0x7f8bb80b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb815e360 0x7f8bb80b0510 secure :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c4d0 tx=0x7f8ba400c990 comp rx=0 tx=0).stop 2024-09-13T08:51:45.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 --2- 172.21.15.44:0/1993494328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bb80aef60 0x7f8bb8166790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 -- 172.21.15.44:0/1993494328 >> 172.21.15.44:0/1993494328 conn(0x7f8bb80cfbc0 msgr2=0x7f8bb813c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:45.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 -- 172.21.15.44:0/1993494328 shutdown_connections 2024-09-13T08:51:45.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.209+0000 7f8bc0600640 1 -- 172.21.15.44:0/1993494328 wait complete. 2024-09-13T08:51:45.237 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-13T08:51:45.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.385+0000 7fac2a200640 1 -- 172.21.15.44:0/3843679758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2415c130 msgr2=0x7fac240cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:45.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.385+0000 7fac2a200640 1 --2- 172.21.15.44:0/3843679758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2415c130 0x7fac240cd050 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fac240d0780 tx=0x7fac1c01cc80 comp rx=0 tx=0).stop 2024-09-13T08:51:45.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.385+0000 7fac2a200640 1 -- 172.21.15.44:0/3843679758 shutdown_connections 2024-09-13T08:51:45.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.385+0000 7fac2a200640 1 --2- 172.21.15.44:0/3843679758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2415c130 0x7fac240cd050 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.385+0000 7fac2a200640 1 -- 172.21.15.44:0/3843679758 >> 172.21.15.44:0/3843679758 conn(0x7fac240d1ea0 msgr2=0x7fac240d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:45.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.385+0000 7fac2a200640 1 -- 172.21.15.44:0/3843679758 shutdown_connections 2024-09-13T08:51:45.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac2a200640 1 -- 172.21.15.44:0/3843679758 wait complete. 2024-09-13T08:51:45.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac2a200640 1 Processor -- start 2024-09-13T08:51:45.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac2a200640 1 -- start start 2024-09-13T08:51:45.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac2a200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fac240cecf0 0x7fac240ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:45.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac2a200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2415bd50 0x7fac240c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:45.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac2a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2415c130 0x7fac2414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:45.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac2a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fac2416a5b0 con 0x7fac2415c130 2024-09-13T08:51:45.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac2a200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac2416a430 con 0x7fac2415bd50 2024-09-13T08:51:45.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac2a200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fac2416a730 con 0x7fac240cecf0 2024-09-13T08:51:45.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac23e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2415c130 0x7fac2414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:45.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac23e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2415c130 0x7fac2414a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34908/0 (socket says 172.21.15.44:34908) 2024-09-13T08:51:45.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac23e00640 1 -- 172.21.15.44:0/4235717533 learned_addr learned my addr 172.21.15.44:0/4235717533 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac1be00640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2415bd50 0x7fac240c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac23400640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fac240cecf0 0x7fac240ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac23e00640 1 -- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fac240cecf0 msgr2=0x7fac240ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac23e00640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fac240cecf0 0x7fac240ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac23e00640 1 -- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2415bd50 msgr2=0x7fac240c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac23e00640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2415bd50 0x7fac240c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac23e00640 1 -- 172.21.15.44:0/4235717533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac240cada0 con 0x7fac2415c130 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac1be00640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2415bd50 0x7fac240c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac23e00640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2415c130 0x7fac2414a6d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fac1c019890 tx=0x7fac1c003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac1b400640 1 -- 172.21.15.44:0/4235717533 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0bc070 con 0x7fac2415c130 2024-09-13T08:51:45.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac1b400640 1 -- 172.21.15.44:0/4235717533 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac1c0ad8e0 con 0x7fac2415c130 2024-09-13T08:51:45.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.389+0000 7fac1b400640 1 -- 172.21.15.44:0/4235717533 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0b2040 con 0x7fac2415c130 2024-09-13T08:51:45.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.393+0000 7fac2a200640 1 -- 172.21.15.44:0/4235717533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac240c97c0 con 0x7fac2415c130 2024-09-13T08:51:45.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.393+0000 7fac2a200640 1 -- 172.21.15.44:0/4235717533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac240ca060 con 0x7fac2415c130 2024-09-13T08:51:45.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.393+0000 7fac2a200640 1 -- 172.21.15.44:0/4235717533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac240ce5e0 con 0x7fac2415c130 2024-09-13T08:51:45.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.393+0000 7fac1b400640 1 -- 172.21.15.44:0/4235717533 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fac1c004000 con 0x7fac2415c130 2024-09-13T08:51:45.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.393+0000 7fac1b400640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fac00074db0 0x7fac00077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:45.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.393+0000 7fac1b400640 1 -- 172.21.15.44:0/4235717533 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5055+0+0 (secure 0 0 0) 0x7fac1c12cdc0 con 0x7fac2415c130 2024-09-13T08:51:45.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.393+0000 7fac23400640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fac00074db0 0x7fac00077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:45.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.397+0000 7fac23400640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fac00074db0 0x7fac00077270 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fac14000f70 tx=0x7fac14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:45.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.397+0000 7fac1b400640 1 -- 172.21.15.44:0/4235717533 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fac1c016990 con 0x7fac2415c130 2024-09-13T08:51:45.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.577+0000 7fac2a200640 1 -- 172.21.15.44:0/4235717533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fac240c94b0 con 0x7fac2415c130 2024-09-13T08:51:45.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac1b400640 1 -- 172.21.15.44:0/4235717533 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v50) ==== 71+0+0 (secure 0 0 0) 0x7fac1c0f9a70 con 0x7fac2415c130 2024-09-13T08:51:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 -- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fac00074db0 msgr2=0x7fac00077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fac00074db0 0x7fac00077270 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fac14000f70 tx=0x7fac14009040 comp rx=0 tx=0).stop 2024-09-13T08:51:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 -- 172.21.15.44:0/4235717533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2415c130 msgr2=0x7fac2414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2415c130 0x7fac2414a6d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fac1c019890 tx=0x7fac1c003f80 comp rx=0 tx=0).stop 2024-09-13T08:51:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 -- 172.21.15.44:0/4235717533 shutdown_connections 2024-09-13T08:51:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fac00074db0 0x7fac00077270 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2415c130 0x7fac2414a6d0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2415bd50 0x7fac240c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 --2- 172.21.15.44:0/4235717533 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fac240cecf0 0x7fac240ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 -- 172.21.15.44:0/4235717533 >> 172.21.15.44:0/4235717533 conn(0x7fac240d1ea0 msgr2=0x7fac240d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 -- 172.21.15.44:0/4235717533 shutdown_connections 2024-09-13T08:51:45.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.581+0000 7fac2a200640 1 -- 172.21.15.44:0/4235717533 wait complete. 2024-09-13T08:51:45.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-13T08:51:45.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.749+0000 7fb25b800640 1 -- 172.21.15.44:0/1327499747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2540b4270 msgr2=0x7fb2540af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:45.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.749+0000 7fb25b800640 1 --2- 172.21.15.44:0/1327499747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2540b4270 0x7fb2540af900 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb2540b3fe0 tx=0x7fb24c01cb80 comp rx=0 tx=0).stop 2024-09-13T08:51:45.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.749+0000 7fb25b800640 1 -- 172.21.15.44:0/1327499747 shutdown_connections 2024-09-13T08:51:45.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.749+0000 7fb25b800640 1 --2- 172.21.15.44:0/1327499747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2540b4270 0x7fb2540af900 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.749+0000 7fb25b800640 1 --2- 172.21.15.44:0/1327499747 >> v2:172.21.15.129:3300/0 conn(0x7fb2540d0af0 0x7fb254068c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.749+0000 7fb25b800640 1 -- 172.21.15.44:0/1327499747 >> 172.21.15.44:0/1327499747 conn(0x7fb2540b2240 msgr2=0x7fb2540b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.749+0000 7fb25b800640 1 -- 172.21.15.44:0/1327499747 shutdown_connections 2024-09-13T08:51:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.749+0000 7fb25b800640 1 -- 172.21.15.44:0/1327499747 wait complete. 2024-09-13T08:51:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.749+0000 7fb25b800640 1 Processor -- start 2024-09-13T08:51:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.749+0000 7fb25b800640 1 -- start start 2024-09-13T08:51:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb25b800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2540b4270 0x7fb254152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb25b800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb2540d0af0 0x7fb25415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb25b800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb25415bde0 0x7fb25415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb25b800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb254168f90 con 0x7fb2540b4270 2024-09-13T08:51:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb25b800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb254168e10 con 0x7fb25415bde0 2024-09-13T08:51:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb25b800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb254169110 con 0x7fb2540d0af0 2024-09-13T08:51:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb258e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2540b4270 0x7fb254152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb258e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2540b4270 0x7fb254152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34932/0 (socket says 172.21.15.44:34932) 2024-09-13T08:51:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb258e00640 1 -- 172.21.15.44:0/3150971801 learned_addr learned my addr 172.21.15.44:0/3150971801 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb259800640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb25415bde0 0x7fb25415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb253e00640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb2540d0af0 0x7fb25415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb258e00640 1 -- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb2540d0af0 msgr2=0x7fb25415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb258e00640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb2540d0af0 0x7fb25415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb258e00640 1 -- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb25415bde0 msgr2=0x7fb25415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb258e00640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb25415bde0 0x7fb25415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb258e00640 1 -- 172.21.15.44:0/3150971801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25415b390 con 0x7fb2540b4270 2024-09-13T08:51:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb258e00640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2540b4270 0x7fb254152ef0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb248001ec0 tx=0x7fb24800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb250c00640 1 -- 172.21.15.44:0/3150971801 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248017070 con 0x7fb2540b4270 2024-09-13T08:51:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb250c00640 1 -- 172.21.15.44:0/3150971801 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb248003600 con 0x7fb2540b4270 2024-09-13T08:51:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb250c00640 1 -- 172.21.15.44:0/3150971801 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248003a50 con 0x7fb2540b4270 2024-09-13T08:51:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb25b800640 1 -- 172.21.15.44:0/3150971801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2541586d0 con 0x7fb2540b4270 2024-09-13T08:51:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb25b800640 1 -- 172.21.15.44:0/3150971801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb254158c90 con 0x7fb2540b4270 2024-09-13T08:51:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.753+0000 7fb25b800640 1 -- 172.21.15.44:0/3150971801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2540af900 con 0x7fb2540b4270 2024-09-13T08:51:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.757+0000 7fb250c00640 1 -- 172.21.15.44:0/3150971801 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fb24801a070 con 0x7fb2540b4270 2024-09-13T08:51:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.757+0000 7fb250c00640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb234074de0 0x7fb2340772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.757+0000 7fb250c00640 1 -- 172.21.15.44:0/3150971801 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5055+0+0 (secure 0 0 0) 0x7fb2480920d0 con 0x7fb2540b4270 2024-09-13T08:51:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.757+0000 7fb253e00640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb234074de0 0x7fb2340772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:45.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.757+0000 7fb253e00640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb234074de0 0x7fb2340772a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb25415d150 tx=0x7fb24000b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:45.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:45.761+0000 7fb250c00640 1 -- 172.21.15.44:0/3150971801 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb24805e730 con 0x7fb2540b4270 2024-09-13T08:51:46.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.013+0000 7fb25b800640 1 -- 172.21.15.44:0/3150971801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7fb2540b1e80 con 0x7fb2540b4270 2024-09-13T08:51:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.205+0000 7fb250c00640 1 -- 172.21.15.44:0/3150971801 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v642) ==== 185+0+0 (secure 0 0 0) 0x7fb248063160 con 0x7fb2540b4270 2024-09-13T08:51:46.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 -- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb234074de0 msgr2=0x7fb2340772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb234074de0 0x7fb2340772a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb25415d150 tx=0x7fb24000b390 comp rx=0 tx=0).stop 2024-09-13T08:51:46.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 -- 172.21.15.44:0/3150971801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2540b4270 msgr2=0x7fb254152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2540b4270 0x7fb254152ef0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb248001ec0 tx=0x7fb24800c890 comp rx=0 tx=0).stop 2024-09-13T08:51:46.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 -- 172.21.15.44:0/3150971801 shutdown_connections 2024-09-13T08:51:46.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb234074de0 0x7fb2340772a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb25415bde0 0x7fb25415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb2540d0af0 0x7fb25415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 --2- 172.21.15.44:0/3150971801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2540b4270 0x7fb254152ef0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 -- 172.21.15.44:0/3150971801 >> 172.21.15.44:0/3150971801 conn(0x7fb2540b2240 msgr2=0x7fb25406cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:46.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 -- 172.21.15.44:0/3150971801 shutdown_connections 2024-09-13T08:51:46.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.209+0000 7fb25b800640 1 -- 172.21.15.44:0/3150971801 wait complete. 2024-09-13T08:51:46.230 INFO:tasks.cephfs_test_runner:Test that progress bars, that printed for 3 ongoing clone jobs, are ... ok 2024-09-13T08:51:46.230 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the progress bar that is printed for 1 ongoing clone job is 2024-09-13T08:51:46.230 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-13T08:51:46.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:51:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.377+0000 7f2f7f800640 1 -- 172.21.15.44:0/672310157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f5c004d50 msgr2=0x7f2f5c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.377+0000 7f2f7f800640 1 --2- 172.21.15.44:0/672310157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f5c004d50 0x7f2f5c0051d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009eb0 tx=0x7f2f7001d5f0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.377+0000 7f2f7d800640 1 -- 172.21.15.44:0/672310157 reap_dead start 2024-09-13T08:51:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.377+0000 7f2f7f800640 1 -- 172.21.15.44:0/672310157 shutdown_connections 2024-09-13T08:51:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.377+0000 7f2f7f800640 1 -- 172.21.15.44:0/672310157 >> 172.21.15.44:0/672310157 conn(0x7f2f780d1dc0 msgr2=0x7f2f780d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.377+0000 7f2f7f800640 1 -- 172.21.15.44:0/672310157 shutdown_connections 2024-09-13T08:51:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.377+0000 7f2f7f800640 1 -- 172.21.15.44:0/672310157 wait complete. 2024-09-13T08:51:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7f800640 1 Processor -- start 2024-09-13T08:51:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7f800640 1 -- start start 2024-09-13T08:51:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7f800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f5c004d50 0x7f2f7815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7f800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f780b1d10 0x7f2f781f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7f800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f7806dd30 0x7f2f7806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7f800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f780d2a60 con 0x7f2f7806dd30 2024-09-13T08:51:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7f800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f780d28e0 con 0x7f2f780b1d10 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7f800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2f780d2be0 con 0x7f2f5c004d50 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f7806dd30 0x7f2f7806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f7806dd30 0x7f2f7806e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34944/0 (socket says 172.21.15.44:34944) 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7d800640 1 -- 172.21.15.44:0/561237214 learned_addr learned my addr 172.21.15.44:0/561237214 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7ce00640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f5c004d50 0x7f2f7815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f6fe00640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f780b1d10 0x7f2f781f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7d800640 1 -- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f5c004d50 msgr2=0x7f2f7815b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7d800640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f5c004d50 0x7f2f7815b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7d800640 1 -- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f780b1d10 msgr2=0x7f2f781f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7d800640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f780b1d10 0x7f2f781f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7d800640 1 -- 172.21.15.44:0/561237214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f781f5010 con 0x7f2f7806dd30 2024-09-13T08:51:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f7d800640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f7806dd30 0x7f2f7806e1e0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009eb0 tx=0x7f2f70004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:46.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f75600640 1 -- 172.21.15.44:0/561237214 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f700ad050 con 0x7f2f7806dd30 2024-09-13T08:51:46.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.381+0000 7f2f75600640 1 -- 172.21.15.44:0/561237214 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f700a7a90 con 0x7f2f7806dd30 2024-09-13T08:51:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.385+0000 7f2f75600640 1 -- 172.21.15.44:0/561237214 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70009200 con 0x7f2f7806dd30 2024-09-13T08:51:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.385+0000 7f2f7f800640 1 -- 172.21.15.44:0/561237214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f781f52f0 con 0x7f2f7806dd30 2024-09-13T08:51:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.385+0000 7f2f7f800640 1 -- 172.21.15.44:0/561237214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f780b2190 con 0x7f2f7806dd30 2024-09-13T08:51:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.385+0000 7f2f7f800640 1 -- 172.21.15.44:0/561237214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f780ce670 con 0x7f2f7806dd30 2024-09-13T08:51:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.385+0000 7f2f75600640 1 -- 172.21.15.44:0/561237214 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f2f700b0070 con 0x7f2f7806dd30 2024-09-13T08:51:46.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.385+0000 7f2f75600640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f54074de0 0x7f2f540772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:46.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.385+0000 7f2f75600640 1 -- 172.21.15.44:0/561237214 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5055+0+0 (secure 0 0 0) 0x7f2f7012def0 con 0x7f2f7806dd30 2024-09-13T08:51:46.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.385+0000 7f2f7ce00640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f54074de0 0x7f2f540772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:46.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.389+0000 7f2f7ce00640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f54074de0 0x7f2f540772a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2f68001870 tx=0x7f2f68000f50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:46.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.389+0000 7f2f75600640 1 -- 172.21.15.44:0/561237214 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2f70017300 con 0x7f2f7806dd30 2024-09-13T08:51:46.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.569+0000 7f2f7f800640 1 -- 172.21.15.44:0/561237214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f781e6aa0 con 0x7f2f7806dd30 2024-09-13T08:51:46.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.569+0000 7f2f75600640 1 -- 172.21.15.44:0/561237214 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+13231 (secure 0 0 0) 0x7f2f700fac20 con 0x7f2f7806dd30 2024-09-13T08:51:46.569 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:46.569 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":122,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:51:45.199811+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6834/1590451084":"2024-09-14T08:51:42.257648+0000","172.21.15.44:6832/1590451084":"2024-09-14T08:51:42.257648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 -- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f54074de0 msgr2=0x7f2f540772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f54074de0 0x7f2f540772a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2f68001870 tx=0x7f2f68000f50 comp rx=0 tx=0).stop 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 -- 172.21.15.44:0/561237214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f7806dd30 msgr2=0x7f2f7806e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f7806dd30 0x7f2f7806e1e0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009eb0 tx=0x7f2f70004ec0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 -- 172.21.15.44:0/561237214 shutdown_connections 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2f54074de0 0x7f2f540772a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f7806dd30 0x7f2f7806e1e0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f780b1d10 0x7f2f781f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 --2- 172.21.15.44:0/561237214 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2f5c004d50 0x7f2f7815b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 -- 172.21.15.44:0/561237214 >> 172.21.15.44:0/561237214 conn(0x7f2f780d1dc0 msgr2=0x7f2f781e4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 -- 172.21.15.44:0/561237214 shutdown_connections 2024-09-13T08:51:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.573+0000 7f2f7f800640 1 -- 172.21.15.44:0/561237214 wait complete. 2024-09-13T08:51:46.591 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T08:37:24.563344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T08:51:46.592 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T08:51:46.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.741+0000 7f4d8c600640 1 -- 172.21.15.44:0/37316856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d6c004810 msgr2=0x7f4d6c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.741+0000 7f4d8c600640 1 --2- 172.21.15.44:0/37316856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d6c004810 0x7f4d6c004c90 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006f20 tx=0x7f4d7001cb60 comp rx=0 tx=0).stop 2024-09-13T08:51:46.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8a600640 1 -- 172.21.15.44:0/37316856 reap_dead start 2024-09-13T08:51:46.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 -- 172.21.15.44:0/37316856 shutdown_connections 2024-09-13T08:51:46.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 -- 172.21.15.44:0/37316856 >> 172.21.15.44:0/37316856 conn(0x7f4d840d1ea0 msgr2=0x7f4d840d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:46.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 -- 172.21.15.44:0/37316856 shutdown_connections 2024-09-13T08:51:46.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 -- 172.21.15.44:0/37316856 wait complete. 2024-09-13T08:51:46.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 Processor -- start 2024-09-13T08:51:46.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 -- start start 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d6c004810 0x7f4d840b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d84153d20 0x7f4d841f8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d84155710 0x7f4d840cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84167e50 con 0x7f4d84155710 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84167cd0 con 0x7f4d84153d20 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8c600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4d84167fd0 con 0x7f4d6c004810 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d84155710 0x7f4d840cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d84155710 0x7f4d840cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34966/0 (socket says 172.21.15.44:34966) 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8a600640 1 -- 172.21.15.44:0/907765292 learned_addr learned my addr 172.21.15.44:0/907765292 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d89c00640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d6c004810 0x7f4d840b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:46.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d89200640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d84153d20 0x7f4d841f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8a600640 1 -- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d6c004810 msgr2=0x7f4d840b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8a600640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d6c004810 0x7f4d840b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8a600640 1 -- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d84153d20 msgr2=0x7f4d841f8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8a600640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d84153d20 0x7f4d841f8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8a600640 1 -- 172.21.15.44:0/907765292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8406dd30 con 0x7f4d84155710 2024-09-13T08:51:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.745+0000 7f4d8a600640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d84155710 0x7f4d840cb920 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f4d840d1470 tx=0x7f4d7c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.749+0000 7f4d81600640 1 -- 172.21.15.44:0/907765292 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c003980 con 0x7f4d84155710 2024-09-13T08:51:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.749+0000 7f4d81600640 1 -- 172.21.15.44:0/907765292 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d7c015070 con 0x7f4d84155710 2024-09-13T08:51:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.749+0000 7f4d81600640 1 -- 172.21.15.44:0/907765292 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c004030 con 0x7f4d84155710 2024-09-13T08:51:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.749+0000 7f4d8c600640 1 -- 172.21.15.44:0/907765292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d840b5ac0 con 0x7f4d84155710 2024-09-13T08:51:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.749+0000 7f4d8c600640 1 -- 172.21.15.44:0/907765292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8406dfd0 con 0x7f4d84155710 2024-09-13T08:51:46.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.749+0000 7f4d8c600640 1 -- 172.21.15.44:0/907765292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d840cc940 con 0x7f4d84155710 2024-09-13T08:51:46.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.749+0000 7f4d81600640 1 -- 172.21.15.44:0/907765292 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f4d7c0041d0 con 0x7f4d84155710 2024-09-13T08:51:46.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.749+0000 7f4d81600640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d64074db0 0x7f4d64077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:46.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.749+0000 7f4d81600640 1 -- 172.21.15.44:0/907765292 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5055+0+0 (secure 0 0 0) 0x7f4d7c093810 con 0x7f4d84155710 2024-09-13T08:51:46.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.749+0000 7f4d89c00640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d64074db0 0x7f4d64077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:46.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.753+0000 7f4d89c00640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d64074db0 0x7f4d64077270 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4d78004ad0 tx=0x7f4d78006200 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:46.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.753+0000 7f4d81600640 1 -- 172.21.15.44:0/907765292 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4d7c05fde0 con 0x7f4d84155710 2024-09-13T08:51:46.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.949+0000 7f4d8c600640 1 -- 172.21.15.44:0/907765292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d840c8c30 con 0x7f4d84155710 2024-09-13T08:51:46.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d81600640 1 -- 172.21.15.44:0/907765292 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7f4d7c064810 con 0x7f4d84155710 2024-09-13T08:51:46.951 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T08:51:46.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 -- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d64074db0 msgr2=0x7f4d64077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d64074db0 0x7f4d64077270 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4d78004ad0 tx=0x7f4d78006200 comp rx=0 tx=0).stop 2024-09-13T08:51:46.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 -- 172.21.15.44:0/907765292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d84155710 msgr2=0x7f4d840cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:46.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d84155710 0x7f4d840cb920 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f4d840d1470 tx=0x7f4d7c010f40 comp rx=0 tx=0).stop 2024-09-13T08:51:46.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 -- 172.21.15.44:0/907765292 shutdown_connections 2024-09-13T08:51:46.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4d64074db0 0x7f4d64077270 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d84155710 0x7f4d840cb920 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d84153d20 0x7f4d841f8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 --2- 172.21.15.44:0/907765292 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4d6c004810 0x7f4d840b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:46.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 -- 172.21.15.44:0/907765292 >> 172.21.15.44:0/907765292 conn(0x7f4d840d1ea0 msgr2=0x7f4d840b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:46.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 -- 172.21.15.44:0/907765292 shutdown_connections 2024-09-13T08:51:46.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:46.953+0000 7f4d8c600640 1 -- 172.21.15.44:0/907765292 wait complete. 2024-09-13T08:51:46.972 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-13T08:51:47.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 -- 172.21.15.44:0/3968679043 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75540d28c0 msgr2=0x7f75540cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:47.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 --2- 172.21.15.44:0/3968679043 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75540d28c0 0x7f75540cea20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f753c009840 tx=0x7f753c01c820 comp rx=0 tx=0).stop 2024-09-13T08:51:47.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 -- 172.21.15.44:0/3968679043 shutdown_connections 2024-09-13T08:51:47.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 --2- 172.21.15.44:0/3968679043 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75540d28c0 0x7f75540cea20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:47.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 --2- 172.21.15.44:0/3968679043 >> v2:172.21.15.129:3300/0 conn(0x7f755415be80 0x7f75540ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:47.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 -- 172.21.15.44:0/3968679043 >> 172.21.15.44:0/3968679043 conn(0x7f75540d1da0 msgr2=0x7f75540d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:47.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 -- 172.21.15.44:0/3968679043 shutdown_connections 2024-09-13T08:51:47.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 -- 172.21.15.44:0/3968679043 wait complete. 2024-09-13T08:51:47.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 Processor -- start 2024-09-13T08:51:47.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 -- start start 2024-09-13T08:51:47.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75540b53f0 0x7f755406f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:47.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75540d28c0 0x7f75541f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:47.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755415be80 0x7f7554071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:47.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7554169160 con 0x7f755415be80 2024-09-13T08:51:47.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7554168fe0 con 0x7f75540b53f0 2024-09-13T08:51:47.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f755a200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f75541692e0 con 0x7f75540d28c0 2024-09-13T08:51:47.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f7553e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755415be80 0x7f7554071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:47.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f7553e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755415be80 0x7f7554071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34968/0 (socket says 172.21.15.44:34968) 2024-09-13T08:51:47.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f7553e00640 1 -- 172.21.15.44:0/3658657635 learned_addr learned my addr 172.21.15.44:0/3658657635 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:47.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f754aa00640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75540d28c0 0x7f75541f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f7553400640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75540b53f0 0x7f755406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f7553e00640 1 -- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75540d28c0 msgr2=0x7f75541f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f7553e00640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75540d28c0 0x7f75541f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f7553e00640 1 -- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75540b53f0 msgr2=0x7f755406f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f7553e00640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75540b53f0 0x7f755406f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.121+0000 7f7553e00640 1 -- 172.21.15.44:0/3658657635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75541fbaf0 con 0x7f755415be80 2024-09-13T08:51:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.125+0000 7f7553e00640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755415be80 0x7f7554071330 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f754c00cb10 tx=0x7f754c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.125+0000 7f754be00640 1 -- 172.21.15.44:0/3658657635 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c016020 con 0x7f755415be80 2024-09-13T08:51:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.125+0000 7f754be00640 1 -- 172.21.15.44:0/3658657635 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f754c00ce50 con 0x7f755415be80 2024-09-13T08:51:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.125+0000 7f754be00640 1 -- 172.21.15.44:0/3658657635 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c00ade0 con 0x7f755415be80 2024-09-13T08:51:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.125+0000 7f755a200640 1 -- 172.21.15.44:0/3658657635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75541fbdd0 con 0x7f755415be80 2024-09-13T08:51:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.125+0000 7f755a200640 1 -- 172.21.15.44:0/3658657635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75541fc390 con 0x7f755415be80 2024-09-13T08:51:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.125+0000 7f755a200640 1 -- 172.21.15.44:0/3658657635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75540b5860 con 0x7f755415be80 2024-09-13T08:51:47.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.125+0000 7f754be00640 1 -- 172.21.15.44:0/3658657635 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f754c019020 con 0x7f755415be80 2024-09-13T08:51:47.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.125+0000 7f754be00640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7530074db0 0x7f7530077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:47.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.125+0000 7f754be00640 1 -- 172.21.15.44:0/3658657635 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5055+0+0 (secure 0 0 0) 0x7f754c092fe0 con 0x7f755415be80 2024-09-13T08:51:47.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.129+0000 7f7553400640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7530074db0 0x7f7530077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:47.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.129+0000 7f7553400640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7530074db0 0x7f7530077270 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7554069730 tx=0x7f75440091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:47.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.129+0000 7f754be00640 1 -- 172.21.15.44:0/3658657635 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f754c020020 con 0x7f755415be80 2024-09-13T08:51:47.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:47.369+0000 7f755a200640 1 -- 172.21.15.44:0/3658657635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7f75540b3400 con 0x7f755415be80 2024-09-13T08:51:48.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f754be00640 1 -- 172.21.15.44:0/3658657635 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v644) ==== 189+0+0 (secure 0 0 0) 0x7f75540d0350 con 0x7f755415be80 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 -- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7530074db0 msgr2=0x7f7530077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7530074db0 0x7f7530077270 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7554069730 tx=0x7f75440091e0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 -- 172.21.15.44:0/3658657635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755415be80 msgr2=0x7f7554071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755415be80 0x7f7554071330 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f754c00cb10 tx=0x7f754c00cc00 comp rx=0 tx=0).stop 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 -- 172.21.15.44:0/3658657635 shutdown_connections 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7530074db0 0x7f7530077270 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755415be80 0x7f7554071330 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f75540d28c0 0x7f75541f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 --2- 172.21.15.44:0/3658657635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75540b53f0 0x7f755406f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 -- 172.21.15.44:0/3658657635 >> 172.21.15.44:0/3658657635 conn(0x7f75540d1da0 msgr2=0x7f7554159260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.217+0000 7f755a200640 1 -- 172.21.15.44:0/3658657635 shutdown_connections 2024-09-13T08:51:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.221+0000 7f755a200640 1 -- 172.21.15.44:0/3658657635 wait complete. 2024-09-13T08:51:48.243 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-13T08:51:48.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.393+0000 7f0839e00640 1 -- 172.21.15.44:0/2152753285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08340b2d00 msgr2=0x7f083415fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.393+0000 7f0839e00640 1 --2- 172.21.15.44:0/2152753285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08340b2d00 0x7f083415fec0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f081c009900 tx=0x7f081c01c850 comp rx=0 tx=0).stop 2024-09-13T08:51:48.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.393+0000 7f0839e00640 1 -- 172.21.15.44:0/2152753285 shutdown_connections 2024-09-13T08:51:48.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.393+0000 7f0839e00640 1 --2- 172.21.15.44:0/2152753285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08340b2d00 0x7f083415fec0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.393+0000 7f0839e00640 1 --2- 172.21.15.44:0/2152753285 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08340cfe80 0x7f08340d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.393+0000 7f0839e00640 1 -- 172.21.15.44:0/2152753285 >> 172.21.15.44:0/2152753285 conn(0x7f08340d1e40 msgr2=0x7f08340d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:48.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.393+0000 7f0839e00640 1 -- 172.21.15.44:0/2152753285 shutdown_connections 2024-09-13T08:51:48.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.393+0000 7f0839e00640 1 -- 172.21.15.44:0/2152753285 wait complete. 2024-09-13T08:51:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0839e00640 1 Processor -- start 2024-09-13T08:51:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0839e00640 1 -- start start 2024-09-13T08:51:48.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0839e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08340b2d00 0x7f083414e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0839e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08340cfe80 0x7f083415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0839e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0834160400 0x7f083415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0839e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f08341695d0 con 0x7f0834160400 2024-09-13T08:51:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0839e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0834169450 con 0x7f08340cfe80 2024-09-13T08:51:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0839e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0834169750 con 0x7f08340b2d00 2024-09-13T08:51:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0833400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0834160400 0x7f083415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0832a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08340b2d00 0x7f083414e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0833400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0834160400 0x7f083415c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34996/0 (socket says 172.21.15.44:34996) 2024-09-13T08:51:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0833400640 1 -- 172.21.15.44:0/2688703701 learned_addr learned my addr 172.21.15.44:0/2688703701 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:48.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f082a000640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08340cfe80 0x7f083415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:48.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0833400640 1 -- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08340b2d00 msgr2=0x7f083414e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0833400640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08340b2d00 0x7f083414e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0833400640 1 -- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08340cfe80 msgr2=0x7f083415a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0833400640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08340cfe80 0x7f083415a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0833400640 1 -- 172.21.15.44:0/2688703701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08341fbd70 con 0x7f0834160400 2024-09-13T08:51:48.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0833400640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0834160400 0x7f083415c2a0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f082c00c740 tx=0x7f082c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:48.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f082be00640 1 -- 172.21.15.44:0/2688703701 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c003b80 con 0x7f0834160400 2024-09-13T08:51:48.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f082be00640 1 -- 172.21.15.44:0/2688703701 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f082c00ce50 con 0x7f0834160400 2024-09-13T08:51:48.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f0839e00640 1 -- 172.21.15.44:0/2688703701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083415acf0 con 0x7f0834160400 2024-09-13T08:51:48.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.397+0000 7f082be00640 1 -- 172.21.15.44:0/2688703701 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c00a850 con 0x7f0834160400 2024-09-13T08:51:48.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.401+0000 7f0839e00640 1 -- 172.21.15.44:0/2688703701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0834158c70 con 0x7f0834160400 2024-09-13T08:51:48.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.401+0000 7f0839e00640 1 -- 172.21.15.44:0/2688703701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083415fec0 con 0x7f0834160400 2024-09-13T08:51:48.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.401+0000 7f082be00640 1 -- 172.21.15.44:0/2688703701 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f082c00a9f0 con 0x7f0834160400 2024-09-13T08:51:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.401+0000 7f082be00640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0810074db0 0x7f0810077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.401+0000 7f082be00640 1 -- 172.21.15.44:0/2688703701 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5055+0+0 (secure 0 0 0) 0x7f082c092af0 con 0x7f0834160400 2024-09-13T08:51:48.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.401+0000 7f0832a00640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0810074db0 0x7f0810077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:48.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.405+0000 7f0832a00640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0810074db0 0x7f0810077270 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0824004630 tx=0x7f082400b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:48.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.405+0000 7f082be00640 1 -- 172.21.15.44:0/2688703701 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f08340d0c80 con 0x7f0834160400 2024-09-13T08:51:48.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 -- 172.21.15.44:0/2688703701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f083406dd30 con 0x7f0834160400 2024-09-13T08:51:48.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f082be00640 1 -- 172.21.15.44:0/2688703701 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v50) ==== 59+0+1998 (secure 0 0 0) 0x7f082c05f1f0 con 0x7f0834160400 2024-09-13T08:51:48.584 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-13T08:51:48.584 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-09-13T08:51:48.584 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-13T08:51:48.584 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4094.620852 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3964.014083 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3978.263337 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4002.298162 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4667.000623 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4664.046790 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6403.016405 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4696.908546 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 2/2 2024-09-13T08:51:48.585 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-09-13T08:51:48.586 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-09-13T08:51:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 -- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0810074db0 msgr2=0x7f0810077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0810074db0 0x7f0810077270 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0824004630 tx=0x7f082400b390 comp rx=0 tx=0).stop 2024-09-13T08:51:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 -- 172.21.15.44:0/2688703701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0834160400 msgr2=0x7f083415c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0834160400 0x7f083415c2a0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f082c00c740 tx=0x7f082c00cc00 comp rx=0 tx=0).stop 2024-09-13T08:51:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 -- 172.21.15.44:0/2688703701 shutdown_connections 2024-09-13T08:51:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0810074db0 0x7f0810077270 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0834160400 0x7f083415c2a0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08340cfe80 0x7f083415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 --2- 172.21.15.44:0/2688703701 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f08340b2d00 0x7f083414e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 -- 172.21.15.44:0/2688703701 >> 172.21.15.44:0/2688703701 conn(0x7f08340d1e40 msgr2=0x7f08340721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 -- 172.21.15.44:0/2688703701 shutdown_connections 2024-09-13T08:51:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.585+0000 7f0839e00640 1 -- 172.21.15.44:0/2688703701 wait complete. 2024-09-13T08:51:48.606 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:51:48.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 -- 172.21.15.44:0/3813640369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49c0b27a0 msgr2=0x7fd49c0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 --2- 172.21.15.44:0/3813640369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49c0b27a0 0x7fd49c0ce0e0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd488009840 tx=0x7fd48801c700 comp rx=0 tx=0).stop 2024-09-13T08:51:48.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 -- 172.21.15.44:0/3813640369 shutdown_connections 2024-09-13T08:51:48.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 --2- 172.21.15.44:0/3813640369 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd49c0ce620 0x7fd49c0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 --2- 172.21.15.44:0/3813640369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49c0b27a0 0x7fd49c0ce0e0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 -- 172.21.15.44:0/3813640369 >> 172.21.15.44:0/3813640369 conn(0x7fd49c0d1e30 msgr2=0x7fd49c0d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:48.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 -- 172.21.15.44:0/3813640369 shutdown_connections 2024-09-13T08:51:48.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 -- 172.21.15.44:0/3813640369 wait complete. 2024-09-13T08:51:48.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 Processor -- start 2024-09-13T08:51:48.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 -- start start 2024-09-13T08:51:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49c0b27a0 0x7fd49c165180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd49c0ce620 0x7fd49c158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd49c15f640 0x7fd49c1f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c0b1010 con 0x7fd49c0b27a0 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c0b0e90 con 0x7fd49c0ce620 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a4200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd49c169390 con 0x7fd49c15f640 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49c0b27a0 0x7fd49c165180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49c0b27a0 0x7fd49c165180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35014/0 (socket says 172.21.15.44:35014) 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a1800640 1 -- 172.21.15.44:0/473731628 learned_addr learned my addr 172.21.15.44:0/473731628 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a0e00640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd49c0ce620 0x7fd49c158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a2200640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd49c15f640 0x7fd49c1f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a1800640 1 -- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd49c15f640 msgr2=0x7fd49c1f8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a1800640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd49c15f640 0x7fd49c1f8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a1800640 1 -- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd49c0ce620 msgr2=0x7fd49c158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a1800640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd49c0ce620 0x7fd49c158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.757+0000 7fd4a1800640 1 -- 172.21.15.44:0/473731628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c1f90a0 con 0x7fd49c0b27a0 2024-09-13T08:51:48.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.761+0000 7fd4a1800640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49c0b27a0 0x7fd49c165180 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fd49000c3f0 tx=0x7fd49000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:48.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.761+0000 7fd499600640 1 -- 172.21.15.44:0/473731628 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490017070 con 0x7fd49c0b27a0 2024-09-13T08:51:48.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.761+0000 7fd499600640 1 -- 172.21.15.44:0/473731628 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd490003600 con 0x7fd49c0b27a0 2024-09-13T08:51:48.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.761+0000 7fd499600640 1 -- 172.21.15.44:0/473731628 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490003a50 con 0x7fd49c0b27a0 2024-09-13T08:51:48.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.761+0000 7fd4a4200640 1 -- 172.21.15.44:0/473731628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49c1640e0 con 0x7fd49c0b27a0 2024-09-13T08:51:48.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.761+0000 7fd4a4200640 1 -- 172.21.15.44:0/473731628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49c1646a0 con 0x7fd49c0b27a0 2024-09-13T08:51:48.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.761+0000 7fd4a4200640 1 -- 172.21.15.44:0/473731628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49c0ce0e0 con 0x7fd49c0b27a0 2024-09-13T08:51:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.761+0000 7fd499600640 1 -- 172.21.15.44:0/473731628 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fd49001a070 con 0x7fd49c0b27a0 2024-09-13T08:51:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.761+0000 7fd499600640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd47c074db0 0x7fd47c077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.761+0000 7fd499600640 1 -- 172.21.15.44:0/473731628 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5055+0+0 (secure 0 0 0) 0x7fd4900920d0 con 0x7fd49c0b27a0 2024-09-13T08:51:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.765+0000 7fd4a0e00640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd47c074db0 0x7fd47c077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.765+0000 7fd4a0e00640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd47c074db0 0x7fd47c077270 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd4880af000 tx=0x7fd48801af40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:48.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.765+0000 7fd499600640 1 -- 172.21.15.44:0/473731628 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd49c1f90a0 con 0x7fd49c0b27a0 2024-09-13T08:51:48.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.941+0000 7fd4a4200640 1 -- 172.21.15.44:0/473731628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd49c0cedf0 con 0x7fd49c0b27a0 2024-09-13T08:51:48.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.945+0000 7fd499600640 1 -- 172.21.15.44:0/473731628 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7fd49005e730 con 0x7fd49c0b27a0 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 -- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd47c074db0 msgr2=0x7fd47c077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd47c074db0 0x7fd47c077270 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd4880af000 tx=0x7fd48801af40 comp rx=0 tx=0).stop 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 -- 172.21.15.44:0/473731628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49c0b27a0 msgr2=0x7fd49c165180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49c0b27a0 0x7fd49c165180 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fd49000c3f0 tx=0x7fd49000c8b0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 -- 172.21.15.44:0/473731628 shutdown_connections 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd47c074db0 0x7fd47c077270 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd49c15f640 0x7fd49c1f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd49c0ce620 0x7fd49c158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 --2- 172.21.15.44:0/473731628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49c0b27a0 0x7fd49c165180 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 -- 172.21.15.44:0/473731628 >> 172.21.15.44:0/473731628 conn(0x7fd49c0d1e30 msgr2=0x7fd49c1661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:48.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 -- 172.21.15.44:0/473731628 shutdown_connections 2024-09-13T08:51:48.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:48.949+0000 7fd4a4200640 1 -- 172.21.15.44:0/473731628 wait complete. 2024-09-13T08:51:48.969 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:51:48.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:51:49.085 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:51:49.166 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:51:49.166 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:51:49.166 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:51:49.175 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:51:49.175 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:51:49.175 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:51:49.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:51:49.185 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:51:49.185 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:51:49.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:51:49.228 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:51:49.229 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:51:49.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:51:49.240 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T08:51:49.240 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:51:49.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 -- 172.21.15.44:0/3583994916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c415bea0 msgr2=0x7f91c40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:49.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 --2- 172.21.15.44:0/3583994916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c415bea0 0x7f91c40ce670 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f91c4073d10 tx=0x7f91b801c8c0 comp rx=0 tx=0).stop 2024-09-13T08:51:49.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 -- 172.21.15.44:0/3583994916 shutdown_connections 2024-09-13T08:51:49.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 --2- 172.21.15.44:0/3583994916 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91c40d28e0 0x7f91c40cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:49.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 --2- 172.21.15.44:0/3583994916 >> v2:172.21.15.129:3300/0 conn(0x7f91c40cdb30 0x7f91c40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:49.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 --2- 172.21.15.44:0/3583994916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c415bea0 0x7f91c40ce670 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:49.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 -- 172.21.15.44:0/3583994916 >> 172.21.15.44:0/3583994916 conn(0x7f91c40d1dc0 msgr2=0x7f91c40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:49.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 -- 172.21.15.44:0/3583994916 shutdown_connections 2024-09-13T08:51:49.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 -- 172.21.15.44:0/3583994916 wait complete. 2024-09-13T08:51:49.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 Processor -- start 2024-09-13T08:51:49.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.417+0000 7f91cc400640 1 -- start start 2024-09-13T08:51:49.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91cc400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91c40cdb30 0x7f91c414b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:49.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91cc400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c40d28e0 0x7f91c40c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:49.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91cc400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91c415bea0 0x7f91c40c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:49.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91cc400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f91c40af070 con 0x7f91c40d28e0 2024-09-13T08:51:49.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91cc400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f91c40aeef0 con 0x7f91c40cdb30 2024-09-13T08:51:49.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91cc400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f91c40af1f0 con 0x7f91c415bea0 2024-09-13T08:51:49.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c40d28e0 0x7f91c40c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:49.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c40d28e0 0x7f91c40c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35026/0 (socket says 172.21.15.44:35026) 2024-09-13T08:51:49.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c9000640 1 -- 172.21.15.44:0/4113421643 learned_addr learned my addr 172.21.15.44:0/4113421643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c9a00640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91c40cdb30 0x7f91c414b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91ca400640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91c415bea0 0x7f91c40c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c9000640 1 -- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91c415bea0 msgr2=0x7f91c40c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c9000640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91c415bea0 0x7f91c40c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c9000640 1 -- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91c40cdb30 msgr2=0x7f91c414b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c9000640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91c40cdb30 0x7f91c414b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:49.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c9000640 1 -- 172.21.15.44:0/4113421643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c40b6cd0 con 0x7f91c40d28e0 2024-09-13T08:51:49.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c9000640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c40d28e0 0x7f91c40c7fe0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f91b000c970 tx=0x7f91b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:49.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c1600640 1 -- 172.21.15.44:0/4113421643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0007c70 con 0x7f91c40d28e0 2024-09-13T08:51:49.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c1600640 1 -- 172.21.15.44:0/4113421643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91b000f450 con 0x7f91c40d28e0 2024-09-13T08:51:49.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91c1600640 1 -- 172.21.15.44:0/4113421643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b000f8a0 con 0x7f91c40d28e0 2024-09-13T08:51:49.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91cc400640 1 -- 172.21.15.44:0/4113421643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c41591e0 con 0x7f91c40d28e0 2024-09-13T08:51:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91cc400640 1 -- 172.21.15.44:0/4113421643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c40b3bc0 con 0x7f91c40d28e0 2024-09-13T08:51:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.421+0000 7f91cc400640 1 -- 172.21.15.44:0/4113421643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c40cc9d0 con 0x7f91c40d28e0 2024-09-13T08:51:49.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.425+0000 7f91c1600640 1 -- 172.21.15.44:0/4113421643 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f91b000fa40 con 0x7f91c40d28e0 2024-09-13T08:51:49.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.425+0000 7f91c1600640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f91a4074db0 0x7f91a4077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:49.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.425+0000 7f91c1600640 1 -- 172.21.15.44:0/4113421643 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5055+0+0 (secure 0 0 0) 0x7f91b00935c0 con 0x7f91c40d28e0 2024-09-13T08:51:49.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.425+0000 7f91c9a00640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f91a4074db0 0x7f91a4077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:49.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.425+0000 7f91c9a00640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f91a4074db0 0x7f91a4077270 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f91b80af000 tx=0x7f91b801b100 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:49.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.429+0000 7f91c1600640 1 -- 172.21.15.44:0/4113421643 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f91b0014030 con 0x7f91c40d28e0 2024-09-13T08:51:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.641+0000 7f91cc400640 1 -- 172.21.15.44:0/4113421643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91c40b4f30 con 0x7f91c40d28e0 2024-09-13T08:51:49.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91c1600640 1 -- 172.21.15.44:0/4113421643 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3917 (secure 0 0 0) 0x7f91b005fc20 con 0x7f91c40d28e0 2024-09-13T08:51:49.643 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:49.644 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":77,"btime":"2024-09-13T08:51:43:260642+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6854,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/3108776730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3108776730},{"type":"v1","addr":"172.21.15.129:6836","nonce":3108776730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1076017330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1076017330},{"type":"v1","addr":"172.21.15.129:6837","nonce":1076017330}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7151,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1660137748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1660137748},{"type":"v1","addr":"172.21.15.44:6834","nonce":1660137748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4098004813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4098004813},{"type":"v1","addr":"172.21.15.44:6835","nonce":4098004813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[]} 2024-09-13T08:51:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 77 2024-09-13T08:51:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 -- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f91a4074db0 msgr2=0x7f91a4077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f91a4074db0 0x7f91a4077270 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f91b80af000 tx=0x7f91b801b100 comp rx=0 tx=0).stop 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 -- 172.21.15.44:0/4113421643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c40d28e0 msgr2=0x7f91c40c7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c40d28e0 0x7f91c40c7fe0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f91b000c970 tx=0x7f91b000ce30 comp rx=0 tx=0).stop 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 -- 172.21.15.44:0/4113421643 shutdown_connections 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f91a4074db0 0x7f91a4077270 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f91c415bea0 0x7f91c40c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c40d28e0 0x7f91c40c7fe0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 --2- 172.21.15.44:0/4113421643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91c40cdb30 0x7f91c414b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 -- 172.21.15.44:0/4113421643 >> 172.21.15.44:0/4113421643 conn(0x7f91c40d1dc0 msgr2=0x7f91c40d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 -- 172.21.15.44:0/4113421643 shutdown_connections 2024-09-13T08:51:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:49.645+0000 7f91cc400640 1 -- 172.21.15.44:0/4113421643 wait complete. 2024-09-13T08:51:49.665 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T08:51:49.665 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T08:51:49.666 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T08:51:49.666 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:51:49.666 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T08:51:49.666 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T08:51:49.666 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T08:51:49.666 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:51:49.666 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T08:51:49.667 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T08:51:49.667 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T08:51:49.667 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:51:49.667 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T08:51:49.667 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T08:51:49.668 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T08:51:49.668 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:51:49.681 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T08:51:49.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T08:51:49.684 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T08:51:49.684 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T08:51:49.686 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T08:51:49.686 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T08:51:49.725 INFO:tasks.ceph.mds.c:Started 2024-09-13T08:51:49.726 DEBUG:teuthology.parallel:result is None 2024-09-13T08:51:49.729 INFO:tasks.ceph.mds.b:Started 2024-09-13T08:51:49.729 DEBUG:teuthology.parallel:result is None 2024-09-13T08:51:49.731 INFO:tasks.ceph.mds.d:Started 2024-09-13T08:51:49.732 DEBUG:teuthology.parallel:result is None 2024-09-13T08:51:49.810 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:51:49.822 INFO:tasks.ceph.mds.b.smithi129.stdout:starting mds.b at 2024-09-13T08:51:49.822 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:51:51.386 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-13T08:51:55.681 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T08:51:55.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T08:51:55.686 INFO:tasks.ceph.mds.a:Started 2024-09-13T08:51:55.686 DEBUG:teuthology.parallel:result is None 2024-09-13T08:51:55.686 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T08:51:55.780 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:51:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.849+0000 7f3e25400640 1 -- 172.21.15.44:0/3176644629 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3e2015e130 msgr2=0x7f3e2006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.849+0000 7f3e25400640 1 --2- 172.21.15.44:0/3176644629 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3e2015e130 0x7f3e2006e740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3e200cce40 tx=0x7f3e1001c760 comp rx=0 tx=0).stop 2024-09-13T08:51:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.849+0000 7f3e25400640 1 -- 172.21.15.44:0/3176644629 shutdown_connections 2024-09-13T08:51:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.849+0000 7f3e25400640 1 --2- 172.21.15.44:0/3176644629 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3e2015e130 0x7f3e2006e740 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.849+0000 7f3e25400640 1 -- 172.21.15.44:0/3176644629 >> 172.21.15.44:0/3176644629 conn(0x7f3e200d1ea0 msgr2=0x7f3e200d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.849+0000 7f3e25400640 1 -- 172.21.15.44:0/3176644629 shutdown_connections 2024-09-13T08:51:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.849+0000 7f3e25400640 1 -- 172.21.15.44:0/3176644629 wait complete. 2024-09-13T08:51:55.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.849+0000 7f3e25400640 1 Processor -- start 2024-09-13T08:51:55.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.849+0000 7f3e25400640 1 -- start start 2024-09-13T08:51:55.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e25400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3e200ce720 0x7f3e20156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:55.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e25400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e200cf250 0x7f3e201f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e25400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e2015e130 0x7f3e201f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e25400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2016c130 con 0x7f3e2015e130 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e25400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2016bfb0 con 0x7f3e200cf250 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e25400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2016c2b0 con 0x7f3e200ce720 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e1f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e2015e130 0x7f3e201f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e1e800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3e200ce720 0x7f3e20156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e1f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e2015e130 0x7f3e201f7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38318/0 (socket says 172.21.15.44:38318) 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e1f200640 1 -- 172.21.15.44:0/2325742508 learned_addr learned my addr 172.21.15.44:0/2325742508 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e17e00640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e200cf250 0x7f3e201f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e1f200640 1 -- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3e200ce720 msgr2=0x7f3e20156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e1f200640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3e200ce720 0x7f3e20156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e1f200640 1 -- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e200cf250 msgr2=0x7f3e201f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:55.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e1f200640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e200cf250 0x7f3e201f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:55.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e1f200640 1 -- 172.21.15.44:0/2325742508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2015a120 con 0x7f3e2015e130 2024-09-13T08:51:55.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e1f200640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e2015e130 0x7f3e201f7cd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800c770 tx=0x7f3e1800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:55.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e16a00640 1 -- 172.21.15.44:0/2325742508 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18003b80 con 0x7f3e2015e130 2024-09-13T08:51:55.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e16a00640 1 -- 172.21.15.44:0/2325742508 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e18003d40 con 0x7f3e2015e130 2024-09-13T08:51:55.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e16a00640 1 -- 172.21.15.44:0/2325742508 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1800aab0 con 0x7f3e2015e130 2024-09-13T08:51:55.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e25400640 1 -- 172.21.15.44:0/2325742508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2015a400 con 0x7f3e2015e130 2024-09-13T08:51:55.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e25400640 1 -- 172.21.15.44:0/2325742508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2015be20 con 0x7f3e2015e130 2024-09-13T08:51:55.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e25400640 1 -- 172.21.15.44:0/2325742508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2015c020 con 0x7f3e2015e130 2024-09-13T08:51:55.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.853+0000 7f3e16a00640 1 -- 172.21.15.44:0/2325742508 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f3e1800ac50 con 0x7f3e2015e130 2024-09-13T08:51:55.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.857+0000 7f3e16a00640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3dfc074db0 0x7f3dfc077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:55.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.857+0000 7f3e16a00640 1 -- 172.21.15.44:0/2325742508 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5055+0+0 (secure 0 0 0) 0x7f3e18092c70 con 0x7f3e2015e130 2024-09-13T08:51:55.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.857+0000 7f3e1e800640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3dfc074db0 0x7f3dfc077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:55.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.857+0000 7f3e1e800640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3dfc074db0 0x7f3dfc077270 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3e100af000 tx=0x7f3e1001afa0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:55.861+0000 7f3e16a00640 1 -- 172.21.15.44:0/2325742508 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f3e2015c020 con 0x7f3e2015e130 2024-09-13T08:51:56.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:56.045+0000 7f3e25400640 1 -- 172.21.15.44:0/2325742508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3e2006d0b0 con 0x7f3e2015e130 2024-09-13T08:51:57.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.253+0000 7f3e16a00640 1 -- 172.21.15.44:0/2325742508 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v127) ==== 97+0+0 (secure 0 0 0) 0x7f3e1805f920 con 0x7f3e2015e130 2024-09-13T08:51:57.254 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-09-13T08:51:57.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 -- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3dfc074db0 msgr2=0x7f3dfc077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:57.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3dfc074db0 0x7f3dfc077270 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3e100af000 tx=0x7f3e1001afa0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 -- 172.21.15.44:0/2325742508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e2015e130 msgr2=0x7f3e201f7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:57.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e2015e130 0x7f3e201f7cd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800c770 tx=0x7f3e1800cc30 comp rx=0 tx=0).stop 2024-09-13T08:51:57.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 -- 172.21.15.44:0/2325742508 shutdown_connections 2024-09-13T08:51:57.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3dfc074db0 0x7f3dfc077270 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e2015e130 0x7f3e201f7cd0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e200cf250 0x7f3e201f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 --2- 172.21.15.44:0/2325742508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f3e200ce720 0x7f3e20156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 -- 172.21.15.44:0/2325742508 >> 172.21.15.44:0/2325742508 conn(0x7f3e200d1ea0 msgr2=0x7f3e20071b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:57.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 -- 172.21.15.44:0/2325742508 shutdown_connections 2024-09-13T08:51:57.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.257+0000 7f3e25400640 1 -- 172.21.15.44:0/2325742508 wait complete. 2024-09-13T08:51:57.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T08:51:57.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.433+0000 7f7f6fe00640 1 -- 172.21.15.44:0/2833606055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f7006d330 msgr2=0x7f7f700ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:57.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.433+0000 7f7f6fe00640 1 --2- 172.21.15.44:0/2833606055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f7006d330 0x7f7f700ce010 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009bc0 tx=0x7f7f6801cb70 comp rx=0 tx=0).stop 2024-09-13T08:51:57.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.433+0000 7f7f6fe00640 1 -- 172.21.15.44:0/2833606055 shutdown_connections 2024-09-13T08:51:57.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.433+0000 7f7f6fe00640 1 --2- 172.21.15.44:0/2833606055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f7006d330 0x7f7f700ce010 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.433+0000 7f7f6fe00640 1 -- 172.21.15.44:0/2833606055 >> 172.21.15.44:0/2833606055 conn(0x7f7f7006dd30 msgr2=0x7f7f7006e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:57.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.433+0000 7f7f6fe00640 1 -- 172.21.15.44:0/2833606055 shutdown_connections 2024-09-13T08:51:57.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.433+0000 7f7f6fe00640 1 -- 172.21.15.44:0/2833606055 wait complete. 2024-09-13T08:51:57.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6fe00640 1 Processor -- start 2024-09-13T08:51:57.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6fe00640 1 -- start start 2024-09-13T08:51:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f7006d330 0x7f7f700c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f7015b890 0x7f7f700d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f7015bc70 0x7f7f701570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70169b40 con 0x7f7f7015b890 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701699c0 con 0x7f7f7015bc70 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6fe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7f70169cc0 con 0x7f7f7006d330 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f7015b890 0x7f7f700d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f7015b890 0x7f7f700d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38332/0 (socket says 172.21.15.44:38332) 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67e00640 1 -- 172.21.15.44:0/203599016 learned_addr learned my addr 172.21.15.44:0/203599016 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6ea00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f7006d330 0x7f7f700c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6f400640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f7015bc70 0x7f7f701570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67e00640 1 -- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f7006d330 msgr2=0x7f7f700c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67e00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f7006d330 0x7f7f700c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67e00640 1 -- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f7015bc70 msgr2=0x7f7f701570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67e00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f7015bc70 0x7f7f701570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67e00640 1 -- 172.21.15.44:0/203599016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f700c6c40 con 0x7f7f7015b890 2024-09-13T08:51:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67e00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f7015b890 0x7f7f700d00a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800c410 tx=0x7f7f5800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67400640 1 -- 172.21.15.44:0/203599016 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58017070 con 0x7f7f7015b890 2024-09-13T08:51:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67400640 1 -- 172.21.15.44:0/203599016 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f580053c0 con 0x7f7f7015b890 2024-09-13T08:51:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6fe00640 1 -- 172.21.15.44:0/203599016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f700b4ae0 con 0x7f7f7015b890 2024-09-13T08:51:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f67400640 1 -- 172.21.15.44:0/203599016 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f580038b0 con 0x7f7f7015b890 2024-09-13T08:51:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.437+0000 7f7f6fe00640 1 -- 172.21.15.44:0/203599016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f700b50a0 con 0x7f7f7015b890 2024-09-13T08:51:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.441+0000 7f7f6fe00640 1 -- 172.21.15.44:0/203599016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f700ce010 con 0x7f7f7015b890 2024-09-13T08:51:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.441+0000 7f7f67400640 1 -- 172.21.15.44:0/203599016 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f7f5801a070 con 0x7f7f7015b890 2024-09-13T08:51:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.441+0000 7f7f67400640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f4c074db0 0x7f7f4c077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.441+0000 7f7f67400640 1 -- 172.21.15.44:0/203599016 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 4969+0+0 (secure 0 0 0) 0x7f7f580922f0 con 0x7f7f7015b890 2024-09-13T08:51:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.441+0000 7f7f6ea00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f4c074db0 0x7f7f4c077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.441+0000 7f7f6ea00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f4c074db0 0x7f7f4c077270 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7f60007fd0 tx=0x7f7f60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:57.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.445+0000 7f7f67400640 1 -- 172.21.15.44:0/203599016 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7f5805ef20 con 0x7f7f7015b890 2024-09-13T08:51:57.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 -- 172.21.15.44:0/203599016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7f7006d0f0 con 0x7f7f7015b890 2024-09-13T08:51:57.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f67400640 1 -- 172.21.15.44:0/203599016 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7f7f5805e820 con 0x7f7f7015b890 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw==", 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ==", 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-09-13T08:51:57.686 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA==", 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw==", 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:51:57.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg==", 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg==", 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw==", 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:51:57.689 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig==", 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.690 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-09-13T08:51:57.691 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ==", 2024-09-13T08:51:57.691 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.691 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:51:57.691 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:51:57.691 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.691 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-09-13T08:51:57.691 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g==", 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg==", 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:51:57.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg==", 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-09-13T08:51:57.693 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA==", 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w==", 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:51:57.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA==", 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.695 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w==", 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmDD2LJxAA3s0jEb2EfPsLcu0w145G3Q==", 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:51:57.696 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ==", 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:51:57.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:57.698 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-09-13T08:51:57.698 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg==", 2024-09-13T08:51:57.698 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:51:57.698 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:51:57.698 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:51:57.698 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:51:57.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:57.698 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T08:51:57.698 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:51:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 -- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f4c074db0 msgr2=0x7f7f4c077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f4c074db0 0x7f7f4c077270 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7f60007fd0 tx=0x7f7f60009040 comp rx=0 tx=0).stop 2024-09-13T08:51:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 -- 172.21.15.44:0/203599016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f7015b890 msgr2=0x7f7f700d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f7015b890 0x7f7f700d00a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800c410 tx=0x7f7f5800c8d0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 -- 172.21.15.44:0/203599016 shutdown_connections 2024-09-13T08:51:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7f4c074db0 0x7f7f4c077270 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f7015bc70 0x7f7f701570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f7015b890 0x7f7f700d00a0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 --2- 172.21.15.44:0/203599016 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7f7006d330 0x7f7f700c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 -- 172.21.15.44:0/203599016 >> 172.21.15.44:0/203599016 conn(0x7f7f7006dd30 msgr2=0x7f7f700b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 -- 172.21.15.44:0/203599016 shutdown_connections 2024-09-13T08:51:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.685+0000 7f7f6fe00640 1 -- 172.21.15.44:0/203599016 wait complete. 2024-09-13T08:51:57.706 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T08:51:57.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 -- 172.21.15.44:0/1998035674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0181585b0 msgr2=0x7fd0180cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 --2- 172.21.15.44:0/1998035674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0181585b0 0x7fd0180cbb60 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd008009900 tx=0x7fd00801f0d0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 -- 172.21.15.44:0/1998035674 shutdown_connections 2024-09-13T08:51:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 --2- 172.21.15.44:0/1998035674 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd01815eed0 0x7fd0180ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 --2- 172.21.15.44:0/1998035674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0181585b0 0x7fd0180cbb60 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 -- 172.21.15.44:0/1998035674 >> 172.21.15.44:0/1998035674 conn(0x7fd0180d0c90 msgr2=0x7fd0180d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 -- 172.21.15.44:0/1998035674 shutdown_connections 2024-09-13T08:51:57.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 -- 172.21.15.44:0/1998035674 wait complete. 2024-09-13T08:51:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 Processor -- start 2024-09-13T08:51:57.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 -- start start 2024-09-13T08:51:57.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd018074490 0x7fd018156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:57.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0181585b0 0x7fd01806ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:57.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd01815eed0 0x7fd0180b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:57.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0180cf130 con 0x7fd01815eed0 2024-09-13T08:51:57.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd0180cefb0 con 0x7fd0181585b0 2024-09-13T08:51:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01f000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd0180cf2b0 con 0x7fd018074490 2024-09-13T08:51:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd01815eed0 0x7fd0180b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd01815eed0 0x7fd0180b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38354/0 (socket says 172.21.15.44:38354) 2024-09-13T08:51:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01d000640 1 -- 172.21.15.44:0/3599844974 learned_addr learned my addr 172.21.15.44:0/3599844974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd00fe00640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0181585b0 0x7fd01806ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd017e00640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd018074490 0x7fd018156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:57.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01d000640 1 -- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd018074490 msgr2=0x7fd018156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:57.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01d000640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd018074490 0x7fd018156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01d000640 1 -- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0181585b0 msgr2=0x7fd01806ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:57.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01d000640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0181585b0 0x7fd01806ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:57.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.857+0000 7fd01d000640 1 -- 172.21.15.44:0/3599844974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd018157780 con 0x7fd01815eed0 2024-09-13T08:51:57.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.861+0000 7fd01d000640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd01815eed0 0x7fd0180b5da0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd01000e940 tx=0x7fd01000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.861+0000 7fd014c00640 1 -- 172.21.15.44:0/3599844974 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010004d30 con 0x7fd01815eed0 2024-09-13T08:51:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.861+0000 7fd014c00640 1 -- 172.21.15.44:0/3599844974 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd01000cb20 con 0x7fd01815eed0 2024-09-13T08:51:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.861+0000 7fd01f000640 1 -- 172.21.15.44:0/3599844974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd018157a60 con 0x7fd01815eed0 2024-09-13T08:51:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.861+0000 7fd014c00640 1 -- 172.21.15.44:0/3599844974 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010005020 con 0x7fd01815eed0 2024-09-13T08:51:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.861+0000 7fd01f000640 1 -- 172.21.15.44:0/3599844974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0181f5ba0 con 0x7fd01815eed0 2024-09-13T08:51:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.861+0000 7fd01f000640 1 -- 172.21.15.44:0/3599844974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0180cf760 con 0x7fd01815eed0 2024-09-13T08:51:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.861+0000 7fd014c00640 1 -- 172.21.15.44:0/3599844974 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fd0100051c0 con 0x7fd01815eed0 2024-09-13T08:51:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.861+0000 7fd014c00640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcff4074db0 0x7fcff4077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.861+0000 7fd014c00640 1 -- 172.21.15.44:0/3599844974 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 4969+0+0 (secure 0 0 0) 0x7fd0100936f0 con 0x7fd01815eed0 2024-09-13T08:51:57.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.865+0000 7fd017e00640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcff4074db0 0x7fcff4077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:57.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.865+0000 7fd017e00640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcff4074db0 0x7fcff4077270 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd0080b2000 tx=0x7fd0080aa040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:57.865+0000 7fd014c00640 1 -- 172.21.15.44:0/3599844974 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd010016030 con 0x7fd01815eed0 2024-09-13T08:51:58.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 -- 172.21.15.44:0/3599844974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd0180d1960 con 0x7fd01815eed0 2024-09-13T08:51:58.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd014c00640 1 -- 172.21.15.44:0/3599844974 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v127) ==== 84+0+65 (secure 0 0 0) 0x7fd01005fd50 con 0x7fd01815eed0 2024-09-13T08:51:58.044 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:51:58.044 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:51:58.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:51:58.044 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T08:51:58.044 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T08:51:58.044 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:51:58.044 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:51:58.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 -- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcff4074db0 msgr2=0x7fcff4077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:58.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcff4074db0 0x7fcff4077270 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd0080b2000 tx=0x7fd0080aa040 comp rx=0 tx=0).stop 2024-09-13T08:51:58.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 -- 172.21.15.44:0/3599844974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd01815eed0 msgr2=0x7fd0180b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:58.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd01815eed0 0x7fd0180b5da0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd01000e940 tx=0x7fd01000ee00 comp rx=0 tx=0).stop 2024-09-13T08:51:58.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 -- 172.21.15.44:0/3599844974 shutdown_connections 2024-09-13T08:51:58.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcff4074db0 0x7fcff4077270 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:58.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd01815eed0 0x7fd0180b5da0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:58.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0181585b0 0x7fd01806ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:58.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 --2- 172.21.15.44:0/3599844974 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd018074490 0x7fd018156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:58.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 -- 172.21.15.44:0/3599844974 >> 172.21.15.44:0/3599844974 conn(0x7fd0180d0c90 msgr2=0x7fd018071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:58.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 -- 172.21.15.44:0/3599844974 shutdown_connections 2024-09-13T08:51:58.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.045+0000 7fd01f000640 1 -- 172.21.15.44:0/3599844974 wait complete. 2024-09-13T08:51:58.066 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T08:51:58.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T08:51:58.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-13T08:51:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.217+0000 7f5f33a00640 1 -- 172.21.15.44:0/3214067646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f10004d50 msgr2=0x7f5f100051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.217+0000 7f5f33a00640 1 --2- 172.21.15.44:0/3214067646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f10004d50 0x7f5f100051b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c640 tx=0x7f5f2401d260 comp rx=0 tx=0).stop 2024-09-13T08:51:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.217+0000 7f5f31a00640 1 -- 172.21.15.44:0/3214067646 reap_dead start 2024-09-13T08:51:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.217+0000 7f5f33a00640 1 -- 172.21.15.44:0/3214067646 shutdown_connections 2024-09-13T08:51:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.217+0000 7f5f33a00640 1 -- 172.21.15.44:0/3214067646 >> 172.21.15.44:0/3214067646 conn(0x7f5f2c0cfbc0 msgr2=0x7f5f2c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.217+0000 7f5f33a00640 1 -- 172.21.15.44:0/3214067646 shutdown_connections 2024-09-13T08:51:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.217+0000 7f5f33a00640 1 -- 172.21.15.44:0/3214067646 wait complete. 2024-09-13T08:51:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f33a00640 1 Processor -- start 2024-09-13T08:51:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f33a00640 1 -- start start 2024-09-13T08:51:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f33a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f10004d50 0x7f5f2c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f33a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f2c1f4c40 0x7f5f2c1f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f33a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c0c98f0 0x7f5f2c0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f33a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c15f480 con 0x7f5f10004d50 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f33a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c15f300 con 0x7f5f2c0c98f0 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f33a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2c15f600 con 0x7f5f2c1f4c40 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f10004d50 0x7f5f2c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f10004d50 0x7f5f2c09aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38370/0 (socket says 172.21.15.44:38370) 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31000640 1 -- 172.21.15.44:0/2030759393 learned_addr learned my addr 172.21.15.44:0/2030759393 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31a00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c0c98f0 0x7f5f2c0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f23e00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f2c1f4c40 0x7f5f2c1f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31000640 1 -- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f2c1f4c40 msgr2=0x7f5f2c1f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31000640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f2c1f4c40 0x7f5f2c1f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31000640 1 -- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c0c98f0 msgr2=0x7f5f2c0c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31000640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c0c98f0 0x7f5f2c0c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:58.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31000640 1 -- 172.21.15.44:0/2030759393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c09c3e0 con 0x7f5f10004d50 2024-09-13T08:51:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31a00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c0c98f0 0x7f5f2c0c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:51:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f23e00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f2c1f4c40 0x7f5f2c1f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:51:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f31000640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f10004d50 0x7f5f2c09aa40 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c005f00 tx=0x7f5f1c011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f29600640 1 -- 172.21.15.44:0/2030759393 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c005320 con 0x7f5f10004d50 2024-09-13T08:51:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f29600640 1 -- 172.21.15.44:0/2030759393 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f1c018020 con 0x7f5f10004d50 2024-09-13T08:51:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f29600640 1 -- 172.21.15.44:0/2030759393 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c003860 con 0x7f5f10004d50 2024-09-13T08:51:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f33a00640 1 -- 172.21.15.44:0/2030759393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c09b340 con 0x7f5f10004d50 2024-09-13T08:51:58.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.221+0000 7f5f33a00640 1 -- 172.21.15.44:0/2030759393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c09b900 con 0x7f5f10004d50 2024-09-13T08:51:58.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.225+0000 7f5f33a00640 1 -- 172.21.15.44:0/2030759393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2c069410 con 0x7f5f10004d50 2024-09-13T08:51:58.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.225+0000 7f5f29600640 1 -- 172.21.15.44:0/2030759393 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f5f1c003a00 con 0x7f5f10004d50 2024-09-13T08:51:58.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.225+0000 7f5f29600640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f08074db0 0x7f5f08077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:58.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.225+0000 7f5f29600640 1 -- 172.21.15.44:0/2030759393 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 4969+0+0 (secure 0 0 0) 0x7f5f1c096630 con 0x7f5f10004d50 2024-09-13T08:51:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.225+0000 7f5f23e00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f08074db0 0x7f5f08077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.229+0000 7f5f23e00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f08074db0 0x7f5f08077270 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c1f5080 tx=0x7f5f140091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:58.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.229+0000 7f5f29600640 1 -- 172.21.15.44:0/2030759393 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5f1c022020 con 0x7f5f10004d50 2024-09-13T08:51:58.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:58.405+0000 7f5f33a00640 1 -- 172.21.15.44:0/2030759393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5f2c0d0850 con 0x7f5f10004d50 2024-09-13T08:51:59.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.273+0000 7f5f29600640 1 -- 172.21.15.44:0/2030759393 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v128) ==== 138+0+0 (secure 0 0 0) 0x7f5f1c063380 con 0x7f5f10004d50 2024-09-13T08:51:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T08:51:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 -- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f08074db0 msgr2=0x7f5f08077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f08074db0 0x7f5f08077270 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c1f5080 tx=0x7f5f140091e0 comp rx=0 tx=0).stop 2024-09-13T08:51:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 -- 172.21.15.44:0/2030759393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f10004d50 msgr2=0x7f5f2c09aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f10004d50 0x7f5f2c09aa40 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c005f00 tx=0x7f5f1c011740 comp rx=0 tx=0).stop 2024-09-13T08:51:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 -- 172.21.15.44:0/2030759393 shutdown_connections 2024-09-13T08:51:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f08074db0 0x7f5f08077270 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c0c98f0 0x7f5f2c0c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f2c1f4c40 0x7f5f2c1f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 --2- 172.21.15.44:0/2030759393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f10004d50 0x7f5f2c09aa40 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 -- 172.21.15.44:0/2030759393 >> 172.21.15.44:0/2030759393 conn(0x7f5f2c0cfbc0 msgr2=0x7f5f2c06e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.277+0000 7f5f33a00640 1 -- 172.21.15.44:0/2030759393 shutdown_connections 2024-09-13T08:51:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.281+0000 7f5f33a00640 1 -- 172.21.15.44:0/2030759393 wait complete. 2024-09-13T08:51:59.306 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T08:51:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.465+0000 7f0e81400640 1 -- 172.21.15.44:0/345797336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e7c1605d0 msgr2=0x7f0e7c0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.465+0000 7f0e81400640 1 --2- 172.21.15.44:0/345797336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e7c1605d0 0x7f0e7c0d0690 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0754a0 tx=0x7f0e6c01c800 comp rx=0 tx=0).stop 2024-09-13T08:51:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.465+0000 7f0e81400640 1 -- 172.21.15.44:0/345797336 shutdown_connections 2024-09-13T08:51:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.465+0000 7f0e81400640 1 --2- 172.21.15.44:0/345797336 >> v2:172.21.15.129:3300/0 conn(0x7f0e7c0afb10 0x7f0e7c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.465+0000 7f0e81400640 1 --2- 172.21.15.44:0/345797336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e7c1605d0 0x7f0e7c0d0690 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.465+0000 7f0e81400640 1 -- 172.21.15.44:0/345797336 >> 172.21.15.44:0/345797336 conn(0x7f0e7c0d1dd0 msgr2=0x7f0e7c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:51:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.465+0000 7f0e81400640 1 -- 172.21.15.44:0/345797336 shutdown_connections 2024-09-13T08:51:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.465+0000 7f0e81400640 1 -- 172.21.15.44:0/345797336 wait complete. 2024-09-13T08:51:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.465+0000 7f0e81400640 1 Processor -- start 2024-09-13T08:51:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.465+0000 7f0e81400640 1 -- start start 2024-09-13T08:51:59.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e81400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e7c0afb10 0x7f0e7c159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:59.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e81400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e7c0b1650 0x7f0e7c1f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:59.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e81400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0e7c1605d0 0x7f0e7c15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:59.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e81400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c0b0ec0 con 0x7f0e7c0afb10 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e81400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c0b1430 con 0x7f0e7c0b1650 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e81400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0e7c169540 con 0x7f0e7c1605d0 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e7c0afb10 0x7f0e7c159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e7c0afb10 0x7f0e7c159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38394/0 (socket says 172.21.15.44:38394) 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7a800640 1 -- 172.21.15.44:0/3563953069 learned_addr learned my addr 172.21.15.44:0/3563953069 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7b200640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0e7c1605d0 0x7f0e7c15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e73e00640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e7c0b1650 0x7f0e7c1f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7a800640 1 -- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0e7c1605d0 msgr2=0x7f0e7c15caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7a800640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0e7c1605d0 0x7f0e7c15caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7a800640 1 -- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e7c0b1650 msgr2=0x7f0e7c1f86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7a800640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e7c0b1650 0x7f0e7c1f86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7a800640 1 -- 172.21.15.44:0/3563953069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c1f8c20 con 0x7f0e7c0afb10 2024-09-13T08:51:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7b200640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0e7c1605d0 0x7f0e7c15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e73e00640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e7c0b1650 0x7f0e7c1f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:51:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e7a800640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e7c0afb10 0x7f0e7c159d00 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0b0040 tx=0x7f0e6c0a64d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e72a00640 1 -- 172.21.15.44:0/3563953069 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0a66c0 con 0x7f0e7c0afb10 2024-09-13T08:51:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e72a00640 1 -- 172.21.15.44:0/3563953069 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e6c004130 con 0x7f0e7c0afb10 2024-09-13T08:51:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e81400640 1 -- 172.21.15.44:0/3563953069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c15b2d0 con 0x7f0e7c0afb10 2024-09-13T08:51:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e72a00640 1 -- 172.21.15.44:0/3563953069 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c004300 con 0x7f0e7c0afb10 2024-09-13T08:51:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e81400640 1 -- 172.21.15.44:0/3563953069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c0c6a50 con 0x7f0e7c0afb10 2024-09-13T08:51:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.469+0000 7f0e81400640 1 -- 172.21.15.44:0/3563953069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e7c0affc0 con 0x7f0e7c0afb10 2024-09-13T08:51:59.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.473+0000 7f0e72a00640 1 -- 172.21.15.44:0/3563953069 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f0e6c0c1030 con 0x7f0e7c0afb10 2024-09-13T08:51:59.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.473+0000 7f0e72a00640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0e58074f80 0x7f0e58077440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:51:59.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.473+0000 7f0e72a00640 1 -- 172.21.15.44:0/3563953069 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5339+0+0 (secure 0 0 0) 0x7f0e6c12dc70 con 0x7f0e7c0afb10 2024-09-13T08:51:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.473+0000 7f0e73e00640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0e58074f80 0x7f0e58077440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:51:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.473+0000 7f0e73e00640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0e58074f80 0x7f0e58077440 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0e64007a60 tx=0x7f0e64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:51:59.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.477+0000 7f0e72a00640 1 -- 172.21.15.44:0/3563953069 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0e6c0c8060 con 0x7f0e7c0afb10 2024-09-13T08:51:59.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:51:59.657+0000 7f0e81400640 1 -- 172.21.15.44:0/3563953069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0e7c06fd00 con 0x7f0e7c0afb10 2024-09-13T08:52:00.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.281+0000 7f0e72a00640 1 -- 172.21.15.44:0/3563953069 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v129) ==== 170+0+0 (secure 0 0 0) 0x7f0e6c0163f0 con 0x7f0e7c0afb10 2024-09-13T08:52:00.282 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 -- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0e58074f80 msgr2=0x7f0e58077440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0e58074f80 0x7f0e58077440 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0e64007a60 tx=0x7f0e64009040 comp rx=0 tx=0).stop 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 -- 172.21.15.44:0/3563953069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e7c0afb10 msgr2=0x7f0e7c159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e7c0afb10 0x7f0e7c159d00 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0b0040 tx=0x7f0e6c0a64d0 comp rx=0 tx=0).stop 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 -- 172.21.15.44:0/3563953069 shutdown_connections 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0e58074f80 0x7f0e58077440 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0e7c1605d0 0x7f0e7c15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e7c0b1650 0x7f0e7c1f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 --2- 172.21.15.44:0/3563953069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e7c0afb10 0x7f0e7c159d00 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 -- 172.21.15.44:0/3563953069 >> 172.21.15.44:0/3563953069 conn(0x7f0e7c0d1dd0 msgr2=0x7f0e7c071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 -- 172.21.15.44:0/3563953069 shutdown_connections 2024-09-13T08:52:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.285+0000 7f0e81400640 1 -- 172.21.15.44:0/3563953069 wait complete. 2024-09-13T08:52:00.321 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T08:52:00.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.473+0000 7fed0a200640 1 -- 172.21.15.44:0/1636164346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fece8003fe0 msgr2=0x7fece8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:00.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.473+0000 7fed0a200640 1 --2- 172.21.15.44:0/1636164346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fece8003fe0 0x7fece8004810 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fecec009b90 tx=0x7fecec01d270 comp rx=0 tx=0).stop 2024-09-13T08:52:00.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.473+0000 7fed03e00640 1 -- 172.21.15.44:0/1636164346 reap_dead start 2024-09-13T08:52:00.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.473+0000 7fed0a200640 1 -- 172.21.15.44:0/1636164346 shutdown_connections 2024-09-13T08:52:00.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.473+0000 7fed0a200640 1 -- 172.21.15.44:0/1636164346 >> 172.21.15.44:0/1636164346 conn(0x7fed040d1dd0 msgr2=0x7fed040d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:00.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.473+0000 7fed0a200640 1 -- 172.21.15.44:0/1636164346 shutdown_connections 2024-09-13T08:52:00.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.473+0000 7fed0a200640 1 -- 172.21.15.44:0/1636164346 wait complete. 2024-09-13T08:52:00.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.473+0000 7fed0a200640 1 Processor -- start 2024-09-13T08:52:00.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.473+0000 7fed0a200640 1 -- start start 2024-09-13T08:52:00.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed0a200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8003fe0 0x7fed04158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:00.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed0a200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed041f49f0 0x7fed041f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:00.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed0a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415ec70 0x7fed0415f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:00.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed0a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed040b06d0 con 0x7fed0415ec70 2024-09-13T08:52:00.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed0a200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed040b2b90 con 0x7fece8003fe0 2024-09-13T08:52:00.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed0a200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fed040b0cd0 con 0x7fed041f49f0 2024-09-13T08:52:00.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415ec70 0x7fed0415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:00.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415ec70 0x7fed0415f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38412/0 (socket says 172.21.15.44:38412) 2024-09-13T08:52:00.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03e00640 1 -- 172.21.15.44:0/1050712193 learned_addr learned my addr 172.21.15.44:0/1050712193 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:00.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03400640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8003fe0 0x7fed04158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:00.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fecfbe00640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed041f49f0 0x7fed041f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:00.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03e00640 1 -- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed041f49f0 msgr2=0x7fed041f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:00.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03e00640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed041f49f0 0x7fed041f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:00.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03e00640 1 -- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8003fe0 msgr2=0x7fed04158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:00.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03e00640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8003fe0 0x7fed04158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:00.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03e00640 1 -- 172.21.15.44:0/1050712193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed041f5490 con 0x7fed0415ec70 2024-09-13T08:52:00.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03400640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8003fe0 0x7fed04158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:52:00.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fecfbe00640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed041f49f0 0x7fed041f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:52:00.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed03e00640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415ec70 0x7fed0415f120 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fed040b1190 tx=0x7fecfc012aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:00.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fecfb400640 1 -- 172.21.15.44:0/1050712193 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc013920 con 0x7fed0415ec70 2024-09-13T08:52:00.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fecfb400640 1 -- 172.21.15.44:0/1050712193 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecfc013ac0 con 0x7fed0415ec70 2024-09-13T08:52:00.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fecfb400640 1 -- 172.21.15.44:0/1050712193 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc010480 con 0x7fed0415ec70 2024-09-13T08:52:00.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed0a200640 1 -- 172.21.15.44:0/1050712193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed041f5770 con 0x7fed0415ec70 2024-09-13T08:52:00.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed0a200640 1 -- 172.21.15.44:0/1050712193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041f6030 con 0x7fed0415ec70 2024-09-13T08:52:00.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.477+0000 7fed0a200640 1 -- 172.21.15.44:0/1050712193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0406adb0 con 0x7fed0415ec70 2024-09-13T08:52:00.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.481+0000 7fecfb400640 1 -- 172.21.15.44:0/1050712193 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fecfc010620 con 0x7fed0415ec70 2024-09-13T08:52:00.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.481+0000 7fecfb400640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0074db0 0x7fece0077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:00.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.481+0000 7fecfb400640 1 -- 172.21.15.44:0/1050712193 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5721+0+0 (secure 0 0 0) 0x7fecfc097170 con 0x7fed0415ec70 2024-09-13T08:52:00.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.481+0000 7fed03400640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0074db0 0x7fece0077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:00.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.481+0000 7fed03400640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0074db0 0x7fece0077270 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fed040b02f0 tx=0x7fecf40091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:00.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.485+0000 7fecfb400640 1 -- 172.21.15.44:0/1050712193 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fecfc063ea0 con 0x7fed0415ec70 2024-09-13T08:52:00.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:00.693+0000 7fed0a200640 1 -- 172.21.15.44:0/1050712193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fed040b1220 con 0x7fed0415ec70 2024-09-13T08:52:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.285+0000 7fecfb400640 1 -- 172.21.15.44:0/1050712193 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v81) ==== 174+0+0 (secure 0 0 0) 0x7fecfc0637a0 con 0x7fed0415ec70 2024-09-13T08:52:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 18 and data pool 19 2024-09-13T08:52:01.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 -- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0074db0 msgr2=0x7fece0077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:01.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0074db0 0x7fece0077270 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fed040b02f0 tx=0x7fecf40091e0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 -- 172.21.15.44:0/1050712193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415ec70 msgr2=0x7fed0415f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:01.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415ec70 0x7fed0415f120 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fed040b1190 tx=0x7fecfc012aa0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 -- 172.21.15.44:0/1050712193 shutdown_connections 2024-09-13T08:52:01.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fece0074db0 0x7fece0077270 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415ec70 0x7fed0415f120 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed041f49f0 0x7fed041f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 --2- 172.21.15.44:0/1050712193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fece8003fe0 0x7fed04158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 -- 172.21.15.44:0/1050712193 >> 172.21.15.44:0/1050712193 conn(0x7fed040d1dd0 msgr2=0x7fed0406f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 -- 172.21.15.44:0/1050712193 shutdown_connections 2024-09-13T08:52:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.289+0000 7fed0a200640 1 -- 172.21.15.44:0/1050712193 wait complete. 2024-09-13T08:52:01.342 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:52:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.489+0000 7f9e8b400640 1 -- 172.21.15.44:0/3204690949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e68006400 msgr2=0x7f9e6800cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.489+0000 7f9e8b400640 1 --2- 172.21.15.44:0/3204690949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e68006400 0x7f9e6800cae0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c630 tx=0x7f9e7c01fa20 comp rx=0 tx=0).stop 2024-09-13T08:52:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.489+0000 7f9e8b400640 1 -- 172.21.15.44:0/3204690949 shutdown_connections 2024-09-13T08:52:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.489+0000 7f9e8b400640 1 --2- 172.21.15.44:0/3204690949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e68006400 0x7f9e6800cae0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.489+0000 7f9e8b400640 1 -- 172.21.15.44:0/3204690949 >> 172.21.15.44:0/3204690949 conn(0x7f9e840b18f0 msgr2=0x7f9e840b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.489+0000 7f9e8b400640 1 -- 172.21.15.44:0/3204690949 shutdown_connections 2024-09-13T08:52:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.489+0000 7f9e8b400640 1 -- 172.21.15.44:0/3204690949 wait complete. 2024-09-13T08:52:01.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 Processor -- start 2024-09-13T08:52:01.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 -- start start 2024-09-13T08:52:01.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e68006400 0x7f9e84148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:01.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e8409d620 0x7f9e841f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:01.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e8409da80 0x7f9e841f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:01.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e840ad0d0 con 0x7f9e8409da80 2024-09-13T08:52:01.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9e840acf50 con 0x7f9e68006400 2024-09-13T08:52:01.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9e840b4340 con 0x7f9e8409d620 2024-09-13T08:52:01.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e89400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e8409da80 0x7f9e841f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:01.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e89400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e8409da80 0x7f9e841f7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38436/0 (socket says 172.21.15.44:38436) 2024-09-13T08:52:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e89400640 1 -- 172.21.15.44:0/3743049511 learned_addr learned my addr 172.21.15.44:0/3743049511 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e7be00640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e8409d620 0x7f9e841f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e88a00640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e68006400 0x7f9e84148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e89400640 1 -- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e8409d620 msgr2=0x7f9e841f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e89400640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e8409d620 0x7f9e841f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e89400640 1 -- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e68006400 msgr2=0x7f9e84148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e89400640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e68006400 0x7f9e84148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e89400640 1 -- 172.21.15.44:0/3743049511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8414c4e0 con 0x7f9e8409da80 2024-09-13T08:52:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e88a00640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e68006400 0x7f9e84148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e89400640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e8409da80 0x7f9e841f7fa0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0202c0 tx=0x7f9e7c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:01.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e81600640 1 -- 172.21.15.44:0/3743049511 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c00c280 con 0x7f9e8409da80 2024-09-13T08:52:01.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e81600640 1 -- 172.21.15.44:0/3743049511 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e7c0b7030 con 0x7f9e8409da80 2024-09-13T08:52:01.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 -- 172.21.15.44:0/3743049511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8409cb40 con 0x7f9e8409da80 2024-09-13T08:52:01.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e81600640 1 -- 172.21.15.44:0/3743049511 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c002090 con 0x7f9e8409da80 2024-09-13T08:52:01.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 -- 172.21.15.44:0/3743049511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8409d100 con 0x7f9e8409da80 2024-09-13T08:52:01.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.493+0000 7f9e8b400640 1 -- 172.21.15.44:0/3743049511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e84069410 con 0x7f9e8409da80 2024-09-13T08:52:01.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.497+0000 7f9e81600640 1 -- 172.21.15.44:0/3743049511 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f9e7c0bc420 con 0x7f9e8409da80 2024-09-13T08:52:01.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.497+0000 7f9e81600640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e60074db0 0x7f9e60077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:01.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.497+0000 7f9e81600640 1 -- 172.21.15.44:0/3743049511 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5821+0+0 (secure 0 0 0) 0x7f9e7c12fc10 con 0x7f9e8409da80 2024-09-13T08:52:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.497+0000 7f9e88a00640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e60074db0 0x7f9e60077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.497+0000 7f9e88a00640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e60074db0 0x7f9e60077270 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9e74000c10 tx=0x7f9e74006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.501+0000 7f9e81600640 1 -- 172.21.15.44:0/3743049511 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9e7c0c2020 con 0x7f9e8409da80 2024-09-13T08:52:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.677+0000 7f9e8b400640 1 -- 172.21.15.44:0/3743049511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e8406e780 con 0x7f9e8409da80 2024-09-13T08:52:01.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.677+0000 7f9e81600640 1 -- 172.21.15.44:0/3743049511 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16889 (secure 0 0 0) 0x7f9e7c0197d0 con 0x7f9e8409da80 2024-09-13T08:52:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:01.679 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":130,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:52:00.698051+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:51:58.420485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-13T08:51:59.669296+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:52:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 -- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e60074db0 msgr2=0x7f9e60077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e60074db0 0x7f9e60077270 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9e74000c10 tx=0x7f9e74006b80 comp rx=0 tx=0).stop 2024-09-13T08:52:01.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 -- 172.21.15.44:0/3743049511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e8409da80 msgr2=0x7f9e841f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e8409da80 0x7f9e841f7fa0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0202c0 tx=0x7f9e7c001260 comp rx=0 tx=0).stop 2024-09-13T08:52:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 -- 172.21.15.44:0/3743049511 shutdown_connections 2024-09-13T08:52:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e60074db0 0x7f9e60077270 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e8409da80 0x7f9e841f7fa0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e8409d620 0x7f9e841f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 --2- 172.21.15.44:0/3743049511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e68006400 0x7f9e84148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 -- 172.21.15.44:0/3743049511 >> 172.21.15.44:0/3743049511 conn(0x7f9e840b18f0 msgr2=0x7f9e8415c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 -- 172.21.15.44:0/3743049511 shutdown_connections 2024-09-13T08:52:01.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.681+0000 7f9e8b400640 1 -- 172.21.15.44:0/3743049511 wait complete. 2024-09-13T08:52:01.700 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:52:01.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.849+0000 7fbe2e200640 1 -- 172.21.15.44:0/2462795992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe0c003a40 msgr2=0x7fbe0c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:01.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.849+0000 7fbe2e200640 1 --2- 172.21.15.44:0/2462795992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe0c003a40 0x7fbe0c003e60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800c310 tx=0x7fbe1801cf60 comp rx=0 tx=0).stop 2024-09-13T08:52:01.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.849+0000 7fbe27e00640 1 -- 172.21.15.44:0/2462795992 reap_dead start 2024-09-13T08:52:01.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.849+0000 7fbe2e200640 1 -- 172.21.15.44:0/2462795992 shutdown_connections 2024-09-13T08:52:01.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.849+0000 7fbe2e200640 1 -- 172.21.15.44:0/2462795992 >> 172.21.15.44:0/2462795992 conn(0x7fbe280b18f0 msgr2=0x7fbe280b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:01.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.849+0000 7fbe2e200640 1 -- 172.21.15.44:0/2462795992 shutdown_connections 2024-09-13T08:52:01.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.849+0000 7fbe2e200640 1 -- 172.21.15.44:0/2462795992 wait complete. 2024-09-13T08:52:01.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.849+0000 7fbe2e200640 1 Processor -- start 2024-09-13T08:52:01.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.849+0000 7fbe2e200640 1 -- start start 2024-09-13T08:52:01.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe2e200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe0c003a40 0x7fbe28158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:01.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe2e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe280ad530 0x7fbe28152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:01.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe2e200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe28161020 0x7fbe281614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe2e200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28167630 con 0x7fbe280ad530 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe2e200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281674b0 con 0x7fbe28161020 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe2e200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fbe281677b0 con 0x7fbe0c003a40 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe280ad530 0x7fbe28152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe280ad530 0x7fbe28152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38456/0 (socket says 172.21.15.44:38456) 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1fe00640 1 -- 172.21.15.44:0/1506925014 learned_addr learned my addr 172.21.15.44:0/1506925014 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe27400640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe0c003a40 0x7fbe28158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe27e00640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe28161020 0x7fbe281614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1fe00640 1 -- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe0c003a40 msgr2=0x7fbe28158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1fe00640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe0c003a40 0x7fbe28158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1fe00640 1 -- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe28161020 msgr2=0x7fbe281614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1fe00640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe28161020 0x7fbe281614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1fe00640 1 -- 172.21.15.44:0/1506925014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe280cdec0 con 0x7fbe280ad530 2024-09-13T08:52:01.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1fe00640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe280ad530 0x7fbe28152930 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fbe10011b10 tx=0x7fbe10009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:01.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1f400640 1 -- 172.21.15.44:0/1506925014 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1001e070 con 0x7fbe280ad530 2024-09-13T08:52:01.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1f400640 1 -- 172.21.15.44:0/1506925014 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe10004050 con 0x7fbe280ad530 2024-09-13T08:52:01.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe2e200640 1 -- 172.21.15.44:0/1506925014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe280ce1a0 con 0x7fbe280ad530 2024-09-13T08:52:01.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1f400640 1 -- 172.21.15.44:0/1506925014 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000fea0 con 0x7fbe280ad530 2024-09-13T08:52:01.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe2e200640 1 -- 172.21.15.44:0/1506925014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe280ce760 con 0x7fbe280ad530 2024-09-13T08:52:01.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe2e200640 1 -- 172.21.15.44:0/1506925014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe28069410 con 0x7fbe280ad530 2024-09-13T08:52:01.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.853+0000 7fbe1f400640 1 -- 172.21.15.44:0/1506925014 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fbe10021070 con 0x7fbe280ad530 2024-09-13T08:52:01.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.857+0000 7fbe1f400640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe04074de0 0x7fbe040772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:01.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.857+0000 7fbe1f400640 1 -- 172.21.15.44:0/1506925014 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5821+0+0 (secure 0 0 0) 0x7fbe100938a0 con 0x7fbe280ad530 2024-09-13T08:52:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.857+0000 7fbe27400640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe04074de0 0x7fbe040772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.857+0000 7fbe27400640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe04074de0 0x7fbe040772a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbe180ab000 tx=0x7fbe18019da0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:01.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:01.861+0000 7fbe1f400640 1 -- 172.21.15.44:0/1506925014 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fbe1009ae20 con 0x7fbe280ad530 2024-09-13T08:52:02.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.037+0000 7fbe2e200640 1 -- 172.21.15.44:0/1506925014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe2806f1f0 con 0x7fbe280ad530 2024-09-13T08:52:02.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe1f400640 1 -- 172.21.15.44:0/1506925014 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16889 (secure 0 0 0) 0x7fbe1005ff30 con 0x7fbe280ad530 2024-09-13T08:52:02.040 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:02.040 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":130,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:52:00.698051+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:51:58.420485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-13T08:51:59.669296+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 -- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe04074de0 msgr2=0x7fbe040772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe04074de0 0x7fbe040772a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbe180ab000 tx=0x7fbe18019da0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 -- 172.21.15.44:0/1506925014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe280ad530 msgr2=0x7fbe28152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe280ad530 0x7fbe28152930 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fbe10011b10 tx=0x7fbe10009ea0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 -- 172.21.15.44:0/1506925014 shutdown_connections 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbe04074de0 0x7fbe040772a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe28161020 0x7fbe281614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe280ad530 0x7fbe28152930 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 --2- 172.21.15.44:0/1506925014 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbe0c003a40 0x7fbe28158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 -- 172.21.15.44:0/1506925014 >> 172.21.15.44:0/1506925014 conn(0x7fbe280b18f0 msgr2=0x7fbe28155890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:02.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 -- 172.21.15.44:0/1506925014 shutdown_connections 2024-09-13T08:52:02.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.041+0000 7fbe2e200640 1 -- 172.21.15.44:0/1506925014 wait complete. 2024-09-13T08:52:02.061 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T08:52:02.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.209+0000 7f1cda800640 1 -- 172.21.15.44:0/3489975414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd40b39e0 msgr2=0x7f1cd40b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:02.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.209+0000 7f1cda800640 1 --2- 172.21.15.44:0/3489975414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd40b39e0 0x7f1cd40b2330 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40738e0 tx=0x7f1ccc01cba0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.209+0000 7f1cda800640 1 -- 172.21.15.44:0/3489975414 shutdown_connections 2024-09-13T08:52:02.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.209+0000 7f1cda800640 1 --2- 172.21.15.44:0/3489975414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd40b39e0 0x7f1cd40b2330 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.209+0000 7f1cda800640 1 --2- 172.21.15.44:0/3489975414 >> v2:172.21.15.129:3300/0 conn(0x7f1cd415f340 0x7f1cd40b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.209+0000 7f1cda800640 1 -- 172.21.15.44:0/3489975414 >> 172.21.15.44:0/3489975414 conn(0x7f1cd40d1e00 msgr2=0x7f1cd40d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:02.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.209+0000 7f1cda800640 1 -- 172.21.15.44:0/3489975414 shutdown_connections 2024-09-13T08:52:02.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.209+0000 7f1cda800640 1 -- 172.21.15.44:0/3489975414 wait complete. 2024-09-13T08:52:02.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 Processor -- start 2024-09-13T08:52:02.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 -- start start 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd40b2a00 0x7f1cd40b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cd40b39e0 0x7f1cd40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cd415f340 0x7f1cd41fb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4168cd0 con 0x7f1cd40b2a00 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4168b50 con 0x7f1cd415f340 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd4168e50 con 0x7f1cd40b39e0 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cd3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd40b2a00 0x7f1cd40b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cd3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd40b2a00 0x7f1cd40b73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36742/0 (socket says 172.21.15.44:36742) 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cd3400640 1 -- 172.21.15.44:0/1748340838 learned_addr learned my addr 172.21.15.44:0/1748340838 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1ccaa00640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cd40b39e0 0x7f1cd40b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:02.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cd3e00640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cd415f340 0x7f1cd41fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cd3400640 1 -- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cd40b39e0 msgr2=0x7f1cd40b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cd3400640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cd40b39e0 0x7f1cd40b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cd3400640 1 -- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cd415f340 msgr2=0x7f1cd41fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cd3400640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cd415f340 0x7f1cd41fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cd3400640 1 -- 172.21.15.44:0/1748340838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd4163fb0 con 0x7f1cd40b2a00 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cd3400640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd40b2a00 0x7f1cd40b73d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400d720 tx=0x7f1cc400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1ccbe00640 1 -- 172.21.15.44:0/1748340838 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4017070 con 0x7f1cd40b2a00 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1ccbe00640 1 -- 172.21.15.44:0/1748340838 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cc4009d20 con 0x7f1cd40b2a00 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1ccbe00640 1 -- 172.21.15.44:0/1748340838 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4005280 con 0x7f1cd40b2a00 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 -- 172.21.15.44:0/1748340838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd4164290 con 0x7f1cd40b2a00 2024-09-13T08:52:02.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 -- 172.21.15.44:0/1748340838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd40b7910 con 0x7f1cd40b2a00 2024-09-13T08:52:02.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.213+0000 7f1cda800640 1 -- 172.21.15.44:0/1748340838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd41644d0 con 0x7f1cd40b2a00 2024-09-13T08:52:02.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.217+0000 7f1ccbe00640 1 -- 172.21.15.44:0/1748340838 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f1cc4005420 con 0x7f1cd40b2a00 2024-09-13T08:52:02.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.217+0000 7f1ccbe00640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb0074db0 0x7f1cb0077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:02.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.217+0000 7f1ccbe00640 1 -- 172.21.15.44:0/1748340838 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5821+0+0 (secure 0 0 0) 0x7f1cc4092fb0 con 0x7f1cd40b2a00 2024-09-13T08:52:02.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.217+0000 7f1ccaa00640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb0074db0 0x7f1cb0077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:02.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.217+0000 7f1ccaa00640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb0074db0 0x7f1cb0077270 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4166730 tx=0x7f1cbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:02.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.221+0000 7f1ccbe00640 1 -- 172.21.15.44:0/1748340838 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1cd41644d0 con 0x7f1cd40b2a00 2024-09-13T08:52:02.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:02.437+0000 7f1cda800640 1 -- 172.21.15.44:0/1748340838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1cd40b34a0 con 0x7f1cd40b2a00 2024-09-13T08:52:02.950 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:52:02.953 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:52:03.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.313+0000 7f1ccbe00640 1 -- 172.21.15.44:0/1748340838 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) ==== 118+0+0 (secure 0 0 0) 0x7f1cc405fc60 con 0x7f1cd40b2a00 2024-09-13T08:52:03.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 -- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb0074db0 msgr2=0x7f1cb0077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:03.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb0074db0 0x7f1cb0077270 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4166730 tx=0x7f1cbc009040 comp rx=0 tx=0).stop 2024-09-13T08:52:03.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 -- 172.21.15.44:0/1748340838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd40b2a00 msgr2=0x7f1cd40b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:03.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd40b2a00 0x7f1cd40b73d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400d720 tx=0x7f1cc400dbe0 comp rx=0 tx=0).stop 2024-09-13T08:52:03.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 -- 172.21.15.44:0/1748340838 shutdown_connections 2024-09-13T08:52:03.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1cb0074db0 0x7f1cb0077270 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:03.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cd415f340 0x7f1cd41fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:03.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1cd40b39e0 0x7f1cd40b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:03.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 --2- 172.21.15.44:0/1748340838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd40b2a00 0x7f1cd40b73d0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:03.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 -- 172.21.15.44:0/1748340838 >> 172.21.15.44:0/1748340838 conn(0x7f1cd40d1e00 msgr2=0x7f1cd40b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:03.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 -- 172.21.15.44:0/1748340838 shutdown_connections 2024-09-13T08:52:03.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.317+0000 7f1cda800640 1 -- 172.21.15.44:0/1748340838 wait complete. 2024-09-13T08:52:03.336 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T08:52:03.336 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T08:52:03.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.485+0000 7f475e800640 1 -- 172.21.15.44:0/2046498916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47581605d0 msgr2=0x7f47580d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:03.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.485+0000 7f475e800640 1 --2- 172.21.15.44:0/2046498916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47581605d0 0x7f47580d0690 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4748009840 tx=0x7f474801c820 comp rx=0 tx=0).stop 2024-09-13T08:52:03.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.485+0000 7f475e800640 1 -- 172.21.15.44:0/2046498916 shutdown_connections 2024-09-13T08:52:03.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.485+0000 7f475e800640 1 --2- 172.21.15.44:0/2046498916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47581605d0 0x7f47580d0690 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:03.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.485+0000 7f475e800640 1 -- 172.21.15.44:0/2046498916 >> 172.21.15.44:0/2046498916 conn(0x7f47580d1dd0 msgr2=0x7f47580d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:03.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.485+0000 7f475e800640 1 -- 172.21.15.44:0/2046498916 shutdown_connections 2024-09-13T08:52:03.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.485+0000 7f475e800640 1 -- 172.21.15.44:0/2046498916 wait complete. 2024-09-13T08:52:03.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 Processor -- start 2024-09-13T08:52:03.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 -- start start 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47580afb10 0x7f475806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47580b1650 0x7f47581fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47581605d0 0x7f475815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f47580b13a0 con 0x7f47580b1650 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47580b1220 con 0x7f47580afb10 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f475816a550 con 0x7f47581605d0 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47580b1650 0x7f47581fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47580b1650 0x7f47581fbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36762/0 (socket says 172.21.15.44:36762) 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474ea00640 1 -- 172.21.15.44:0/1088192222 learned_addr learned my addr 172.21.15.44:0/1088192222 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474ea00640 1 -- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47581605d0 msgr2=0x7f475815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474ea00640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47581605d0 0x7f475815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474ea00640 1 -- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47580afb10 msgr2=0x7f475806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:03.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f4757400640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47580afb10 0x7f475806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:03.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474ea00640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47580afb10 0x7f475806edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:03.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474ea00640 1 -- 172.21.15.44:0/1088192222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47580cf980 con 0x7f47580b1650 2024-09-13T08:52:03.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f4757400640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47580afb10 0x7f475806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:52:03.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474ea00640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47580b1650 0x7f47581fbe50 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f474000c480 tx=0x7f474000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:03.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474fe00640 1 -- 172.21.15.44:0/1088192222 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740016020 con 0x7f47580b1650 2024-09-13T08:52:03.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474fe00640 1 -- 172.21.15.44:0/1088192222 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4740012920 con 0x7f47580b1650 2024-09-13T08:52:03.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f474fe00640 1 -- 172.21.15.44:0/1088192222 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740012d70 con 0x7f47580b1650 2024-09-13T08:52:03.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 -- 172.21.15.44:0/1088192222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47581601a0 con 0x7f47580b1650 2024-09-13T08:52:03.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 -- 172.21.15.44:0/1088192222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475806f3d0 con 0x7f47580b1650 2024-09-13T08:52:03.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.489+0000 7f475e800640 1 -- 172.21.15.44:0/1088192222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47580d0690 con 0x7f47580b1650 2024-09-13T08:52:03.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.493+0000 7f474fe00640 1 -- 172.21.15.44:0/1088192222 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f4740019020 con 0x7f47580b1650 2024-09-13T08:52:03.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.493+0000 7f474fe00640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4734074db0 0x7f4734077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:03.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.493+0000 7f474fe00640 1 -- 172.21.15.44:0/1088192222 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f4740092ee0 con 0x7f47580b1650 2024-09-13T08:52:03.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.493+0000 7f4757400640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4734074db0 0x7f4734077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:03.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.493+0000 7f4757400640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4734074db0 0x7f4734077270 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f47480af000 tx=0x7f474801b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:03.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.497+0000 7f474fe00640 1 -- 172.21.15.44:0/1088192222 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4740020020 con 0x7f47580b1650 2024-09-13T08:52:03.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:03.709+0000 7f475e800640 1 -- 172.21.15.44:0/1088192222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f47580b2280 con 0x7f47580b1650 2024-09-13T08:52:04.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.317+0000 7f474fe00640 1 -- 172.21.15.44:0/1088192222 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) ==== 122+0+0 (secure 0 0 0) 0x7f474005fb90 con 0x7f47580b1650 2024-09-13T08:52:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 -- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4734074db0 msgr2=0x7f4734077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4734074db0 0x7f4734077270 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f47480af000 tx=0x7f474801b060 comp rx=0 tx=0).stop 2024-09-13T08:52:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 -- 172.21.15.44:0/1088192222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47580b1650 msgr2=0x7f47581fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47580b1650 0x7f47581fbe50 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f474000c480 tx=0x7f474000c940 comp rx=0 tx=0).stop 2024-09-13T08:52:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 -- 172.21.15.44:0/1088192222 shutdown_connections 2024-09-13T08:52:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4734074db0 0x7f4734077270 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f47581605d0 0x7f475815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47580b1650 0x7f47581fbe50 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 --2- 172.21.15.44:0/1088192222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47580afb10 0x7f475806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 -- 172.21.15.44:0/1088192222 >> 172.21.15.44:0/1088192222 conn(0x7f47580d1dd0 msgr2=0x7f4758071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 -- 172.21.15.44:0/1088192222 shutdown_connections 2024-09-13T08:52:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.321+0000 7f475e800640 1 -- 172.21.15.44:0/1088192222 wait complete. 2024-09-13T08:52:04.343 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T08:52:04.343 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:52:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.493+0000 7f1997200640 1 -- 172.21.15.44:0/454533029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f197c005290 msgr2=0x7f197c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.493+0000 7f1997200640 1 --2- 172.21.15.44:0/454533029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f197c005290 0x7f197c00b9d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1990009ee0 tx=0x7f199001d630 comp rx=0 tx=0).stop 2024-09-13T08:52:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.493+0000 7f1997200640 1 -- 172.21.15.44:0/454533029 shutdown_connections 2024-09-13T08:52:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.493+0000 7f1997200640 1 --2- 172.21.15.44:0/454533029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f197c005290 0x7f197c00b9d0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.493+0000 7f1997200640 1 -- 172.21.15.44:0/454533029 >> 172.21.15.44:0/454533029 conn(0x7f199809aa10 msgr2=0x7f199809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.493+0000 7f1997200640 1 -- 172.21.15.44:0/454533029 shutdown_connections 2024-09-13T08:52:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.493+0000 7f1997200640 1 -- 172.21.15.44:0/454533029 wait complete. 2024-09-13T08:52:04.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 Processor -- start 2024-09-13T08:52:04.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 -- start start 2024-09-13T08:52:04.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f197c005290 0x7f199815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:04.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19980b6970 0x7f19980afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19980b6df0 0x7f19980b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f19980cf390 con 0x7f19980b6970 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19980cf210 con 0x7f197c005290 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f19980b2590 con 0x7f19980b6df0 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19980b6970 0x7f19980afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19980b6970 0x7f19980afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36770/0 (socket says 172.21.15.44:36770) 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198fe00640 1 -- 172.21.15.44:0/2823122252 learned_addr learned my addr 172.21.15.44:0/2823122252 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1996800640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19980b6df0 0x7f19980b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1995e00640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f197c005290 0x7f199815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198fe00640 1 -- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19980b6df0 msgr2=0x7f19980b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198fe00640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19980b6df0 0x7f19980b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198fe00640 1 -- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f197c005290 msgr2=0x7f199815dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198fe00640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f197c005290 0x7f199815dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198fe00640 1 -- 172.21.15.44:0/2823122252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19981f7da0 con 0x7f19980b6970 2024-09-13T08:52:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198fe00640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19980b6970 0x7f19980afd00 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f19800063d0 tx=0x7f19800039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198e000640 1 -- 172.21.15.44:0/2823122252 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980015070 con 0x7f19980b6970 2024-09-13T08:52:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198e000640 1 -- 172.21.15.44:0/2823122252 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1980011840 con 0x7f19980b6970 2024-09-13T08:52:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 -- 172.21.15.44:0/2823122252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19981f8040 con 0x7f19980b6970 2024-09-13T08:52:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f198e000640 1 -- 172.21.15.44:0/2823122252 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980011c90 con 0x7f19980b6970 2024-09-13T08:52:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 -- 172.21.15.44:0/2823122252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19981f8600 con 0x7f19980b6970 2024-09-13T08:52:04.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.497+0000 7f1997200640 1 -- 172.21.15.44:0/2823122252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19980b2a90 con 0x7f19980b6970 2024-09-13T08:52:04.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.501+0000 7f198e000640 1 -- 172.21.15.44:0/2823122252 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f1980018070 con 0x7f19980b6970 2024-09-13T08:52:04.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.501+0000 7f198e000640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1974074de0 0x7f19740772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:04.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.501+0000 7f198e000640 1 -- 172.21.15.44:0/2823122252 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f1980093990 con 0x7f19980b6970 2024-09-13T08:52:04.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.501+0000 7f1995e00640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1974074de0 0x7f19740772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:04.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.501+0000 7f1995e00640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1974074de0 0x7f19740772a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1988000c10 tx=0x7f1988006b20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:04.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.505+0000 7f198e000640 1 -- 172.21.15.44:0/2823122252 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f198005fff0 con 0x7f19980b6970 2024-09-13T08:52:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 -- 172.21.15.44:0/2823122252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19980ce360 con 0x7f19980b6970 2024-09-13T08:52:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f198e000640 1 -- 172.21.15.44:0/2823122252 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5228 (secure 0 0 0) 0x7f19800647e0 con 0x7f19980b6970 2024-09-13T08:52:04.716 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:04.716 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":85,"btime":"2024-09-13T08:52:04:315400+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/771921648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":771921648},{"type":"v1","addr":"172.21.15.129:6836","nonce":771921648}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8104,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1464443005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1464443005},{"type":"v1","addr":"172.21.15.129:6837","nonce":1464443005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2601537201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2601537201},{"type":"v1","addr":"172.21.15.44:6833","nonce":2601537201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:52:00.698019+0000","modified":"2024-09-13T08:52:03.712285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8763":{"gid":8763,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2181881572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2181881572},{"type":"v1","addr":"172.21.15.44:6835","nonce":2181881572}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8763,"qdb_cluster":[8763]},"id":9}]} 2024-09-13T08:52:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 85 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 -- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1974074de0 msgr2=0x7f19740772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1974074de0 0x7f19740772a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1988000c10 tx=0x7f1988006b20 comp rx=0 tx=0).stop 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 -- 172.21.15.44:0/2823122252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19980b6970 msgr2=0x7f19980afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19980b6970 0x7f19980afd00 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f19800063d0 tx=0x7f19800039d0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 -- 172.21.15.44:0/2823122252 shutdown_connections 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1974074de0 0x7f19740772a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f19980b6df0 0x7f19980b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19980b6970 0x7f19980afd00 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 --2- 172.21.15.44:0/2823122252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f197c005290 0x7f199815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 -- 172.21.15.44:0/2823122252 >> 172.21.15.44:0/2823122252 conn(0x7f199809aa10 msgr2=0x7f19980b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 -- 172.21.15.44:0/2823122252 shutdown_connections 2024-09-13T08:52:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.717+0000 7f1997200640 1 -- 172.21.15.44:0/2823122252 wait complete. 2024-09-13T08:52:04.738 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:52:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 -- 172.21.15.44:0/462835246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c15bd50 msgr2=0x7f6a3c0ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 --2- 172.21.15.44:0/462835246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c15bd50 0x7f6a3c0ce5e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0d0a60 tx=0x7f6a2c01c780 comp rx=0 tx=0).stop 2024-09-13T08:52:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 -- 172.21.15.44:0/462835246 shutdown_connections 2024-09-13T08:52:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 --2- 172.21.15.44:0/462835246 >> v2:172.21.15.129:3300/0 conn(0x7f6a3c0cf4e0 0x7f6a3c0c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 --2- 172.21.15.44:0/462835246 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a3c15c130 0x7f6a3c0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 --2- 172.21.15.44:0/462835246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c15bd50 0x7f6a3c0ce5e0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 -- 172.21.15.44:0/462835246 >> 172.21.15.44:0/462835246 conn(0x7f6a3c0d1ea0 msgr2=0x7f6a3c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 -- 172.21.15.44:0/462835246 shutdown_connections 2024-09-13T08:52:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 -- 172.21.15.44:0/462835246 wait complete. 2024-09-13T08:52:04.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 Processor -- start 2024-09-13T08:52:04.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.885+0000 7f6a40e00640 1 -- start start 2024-09-13T08:52:04.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a40e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a3c0cf4e0 0x7f6a3c156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:04.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a40e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c15bd50 0x7f6a3c09d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:04.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a40e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3c15c130 0x7f6a3c09de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a40e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c0ccac0 con 0x7f6a3c15bd50 2024-09-13T08:52:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a40e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c0cc940 con 0x7f6a3c15c130 2024-09-13T08:52:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a40e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3c0ccc40 con 0x7f6a3c0cf4e0 2024-09-13T08:52:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a33e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c15bd50 0x7f6a3c09d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a39e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a3c0cf4e0 0x7f6a3c156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a33e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c15bd50 0x7f6a3c09d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36792/0 (socket says 172.21.15.44:36792) 2024-09-13T08:52:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a33e00640 1 -- 172.21.15.44:0/130415339 learned_addr learned my addr 172.21.15.44:0/130415339 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a3a800640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3c15c130 0x7f6a3c09de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:04.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a33e00640 1 -- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a3c0cf4e0 msgr2=0x7f6a3c156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:04.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a33e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a3c0cf4e0 0x7f6a3c156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a33e00640 1 -- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3c15c130 msgr2=0x7f6a3c09de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:04.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a33e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3c15c130 0x7f6a3c09de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:04.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a33e00640 1 -- 172.21.15.44:0/130415339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3c0b2670 con 0x7f6a3c15bd50 2024-09-13T08:52:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a33e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c15bd50 0x7f6a3c09d900 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6a24005d10 tx=0x7f6a2400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a32000640 1 -- 172.21.15.44:0/130415339 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24017070 con 0x7f6a3c15bd50 2024-09-13T08:52:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a32000640 1 -- 172.21.15.44:0/130415339 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a24009b20 con 0x7f6a3c15bd50 2024-09-13T08:52:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a32000640 1 -- 172.21.15.44:0/130415339 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24005020 con 0x7f6a3c15bd50 2024-09-13T08:52:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a40e00640 1 -- 172.21.15.44:0/130415339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3c0b2950 con 0x7f6a3c15bd50 2024-09-13T08:52:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a40e00640 1 -- 172.21.15.44:0/130415339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3c09c580 con 0x7f6a3c15bd50 2024-09-13T08:52:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.889+0000 7f6a40e00640 1 -- 172.21.15.44:0/130415339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a3c0c5dd0 con 0x7f6a3c15bd50 2024-09-13T08:52:04.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.893+0000 7f6a32000640 1 -- 172.21.15.44:0/130415339 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f6a240051c0 con 0x7f6a3c15bd50 2024-09-13T08:52:04.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.893+0000 7f6a32000640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074de0 0x7f6a180772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:04.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.893+0000 7f6a32000640 1 -- 172.21.15.44:0/130415339 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f6a24093340 con 0x7f6a3c15bd50 2024-09-13T08:52:04.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.893+0000 7f6a39e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074de0 0x7f6a180772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:04.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.893+0000 7f6a39e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074de0 0x7f6a180772a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0af000 tx=0x7f6a2c01afc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:04.893+0000 7f6a32000640 1 -- 172.21.15.44:0/130415339 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6a2405f9a0 con 0x7f6a3c15bd50 2024-09-13T08:52:05.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.073+0000 7f6a40e00640 1 -- 172.21.15.44:0/130415339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a3c071cd0 con 0x7f6a3c15bd50 2024-09-13T08:52:05.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.073+0000 7f6a32000640 1 -- 172.21.15.44:0/130415339 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16876 (secure 0 0 0) 0x7f6a240643d0 con 0x7f6a3c15bd50 2024-09-13T08:52:05.074 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:05.074 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":131,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:52:02.300428+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:51:58.420485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-13T08:51:59.669296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:52:05.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 -- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074de0 msgr2=0x7f6a180772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074de0 0x7f6a180772a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0af000 tx=0x7f6a2c01afc0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 -- 172.21.15.44:0/130415339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c15bd50 msgr2=0x7f6a3c09d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c15bd50 0x7f6a3c09d900 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6a24005d10 tx=0x7f6a2400dbb0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 -- 172.21.15.44:0/130415339 shutdown_connections 2024-09-13T08:52:05.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6a18074de0 0x7f6a180772a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a3c15c130 0x7f6a3c09de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c15bd50 0x7f6a3c09d900 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 --2- 172.21.15.44:0/130415339 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6a3c0cf4e0 0x7f6a3c156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 -- 172.21.15.44:0/130415339 >> 172.21.15.44:0/130415339 conn(0x7f6a3c0d1ea0 msgr2=0x7f6a3c0d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:05.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 -- 172.21.15.44:0/130415339 shutdown_connections 2024-09-13T08:52:05.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.077+0000 7f6a40e00640 1 -- 172.21.15.44:0/130415339 wait complete. 2024-09-13T08:52:05.097 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:52:05.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:52:05.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.245+0000 7fb5c8a00640 1 -- 172.21.15.44:0/2500713908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5a8004d50 msgr2=0x7fb5a80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.245+0000 7fb5c8a00640 1 --2- 172.21.15.44:0/2500713908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5a8004d50 0x7fb5a80051d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00c640 tx=0x7fb5bc01fa60 comp rx=0 tx=0).stop 2024-09-13T08:52:05.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.245+0000 7fb5c2800640 1 -- 172.21.15.44:0/2500713908 reap_dead start 2024-09-13T08:52:05.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.245+0000 7fb5c8a00640 1 -- 172.21.15.44:0/2500713908 shutdown_connections 2024-09-13T08:52:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.245+0000 7fb5c8a00640 1 -- 172.21.15.44:0/2500713908 >> 172.21.15.44:0/2500713908 conn(0x7fb5c40b18f0 msgr2=0x7fb5c40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.245+0000 7fb5c8a00640 1 -- 172.21.15.44:0/2500713908 shutdown_connections 2024-09-13T08:52:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.245+0000 7fb5c8a00640 1 -- 172.21.15.44:0/2500713908 wait complete. 2024-09-13T08:52:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.245+0000 7fb5c8a00640 1 Processor -- start 2024-09-13T08:52:05.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.245+0000 7fb5c8a00640 1 -- start start 2024-09-13T08:52:05.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c8a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb5a8004d50 0x7fb5c409cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:05.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c4147880 0x7fb5c415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:05.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c8a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c415aa10 0x7fb5c409ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:05.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c8a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c415b760 con 0x7fb5c4147880 2024-09-13T08:52:05.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c8a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c415b5e0 con 0x7fb5c415aa10 2024-09-13T08:52:05.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c8a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c40b46e0 con 0x7fb5a8004d50 2024-09-13T08:52:05.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c4147880 0x7fb5c415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c4147880 0x7fb5c415a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36798/0 (socket says 172.21.15.44:36798) 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5bbe00640 1 -- 172.21.15.44:0/1847441821 learned_addr learned my addr 172.21.15.44:0/1847441821 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c1e00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb5a8004d50 0x7fb5c409cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c2800640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c415aa10 0x7fb5c409ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5bbe00640 1 -- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb5a8004d50 msgr2=0x7fb5c409cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5bbe00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb5a8004d50 0x7fb5c409cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5bbe00640 1 -- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c415aa10 msgr2=0x7fb5c409ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5bbe00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c415aa10 0x7fb5c409ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5bbe00640 1 -- 172.21.15.44:0/1847441821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c41f7da0 con 0x7fb5c4147880 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c2800640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c415aa10 0x7fb5c409ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5bbe00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c4147880 0x7fb5c415a4d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac011ae0 tx=0x7fb5ac009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5ba000640 1 -- 172.21.15.44:0/1847441821 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac01e070 con 0x7fb5c4147880 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5ba000640 1 -- 172.21.15.44:0/1847441821 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5ac004460 con 0x7fb5c4147880 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c8a00640 1 -- 172.21.15.44:0/1847441821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c41f8080 con 0x7fb5c4147880 2024-09-13T08:52:05.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5ba000640 1 -- 172.21.15.44:0/1847441821 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac00fc30 con 0x7fb5c4147880 2024-09-13T08:52:05.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c8a00640 1 -- 172.21.15.44:0/1847441821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c41f8640 con 0x7fb5c4147880 2024-09-13T08:52:05.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.249+0000 7fb5c8a00640 1 -- 172.21.15.44:0/1847441821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c4069410 con 0x7fb5c4147880 2024-09-13T08:52:05.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.253+0000 7fb5ba000640 1 -- 172.21.15.44:0/1847441821 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fb5ac021070 con 0x7fb5c4147880 2024-09-13T08:52:05.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.253+0000 7fb5ba000640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0074de0 0x7fb5a00772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:05.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.253+0000 7fb5ba000640 1 -- 172.21.15.44:0/1847441821 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7fb5ac093810 con 0x7fb5c4147880 2024-09-13T08:52:05.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.253+0000 7fb5c1e00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0074de0 0x7fb5a00772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:05.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.253+0000 7fb5c1e00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0074de0 0x7fb5a00772a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4005e10 tx=0x7fb5b40091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:05.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.257+0000 7fb5ba000640 1 -- 172.21.15.44:0/1847441821 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb5ac05ff30 con 0x7fb5c4147880 2024-09-13T08:52:05.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.433+0000 7fb5c8a00640 1 -- 172.21.15.44:0/1847441821 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5c40cc000 con 0x7fb5a0074de0 2024-09-13T08:52:05.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.437+0000 7fb5ba000640 1 -- 172.21.15.44:0/1847441821 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252498 (secure 0 0 0) 0x7fb5c40cc000 con 0x7fb5a0074de0 2024-09-13T08:52:05.440 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:05.442 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:52:05.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 -- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0074de0 msgr2=0x7fb5a00772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0074de0 0x7fb5a00772a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4005e10 tx=0x7fb5b40091e0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 -- 172.21.15.44:0/1847441821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c4147880 msgr2=0x7fb5c415a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c4147880 0x7fb5c415a4d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac011ae0 tx=0x7fb5ac009ea0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 -- 172.21.15.44:0/1847441821 shutdown_connections 2024-09-13T08:52:05.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5a0074de0 0x7fb5a00772a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c415aa10 0x7fb5c409ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c4147880 0x7fb5c415a4d0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 --2- 172.21.15.44:0/1847441821 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb5a8004d50 0x7fb5c409cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 -- 172.21.15.44:0/1847441821 >> 172.21.15.44:0/1847441821 conn(0x7fb5c40b18f0 msgr2=0x7fb5c4072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:05.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 -- 172.21.15.44:0/1847441821 shutdown_connections 2024-09-13T08:52:05.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.441+0000 7fb5c8a00640 1 -- 172.21.15.44:0/1847441821 wait complete. 2024-09-13T08:52:05.461 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":565,"stamp":"2024-09-13T08:52:04.816487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1094516,"kb_used_data":13928,"kb_used_omap":12,"kb_used_meta":1080371,"kb_avail":753880204,"statfs":{"total":773094113280,"available":771973328896,"internally_reserved":0,"allocated":14262272,"data_stored":6926868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1106300465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.536333"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316252+0000","last_change":"2024-09-13T08:52:01.322527+0000","last_active":"2024-09-13T08:52:02.316252+0000","last_peered":"2024-09-13T08:52:02.316252+0000","last_clean":"2024-09-13T08:52:02.316252+0000","last_became_active":"2024-09-13T08:52:01.322224+0000","last_became_peered":"2024-09-13T08:52:01.322224+0000","last_unstale":"2024-09-13T08:52:02.316252+0000","last_undegraded":"2024-09-13T08:52:02.316252+0000","last_fullsized":"2024-09-13T08:52:02.316252+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:30:12.276749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315667+0000","last_change":"2024-09-13T08:52:01.312181+0000","last_active":"2024-09-13T08:52:02.315667+0000","last_peered":"2024-09-13T08:52:02.315667+0000","last_clean":"2024-09-13T08:52:02.315667+0000","last_became_active":"2024-09-13T08:52:01.311857+0000","last_became_peered":"2024-09-13T08:52:01.311857+0000","last_unstale":"2024-09-13T08:52:02.315667+0000","last_undegraded":"2024-09-13T08:52:02.315667+0000","last_fullsized":"2024-09-13T08:52:02.315667+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:39:56.718999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741781+0000","last_change":"2024-09-13T08:52:01.317101+0000","last_active":"2024-09-13T08:52:02.741781+0000","last_peered":"2024-09-13T08:52:02.741781+0000","last_clean":"2024-09-13T08:52:02.741781+0000","last_became_active":"2024-09-13T08:52:01.316665+0000","last_became_peered":"2024-09-13T08:52:01.316665+0000","last_unstale":"2024-09-13T08:52:02.741781+0000","last_undegraded":"2024-09-13T08:52:02.741781+0000","last_fullsized":"2024-09-13T08:52:02.741781+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:29:55.643905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317455+0000","last_change":"2024-09-13T08:52:01.318694+0000","last_active":"2024-09-13T08:52:02.317455+0000","last_peered":"2024-09-13T08:52:02.317455+0000","last_clean":"2024-09-13T08:52:02.317455+0000","last_became_active":"2024-09-13T08:52:01.318034+0000","last_became_peered":"2024-09-13T08:52:01.318034+0000","last_unstale":"2024-09-13T08:52:02.317455+0000","last_undegraded":"2024-09-13T08:52:02.317455+0000","last_fullsized":"2024-09-13T08:52:02.317455+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:08.298145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741549+0000","last_change":"2024-09-13T08:52:01.329818+0000","last_active":"2024-09-13T08:52:02.741549+0000","last_peered":"2024-09-13T08:52:02.741549+0000","last_clean":"2024-09-13T08:52:02.741549+0000","last_became_active":"2024-09-13T08:52:01.329412+0000","last_became_peered":"2024-09-13T08:52:01.329412+0000","last_unstale":"2024-09-13T08:52:02.741549+0000","last_undegraded":"2024-09-13T08:52:02.741549+0000","last_fullsized":"2024-09-13T08:52:02.741549+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:54:04.345707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311175+0000","last_change":"2024-09-13T08:52:01.312938+0000","last_active":"2024-09-13T08:52:02.311175+0000","last_peered":"2024-09-13T08:52:02.311175+0000","last_clean":"2024-09-13T08:52:02.311175+0000","last_became_active":"2024-09-13T08:52:01.312609+0000","last_became_peered":"2024-09-13T08:52:01.312609+0000","last_unstale":"2024-09-13T08:52:02.311175+0000","last_undegraded":"2024-09-13T08:52:02.311175+0000","last_fullsized":"2024-09-13T08:52:02.311175+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:16.750964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741080+0000","last_change":"2024-09-13T08:52:01.321368+0000","last_active":"2024-09-13T08:52:02.741080+0000","last_peered":"2024-09-13T08:52:02.741080+0000","last_clean":"2024-09-13T08:52:02.741080+0000","last_became_active":"2024-09-13T08:52:01.321063+0000","last_became_peered":"2024-09-13T08:52:01.321063+0000","last_unstale":"2024-09-13T08:52:02.741080+0000","last_undegraded":"2024-09-13T08:52:02.741080+0000","last_fullsized":"2024-09-13T08:52:02.741080+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:22:47.683542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.740802+0000","last_change":"2024-09-13T08:52:01.304827+0000","last_active":"2024-09-13T08:52:02.740802+0000","last_peered":"2024-09-13T08:52:02.740802+0000","last_clean":"2024-09-13T08:52:02.740802+0000","last_became_active":"2024-09-13T08:52:01.304500+0000","last_became_peered":"2024-09-13T08:52:01.304500+0000","last_unstale":"2024-09-13T08:52:02.740802+0000","last_undegraded":"2024-09-13T08:52:02.740802+0000","last_fullsized":"2024-09-13T08:52:02.740802+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:15.390950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742192+0000","last_change":"2024-09-13T08:52:01.319989+0000","last_active":"2024-09-13T08:52:02.742192+0000","last_peered":"2024-09-13T08:52:02.742192+0000","last_clean":"2024-09-13T08:52:02.742192+0000","last_became_active":"2024-09-13T08:52:01.318859+0000","last_became_peered":"2024-09-13T08:52:01.318859+0000","last_unstale":"2024-09-13T08:52:02.742192+0000","last_undegraded":"2024-09-13T08:52:02.742192+0000","last_fullsized":"2024-09-13T08:52:02.742192+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:06.887326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315981+0000","last_change":"2024-09-13T08:52:01.323289+0000","last_active":"2024-09-13T08:52:02.315981+0000","last_peered":"2024-09-13T08:52:02.315981+0000","last_clean":"2024-09-13T08:52:02.315981+0000","last_became_active":"2024-09-13T08:52:01.322978+0000","last_became_peered":"2024-09-13T08:52:01.322978+0000","last_unstale":"2024-09-13T08:52:02.315981+0000","last_undegraded":"2024-09-13T08:52:02.315981+0000","last_fullsized":"2024-09-13T08:52:02.315981+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:56.774369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317461+0000","last_change":"2024-09-13T08:52:01.314548+0000","last_active":"2024-09-13T08:52:02.317461+0000","last_peered":"2024-09-13T08:52:02.317461+0000","last_clean":"2024-09-13T08:52:02.317461+0000","last_became_active":"2024-09-13T08:52:01.314117+0000","last_became_peered":"2024-09-13T08:52:01.314117+0000","last_unstale":"2024-09-13T08:52:02.317461+0000","last_undegraded":"2024-09-13T08:52:02.317461+0000","last_fullsized":"2024-09-13T08:52:02.317461+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:13:00.802333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311696+0000","last_change":"2024-09-13T08:52:01.320522+0000","last_active":"2024-09-13T08:52:02.311696+0000","last_peered":"2024-09-13T08:52:02.311696+0000","last_clean":"2024-09-13T08:52:02.311696+0000","last_became_active":"2024-09-13T08:52:01.320133+0000","last_became_peered":"2024-09-13T08:52:01.320133+0000","last_unstale":"2024-09-13T08:52:02.311696+0000","last_undegraded":"2024-09-13T08:52:02.311696+0000","last_fullsized":"2024-09-13T08:52:02.311696+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:23.797197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317238+0000","last_change":"2024-09-13T08:52:01.298595+0000","last_active":"2024-09-13T08:52:02.317238+0000","last_peered":"2024-09-13T08:52:02.317238+0000","last_clean":"2024-09-13T08:52:02.317238+0000","last_became_active":"2024-09-13T08:52:01.298273+0000","last_became_peered":"2024-09-13T08:52:01.298273+0000","last_unstale":"2024-09-13T08:52:02.317238+0000","last_undegraded":"2024-09-13T08:52:02.317238+0000","last_fullsized":"2024-09-13T08:52:02.317238+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:00.466613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742511+0000","last_change":"2024-09-13T08:52:01.319951+0000","last_active":"2024-09-13T08:52:02.742511+0000","last_peered":"2024-09-13T08:52:02.742511+0000","last_clean":"2024-09-13T08:52:02.742511+0000","last_became_active":"2024-09-13T08:52:01.319635+0000","last_became_peered":"2024-09-13T08:52:01.319635+0000","last_unstale":"2024-09-13T08:52:02.742511+0000","last_undegraded":"2024-09-13T08:52:02.742511+0000","last_fullsized":"2024-09-13T08:52:02.742511+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:20:22.380961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742641+0000","last_change":"2024-09-13T08:52:01.316882+0000","last_active":"2024-09-13T08:52:02.742641+0000","last_peered":"2024-09-13T08:52:02.742641+0000","last_clean":"2024-09-13T08:52:02.742641+0000","last_became_active":"2024-09-13T08:52:01.316625+0000","last_became_peered":"2024-09-13T08:52:01.316625+0000","last_unstale":"2024-09-13T08:52:02.742641+0000","last_undegraded":"2024-09-13T08:52:02.742641+0000","last_fullsized":"2024-09-13T08:52:02.742641+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:17.107498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743502+0000","last_change":"2024-09-13T08:52:01.321436+0000","last_active":"2024-09-13T08:52:02.743502+0000","last_peered":"2024-09-13T08:52:02.743502+0000","last_clean":"2024-09-13T08:52:02.743502+0000","last_became_active":"2024-09-13T08:52:01.321179+0000","last_became_peered":"2024-09-13T08:52:01.321179+0000","last_unstale":"2024-09-13T08:52:02.743502+0000","last_undegraded":"2024-09-13T08:52:02.743502+0000","last_fullsized":"2024-09-13T08:52:02.743502+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:22:15.251227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742403+0000","last_change":"2024-09-13T08:52:01.320175+0000","last_active":"2024-09-13T08:52:02.742403+0000","last_peered":"2024-09-13T08:52:02.742403+0000","last_clean":"2024-09-13T08:52:02.742403+0000","last_became_active":"2024-09-13T08:52:01.317764+0000","last_became_peered":"2024-09-13T08:52:01.317764+0000","last_unstale":"2024-09-13T08:52:02.742403+0000","last_undegraded":"2024-09-13T08:52:02.742403+0000","last_fullsized":"2024-09-13T08:52:02.742403+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:44.605557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743111+0000","last_change":"2024-09-13T08:52:01.322915+0000","last_active":"2024-09-13T08:52:02.743111+0000","last_peered":"2024-09-13T08:52:02.743111+0000","last_clean":"2024-09-13T08:52:02.743111+0000","last_became_active":"2024-09-13T08:52:01.322616+0000","last_became_peered":"2024-09-13T08:52:01.322616+0000","last_unstale":"2024-09-13T08:52:02.743111+0000","last_undegraded":"2024-09-13T08:52:02.743111+0000","last_fullsized":"2024-09-13T08:52:02.743111+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.759066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315583+0000","last_change":"2024-09-13T08:52:01.302162+0000","last_active":"2024-09-13T08:52:02.315583+0000","last_peered":"2024-09-13T08:52:02.315583+0000","last_clean":"2024-09-13T08:52:02.315583+0000","last_became_active":"2024-09-13T08:52:01.301525+0000","last_became_peered":"2024-09-13T08:52:01.301525+0000","last_unstale":"2024-09-13T08:52:02.315583+0000","last_undegraded":"2024-09-13T08:52:02.315583+0000","last_fullsized":"2024-09-13T08:52:02.315583+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:32.349846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743400+0000","last_change":"2024-09-13T08:52:01.321288+0000","last_active":"2024-09-13T08:52:02.743400+0000","last_peered":"2024-09-13T08:52:02.743400+0000","last_clean":"2024-09-13T08:52:02.743400+0000","last_became_active":"2024-09-13T08:52:01.320996+0000","last_became_peered":"2024-09-13T08:52:01.320996+0000","last_unstale":"2024-09-13T08:52:02.743400+0000","last_undegraded":"2024-09-13T08:52:02.743400+0000","last_fullsized":"2024-09-13T08:52:02.743400+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:36:01.397745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311415+0000","last_change":"2024-09-13T08:52:01.327882+0000","last_active":"2024-09-13T08:52:02.311415+0000","last_peered":"2024-09-13T08:52:02.311415+0000","last_clean":"2024-09-13T08:52:02.311415+0000","last_became_active":"2024-09-13T08:52:01.327465+0000","last_became_peered":"2024-09-13T08:52:01.327465+0000","last_unstale":"2024-09-13T08:52:02.311415+0000","last_undegraded":"2024-09-13T08:52:02.311415+0000","last_fullsized":"2024-09-13T08:52:02.311415+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:00.614178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315546+0000","last_change":"2024-09-13T08:52:01.300377+0000","last_active":"2024-09-13T08:52:02.315546+0000","last_peered":"2024-09-13T08:52:02.315546+0000","last_clean":"2024-09-13T08:52:02.315546+0000","last_became_active":"2024-09-13T08:52:01.299895+0000","last_became_peered":"2024-09-13T08:52:01.299895+0000","last_unstale":"2024-09-13T08:52:02.315546+0000","last_undegraded":"2024-09-13T08:52:02.315546+0000","last_fullsized":"2024-09-13T08:52:02.315546+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:01:05.858267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741424+0000","last_change":"2024-09-13T08:52:01.310370+0000","last_active":"2024-09-13T08:52:02.741424+0000","last_peered":"2024-09-13T08:52:02.741424+0000","last_clean":"2024-09-13T08:52:02.741424+0000","last_became_active":"2024-09-13T08:52:01.310069+0000","last_became_peered":"2024-09-13T08:52:01.310069+0000","last_unstale":"2024-09-13T08:52:02.741424+0000","last_undegraded":"2024-09-13T08:52:02.741424+0000","last_fullsized":"2024-09-13T08:52:02.741424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:53.281383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742535+0000","last_change":"2024-09-13T08:52:01.327212+0000","last_active":"2024-09-13T08:52:02.742535+0000","last_peered":"2024-09-13T08:52:02.742535+0000","last_clean":"2024-09-13T08:52:02.742535+0000","last_became_active":"2024-09-13T08:52:01.326928+0000","last_became_peered":"2024-09-13T08:52:01.326928+0000","last_unstale":"2024-09-13T08:52:02.742535+0000","last_undegraded":"2024-09-13T08:52:02.742535+0000","last_fullsized":"2024-09-13T08:52:02.742535+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:59.345020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317786+0000","last_change":"2024-09-13T08:52:01.300357+0000","last_active":"2024-09-13T08:52:02.317786+0000","last_peered":"2024-09-13T08:52:02.317786+0000","last_clean":"2024-09-13T08:52:02.317786+0000","last_became_active":"2024-09-13T08:52:01.299967+0000","last_became_peered":"2024-09-13T08:52:01.299967+0000","last_unstale":"2024-09-13T08:52:02.317786+0000","last_undegraded":"2024-09-13T08:52:02.317786+0000","last_fullsized":"2024-09-13T08:52:02.317786+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:37.958421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316013+0000","last_change":"2024-09-13T08:52:01.306645+0000","last_active":"2024-09-13T08:52:02.316013+0000","last_peered":"2024-09-13T08:52:02.316013+0000","last_clean":"2024-09-13T08:52:02.316013+0000","last_became_active":"2024-09-13T08:52:01.306324+0000","last_became_peered":"2024-09-13T08:52:01.306324+0000","last_unstale":"2024-09-13T08:52:02.316013+0000","last_undegraded":"2024-09-13T08:52:02.316013+0000","last_fullsized":"2024-09-13T08:52:02.316013+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:57.164277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743469+0000","last_change":"2024-09-13T08:52:01.318429+0000","last_active":"2024-09-13T08:52:02.743469+0000","last_peered":"2024-09-13T08:52:02.743469+0000","last_clean":"2024-09-13T08:52:02.743469+0000","last_became_active":"2024-09-13T08:52:01.318131+0000","last_became_peered":"2024-09-13T08:52:01.318131+0000","last_unstale":"2024-09-13T08:52:02.743469+0000","last_undegraded":"2024-09-13T08:52:02.743469+0000","last_fullsized":"2024-09-13T08:52:02.743469+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:08.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742300+0000","last_change":"2024-09-13T08:52:01.315432+0000","last_active":"2024-09-13T08:52:02.742300+0000","last_peered":"2024-09-13T08:52:02.742300+0000","last_clean":"2024-09-13T08:52:02.742300+0000","last_became_active":"2024-09-13T08:52:01.315141+0000","last_became_peered":"2024-09-13T08:52:01.315141+0000","last_unstale":"2024-09-13T08:52:02.742300+0000","last_undegraded":"2024-09-13T08:52:02.742300+0000","last_fullsized":"2024-09-13T08:52:02.742300+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:07:31.291030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315985+0000","last_change":"2024-09-13T08:52:01.311484+0000","last_active":"2024-09-13T08:52:02.315985+0000","last_peered":"2024-09-13T08:52:02.315985+0000","last_clean":"2024-09-13T08:52:02.315985+0000","last_became_active":"2024-09-13T08:52:01.311200+0000","last_became_peered":"2024-09-13T08:52:01.311200+0000","last_unstale":"2024-09-13T08:52:02.315985+0000","last_undegraded":"2024-09-13T08:52:02.315985+0000","last_fullsized":"2024-09-13T08:52:02.315985+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:41.118368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315646+0000","last_change":"2024-09-13T08:52:01.316859+0000","last_active":"2024-09-13T08:52:02.315646+0000","last_peered":"2024-09-13T08:52:02.315646+0000","last_clean":"2024-09-13T08:52:02.315646+0000","last_became_active":"2024-09-13T08:52:01.316549+0000","last_became_peered":"2024-09-13T08:52:01.316549+0000","last_unstale":"2024-09-13T08:52:02.315646+0000","last_undegraded":"2024-09-13T08:52:02.315646+0000","last_fullsized":"2024-09-13T08:52:02.315646+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:10:02.167304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741628+0000","last_change":"2024-09-13T08:52:01.312535+0000","last_active":"2024-09-13T08:52:02.741628+0000","last_peered":"2024-09-13T08:52:02.741628+0000","last_clean":"2024-09-13T08:52:02.741628+0000","last_became_active":"2024-09-13T08:52:01.312206+0000","last_became_peered":"2024-09-13T08:52:01.312206+0000","last_unstale":"2024-09-13T08:52:02.741628+0000","last_undegraded":"2024-09-13T08:52:02.741628+0000","last_fullsized":"2024-09-13T08:52:02.741628+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:32.739615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741025+0000","last_change":"2024-09-13T08:52:01.310332+0000","last_active":"2024-09-13T08:52:02.741025+0000","last_peered":"2024-09-13T08:52:02.741025+0000","last_clean":"2024-09-13T08:52:02.741025+0000","last_became_active":"2024-09-13T08:52:01.309812+0000","last_became_peered":"2024-09-13T08:52:01.309812+0000","last_unstale":"2024-09-13T08:52:02.741025+0000","last_undegraded":"2024-09-13T08:52:02.741025+0000","last_fullsized":"2024-09-13T08:52:02.741025+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:40:56.189622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743492+0000","last_change":"2024-09-13T08:52:01.322757+0000","last_active":"2024-09-13T08:52:02.743492+0000","last_peered":"2024-09-13T08:52:02.743492+0000","last_clean":"2024-09-13T08:52:02.743492+0000","last_became_active":"2024-09-13T08:52:01.322499+0000","last_became_peered":"2024-09-13T08:52:01.322499+0000","last_unstale":"2024-09-13T08:52:02.743492+0000","last_undegraded":"2024-09-13T08:52:02.743492+0000","last_fullsized":"2024-09-13T08:52:02.743492+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:41.534273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"130'4","reported_seq":19,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741663+0000","last_change":"2024-09-13T08:52:00.309359+0000","last_active":"2024-09-13T08:52:02.741663+0000","last_peered":"2024-09-13T08:52:02.741663+0000","last_clean":"2024-09-13T08:52:02.741663+0000","last_became_active":"2024-09-13T08:52:00.308056+0000","last_became_peered":"2024-09-13T08:52:00.308056+0000","last_unstale":"2024-09-13T08:52:02.741663+0000","last_undegraded":"2024-09-13T08:52:02.741663+0000","last_fullsized":"2024-09-13T08:52:02.741663+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:36:21.582936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742453+0000","last_change":"2024-09-13T08:52:01.318029+0000","last_active":"2024-09-13T08:52:02.742453+0000","last_peered":"2024-09-13T08:52:02.742453+0000","last_clean":"2024-09-13T08:52:02.742453+0000","last_became_active":"2024-09-13T08:52:01.317483+0000","last_became_peered":"2024-09-13T08:52:01.317483+0000","last_unstale":"2024-09-13T08:52:02.742453+0000","last_undegraded":"2024-09-13T08:52:02.742453+0000","last_fullsized":"2024-09-13T08:52:02.742453+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:01:37.722600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"130'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742171+0000","last_change":"2024-09-13T08:52:00.305531+0000","last_active":"2024-09-13T08:52:02.742171+0000","last_peered":"2024-09-13T08:52:02.742171+0000","last_clean":"2024-09-13T08:52:02.742171+0000","last_became_active":"2024-09-13T08:52:00.305247+0000","last_became_peered":"2024-09-13T08:52:00.305247+0000","last_unstale":"2024-09-13T08:52:02.742171+0000","last_undegraded":"2024-09-13T08:52:02.742171+0000","last_fullsized":"2024-09-13T08:52:02.742171+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:45.296251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741664+0000","last_change":"2024-09-13T08:52:01.316971+0000","last_active":"2024-09-13T08:52:02.741664+0000","last_peered":"2024-09-13T08:52:02.741664+0000","last_clean":"2024-09-13T08:52:02.741664+0000","last_became_active":"2024-09-13T08:52:01.316688+0000","last_became_peered":"2024-09-13T08:52:01.316688+0000","last_unstale":"2024-09-13T08:52:02.741664+0000","last_undegraded":"2024-09-13T08:52:02.741664+0000","last_fullsized":"2024-09-13T08:52:02.741664+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:48:31.309365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317309+0000","last_change":"2024-09-13T08:52:00.307998+0000","last_active":"2024-09-13T08:52:02.317309+0000","last_peered":"2024-09-13T08:52:02.317309+0000","last_clean":"2024-09-13T08:52:02.317309+0000","last_became_active":"2024-09-13T08:52:00.307690+0000","last_became_peered":"2024-09-13T08:52:00.307690+0000","last_unstale":"2024-09-13T08:52:02.317309+0000","last_undegraded":"2024-09-13T08:52:02.317309+0000","last_fullsized":"2024-09-13T08:52:02.317309+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:17:54.277481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311646+0000","last_change":"2024-09-13T08:52:01.319805+0000","last_active":"2024-09-13T08:52:02.311646+0000","last_peered":"2024-09-13T08:52:02.311646+0000","last_clean":"2024-09-13T08:52:02.311646+0000","last_became_active":"2024-09-13T08:52:01.319399+0000","last_became_peered":"2024-09-13T08:52:01.319399+0000","last_unstale":"2024-09-13T08:52:02.311646+0000","last_undegraded":"2024-09-13T08:52:02.311646+0000","last_fullsized":"2024-09-13T08:52:02.311646+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:41.118368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316083+0000","last_change":"2024-09-13T08:52:00.302580+0000","last_active":"2024-09-13T08:52:02.316083+0000","last_peered":"2024-09-13T08:52:02.316083+0000","last_clean":"2024-09-13T08:52:02.316083+0000","last_became_active":"2024-09-13T08:52:00.302293+0000","last_became_peered":"2024-09-13T08:52:00.302293+0000","last_unstale":"2024-09-13T08:52:02.316083+0000","last_undegraded":"2024-09-13T08:52:02.316083+0000","last_fullsized":"2024-09-13T08:52:02.316083+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:00:20.752180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742989+0000","last_change":"2024-09-13T08:52:01.321225+0000","last_active":"2024-09-13T08:52:02.742989+0000","last_peered":"2024-09-13T08:52:02.742989+0000","last_clean":"2024-09-13T08:52:02.742989+0000","last_became_active":"2024-09-13T08:52:01.320936+0000","last_became_peered":"2024-09-13T08:52:01.320936+0000","last_unstale":"2024-09-13T08:52:02.742989+0000","last_undegraded":"2024-09-13T08:52:02.742989+0000","last_fullsized":"2024-09-13T08:52:02.742989+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:25.402433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"130'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315264+0000","last_change":"2024-09-13T08:52:00.301831+0000","last_active":"2024-09-13T08:52:02.315264+0000","last_peered":"2024-09-13T08:52:02.315264+0000","last_clean":"2024-09-13T08:52:02.315264+0000","last_became_active":"2024-09-13T08:52:00.301550+0000","last_became_peered":"2024-09-13T08:52:00.301550+0000","last_unstale":"2024-09-13T08:52:02.315264+0000","last_undegraded":"2024-09-13T08:52:02.315264+0000","last_fullsized":"2024-09-13T08:52:02.315264+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:05.223602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311513+0000","last_change":"2024-09-13T08:52:01.327795+0000","last_active":"2024-09-13T08:52:02.311513+0000","last_peered":"2024-09-13T08:52:02.311513+0000","last_clean":"2024-09-13T08:52:02.311513+0000","last_became_active":"2024-09-13T08:52:01.326730+0000","last_became_peered":"2024-09-13T08:52:01.326730+0000","last_unstale":"2024-09-13T08:52:02.311513+0000","last_undegraded":"2024-09-13T08:52:02.311513+0000","last_fullsized":"2024-09-13T08:52:02.311513+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:20:01.148707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"130'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743105+0000","last_change":"2024-09-13T08:52:00.305456+0000","last_active":"2024-09-13T08:52:02.743105+0000","last_peered":"2024-09-13T08:52:02.743105+0000","last_clean":"2024-09-13T08:52:02.743105+0000","last_became_active":"2024-09-13T08:52:00.305190+0000","last_became_peered":"2024-09-13T08:52:00.305190+0000","last_unstale":"2024-09-13T08:52:02.743105+0000","last_undegraded":"2024-09-13T08:52:02.743105+0000","last_fullsized":"2024-09-13T08:52:02.743105+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:52.271592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311370+0000","last_change":"2024-09-13T08:52:01.323808+0000","last_active":"2024-09-13T08:52:02.311370+0000","last_peered":"2024-09-13T08:52:02.311370+0000","last_clean":"2024-09-13T08:52:02.311370+0000","last_became_active":"2024-09-13T08:52:01.322660+0000","last_became_peered":"2024-09-13T08:52:01.322660+0000","last_unstale":"2024-09-13T08:52:02.311370+0000","last_undegraded":"2024-09-13T08:52:02.311370+0000","last_fullsized":"2024-09-13T08:52:02.311370+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:22.751896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741676+0000","last_change":"2024-09-13T08:52:00.304348+0000","last_active":"2024-09-13T08:52:02.741676+0000","last_peered":"2024-09-13T08:52:02.741676+0000","last_clean":"2024-09-13T08:52:02.741676+0000","last_became_active":"2024-09-13T08:52:00.304098+0000","last_became_peered":"2024-09-13T08:52:00.304098+0000","last_unstale":"2024-09-13T08:52:02.741676+0000","last_undegraded":"2024-09-13T08:52:02.741676+0000","last_fullsized":"2024-09-13T08:52:02.741676+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:46.449811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315496+0000","last_change":"2024-09-13T08:52:01.298285+0000","last_active":"2024-09-13T08:52:02.315496+0000","last_peered":"2024-09-13T08:52:02.315496+0000","last_clean":"2024-09-13T08:52:02.315496+0000","last_became_active":"2024-09-13T08:52:01.297963+0000","last_became_peered":"2024-09-13T08:52:01.297963+0000","last_unstale":"2024-09-13T08:52:02.315496+0000","last_undegraded":"2024-09-13T08:52:02.315496+0000","last_fullsized":"2024-09-13T08:52:02.315496+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:01.152580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315102+0000","last_change":"2024-09-13T08:52:00.304666+0000","last_active":"2024-09-13T08:52:02.315102+0000","last_peered":"2024-09-13T08:52:02.315102+0000","last_clean":"2024-09-13T08:52:02.315102+0000","last_became_active":"2024-09-13T08:52:00.303812+0000","last_became_peered":"2024-09-13T08:52:00.303812+0000","last_unstale":"2024-09-13T08:52:02.315102+0000","last_undegraded":"2024-09-13T08:52:02.315102+0000","last_fullsized":"2024-09-13T08:52:02.315102+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:28:38.371986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316251+0000","last_change":"2024-09-13T08:52:01.300199+0000","last_active":"2024-09-13T08:52:02.316251+0000","last_peered":"2024-09-13T08:52:02.316251+0000","last_clean":"2024-09-13T08:52:02.316251+0000","last_became_active":"2024-09-13T08:52:01.299819+0000","last_became_peered":"2024-09-13T08:52:01.299819+0000","last_unstale":"2024-09-13T08:52:02.316251+0000","last_undegraded":"2024-09-13T08:52:02.316251+0000","last_fullsized":"2024-09-13T08:52:02.316251+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:54.373221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311827+0000","last_change":"2024-09-13T08:52:00.313505+0000","last_active":"2024-09-13T08:52:02.311827+0000","last_peered":"2024-09-13T08:52:02.311827+0000","last_clean":"2024-09-13T08:52:02.311827+0000","last_became_active":"2024-09-13T08:52:00.312377+0000","last_became_peered":"2024-09-13T08:52:00.312377+0000","last_unstale":"2024-09-13T08:52:02.311827+0000","last_undegraded":"2024-09-13T08:52:02.311827+0000","last_fullsized":"2024-09-13T08:52:02.311827+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:14.867528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741575+0000","last_change":"2024-09-13T08:52:01.312669+0000","last_active":"2024-09-13T08:52:02.741575+0000","last_peered":"2024-09-13T08:52:02.741575+0000","last_clean":"2024-09-13T08:52:02.741575+0000","last_became_active":"2024-09-13T08:52:01.312321+0000","last_became_peered":"2024-09-13T08:52:01.312321+0000","last_unstale":"2024-09-13T08:52:02.741575+0000","last_undegraded":"2024-09-13T08:52:02.741575+0000","last_fullsized":"2024-09-13T08:52:02.741575+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:18:55.267451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743247+0000","last_change":"2024-09-13T08:52:00.309464+0000","last_active":"2024-09-13T08:52:02.743247+0000","last_peered":"2024-09-13T08:52:02.743247+0000","last_clean":"2024-09-13T08:52:02.743247+0000","last_became_active":"2024-09-13T08:52:00.309169+0000","last_became_peered":"2024-09-13T08:52:00.309169+0000","last_unstale":"2024-09-13T08:52:02.743247+0000","last_undegraded":"2024-09-13T08:52:02.743247+0000","last_fullsized":"2024-09-13T08:52:02.743247+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:13.173905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311891+0000","last_change":"2024-09-13T08:52:01.320481+0000","last_active":"2024-09-13T08:52:02.311891+0000","last_peered":"2024-09-13T08:52:02.311891+0000","last_clean":"2024-09-13T08:52:02.311891+0000","last_became_active":"2024-09-13T08:52:01.320084+0000","last_became_peered":"2024-09-13T08:52:01.320084+0000","last_unstale":"2024-09-13T08:52:02.311891+0000","last_undegraded":"2024-09-13T08:52:02.311891+0000","last_fullsized":"2024-09-13T08:52:02.311891+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:40.653366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"130'3","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741201+0000","last_change":"2024-09-13T08:52:00.308398+0000","last_active":"2024-09-13T08:52:02.741201+0000","last_peered":"2024-09-13T08:52:02.741201+0000","last_clean":"2024-09-13T08:52:02.741201+0000","last_became_active":"2024-09-13T08:52:00.307459+0000","last_became_peered":"2024-09-13T08:52:00.307459+0000","last_unstale":"2024-09-13T08:52:02.741201+0000","last_undegraded":"2024-09-13T08:52:02.741201+0000","last_fullsized":"2024-09-13T08:52:02.741201+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:00:20.752180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742172+0000","last_change":"2024-09-13T08:52:01.321620+0000","last_active":"2024-09-13T08:52:02.742172+0000","last_peered":"2024-09-13T08:52:02.742172+0000","last_clean":"2024-09-13T08:52:02.742172+0000","last_became_active":"2024-09-13T08:52:01.320943+0000","last_became_peered":"2024-09-13T08:52:01.320943+0000","last_unstale":"2024-09-13T08:52:02.742172+0000","last_undegraded":"2024-09-13T08:52:02.742172+0000","last_fullsized":"2024-09-13T08:52:02.742172+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:43:35.081647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"130'6","reported_seq":21,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742358+0000","last_change":"2024-09-13T08:52:00.304788+0000","last_active":"2024-09-13T08:52:02.742358+0000","last_peered":"2024-09-13T08:52:02.742358+0000","last_clean":"2024-09-13T08:52:02.742358+0000","last_became_active":"2024-09-13T08:52:00.304499+0000","last_became_peered":"2024-09-13T08:52:00.304499+0000","last_unstale":"2024-09-13T08:52:02.742358+0000","last_undegraded":"2024-09-13T08:52:02.742358+0000","last_fullsized":"2024-09-13T08:52:02.742358+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:44.767320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743047+0000","last_change":"2024-09-13T08:52:01.315708+0000","last_active":"2024-09-13T08:52:02.743047+0000","last_peered":"2024-09-13T08:52:02.743047+0000","last_clean":"2024-09-13T08:52:02.743047+0000","last_became_active":"2024-09-13T08:52:01.315427+0000","last_became_peered":"2024-09-13T08:52:01.315427+0000","last_unstale":"2024-09-13T08:52:02.743047+0000","last_undegraded":"2024-09-13T08:52:02.743047+0000","last_fullsized":"2024-09-13T08:52:02.743047+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:52.518065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742613+0000","last_change":"2024-09-13T08:52:00.306576+0000","last_active":"2024-09-13T08:52:02.742613+0000","last_peered":"2024-09-13T08:52:02.742613+0000","last_clean":"2024-09-13T08:52:02.742613+0000","last_became_active":"2024-09-13T08:52:00.305457+0000","last_became_peered":"2024-09-13T08:52:00.305457+0000","last_unstale":"2024-09-13T08:52:02.742613+0000","last_undegraded":"2024-09-13T08:52:02.742613+0000","last_fullsized":"2024-09-13T08:52:02.742613+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:55.302117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742431+0000","last_change":"2024-09-13T08:52:01.320474+0000","last_active":"2024-09-13T08:52:02.742431+0000","last_peered":"2024-09-13T08:52:02.742431+0000","last_clean":"2024-09-13T08:52:02.742431+0000","last_became_active":"2024-09-13T08:52:01.320146+0000","last_became_peered":"2024-09-13T08:52:01.320146+0000","last_unstale":"2024-09-13T08:52:02.742431+0000","last_undegraded":"2024-09-13T08:52:02.742431+0000","last_fullsized":"2024-09-13T08:52:02.742431+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:55:51.095792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316786+0000","last_change":"2024-09-13T08:52:00.301084+0000","last_active":"2024-09-13T08:52:02.316786+0000","last_peered":"2024-09-13T08:52:02.316786+0000","last_clean":"2024-09-13T08:52:02.316786+0000","last_became_active":"2024-09-13T08:52:00.300449+0000","last_became_peered":"2024-09-13T08:52:00.300449+0000","last_unstale":"2024-09-13T08:52:02.316786+0000","last_undegraded":"2024-09-13T08:52:02.316786+0000","last_fullsized":"2024-09-13T08:52:02.316786+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:40.768515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311291+0000","last_change":"2024-09-13T08:52:01.323667+0000","last_active":"2024-09-13T08:52:02.311291+0000","last_peered":"2024-09-13T08:52:02.311291+0000","last_clean":"2024-09-13T08:52:02.311291+0000","last_became_active":"2024-09-13T08:52:01.323392+0000","last_became_peered":"2024-09-13T08:52:01.323392+0000","last_unstale":"2024-09-13T08:52:02.311291+0000","last_undegraded":"2024-09-13T08:52:02.311291+0000","last_fullsized":"2024-09-13T08:52:02.311291+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:45:28.172536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"130'8","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311054+0000","last_change":"2024-09-13T08:52:00.311593+0000","last_active":"2024-09-13T08:52:02.311054+0000","last_peered":"2024-09-13T08:52:02.311054+0000","last_clean":"2024-09-13T08:52:02.311054+0000","last_became_active":"2024-09-13T08:52:00.309664+0000","last_became_peered":"2024-09-13T08:52:00.309664+0000","last_unstale":"2024-09-13T08:52:02.311054+0000","last_undegraded":"2024-09-13T08:52:02.311054+0000","last_fullsized":"2024-09-13T08:52:02.311054+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:27.749275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741314+0000","last_change":"2024-09-13T08:52:01.318196+0000","last_active":"2024-09-13T08:52:02.741314+0000","last_peered":"2024-09-13T08:52:02.741314+0000","last_clean":"2024-09-13T08:52:02.741314+0000","last_became_active":"2024-09-13T08:52:01.316334+0000","last_became_peered":"2024-09-13T08:52:01.316334+0000","last_unstale":"2024-09-13T08:52:02.741314+0000","last_undegraded":"2024-09-13T08:52:02.741314+0000","last_fullsized":"2024-09-13T08:52:02.741314+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:54.078357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"130'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311879+0000","last_change":"2024-09-13T08:52:00.303719+0000","last_active":"2024-09-13T08:52:02.311879+0000","last_peered":"2024-09-13T08:52:02.311879+0000","last_clean":"2024-09-13T08:52:02.311879+0000","last_became_active":"2024-09-13T08:52:00.303414+0000","last_became_peered":"2024-09-13T08:52:00.303414+0000","last_unstale":"2024-09-13T08:52:02.311879+0000","last_undegraded":"2024-09-13T08:52:02.311879+0000","last_fullsized":"2024-09-13T08:52:02.311879+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:29:26.702947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741910+0000","last_change":"2024-09-13T08:52:01.311936+0000","last_active":"2024-09-13T08:52:02.741910+0000","last_peered":"2024-09-13T08:52:02.741910+0000","last_clean":"2024-09-13T08:52:02.741910+0000","last_became_active":"2024-09-13T08:52:01.311521+0000","last_became_peered":"2024-09-13T08:52:01.311521+0000","last_unstale":"2024-09-13T08:52:02.741910+0000","last_undegraded":"2024-09-13T08:52:02.741910+0000","last_fullsized":"2024-09-13T08:52:02.741910+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:01.152580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317148+0000","last_change":"2024-09-13T08:52:00.304308+0000","last_active":"2024-09-13T08:52:02.317148+0000","last_peered":"2024-09-13T08:52:02.317148+0000","last_clean":"2024-09-13T08:52:02.317148+0000","last_became_active":"2024-09-13T08:52:00.303052+0000","last_became_peered":"2024-09-13T08:52:00.303052+0000","last_unstale":"2024-09-13T08:52:02.317148+0000","last_undegraded":"2024-09-13T08:52:02.317148+0000","last_fullsized":"2024-09-13T08:52:02.317148+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:22.484472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316336+0000","last_change":"2024-09-13T08:52:01.314222+0000","last_active":"2024-09-13T08:52:02.316336+0000","last_peered":"2024-09-13T08:52:02.316336+0000","last_clean":"2024-09-13T08:52:02.316336+0000","last_became_active":"2024-09-13T08:52:01.313910+0000","last_became_peered":"2024-09-13T08:52:01.313910+0000","last_unstale":"2024-09-13T08:52:02.316336+0000","last_undegraded":"2024-09-13T08:52:02.316336+0000","last_fullsized":"2024-09-13T08:52:02.316336+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:51.220023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"130'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741902+0000","last_change":"2024-09-13T08:52:00.305737+0000","last_active":"2024-09-13T08:52:02.741902+0000","last_peered":"2024-09-13T08:52:02.741902+0000","last_clean":"2024-09-13T08:52:02.741902+0000","last_became_active":"2024-09-13T08:52:00.305460+0000","last_became_peered":"2024-09-13T08:52:00.305460+0000","last_unstale":"2024-09-13T08:52:02.741902+0000","last_undegraded":"2024-09-13T08:52:02.741902+0000","last_fullsized":"2024-09-13T08:52:02.741902+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:27.642818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315790+0000","last_change":"2024-09-13T08:52:01.300837+0000","last_active":"2024-09-13T08:52:02.315790+0000","last_peered":"2024-09-13T08:52:02.315790+0000","last_clean":"2024-09-13T08:52:02.315790+0000","last_became_active":"2024-09-13T08:52:01.300548+0000","last_became_peered":"2024-09-13T08:52:01.300548+0000","last_unstale":"2024-09-13T08:52:02.315790+0000","last_undegraded":"2024-09-13T08:52:02.315790+0000","last_fullsized":"2024-09-13T08:52:02.315790+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:37.434546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741071+0000","last_change":"2024-09-13T08:52:00.308406+0000","last_active":"2024-09-13T08:52:02.741071+0000","last_peered":"2024-09-13T08:52:02.741071+0000","last_clean":"2024-09-13T08:52:02.741071+0000","last_became_active":"2024-09-13T08:52:00.308066+0000","last_became_peered":"2024-09-13T08:52:00.308066+0000","last_unstale":"2024-09-13T08:52:02.741071+0000","last_undegraded":"2024-09-13T08:52:02.741071+0000","last_fullsized":"2024-09-13T08:52:02.741071+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:32.849131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315765+0000","last_change":"2024-09-13T08:52:01.314562+0000","last_active":"2024-09-13T08:52:02.315765+0000","last_peered":"2024-09-13T08:52:02.315765+0000","last_clean":"2024-09-13T08:52:02.315765+0000","last_became_active":"2024-09-13T08:52:01.314274+0000","last_became_peered":"2024-09-13T08:52:01.314274+0000","last_unstale":"2024-09-13T08:52:02.315765+0000","last_undegraded":"2024-09-13T08:52:02.315765+0000","last_fullsized":"2024-09-13T08:52:02.315765+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:15.167781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741902+0000","last_change":"2024-09-13T08:52:00.306739+0000","last_active":"2024-09-13T08:52:02.741902+0000","last_peered":"2024-09-13T08:52:02.741902+0000","last_clean":"2024-09-13T08:52:02.741902+0000","last_became_active":"2024-09-13T08:52:00.306395+0000","last_became_peered":"2024-09-13T08:52:00.306395+0000","last_unstale":"2024-09-13T08:52:02.741902+0000","last_undegraded":"2024-09-13T08:52:02.741902+0000","last_fullsized":"2024-09-13T08:52:02.741902+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:11:19.672649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741866+0000","last_change":"2024-09-13T08:52:01.313473+0000","last_active":"2024-09-13T08:52:02.741866+0000","last_peered":"2024-09-13T08:52:02.741866+0000","last_clean":"2024-09-13T08:52:02.741866+0000","last_became_active":"2024-09-13T08:52:01.313205+0000","last_became_peered":"2024-09-13T08:52:01.313205+0000","last_unstale":"2024-09-13T08:52:02.741866+0000","last_undegraded":"2024-09-13T08:52:02.741866+0000","last_fullsized":"2024-09-13T08:52:02.741866+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:00.109380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743189+0000","last_change":"2024-09-13T08:52:00.311117+0000","last_active":"2024-09-13T08:52:02.743189+0000","last_peered":"2024-09-13T08:52:02.743189+0000","last_clean":"2024-09-13T08:52:02.743189+0000","last_became_active":"2024-09-13T08:52:00.310555+0000","last_became_peered":"2024-09-13T08:52:00.310555+0000","last_unstale":"2024-09-13T08:52:02.743189+0000","last_undegraded":"2024-09-13T08:52:02.743189+0000","last_fullsized":"2024-09-13T08:52:02.743189+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:31.729824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742640+0000","last_change":"2024-09-13T08:52:01.316637+0000","last_active":"2024-09-13T08:52:02.742640+0000","last_peered":"2024-09-13T08:52:02.742640+0000","last_clean":"2024-09-13T08:52:02.742640+0000","last_became_active":"2024-09-13T08:52:01.316356+0000","last_became_peered":"2024-09-13T08:52:01.316356+0000","last_unstale":"2024-09-13T08:52:02.742640+0000","last_undegraded":"2024-09-13T08:52:02.742640+0000","last_fullsized":"2024-09-13T08:52:02.742640+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:46.110526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743090+0000","last_change":"2024-09-13T08:52:00.314043+0000","last_active":"2024-09-13T08:52:02.743090+0000","last_peered":"2024-09-13T08:52:02.743090+0000","last_clean":"2024-09-13T08:52:02.743090+0000","last_became_active":"2024-09-13T08:52:00.313702+0000","last_became_peered":"2024-09-13T08:52:00.313702+0000","last_unstale":"2024-09-13T08:52:02.743090+0000","last_undegraded":"2024-09-13T08:52:02.743090+0000","last_fullsized":"2024-09-13T08:52:02.743090+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:40.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311139+0000","last_change":"2024-09-13T08:52:01.323589+0000","last_active":"2024-09-13T08:52:02.311139+0000","last_peered":"2024-09-13T08:52:02.311139+0000","last_clean":"2024-09-13T08:52:02.311139+0000","last_became_active":"2024-09-13T08:52:01.323364+0000","last_became_peered":"2024-09-13T08:52:01.323364+0000","last_unstale":"2024-09-13T08:52:02.311139+0000","last_undegraded":"2024-09-13T08:52:02.311139+0000","last_fullsized":"2024-09-13T08:52:02.311139+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:43.374921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316681+0000","last_change":"2024-09-13T08:52:00.307654+0000","last_active":"2024-09-13T08:52:02.316681+0000","last_peered":"2024-09-13T08:52:02.316681+0000","last_clean":"2024-09-13T08:52:02.316681+0000","last_became_active":"2024-09-13T08:52:00.307314+0000","last_became_peered":"2024-09-13T08:52:00.307314+0000","last_unstale":"2024-09-13T08:52:02.316681+0000","last_undegraded":"2024-09-13T08:52:02.316681+0000","last_fullsized":"2024-09-13T08:52:02.316681+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:39.643575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742632+0000","last_change":"2024-09-13T08:52:01.329275+0000","last_active":"2024-09-13T08:52:02.742632+0000","last_peered":"2024-09-13T08:52:02.742632+0000","last_clean":"2024-09-13T08:52:02.742632+0000","last_became_active":"2024-09-13T08:52:01.327989+0000","last_became_peered":"2024-09-13T08:52:01.327989+0000","last_unstale":"2024-09-13T08:52:02.742632+0000","last_undegraded":"2024-09-13T08:52:02.742632+0000","last_fullsized":"2024-09-13T08:52:02.742632+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:37:36.161702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743542+0000","last_change":"2024-09-13T08:52:00.312748+0000","last_active":"2024-09-13T08:52:02.743542+0000","last_peered":"2024-09-13T08:52:02.743542+0000","last_clean":"2024-09-13T08:52:02.743542+0000","last_became_active":"2024-09-13T08:52:00.312464+0000","last_became_peered":"2024-09-13T08:52:00.312464+0000","last_unstale":"2024-09-13T08:52:02.743542+0000","last_undegraded":"2024-09-13T08:52:02.743542+0000","last_fullsized":"2024-09-13T08:52:02.743542+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:43:34.071856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316379+0000","last_change":"2024-09-13T08:52:01.319213+0000","last_active":"2024-09-13T08:52:02.316379+0000","last_peered":"2024-09-13T08:52:02.316379+0000","last_clean":"2024-09-13T08:52:02.316379+0000","last_became_active":"2024-09-13T08:52:01.318806+0000","last_became_peered":"2024-09-13T08:52:01.318806+0000","last_unstale":"2024-09-13T08:52:02.316379+0000","last_undegraded":"2024-09-13T08:52:02.316379+0000","last_fullsized":"2024-09-13T08:52:02.316379+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:08.286690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"131'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741422+0000","last_change":"2024-09-13T08:52:00.309662+0000","last_active":"2024-09-13T08:52:02.741422+0000","last_peered":"2024-09-13T08:52:02.741422+0000","last_clean":"2024-09-13T08:52:02.741422+0000","last_became_active":"2024-09-13T08:52:00.309379+0000","last_became_peered":"2024-09-13T08:52:00.309379+0000","last_unstale":"2024-09-13T08:52:02.741422+0000","last_undegraded":"2024-09-13T08:52:02.741422+0000","last_fullsized":"2024-09-13T08:52:02.741422+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:44:57.715520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743621+0000","last_change":"2024-09-13T08:52:01.315634+0000","last_active":"2024-09-13T08:52:02.743621+0000","last_peered":"2024-09-13T08:52:02.743621+0000","last_clean":"2024-09-13T08:52:02.743621+0000","last_became_active":"2024-09-13T08:52:01.315262+0000","last_became_peered":"2024-09-13T08:52:01.315262+0000","last_unstale":"2024-09-13T08:52:02.743621+0000","last_undegraded":"2024-09-13T08:52:02.743621+0000","last_fullsized":"2024-09-13T08:52:02.743621+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:30.042810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742822+0000","last_change":"2024-09-13T08:52:00.314269+0000","last_active":"2024-09-13T08:52:02.742822+0000","last_peered":"2024-09-13T08:52:02.742822+0000","last_clean":"2024-09-13T08:52:02.742822+0000","last_became_active":"2024-09-13T08:52:00.313830+0000","last_became_peered":"2024-09-13T08:52:00.313830+0000","last_unstale":"2024-09-13T08:52:02.742822+0000","last_undegraded":"2024-09-13T08:52:02.742822+0000","last_fullsized":"2024-09-13T08:52:02.742822+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:48:59.099589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741900+0000","last_change":"2024-09-13T08:52:01.314242+0000","last_active":"2024-09-13T08:52:02.741900+0000","last_peered":"2024-09-13T08:52:02.741900+0000","last_clean":"2024-09-13T08:52:02.741900+0000","last_became_active":"2024-09-13T08:52:01.313973+0000","last_became_peered":"2024-09-13T08:52:01.313973+0000","last_unstale":"2024-09-13T08:52:02.741900+0000","last_undegraded":"2024-09-13T08:52:02.741900+0000","last_fullsized":"2024-09-13T08:52:02.741900+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:55:28.898816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743017+0000","last_change":"2024-09-13T08:52:00.313237+0000","last_active":"2024-09-13T08:52:02.743017+0000","last_peered":"2024-09-13T08:52:02.743017+0000","last_clean":"2024-09-13T08:52:02.743017+0000","last_became_active":"2024-09-13T08:52:00.312932+0000","last_became_peered":"2024-09-13T08:52:00.312932+0000","last_unstale":"2024-09-13T08:52:02.743017+0000","last_undegraded":"2024-09-13T08:52:02.743017+0000","last_fullsized":"2024-09-13T08:52:02.743017+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:29:23.309222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311228+0000","last_change":"2024-09-13T08:52:01.327132+0000","last_active":"2024-09-13T08:52:02.311228+0000","last_peered":"2024-09-13T08:52:02.311228+0000","last_clean":"2024-09-13T08:52:02.311228+0000","last_became_active":"2024-09-13T08:52:01.326305+0000","last_became_peered":"2024-09-13T08:52:01.326305+0000","last_unstale":"2024-09-13T08:52:02.311228+0000","last_undegraded":"2024-09-13T08:52:02.311228+0000","last_fullsized":"2024-09-13T08:52:02.311228+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:32:34.733024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315950+0000","last_change":"2024-09-13T08:52:00.305900+0000","last_active":"2024-09-13T08:52:02.315950+0000","last_peered":"2024-09-13T08:52:02.315950+0000","last_clean":"2024-09-13T08:52:02.315950+0000","last_became_active":"2024-09-13T08:52:00.305583+0000","last_became_peered":"2024-09-13T08:52:00.305583+0000","last_unstale":"2024-09-13T08:52:02.315950+0000","last_undegraded":"2024-09-13T08:52:02.315950+0000","last_fullsized":"2024-09-13T08:52:02.315950+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:38.297880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311618+0000","last_change":"2024-09-13T08:52:01.328194+0000","last_active":"2024-09-13T08:52:02.311618+0000","last_peered":"2024-09-13T08:52:02.311618+0000","last_clean":"2024-09-13T08:52:02.311618+0000","last_became_active":"2024-09-13T08:52:01.327585+0000","last_became_peered":"2024-09-13T08:52:01.327585+0000","last_unstale":"2024-09-13T08:52:02.311618+0000","last_undegraded":"2024-09-13T08:52:02.311618+0000","last_fullsized":"2024-09-13T08:52:02.311618+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:27.166328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311279+0000","last_change":"2024-09-13T08:52:00.308764+0000","last_active":"2024-09-13T08:52:02.311279+0000","last_peered":"2024-09-13T08:52:02.311279+0000","last_clean":"2024-09-13T08:52:02.311279+0000","last_became_active":"2024-09-13T08:52:00.308488+0000","last_became_peered":"2024-09-13T08:52:00.308488+0000","last_unstale":"2024-09-13T08:52:02.311279+0000","last_undegraded":"2024-09-13T08:52:02.311279+0000","last_fullsized":"2024-09-13T08:52:02.311279+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:36:00.387954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742785+0000","last_change":"2024-09-13T08:52:01.327312+0000","last_active":"2024-09-13T08:52:02.742785+0000","last_peered":"2024-09-13T08:52:02.742785+0000","last_clean":"2024-09-13T08:52:02.742785+0000","last_became_active":"2024-09-13T08:52:01.326968+0000","last_became_peered":"2024-09-13T08:52:01.326968+0000","last_unstale":"2024-09-13T08:52:02.742785+0000","last_undegraded":"2024-09-13T08:52:02.742785+0000","last_fullsized":"2024-09-13T08:52:02.742785+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:35:05.037339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316895+0000","last_change":"2024-09-13T08:52:00.301381+0000","last_active":"2024-09-13T08:52:02.316895+0000","last_peered":"2024-09-13T08:52:02.316895+0000","last_clean":"2024-09-13T08:52:02.316895+0000","last_became_active":"2024-09-13T08:52:00.300814+0000","last_became_peered":"2024-09-13T08:52:00.300814+0000","last_unstale":"2024-09-13T08:52:02.316895+0000","last_undegraded":"2024-09-13T08:52:02.316895+0000","last_fullsized":"2024-09-13T08:52:02.316895+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:30:11.266958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742954+0000","last_change":"2024-09-13T08:52:01.312785+0000","last_active":"2024-09-13T08:52:02.742954+0000","last_peered":"2024-09-13T08:52:02.742954+0000","last_clean":"2024-09-13T08:52:02.742954+0000","last_became_active":"2024-09-13T08:52:01.312502+0000","last_became_peered":"2024-09-13T08:52:01.312502+0000","last_unstale":"2024-09-13T08:52:02.742954+0000","last_undegraded":"2024-09-13T08:52:02.742954+0000","last_fullsized":"2024-09-13T08:52:02.742954+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:54:50.643171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742751+0000","last_change":"2024-09-13T08:52:00.305996+0000","last_active":"2024-09-13T08:52:02.742751+0000","last_peered":"2024-09-13T08:52:02.742751+0000","last_clean":"2024-09-13T08:52:02.742751+0000","last_became_active":"2024-09-13T08:52:00.305186+0000","last_became_peered":"2024-09-13T08:52:00.305186+0000","last_unstale":"2024-09-13T08:52:02.742751+0000","last_undegraded":"2024-09-13T08:52:02.742751+0000","last_fullsized":"2024-09-13T08:52:02.742751+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:55.693460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741547+0000","last_change":"2024-09-13T08:52:01.304641+0000","last_active":"2024-09-13T08:52:02.741547+0000","last_peered":"2024-09-13T08:52:02.741547+0000","last_clean":"2024-09-13T08:52:02.741547+0000","last_became_active":"2024-09-13T08:52:01.304281+0000","last_became_peered":"2024-09-13T08:52:01.304281+0000","last_unstale":"2024-09-13T08:52:02.741547+0000","last_undegraded":"2024-09-13T08:52:02.741547+0000","last_fullsized":"2024-09-13T08:52:02.741547+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:16:02.732823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311717+0000","last_change":"2024-09-13T08:52:00.308341+0000","last_active":"2024-09-13T08:52:02.311717+0000","last_peered":"2024-09-13T08:52:02.311717+0000","last_clean":"2024-09-13T08:52:02.311717+0000","last_became_active":"2024-09-13T08:52:00.308032+0000","last_became_peered":"2024-09-13T08:52:00.308032+0000","last_unstale":"2024-09-13T08:52:02.311717+0000","last_undegraded":"2024-09-13T08:52:02.311717+0000","last_fullsized":"2024-09-13T08:52:02.311717+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:46:27.815150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":321,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.740679+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:52:02.740679+0000","last_peered":"2024-09-13T08:52:02.740679+0000","last_clean":"2024-09-13T08:52:02.740679+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:52:02.740679+0000","last_undegraded":"2024-09-13T08:52:02.740679+0000","last_fullsized":"2024-09-13T08:52:02.740679+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509618,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127124,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244716,"statfs":{"total":96636764160,"available":96506589184,"internally_reserved":0,"allocated":1630208,"data_stored":717698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509618,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131864,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239976,"statfs":{"total":96636764160,"available":96501735424,"internally_reserved":0,"allocated":2232320,"data_stored":1308774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509619,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149328,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222512,"statfs":{"total":96636764160,"available":96483852288,"internally_reserved":0,"allocated":1626112,"data_stored":717664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509619,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131280,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240560,"statfs":{"total":96636764160,"available":96502333440,"internally_reserved":0,"allocated":1630208,"data_stored":717734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509618,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127128,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244712,"statfs":{"total":96636764160,"available":96506585088,"internally_reserved":0,"allocated":1634304,"data_stored":718336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509620,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145764,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226076,"statfs":{"total":96636764160,"available":96487501824,"internally_reserved":0,"allocated":2236416,"data_stored":1309666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509618,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145188,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226652,"statfs":{"total":96636764160,"available":96488091648,"internally_reserved":0,"allocated":1646592,"data_stored":719332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542324,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136840,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235000,"statfs":{"total":96636764160,"available":96496640000,"internally_reserved":0,"allocated":1626112,"data_stored":717664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:52:05.466 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:52:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.653+0000 7fd53e600640 1 -- 172.21.15.44:0/9729830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd520004810 msgr2=0x7fd520004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.653+0000 7fd53e600640 1 --2- 172.21.15.44:0/9729830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd520004810 0x7fd520004c90 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fd5240016f0 tx=0x7fd52401c750 comp rx=0 tx=0).stop 2024-09-13T08:52:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.653+0000 7fd53e600640 1 -- 172.21.15.44:0/9729830 shutdown_connections 2024-09-13T08:52:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.653+0000 7fd53e600640 1 --2- 172.21.15.44:0/9729830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd520004810 0x7fd520004c90 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.653+0000 7fd53e600640 1 -- 172.21.15.44:0/9729830 >> 172.21.15.44:0/9729830 conn(0x7fd5380d1be0 msgr2=0x7fd5380d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.653+0000 7fd53e600640 1 -- 172.21.15.44:0/9729830 shutdown_connections 2024-09-13T08:52:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.653+0000 7fd53e600640 1 -- 172.21.15.44:0/9729830 wait complete. 2024-09-13T08:52:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53e600640 1 Processor -- start 2024-09-13T08:52:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53e600640 1 -- start start 2024-09-13T08:52:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53e600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd520004810 0x7fd5381f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53e600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5381f4990 0x7fd5380b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd53806e460 0x7fd53806e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd538070980 con 0x7fd53806e460 2024-09-13T08:52:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53e600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd538072410 con 0x7fd5381f4990 2024-09-13T08:52:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53e600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd538070b00 con 0x7fd520004810 2024-09-13T08:52:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd53806e460 0x7fd53806e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd53806e460 0x7fd53806e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36810/0 (socket says 172.21.15.44:36810) 2024-09-13T08:52:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53dc00640 1 -- 172.21.15.44:0/3813930269 learned_addr learned my addr 172.21.15.44:0/3813930269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53d200640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd520004810 0x7fd5381f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd537e00640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5381f4990 0x7fd5380b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53dc00640 1 -- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd520004810 msgr2=0x7fd5381f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53dc00640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd520004810 0x7fd5381f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53dc00640 1 -- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5381f4990 msgr2=0x7fd5380b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53dc00640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5381f4990 0x7fd5380b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53dc00640 1 -- 172.21.15.44:0/3813930269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5380b3360 con 0x7fd53806e460 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53dc00640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd53806e460 0x7fd53806e8e0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fd5380d0ce0 tx=0x7fd530011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd534c00640 1 -- 172.21.15.44:0/3813930269 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5300139e0 con 0x7fd53806e460 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd534c00640 1 -- 172.21.15.44:0/3813930269 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd530013c70 con 0x7fd53806e460 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53e600640 1 -- 172.21.15.44:0/3813930269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5380b30a0 con 0x7fd53806e460 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd534c00640 1 -- 172.21.15.44:0/3813930269 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530012640 con 0x7fd53806e460 2024-09-13T08:52:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.657+0000 7fd53e600640 1 -- 172.21.15.44:0/3813930269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5381f5410 con 0x7fd53806e460 2024-09-13T08:52:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.661+0000 7fd53e600640 1 -- 172.21.15.44:0/3813930269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd538071060 con 0x7fd53806e460 2024-09-13T08:52:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.661+0000 7fd534c00640 1 -- 172.21.15.44:0/3813930269 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fd5300127e0 con 0x7fd53806e460 2024-09-13T08:52:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.661+0000 7fd534c00640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd518074db0 0x7fd518077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.661+0000 7fd534c00640 1 -- 172.21.15.44:0/3813930269 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7fd530096aa0 con 0x7fd53806e460 2024-09-13T08:52:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.661+0000 7fd53d200640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd518074db0 0x7fd518077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.661+0000 7fd53d200640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd518074db0 0x7fd518077270 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009ce0 tx=0x7fd52c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:05.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.665+0000 7fd534c00640 1 -- 172.21.15.44:0/3813930269 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd530063750 con 0x7fd53806e460 2024-09-13T08:52:05.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.837+0000 7fd53e600640 1 -- 172.21.15.44:0/3813930269 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5380b0320 con 0x7fd518074db0 2024-09-13T08:52:05.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.845+0000 7fd534c00640 1 -- 172.21.15.44:0/3813930269 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252498 (secure 0 0 0) 0x7fd52c05ea70 con 0x7fd518074db0 2024-09-13T08:52:05.846 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:05.847 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 -- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd518074db0 msgr2=0x7fd518077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd518074db0 0x7fd518077270 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009ce0 tx=0x7fd52c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 -- 172.21.15.44:0/3813930269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd53806e460 msgr2=0x7fd53806e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd53806e460 0x7fd53806e8e0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fd5380d0ce0 tx=0x7fd530011970 comp rx=0 tx=0).stop 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 -- 172.21.15.44:0/3813930269 shutdown_connections 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd518074db0 0x7fd518077270 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd53806e460 0x7fd53806e8e0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5381f4990 0x7fd5380b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 --2- 172.21.15.44:0/3813930269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd520004810 0x7fd5381f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 -- 172.21.15.44:0/3813930269 >> 172.21.15.44:0/3813930269 conn(0x7fd5380d1be0 msgr2=0x7fd5380b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 -- 172.21.15.44:0/3813930269 shutdown_connections 2024-09-13T08:52:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:05.849+0000 7fd53e600640 1 -- 172.21.15.44:0/3813930269 wait complete. 2024-09-13T08:52:05.866 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":565,"stamp":"2024-09-13T08:52:04.816487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1094516,"kb_used_data":13928,"kb_used_omap":12,"kb_used_meta":1080371,"kb_avail":753880204,"statfs":{"total":773094113280,"available":771973328896,"internally_reserved":0,"allocated":14262272,"data_stored":6926868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1106300465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.536333"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316252+0000","last_change":"2024-09-13T08:52:01.322527+0000","last_active":"2024-09-13T08:52:02.316252+0000","last_peered":"2024-09-13T08:52:02.316252+0000","last_clean":"2024-09-13T08:52:02.316252+0000","last_became_active":"2024-09-13T08:52:01.322224+0000","last_became_peered":"2024-09-13T08:52:01.322224+0000","last_unstale":"2024-09-13T08:52:02.316252+0000","last_undegraded":"2024-09-13T08:52:02.316252+0000","last_fullsized":"2024-09-13T08:52:02.316252+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:30:12.276749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315667+0000","last_change":"2024-09-13T08:52:01.312181+0000","last_active":"2024-09-13T08:52:02.315667+0000","last_peered":"2024-09-13T08:52:02.315667+0000","last_clean":"2024-09-13T08:52:02.315667+0000","last_became_active":"2024-09-13T08:52:01.311857+0000","last_became_peered":"2024-09-13T08:52:01.311857+0000","last_unstale":"2024-09-13T08:52:02.315667+0000","last_undegraded":"2024-09-13T08:52:02.315667+0000","last_fullsized":"2024-09-13T08:52:02.315667+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:39:56.718999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741781+0000","last_change":"2024-09-13T08:52:01.317101+0000","last_active":"2024-09-13T08:52:02.741781+0000","last_peered":"2024-09-13T08:52:02.741781+0000","last_clean":"2024-09-13T08:52:02.741781+0000","last_became_active":"2024-09-13T08:52:01.316665+0000","last_became_peered":"2024-09-13T08:52:01.316665+0000","last_unstale":"2024-09-13T08:52:02.741781+0000","last_undegraded":"2024-09-13T08:52:02.741781+0000","last_fullsized":"2024-09-13T08:52:02.741781+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:29:55.643905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317455+0000","last_change":"2024-09-13T08:52:01.318694+0000","last_active":"2024-09-13T08:52:02.317455+0000","last_peered":"2024-09-13T08:52:02.317455+0000","last_clean":"2024-09-13T08:52:02.317455+0000","last_became_active":"2024-09-13T08:52:01.318034+0000","last_became_peered":"2024-09-13T08:52:01.318034+0000","last_unstale":"2024-09-13T08:52:02.317455+0000","last_undegraded":"2024-09-13T08:52:02.317455+0000","last_fullsized":"2024-09-13T08:52:02.317455+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:08.298145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741549+0000","last_change":"2024-09-13T08:52:01.329818+0000","last_active":"2024-09-13T08:52:02.741549+0000","last_peered":"2024-09-13T08:52:02.741549+0000","last_clean":"2024-09-13T08:52:02.741549+0000","last_became_active":"2024-09-13T08:52:01.329412+0000","last_became_peered":"2024-09-13T08:52:01.329412+0000","last_unstale":"2024-09-13T08:52:02.741549+0000","last_undegraded":"2024-09-13T08:52:02.741549+0000","last_fullsized":"2024-09-13T08:52:02.741549+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:54:04.345707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311175+0000","last_change":"2024-09-13T08:52:01.312938+0000","last_active":"2024-09-13T08:52:02.311175+0000","last_peered":"2024-09-13T08:52:02.311175+0000","last_clean":"2024-09-13T08:52:02.311175+0000","last_became_active":"2024-09-13T08:52:01.312609+0000","last_became_peered":"2024-09-13T08:52:01.312609+0000","last_unstale":"2024-09-13T08:52:02.311175+0000","last_undegraded":"2024-09-13T08:52:02.311175+0000","last_fullsized":"2024-09-13T08:52:02.311175+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:16.750964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741080+0000","last_change":"2024-09-13T08:52:01.321368+0000","last_active":"2024-09-13T08:52:02.741080+0000","last_peered":"2024-09-13T08:52:02.741080+0000","last_clean":"2024-09-13T08:52:02.741080+0000","last_became_active":"2024-09-13T08:52:01.321063+0000","last_became_peered":"2024-09-13T08:52:01.321063+0000","last_unstale":"2024-09-13T08:52:02.741080+0000","last_undegraded":"2024-09-13T08:52:02.741080+0000","last_fullsized":"2024-09-13T08:52:02.741080+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:22:47.683542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.740802+0000","last_change":"2024-09-13T08:52:01.304827+0000","last_active":"2024-09-13T08:52:02.740802+0000","last_peered":"2024-09-13T08:52:02.740802+0000","last_clean":"2024-09-13T08:52:02.740802+0000","last_became_active":"2024-09-13T08:52:01.304500+0000","last_became_peered":"2024-09-13T08:52:01.304500+0000","last_unstale":"2024-09-13T08:52:02.740802+0000","last_undegraded":"2024-09-13T08:52:02.740802+0000","last_fullsized":"2024-09-13T08:52:02.740802+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:15.390950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742192+0000","last_change":"2024-09-13T08:52:01.319989+0000","last_active":"2024-09-13T08:52:02.742192+0000","last_peered":"2024-09-13T08:52:02.742192+0000","last_clean":"2024-09-13T08:52:02.742192+0000","last_became_active":"2024-09-13T08:52:01.318859+0000","last_became_peered":"2024-09-13T08:52:01.318859+0000","last_unstale":"2024-09-13T08:52:02.742192+0000","last_undegraded":"2024-09-13T08:52:02.742192+0000","last_fullsized":"2024-09-13T08:52:02.742192+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:06.887326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315981+0000","last_change":"2024-09-13T08:52:01.323289+0000","last_active":"2024-09-13T08:52:02.315981+0000","last_peered":"2024-09-13T08:52:02.315981+0000","last_clean":"2024-09-13T08:52:02.315981+0000","last_became_active":"2024-09-13T08:52:01.322978+0000","last_became_peered":"2024-09-13T08:52:01.322978+0000","last_unstale":"2024-09-13T08:52:02.315981+0000","last_undegraded":"2024-09-13T08:52:02.315981+0000","last_fullsized":"2024-09-13T08:52:02.315981+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:56.774369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317461+0000","last_change":"2024-09-13T08:52:01.314548+0000","last_active":"2024-09-13T08:52:02.317461+0000","last_peered":"2024-09-13T08:52:02.317461+0000","last_clean":"2024-09-13T08:52:02.317461+0000","last_became_active":"2024-09-13T08:52:01.314117+0000","last_became_peered":"2024-09-13T08:52:01.314117+0000","last_unstale":"2024-09-13T08:52:02.317461+0000","last_undegraded":"2024-09-13T08:52:02.317461+0000","last_fullsized":"2024-09-13T08:52:02.317461+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:13:00.802333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311696+0000","last_change":"2024-09-13T08:52:01.320522+0000","last_active":"2024-09-13T08:52:02.311696+0000","last_peered":"2024-09-13T08:52:02.311696+0000","last_clean":"2024-09-13T08:52:02.311696+0000","last_became_active":"2024-09-13T08:52:01.320133+0000","last_became_peered":"2024-09-13T08:52:01.320133+0000","last_unstale":"2024-09-13T08:52:02.311696+0000","last_undegraded":"2024-09-13T08:52:02.311696+0000","last_fullsized":"2024-09-13T08:52:02.311696+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:23.797197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317238+0000","last_change":"2024-09-13T08:52:01.298595+0000","last_active":"2024-09-13T08:52:02.317238+0000","last_peered":"2024-09-13T08:52:02.317238+0000","last_clean":"2024-09-13T08:52:02.317238+0000","last_became_active":"2024-09-13T08:52:01.298273+0000","last_became_peered":"2024-09-13T08:52:01.298273+0000","last_unstale":"2024-09-13T08:52:02.317238+0000","last_undegraded":"2024-09-13T08:52:02.317238+0000","last_fullsized":"2024-09-13T08:52:02.317238+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:00.466613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742511+0000","last_change":"2024-09-13T08:52:01.319951+0000","last_active":"2024-09-13T08:52:02.742511+0000","last_peered":"2024-09-13T08:52:02.742511+0000","last_clean":"2024-09-13T08:52:02.742511+0000","last_became_active":"2024-09-13T08:52:01.319635+0000","last_became_peered":"2024-09-13T08:52:01.319635+0000","last_unstale":"2024-09-13T08:52:02.742511+0000","last_undegraded":"2024-09-13T08:52:02.742511+0000","last_fullsized":"2024-09-13T08:52:02.742511+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:20:22.380961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742641+0000","last_change":"2024-09-13T08:52:01.316882+0000","last_active":"2024-09-13T08:52:02.742641+0000","last_peered":"2024-09-13T08:52:02.742641+0000","last_clean":"2024-09-13T08:52:02.742641+0000","last_became_active":"2024-09-13T08:52:01.316625+0000","last_became_peered":"2024-09-13T08:52:01.316625+0000","last_unstale":"2024-09-13T08:52:02.742641+0000","last_undegraded":"2024-09-13T08:52:02.742641+0000","last_fullsized":"2024-09-13T08:52:02.742641+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:17.107498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743502+0000","last_change":"2024-09-13T08:52:01.321436+0000","last_active":"2024-09-13T08:52:02.743502+0000","last_peered":"2024-09-13T08:52:02.743502+0000","last_clean":"2024-09-13T08:52:02.743502+0000","last_became_active":"2024-09-13T08:52:01.321179+0000","last_became_peered":"2024-09-13T08:52:01.321179+0000","last_unstale":"2024-09-13T08:52:02.743502+0000","last_undegraded":"2024-09-13T08:52:02.743502+0000","last_fullsized":"2024-09-13T08:52:02.743502+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:22:15.251227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742403+0000","last_change":"2024-09-13T08:52:01.320175+0000","last_active":"2024-09-13T08:52:02.742403+0000","last_peered":"2024-09-13T08:52:02.742403+0000","last_clean":"2024-09-13T08:52:02.742403+0000","last_became_active":"2024-09-13T08:52:01.317764+0000","last_became_peered":"2024-09-13T08:52:01.317764+0000","last_unstale":"2024-09-13T08:52:02.742403+0000","last_undegraded":"2024-09-13T08:52:02.742403+0000","last_fullsized":"2024-09-13T08:52:02.742403+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:44.605557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743111+0000","last_change":"2024-09-13T08:52:01.322915+0000","last_active":"2024-09-13T08:52:02.743111+0000","last_peered":"2024-09-13T08:52:02.743111+0000","last_clean":"2024-09-13T08:52:02.743111+0000","last_became_active":"2024-09-13T08:52:01.322616+0000","last_became_peered":"2024-09-13T08:52:01.322616+0000","last_unstale":"2024-09-13T08:52:02.743111+0000","last_undegraded":"2024-09-13T08:52:02.743111+0000","last_fullsized":"2024-09-13T08:52:02.743111+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.759066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315583+0000","last_change":"2024-09-13T08:52:01.302162+0000","last_active":"2024-09-13T08:52:02.315583+0000","last_peered":"2024-09-13T08:52:02.315583+0000","last_clean":"2024-09-13T08:52:02.315583+0000","last_became_active":"2024-09-13T08:52:01.301525+0000","last_became_peered":"2024-09-13T08:52:01.301525+0000","last_unstale":"2024-09-13T08:52:02.315583+0000","last_undegraded":"2024-09-13T08:52:02.315583+0000","last_fullsized":"2024-09-13T08:52:02.315583+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:32.349846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743400+0000","last_change":"2024-09-13T08:52:01.321288+0000","last_active":"2024-09-13T08:52:02.743400+0000","last_peered":"2024-09-13T08:52:02.743400+0000","last_clean":"2024-09-13T08:52:02.743400+0000","last_became_active":"2024-09-13T08:52:01.320996+0000","last_became_peered":"2024-09-13T08:52:01.320996+0000","last_unstale":"2024-09-13T08:52:02.743400+0000","last_undegraded":"2024-09-13T08:52:02.743400+0000","last_fullsized":"2024-09-13T08:52:02.743400+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:36:01.397745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311415+0000","last_change":"2024-09-13T08:52:01.327882+0000","last_active":"2024-09-13T08:52:02.311415+0000","last_peered":"2024-09-13T08:52:02.311415+0000","last_clean":"2024-09-13T08:52:02.311415+0000","last_became_active":"2024-09-13T08:52:01.327465+0000","last_became_peered":"2024-09-13T08:52:01.327465+0000","last_unstale":"2024-09-13T08:52:02.311415+0000","last_undegraded":"2024-09-13T08:52:02.311415+0000","last_fullsized":"2024-09-13T08:52:02.311415+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:00.614178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315546+0000","last_change":"2024-09-13T08:52:01.300377+0000","last_active":"2024-09-13T08:52:02.315546+0000","last_peered":"2024-09-13T08:52:02.315546+0000","last_clean":"2024-09-13T08:52:02.315546+0000","last_became_active":"2024-09-13T08:52:01.299895+0000","last_became_peered":"2024-09-13T08:52:01.299895+0000","last_unstale":"2024-09-13T08:52:02.315546+0000","last_undegraded":"2024-09-13T08:52:02.315546+0000","last_fullsized":"2024-09-13T08:52:02.315546+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:01:05.858267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741424+0000","last_change":"2024-09-13T08:52:01.310370+0000","last_active":"2024-09-13T08:52:02.741424+0000","last_peered":"2024-09-13T08:52:02.741424+0000","last_clean":"2024-09-13T08:52:02.741424+0000","last_became_active":"2024-09-13T08:52:01.310069+0000","last_became_peered":"2024-09-13T08:52:01.310069+0000","last_unstale":"2024-09-13T08:52:02.741424+0000","last_undegraded":"2024-09-13T08:52:02.741424+0000","last_fullsized":"2024-09-13T08:52:02.741424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:53.281383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742535+0000","last_change":"2024-09-13T08:52:01.327212+0000","last_active":"2024-09-13T08:52:02.742535+0000","last_peered":"2024-09-13T08:52:02.742535+0000","last_clean":"2024-09-13T08:52:02.742535+0000","last_became_active":"2024-09-13T08:52:01.326928+0000","last_became_peered":"2024-09-13T08:52:01.326928+0000","last_unstale":"2024-09-13T08:52:02.742535+0000","last_undegraded":"2024-09-13T08:52:02.742535+0000","last_fullsized":"2024-09-13T08:52:02.742535+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:59.345020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317786+0000","last_change":"2024-09-13T08:52:01.300357+0000","last_active":"2024-09-13T08:52:02.317786+0000","last_peered":"2024-09-13T08:52:02.317786+0000","last_clean":"2024-09-13T08:52:02.317786+0000","last_became_active":"2024-09-13T08:52:01.299967+0000","last_became_peered":"2024-09-13T08:52:01.299967+0000","last_unstale":"2024-09-13T08:52:02.317786+0000","last_undegraded":"2024-09-13T08:52:02.317786+0000","last_fullsized":"2024-09-13T08:52:02.317786+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:37.958421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316013+0000","last_change":"2024-09-13T08:52:01.306645+0000","last_active":"2024-09-13T08:52:02.316013+0000","last_peered":"2024-09-13T08:52:02.316013+0000","last_clean":"2024-09-13T08:52:02.316013+0000","last_became_active":"2024-09-13T08:52:01.306324+0000","last_became_peered":"2024-09-13T08:52:01.306324+0000","last_unstale":"2024-09-13T08:52:02.316013+0000","last_undegraded":"2024-09-13T08:52:02.316013+0000","last_fullsized":"2024-09-13T08:52:02.316013+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:57.164277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743469+0000","last_change":"2024-09-13T08:52:01.318429+0000","last_active":"2024-09-13T08:52:02.743469+0000","last_peered":"2024-09-13T08:52:02.743469+0000","last_clean":"2024-09-13T08:52:02.743469+0000","last_became_active":"2024-09-13T08:52:01.318131+0000","last_became_peered":"2024-09-13T08:52:01.318131+0000","last_unstale":"2024-09-13T08:52:02.743469+0000","last_undegraded":"2024-09-13T08:52:02.743469+0000","last_fullsized":"2024-09-13T08:52:02.743469+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:08.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742300+0000","last_change":"2024-09-13T08:52:01.315432+0000","last_active":"2024-09-13T08:52:02.742300+0000","last_peered":"2024-09-13T08:52:02.742300+0000","last_clean":"2024-09-13T08:52:02.742300+0000","last_became_active":"2024-09-13T08:52:01.315141+0000","last_became_peered":"2024-09-13T08:52:01.315141+0000","last_unstale":"2024-09-13T08:52:02.742300+0000","last_undegraded":"2024-09-13T08:52:02.742300+0000","last_fullsized":"2024-09-13T08:52:02.742300+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:07:31.291030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315985+0000","last_change":"2024-09-13T08:52:01.311484+0000","last_active":"2024-09-13T08:52:02.315985+0000","last_peered":"2024-09-13T08:52:02.315985+0000","last_clean":"2024-09-13T08:52:02.315985+0000","last_became_active":"2024-09-13T08:52:01.311200+0000","last_became_peered":"2024-09-13T08:52:01.311200+0000","last_unstale":"2024-09-13T08:52:02.315985+0000","last_undegraded":"2024-09-13T08:52:02.315985+0000","last_fullsized":"2024-09-13T08:52:02.315985+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:41.118368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315646+0000","last_change":"2024-09-13T08:52:01.316859+0000","last_active":"2024-09-13T08:52:02.315646+0000","last_peered":"2024-09-13T08:52:02.315646+0000","last_clean":"2024-09-13T08:52:02.315646+0000","last_became_active":"2024-09-13T08:52:01.316549+0000","last_became_peered":"2024-09-13T08:52:01.316549+0000","last_unstale":"2024-09-13T08:52:02.315646+0000","last_undegraded":"2024-09-13T08:52:02.315646+0000","last_fullsized":"2024-09-13T08:52:02.315646+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:10:02.167304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741628+0000","last_change":"2024-09-13T08:52:01.312535+0000","last_active":"2024-09-13T08:52:02.741628+0000","last_peered":"2024-09-13T08:52:02.741628+0000","last_clean":"2024-09-13T08:52:02.741628+0000","last_became_active":"2024-09-13T08:52:01.312206+0000","last_became_peered":"2024-09-13T08:52:01.312206+0000","last_unstale":"2024-09-13T08:52:02.741628+0000","last_undegraded":"2024-09-13T08:52:02.741628+0000","last_fullsized":"2024-09-13T08:52:02.741628+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:32.739615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741025+0000","last_change":"2024-09-13T08:52:01.310332+0000","last_active":"2024-09-13T08:52:02.741025+0000","last_peered":"2024-09-13T08:52:02.741025+0000","last_clean":"2024-09-13T08:52:02.741025+0000","last_became_active":"2024-09-13T08:52:01.309812+0000","last_became_peered":"2024-09-13T08:52:01.309812+0000","last_unstale":"2024-09-13T08:52:02.741025+0000","last_undegraded":"2024-09-13T08:52:02.741025+0000","last_fullsized":"2024-09-13T08:52:02.741025+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:40:56.189622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743492+0000","last_change":"2024-09-13T08:52:01.322757+0000","last_active":"2024-09-13T08:52:02.743492+0000","last_peered":"2024-09-13T08:52:02.743492+0000","last_clean":"2024-09-13T08:52:02.743492+0000","last_became_active":"2024-09-13T08:52:01.322499+0000","last_became_peered":"2024-09-13T08:52:01.322499+0000","last_unstale":"2024-09-13T08:52:02.743492+0000","last_undegraded":"2024-09-13T08:52:02.743492+0000","last_fullsized":"2024-09-13T08:52:02.743492+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:41.534273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"130'4","reported_seq":19,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741663+0000","last_change":"2024-09-13T08:52:00.309359+0000","last_active":"2024-09-13T08:52:02.741663+0000","last_peered":"2024-09-13T08:52:02.741663+0000","last_clean":"2024-09-13T08:52:02.741663+0000","last_became_active":"2024-09-13T08:52:00.308056+0000","last_became_peered":"2024-09-13T08:52:00.308056+0000","last_unstale":"2024-09-13T08:52:02.741663+0000","last_undegraded":"2024-09-13T08:52:02.741663+0000","last_fullsized":"2024-09-13T08:52:02.741663+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:36:21.582936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742453+0000","last_change":"2024-09-13T08:52:01.318029+0000","last_active":"2024-09-13T08:52:02.742453+0000","last_peered":"2024-09-13T08:52:02.742453+0000","last_clean":"2024-09-13T08:52:02.742453+0000","last_became_active":"2024-09-13T08:52:01.317483+0000","last_became_peered":"2024-09-13T08:52:01.317483+0000","last_unstale":"2024-09-13T08:52:02.742453+0000","last_undegraded":"2024-09-13T08:52:02.742453+0000","last_fullsized":"2024-09-13T08:52:02.742453+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:01:37.722600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"130'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742171+0000","last_change":"2024-09-13T08:52:00.305531+0000","last_active":"2024-09-13T08:52:02.742171+0000","last_peered":"2024-09-13T08:52:02.742171+0000","last_clean":"2024-09-13T08:52:02.742171+0000","last_became_active":"2024-09-13T08:52:00.305247+0000","last_became_peered":"2024-09-13T08:52:00.305247+0000","last_unstale":"2024-09-13T08:52:02.742171+0000","last_undegraded":"2024-09-13T08:52:02.742171+0000","last_fullsized":"2024-09-13T08:52:02.742171+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:45.296251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741664+0000","last_change":"2024-09-13T08:52:01.316971+0000","last_active":"2024-09-13T08:52:02.741664+0000","last_peered":"2024-09-13T08:52:02.741664+0000","last_clean":"2024-09-13T08:52:02.741664+0000","last_became_active":"2024-09-13T08:52:01.316688+0000","last_became_peered":"2024-09-13T08:52:01.316688+0000","last_unstale":"2024-09-13T08:52:02.741664+0000","last_undegraded":"2024-09-13T08:52:02.741664+0000","last_fullsized":"2024-09-13T08:52:02.741664+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:48:31.309365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317309+0000","last_change":"2024-09-13T08:52:00.307998+0000","last_active":"2024-09-13T08:52:02.317309+0000","last_peered":"2024-09-13T08:52:02.317309+0000","last_clean":"2024-09-13T08:52:02.317309+0000","last_became_active":"2024-09-13T08:52:00.307690+0000","last_became_peered":"2024-09-13T08:52:00.307690+0000","last_unstale":"2024-09-13T08:52:02.317309+0000","last_undegraded":"2024-09-13T08:52:02.317309+0000","last_fullsized":"2024-09-13T08:52:02.317309+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:17:54.277481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311646+0000","last_change":"2024-09-13T08:52:01.319805+0000","last_active":"2024-09-13T08:52:02.311646+0000","last_peered":"2024-09-13T08:52:02.311646+0000","last_clean":"2024-09-13T08:52:02.311646+0000","last_became_active":"2024-09-13T08:52:01.319399+0000","last_became_peered":"2024-09-13T08:52:01.319399+0000","last_unstale":"2024-09-13T08:52:02.311646+0000","last_undegraded":"2024-09-13T08:52:02.311646+0000","last_fullsized":"2024-09-13T08:52:02.311646+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:41.118368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316083+0000","last_change":"2024-09-13T08:52:00.302580+0000","last_active":"2024-09-13T08:52:02.316083+0000","last_peered":"2024-09-13T08:52:02.316083+0000","last_clean":"2024-09-13T08:52:02.316083+0000","last_became_active":"2024-09-13T08:52:00.302293+0000","last_became_peered":"2024-09-13T08:52:00.302293+0000","last_unstale":"2024-09-13T08:52:02.316083+0000","last_undegraded":"2024-09-13T08:52:02.316083+0000","last_fullsized":"2024-09-13T08:52:02.316083+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:00:20.752180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742989+0000","last_change":"2024-09-13T08:52:01.321225+0000","last_active":"2024-09-13T08:52:02.742989+0000","last_peered":"2024-09-13T08:52:02.742989+0000","last_clean":"2024-09-13T08:52:02.742989+0000","last_became_active":"2024-09-13T08:52:01.320936+0000","last_became_peered":"2024-09-13T08:52:01.320936+0000","last_unstale":"2024-09-13T08:52:02.742989+0000","last_undegraded":"2024-09-13T08:52:02.742989+0000","last_fullsized":"2024-09-13T08:52:02.742989+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:25.402433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"130'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315264+0000","last_change":"2024-09-13T08:52:00.301831+0000","last_active":"2024-09-13T08:52:02.315264+0000","last_peered":"2024-09-13T08:52:02.315264+0000","last_clean":"2024-09-13T08:52:02.315264+0000","last_became_active":"2024-09-13T08:52:00.301550+0000","last_became_peered":"2024-09-13T08:52:00.301550+0000","last_unstale":"2024-09-13T08:52:02.315264+0000","last_undegraded":"2024-09-13T08:52:02.315264+0000","last_fullsized":"2024-09-13T08:52:02.315264+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:05.223602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311513+0000","last_change":"2024-09-13T08:52:01.327795+0000","last_active":"2024-09-13T08:52:02.311513+0000","last_peered":"2024-09-13T08:52:02.311513+0000","last_clean":"2024-09-13T08:52:02.311513+0000","last_became_active":"2024-09-13T08:52:01.326730+0000","last_became_peered":"2024-09-13T08:52:01.326730+0000","last_unstale":"2024-09-13T08:52:02.311513+0000","last_undegraded":"2024-09-13T08:52:02.311513+0000","last_fullsized":"2024-09-13T08:52:02.311513+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:20:01.148707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"130'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743105+0000","last_change":"2024-09-13T08:52:00.305456+0000","last_active":"2024-09-13T08:52:02.743105+0000","last_peered":"2024-09-13T08:52:02.743105+0000","last_clean":"2024-09-13T08:52:02.743105+0000","last_became_active":"2024-09-13T08:52:00.305190+0000","last_became_peered":"2024-09-13T08:52:00.305190+0000","last_unstale":"2024-09-13T08:52:02.743105+0000","last_undegraded":"2024-09-13T08:52:02.743105+0000","last_fullsized":"2024-09-13T08:52:02.743105+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:52.271592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311370+0000","last_change":"2024-09-13T08:52:01.323808+0000","last_active":"2024-09-13T08:52:02.311370+0000","last_peered":"2024-09-13T08:52:02.311370+0000","last_clean":"2024-09-13T08:52:02.311370+0000","last_became_active":"2024-09-13T08:52:01.322660+0000","last_became_peered":"2024-09-13T08:52:01.322660+0000","last_unstale":"2024-09-13T08:52:02.311370+0000","last_undegraded":"2024-09-13T08:52:02.311370+0000","last_fullsized":"2024-09-13T08:52:02.311370+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:22.751896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741676+0000","last_change":"2024-09-13T08:52:00.304348+0000","last_active":"2024-09-13T08:52:02.741676+0000","last_peered":"2024-09-13T08:52:02.741676+0000","last_clean":"2024-09-13T08:52:02.741676+0000","last_became_active":"2024-09-13T08:52:00.304098+0000","last_became_peered":"2024-09-13T08:52:00.304098+0000","last_unstale":"2024-09-13T08:52:02.741676+0000","last_undegraded":"2024-09-13T08:52:02.741676+0000","last_fullsized":"2024-09-13T08:52:02.741676+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:46.449811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315496+0000","last_change":"2024-09-13T08:52:01.298285+0000","last_active":"2024-09-13T08:52:02.315496+0000","last_peered":"2024-09-13T08:52:02.315496+0000","last_clean":"2024-09-13T08:52:02.315496+0000","last_became_active":"2024-09-13T08:52:01.297963+0000","last_became_peered":"2024-09-13T08:52:01.297963+0000","last_unstale":"2024-09-13T08:52:02.315496+0000","last_undegraded":"2024-09-13T08:52:02.315496+0000","last_fullsized":"2024-09-13T08:52:02.315496+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:01.152580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315102+0000","last_change":"2024-09-13T08:52:00.304666+0000","last_active":"2024-09-13T08:52:02.315102+0000","last_peered":"2024-09-13T08:52:02.315102+0000","last_clean":"2024-09-13T08:52:02.315102+0000","last_became_active":"2024-09-13T08:52:00.303812+0000","last_became_peered":"2024-09-13T08:52:00.303812+0000","last_unstale":"2024-09-13T08:52:02.315102+0000","last_undegraded":"2024-09-13T08:52:02.315102+0000","last_fullsized":"2024-09-13T08:52:02.315102+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:28:38.371986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316251+0000","last_change":"2024-09-13T08:52:01.300199+0000","last_active":"2024-09-13T08:52:02.316251+0000","last_peered":"2024-09-13T08:52:02.316251+0000","last_clean":"2024-09-13T08:52:02.316251+0000","last_became_active":"2024-09-13T08:52:01.299819+0000","last_became_peered":"2024-09-13T08:52:01.299819+0000","last_unstale":"2024-09-13T08:52:02.316251+0000","last_undegraded":"2024-09-13T08:52:02.316251+0000","last_fullsized":"2024-09-13T08:52:02.316251+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:54.373221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311827+0000","last_change":"2024-09-13T08:52:00.313505+0000","last_active":"2024-09-13T08:52:02.311827+0000","last_peered":"2024-09-13T08:52:02.311827+0000","last_clean":"2024-09-13T08:52:02.311827+0000","last_became_active":"2024-09-13T08:52:00.312377+0000","last_became_peered":"2024-09-13T08:52:00.312377+0000","last_unstale":"2024-09-13T08:52:02.311827+0000","last_undegraded":"2024-09-13T08:52:02.311827+0000","last_fullsized":"2024-09-13T08:52:02.311827+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:14.867528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741575+0000","last_change":"2024-09-13T08:52:01.312669+0000","last_active":"2024-09-13T08:52:02.741575+0000","last_peered":"2024-09-13T08:52:02.741575+0000","last_clean":"2024-09-13T08:52:02.741575+0000","last_became_active":"2024-09-13T08:52:01.312321+0000","last_became_peered":"2024-09-13T08:52:01.312321+0000","last_unstale":"2024-09-13T08:52:02.741575+0000","last_undegraded":"2024-09-13T08:52:02.741575+0000","last_fullsized":"2024-09-13T08:52:02.741575+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:18:55.267451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743247+0000","last_change":"2024-09-13T08:52:00.309464+0000","last_active":"2024-09-13T08:52:02.743247+0000","last_peered":"2024-09-13T08:52:02.743247+0000","last_clean":"2024-09-13T08:52:02.743247+0000","last_became_active":"2024-09-13T08:52:00.309169+0000","last_became_peered":"2024-09-13T08:52:00.309169+0000","last_unstale":"2024-09-13T08:52:02.743247+0000","last_undegraded":"2024-09-13T08:52:02.743247+0000","last_fullsized":"2024-09-13T08:52:02.743247+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:13.173905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311891+0000","last_change":"2024-09-13T08:52:01.320481+0000","last_active":"2024-09-13T08:52:02.311891+0000","last_peered":"2024-09-13T08:52:02.311891+0000","last_clean":"2024-09-13T08:52:02.311891+0000","last_became_active":"2024-09-13T08:52:01.320084+0000","last_became_peered":"2024-09-13T08:52:01.320084+0000","last_unstale":"2024-09-13T08:52:02.311891+0000","last_undegraded":"2024-09-13T08:52:02.311891+0000","last_fullsized":"2024-09-13T08:52:02.311891+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:40.653366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"130'3","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741201+0000","last_change":"2024-09-13T08:52:00.308398+0000","last_active":"2024-09-13T08:52:02.741201+0000","last_peered":"2024-09-13T08:52:02.741201+0000","last_clean":"2024-09-13T08:52:02.741201+0000","last_became_active":"2024-09-13T08:52:00.307459+0000","last_became_peered":"2024-09-13T08:52:00.307459+0000","last_unstale":"2024-09-13T08:52:02.741201+0000","last_undegraded":"2024-09-13T08:52:02.741201+0000","last_fullsized":"2024-09-13T08:52:02.741201+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:00:20.752180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742172+0000","last_change":"2024-09-13T08:52:01.321620+0000","last_active":"2024-09-13T08:52:02.742172+0000","last_peered":"2024-09-13T08:52:02.742172+0000","last_clean":"2024-09-13T08:52:02.742172+0000","last_became_active":"2024-09-13T08:52:01.320943+0000","last_became_peered":"2024-09-13T08:52:01.320943+0000","last_unstale":"2024-09-13T08:52:02.742172+0000","last_undegraded":"2024-09-13T08:52:02.742172+0000","last_fullsized":"2024-09-13T08:52:02.742172+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:43:35.081647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"130'6","reported_seq":21,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742358+0000","last_change":"2024-09-13T08:52:00.304788+0000","last_active":"2024-09-13T08:52:02.742358+0000","last_peered":"2024-09-13T08:52:02.742358+0000","last_clean":"2024-09-13T08:52:02.742358+0000","last_became_active":"2024-09-13T08:52:00.304499+0000","last_became_peered":"2024-09-13T08:52:00.304499+0000","last_unstale":"2024-09-13T08:52:02.742358+0000","last_undegraded":"2024-09-13T08:52:02.742358+0000","last_fullsized":"2024-09-13T08:52:02.742358+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:44.767320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743047+0000","last_change":"2024-09-13T08:52:01.315708+0000","last_active":"2024-09-13T08:52:02.743047+0000","last_peered":"2024-09-13T08:52:02.743047+0000","last_clean":"2024-09-13T08:52:02.743047+0000","last_became_active":"2024-09-13T08:52:01.315427+0000","last_became_peered":"2024-09-13T08:52:01.315427+0000","last_unstale":"2024-09-13T08:52:02.743047+0000","last_undegraded":"2024-09-13T08:52:02.743047+0000","last_fullsized":"2024-09-13T08:52:02.743047+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:52.518065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742613+0000","last_change":"2024-09-13T08:52:00.306576+0000","last_active":"2024-09-13T08:52:02.742613+0000","last_peered":"2024-09-13T08:52:02.742613+0000","last_clean":"2024-09-13T08:52:02.742613+0000","last_became_active":"2024-09-13T08:52:00.305457+0000","last_became_peered":"2024-09-13T08:52:00.305457+0000","last_unstale":"2024-09-13T08:52:02.742613+0000","last_undegraded":"2024-09-13T08:52:02.742613+0000","last_fullsized":"2024-09-13T08:52:02.742613+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:55.302117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742431+0000","last_change":"2024-09-13T08:52:01.320474+0000","last_active":"2024-09-13T08:52:02.742431+0000","last_peered":"2024-09-13T08:52:02.742431+0000","last_clean":"2024-09-13T08:52:02.742431+0000","last_became_active":"2024-09-13T08:52:01.320146+0000","last_became_peered":"2024-09-13T08:52:01.320146+0000","last_unstale":"2024-09-13T08:52:02.742431+0000","last_undegraded":"2024-09-13T08:52:02.742431+0000","last_fullsized":"2024-09-13T08:52:02.742431+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:55:51.095792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316786+0000","last_change":"2024-09-13T08:52:00.301084+0000","last_active":"2024-09-13T08:52:02.316786+0000","last_peered":"2024-09-13T08:52:02.316786+0000","last_clean":"2024-09-13T08:52:02.316786+0000","last_became_active":"2024-09-13T08:52:00.300449+0000","last_became_peered":"2024-09-13T08:52:00.300449+0000","last_unstale":"2024-09-13T08:52:02.316786+0000","last_undegraded":"2024-09-13T08:52:02.316786+0000","last_fullsized":"2024-09-13T08:52:02.316786+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:40.768515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311291+0000","last_change":"2024-09-13T08:52:01.323667+0000","last_active":"2024-09-13T08:52:02.311291+0000","last_peered":"2024-09-13T08:52:02.311291+0000","last_clean":"2024-09-13T08:52:02.311291+0000","last_became_active":"2024-09-13T08:52:01.323392+0000","last_became_peered":"2024-09-13T08:52:01.323392+0000","last_unstale":"2024-09-13T08:52:02.311291+0000","last_undegraded":"2024-09-13T08:52:02.311291+0000","last_fullsized":"2024-09-13T08:52:02.311291+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:45:28.172536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"130'8","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311054+0000","last_change":"2024-09-13T08:52:00.311593+0000","last_active":"2024-09-13T08:52:02.311054+0000","last_peered":"2024-09-13T08:52:02.311054+0000","last_clean":"2024-09-13T08:52:02.311054+0000","last_became_active":"2024-09-13T08:52:00.309664+0000","last_became_peered":"2024-09-13T08:52:00.309664+0000","last_unstale":"2024-09-13T08:52:02.311054+0000","last_undegraded":"2024-09-13T08:52:02.311054+0000","last_fullsized":"2024-09-13T08:52:02.311054+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:27.749275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741314+0000","last_change":"2024-09-13T08:52:01.318196+0000","last_active":"2024-09-13T08:52:02.741314+0000","last_peered":"2024-09-13T08:52:02.741314+0000","last_clean":"2024-09-13T08:52:02.741314+0000","last_became_active":"2024-09-13T08:52:01.316334+0000","last_became_peered":"2024-09-13T08:52:01.316334+0000","last_unstale":"2024-09-13T08:52:02.741314+0000","last_undegraded":"2024-09-13T08:52:02.741314+0000","last_fullsized":"2024-09-13T08:52:02.741314+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:54.078357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"130'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311879+0000","last_change":"2024-09-13T08:52:00.303719+0000","last_active":"2024-09-13T08:52:02.311879+0000","last_peered":"2024-09-13T08:52:02.311879+0000","last_clean":"2024-09-13T08:52:02.311879+0000","last_became_active":"2024-09-13T08:52:00.303414+0000","last_became_peered":"2024-09-13T08:52:00.303414+0000","last_unstale":"2024-09-13T08:52:02.311879+0000","last_undegraded":"2024-09-13T08:52:02.311879+0000","last_fullsized":"2024-09-13T08:52:02.311879+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:29:26.702947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741910+0000","last_change":"2024-09-13T08:52:01.311936+0000","last_active":"2024-09-13T08:52:02.741910+0000","last_peered":"2024-09-13T08:52:02.741910+0000","last_clean":"2024-09-13T08:52:02.741910+0000","last_became_active":"2024-09-13T08:52:01.311521+0000","last_became_peered":"2024-09-13T08:52:01.311521+0000","last_unstale":"2024-09-13T08:52:02.741910+0000","last_undegraded":"2024-09-13T08:52:02.741910+0000","last_fullsized":"2024-09-13T08:52:02.741910+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:01.152580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.317148+0000","last_change":"2024-09-13T08:52:00.304308+0000","last_active":"2024-09-13T08:52:02.317148+0000","last_peered":"2024-09-13T08:52:02.317148+0000","last_clean":"2024-09-13T08:52:02.317148+0000","last_became_active":"2024-09-13T08:52:00.303052+0000","last_became_peered":"2024-09-13T08:52:00.303052+0000","last_unstale":"2024-09-13T08:52:02.317148+0000","last_undegraded":"2024-09-13T08:52:02.317148+0000","last_fullsized":"2024-09-13T08:52:02.317148+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:22.484472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316336+0000","last_change":"2024-09-13T08:52:01.314222+0000","last_active":"2024-09-13T08:52:02.316336+0000","last_peered":"2024-09-13T08:52:02.316336+0000","last_clean":"2024-09-13T08:52:02.316336+0000","last_became_active":"2024-09-13T08:52:01.313910+0000","last_became_peered":"2024-09-13T08:52:01.313910+0000","last_unstale":"2024-09-13T08:52:02.316336+0000","last_undegraded":"2024-09-13T08:52:02.316336+0000","last_fullsized":"2024-09-13T08:52:02.316336+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:51.220023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"130'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741902+0000","last_change":"2024-09-13T08:52:00.305737+0000","last_active":"2024-09-13T08:52:02.741902+0000","last_peered":"2024-09-13T08:52:02.741902+0000","last_clean":"2024-09-13T08:52:02.741902+0000","last_became_active":"2024-09-13T08:52:00.305460+0000","last_became_peered":"2024-09-13T08:52:00.305460+0000","last_unstale":"2024-09-13T08:52:02.741902+0000","last_undegraded":"2024-09-13T08:52:02.741902+0000","last_fullsized":"2024-09-13T08:52:02.741902+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:27.642818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315790+0000","last_change":"2024-09-13T08:52:01.300837+0000","last_active":"2024-09-13T08:52:02.315790+0000","last_peered":"2024-09-13T08:52:02.315790+0000","last_clean":"2024-09-13T08:52:02.315790+0000","last_became_active":"2024-09-13T08:52:01.300548+0000","last_became_peered":"2024-09-13T08:52:01.300548+0000","last_unstale":"2024-09-13T08:52:02.315790+0000","last_undegraded":"2024-09-13T08:52:02.315790+0000","last_fullsized":"2024-09-13T08:52:02.315790+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:37.434546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741071+0000","last_change":"2024-09-13T08:52:00.308406+0000","last_active":"2024-09-13T08:52:02.741071+0000","last_peered":"2024-09-13T08:52:02.741071+0000","last_clean":"2024-09-13T08:52:02.741071+0000","last_became_active":"2024-09-13T08:52:00.308066+0000","last_became_peered":"2024-09-13T08:52:00.308066+0000","last_unstale":"2024-09-13T08:52:02.741071+0000","last_undegraded":"2024-09-13T08:52:02.741071+0000","last_fullsized":"2024-09-13T08:52:02.741071+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:32.849131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315765+0000","last_change":"2024-09-13T08:52:01.314562+0000","last_active":"2024-09-13T08:52:02.315765+0000","last_peered":"2024-09-13T08:52:02.315765+0000","last_clean":"2024-09-13T08:52:02.315765+0000","last_became_active":"2024-09-13T08:52:01.314274+0000","last_became_peered":"2024-09-13T08:52:01.314274+0000","last_unstale":"2024-09-13T08:52:02.315765+0000","last_undegraded":"2024-09-13T08:52:02.315765+0000","last_fullsized":"2024-09-13T08:52:02.315765+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:15.167781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741902+0000","last_change":"2024-09-13T08:52:00.306739+0000","last_active":"2024-09-13T08:52:02.741902+0000","last_peered":"2024-09-13T08:52:02.741902+0000","last_clean":"2024-09-13T08:52:02.741902+0000","last_became_active":"2024-09-13T08:52:00.306395+0000","last_became_peered":"2024-09-13T08:52:00.306395+0000","last_unstale":"2024-09-13T08:52:02.741902+0000","last_undegraded":"2024-09-13T08:52:02.741902+0000","last_fullsized":"2024-09-13T08:52:02.741902+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:11:19.672649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741866+0000","last_change":"2024-09-13T08:52:01.313473+0000","last_active":"2024-09-13T08:52:02.741866+0000","last_peered":"2024-09-13T08:52:02.741866+0000","last_clean":"2024-09-13T08:52:02.741866+0000","last_became_active":"2024-09-13T08:52:01.313205+0000","last_became_peered":"2024-09-13T08:52:01.313205+0000","last_unstale":"2024-09-13T08:52:02.741866+0000","last_undegraded":"2024-09-13T08:52:02.741866+0000","last_fullsized":"2024-09-13T08:52:02.741866+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:00.109380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743189+0000","last_change":"2024-09-13T08:52:00.311117+0000","last_active":"2024-09-13T08:52:02.743189+0000","last_peered":"2024-09-13T08:52:02.743189+0000","last_clean":"2024-09-13T08:52:02.743189+0000","last_became_active":"2024-09-13T08:52:00.310555+0000","last_became_peered":"2024-09-13T08:52:00.310555+0000","last_unstale":"2024-09-13T08:52:02.743189+0000","last_undegraded":"2024-09-13T08:52:02.743189+0000","last_fullsized":"2024-09-13T08:52:02.743189+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:31.729824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742640+0000","last_change":"2024-09-13T08:52:01.316637+0000","last_active":"2024-09-13T08:52:02.742640+0000","last_peered":"2024-09-13T08:52:02.742640+0000","last_clean":"2024-09-13T08:52:02.742640+0000","last_became_active":"2024-09-13T08:52:01.316356+0000","last_became_peered":"2024-09-13T08:52:01.316356+0000","last_unstale":"2024-09-13T08:52:02.742640+0000","last_undegraded":"2024-09-13T08:52:02.742640+0000","last_fullsized":"2024-09-13T08:52:02.742640+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:46.110526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743090+0000","last_change":"2024-09-13T08:52:00.314043+0000","last_active":"2024-09-13T08:52:02.743090+0000","last_peered":"2024-09-13T08:52:02.743090+0000","last_clean":"2024-09-13T08:52:02.743090+0000","last_became_active":"2024-09-13T08:52:00.313702+0000","last_became_peered":"2024-09-13T08:52:00.313702+0000","last_unstale":"2024-09-13T08:52:02.743090+0000","last_undegraded":"2024-09-13T08:52:02.743090+0000","last_fullsized":"2024-09-13T08:52:02.743090+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:40.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311139+0000","last_change":"2024-09-13T08:52:01.323589+0000","last_active":"2024-09-13T08:52:02.311139+0000","last_peered":"2024-09-13T08:52:02.311139+0000","last_clean":"2024-09-13T08:52:02.311139+0000","last_became_active":"2024-09-13T08:52:01.323364+0000","last_became_peered":"2024-09-13T08:52:01.323364+0000","last_unstale":"2024-09-13T08:52:02.311139+0000","last_undegraded":"2024-09-13T08:52:02.311139+0000","last_fullsized":"2024-09-13T08:52:02.311139+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:43.374921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316681+0000","last_change":"2024-09-13T08:52:00.307654+0000","last_active":"2024-09-13T08:52:02.316681+0000","last_peered":"2024-09-13T08:52:02.316681+0000","last_clean":"2024-09-13T08:52:02.316681+0000","last_became_active":"2024-09-13T08:52:00.307314+0000","last_became_peered":"2024-09-13T08:52:00.307314+0000","last_unstale":"2024-09-13T08:52:02.316681+0000","last_undegraded":"2024-09-13T08:52:02.316681+0000","last_fullsized":"2024-09-13T08:52:02.316681+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:39.643575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742632+0000","last_change":"2024-09-13T08:52:01.329275+0000","last_active":"2024-09-13T08:52:02.742632+0000","last_peered":"2024-09-13T08:52:02.742632+0000","last_clean":"2024-09-13T08:52:02.742632+0000","last_became_active":"2024-09-13T08:52:01.327989+0000","last_became_peered":"2024-09-13T08:52:01.327989+0000","last_unstale":"2024-09-13T08:52:02.742632+0000","last_undegraded":"2024-09-13T08:52:02.742632+0000","last_fullsized":"2024-09-13T08:52:02.742632+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:37:36.161702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743542+0000","last_change":"2024-09-13T08:52:00.312748+0000","last_active":"2024-09-13T08:52:02.743542+0000","last_peered":"2024-09-13T08:52:02.743542+0000","last_clean":"2024-09-13T08:52:02.743542+0000","last_became_active":"2024-09-13T08:52:00.312464+0000","last_became_peered":"2024-09-13T08:52:00.312464+0000","last_unstale":"2024-09-13T08:52:02.743542+0000","last_undegraded":"2024-09-13T08:52:02.743542+0000","last_fullsized":"2024-09-13T08:52:02.743542+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:43:34.071856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316379+0000","last_change":"2024-09-13T08:52:01.319213+0000","last_active":"2024-09-13T08:52:02.316379+0000","last_peered":"2024-09-13T08:52:02.316379+0000","last_clean":"2024-09-13T08:52:02.316379+0000","last_became_active":"2024-09-13T08:52:01.318806+0000","last_became_peered":"2024-09-13T08:52:01.318806+0000","last_unstale":"2024-09-13T08:52:02.316379+0000","last_undegraded":"2024-09-13T08:52:02.316379+0000","last_fullsized":"2024-09-13T08:52:02.316379+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:08.286690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"131'1","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741422+0000","last_change":"2024-09-13T08:52:00.309662+0000","last_active":"2024-09-13T08:52:02.741422+0000","last_peered":"2024-09-13T08:52:02.741422+0000","last_clean":"2024-09-13T08:52:02.741422+0000","last_became_active":"2024-09-13T08:52:00.309379+0000","last_became_peered":"2024-09-13T08:52:00.309379+0000","last_unstale":"2024-09-13T08:52:02.741422+0000","last_undegraded":"2024-09-13T08:52:02.741422+0000","last_fullsized":"2024-09-13T08:52:02.741422+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:44:57.715520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743621+0000","last_change":"2024-09-13T08:52:01.315634+0000","last_active":"2024-09-13T08:52:02.743621+0000","last_peered":"2024-09-13T08:52:02.743621+0000","last_clean":"2024-09-13T08:52:02.743621+0000","last_became_active":"2024-09-13T08:52:01.315262+0000","last_became_peered":"2024-09-13T08:52:01.315262+0000","last_unstale":"2024-09-13T08:52:02.743621+0000","last_undegraded":"2024-09-13T08:52:02.743621+0000","last_fullsized":"2024-09-13T08:52:02.743621+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:30.042810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742822+0000","last_change":"2024-09-13T08:52:00.314269+0000","last_active":"2024-09-13T08:52:02.742822+0000","last_peered":"2024-09-13T08:52:02.742822+0000","last_clean":"2024-09-13T08:52:02.742822+0000","last_became_active":"2024-09-13T08:52:00.313830+0000","last_became_peered":"2024-09-13T08:52:00.313830+0000","last_unstale":"2024-09-13T08:52:02.742822+0000","last_undegraded":"2024-09-13T08:52:02.742822+0000","last_fullsized":"2024-09-13T08:52:02.742822+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:48:59.099589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741900+0000","last_change":"2024-09-13T08:52:01.314242+0000","last_active":"2024-09-13T08:52:02.741900+0000","last_peered":"2024-09-13T08:52:02.741900+0000","last_clean":"2024-09-13T08:52:02.741900+0000","last_became_active":"2024-09-13T08:52:01.313973+0000","last_became_peered":"2024-09-13T08:52:01.313973+0000","last_unstale":"2024-09-13T08:52:02.741900+0000","last_undegraded":"2024-09-13T08:52:02.741900+0000","last_fullsized":"2024-09-13T08:52:02.741900+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:55:28.898816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.743017+0000","last_change":"2024-09-13T08:52:00.313237+0000","last_active":"2024-09-13T08:52:02.743017+0000","last_peered":"2024-09-13T08:52:02.743017+0000","last_clean":"2024-09-13T08:52:02.743017+0000","last_became_active":"2024-09-13T08:52:00.312932+0000","last_became_peered":"2024-09-13T08:52:00.312932+0000","last_unstale":"2024-09-13T08:52:02.743017+0000","last_undegraded":"2024-09-13T08:52:02.743017+0000","last_fullsized":"2024-09-13T08:52:02.743017+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:29:23.309222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311228+0000","last_change":"2024-09-13T08:52:01.327132+0000","last_active":"2024-09-13T08:52:02.311228+0000","last_peered":"2024-09-13T08:52:02.311228+0000","last_clean":"2024-09-13T08:52:02.311228+0000","last_became_active":"2024-09-13T08:52:01.326305+0000","last_became_peered":"2024-09-13T08:52:01.326305+0000","last_unstale":"2024-09-13T08:52:02.311228+0000","last_undegraded":"2024-09-13T08:52:02.311228+0000","last_fullsized":"2024-09-13T08:52:02.311228+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:32:34.733024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.315950+0000","last_change":"2024-09-13T08:52:00.305900+0000","last_active":"2024-09-13T08:52:02.315950+0000","last_peered":"2024-09-13T08:52:02.315950+0000","last_clean":"2024-09-13T08:52:02.315950+0000","last_became_active":"2024-09-13T08:52:00.305583+0000","last_became_peered":"2024-09-13T08:52:00.305583+0000","last_unstale":"2024-09-13T08:52:02.315950+0000","last_undegraded":"2024-09-13T08:52:02.315950+0000","last_fullsized":"2024-09-13T08:52:02.315950+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:38.297880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311618+0000","last_change":"2024-09-13T08:52:01.328194+0000","last_active":"2024-09-13T08:52:02.311618+0000","last_peered":"2024-09-13T08:52:02.311618+0000","last_clean":"2024-09-13T08:52:02.311618+0000","last_became_active":"2024-09-13T08:52:01.327585+0000","last_became_peered":"2024-09-13T08:52:01.327585+0000","last_unstale":"2024-09-13T08:52:02.311618+0000","last_undegraded":"2024-09-13T08:52:02.311618+0000","last_fullsized":"2024-09-13T08:52:02.311618+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:27.166328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311279+0000","last_change":"2024-09-13T08:52:00.308764+0000","last_active":"2024-09-13T08:52:02.311279+0000","last_peered":"2024-09-13T08:52:02.311279+0000","last_clean":"2024-09-13T08:52:02.311279+0000","last_became_active":"2024-09-13T08:52:00.308488+0000","last_became_peered":"2024-09-13T08:52:00.308488+0000","last_unstale":"2024-09-13T08:52:02.311279+0000","last_undegraded":"2024-09-13T08:52:02.311279+0000","last_fullsized":"2024-09-13T08:52:02.311279+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:36:00.387954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742785+0000","last_change":"2024-09-13T08:52:01.327312+0000","last_active":"2024-09-13T08:52:02.742785+0000","last_peered":"2024-09-13T08:52:02.742785+0000","last_clean":"2024-09-13T08:52:02.742785+0000","last_became_active":"2024-09-13T08:52:01.326968+0000","last_became_peered":"2024-09-13T08:52:01.326968+0000","last_unstale":"2024-09-13T08:52:02.742785+0000","last_undegraded":"2024-09-13T08:52:02.742785+0000","last_fullsized":"2024-09-13T08:52:02.742785+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:35:05.037339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.316895+0000","last_change":"2024-09-13T08:52:00.301381+0000","last_active":"2024-09-13T08:52:02.316895+0000","last_peered":"2024-09-13T08:52:02.316895+0000","last_clean":"2024-09-13T08:52:02.316895+0000","last_became_active":"2024-09-13T08:52:00.300814+0000","last_became_peered":"2024-09-13T08:52:00.300814+0000","last_unstale":"2024-09-13T08:52:02.316895+0000","last_undegraded":"2024-09-13T08:52:02.316895+0000","last_fullsized":"2024-09-13T08:52:02.316895+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:30:11.266958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742954+0000","last_change":"2024-09-13T08:52:01.312785+0000","last_active":"2024-09-13T08:52:02.742954+0000","last_peered":"2024-09-13T08:52:02.742954+0000","last_clean":"2024-09-13T08:52:02.742954+0000","last_became_active":"2024-09-13T08:52:01.312502+0000","last_became_peered":"2024-09-13T08:52:01.312502+0000","last_unstale":"2024-09-13T08:52:02.742954+0000","last_undegraded":"2024-09-13T08:52:02.742954+0000","last_fullsized":"2024-09-13T08:52:02.742954+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:54:50.643171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.742751+0000","last_change":"2024-09-13T08:52:00.305996+0000","last_active":"2024-09-13T08:52:02.742751+0000","last_peered":"2024-09-13T08:52:02.742751+0000","last_clean":"2024-09-13T08:52:02.742751+0000","last_became_active":"2024-09-13T08:52:00.305186+0000","last_became_peered":"2024-09-13T08:52:00.305186+0000","last_unstale":"2024-09-13T08:52:02.742751+0000","last_undegraded":"2024-09-13T08:52:02.742751+0000","last_fullsized":"2024-09-13T08:52:02.742751+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:55.693460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.741547+0000","last_change":"2024-09-13T08:52:01.304641+0000","last_active":"2024-09-13T08:52:02.741547+0000","last_peered":"2024-09-13T08:52:02.741547+0000","last_clean":"2024-09-13T08:52:02.741547+0000","last_became_active":"2024-09-13T08:52:01.304281+0000","last_became_peered":"2024-09-13T08:52:01.304281+0000","last_unstale":"2024-09-13T08:52:02.741547+0000","last_undegraded":"2024-09-13T08:52:02.741547+0000","last_fullsized":"2024-09-13T08:52:02.741547+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:52:00.278458+0000","last_clean_scrub_stamp":"2024-09-13T08:52:00.278458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:16:02.732823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"130'2","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.311717+0000","last_change":"2024-09-13T08:52:00.308341+0000","last_active":"2024-09-13T08:52:02.311717+0000","last_peered":"2024-09-13T08:52:02.311717+0000","last_clean":"2024-09-13T08:52:02.311717+0000","last_became_active":"2024-09-13T08:52:00.308032+0000","last_became_peered":"2024-09-13T08:52:00.308032+0000","last_unstale":"2024-09-13T08:52:02.311717+0000","last_undegraded":"2024-09-13T08:52:02.311717+0000","last_fullsized":"2024-09-13T08:52:02.311717+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:51:59.268667+0000","last_clean_scrub_stamp":"2024-09-13T08:51:59.268667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:46:27.815150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":321,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T08:52:02.740679+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:52:02.740679+0000","last_peered":"2024-09-13T08:52:02.740679+0000","last_clean":"2024-09-13T08:52:02.740679+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:52:02.740679+0000","last_undegraded":"2024-09-13T08:52:02.740679+0000","last_fullsized":"2024-09-13T08:52:02.740679+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509618,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127124,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244716,"statfs":{"total":96636764160,"available":96506589184,"internally_reserved":0,"allocated":1630208,"data_stored":717698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509618,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131864,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239976,"statfs":{"total":96636764160,"available":96501735424,"internally_reserved":0,"allocated":2232320,"data_stored":1308774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509619,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149328,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222512,"statfs":{"total":96636764160,"available":96483852288,"internally_reserved":0,"allocated":1626112,"data_stored":717664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509619,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131280,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240560,"statfs":{"total":96636764160,"available":96502333440,"internally_reserved":0,"allocated":1630208,"data_stored":717734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509618,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127128,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244712,"statfs":{"total":96636764160,"available":96506585088,"internally_reserved":0,"allocated":1634304,"data_stored":718336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509620,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145764,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226076,"statfs":{"total":96636764160,"available":96487501824,"internally_reserved":0,"allocated":2236416,"data_stored":1309666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509618,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145188,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226652,"statfs":{"total":96636764160,"available":96488091648,"internally_reserved":0,"allocated":1646592,"data_stored":719332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542324,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136840,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235000,"statfs":{"total":96636764160,"available":96496640000,"internally_reserved":0,"allocated":1626112,"data_stored":717664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:52:05.870 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:52:05.870 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T08:52:06.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 -- 172.21.15.44:0/301851199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67840cd100 msgr2=0x7f678406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 --2- 172.21.15.44:0/301851199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67840cd100 0x7f678406f7b0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f67840708e0 tx=0x7f676c01c7a0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 -- 172.21.15.44:0/301851199 shutdown_connections 2024-09-13T08:52:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 --2- 172.21.15.44:0/301851199 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67840b49e0 0x7f67840acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 --2- 172.21.15.44:0/301851199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67840cd100 0x7f678406f7b0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 -- 172.21.15.44:0/301851199 >> 172.21.15.44:0/301851199 conn(0x7f67840b18f0 msgr2=0x7f67840b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 -- 172.21.15.44:0/301851199 shutdown_connections 2024-09-13T08:52:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 -- 172.21.15.44:0/301851199 wait complete. 2024-09-13T08:52:06.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 Processor -- start 2024-09-13T08:52:06.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 -- start start 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.065+0000 7f678a800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67840b49e0 0x7f67841fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f678a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67840cd100 0x7f67841649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f678a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678415dae0 0x7f678409d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f678a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f67840cf0f0 con 0x7f678415dae0 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f678a800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67840cef70 con 0x7f67840cd100 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f678a800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f67840cf270 con 0x7f67840b49e0 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678415dae0 0x7f678409d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678415dae0 0x7f678409d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36838/0 (socket says 172.21.15.44:36838) 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783e00640 1 -- 172.21.15.44:0/4011528883 learned_addr learned my addr 172.21.15.44:0/4011528883 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f677be00640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67840cd100 0x7f67841649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:06.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783400640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67840b49e0 0x7f67841fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:06.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783e00640 1 -- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67840b49e0 msgr2=0x7f67841fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:06.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783e00640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67840b49e0 0x7f67841fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783e00640 1 -- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67840cd100 msgr2=0x7f67841649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:06.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783e00640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67840cd100 0x7f67841649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783e00640 1 -- 172.21.15.44:0/4011528883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67841fbe80 con 0x7f678415dae0 2024-09-13T08:52:06.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783400640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67840b49e0 0x7f67841fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:06.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f6783e00640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678415dae0 0x7f678409d2a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f677c00dd80 tx=0x7f677c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f677b400640 1 -- 172.21.15.44:0/4011528883 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c004d30 con 0x7f678415dae0 2024-09-13T08:52:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f677b400640 1 -- 172.21.15.44:0/4011528883 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f677c00be40 con 0x7f678415dae0 2024-09-13T08:52:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f677b400640 1 -- 172.21.15.44:0/4011528883 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c00e8c0 con 0x7f678415dae0 2024-09-13T08:52:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f678a800640 1 -- 172.21.15.44:0/4011528883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6784164f10 con 0x7f678415dae0 2024-09-13T08:52:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f678a800640 1 -- 172.21.15.44:0/4011528883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6784163ef0 con 0x7f678415dae0 2024-09-13T08:52:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f678a800640 1 -- 172.21.15.44:0/4011528883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6784069410 con 0x7f678415dae0 2024-09-13T08:52:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.069+0000 7f677b400640 1 -- 172.21.15.44:0/4011528883 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f677c00ea60 con 0x7f678415dae0 2024-09-13T08:52:06.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.073+0000 7f677b400640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6760074db0 0x7f6760077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:06.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.073+0000 7f677b400640 1 -- 172.21.15.44:0/4011528883 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f677c093ac0 con 0x7f678415dae0 2024-09-13T08:52:06.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.073+0000 7f6783400640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6760074db0 0x7f6760077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:06.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.073+0000 7f6783400640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6760074db0 0x7f6760077270 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f67740098b0 tx=0x7f6774008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:06.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.077+0000 7f677b400640 1 -- 172.21.15.44:0/4011528883 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f67841fbe80 con 0x7f678415dae0 2024-09-13T08:52:06.260 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-09-13T08:52:06.261+0000 7fe512000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:52:06.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.317+0000 7f678a800640 1 -- 172.21.15.44:0/4011528883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f67840add00 con 0x7f678415dae0 2024-09-13T08:52:06.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f677b400640 1 -- 172.21.15.44:0/4011528883 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f677c060770 con 0x7f678415dae0 2024-09-13T08:52:06.320 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T08:52:06.320 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:52:06.320 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-09-13T08:52:06.320 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-09-13T08:52:06.321 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-13T08:52:06.321 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-09-13T08:52:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 -- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6760074db0 msgr2=0x7f6760077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6760074db0 0x7f6760077270 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f67740098b0 tx=0x7f6774008040 comp rx=0 tx=0).stop 2024-09-13T08:52:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 -- 172.21.15.44:0/4011528883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678415dae0 msgr2=0x7f678409d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678415dae0 0x7f678409d2a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f677c00dd80 tx=0x7f677c00b730 comp rx=0 tx=0).stop 2024-09-13T08:52:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 -- 172.21.15.44:0/4011528883 shutdown_connections 2024-09-13T08:52:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6760074db0 0x7f6760077270 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678415dae0 0x7f678409d2a0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67840cd100 0x7f67841649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 --2- 172.21.15.44:0/4011528883 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f67840b49e0 0x7f67841fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 -- 172.21.15.44:0/4011528883 >> 172.21.15.44:0/4011528883 conn(0x7f67840b18f0 msgr2=0x7f678415c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:06.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 -- 172.21.15.44:0/4011528883 shutdown_connections 2024-09-13T08:52:06.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.321+0000 7f678a800640 1 -- 172.21.15.44:0/4011528883 wait complete. 2024-09-13T08:52:06.343 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:52:06.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.489+0000 7f00f7200640 1 -- 172.21.15.44:0/2847527209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f00d28e0 msgr2=0x7f00f00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:06.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.489+0000 7f00f7200640 1 --2- 172.21.15.44:0/2847527209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f00d28e0 0x7f00f00cebb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009c80 tx=0x7f00e801cda0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.489+0000 7f00f7200640 1 -- 172.21.15.44:0/2847527209 shutdown_connections 2024-09-13T08:52:06.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.489+0000 7f00f7200640 1 --2- 172.21.15.44:0/2847527209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f00d28e0 0x7f00f00cebb0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.489+0000 7f00f7200640 1 --2- 172.21.15.44:0/2847527209 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00f015bea0 0x7f00f00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.489+0000 7f00f7200640 1 -- 172.21.15.44:0/2847527209 >> 172.21.15.44:0/2847527209 conn(0x7f00f00d1dc0 msgr2=0x7f00f00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:06.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.489+0000 7f00f7200640 1 -- 172.21.15.44:0/2847527209 shutdown_connections 2024-09-13T08:52:06.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.489+0000 7f00f7200640 1 -- 172.21.15.44:0/2847527209 wait complete. 2024-09-13T08:52:06.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.489+0000 7f00f7200640 1 Processor -- start 2024-09-13T08:52:06.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.489+0000 7f00f7200640 1 -- start start 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f7200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00f00cdb30 0x7f00f0159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f7200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00f00d28e0 0x7f00f00cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f015bea0 0x7f00f00717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0169020 con 0x7f00f015bea0 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f7200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0168ea0 con 0x7f00f00cdb30 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f7200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f00f01691a0 con 0x7f00f00d28e0 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f015bea0 0x7f00f00717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f015bea0 0x7f00f00717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36848/0 (socket says 172.21.15.44:36848) 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f5200640 1 -- 172.21.15.44:0/2417816545 learned_addr learned my addr 172.21.15.44:0/2417816545 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00e7e00640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00f00d28e0 0x7f00f00cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00efe00640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00f00cdb30 0x7f00f0159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:06.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f5200640 1 -- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00f00d28e0 msgr2=0x7f00f00cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f5200640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00f00d28e0 0x7f00f00cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f5200640 1 -- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00f00cdb30 msgr2=0x7f00f0159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f5200640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00f00cdb30 0x7f00f0159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f5200640 1 -- 172.21.15.44:0/2417816545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f01fb400 con 0x7f00f015bea0 2024-09-13T08:52:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f5200640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f015bea0 0x7f00f00717c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009bd0 tx=0x7f00e80a5ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00ecc00640 1 -- 172.21.15.44:0/2417816545 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e80bb070 con 0x7f00f015bea0 2024-09-13T08:52:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00ecc00640 1 -- 172.21.15.44:0/2417816545 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00e8004d40 con 0x7f00f015bea0 2024-09-13T08:52:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f7200640 1 -- 172.21.15.44:0/2417816545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f01fb6e0 con 0x7f00f015bea0 2024-09-13T08:52:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00ecc00640 1 -- 172.21.15.44:0/2417816545 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e80b2040 con 0x7f00f015bea0 2024-09-13T08:52:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f7200640 1 -- 172.21.15.44:0/2417816545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f01fbca0 con 0x7f00f015bea0 2024-09-13T08:52:06.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.493+0000 7f00f7200640 1 -- 172.21.15.44:0/2417816545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f00cc9d0 con 0x7f00f015bea0 2024-09-13T08:52:06.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.497+0000 7f00ecc00640 1 -- 172.21.15.44:0/2417816545 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f00e80be070 con 0x7f00f015bea0 2024-09-13T08:52:06.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.497+0000 7f00ecc00640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00cc074db0 0x7f00cc077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:06.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.497+0000 7f00ecc00640 1 -- 172.21.15.44:0/2417816545 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f00e812cc70 con 0x7f00f015bea0 2024-09-13T08:52:06.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.497+0000 7f00efe00640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00cc074db0 0x7f00cc077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.497+0000 7f00efe00640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00cc074db0 0x7f00cc077270 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f00e0006fd0 tx=0x7f00e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:06.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.501+0000 7f00ecc00640 1 -- 172.21.15.44:0/2417816545 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f00e8016990 con 0x7f00f015bea0 2024-09-13T08:52:06.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 -- 172.21.15.44:0/2417816545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00f0070d00 con 0x7f00f015bea0 2024-09-13T08:52:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00ecc00640 1 -- 172.21.15.44:0/2417816545 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5228 (secure 0 0 0) 0x7f00e80f99c0 con 0x7f00f015bea0 2024-09-13T08:52:06.712 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:06.712 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":85,"btime":"2024-09-13T08:52:04:315400+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/771921648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":771921648},{"type":"v1","addr":"172.21.15.129:6836","nonce":771921648}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8104,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1464443005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1464443005},{"type":"v1","addr":"172.21.15.129:6837","nonce":1464443005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2601537201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2601537201},{"type":"v1","addr":"172.21.15.44:6833","nonce":2601537201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:52:00.698019+0000","modified":"2024-09-13T08:52:03.712285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8763":{"gid":8763,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2181881572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2181881572},{"type":"v1","addr":"172.21.15.44:6835","nonce":2181881572}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8763,"qdb_cluster":[8763]},"id":9}]} 2024-09-13T08:52:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 85 2024-09-13T08:52:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 -- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00cc074db0 msgr2=0x7f00cc077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00cc074db0 0x7f00cc077270 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f00e0006fd0 tx=0x7f00e0008040 comp rx=0 tx=0).stop 2024-09-13T08:52:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 -- 172.21.15.44:0/2417816545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f015bea0 msgr2=0x7f00f00717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f015bea0 0x7f00f00717c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009bd0 tx=0x7f00e80a5ba0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 -- 172.21.15.44:0/2417816545 shutdown_connections 2024-09-13T08:52:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f00cc074db0 0x7f00cc077270 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f015bea0 0x7f00f00717c0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f00f00d28e0 0x7f00f00cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 --2- 172.21.15.44:0/2417816545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00f00cdb30 0x7f00f0159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 -- 172.21.15.44:0/2417816545 >> 172.21.15.44:0/2417816545 conn(0x7f00f00d1dc0 msgr2=0x7f00f00b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 -- 172.21.15.44:0/2417816545 shutdown_connections 2024-09-13T08:52:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:06.713+0000 7f00f7200640 1 -- 172.21.15.44:0/2417816545 wait complete. 2024-09-13T08:52:07.136 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T08:52:00.698019+0000', 'modified': '2024-09-13T08:52:03.712285+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8763}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8763': {'gid': 8763, 'name': 'a', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/2181881572', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 2181881572}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2181881572}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8763, 'qdb_cluster': [8763]} 2024-09-13T08:52:07.137 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T08:52:07.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T08:52:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.289+0000 7f9470c00640 1 -- 172.21.15.44:0/3813783594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c0d28e0 msgr2=0x7f946c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.289+0000 7f9470c00640 1 --2- 172.21.15.44:0/3813783594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c0d28e0 0x7f946c0cebb0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f946c0745d0 tx=0x7f946401caf0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.289+0000 7f9470c00640 1 -- 172.21.15.44:0/3813783594 shutdown_connections 2024-09-13T08:52:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.289+0000 7f9470c00640 1 --2- 172.21.15.44:0/3813783594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c0d28e0 0x7f946c0cebb0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.289+0000 7f9470c00640 1 --2- 172.21.15.44:0/3813783594 >> v2:172.21.15.129:3300/0 conn(0x7f946c15bea0 0x7f946c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.289+0000 7f9470c00640 1 -- 172.21.15.44:0/3813783594 >> 172.21.15.44:0/3813783594 conn(0x7f946c0d1dc0 msgr2=0x7f946c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.289+0000 7f9470c00640 1 -- 172.21.15.44:0/3813783594 shutdown_connections 2024-09-13T08:52:07.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.289+0000 7f9470c00640 1 -- 172.21.15.44:0/3813783594 wait complete. 2024-09-13T08:52:07.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.289+0000 7f9470c00640 1 Processor -- start 2024-09-13T08:52:07.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.289+0000 7f9470c00640 1 -- start start 2024-09-13T08:52:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9470c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f946c0cdb30 0x7f946c1660b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9470c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f946c0d28e0 0x7f946c06f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9470c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c15bea0 0x7f946c0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9470c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f946c168fc0 con 0x7f946c15bea0 2024-09-13T08:52:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9470c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f946c168e40 con 0x7f946c0cdb30 2024-09-13T08:52:07.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9470c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f946c169140 con 0x7f946c0d28e0 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f946a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c15bea0 0x7f946c0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f946a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c15bea0 0x7f946c0b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36860/0 (socket says 172.21.15.44:36860) 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f946a800640 1 -- 172.21.15.44:0/1575959505 learned_addr learned my addr 172.21.15.44:0/1575959505 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9469e00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f946c0cdb30 0x7f946c1660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9463e00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f946c0d28e0 0x7f946c06f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f946a800640 1 -- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f946c0d28e0 msgr2=0x7f946c06f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f946a800640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f946c0d28e0 0x7f946c06f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f946a800640 1 -- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f946c0cdb30 msgr2=0x7f946c1660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f946a800640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f946c0cdb30 0x7f946c1660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f946a800640 1 -- 172.21.15.44:0/1575959505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c06fab0 con 0x7f946c15bea0 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9463e00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f946c0d28e0 0x7f946c06f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:52:07.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f946a800640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c15bea0 0x7f946c0b53f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f9464019700 tx=0x7f94640a58f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:07.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9462000640 1 -- 172.21.15.44:0/1575959505 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94640ba020 con 0x7f946c15bea0 2024-09-13T08:52:07.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9462000640 1 -- 172.21.15.44:0/1575959505 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9464004d40 con 0x7f946c15bea0 2024-09-13T08:52:07.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9470c00640 1 -- 172.21.15.44:0/1575959505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946c06deb0 con 0x7f946c15bea0 2024-09-13T08:52:07.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9462000640 1 -- 172.21.15.44:0/1575959505 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94640b2040 con 0x7f946c15bea0 2024-09-13T08:52:07.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.293+0000 7f9470c00640 1 -- 172.21.15.44:0/1575959505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946c06e470 con 0x7f946c15bea0 2024-09-13T08:52:07.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9462000640 1 -- 172.21.15.44:0/1575959505 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f9464004300 con 0x7f946c15bea0 2024-09-13T08:52:07.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9462000640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9448074db0 0x7f9448077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9473f6a000 1 Processor -- start 2024-09-13T08:52:07.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9462000640 1 -- 172.21.15.44:0/1575959505 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f946412d670 con 0x7f946c15bea0 2024-09-13T08:52:07.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9473f6a000 1 -- start start 2024-09-13T08:52:07.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9469e00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9448074db0 0x7f9448077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4eb9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eba2f0 0x5651a4e6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4e6de10 0x5651a4e6e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9473f6a000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5651a4d8e3f0 con 0x5651a4eb99d0 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9473f6a000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5651a4ccbb20 con 0x5651a4e6de10 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9473f6a000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x5651a4cbaf50 con 0x5651a4eba2f0 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9469e00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9448074db0 0x7f9448077270 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f945c0058d0 tx=0x7f945c005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9469e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4eb9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f946a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4e6de10 0x5651a4e6e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9469e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4eb9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36870/0 (socket says 172.21.15.44:36870) 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9463e00640 1 --2- 172.21.15.44:0/2390698416 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eba2f0 0x5651a4e6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9469e00640 1 -- 172.21.15.44:0/2390698416 learned_addr learned my addr 172.21.15.44:0/2390698416 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:07.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9469e00640 1 -- 172.21.15.44:0/2390698416 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eba2f0 msgr2=0x5651a4e6d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:52:07.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9469e00640 1 --2- 172.21.15.44:0/2390698416 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eba2f0 0x5651a4e6d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9469e00640 1 -- 172.21.15.44:0/2390698416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4e6de10 msgr2=0x5651a4e6e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:52:07.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9469e00640 1 --2- 172.21.15.44:0/2390698416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4e6de10 0x5651a4e6e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.297+0000 7f9469e00640 1 -- 172.21.15.44:0/2390698416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a4c82590 con 0x5651a4eb99d0 2024-09-13T08:52:07.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9469e00640 1 --2- 172.21.15.44:0/2390698416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4eb9db0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f945c00fdd0 tx=0x7f945c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b814e0ade8aa37 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:07.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f945aa00640 1 -- 172.21.15.44:0/2390698416 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c022070 con 0x5651a4eb99d0 2024-09-13T08:52:07.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f945aa00640 1 -- 172.21.15.44:0/2390698416 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f945c01e450 con 0x5651a4eb99d0 2024-09-13T08:52:07.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f945aa00640 1 -- 172.21.15.44:0/2390698416 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c01e860 con 0x5651a4eb99d0 2024-09-13T08:52:07.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 -- 172.21.15.44:0/2390698416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 msgr2=0x5651a4eb9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 --2- 172.21.15.44:0/2390698416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4eb9db0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f945c00fdd0 tx=0x7f945c012000 comp rx=0 tx=0).stop 2024-09-13T08:52:07.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 -- 172.21.15.44:0/2390698416 shutdown_connections 2024-09-13T08:52:07.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 --2- 172.21.15.44:0/2390698416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4e6de10 0x5651a4e6e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 --2- 172.21.15.44:0/2390698416 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eba2f0 0x5651a4e6d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 --2- 172.21.15.44:0/2390698416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4eb9db0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 -- 172.21.15.44:0/2390698416 >> 172.21.15.44:0/2390698416 conn(0x5651a4d57130 msgr2=0x5651a4e9ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:07.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 -- 172.21.15.44:0/2390698416 shutdown_connections 2024-09-13T08:52:07.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 -- 172.21.15.44:0/2390698416 wait complete. 2024-09-13T08:52:07.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 -- start start 2024-09-13T08:52:07.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 Processor -- start 2024-09-13T08:52:07.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4e6de10 0x5651a4e6f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eb99d0 0x5651a4e6f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 0x5651a4e6fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5651a4c555d0 con 0x5651a4e6de10 2024-09-13T08:52:07.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5651a4cbed60 con 0x5651a4eba2f0 2024-09-13T08:52:07.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x5651a4c61500 con 0x5651a4eb99d0 2024-09-13T08:52:07.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9473f6a000 10 client.0 _ll_register_callbacks cb 0x5651a4c65810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:52:07.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9469e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4e6de10 0x5651a4e6f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9469e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4e6de10 0x5651a4e6f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36872/0 (socket says 172.21.15.44:36872) 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9469e00640 1 -- 172.21.15.44:0/3443701189 learned_addr learned my addr 172.21.15.44:0/3443701189 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f9463e00640 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eb99d0 0x5651a4e6f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.301+0000 7f946a800640 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 0x5651a4e6fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9469e00640 1 -- 172.21.15.44:0/3443701189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eb99d0 msgr2=0x5651a4e6f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9469e00640 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eb99d0 0x5651a4e6f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9469e00640 1 -- 172.21.15.44:0/3443701189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 msgr2=0x5651a4e6fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9469e00640 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 0x5651a4e6fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9469e00640 1 -- 172.21.15.44:0/3443701189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a4c59170 con 0x5651a4e6de10 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f946a800640 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 0x5651a4e6fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9469e00640 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4e6de10 0x5651a4e6f240 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c010 tx=0x7f945c00b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bcdcf065a859df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f944f400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:52:07.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f944f400640 1 -- 172.21.15.44:0/3443701189 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c035060 con 0x5651a4e6de10 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 -- 172.21.15.44:0/3443701189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651a4c530f0 con 0x5651a4e6de10 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f944f400640 1 -- 172.21.15.44:0/3443701189 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f945c0205f0 con 0x5651a4e6de10 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f944f400640 1 -- 172.21.15.44:0/3443701189 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f945c03b050 con 0x5651a4e6de10 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 10 client.8883 fetch_fsmap learned FSMap version 85 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 -- 172.21.15.44:0/3443701189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x5651a4d3fef0 con 0x5651a4e6de10 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f944f400640 1 -- 172.21.15.44:0/3443701189 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5651a4c530f0 con 0x5651a4e6de10 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f944f400640 1 -- 172.21.15.44:0/3443701189 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7f945c03e020 con 0x5651a4e6de10 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 10 client.8883 fetch_fsmap finished waiting for FSMap version 85 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 10 client.8883 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 20 client.8883 populate_metadata read hostname 'smithi044' 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.44:6834/2181881572,v1:172.21.15.44:6835/2181881572] conn(0x5651a4ec8c70 0x5651a4ecb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 4 client.8883 mds_command: new command op to 8763 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T08:52:07.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 -- 172.21.15.44:0/3443701189 --> [v2:172.21.15.44:6834/2181881572,v1:172.21.15.44:6835/2181881572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a4ccda70 con 0x5651a4ec8c70 2024-09-13T08:52:07.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9463e00640 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.44:6834/2181881572,v1:172.21.15.44:6835/2181881572] conn(0x5651a4ec8c70 0x5651a4ecb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9463e00640 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.44:6834/2181881572,v1:172.21.15.44:6835/2181881572] conn(0x5651a4ec8c70 0x5651a4ecb060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=876d196ef98f7d8b server_cookie=d9c97d7e8f7971f3 in_seq=0 out_seq=0 2024-09-13T08:52:07.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f944f400640 10 client.8883 ms_handle_connect on v2:172.21.15.44:6834/2181881572 2024-09-13T08:52:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f944f400640 1 -- 172.21.15.44:0/3443701189 <== mds.0 v2:172.21.15.44:6834/2181881572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651a4ccda70 con 0x5651a4ec8c70 2024-09-13T08:52:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f944f400640 10 client.8883 handle_command_reply: tid=0 2024-09-13T08:52:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 client.8883 shutdown 2024-09-13T08:52:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 -- 172.21.15.44:0/3443701189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4e6de10 msgr2=0x5651a4e6f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4e6de10 0x5651a4e6f240 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c010 tx=0x7f945c00b3c0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 -- 172.21.15.44:0/3443701189 shutdown_connections 2024-09-13T08:52:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.44:6834/2181881572,v1:172.21.15.44:6835/2181881572] conn(0x5651a4ec8c70 0x5651a4ecb060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 0x5651a4e6fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eb99d0 0x5651a4e6f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3443701189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4e6de10 0x5651a4e6f240 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 -- 172.21.15.44:0/3443701189 >> 172.21.15.44:0/3443701189 conn(0x5651a4d57130 msgr2=0x5651a4e6cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:07.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f944f400640 0 client.8883 ms_handle_reset on v2:172.21.15.44:6834/2181881572 2024-09-13T08:52:07.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 -- 172.21.15.44:0/3443701189 shutdown_connections 2024-09-13T08:52:07.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 1 -- 172.21.15.44:0/3443701189 wait complete. 2024-09-13T08:52:07.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.305+0000 7f9473f6a000 20 client.8883 trim_cache size 0 max 16384 2024-09-13T08:52:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.325+0000 7f9473f6a000 1 Processor -- start 2024-09-13T08:52:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.325+0000 7f9473f6a000 1 -- start start 2024-09-13T08:52:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.325+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4e6de10 0x5651a4f2b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.325+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4f2bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.325+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 0x5651a4f2c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.325+0000 7f9473f6a000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f946c2030f0 con 0x5651a4eb99d0 2024-09-13T08:52:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.325+0000 7f9473f6a000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f946c0b38e0 con 0x5651a4eba2f0 2024-09-13T08:52:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.325+0000 7f9473f6a000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f946c154af0 con 0x5651a4e6de10 2024-09-13T08:52:07.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9469e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4e6de10 0x5651a4f2b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f946a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 0x5651a4f2c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9463e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4f2bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9469e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4e6de10 0x5651a4f2b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:45530/0 (socket says 172.21.15.44:45530) 2024-09-13T08:52:07.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9469e00640 1 -- 172.21.15.44:0/2986076343 learned_addr learned my addr 172.21.15.44:0/2986076343 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:07.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9463e00640 1 -- 172.21.15.44:0/2986076343 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4e6de10 msgr2=0x5651a4f2b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:52:07.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9463e00640 1 --2- 172.21.15.44:0/2986076343 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4e6de10 0x5651a4f2b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9463e00640 1 -- 172.21.15.44:0/2986076343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 msgr2=0x5651a4f2c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:52:07.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9463e00640 1 --2- 172.21.15.44:0/2986076343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 0x5651a4f2c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9463e00640 1 -- 172.21.15.44:0/2986076343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a4d16d90 con 0x5651a4eb99d0 2024-09-13T08:52:07.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9463e00640 1 --2- 172.21.15.44:0/2986076343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4f2bf30 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x5651a4c70120 tx=0x7f9454001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e12000e873d28d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:07.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f945a000640 1 -- 172.21.15.44:0/2986076343 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454005310 con 0x5651a4eb99d0 2024-09-13T08:52:07.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f945a000640 1 -- 172.21.15.44:0/2986076343 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9454022070 con 0x5651a4eb99d0 2024-09-13T08:52:07.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f945a000640 1 -- 172.21.15.44:0/2986076343 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454015ac0 con 0x5651a4eb99d0 2024-09-13T08:52:07.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9473f6a000 1 -- 172.21.15.44:0/2986076343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 msgr2=0x5651a4f2bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9473f6a000 1 --2- 172.21.15.44:0/2986076343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4f2bf30 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x5651a4c70120 tx=0x7f9454001090 comp rx=0 tx=0).stop 2024-09-13T08:52:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9473f6a000 1 -- 172.21.15.44:0/2986076343 shutdown_connections 2024-09-13T08:52:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9473f6a000 1 --2- 172.21.15.44:0/2986076343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4eba2f0 0x5651a4f2c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9473f6a000 1 --2- 172.21.15.44:0/2986076343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eb99d0 0x5651a4f2bf30 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9473f6a000 1 --2- 172.21.15.44:0/2986076343 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4e6de10 0x5651a4f2b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9473f6a000 1 -- 172.21.15.44:0/2986076343 >> 172.21.15.44:0/2986076343 conn(0x5651a4d57130 msgr2=0x5651a4ecb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9473f6a000 1 -- 172.21.15.44:0/2986076343 shutdown_connections 2024-09-13T08:52:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.329+0000 7f9473f6a000 1 -- 172.21.15.44:0/2986076343 wait complete. 2024-09-13T08:52:07.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 -- start start 2024-09-13T08:52:07.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 Processor -- start 2024-09-13T08:52:07.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4e6de10 0x5651a4ed0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eb99d0 0x5651a4eeb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eba2f0 0x5651a4eebac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5651a4cbe070 con 0x5651a4eba2f0 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5651a4cc3390 con 0x5651a4e6de10 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x5651a4c86ec0 con 0x5651a4eb99d0 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 10 client.0 _ll_register_callbacks cb 0x7f9464005b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f946a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eba2f0 0x5651a4eebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f946a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eba2f0 0x5651a4eebac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36896/0 (socket says 172.21.15.44:36896) 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f946a800640 1 -- 172.21.15.44:0/3228101140 learned_addr learned my addr 172.21.15.44:0/3228101140 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9469e00640 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4e6de10 0x5651a4ed0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9463e00640 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eb99d0 0x5651a4eeb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f946a800640 1 -- 172.21.15.44:0/3228101140 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eb99d0 msgr2=0x5651a4eeb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f946a800640 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eb99d0 0x5651a4eeb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f946a800640 1 -- 172.21.15.44:0/3228101140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4e6de10 msgr2=0x5651a4ed0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f946a800640 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4e6de10 0x5651a4ed0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f946a800640 1 -- 172.21.15.44:0/3228101140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a4c81a90 con 0x5651a4eba2f0 2024-09-13T08:52:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f946a800640 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eba2f0 0x5651a4eebac0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f946401cfd0 tx=0x7f946411f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f034072e0ef520 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:07.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f944f400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:52:07.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f944f400640 1 -- 172.21.15.44:0/3228101140 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94640fa320 con 0x5651a4eba2f0 2024-09-13T08:52:07.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f944f400640 1 -- 172.21.15.44:0/3228101140 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f94640f9e80 con 0x5651a4eba2f0 2024-09-13T08:52:07.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 -- 172.21.15.44:0/3228101140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9454022070 con 0x5651a4eba2f0 2024-09-13T08:52:07.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f944f400640 1 -- 172.21.15.44:0/3228101140 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f94640f9c20 con 0x5651a4eba2f0 2024-09-13T08:52:07.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 10 client.8889 fetch_fsmap learned FSMap version 85 2024-09-13T08:52:07.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 -- 172.21.15.44:0/3228101140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x5651a4c60050 con 0x5651a4eba2f0 2024-09-13T08:52:07.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f944f400640 1 -- 172.21.15.44:0/3228101140 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9454022070 con 0x5651a4eba2f0 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f944f400640 1 -- 172.21.15.44:0/3228101140 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7f94640b5070 con 0x5651a4eba2f0 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 10 client.8889 fetch_fsmap finished waiting for FSMap version 85 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 10 client.8889 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 20 client.8889 populate_metadata read hostname 'smithi044' 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.44:6834/2181881572,v1:172.21.15.44:6835/2181881572] conn(0x5651a4eec000 0x5651a4f2fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 4 client.8889 mds_command: new command op to 8763 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.333+0000 7f9473f6a000 1 -- 172.21.15.44:0/3228101140 --> [v2:172.21.15.44:6834/2181881572,v1:172.21.15.44:6835/2181881572] -- command(tid 0: {"prefix": "status"}) -- 0x5651a4cc9520 con 0x5651a4eec000 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9469e00640 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.44:6834/2181881572,v1:172.21.15.44:6835/2181881572] conn(0x5651a4eec000 0x5651a4f2fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9469e00640 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.44:6834/2181881572,v1:172.21.15.44:6835/2181881572] conn(0x5651a4eec000 0x5651a4f2fc40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c5ff841afe1bdf server_cookie=e9afe0bd8cdbb11e in_seq=0 out_seq=0 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f944f400640 10 client.8889 ms_handle_connect on v2:172.21.15.44:6834/2181881572 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f944f400640 1 -- 172.21.15.44:0/3228101140 <== mds.0 v2:172.21.15.44:6834/2181881572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5651a4cc9520 con 0x5651a4eec000 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f944f400640 10 client.8889 handle_command_reply: tid=0 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 client.8889 shutdown 2024-09-13T08:52:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 -- 172.21.15.44:0/3228101140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eba2f0 msgr2=0x5651a4eebac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eba2f0 0x5651a4eebac0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f946401cfd0 tx=0x7f946411f2f0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 -- 172.21.15.44:0/3228101140 shutdown_connections 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.44:6834/2181881572,v1:172.21.15.44:6835/2181881572] conn(0x5651a4eec000 0x5651a4f2fc40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651a4eba2f0 0x5651a4eebac0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5651a4eb99d0 0x5651a4eeb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 --2- 172.21.15.44:0/3228101140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651a4e6de10 0x5651a4ed0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f944f400640 0 client.8889 ms_handle_reset on v2:172.21.15.44:6834/2181881572 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 -- 172.21.15.44:0/3228101140 >> 172.21.15.44:0/3228101140 conn(0x5651a4d57130 msgr2=0x5651a4ecb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 -- 172.21.15.44:0/3228101140 shutdown_connections 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 1 -- 172.21.15.44:0/3228101140 wait complete. 2024-09-13T08:52:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9473f6a000 20 client.8889 trim_cache size 0 max 16384 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 8763, 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.0508855160000001, 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 85, 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 131, 2024-09-13T08:52:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 131, 2024-09-13T08:52:07.341 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.551835063 2024-09-13T08:52:07.341 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 -- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9448074db0 msgr2=0x7f9448077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9448074db0 0x7f9448077270 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f945c0058d0 tx=0x7f945c005860 comp rx=0 tx=0).stop 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 -- 172.21.15.44:0/1575959505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c15bea0 msgr2=0x7f946c0b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c15bea0 0x7f946c0b53f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f9464019700 tx=0x7f94640a58f0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 -- 172.21.15.44:0/1575959505 shutdown_connections 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9448074db0 0x7f9448077270 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f946c15bea0 0x7f946c0b53f0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f946c0d28e0 0x7f946c06f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 --2- 172.21.15.44:0/1575959505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f946c0cdb30 0x7f946c1660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 -- 172.21.15.44:0/1575959505 >> 172.21.15.44:0/1575959505 conn(0x7f946c0d1dc0 msgr2=0x7f946c0b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 -- 172.21.15.44:0/1575959505 shutdown_connections 2024-09-13T08:52:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.337+0000 7f9470c00640 1 -- 172.21.15.44:0/1575959505 wait complete. 2024-09-13T08:52:07.357 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:52:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 -- 172.21.15.44:0/2425679078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6700d2b50 msgr2=0x7ff6700cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 --2- 172.21.15.44:0/2425679078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6700d2b50 0x7ff6700cf660 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7ff658009840 tx=0x7ff65801c820 comp rx=0 tx=0).stop 2024-09-13T08:52:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 -- 172.21.15.44:0/2425679078 shutdown_connections 2024-09-13T08:52:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 --2- 172.21.15.44:0/2425679078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6700d2b50 0x7ff6700cf660 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 -- 172.21.15.44:0/2425679078 >> 172.21.15.44:0/2425679078 conn(0x7ff6700d3fe0 msgr2=0x7ff6700d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 -- 172.21.15.44:0/2425679078 shutdown_connections 2024-09-13T08:52:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 -- 172.21.15.44:0/2425679078 wait complete. 2024-09-13T08:52:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 Processor -- start 2024-09-13T08:52:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 -- start start 2024-09-13T08:52:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff6700b7ab0 0x7ff6700cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6700d2060 0x7ff670156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6700d2b50 0x7ff6700b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff67016c560 con 0x7ff6700d2b50 2024-09-13T08:52:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff67016c3e0 con 0x7ff6700d2060 2024-09-13T08:52:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.505+0000 7ff676a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff67016c6e0 con 0x7ff6700b7ab0 2024-09-13T08:52:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff674a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6700d2b50 0x7ff6700b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff674a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6700d2b50 0x7ff6700b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36910/0 (socket says 172.21.15.44:36910) 2024-09-13T08:52:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff674a00640 1 -- 172.21.15.44:0/1637951270 learned_addr learned my addr 172.21.15.44:0/1637951270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:07.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff66fe00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff6700b7ab0 0x7ff6700cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff667e00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6700d2060 0x7ff670156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff674a00640 1 -- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff6700b7ab0 msgr2=0x7ff6700cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff674a00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff6700b7ab0 0x7ff6700cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff674a00640 1 -- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6700d2060 msgr2=0x7ff670156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff674a00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6700d2060 0x7ff670156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff674a00640 1 -- 172.21.15.44:0/1637951270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff670069b20 con 0x7ff6700d2b50 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff674a00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6700d2b50 0x7ff6700b1500 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ff66800c730 tx=0x7ff66800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff66cc00640 1 -- 172.21.15.44:0/1637951270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668003db0 con 0x7ff6700d2b50 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff66cc00640 1 -- 172.21.15.44:0/1637951270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff66800ce40 con 0x7ff6700d2b50 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff66cc00640 1 -- 172.21.15.44:0/1637951270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6680052d0 con 0x7ff6700d2b50 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff676a00640 1 -- 172.21.15.44:0/1637951270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6700b0c10 con 0x7ff6700d2b50 2024-09-13T08:52:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff676a00640 1 -- 172.21.15.44:0/1637951270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6700b40c0 con 0x7ff6700d2b50 2024-09-13T08:52:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff676a00640 1 -- 172.21.15.44:0/1637951270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6700cf660 con 0x7ff6700d2b50 2024-09-13T08:52:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.509+0000 7ff66cc00640 1 -- 172.21.15.44:0/1637951270 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7ff668005470 con 0x7ff6700d2b50 2024-09-13T08:52:07.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.513+0000 7ff66cc00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff64c074de0 0x7ff64c0772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:07.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.513+0000 7ff66cc00640 1 -- 172.21.15.44:0/1637951270 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7ff668092b00 con 0x7ff6700d2b50 2024-09-13T08:52:07.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.513+0000 7ff66fe00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff64c074de0 0x7ff64c0772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:07.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.513+0000 7ff66fe00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff64c074de0 0x7ff64c0772a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff660007fd0 tx=0x7ff660009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:07.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.517+0000 7ff66cc00640 1 -- 172.21.15.44:0/1637951270 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff66805f1e0 con 0x7ff6700d2b50 2024-09-13T08:52:07.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 -- 172.21.15.44:0/1637951270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff6700d47d0 con 0x7ff6700d2b50 2024-09-13T08:52:07.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff66cc00640 1 -- 172.21.15.44:0/1637951270 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) ==== 53+0+73 (secure 0 0 0) 0x7ff668063c10 con 0x7ff6700d2b50 2024-09-13T08:52:07.732 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:52:07.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 -- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff64c074de0 msgr2=0x7ff64c0772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff64c074de0 0x7ff64c0772a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff660007fd0 tx=0x7ff660009040 comp rx=0 tx=0).stop 2024-09-13T08:52:07.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 -- 172.21.15.44:0/1637951270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6700d2b50 msgr2=0x7ff6700b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6700d2b50 0x7ff6700b1500 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ff66800c730 tx=0x7ff66800cbf0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 -- 172.21.15.44:0/1637951270 shutdown_connections 2024-09-13T08:52:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff64c074de0 0x7ff64c0772a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6700d2b50 0x7ff6700b1500 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6700d2060 0x7ff670156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 --2- 172.21.15.44:0/1637951270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff6700b7ab0 0x7ff6700cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 -- 172.21.15.44:0/1637951270 >> 172.21.15.44:0/1637951270 conn(0x7ff6700d3fe0 msgr2=0x7ff670160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 -- 172.21.15.44:0/1637951270 shutdown_connections 2024-09-13T08:52:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07.733+0000 7ff676a00640 1 -- 172.21.15.44:0/1637951270 wait complete. 2024-09-13T08:52:07.754 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T08:52:07.754 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:52:07.754 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:52:07.754 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T08:52:07.755 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:52:07.755 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T08:52:07.755 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T08:52:07.755 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:52:07.755 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:52:07.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:07.755 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:52:07.765 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:52:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:52:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:52:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:52:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:52:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:52:07.765 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:52:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:52:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41555sec preferred_lft 41555sec 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:52:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:52:07.767 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:52:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:52:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T08:52:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:52:07.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:07.768 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:52:07.824 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:52:07.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:07.827 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:52:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:52:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:52:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:52:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:52:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:52:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:52:07.885 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:52:07.885 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:52:07.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:52:07.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:52:07.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:52:07.885 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:52:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:52:08.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:52:08.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:08.023 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:52:08.033 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:52:08.033 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T08:52:08.033 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:52:08.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:08.033 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T08:52:08.095 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:52:08.095 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T08:52:08.151 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:52:08.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:52:08.152 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:52:08.206 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:52:08.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:08.206 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:52:08.295 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:52:08.296 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:52:08.296 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:52:08.312 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:52:08.312 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:52:08.312 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:52:08.312 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:52:08.312 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:52:08.320 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:52:08.320 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:52:08.321 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:52:08.321 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:52:08.321 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:52:08.321 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:52:08.321 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:52:08.321 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:52:08.324 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:52:08.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:08.325 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:52:08.345 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:52:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:52:08.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:52:08.453 INFO:teuthology.orchestra.run.smithi044.stdout:client8901 2024-09-13T08:52:08.453 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:52:08.453 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:52:08.462 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-13T08:52:08.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.653+0000 7f2e75e00640 1 -- 172.21.15.44:0/2336972626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e7015c1f0 msgr2=0x7f2e700cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:08.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.653+0000 7f2e75e00640 1 --2- 172.21.15.44:0/2336972626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e7015c1f0 0x7f2e700cee10 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009840 tx=0x7f2e5801f010 comp rx=0 tx=0).stop 2024-09-13T08:52:08.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.653+0000 7f2e75e00640 1 -- 172.21.15.44:0/2336972626 shutdown_connections 2024-09-13T08:52:08.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.653+0000 7f2e75e00640 1 --2- 172.21.15.44:0/2336972626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e7015c1f0 0x7f2e700cee10 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:08.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.653+0000 7f2e75e00640 1 -- 172.21.15.44:0/2336972626 >> 172.21.15.44:0/2336972626 conn(0x7f2e700d1da0 msgr2=0x7f2e700d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:08.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.653+0000 7f2e75e00640 1 -- 172.21.15.44:0/2336972626 shutdown_connections 2024-09-13T08:52:08.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.653+0000 7f2e75e00640 1 -- 172.21.15.44:0/2336972626 wait complete. 2024-09-13T08:52:08.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.653+0000 7f2e75e00640 1 Processor -- start 2024-09-13T08:52:08.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.653+0000 7f2e75e00640 1 -- start start 2024-09-13T08:52:08.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e75e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e700b0b60 0x7f2e7014eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:08.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e75e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e7015be10 0x7f2e701f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:08.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e75e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2e7015c1f0 0x7f2e700b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:08.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e75e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2e700cd670 con 0x7f2e700b0b60 2024-09-13T08:52:08.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e75e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2e700cd4f0 con 0x7f2e7015be10 2024-09-13T08:52:08.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e75e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2e700cd7f0 con 0x7f2e7015c1f0 2024-09-13T08:52:08.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e6ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e700b0b60 0x7f2e7014eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:08.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e6ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e700b0b60 0x7f2e7014eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36934/0 (socket says 172.21.15.44:36934) 2024-09-13T08:52:08.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e6ea00640 1 -- 172.21.15.44:0/1590137833 learned_addr learned my addr 172.21.15.44:0/1590137833 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:08.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e67e00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e7015be10 0x7f2e701f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e6f400640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2e7015c1f0 0x7f2e700b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e6ea00640 1 -- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2e7015c1f0 msgr2=0x7f2e700b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e6ea00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2e7015c1f0 0x7f2e700b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e6ea00640 1 -- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e7015be10 msgr2=0x7f2e701f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e6ea00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e7015be10 0x7f2e701f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e6ea00640 1 -- 172.21.15.44:0/1590137833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e700c5e20 con 0x7f2e700b0b60 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e6ea00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e700b0b60 0x7f2e7014eb80 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c450 tx=0x7f2e6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e67400640 1 -- 172.21.15.44:0/1590137833 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60012830 con 0x7f2e700b0b60 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e67400640 1 -- 172.21.15.44:0/1590137833 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e60017070 con 0x7f2e700b0b60 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e67400640 1 -- 172.21.15.44:0/1590137833 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60012d80 con 0x7f2e700b0b60 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e75e00640 1 -- 172.21.15.44:0/1590137833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e700c6100 con 0x7f2e700b0b60 2024-09-13T08:52:08.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e75e00640 1 -- 172.21.15.44:0/1590137833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e700c66c0 con 0x7f2e700b0b60 2024-09-13T08:52:08.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.657+0000 7f2e75e00640 1 -- 172.21.15.44:0/1590137833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e700cdca0 con 0x7f2e700b0b60 2024-09-13T08:52:08.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.661+0000 7f2e67400640 1 -- 172.21.15.44:0/1590137833 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f2e6001c460 con 0x7f2e700b0b60 2024-09-13T08:52:08.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.661+0000 7f2e67400640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e4c074db0 0x7f2e4c077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:08.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.661+0000 7f2e67400640 1 -- 172.21.15.44:0/1590137833 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f2e60092c30 con 0x7f2e700b0b60 2024-09-13T08:52:08.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.661+0000 7f2e67e00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e4c074db0 0x7f2e4c077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:08.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.661+0000 7f2e67e00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e4c074db0 0x7f2e4c077270 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2e580b2000 tx=0x7f2e5801d850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:08.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.665+0000 7f2e67400640 1 -- 172.21.15.44:0/1590137833 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2e6009b030 con 0x7f2e700b0b60 2024-09-13T08:52:08.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:08.877+0000 7f2e75e00640 1 -- 172.21.15.44:0/1590137833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f2e7015b930 con 0x7f2e700b0b60 2024-09-13T08:52:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e67400640 1 -- 172.21.15.44:0/1590137833 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v86) ==== 136+0+0 (secure 0 0 0) 0x7f2e6005f8e0 con 0x7f2e700b0b60 2024-09-13T08:52:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 -- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e4c074db0 msgr2=0x7f2e4c077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e4c074db0 0x7f2e4c077270 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2e580b2000 tx=0x7f2e5801d850 comp rx=0 tx=0).stop 2024-09-13T08:52:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 -- 172.21.15.44:0/1590137833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e700b0b60 msgr2=0x7f2e7014eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e700b0b60 0x7f2e7014eb80 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c450 tx=0x7f2e6000c910 comp rx=0 tx=0).stop 2024-09-13T08:52:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 -- 172.21.15.44:0/1590137833 shutdown_connections 2024-09-13T08:52:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2e4c074db0 0x7f2e4c077270 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2e7015c1f0 0x7f2e700b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e7015be10 0x7f2e701f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 --2- 172.21.15.44:0/1590137833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e700b0b60 0x7f2e7014eb80 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 -- 172.21.15.44:0/1590137833 >> 172.21.15.44:0/1590137833 conn(0x7f2e700d1da0 msgr2=0x7f2e70070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 -- 172.21.15.44:0/1590137833 shutdown_connections 2024-09-13T08:52:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.341+0000 7f2e75e00640 1 -- 172.21.15.44:0/1590137833 wait complete. 2024-09-13T08:52:09.362 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:52:09.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.509+0000 7fb46b600640 1 -- 172.21.15.44:0/2729395839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb46415c130 msgr2=0x7fb4640cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.509+0000 7fb46b600640 1 --2- 172.21.15.44:0/2729395839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb46415c130 0x7fb4640cd050 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb4640d0780 tx=0x7fb45c01ca80 comp rx=0 tx=0).stop 2024-09-13T08:52:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 -- 172.21.15.44:0/2729395839 shutdown_connections 2024-09-13T08:52:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 --2- 172.21.15.44:0/2729395839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb46415c130 0x7fb4640cd050 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 -- 172.21.15.44:0/2729395839 >> 172.21.15.44:0/2729395839 conn(0x7fb4640d1ea0 msgr2=0x7fb4640d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 -- 172.21.15.44:0/2729395839 shutdown_connections 2024-09-13T08:52:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 -- 172.21.15.44:0/2729395839 wait complete. 2024-09-13T08:52:09.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 Processor -- start 2024-09-13T08:52:09.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 -- start start 2024-09-13T08:52:09.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4640cecf0 0x7fb464166c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:09.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb46415bd50 0x7fb4640714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb46415c130 0x7fb464071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb46416a580 con 0x7fb46415bd50 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb46416a400 con 0x7fb46415c130 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb46b600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb46416a700 con 0x7fb4640cecf0 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb463e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb46415bd50 0x7fb4640714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb463e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb46415bd50 0x7fb4640714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36960/0 (socket says 172.21.15.44:36960) 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb463e00640 1 -- 172.21.15.44:0/2054693550 learned_addr learned my addr 172.21.15.44:0/2054693550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb463e00640 1 -- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4640cecf0 msgr2=0x7fb464166c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb463e00640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4640cecf0 0x7fb464166c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb463e00640 1 -- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb46415c130 msgr2=0x7fb464071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb469600640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb46415c130 0x7fb464071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb463e00640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb46415c130 0x7fb464071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb463e00640 1 -- 172.21.15.44:0/2054693550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46406dd30 con 0x7fb46415bd50 2024-09-13T08:52:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb469600640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb46415c130 0x7fb464071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:52:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.513+0000 7fb463e00640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb46415bd50 0x7fb4640714d0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c4e0 tx=0x7fb45000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.517+0000 7fb460c00640 1 -- 172.21.15.44:0/2054693550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450016020 con 0x7fb46415bd50 2024-09-13T08:52:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.517+0000 7fb460c00640 1 -- 172.21.15.44:0/2054693550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb450012920 con 0x7fb46415bd50 2024-09-13T08:52:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.517+0000 7fb460c00640 1 -- 172.21.15.44:0/2054693550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450012d50 con 0x7fb46415bd50 2024-09-13T08:52:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.517+0000 7fb46b600640 1 -- 172.21.15.44:0/2054693550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4641675a0 con 0x7fb46415bd50 2024-09-13T08:52:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.517+0000 7fb46b600640 1 -- 172.21.15.44:0/2054693550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb464155720 con 0x7fb46415bd50 2024-09-13T08:52:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.517+0000 7fb46b600640 1 -- 172.21.15.44:0/2054693550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4640ce5e0 con 0x7fb46415bd50 2024-09-13T08:52:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.517+0000 7fb460c00640 1 -- 172.21.15.44:0/2054693550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fb450019020 con 0x7fb46415bd50 2024-09-13T08:52:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.517+0000 7fb460c00640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb444074db0 0x7fb444077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:09.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.517+0000 7fb460c00640 1 -- 172.21.15.44:0/2054693550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7fb4500933f0 con 0x7fb46415bd50 2024-09-13T08:52:09.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.517+0000 7fb468c00640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb444074db0 0x7fb444077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:09.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.521+0000 7fb468c00640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb444074db0 0x7fb444077270 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb458000f70 tx=0x7fb4580079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:09.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.521+0000 7fb460c00640 1 -- 172.21.15.44:0/2054693550 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb450020030 con 0x7fb46415bd50 2024-09-13T08:52:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.717+0000 7fb46b600640 1 -- 172.21.15.44:0/2054693550 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb4640d2320 con 0x7fb444074db0 2024-09-13T08:52:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb460c00640 1 -- 172.21.15.44:0/2054693550 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb4640d2320 con 0x7fb444074db0 2024-09-13T08:52:09.720 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:52:09.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:52:09.720 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:52:09.720 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:52:09.721 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:52:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 -- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb444074db0 msgr2=0x7fb444077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb444074db0 0x7fb444077270 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb458000f70 tx=0x7fb4580079c0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 -- 172.21.15.44:0/2054693550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb46415bd50 msgr2=0x7fb4640714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb46415bd50 0x7fb4640714d0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c4e0 tx=0x7fb45000c9a0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 -- 172.21.15.44:0/2054693550 shutdown_connections 2024-09-13T08:52:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb444074db0 0x7fb444077270 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb46415c130 0x7fb464071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb46415bd50 0x7fb4640714d0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 --2- 172.21.15.44:0/2054693550 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb4640cecf0 0x7fb464166c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 -- 172.21.15.44:0/2054693550 >> 172.21.15.44:0/2054693550 conn(0x7fb4640d1ea0 msgr2=0x7fb464157e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 -- 172.21.15.44:0/2054693550 shutdown_connections 2024-09-13T08:52:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.721+0000 7fb46b600640 1 -- 172.21.15.44:0/2054693550 wait complete. 2024-09-13T08:52:09.742 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:52:09.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 -- 172.21.15.44:0/4193993287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0d22c0 msgr2=0x7fe71c166800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:09.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 --2- 172.21.15.44:0/4193993287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0d22c0 0x7fe71c166800 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fe714009bc0 tx=0x7fe71401cba0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 -- 172.21.15.44:0/4193993287 shutdown_connections 2024-09-13T08:52:09.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 --2- 172.21.15.44:0/4193993287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0d22c0 0x7fe71c166800 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 -- 172.21.15.44:0/4193993287 >> 172.21.15.44:0/4193993287 conn(0x7fe71c09aa10 msgr2=0x7fe71c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:09.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 -- 172.21.15.44:0/4193993287 shutdown_connections 2024-09-13T08:52:09.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 -- 172.21.15.44:0/4193993287 wait complete. 2024-09-13T08:52:09.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 Processor -- start 2024-09-13T08:52:09.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 -- start start 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c0b0d10 0x7fe71c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0d22c0 0x7fe71c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe71c15bd40 0x7fe71c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c169eb0 con 0x7fe71c0d22c0 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c169d30 con 0x7fe71c0b0d10 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.893+0000 7fe722200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe71c16a030 con 0x7fe71c15bd40 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe71be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0d22c0 0x7fe71c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe71be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0d22c0 0x7fe71c0b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36978/0 (socket says 172.21.15.44:36978) 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe71be00640 1 -- 172.21.15.44:0/3951751304 learned_addr learned my addr 172.21.15.44:0/3951751304 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe720e00640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c0b0d10 0x7fe71c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe721800640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe71c15bd40 0x7fe71c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe71be00640 1 -- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe71c15bd40 msgr2=0x7fe71c0b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe71be00640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe71c15bd40 0x7fe71c0b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe71be00640 1 -- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c0b0d10 msgr2=0x7fe71c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe71be00640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c0b0d10 0x7fe71c156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe71be00640 1 -- 172.21.15.44:0/3951751304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c0b5b50 con 0x7fe71c0d22c0 2024-09-13T08:52:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe71be00640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0d22c0 0x7fe71c0b4ab0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c480 tx=0x7fe70800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:09.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe718c00640 1 -- 172.21.15.44:0/3951751304 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708012830 con 0x7fe71c0d22c0 2024-09-13T08:52:09.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe718c00640 1 -- 172.21.15.44:0/3951751304 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe708017070 con 0x7fe71c0d22c0 2024-09-13T08:52:09.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe722200640 1 -- 172.21.15.44:0/3951751304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c1650f0 con 0x7fe71c0d22c0 2024-09-13T08:52:09.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe718c00640 1 -- 172.21.15.44:0/3951751304 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708012d80 con 0x7fe71c0d22c0 2024-09-13T08:52:09.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe722200640 1 -- 172.21.15.44:0/3951751304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c1656b0 con 0x7fe71c0d22c0 2024-09-13T08:52:09.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe722200640 1 -- 172.21.15.44:0/3951751304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c0d1bb0 con 0x7fe71c0d22c0 2024-09-13T08:52:09.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.897+0000 7fe718c00640 1 -- 172.21.15.44:0/3951751304 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fe70801c460 con 0x7fe71c0d22c0 2024-09-13T08:52:09.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.901+0000 7fe718c00640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc074db0 0x7fe6fc077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:09.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.901+0000 7fe718c00640 1 -- 172.21.15.44:0/3951751304 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7fe708092a80 con 0x7fe71c0d22c0 2024-09-13T08:52:09.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.901+0000 7fe720e00640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc074db0 0x7fe6fc077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:09.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.901+0000 7fe720e00640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc074db0 0x7fe6fc077270 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe710007fd0 tx=0x7fe710007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:09.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:09.905+0000 7fe718c00640 1 -- 172.21.15.44:0/3951751304 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe70805f180 con 0x7fe71c0d22c0 2024-09-13T08:52:10.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 -- 172.21.15.44:0/3951751304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe71c159680 con 0x7fe71c0d22c0 2024-09-13T08:52:10.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe718c00640 1 -- 172.21.15.44:0/3951751304 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7fe708063bb0 con 0x7fe71c0d22c0 2024-09-13T08:52:10.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 -- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc074db0 msgr2=0x7fe6fc077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc074db0 0x7fe6fc077270 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe710007fd0 tx=0x7fe710007ea0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 -- 172.21.15.44:0/3951751304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0d22c0 msgr2=0x7fe71c0b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0d22c0 0x7fe71c0b4ab0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c480 tx=0x7fe70800c940 comp rx=0 tx=0).stop 2024-09-13T08:52:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 -- 172.21.15.44:0/3951751304 shutdown_connections 2024-09-13T08:52:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe6fc074db0 0x7fe6fc077270 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe71c15bd40 0x7fe71c0b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe71c0d22c0 0x7fe71c0b4ab0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 --2- 172.21.15.44:0/3951751304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c0b0d10 0x7fe71c156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 -- 172.21.15.44:0/3951751304 >> 172.21.15.44:0/3951751304 conn(0x7fe71c09aa10 msgr2=0x7fe71c09adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 -- 172.21.15.44:0/3951751304 shutdown_connections 2024-09-13T08:52:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.081+0000 7fe722200640 1 -- 172.21.15.44:0/3951751304 wait complete. 2024-09-13T08:52:10.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-13T08:52:10.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.249+0000 7ff99de00640 1 -- 172.21.15.44:0/1034395289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9980b0d10 msgr2=0x7ff9980d1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.249+0000 7ff99de00640 1 --2- 172.21.15.44:0/1034395289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9980b0d10 0x7ff9980d1bb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7ff9980cf780 tx=0x7ff98001c600 comp rx=0 tx=0).stop 2024-09-13T08:52:10.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 -- 172.21.15.44:0/1034395289 shutdown_connections 2024-09-13T08:52:10.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 --2- 172.21.15.44:0/1034395289 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9980d20f0 0x7ff9980ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 --2- 172.21.15.44:0/1034395289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9980b0d10 0x7ff9980d1bb0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 --2- 172.21.15.44:0/1034395289 >> v2:172.21.15.129:3300/0 conn(0x7ff99815bd40 0x7ff9980d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 -- 172.21.15.44:0/1034395289 >> 172.21.15.44:0/1034395289 conn(0x7ff99809aa10 msgr2=0x7ff99809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:10.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 -- 172.21.15.44:0/1034395289 shutdown_connections 2024-09-13T08:52:10.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 -- 172.21.15.44:0/1034395289 wait complete. 2024-09-13T08:52:10.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 Processor -- start 2024-09-13T08:52:10.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 -- start start 2024-09-13T08:52:10.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9980b0d10 0x7ff99814b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9980d20f0 0x7ff998154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99815bd40 0x7ff99814bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9980ccdc0 con 0x7ff99815bd40 2024-09-13T08:52:10.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9980ccc40 con 0x7ff9980b0d10 2024-09-13T08:52:10.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99de00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff9980ccf40 con 0x7ff9980d20f0 2024-09-13T08:52:10.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99815bd40 0x7ff99814bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff98fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9980d20f0 0x7ff998154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9980b0d10 0x7ff99814b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99815bd40 0x7ff99814bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36990/0 (socket says 172.21.15.44:36990) 2024-09-13T08:52:10.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff98fe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9980d20f0 0x7ff998154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3301/0 says I am v2:172.21.15.44:45570/0 (socket says 172.21.15.44:45570) 2024-09-13T08:52:10.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99d400640 1 -- 172.21.15.44:0/237638117 learned_addr learned my addr 172.21.15.44:0/237638117 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:10.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99d400640 1 -- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9980d20f0 msgr2=0x7ff998154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99d400640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9980d20f0 0x7ff998154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99d400640 1 -- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9980b0d10 msgr2=0x7ff99814b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99d400640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9980b0d10 0x7ff99814b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99d400640 1 -- 172.21.15.44:0/237638117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9980b7290 con 0x7ff99815bd40 2024-09-13T08:52:10.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff98fe00640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9980d20f0 0x7ff998154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:52:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.253+0000 7ff99d400640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99815bd40 0x7ff99814bb40 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff99000eab0 tx=0x7ff99000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.257+0000 7ff995600640 1 -- 172.21.15.44:0/237638117 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99000ce40 con 0x7ff99815bd40 2024-09-13T08:52:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.257+0000 7ff995600640 1 -- 172.21.15.44:0/237638117 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff990004c20 con 0x7ff99815bd40 2024-09-13T08:52:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.257+0000 7ff995600640 1 -- 172.21.15.44:0/237638117 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990005020 con 0x7ff99815bd40 2024-09-13T08:52:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.257+0000 7ff99de00640 1 -- 172.21.15.44:0/237638117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9980b4b50 con 0x7ff99815bd40 2024-09-13T08:52:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.257+0000 7ff99de00640 1 -- 172.21.15.44:0/237638117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9980b7520 con 0x7ff99815bd40 2024-09-13T08:52:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.257+0000 7ff99de00640 1 -- 172.21.15.44:0/237638117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9980b4660 con 0x7ff99815bd40 2024-09-13T08:52:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.257+0000 7ff995600640 1 -- 172.21.15.44:0/237638117 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7ff9900051c0 con 0x7ff99815bd40 2024-09-13T08:52:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.257+0000 7ff995600640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff974074db0 0x7ff974077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.257+0000 7ff995600640 1 -- 172.21.15.44:0/237638117 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7ff990093af0 con 0x7ff99815bd40 2024-09-13T08:52:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.257+0000 7ff99ca00640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff974074db0 0x7ff974077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.261+0000 7ff99ca00640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff974074db0 0x7ff974077270 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff988010e60 tx=0x7ff988012040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:10.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.261+0000 7ff995600640 1 -- 172.21.15.44:0/237638117 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff990060150 con 0x7ff99815bd40 2024-09-13T08:52:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.437+0000 7ff99de00640 1 -- 172.21.15.44:0/237638117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7ff9980b7860 con 0x7ff99815bd40 2024-09-13T08:52:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.437+0000 7ff995600640 1 -- 172.21.15.44:0/237638117 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v51) ==== 116+0+2 (secure 0 0 0) 0x7ff990064b80 con 0x7ff99815bd40 2024-09-13T08:52:10.439 INFO:teuthology.orchestra.run.smithi044.stdout:4 2024-09-13T08:52:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 -- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff974074db0 msgr2=0x7ff974077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff974074db0 0x7ff974077270 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff988010e60 tx=0x7ff988012040 comp rx=0 tx=0).stop 2024-09-13T08:52:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 -- 172.21.15.44:0/237638117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99815bd40 msgr2=0x7ff99814bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99815bd40 0x7ff99814bb40 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff99000eab0 tx=0x7ff99000ef70 comp rx=0 tx=0).stop 2024-09-13T08:52:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 -- 172.21.15.44:0/237638117 shutdown_connections 2024-09-13T08:52:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff974074db0 0x7ff974077270 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99815bd40 0x7ff99814bb40 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff9980d20f0 0x7ff998154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 --2- 172.21.15.44:0/237638117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9980b0d10 0x7ff99814b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 -- 172.21.15.44:0/237638117 >> 172.21.15.44:0/237638117 conn(0x7ff99809aa10 msgr2=0x7ff998155f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 -- 172.21.15.44:0/237638117 shutdown_connections 2024-09-13T08:52:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.441+0000 7ff99de00640 1 -- 172.21.15.44:0/237638117 wait complete. 2024-09-13T08:52:10.460 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:52:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.609+0000 7f90c2000640 1 -- 172.21.15.44:0/853877599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc071060 msgr2=0x7f90bc0afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.609+0000 7f90c2000640 1 --2- 172.21.15.44:0/853877599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc071060 0x7f90bc0afa10 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f90bc15a910 tx=0x7f90ac01c7a0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.609+0000 7f90c2000640 1 -- 172.21.15.44:0/853877599 shutdown_connections 2024-09-13T08:52:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.609+0000 7f90c2000640 1 --2- 172.21.15.44:0/853877599 >> v2:172.21.15.129:3300/0 conn(0x7f90bc0cd650 0x7f90bc0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.609+0000 7f90c2000640 1 --2- 172.21.15.44:0/853877599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc071060 0x7f90bc0afa10 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.609+0000 7f90c2000640 1 -- 172.21.15.44:0/853877599 >> 172.21.15.44:0/853877599 conn(0x7f90bc0b2060 msgr2=0x7f90bc0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.609+0000 7f90c2000640 1 -- 172.21.15.44:0/853877599 shutdown_connections 2024-09-13T08:52:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.609+0000 7f90c2000640 1 -- 172.21.15.44:0/853877599 wait complete. 2024-09-13T08:52:10.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90c2000640 1 Processor -- start 2024-09-13T08:52:10.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90c2000640 1 -- start start 2024-09-13T08:52:10.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90c2000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc071060 0x7f90bc1f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90c2000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0cd650 0x7f90bc159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90c2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15e910 0x7f90bc15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90c2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc0cfa40 con 0x7f90bc15e910 2024-09-13T08:52:10.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90c2000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc0cf8c0 con 0x7f90bc0cd650 2024-09-13T08:52:10.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90c2000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f90bc0cfbc0 con 0x7f90bc071060 2024-09-13T08:52:10.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15e910 0x7f90bc15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15e910 0x7f90bc15cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37006/0 (socket says 172.21.15.44:37006) 2024-09-13T08:52:10.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bbe00640 1 -- 172.21.15.44:0/1040771923 learned_addr learned my addr 172.21.15.44:0/1040771923 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bb400640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc071060 0x7f90bc1f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90b3e00640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0cd650 0x7f90bc159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bbe00640 1 -- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc071060 msgr2=0x7f90bc1f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bbe00640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc071060 0x7f90bc1f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bbe00640 1 -- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0cd650 msgr2=0x7f90bc159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bbe00640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0cd650 0x7f90bc159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bbe00640 1 -- 172.21.15.44:0/1040771923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc15a2a0 con 0x7f90bc15e910 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90b3e00640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0cd650 0x7f90bc159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bb400640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc071060 0x7f90bc1f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90bbe00640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15e910 0x7f90bc15cd60 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f90b400dd80 tx=0x7f90b400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90b3400640 1 -- 172.21.15.44:0/1040771923 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b400be40 con 0x7f90bc15e910 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90b3400640 1 -- 172.21.15.44:0/1040771923 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90b4003ac0 con 0x7f90bc15e910 2024-09-13T08:52:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90c2000640 1 -- 172.21.15.44:0/1040771923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc1f8090 con 0x7f90bc15e910 2024-09-13T08:52:10.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.613+0000 7f90b3400640 1 -- 172.21.15.44:0/1040771923 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4004030 con 0x7f90bc15e910 2024-09-13T08:52:10.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.617+0000 7f90c2000640 1 -- 172.21.15.44:0/1040771923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc1f8650 con 0x7f90bc15e910 2024-09-13T08:52:10.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.617+0000 7f90c2000640 1 -- 172.21.15.44:0/1040771923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90bc146ae0 con 0x7f90bc15e910 2024-09-13T08:52:10.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.617+0000 7f90b3400640 1 -- 172.21.15.44:0/1040771923 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f90b40041d0 con 0x7f90bc15e910 2024-09-13T08:52:10.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.617+0000 7f90b3400640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9098074db0 0x7f9098077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.617+0000 7f90b3400640 1 -- 172.21.15.44:0/1040771923 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f90b40939e0 con 0x7f90bc15e910 2024-09-13T08:52:10.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.617+0000 7f90bb400640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9098074db0 0x7f9098077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.621+0000 7f90bb400640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9098074db0 0x7f9098077270 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0af000 tx=0x7f90ac01afe0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:10.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.621+0000 7f90b3400640 1 -- 172.21.15.44:0/1040771923 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f90b4016e90 con 0x7f90bc15e910 2024-09-13T08:52:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.797+0000 7f90c2000640 1 -- 172.21.15.44:0/1040771923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f90bc0af690 con 0x7f90bc15e910 2024-09-13T08:52:10.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90b3400640 1 -- 172.21.15.44:0/1040771923 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v52)=0 v52) ==== 131+0+0 (secure 0 0 0) 0x7f90b4060690 con 0x7f90bc15e910 2024-09-13T08:52:10.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 -- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9098074db0 msgr2=0x7f9098077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9098074db0 0x7f9098077270 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0af000 tx=0x7f90ac01afe0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 -- 172.21.15.44:0/1040771923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15e910 msgr2=0x7f90bc15cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15e910 0x7f90bc15cd60 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f90b400dd80 tx=0x7f90b400b730 comp rx=0 tx=0).stop 2024-09-13T08:52:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 -- 172.21.15.44:0/1040771923 shutdown_connections 2024-09-13T08:52:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9098074db0 0x7f9098077270 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15e910 0x7f90bc15cd60 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc0cd650 0x7f90bc159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 --2- 172.21.15.44:0/1040771923 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90bc071060 0x7f90bc1f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 -- 172.21.15.44:0/1040771923 >> 172.21.15.44:0/1040771923 conn(0x7f90bc0b2060 msgr2=0x7f90bc0b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:10.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 -- 172.21.15.44:0/1040771923 shutdown_connections 2024-09-13T08:52:10.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.801+0000 7f90c2000640 1 -- 172.21.15.44:0/1040771923 wait complete. 2024-09-13T08:52:10.822 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-13T08:52:10.823 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:10.822+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:10.823 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:10.822+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:10.823 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:10.822+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:10.823 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:10.822+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:10.823 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:10.822+0000 7f8ac7000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.973+0000 7f57f6000640 1 -- 172.21.15.44:0/4263769331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f00ce890 msgr2=0x7f57f006eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.973+0000 7f57f6000640 1 --2- 172.21.15.44:0/4263769331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f00ce890 0x7f57f006eb40 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f57f00767a0 tx=0x7f57e801deb0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.973+0000 7f57f6000640 1 -- 172.21.15.44:0/4263769331 shutdown_connections 2024-09-13T08:52:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.973+0000 7f57f6000640 1 --2- 172.21.15.44:0/4263769331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f00ce890 0x7f57f006eb40 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.973+0000 7f57f6000640 1 --2- 172.21.15.44:0/4263769331 >> v2:172.21.15.129:3300/0 conn(0x7f57f015a9d0 0x7f57f00ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.973+0000 7f57f6000640 1 -- 172.21.15.44:0/4263769331 >> 172.21.15.44:0/4263769331 conn(0x7f57f00cfbc0 msgr2=0x7f57f00cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.973+0000 7f57f6000640 1 -- 172.21.15.44:0/4263769331 shutdown_connections 2024-09-13T08:52:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.973+0000 7f57f6000640 1 -- 172.21.15.44:0/4263769331 wait complete. 2024-09-13T08:52:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57f6000640 1 Processor -- start 2024-09-13T08:52:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57f6000640 1 -- start start 2024-09-13T08:52:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57f6000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f57f00734f0 0x7f57f01f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57f6000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57f00ce890 0x7f57f0159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57f6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f015a9d0 0x7f57f01599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57f6000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f57f006f9a0 con 0x7f57f015a9d0 2024-09-13T08:52:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57f6000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57f006f820 con 0x7f57f00ce890 2024-09-13T08:52:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57f6000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f57f006fb20 con 0x7f57f00734f0 2024-09-13T08:52:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f015a9d0 0x7f57f01599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f015a9d0 0x7f57f01599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37022/0 (socket says 172.21.15.44:37022) 2024-09-13T08:52:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57efe00640 1 -- 172.21.15.44:0/55557815 learned_addr learned my addr 172.21.15.44:0/55557815 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57e7e00640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57f00ce890 0x7f57f0159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57efe00640 1 -- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f57f00734f0 msgr2=0x7f57f01f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57ef400640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f57f00734f0 0x7f57f01f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57efe00640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f57f00734f0 0x7f57f01f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57efe00640 1 -- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57f00ce890 msgr2=0x7f57f0159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57efe00640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57f00ce890 0x7f57f0159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57efe00640 1 -- 172.21.15.44:0/55557815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f0150340 con 0x7f57f015a9d0 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57e7e00640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57f00ce890 0x7f57f0159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57ef400640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f57f00734f0 0x7f57f01f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57efe00640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f015a9d0 0x7f57f01599c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f57e800ad90 tx=0x7f57e8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57e7400640 1 -- 172.21.15.44:0/55557815 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e80bd070 con 0x7f57f015a9d0 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57e7400640 1 -- 172.21.15.44:0/55557815 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57e8004130 con 0x7f57f015a9d0 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57e7400640 1 -- 172.21.15.44:0/55557815 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e80b3040 con 0x7f57f015a9d0 2024-09-13T08:52:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57f6000640 1 -- 172.21.15.44:0/55557815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f014faa0 con 0x7f57f015a9d0 2024-09-13T08:52:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.977+0000 7f57f6000640 1 -- 172.21.15.44:0/55557815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f0150060 con 0x7f57f015a9d0 2024-09-13T08:52:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.981+0000 7f57f6000640 1 -- 172.21.15.44:0/55557815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f0069410 con 0x7f57f015a9d0 2024-09-13T08:52:10.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.981+0000 7f57e7400640 1 -- 172.21.15.44:0/55557815 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f57e80c0070 con 0x7f57f015a9d0 2024-09-13T08:52:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.981+0000 7f57e7400640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57cc074db0 0x7f57cc077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.981+0000 7f57e7400640 1 -- 172.21.15.44:0/55557815 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f57e812d0f0 con 0x7f57f015a9d0 2024-09-13T08:52:10.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.981+0000 7f57ef400640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57cc074db0 0x7f57cc077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:10.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.981+0000 7f57ef400640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57cc074db0 0x7f57cc077270 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f57e0001230 tx=0x7f57e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:10.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:10.985+0000 7f57e7400640 1 -- 172.21.15.44:0/55557815 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f57e8017aa0 con 0x7f57f015a9d0 2024-09-13T08:52:11.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.185+0000 7f57f6000640 1 -- 172.21.15.44:0/55557815 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f57f00703b0 con 0x7f57cc074db0 2024-09-13T08:52:11.196 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:11.198+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:11.196 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:11.198+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:11.197 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:11.198+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:11.197 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:11.198+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:11.197 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:11.198+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:11.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.229+0000 7f57e7400640 1 -- 172.21.15.44:0/55557815 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f57f00703b0 con 0x7f57cc074db0 2024-09-13T08:52:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 -- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57cc074db0 msgr2=0x7f57cc077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57cc074db0 0x7f57cc077270 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f57e0001230 tx=0x7f57e0009040 comp rx=0 tx=0).stop 2024-09-13T08:52:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 -- 172.21.15.44:0/55557815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f015a9d0 msgr2=0x7f57f01599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f015a9d0 0x7f57f01599c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f57e800ad90 tx=0x7f57e8005b00 comp rx=0 tx=0).stop 2024-09-13T08:52:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 -- 172.21.15.44:0/55557815 shutdown_connections 2024-09-13T08:52:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f57cc074db0 0x7f57cc077270 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f015a9d0 0x7f57f01599c0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57f00ce890 0x7f57f0159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 --2- 172.21.15.44:0/55557815 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f57f00734f0 0x7f57f01f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 -- 172.21.15.44:0/55557815 >> 172.21.15.44:0/55557815 conn(0x7f57f00cfbc0 msgr2=0x7f57f00b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:11.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 -- 172.21.15.44:0/55557815 shutdown_connections 2024-09-13T08:52:11.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.233+0000 7f57f6000640 1 -- 172.21.15.44:0/55557815 wait complete. 2024-09-13T08:52:11.252 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:52:11.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 -- 172.21.15.44:0/2284063940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7780063e0 msgr2=0x7fe77800cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:11.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 --2- 172.21.15.44:0/2284063940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7780063e0 0x7fe77800cae0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe78c001e60 tx=0x7fe78c01cbe0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 -- 172.21.15.44:0/2284063940 shutdown_connections 2024-09-13T08:52:11.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 --2- 172.21.15.44:0/2284063940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7780063e0 0x7fe77800cae0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 -- 172.21.15.44:0/2284063940 >> 172.21.15.44:0/2284063940 conn(0x7fe7940b18f0 msgr2=0x7fe7940b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:11.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 -- 172.21.15.44:0/2284063940 shutdown_connections 2024-09-13T08:52:11.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 -- 172.21.15.44:0/2284063940 wait complete. 2024-09-13T08:52:11.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 Processor -- start 2024-09-13T08:52:11.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 -- start start 2024-09-13T08:52:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7780063e0 0x7fe79414f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.405+0000 7fe798e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe79409be30 0x7fe7940b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe798e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7940b0cc0 0x7fe7941592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe798e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7940ce990 con 0x7fe7940b0cc0 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe798e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7940ce810 con 0x7fe79409be30 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe798e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe7940ceb10 con 0x7fe7780063e0 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe792800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7940b0cc0 0x7fe7941592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe792800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7940b0cc0 0x7fe7941592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37048/0 (socket says 172.21.15.44:37048) 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe792800640 1 -- 172.21.15.44:0/2213929113 learned_addr learned my addr 172.21.15.44:0/2213929113 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe791e00640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7780063e0 0x7fe79414f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe78be00640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe79409be30 0x7fe7940b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe792800640 1 -- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7780063e0 msgr2=0x7fe79414f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe792800640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7780063e0 0x7fe79414f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe792800640 1 -- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe79409be30 msgr2=0x7fe7940b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe792800640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe79409be30 0x7fe7940b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe792800640 1 -- 172.21.15.44:0/2213929113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7941f7a90 con 0x7fe7940b0cc0 2024-09-13T08:52:11.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe792800640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7940b0cc0 0x7fe7941592e0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe78c01d480 tx=0x7fe78c006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:11.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe78a000640 1 -- 172.21.15.44:0/2213929113 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c003b80 con 0x7fe7940b0cc0 2024-09-13T08:52:11.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe78a000640 1 -- 172.21.15.44:0/2213929113 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe78c0b7070 con 0x7fe7940b0cc0 2024-09-13T08:52:11.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe78a000640 1 -- 172.21.15.44:0/2213929113 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c006380 con 0x7fe7940b0cc0 2024-09-13T08:52:11.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe798e00640 1 -- 172.21.15.44:0/2213929113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7941f7d70 con 0x7fe7940b0cc0 2024-09-13T08:52:11.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe798e00640 1 -- 172.21.15.44:0/2213929113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7941f8330 con 0x7fe7940b0cc0 2024-09-13T08:52:11.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe798e00640 1 -- 172.21.15.44:0/2213929113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe794069410 con 0x7fe7940b0cc0 2024-09-13T08:52:11.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.409+0000 7fe78a000640 1 -- 172.21.15.44:0/2213929113 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7fe78c006520 con 0x7fe7940b0cc0 2024-09-13T08:52:11.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.413+0000 7fe78a000640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe770074de0 0x7fe7700772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:11.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.413+0000 7fe78a000640 1 -- 172.21.15.44:0/2213929113 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7fe78c12dcc0 con 0x7fe7940b0cc0 2024-09-13T08:52:11.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.413+0000 7fe791e00640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe770074de0 0x7fe7700772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:11.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.413+0000 7fe791e00640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe770074de0 0x7fe7700772a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe784001ec0 tx=0x7fe784001740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:11.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.413+0000 7fe78a000640 1 -- 172.21.15.44:0/2213929113 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe78c0168f0 con 0x7fe7940b0cc0 2024-09-13T08:52:11.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.613+0000 7fe798e00640 1 -- 172.21.15.44:0/2213929113 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe7940b4420 con 0x7fe770074de0 2024-09-13T08:52:11.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe78a000640 1 -- 172.21.15.44:0/2213929113 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fe7940b4420 con 0x7fe770074de0 2024-09-13T08:52:11.620 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:11.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 -- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe770074de0 msgr2=0x7fe7700772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:11.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe770074de0 0x7fe7700772a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe784001ec0 tx=0x7fe784001740 comp rx=0 tx=0).stop 2024-09-13T08:52:11.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 -- 172.21.15.44:0/2213929113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7940b0cc0 msgr2=0x7fe7941592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:11.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7940b0cc0 0x7fe7941592e0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe78c01d480 tx=0x7fe78c006c50 comp rx=0 tx=0).stop 2024-09-13T08:52:11.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 -- 172.21.15.44:0/2213929113 shutdown_connections 2024-09-13T08:52:11.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe770074de0 0x7fe7700772a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7940b0cc0 0x7fe7941592e0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe79409be30 0x7fe7940b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 --2- 172.21.15.44:0/2213929113 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe7780063e0 0x7fe79414f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 -- 172.21.15.44:0/2213929113 >> 172.21.15.44:0/2213929113 conn(0x7fe7940b18f0 msgr2=0x7fe794151500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:11.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 -- 172.21.15.44:0/2213929113 shutdown_connections 2024-09-13T08:52:11.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.621+0000 7fe798e00640 1 -- 172.21.15.44:0/2213929113 wait complete. 2024-09-13T08:52:11.642 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:52:11.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 -- 172.21.15.44:0/670047462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11d81605d0 msgr2=0x7f11d80d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:11.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 --2- 172.21.15.44:0/670047462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11d81605d0 0x7f11d80d0690 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f11d8073990 tx=0x7f11c801c800 comp rx=0 tx=0).stop 2024-09-13T08:52:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 -- 172.21.15.44:0/670047462 shutdown_connections 2024-09-13T08:52:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 --2- 172.21.15.44:0/670047462 >> v2:172.21.15.129:3300/0 conn(0x7f11d80b1650 0x7f11d80b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 --2- 172.21.15.44:0/670047462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11d81605d0 0x7f11d80d0690 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 -- 172.21.15.44:0/670047462 >> 172.21.15.44:0/670047462 conn(0x7f11d80d1dd0 msgr2=0x7f11d80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 -- 172.21.15.44:0/670047462 shutdown_connections 2024-09-13T08:52:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 -- 172.21.15.44:0/670047462 wait complete. 2024-09-13T08:52:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 Processor -- start 2024-09-13T08:52:11.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 -- start start 2024-09-13T08:52:11.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11d80aeb00 0x7f11d80c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:11.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d80b1650 0x7f11d814e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:11.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11d81605d0 0x7f11d81f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:11.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f11d80b0810 con 0x7f11d81605d0 2024-09-13T08:52:11.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11d80b0ee0 con 0x7f11d80b1650 2024-09-13T08:52:11.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11df200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f11d80b1220 con 0x7f11d80aeb00 2024-09-13T08:52:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11dd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11d81605d0 0x7f11d81f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11dd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11d81605d0 0x7f11d81f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37076/0 (socket says 172.21.15.44:37076) 2024-09-13T08:52:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11dd200640 1 -- 172.21.15.44:0/2455987548 learned_addr learned my addr 172.21.15.44:0/2455987548 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11d7e00640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11d80aeb00 0x7f11d80c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.793+0000 7f11cfe00640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d80b1650 0x7f11d814e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11dd200640 1 -- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11d80aeb00 msgr2=0x7f11d80c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11dd200640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11d80aeb00 0x7f11d80c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11dd200640 1 -- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d80b1650 msgr2=0x7f11d814e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11dd200640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d80b1650 0x7f11d814e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:11.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11dd200640 1 -- 172.21.15.44:0/2455987548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d81f8890 con 0x7f11d81605d0 2024-09-13T08:52:11.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11dd200640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11d81605d0 0x7f11d81f8350 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f11d000cbf0 tx=0x7f11d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11d4c00640 1 -- 172.21.15.44:0/2455987548 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0016020 con 0x7f11d81605d0 2024-09-13T08:52:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11d4c00640 1 -- 172.21.15.44:0/2455987548 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11d000ab00 con 0x7f11d81605d0 2024-09-13T08:52:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11d4c00640 1 -- 172.21.15.44:0/2455987548 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0012420 con 0x7f11d81605d0 2024-09-13T08:52:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11df200640 1 -- 172.21.15.44:0/2455987548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d80c70e0 con 0x7f11d81605d0 2024-09-13T08:52:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11df200640 1 -- 172.21.15.44:0/2455987548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d80c76a0 con 0x7f11d81605d0 2024-09-13T08:52:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11df200640 1 -- 172.21.15.44:0/2455987548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d80b1a30 con 0x7f11d81605d0 2024-09-13T08:52:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11d4c00640 1 -- 172.21.15.44:0/2455987548 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 51) ==== 99885+0+0 (secure 0 0 0) 0x7f11d0019020 con 0x7f11d81605d0 2024-09-13T08:52:11.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.797+0000 7f11d4c00640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b4074db0 0x7f11b4077270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:11.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.801+0000 7f11d4c00640 1 -- 172.21.15.44:0/2455987548 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f11d0093080 con 0x7f11d81605d0 2024-09-13T08:52:11.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.801+0000 7f11d7e00640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b4074db0 0x7f11b4077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.801+0000 7f11d7e00640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b4074db0 0x7f11b4077270 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f11c80af000 tx=0x7f11c801b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:11.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.805+0000 7f11d4c00640 1 -- 172.21.15.44:0/2455987548 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f11d0020020 con 0x7f11d81605d0 2024-09-13T08:52:11.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:11.813+0000 7f11d4c00640 1 -- 172.21.15.44:0/2455987548 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 52) ==== 99936+0+0 (secure 0 0 0) 0x7f11d005b950 con 0x7f11d81605d0 2024-09-13T08:52:12.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.009+0000 7f11df200640 1 -- 172.21.15.44:0/2455987548 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f11d806dfb0 con 0x7f11b4074db0 2024-09-13T08:52:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.013+0000 7f11d4c00640 1 -- 172.21.15.44:0/2455987548 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f11d806dfb0 con 0x7f11b4074db0 2024-09-13T08:52:12.014 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 -- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b4074db0 msgr2=0x7f11b4077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b4074db0 0x7f11b4077270 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f11c80af000 tx=0x7f11c801b040 comp rx=0 tx=0).stop 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 -- 172.21.15.44:0/2455987548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11d81605d0 msgr2=0x7f11d81f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11d81605d0 0x7f11d81f8350 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f11d000cbf0 tx=0x7f11d000cce0 comp rx=0 tx=0).stop 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 -- 172.21.15.44:0/2455987548 shutdown_connections 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11b4074db0 0x7f11b4077270 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11d81605d0 0x7f11d81f8350 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d80b1650 0x7f11d814e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 --2- 172.21.15.44:0/2455987548 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11d80aeb00 0x7f11d80c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 -- 172.21.15.44:0/2455987548 >> 172.21.15.44:0/2455987548 conn(0x7f11d80d1dd0 msgr2=0x7f11d815bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 -- 172.21.15.44:0/2455987548 shutdown_connections 2024-09-13T08:52:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:12.017+0000 7f11df200640 1 -- 172.21.15.44:0/2455987548 wait complete. 2024-09-13T08:52:12.036 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:12.036 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:52:12.037 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:52:12.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:12.045 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:52:22.058 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:52:22.058 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:52:22.058 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.95122 s, 108 MB/s 2024-09-13T08:52:22.059 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:52:22.059 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:52:22.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:22.069 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:52:31.605 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:52:31.605 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:52:31.605 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.48445 s, 113 MB/s 2024-09-13T08:52:31.606 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:52:31.607 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:52:31.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:31.616 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:52:33.006 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:52:33.011 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:52:41.144 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:52:41.144 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:52:41.145 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.47674 s, 113 MB/s 2024-09-13T08:52:41.146 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-13T08:52:41.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.296+0000 7f9eda400640 1 -- 172.21.15.44:0/2063918308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003a40 msgr2=0x7f9eb8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:41.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.296+0000 7f9eda400640 1 --2- 172.21.15.44:0/2063918308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003a40 0x7f9eb8003e60 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c660 tx=0x7f9ecc01fa40 comp rx=0 tx=0).stop 2024-09-13T08:52:41.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.296+0000 7f9ed3e00640 1 -- 172.21.15.44:0/2063918308 reap_dead start 2024-09-13T08:52:41.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.296+0000 7f9eda400640 1 -- 172.21.15.44:0/2063918308 shutdown_connections 2024-09-13T08:52:41.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.296+0000 7f9eda400640 1 -- 172.21.15.44:0/2063918308 >> 172.21.15.44:0/2063918308 conn(0x7f9ed40d1da0 msgr2=0x7f9ed40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:41.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.296+0000 7f9eda400640 1 -- 172.21.15.44:0/2063918308 shutdown_connections 2024-09-13T08:52:41.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.296+0000 7f9eda400640 1 -- 172.21.15.44:0/2063918308 wait complete. 2024-09-13T08:52:41.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9eda400640 1 Processor -- start 2024-09-13T08:52:41.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9eda400640 1 -- start start 2024-09-13T08:52:41.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9eda400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003a40 0x7f9ed41f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:41.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9eda400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ed406f920 0x7f9ed40b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:41.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9eda400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ed40c8e70 0x7f9ed40c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:41.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9eda400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed40cf3e0 con 0x7f9eb8003a40 2024-09-13T08:52:41.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9eda400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed40cf260 con 0x7f9ed40c8e70 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9eda400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed40cf560 con 0x7f9ed406f920 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003a40 0x7f9ed41f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003a40 0x7f9ed41f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42984/0 (socket says 172.21.15.44:42984) 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed3400640 1 -- 172.21.15.44:0/1265634917 learned_addr learned my addr 172.21.15.44:0/1265634917 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed2a00640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ed406f920 0x7f9ed40b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed3e00640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ed40c8e70 0x7f9ed40c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed3400640 1 -- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ed406f920 msgr2=0x7f9ed40b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed3400640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ed406f920 0x7f9ed40b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed3400640 1 -- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ed40c8e70 msgr2=0x7f9ed40c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed3400640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ed40c8e70 0x7f9ed40c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed3400640 1 -- 172.21.15.44:0/1265634917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed41f5010 con 0x7f9eb8003a40 2024-09-13T08:52:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ed3400640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003a40 0x7f9ed41f49c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc01e310 tx=0x7f9ecc00c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.300+0000 7f9ecb400640 1 -- 172.21.15.44:0/1265634917 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc004e40 con 0x7f9eb8003a40 2024-09-13T08:52:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.304+0000 7f9ecb400640 1 -- 172.21.15.44:0/1265634917 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ecc001840 con 0x7f9eb8003a40 2024-09-13T08:52:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.304+0000 7f9eda400640 1 -- 172.21.15.44:0/1265634917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed41f52f0 con 0x7f9eb8003a40 2024-09-13T08:52:41.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.304+0000 7f9ecb400640 1 -- 172.21.15.44:0/1265634917 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc003980 con 0x7f9eb8003a40 2024-09-13T08:52:41.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.304+0000 7f9eda400640 1 -- 172.21.15.44:0/1265634917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed41f58b0 con 0x7f9eb8003a40 2024-09-13T08:52:41.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.304+0000 7f9eda400640 1 -- 172.21.15.44:0/1265634917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed40ccbd0 con 0x7f9eb8003a40 2024-09-13T08:52:41.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.304+0000 7f9ecb400640 1 -- 172.21.15.44:0/1265634917 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 53) ==== 99987+0+0 (secure 0 0 0) 0x7f9ecc0a9610 con 0x7f9eb8003a40 2024-09-13T08:52:41.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.304+0000 7f9ecb400640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9eb4074ed0 0x7f9eb4077390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:41.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.304+0000 7f9ecb400640 1 -- 172.21.15.44:0/1265634917 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f9ecc12f870 con 0x7f9eb8003a40 2024-09-13T08:52:41.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.304+0000 7f9ed2a00640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9eb4074ed0 0x7f9eb4077390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:41.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.308+0000 7f9ed2a00640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9eb4074ed0 0x7f9eb4077390 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40715b0 tx=0x7f9ec40091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:41.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.312+0000 7f9ecb400640 1 -- 172.21.15.44:0/1265634917 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9ecc0197f0 con 0x7f9eb8003a40 2024-09-13T08:52:41.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.516+0000 7f9eda400640 1 -- 172.21.15.44:0/1265634917 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f9ed40715e0 con 0x7f9eb4074ed0 2024-09-13T08:52:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9ecb400640 1 -- 172.21.15.44:0/1265634917 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ed40715e0 con 0x7f9eb4074ed0 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 -- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9eb4074ed0 msgr2=0x7f9eb4077390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9eb4074ed0 0x7f9eb4077390 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40715b0 tx=0x7f9ec40091e0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 -- 172.21.15.44:0/1265634917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003a40 msgr2=0x7f9ed41f49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003a40 0x7f9ed41f49c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc01e310 tx=0x7f9ecc00c350 comp rx=0 tx=0).stop 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 -- 172.21.15.44:0/1265634917 shutdown_connections 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9eb4074ed0 0x7f9eb4077390 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ed40c8e70 0x7f9ed40c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9ed406f920 0x7f9ed40b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 --2- 172.21.15.44:0/1265634917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003a40 0x7f9ed41f49c0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 -- 172.21.15.44:0/1265634917 >> 172.21.15.44:0/1265634917 conn(0x7f9ed40d1da0 msgr2=0x7f9ed4071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 -- 172.21.15.44:0/1265634917 shutdown_connections 2024-09-13T08:52:41.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.532+0000 7f9eda400640 1 -- 172.21.15.44:0/1265634917 wait complete. 2024-09-13T08:52:41.553 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:52:41.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.704+0000 7fb8a9a00640 1 -- 172.21.15.44:0/2833691776 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8a415df10 msgr2=0x7fb8a406eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:41.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.704+0000 7fb8a9a00640 1 --2- 172.21.15.44:0/2833691776 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8a415df10 0x7fb8a406eb70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb894009900 tx=0x7fb89401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.704+0000 7fb8a9a00640 1 -- 172.21.15.44:0/2833691776 shutdown_connections 2024-09-13T08:52:41.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.704+0000 7fb8a9a00640 1 --2- 172.21.15.44:0/2833691776 >> v2:172.21.15.129:3300/0 conn(0x7fb8a40d2300 0x7fb8a40b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.704+0000 7fb8a9a00640 1 --2- 172.21.15.44:0/2833691776 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8a415df10 0x7fb8a406eb70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.704+0000 7fb8a9a00640 1 -- 172.21.15.44:0/2833691776 >> 172.21.15.44:0/2833691776 conn(0x7fb8a409aa10 msgr2=0x7fb8a409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:41.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.704+0000 7fb8a9a00640 1 -- 172.21.15.44:0/2833691776 shutdown_connections 2024-09-13T08:52:41.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.704+0000 7fb8a9a00640 1 -- 172.21.15.44:0/2833691776 wait complete. 2024-09-13T08:52:41.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.704+0000 7fb8a9a00640 1 Processor -- start 2024-09-13T08:52:41.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.704+0000 7fb8a9a00640 1 -- start start 2024-09-13T08:52:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a9a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8a40cfd80 0x7fb8a4159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a9a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8a40d2300 0x7fb8a415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8a415df10 0x7fb8a40b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a9a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a416ad30 con 0x7fb8a415df10 2024-09-13T08:52:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a9a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a416abb0 con 0x7fb8a40cfd80 2024-09-13T08:52:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a9a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a416aeb0 con 0x7fb8a40d2300 2024-09-13T08:52:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8a415df10 0x7fb8a40b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8a40cfd80 0x7fb8a4159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8a40cfd80 0x7fb8a4159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:43834/0 (socket says 172.21.15.44:43834) 2024-09-13T08:52:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a3e00640 1 -- 172.21.15.44:0/1542670905 learned_addr learned my addr 172.21.15.44:0/1542670905 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a3400640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8a40d2300 0x7fb8a415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a3e00640 1 -- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8a40d2300 msgr2=0x7fb8a415a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a3e00640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8a40d2300 0x7fb8a415a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a3e00640 1 -- 172.21.15.44:0/1542670905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8a415df10 msgr2=0x7fb8a40b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a3e00640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8a415df10 0x7fb8a40b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a3e00640 1 -- 172.21.15.44:0/1542670905 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a415a730 con 0x7fb8a40cfd80 2024-09-13T08:52:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a3e00640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8a40cfd80 0x7fb8a4159670 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb894009850 tx=0x7fb8940a5730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb89be00640 1 -- 172.21.15.44:0/1542670905 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940b2040 con 0x7fb8a40cfd80 2024-09-13T08:52:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb89be00640 1 -- 172.21.15.44:0/1542670905 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8940043b0 con 0x7fb8a40cfd80 2024-09-13T08:52:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb89be00640 1 -- 172.21.15.44:0/1542670905 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894005290 con 0x7fb8a40cfd80 2024-09-13T08:52:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a9a00640 1 -- 172.21.15.44:0/1542670905 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a415abf0 con 0x7fb8a40cfd80 2024-09-13T08:52:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a9a00640 1 -- 172.21.15.44:0/1542670905 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a41facd0 con 0x7fb8a40cfd80 2024-09-13T08:52:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.708+0000 7fb8a9a00640 1 -- 172.21.15.44:0/1542670905 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a40b16d0 con 0x7fb8a40cfd80 2024-09-13T08:52:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.712+0000 7fb89be00640 1 -- 172.21.15.44:0/1542670905 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 53) ==== 99987+0+0 (secure 0 0 0) 0x7fb8940bd060 con 0x7fb8a40cfd80 2024-09-13T08:52:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.712+0000 7fb89be00640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb884074ed0 0x7fb884077390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.712+0000 7fb89be00640 1 -- 172.21.15.44:0/1542670905 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7fb89412c0d0 con 0x7fb8a40cfd80 2024-09-13T08:52:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.712+0000 7fb8a3400640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb884074ed0 0x7fb884077390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.712+0000 7fb8a3400640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb884074ed0 0x7fb884077390 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40cdf30 tx=0x7fb890009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:41.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.716+0000 7fb89be00640 1 -- 172.21.15.44:0/1542670905 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb8940164d0 con 0x7fb8a40cfd80 2024-09-13T08:52:41.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.920+0000 7fb8a9a00640 1 -- 172.21.15.44:0/1542670905 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb8a41571f0 con 0x7fb884074ed0 2024-09-13T08:52:41.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb89be00640 1 -- 172.21.15.44:0/1542670905 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fb8a41571f0 con 0x7fb884074ed0 2024-09-13T08:52:41.924 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:41.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 -- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb884074ed0 msgr2=0x7fb884077390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:41.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb884074ed0 0x7fb884077390 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40cdf30 tx=0x7fb890009040 comp rx=0 tx=0).stop 2024-09-13T08:52:41.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 -- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8a40cfd80 msgr2=0x7fb8a4159670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:41.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8a40cfd80 0x7fb8a4159670 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb894009850 tx=0x7fb8940a5730 comp rx=0 tx=0).stop 2024-09-13T08:52:41.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 -- 172.21.15.44:0/1542670905 shutdown_connections 2024-09-13T08:52:41.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb884074ed0 0x7fb884077390 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8a415df10 0x7fb8a40b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb8a40d2300 0x7fb8a415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 --2- 172.21.15.44:0/1542670905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8a40cfd80 0x7fb8a4159670 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:41.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 -- 172.21.15.44:0/1542670905 >> 172.21.15.44:0/1542670905 conn(0x7fb8a409aa10 msgr2=0x7fb8a409adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:41.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 -- 172.21.15.44:0/1542670905 shutdown_connections 2024-09-13T08:52:41.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:41.924+0000 7fb8a9a00640 1 -- 172.21.15.44:0/1542670905 wait complete. 2024-09-13T08:52:41.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:41.945 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:41.945 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:52:41.955 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:41.955 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3055190016" 2024-09-13T08:52:41.955 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:41.956 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-13T08:52:43.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:43.957 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:43.957 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:52:43.968 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:43.968 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3055190016" 2024-09-13T08:52:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:43.969 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-13T08:52:45.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:45.970 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:45.970 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:52:45.980 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:45.981 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3221225472" 2024-09-13T08:52:45.981 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:45.981 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-13T08:52:45.982 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-13T08:52:46.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.184+0000 7f1f30000640 1 -- 172.21.15.44:0/2489252367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f2815bfa0 msgr2=0x7f1f280cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:46.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.184+0000 7f1f30000640 1 --2- 172.21.15.44:0/2489252367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f2815bfa0 0x7f1f280cd000 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009840 tx=0x7f1f1401efd0 comp rx=0 tx=0).stop 2024-09-13T08:52:46.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.184+0000 7f1f30000640 1 -- 172.21.15.44:0/2489252367 shutdown_connections 2024-09-13T08:52:46.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.184+0000 7f1f30000640 1 --2- 172.21.15.44:0/2489252367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f2815bfa0 0x7f1f280cd000 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:46.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.184+0000 7f1f30000640 1 --2- 172.21.15.44:0/2489252367 >> v2:172.21.15.129:3300/0 conn(0x7f1f2815c3e0 0x7f1f2815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:46.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.184+0000 7f1f30000640 1 -- 172.21.15.44:0/2489252367 >> 172.21.15.44:0/2489252367 conn(0x7f1f280b18f0 msgr2=0x7f1f280b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:46.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.184+0000 7f1f30000640 1 -- 172.21.15.44:0/2489252367 shutdown_connections 2024-09-13T08:52:46.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.184+0000 7f1f30000640 1 -- 172.21.15.44:0/2489252367 wait complete. 2024-09-13T08:52:46.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f30000640 1 Processor -- start 2024-09-13T08:52:46.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f30000640 1 -- start start 2024-09-13T08:52:46.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f30000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f280cd540 0x7f1f280cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f30000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f2815bfa0 0x7f1f280cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f30000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f2815c3e0 0x7f1f2815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f30000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28160020 con 0x7f1f2815bfa0 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f30000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28073440 con 0x7f1f280cd540 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f30000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1f281601a0 con 0x7f1f2815c3e0 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f2815bfa0 0x7f1f280cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f2815bfa0 0x7f1f280cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41860/0 (socket says 172.21.15.44:41860) 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2cc00640 1 -- 172.21.15.44:0/1079211136 learned_addr learned my addr 172.21.15.44:0/1079211136 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2d600640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f280cd540 0x7f1f280cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2e000640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f2815c3e0 0x7f1f2815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:46.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2cc00640 1 -- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f2815c3e0 msgr2=0x7f1f2815a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2cc00640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f2815c3e0 0x7f1f2815a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2cc00640 1 -- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f280cd540 msgr2=0x7f1f280cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2cc00640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f280cd540 0x7f1f280cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2cc00640 1 -- 172.21.15.44:0/1079211136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f281f4630 con 0x7f1f2815bfa0 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2e000640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f2815c3e0 0x7f1f2815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f2cc00640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f2815bfa0 0x7f1f280cdae0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009930 tx=0x7f1f14005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f25600640 1 -- 172.21.15.44:0/1079211136 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f140bd020 con 0x7f1f2815bfa0 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f25600640 1 -- 172.21.15.44:0/1079211136 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f14005310 con 0x7f1f2815bfa0 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f30000640 1 -- 172.21.15.44:0/1079211136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f281f48f0 con 0x7f1f2815bfa0 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f25600640 1 -- 172.21.15.44:0/1079211136 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f140b5040 con 0x7f1f2815bfa0 2024-09-13T08:52:46.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.188+0000 7f1f30000640 1 -- 172.21.15.44:0/1079211136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f281f4eb0 con 0x7f1f2815bfa0 2024-09-13T08:52:46.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.192+0000 7f1f30000640 1 -- 172.21.15.44:0/1079211136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f28160680 con 0x7f1f2815bfa0 2024-09-13T08:52:46.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.192+0000 7f1f25600640 1 -- 172.21.15.44:0/1079211136 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 53) ==== 99987+0+0 (secure 0 0 0) 0x7f1f140c0070 con 0x7f1f2815bfa0 2024-09-13T08:52:46.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.192+0000 7f1f25600640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f08074ed0 0x7f1f08077390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:46.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.192+0000 7f1f25600640 1 -- 172.21.15.44:0/1079211136 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f1f1412fa70 con 0x7f1f2815bfa0 2024-09-13T08:52:46.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.192+0000 7f1f2d600640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f08074ed0 0x7f1f08077390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:46.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.196+0000 7f1f2d600640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f08074ed0 0x7f1f08077390 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c005fd0 tx=0x7f1f1c005e40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:46.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.196+0000 7f1f25600640 1 -- 172.21.15.44:0/1079211136 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1f14018b30 con 0x7f1f2815bfa0 2024-09-13T08:52:46.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.404+0000 7f1f30000640 1 -- 172.21.15.44:0/1079211136 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f1f2806fc10 con 0x7f1f08074ed0 2024-09-13T08:52:46.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f25600640 1 -- 172.21.15.44:0/1079211136 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1f2806fc10 con 0x7f1f08074ed0 2024-09-13T08:52:46.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f30000640 1 -- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f08074ed0 msgr2=0x7f1f08077390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:46.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f30000640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f08074ed0 0x7f1f08077390 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c005fd0 tx=0x7f1f1c005e40 comp rx=0 tx=0).stop 2024-09-13T08:52:46.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f30000640 1 -- 172.21.15.44:0/1079211136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f2815bfa0 msgr2=0x7f1f280cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:46.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f30000640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f2815bfa0 0x7f1f280cdae0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009930 tx=0x7f1f14005d20 comp rx=0 tx=0).stop 2024-09-13T08:52:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f30000640 1 -- 172.21.15.44:0/1079211136 shutdown_connections 2024-09-13T08:52:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f30000640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f08074ed0 0x7f1f08077390 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f30000640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f2815c3e0 0x7f1f2815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f30000640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f2815bfa0 0x7f1f280cdae0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f30000640 1 --2- 172.21.15.44:0/1079211136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f280cd540 0x7f1f280cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.488+0000 7f1f30000640 1 -- 172.21.15.44:0/1079211136 >> 172.21.15.44:0/1079211136 conn(0x7f1f280b18f0 msgr2=0x7f1f280b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.492+0000 7f1f30000640 1 -- 172.21.15.44:0/1079211136 shutdown_connections 2024-09-13T08:52:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:46.492+0000 7f1f30000640 1 -- 172.21.15.44:0/1079211136 wait complete. 2024-09-13T08:52:46.493 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:46.493+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:46.493 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:46.493+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:46.493 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:46.493+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:46.493 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:46.493+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:46.493 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:46.493+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:46.510 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:46.509+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:46.510 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:46.509+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:46.511 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:46.509+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:46.511 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:46.509+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:46.511 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:46.509+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:47.498 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:47.497+0000 7f8a40000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:47.498 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:47.497+0000 7f8a40000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:47.498 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:47.497+0000 7f8a40000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:47.498 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:47.497+0000 7f8a40000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:47.498 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:47.497+0000 7f8a40000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:47.514 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-13T08:52:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 -- 172.21.15.44:0/20176864 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8d40c41c0 msgr2=0x7ff8d406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 --2- 172.21.15.44:0/20176864 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8d40c41c0 0x7ff8d406d160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009bc0 tx=0x7ff8d001f3b0 comp rx=0 tx=0).stop 2024-09-13T08:52:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 -- 172.21.15.44:0/20176864 shutdown_connections 2024-09-13T08:52:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 --2- 172.21.15.44:0/20176864 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8d40c41c0 0x7ff8d406d160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 --2- 172.21.15.44:0/20176864 >> v2:172.21.15.129:3300/0 conn(0x7ff8d415baf0 0x7ff8d406cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 -- 172.21.15.44:0/20176864 >> 172.21.15.44:0/20176864 conn(0x7ff8d40d0b30 msgr2=0x7ff8d40d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 -- 172.21.15.44:0/20176864 shutdown_connections 2024-09-13T08:52:47.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 -- 172.21.15.44:0/20176864 wait complete. 2024-09-13T08:52:47.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 Processor -- start 2024-09-13T08:52:47.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 -- start start 2024-09-13T08:52:47.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8d40c41c0 0x7ff8d40c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:47.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8d40cd180 0x7ff8d40c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:47.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8d415baf0 0x7ff8d40af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d4074820 con 0x7ff8d415baf0 2024-09-13T08:52:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d40746a0 con 0x7ff8d40c41c0 2024-09-13T08:52:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8db200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d40cb890 con 0x7ff8d40cd180 2024-09-13T08:52:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8d415baf0 0x7ff8d40af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8d415baf0 0x7ff8d40af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41870/0 (socket says 172.21.15.44:41870) 2024-09-13T08:52:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.744+0000 7ff8da800640 1 -- 172.21.15.44:0/622713221 learned_addr learned my addr 172.21.15.44:0/622713221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8da800640 1 -- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8d40cd180 msgr2=0x7ff8d40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8da800640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8d40cd180 0x7ff8d40c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8da800640 1 -- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8d40c41c0 msgr2=0x7ff8d40c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:47.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8da800640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8d40c41c0 0x7ff8d40c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:47.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8da800640 1 -- 172.21.15.44:0/622713221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d40c5110 con 0x7ff8d415baf0 2024-09-13T08:52:47.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8da800640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8d415baf0 0x7ff8d40af590 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009b10 tx=0x7ff8d00b0940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:47.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8ce000640 1 -- 172.21.15.44:0/622713221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00b4070 con 0x7ff8d415baf0 2024-09-13T08:52:47.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8db200640 1 -- 172.21.15.44:0/622713221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d40c53f0 con 0x7ff8d415baf0 2024-09-13T08:52:47.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8ce000640 1 -- 172.21.15.44:0/622713221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8d00bf070 con 0x7ff8d415baf0 2024-09-13T08:52:47.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8ce000640 1 -- 172.21.15.44:0/622713221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00b0e60 con 0x7ff8d415baf0 2024-09-13T08:52:47.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8db200640 1 -- 172.21.15.44:0/622713221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d40c59b0 con 0x7ff8d415baf0 2024-09-13T08:52:47.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8db200640 1 -- 172.21.15.44:0/622713221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d40c7460 con 0x7ff8d415baf0 2024-09-13T08:52:47.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.748+0000 7ff8ce000640 1 -- 172.21.15.44:0/622713221 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 54) ==== 100089+0+0 (secure 0 0 0) 0x7ff8d0003be0 con 0x7ff8d415baf0 2024-09-13T08:52:47.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.752+0000 7ff8ce000640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8b8074ff0 0x7ff8b80774b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:47.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.752+0000 7ff8ce000640 1 -- 172.21.15.44:0/622713221 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7ff8d01311f0 con 0x7ff8d415baf0 2024-09-13T08:52:47.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.752+0000 7ff8d9e00640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8b8074ff0 0x7ff8b80774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:47.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.756+0000 7ff8ce000640 1 -- 172.21.15.44:0/622713221 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff8d0018f10 con 0x7ff8d415baf0 2024-09-13T08:52:47.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:47.756+0000 7ff8d9e00640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8b8074ff0 0x7ff8b80774b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80058d0 tx=0x7ff8c8005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:48.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.068+0000 7ff8db200640 1 -- 172.21.15.44:0/622713221 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7ff8d40cbd40 con 0x7ff8b8074ff0 2024-09-13T08:52:48.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.188+0000 7ff8ce000640 1 -- 172.21.15.44:0/622713221 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff8d40cbd40 con 0x7ff8b8074ff0 2024-09-13T08:52:48.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 -- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8b8074ff0 msgr2=0x7ff8b80774b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8b8074ff0 0x7ff8b80774b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80058d0 tx=0x7ff8c8005860 comp rx=0 tx=0).stop 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 -- 172.21.15.44:0/622713221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8d415baf0 msgr2=0x7ff8d40af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8d415baf0 0x7ff8d40af590 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009b10 tx=0x7ff8d00b0940 comp rx=0 tx=0).stop 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 -- 172.21.15.44:0/622713221 shutdown_connections 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff8b8074ff0 0x7ff8b80774b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8d415baf0 0x7ff8d40af590 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff8d40cd180 0x7ff8d40c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 --2- 172.21.15.44:0/622713221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8d40c41c0 0x7ff8d40c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 -- 172.21.15.44:0/622713221 >> 172.21.15.44:0/622713221 conn(0x7ff8d40d0b30 msgr2=0x7ff8d40b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 -- 172.21.15.44:0/622713221 shutdown_connections 2024-09-13T08:52:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.192+0000 7ff8db200640 1 -- 172.21.15.44:0/622713221 wait complete. 2024-09-13T08:52:48.213 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-13T08:52:48.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.368+0000 7f2594600640 1 -- 172.21.15.44:0/3056788364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2574005e60 msgr2=0x7f25740062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:48.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.368+0000 7f2594600640 1 --2- 172.21.15.44:0/3056788364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2574005e60 0x7f25740062e0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f2584009ee0 tx=0x7f258401d5f0 comp rx=0 tx=0).stop 2024-09-13T08:52:48.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.368+0000 7f2592600640 1 -- 172.21.15.44:0/3056788364 reap_dead start 2024-09-13T08:52:48.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.368+0000 7f2594600640 1 -- 172.21.15.44:0/3056788364 shutdown_connections 2024-09-13T08:52:48.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.368+0000 7f2594600640 1 -- 172.21.15.44:0/3056788364 >> 172.21.15.44:0/3056788364 conn(0x7f258c0b18f0 msgr2=0x7f258c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:48.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.368+0000 7f2594600640 1 -- 172.21.15.44:0/3056788364 shutdown_connections 2024-09-13T08:52:48.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.368+0000 7f2594600640 1 -- 172.21.15.44:0/3056788364 wait complete. 2024-09-13T08:52:48.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 Processor -- start 2024-09-13T08:52:48.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 -- start start 2024-09-13T08:52:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2574005e60 0x7f258c06f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f258c1e2b90 0x7f258c14b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f258c0c4e50 0x7f258c0c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f258c0cc660 con 0x7f258c1e2b90 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f258c0cd5e0 con 0x7f258c0c4e50 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f258c0cc7e0 con 0x7f2574005e60 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2591200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f258c1e2b90 0x7f258c14b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2591200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f258c1e2b90 0x7f258c14b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41888/0 (socket says 172.21.15.44:41888) 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2591200640 1 -- 172.21.15.44:0/2852220519 learned_addr learned my addr 172.21.15.44:0/2852220519 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2591c00640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2574005e60 0x7f258c06f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2592600640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f258c0c4e50 0x7f258c0c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2591200640 1 -- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2574005e60 msgr2=0x7f258c06f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2591200640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2574005e60 0x7f258c06f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2591200640 1 -- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f258c0c4e50 msgr2=0x7f258c0c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2591200640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f258c0c4e50 0x7f258c0c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2591200640 1 -- 172.21.15.44:0/2852220519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c1e31e0 con 0x7f258c1e2b90 2024-09-13T08:52:48.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2591200640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f258c1e2b90 0x7f258c14b5a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f2578003ed0 tx=0x7f2578001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:48.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2589600640 1 -- 172.21.15.44:0/2852220519 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578015070 con 0x7f258c1e2b90 2024-09-13T08:52:48.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2589600640 1 -- 172.21.15.44:0/2852220519 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2578011840 con 0x7f258c1e2b90 2024-09-13T08:52:48.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 -- 172.21.15.44:0/2852220519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c1e34c0 con 0x7f258c1e2b90 2024-09-13T08:52:48.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2589600640 1 -- 172.21.15.44:0/2852220519 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578011c70 con 0x7f258c1e2b90 2024-09-13T08:52:48.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 -- 172.21.15.44:0/2852220519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c1e3a80 con 0x7f258c1e2b90 2024-09-13T08:52:48.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.372+0000 7f2594600640 1 -- 172.21.15.44:0/2852220519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258c069410 con 0x7f258c1e2b90 2024-09-13T08:52:48.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.376+0000 7f2589600640 1 -- 172.21.15.44:0/2852220519 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 54) ==== 100089+0+0 (secure 0 0 0) 0x7f2578018070 con 0x7f258c1e2b90 2024-09-13T08:52:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.376+0000 7f2589600640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f256c075020 0x7f256c0774e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.376+0000 7f2589600640 1 -- 172.21.15.44:0/2852220519 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f2578093de0 con 0x7f258c1e2b90 2024-09-13T08:52:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.376+0000 7f2591c00640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f256c075020 0x7f256c0774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:48.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.376+0000 7f2591c00640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f256c075020 0x7f256c0774e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2580006bb0 tx=0x7f2580006730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:48.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.380+0000 7f2589600640 1 -- 172.21.15.44:0/2852220519 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2578060440 con 0x7f258c1e2b90 2024-09-13T08:52:48.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:48.592+0000 7f2594600640 1 -- 172.21.15.44:0/2852220519 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f258c0b42d0 con 0x7f256c075020 2024-09-13T08:52:49.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:49.456+0000 7f2589600640 1 -- 172.21.15.44:0/2852220519 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 55) ==== 100140+0+0 (secure 0 0 0) 0x7f257805c6b0 con 0x7f258c1e2b90 2024-09-13T08:52:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2589600640 1 -- 172.21.15.44:0/2852220519 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f258c0b42d0 con 0x7f256c075020 2024-09-13T08:52:50.804 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:52:50.804 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:52:50.804 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:52:50.804 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:52:50.804 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:52:50.804 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:52:50.804 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:52:50.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:52:50.805 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:52:50.805 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:52:50.805 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:52:50.805 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:52:50.805 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:52:50.805 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:52:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 -- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f256c075020 msgr2=0x7f256c0774e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f256c075020 0x7f256c0774e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2580006bb0 tx=0x7f2580006730 comp rx=0 tx=0).stop 2024-09-13T08:52:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 -- 172.21.15.44:0/2852220519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f258c1e2b90 msgr2=0x7f258c14b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f258c1e2b90 0x7f258c14b5a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f2578003ed0 tx=0x7f2578001720 comp rx=0 tx=0).stop 2024-09-13T08:52:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 -- 172.21.15.44:0/2852220519 shutdown_connections 2024-09-13T08:52:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f256c075020 0x7f256c0774e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f258c0c4e50 0x7f258c0c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f258c1e2b90 0x7f258c14b5a0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 --2- 172.21.15.44:0/2852220519 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2574005e60 0x7f258c06f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 -- 172.21.15.44:0/2852220519 >> 172.21.15.44:0/2852220519 conn(0x7f258c0b18f0 msgr2=0x7f258c0b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 -- 172.21.15.44:0/2852220519 shutdown_connections 2024-09-13T08:52:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.804+0000 7f2594600640 1 -- 172.21.15.44:0/2852220519 wait complete. 2024-09-13T08:52:50.827 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:52:50.827 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:52:50.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.972+0000 7f90dc400640 1 -- 172.21.15.44:0/1779804712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d4073b90 msgr2=0x7f90d40b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:50.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.972+0000 7f90dc400640 1 --2- 172.21.15.44:0/1779804712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d4073b90 0x7f90d40b3320 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009bc0 tx=0x7f90cc01cb10 comp rx=0 tx=0).stop 2024-09-13T08:52:50.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.972+0000 7f90dc400640 1 -- 172.21.15.44:0/1779804712 shutdown_connections 2024-09-13T08:52:50.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.972+0000 7f90dc400640 1 --2- 172.21.15.44:0/1779804712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d4073b90 0x7f90d40b3320 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:50.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.972+0000 7f90dc400640 1 --2- 172.21.15.44:0/1779804712 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d415a480 0x7f90d4072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:50.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.972+0000 7f90dc400640 1 -- 172.21.15.44:0/1779804712 >> 172.21.15.44:0/1779804712 conn(0x7f90d406cdf0 msgr2=0x7f90d406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:50.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.972+0000 7f90dc400640 1 -- 172.21.15.44:0/1779804712 shutdown_connections 2024-09-13T08:52:50.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.972+0000 7f90dc400640 1 -- 172.21.15.44:0/1779804712 wait complete. 2024-09-13T08:52:50.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90dc400640 1 Processor -- start 2024-09-13T08:52:50.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90dc400640 1 -- start start 2024-09-13T08:52:50.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90dc400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d4073b90 0x7f90d41f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90dc400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d40cf910 0x7f90d414d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90dc400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d415a480 0x7f90d40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90dc400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4167260 con 0x7f90d4073b90 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90dc400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41670e0 con 0x7f90d415a480 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90dc400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f90d41673e0 con 0x7f90d40cf910 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d4073b90 0x7f90d41f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d4073b90 0x7f90d41f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41900/0 (socket says 172.21.15.44:41900) 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d9a00640 1 -- 172.21.15.44:0/1118663949 learned_addr learned my addr 172.21.15.44:0/1118663949 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d9000640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d40cf910 0x7f90d414d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90da400640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d415a480 0x7f90d40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d9a00640 1 -- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d40cf910 msgr2=0x7f90d414d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:50.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d9a00640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d40cf910 0x7f90d414d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d9a00640 1 -- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d415a480 msgr2=0x7f90d40b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d9a00640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d415a480 0x7f90d40b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d9a00640 1 -- 172.21.15.44:0/1118663949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d41f84b0 con 0x7f90d4073b90 2024-09-13T08:52:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d9a00640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d4073b90 0x7f90d41f7d10 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f90c800e970 tx=0x7f90c800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d1600640 1 -- 172.21.15.44:0/1118663949 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c800cd80 con 0x7f90d4073b90 2024-09-13T08:52:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d1600640 1 -- 172.21.15.44:0/1118663949 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90c8004020 con 0x7f90d4073b90 2024-09-13T08:52:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90dc400640 1 -- 172.21.15.44:0/1118663949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d41f8790 con 0x7f90d4073b90 2024-09-13T08:52:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90d1600640 1 -- 172.21.15.44:0/1118663949 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8004470 con 0x7f90d4073b90 2024-09-13T08:52:50.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.976+0000 7f90dc400640 1 -- 172.21.15.44:0/1118663949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d4152d50 con 0x7f90d4073b90 2024-09-13T08:52:50.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.980+0000 7f90dc400640 1 -- 172.21.15.44:0/1118663949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d40b3320 con 0x7f90d4073b90 2024-09-13T08:52:50.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.980+0000 7f90d1600640 1 -- 172.21.15.44:0/1118663949 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 55) ==== 100140+0+0 (secure 0 0 0) 0x7f90c8003600 con 0x7f90d4073b90 2024-09-13T08:52:50.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.980+0000 7f90d1600640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4075080 0x7f90b4077540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:50.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.980+0000 7f90d1600640 1 -- 172.21.15.44:0/1118663949 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f90c8093930 con 0x7f90d4073b90 2024-09-13T08:52:50.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.980+0000 7f90d9000640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4075080 0x7f90b4077540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:50.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.980+0000 7f90d9000640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4075080 0x7f90b4077540 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f90d4150830 tx=0x7f90c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:50.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:50.984+0000 7f90d1600640 1 -- 172.21.15.44:0/1118663949 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f90c80605e0 con 0x7f90d4073b90 2024-09-13T08:52:51.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.224+0000 7f90dc400640 1 -- 172.21.15.44:0/1118663949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f90d4072c80 con 0x7f90d4073b90 2024-09-13T08:52:51.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90d1600640 1 -- 172.21.15.44:0/1118663949 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1915 (secure 0 0 0) 0x7f90c805fee0 con 0x7f90d4073b90 2024-09-13T08:52:51.227 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:51.228 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:52:51.228 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:52:51.228 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:52:51.228 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:52:51.228 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:52:51.228 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:52:51.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:52:51.229 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:52:51.229 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:52:51.229 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:52:51.229 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:52:51.230 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:52:51.230 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:52:51.230 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:52:51.230 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:52:51.230 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:52:51.231 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:52:51.231 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:52:51.231 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 941, 2024-09-13T08:52:51.231 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:52:51.231 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:52:51.231 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:52:51.232 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:52:51.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:52:51.232 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:52:51.232 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 131, 2024-09-13T08:52:51.232 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:52:51.232 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:52:51.232 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:52:51.233 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:52:51.233 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:52:51.233 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:52:51.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:52:51.233 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:52:51.233 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:52:51.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:52:51.233 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:52:51.233 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:52:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:52:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:52:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:52:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:52:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 799, 2024-09-13T08:52:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 3247176636, 2024-09-13T08:52:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 7563550720, 2024-09-13T08:52:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 765530562560, 2024-09-13T08:52:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:52:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 2446054, 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 91554712, 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 0, 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 22 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 86, 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:52:09:337186+0000", 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 9, 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:52:51.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 9, 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 8763 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:52:51.236 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 21, 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:51:42.811499+0000", 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:52:51.237 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": {} 2024-09-13T08:52:51.238 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:52:51.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 -- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4075080 msgr2=0x7f90b4077540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4075080 0x7f90b4077540 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f90d4150830 tx=0x7f90c0009040 comp rx=0 tx=0).stop 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 -- 172.21.15.44:0/1118663949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d4073b90 msgr2=0x7f90d41f7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d4073b90 0x7f90d41f7d10 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f90c800e970 tx=0x7f90c800ee30 comp rx=0 tx=0).stop 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 -- 172.21.15.44:0/1118663949 shutdown_connections 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f90b4075080 0x7f90b4077540 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90d415a480 0x7f90d40b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90d40cf910 0x7f90d414d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 --2- 172.21.15.44:0/1118663949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d4073b90 0x7f90d41f7d10 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 -- 172.21.15.44:0/1118663949 >> 172.21.15.44:0/1118663949 conn(0x7f90d406cdf0 msgr2=0x7f90d40aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 -- 172.21.15.44:0/1118663949 shutdown_connections 2024-09-13T08:52:51.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.228+0000 7f90dc400640 1 -- 172.21.15.44:0/1118663949 wait complete. 2024-09-13T08:52:51.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:52:51.249 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/5f5c748c-d23c-413f-9690-1a82dc3a17c6 2024-09-13T08:52:51.249 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:52:51.259 INFO:teuthology.orchestra.run.smithi044.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/5f5c748c-d23c-413f-9690-1a82dc3a17c6': No such file or directory 2024-09-13T08:52:51.260 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-13T08:52:51.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:52:51.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.448+0000 7f7033000640 1 -- 172.21.15.44:0/1346647573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7014005220 msgr2=0x7f701400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:51.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.448+0000 7f7033000640 1 --2- 172.21.15.44:0/1346647573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7014005220 0x7f701400b9d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f702400c660 tx=0x7f702401d2e0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.448+0000 7f7033000640 1 -- 172.21.15.44:0/1346647573 shutdown_connections 2024-09-13T08:52:51.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.448+0000 7f7033000640 1 --2- 172.21.15.44:0/1346647573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7014005220 0x7f701400b9d0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.448+0000 7f7033000640 1 -- 172.21.15.44:0/1346647573 >> 172.21.15.44:0/1346647573 conn(0x7f702c06dd30 msgr2=0x7f702c06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:51.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.448+0000 7f7033000640 1 -- 172.21.15.44:0/1346647573 shutdown_connections 2024-09-13T08:52:51.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7033000640 1 -- 172.21.15.44:0/1346647573 wait complete. 2024-09-13T08:52:51.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7033000640 1 Processor -- start 2024-09-13T08:52:51.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7033000640 1 -- start start 2024-09-13T08:52:51.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7033000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7014005220 0x7f702c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:51.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7033000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f702c0b0640 0x7f702c1f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:51.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7033000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f702c1f4ca0 0x7f702c158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:51.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7033000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f702c0ce3a0 con 0x7f7014005220 2024-09-13T08:52:51.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7033000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f702c0ce220 con 0x7f702c1f4ca0 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7033000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f702c1605a0 con 0x7f702c0b0640 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7014005220 0x7f702c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7014005220 0x7f702c0b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41920/0 (socket says 172.21.15.44:41920) 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031c00640 1 -- 172.21.15.44:0/1650585752 learned_addr learned my addr 172.21.15.44:0/1650585752 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7032600640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f702c1f4ca0 0x7f702c158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031200640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f702c0b0640 0x7f702c1f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031c00640 1 -- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f702c0b0640 msgr2=0x7f702c1f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031c00640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f702c0b0640 0x7f702c1f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031c00640 1 -- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f702c1f4ca0 msgr2=0x7f702c158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031c00640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f702c1f4ca0 0x7f702c158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031c00640 1 -- 172.21.15.44:0/1650585752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c0d0060 con 0x7f7014005220 2024-09-13T08:52:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031200640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f702c0b0640 0x7f702c1f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:51.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7032600640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f702c1f4ca0 0x7f702c158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:51.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.452+0000 7f7031c00640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7014005220 0x7f702c0b5c60 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7020011860 tx=0x7f7020011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:51.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.456+0000 7f7029600640 1 -- 172.21.15.44:0/1650585752 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702000f9a0 con 0x7f7014005220 2024-09-13T08:52:51.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.456+0000 7f7029600640 1 -- 172.21.15.44:0/1650585752 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f702000fb40 con 0x7f7014005220 2024-09-13T08:52:51.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.456+0000 7f7029600640 1 -- 172.21.15.44:0/1650585752 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020003600 con 0x7f7014005220 2024-09-13T08:52:51.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.456+0000 7f7033000640 1 -- 172.21.15.44:0/1650585752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c0d04c0 con 0x7f7014005220 2024-09-13T08:52:51.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.456+0000 7f7033000640 1 -- 172.21.15.44:0/1650585752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c09f030 con 0x7f7014005220 2024-09-13T08:52:51.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.456+0000 7f7033000640 1 -- 172.21.15.44:0/1650585752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702c09ee20 con 0x7f7014005220 2024-09-13T08:52:51.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.456+0000 7f7029600640 1 -- 172.21.15.44:0/1650585752 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 55) ==== 100140+0+0 (secure 0 0 0) 0x7f70200037a0 con 0x7f7014005220 2024-09-13T08:52:51.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.456+0000 7f7029600640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f700c0750b0 0x7f700c077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:51.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.460+0000 7f7029600640 1 -- 172.21.15.44:0/1650585752 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f7020094050 con 0x7f7014005220 2024-09-13T08:52:51.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.460+0000 7f7031200640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f700c0750b0 0x7f700c077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:51.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.460+0000 7f7031200640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f700c0750b0 0x7f700c077570 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f702c06ea90 tx=0x7f7018006bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.464+0000 7f7029600640 1 -- 172.21.15.44:0/1650585752 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f702c0d0060 con 0x7f7014005220 2024-09-13T08:52:51.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.664+0000 7f7033000640 1 -- 172.21.15.44:0/1650585752 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f702c0d0270 con 0x7f700c0750b0 2024-09-13T08:52:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7029600640 1 -- 172.21.15.44:0/1650585752 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f702c0d0270 con 0x7f700c0750b0 2024-09-13T08:52:51.668 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:52:51.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:52:51.668 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:52:51.669 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:52:51.669 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 -- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f700c0750b0 msgr2=0x7f700c077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f700c0750b0 0x7f700c077570 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f702c06ea90 tx=0x7f7018006bd0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 -- 172.21.15.44:0/1650585752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7014005220 msgr2=0x7f702c0b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7014005220 0x7f702c0b5c60 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7020011860 tx=0x7f7020011d20 comp rx=0 tx=0).stop 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 -- 172.21.15.44:0/1650585752 shutdown_connections 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f700c0750b0 0x7f700c077570 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f702c1f4ca0 0x7f702c158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f702c0b0640 0x7f702c1f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 --2- 172.21.15.44:0/1650585752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7014005220 0x7f702c0b5c60 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 -- 172.21.15.44:0/1650585752 >> 172.21.15.44:0/1650585752 conn(0x7f702c06dd30 msgr2=0x7f702c070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 -- 172.21.15.44:0/1650585752 shutdown_connections 2024-09-13T08:52:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.668+0000 7f7033000640 1 -- 172.21.15.44:0/1650585752 wait complete. 2024-09-13T08:52:51.690 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-13T08:52:51.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.836+0000 7f0f18a00640 1 -- 172.21.15.44:0/3641409708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f1415bff0 msgr2=0x7f0f1406f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:51.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.836+0000 7f0f18a00640 1 --2- 172.21.15.44:0/3641409708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f1415bff0 0x7f0f1406f650 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009900 tx=0x7f0f0401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.836+0000 7f0f18a00640 1 -- 172.21.15.44:0/3641409708 shutdown_connections 2024-09-13T08:52:51.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.836+0000 7f0f18a00640 1 --2- 172.21.15.44:0/3641409708 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f1406d880 0x7f0f1406fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.836+0000 7f0f18a00640 1 --2- 172.21.15.44:0/3641409708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f1415bff0 0x7f0f1406f650 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.836+0000 7f0f18a00640 1 -- 172.21.15.44:0/3641409708 >> 172.21.15.44:0/3641409708 conn(0x7f0f140d0ab0 msgr2=0x7f0f140d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:51.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 -- 172.21.15.44:0/3641409708 shutdown_connections 2024-09-13T08:52:51.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 -- 172.21.15.44:0/3641409708 wait complete. 2024-09-13T08:52:51.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 Processor -- start 2024-09-13T08:52:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 -- start start 2024-09-13T08:52:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f1406d880 0x7f0f140ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f14070de0 0x7f0f14152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1415bff0 0x7f0f14152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14168ff0 con 0x7f0f14070de0 2024-09-13T08:52:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14168e70 con 0x7f0f1415bff0 2024-09-13T08:52:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0f14169170 con 0x7f0f1406d880 2024-09-13T08:52:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f14070de0 0x7f0f14152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f14070de0 0x7f0f14152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41942/0 (socket says 172.21.15.44:41942) 2024-09-13T08:52:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0aa00640 1 -- 172.21.15.44:0/790756324 learned_addr learned my addr 172.21.15.44:0/790756324 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f13400640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f1406d880 0x7f0f140ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f13e00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1415bff0 0x7f0f14152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0aa00640 1 -- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f1406d880 msgr2=0x7f0f140ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0aa00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f1406d880 0x7f0f140ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0aa00640 1 -- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1415bff0 msgr2=0x7f0f14152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0aa00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1415bff0 0x7f0f14152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0aa00640 1 -- 172.21.15.44:0/790756324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f140c97d0 con 0x7f0f14070de0 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0aa00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f14070de0 0x7f0f14152830 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c450 tx=0x7f0efc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0be00640 1 -- 172.21.15.44:0/790756324 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc012830 con 0x7f0f14070de0 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0be00640 1 -- 172.21.15.44:0/790756324 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0efc017070 con 0x7f0f14070de0 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f0be00640 1 -- 172.21.15.44:0/790756324 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc012d80 con 0x7f0f14070de0 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.840+0000 7f0f18a00640 1 -- 172.21.15.44:0/790756324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f140c9ab0 con 0x7f0f14070de0 2024-09-13T08:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.844+0000 7f0f18a00640 1 -- 172.21.15.44:0/790756324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f140ca070 con 0x7f0f14070de0 2024-09-13T08:52:51.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.844+0000 7f0f18a00640 1 -- 172.21.15.44:0/790756324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f14073630 con 0x7f0f14070de0 2024-09-13T08:52:51.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.844+0000 7f0f0be00640 1 -- 172.21.15.44:0/790756324 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 55) ==== 100140+0+0 (secure 0 0 0) 0x7f0efc01c460 con 0x7f0f14070de0 2024-09-13T08:52:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.844+0000 7f0f0be00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ef00750b0 0x7f0ef0077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.844+0000 7f0f0be00640 1 -- 172.21.15.44:0/790756324 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f0efc092c40 con 0x7f0f14070de0 2024-09-13T08:52:51.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.844+0000 7f0f13400640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ef00750b0 0x7f0ef0077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:51.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.848+0000 7f0f13400640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ef00750b0 0x7f0ef0077570 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0f040af000 tx=0x7f0f0401b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:51.848+0000 7f0f0be00640 1 -- 172.21.15.44:0/790756324 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0efc05f8f0 con 0x7f0f14070de0 2024-09-13T08:52:52.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.052+0000 7f0f18a00640 1 -- 172.21.15.44:0/790756324 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f14072b50 con 0x7f0ef00750b0 2024-09-13T08:52:52.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f0be00640 1 -- 172.21.15.44:0/790756324 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f0f14072b50 con 0x7f0ef00750b0 2024-09-13T08:52:52.056 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:52.056 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:52:52.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:52:52.057 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sv1" 2024-09-13T08:52:52.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:52:52.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:52:52.057 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1clone1" 2024-09-13T08:52:52.057 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:52:52.057 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:52:52.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 -- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ef00750b0 msgr2=0x7f0ef0077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ef00750b0 0x7f0ef0077570 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0f040af000 tx=0x7f0f0401b120 comp rx=0 tx=0).stop 2024-09-13T08:52:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 -- 172.21.15.44:0/790756324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f14070de0 msgr2=0x7f0f14152830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f14070de0 0x7f0f14152830 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c450 tx=0x7f0efc00c910 comp rx=0 tx=0).stop 2024-09-13T08:52:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 -- 172.21.15.44:0/790756324 shutdown_connections 2024-09-13T08:52:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0ef00750b0 0x7f0ef0077570 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1415bff0 0x7f0f14152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f14070de0 0x7f0f14152830 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 --2- 172.21.15.44:0/790756324 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0f1406d880 0x7f0f140ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 -- 172.21.15.44:0/790756324 >> 172.21.15.44:0/790756324 conn(0x7f0f140d0ab0 msgr2=0x7f0f140b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 -- 172.21.15.44:0/790756324 shutdown_connections 2024-09-13T08:52:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.056+0000 7f0f18a00640 1 -- 172.21.15.44:0/790756324 wait complete. 2024-09-13T08:52:52.078 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-13T08:52:52.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.228+0000 7f92da400640 1 -- 172.21.15.44:0/2675319434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b8004810 msgr2=0x7f92b8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.228+0000 7f92da400640 1 --2- 172.21.15.44:0/2675319434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b8004810 0x7f92b8004c90 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009b30 tx=0x7f92bc01d230 comp rx=0 tx=0).stop 2024-09-13T08:52:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.228+0000 7f92da400640 1 -- 172.21.15.44:0/2675319434 shutdown_connections 2024-09-13T08:52:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.228+0000 7f92da400640 1 --2- 172.21.15.44:0/2675319434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b8004810 0x7f92b8004c90 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.228+0000 7f92da400640 1 -- 172.21.15.44:0/2675319434 >> 172.21.15.44:0/2675319434 conn(0x7f92d40d1dc0 msgr2=0x7f92d40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.228+0000 7f92da400640 1 -- 172.21.15.44:0/2675319434 shutdown_connections 2024-09-13T08:52:52.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.228+0000 7f92da400640 1 -- 172.21.15.44:0/2675319434 wait complete. 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 Processor -- start 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 -- start start 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b8004810 0x7f92d4071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92d40b60e0 0x7f92d414fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92d40b4c10 0x7f92d41503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f92d40cffc0 con 0x7f92d40b60e0 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f92d40cfe40 con 0x7f92b8004810 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f92d40d0140 con 0x7f92d40b4c10 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92d40b60e0 0x7f92d414fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92d40b60e0 0x7f92d414fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54016/0 (socket says 172.21.15.44:54016) 2024-09-13T08:52:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cbe00640 1 -- 172.21.15.44:0/2890836640 learned_addr learned my addr 172.21.15.44:0/2890836640 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92d3400640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b8004810 0x7f92d4071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92d3e00640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92d40b4c10 0x7f92d41503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cbe00640 1 -- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92d40b4c10 msgr2=0x7f92d41503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cbe00640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92d40b4c10 0x7f92d41503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cbe00640 1 -- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b8004810 msgr2=0x7f92d4071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cbe00640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b8004810 0x7f92d4071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cbe00640 1 -- 172.21.15.44:0/2890836640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d40cdc90 con 0x7f92d40b60e0 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cbe00640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92d40b60e0 0x7f92d414fea0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f92bc01d060 tx=0x7f92bc005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cb400640 1 -- 172.21.15.44:0/2890836640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0b1040 con 0x7f92d40b60e0 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cb400640 1 -- 172.21.15.44:0/2890836640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92bc004030 con 0x7f92d40b60e0 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92cb400640 1 -- 172.21.15.44:0/2890836640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc004460 con 0x7f92d40b60e0 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 -- 172.21.15.44:0/2890836640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d41564e0 con 0x7f92d40b60e0 2024-09-13T08:52:52.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 -- 172.21.15.44:0/2890836640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d41f8670 con 0x7f92d40b60e0 2024-09-13T08:52:52.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.232+0000 7f92da400640 1 -- 172.21.15.44:0/2890836640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d40d06a0 con 0x7f92d40b60e0 2024-09-13T08:52:52.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.236+0000 7f92cb400640 1 -- 172.21.15.44:0/2890836640 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 55) ==== 100140+0+0 (secure 0 0 0) 0x7f92bc0bd020 con 0x7f92d40b60e0 2024-09-13T08:52:52.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.236+0000 7f92cb400640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92b00750b0 0x7f92b0077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:52.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.236+0000 7f92cb400640 1 -- 172.21.15.44:0/2890836640 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f92bc12d730 con 0x7f92d40b60e0 2024-09-13T08:52:52.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.236+0000 7f92d3400640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92b00750b0 0x7f92b0077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:52.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.236+0000 7f92d3400640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92b00750b0 0x7f92b0077570 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f92d4052940 tx=0x7f92c4001630 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:52.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.240+0000 7f92cb400640 1 -- 172.21.15.44:0/2890836640 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f92bc0c4020 con 0x7f92d40b60e0 2024-09-13T08:52:52.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.452+0000 7f92da400640 1 -- 172.21.15.44:0/2890836640 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92d415c5c0 con 0x7f92b00750b0 2024-09-13T08:52:52.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.468+0000 7f92cb400640 1 -- 172.21.15.44:0/2890836640 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f92d415c5c0 con 0x7f92b00750b0 2024-09-13T08:52:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:52.470 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:52:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:52:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1" 2024-09-13T08:52:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:52:52.470 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:52:52.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 -- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92b00750b0 msgr2=0x7f92b0077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92b00750b0 0x7f92b0077570 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f92d4052940 tx=0x7f92c4001630 comp rx=0 tx=0).stop 2024-09-13T08:52:52.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 -- 172.21.15.44:0/2890836640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92d40b60e0 msgr2=0x7f92d414fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92d40b60e0 0x7f92d414fea0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f92bc01d060 tx=0x7f92bc005640 comp rx=0 tx=0).stop 2024-09-13T08:52:52.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 -- 172.21.15.44:0/2890836640 shutdown_connections 2024-09-13T08:52:52.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f92b00750b0 0x7f92b0077570 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f92d40b4c10 0x7f92d41503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92d40b60e0 0x7f92d414fea0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 --2- 172.21.15.44:0/2890836640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f92b8004810 0x7f92d4071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 -- 172.21.15.44:0/2890836640 >> 172.21.15.44:0/2890836640 conn(0x7f92d40d1dc0 msgr2=0x7f92d40b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:52.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 -- 172.21.15.44:0/2890836640 shutdown_connections 2024-09-13T08:52:52.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.472+0000 7f92da400640 1 -- 172.21.15.44:0/2890836640 wait complete. 2024-09-13T08:52:52.492 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-13T08:52:52.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.644+0000 7f4378600640 1 -- 172.21.15.44:0/3254910564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437006e810 msgr2=0x7f437006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.644+0000 7f4378600640 1 --2- 172.21.15.44:0/3254910564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437006e810 0x7f437006f1f0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f4370070d80 tx=0x7f436801cb50 comp rx=0 tx=0).stop 2024-09-13T08:52:52.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.644+0000 7f4378600640 1 -- 172.21.15.44:0/3254910564 shutdown_connections 2024-09-13T08:52:52.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.644+0000 7f4378600640 1 --2- 172.21.15.44:0/3254910564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437006e810 0x7f437006f1f0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.644+0000 7f4378600640 1 --2- 172.21.15.44:0/3254910564 >> v2:172.21.15.129:3300/0 conn(0x7f43700cf300 0x7f437006ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.644+0000 7f4378600640 1 -- 172.21.15.44:0/3254910564 >> 172.21.15.44:0/3254910564 conn(0x7f43700b18f0 msgr2=0x7f43700b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:52.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.644+0000 7f4378600640 1 -- 172.21.15.44:0/3254910564 shutdown_connections 2024-09-13T08:52:52.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.644+0000 7f4378600640 1 -- 172.21.15.44:0/3254910564 wait complete. 2024-09-13T08:52:52.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.644+0000 7f4378600640 1 Processor -- start 2024-09-13T08:52:52.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.644+0000 7f4378600640 1 -- start start 2024-09-13T08:52:52.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4378600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437006e810 0x7f4370142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:52.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4378600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700cf300 0x7f437013e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:52.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4378600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f437015ade0 0x7f4370152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:52.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4378600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4370168840 con 0x7f437006e810 2024-09-13T08:52:52.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4378600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43701686c0 con 0x7f43700cf300 2024-09-13T08:52:52.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4378600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f43701689c0 con 0x7f437015ade0 2024-09-13T08:52:52.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437006e810 0x7f4370142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:52.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437006e810 0x7f4370142280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54036/0 (socket says 172.21.15.44:54036) 2024-09-13T08:52:52.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375c00640 1 -- 172.21.15.44:0/1870853617 learned_addr learned my addr 172.21.15.44:0/1870853617 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:52.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375200640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700cf300 0x7f437013e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4376600640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f437015ade0 0x7f4370152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375c00640 1 -- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f437015ade0 msgr2=0x7f4370152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375c00640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f437015ade0 0x7f4370152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375c00640 1 -- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700cf300 msgr2=0x7f437013e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375c00640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700cf300 0x7f437013e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375c00640 1 -- 172.21.15.44:0/1870853617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43701427c0 con 0x7f437006e810 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375200640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700cf300 0x7f437013e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4375c00640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437006e810 0x7f4370142280 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f436400c430 tx=0x7f436400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f436d600640 1 -- 172.21.15.44:0/1870853617 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364011790 con 0x7f437006e810 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f436d600640 1 -- 172.21.15.44:0/1870853617 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4364017070 con 0x7f437006e810 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f436d600640 1 -- 172.21.15.44:0/1870853617 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364011ce0 con 0x7f437006e810 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4378600640 1 -- 172.21.15.44:0/1870853617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437013ef30 con 0x7f437006e810 2024-09-13T08:52:52.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4378600640 1 -- 172.21.15.44:0/1870853617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437013f4b0 con 0x7f437006e810 2024-09-13T08:52:52.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f4378600640 1 -- 172.21.15.44:0/1870853617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4370069410 con 0x7f437006e810 2024-09-13T08:52:52.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.648+0000 7f436d600640 1 -- 172.21.15.44:0/1870853617 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 55) ==== 100140+0+0 (secure 0 0 0) 0x7f436401c460 con 0x7f437006e810 2024-09-13T08:52:52.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.652+0000 7f436d600640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43500750b0 0x7f4350077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:52.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.652+0000 7f436d600640 1 -- 172.21.15.44:0/1870853617 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f4364092ce0 con 0x7f437006e810 2024-09-13T08:52:52.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.652+0000 7f4375200640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43500750b0 0x7f4350077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:52.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.652+0000 7f4375200640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43500750b0 0x7f4350077570 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f43700cc2f0 tx=0x7f435c00b2c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:52.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.656+0000 7f436d600640 1 -- 172.21.15.44:0/1870853617 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f436405fa50 con 0x7f437006e810 2024-09-13T08:52:52.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.864+0000 7f4378600640 1 -- 172.21.15.44:0/1870853617 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43700cf720 con 0x7f43500750b0 2024-09-13T08:52:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f436d600640 1 -- 172.21.15.44:0/1870853617 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f43700cf720 con 0x7f43500750b0 2024-09-13T08:52:52.895 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:52:52.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 -- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43500750b0 msgr2=0x7f4350077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43500750b0 0x7f4350077570 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f43700cc2f0 tx=0x7f435c00b2c0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 -- 172.21.15.44:0/1870853617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437006e810 msgr2=0x7f4370142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:52.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437006e810 0x7f4370142280 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f436400c430 tx=0x7f436400c8f0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 -- 172.21.15.44:0/1870853617 shutdown_connections 2024-09-13T08:52:52.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f43500750b0 0x7f4350077570 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f437015ade0 0x7f4370152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43700cf300 0x7f437013e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 --2- 172.21.15.44:0/1870853617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437006e810 0x7f4370142280 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:52.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 -- 172.21.15.44:0/1870853617 >> 172.21.15.44:0/1870853617 conn(0x7f43700b18f0 msgr2=0x7f4370072950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:52.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 -- 172.21.15.44:0/1870853617 shutdown_connections 2024-09-13T08:52:52.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:52.896+0000 7f4378600640 1 -- 172.21.15.44:0/1870853617 wait complete. 2024-09-13T08:52:52.918 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-13T08:52:53.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 -- 172.21.15.44:0/3484247398 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43800cdf20 msgr2=0x7f43800b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 --2- 172.21.15.44:0/3484247398 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43800cdf20 0x7f43800b1860 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4378009c80 tx=0x7f437801cd70 comp rx=0 tx=0).stop 2024-09-13T08:52:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 -- 172.21.15.44:0/3484247398 shutdown_connections 2024-09-13T08:52:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 --2- 172.21.15.44:0/3484247398 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43800cdf20 0x7f43800b1860 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 --2- 172.21.15.44:0/3484247398 >> v2:172.21.15.129:3300/0 conn(0x7f43800b3bb0 0x7f43800b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 -- 172.21.15.44:0/3484247398 >> 172.21.15.44:0/3484247398 conn(0x7f43800d1e00 msgr2=0x7f43800d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 -- 172.21.15.44:0/3484247398 shutdown_connections 2024-09-13T08:52:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 -- 172.21.15.44:0/3484247398 wait complete. 2024-09-13T08:52:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 Processor -- start 2024-09-13T08:52:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 -- start start 2024-09-13T08:52:53.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43800b3bb0 0x7f43800b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:53.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43800cdf20 0x7f438015ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:53.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f438015f340 0x7f4380164f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:53.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4380168f60 con 0x7f43800b3bb0 2024-09-13T08:52:53.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4380168de0 con 0x7f438015f340 2024-09-13T08:52:53.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.068+0000 7f4387c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f43801690e0 con 0x7f43800cdf20 2024-09-13T08:52:53.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4385200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43800b3bb0 0x7f43800b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:53.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4385200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43800b3bb0 0x7f43800b6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54046/0 (socket says 172.21.15.44:54046) 2024-09-13T08:52:53.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4385200640 1 -- 172.21.15.44:0/1611847287 learned_addr learned my addr 172.21.15.44:0/1611847287 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:53.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f437fe00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43800cdf20 0x7f438015ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:53.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4385c00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f438015f340 0x7f4380164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4385200640 1 -- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43800cdf20 msgr2=0x7f438015ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4385200640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43800cdf20 0x7f438015ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4385200640 1 -- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f438015f340 msgr2=0x7f4380164f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4385200640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f438015f340 0x7f4380164f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4385200640 1 -- 172.21.15.44:0/1611847287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43800b79f0 con 0x7f43800b3bb0 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4385200640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43800b3bb0 0x7f43800b6730 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f437400c3c0 tx=0x7f437400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f437cc00640 1 -- 172.21.15.44:0/1611847287 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374017070 con 0x7f43800b3bb0 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f437cc00640 1 -- 172.21.15.44:0/1611847287 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43740053c0 con 0x7f43800b3bb0 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4387c00640 1 -- 172.21.15.44:0/1611847287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43800b5970 con 0x7f43800b3bb0 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f437cc00640 1 -- 172.21.15.44:0/1611847287 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374003860 con 0x7f43800b3bb0 2024-09-13T08:52:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4387c00640 1 -- 172.21.15.44:0/1611847287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43800b5f30 con 0x7f43800b3bb0 2024-09-13T08:52:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f4387c00640 1 -- 172.21.15.44:0/1611847287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43800b34a0 con 0x7f43800b3bb0 2024-09-13T08:52:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.072+0000 7f437cc00640 1 -- 172.21.15.44:0/1611847287 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 55) ==== 100140+0+0 (secure 0 0 0) 0x7f437401a070 con 0x7f43800b3bb0 2024-09-13T08:52:53.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.076+0000 7f437cc00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4360075080 0x7f4360077540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:53.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.076+0000 7f437cc00640 1 -- 172.21.15.44:0/1611847287 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f4374092280 con 0x7f43800b3bb0 2024-09-13T08:52:53.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.076+0000 7f437fe00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4360075080 0x7f4360077540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:53.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.076+0000 7f437fe00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4360075080 0x7f4360077540 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f436c009600 tx=0x7f436c0091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:53.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.080+0000 7f437cc00640 1 -- 172.21.15.44:0/1611847287 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f43800d2c90 con 0x7f43800b3bb0 2024-09-13T08:52:53.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.280+0000 7f4387c00640 1 -- 172.21.15.44:0/1611847287 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f438015ad10 con 0x7f4360075080 2024-09-13T08:52:53.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f437cc00640 1 -- 172.21.15.44:0/1611847287 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f438015ad10 con 0x7f4360075080 2024-09-13T08:52:53.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 -- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4360075080 msgr2=0x7f4360077540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:53.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4360075080 0x7f4360077540 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f436c009600 tx=0x7f436c0091e0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 -- 172.21.15.44:0/1611847287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43800b3bb0 msgr2=0x7f43800b6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:53.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43800b3bb0 0x7f43800b6730 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f437400c3c0 tx=0x7f437400c880 comp rx=0 tx=0).stop 2024-09-13T08:52:53.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 -- 172.21.15.44:0/1611847287 shutdown_connections 2024-09-13T08:52:53.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4360075080 0x7f4360077540 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f438015f340 0x7f4380164f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43800cdf20 0x7f438015ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 --2- 172.21.15.44:0/1611847287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43800b3bb0 0x7f43800b6730 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 -- 172.21.15.44:0/1611847287 >> 172.21.15.44:0/1611847287 conn(0x7f43800d1e00 msgr2=0x7f438015dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:53.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 -- 172.21.15.44:0/1611847287 shutdown_connections 2024-09-13T08:52:53.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.304+0000 7f4387c00640 1 -- 172.21.15.44:0/1611847287 wait complete. 2024-09-13T08:52:53.318 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:53.317+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:53.318 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:53.317+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:53.318 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:53.317+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:53.318 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:53.317+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:53.319 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:52:53.317+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:52:53.327 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-13T08:52:53.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.480+0000 7fa924200640 1 -- 172.21.15.44:0/1319013455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c15be10 msgr2=0x7fa91c0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:53.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.480+0000 7fa924200640 1 --2- 172.21.15.44:0/1319013455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c15be10 0x7fa91c0ce700 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa910009840 tx=0x7fa91001f010 comp rx=0 tx=0).stop 2024-09-13T08:52:53.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.480+0000 7fa924200640 1 -- 172.21.15.44:0/1319013455 shutdown_connections 2024-09-13T08:52:53.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.480+0000 7fa924200640 1 --2- 172.21.15.44:0/1319013455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c15be10 0x7fa91c0ce700 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.480+0000 7fa924200640 1 -- 172.21.15.44:0/1319013455 >> 172.21.15.44:0/1319013455 conn(0x7fa91c0d1da0 msgr2=0x7fa91c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:53.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.480+0000 7fa924200640 1 -- 172.21.15.44:0/1319013455 shutdown_connections 2024-09-13T08:52:53.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.480+0000 7fa924200640 1 -- 172.21.15.44:0/1319013455 wait complete. 2024-09-13T08:52:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.480+0000 7fa924200640 1 Processor -- start 2024-09-13T08:52:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.480+0000 7fa924200640 1 -- start start 2024-09-13T08:52:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.480+0000 7fa924200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa91c0b0b60 0x7fa91c149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa924200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c15be10 0x7fa91c0c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa924200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c15c1f0 0x7fa91c0cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa924200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c0cd670 con 0x7fa91c15be10 2024-09-13T08:52:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa924200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c0cd4f0 con 0x7fa91c15c1f0 2024-09-13T08:52:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa924200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa91c0cd7f0 con 0x7fa91c0b0b60 2024-09-13T08:52:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa920e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c15be10 0x7fa91c0c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa920e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c15be10 0x7fa91c0c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54062/0 (socket says 172.21.15.44:54062) 2024-09-13T08:52:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa920e00640 1 -- 172.21.15.44:0/2814876753 learned_addr learned my addr 172.21.15.44:0/2814876753 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa922200640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c15c1f0 0x7fa91c0cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:53.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa921800640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa91c0b0b60 0x7fa91c149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:53.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa920e00640 1 -- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa91c0b0b60 msgr2=0x7fa91c149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:53.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa920e00640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa91c0b0b60 0x7fa91c149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa920e00640 1 -- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c15c1f0 msgr2=0x7fa91c0cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:53.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa920e00640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c15c1f0 0x7fa91c0cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:53.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa920e00640 1 -- 172.21.15.44:0/2814876753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c1f5850 con 0x7fa91c15be10 2024-09-13T08:52:53.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa922200640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c15c1f0 0x7fa91c0cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:53.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa921800640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa91c0b0b60 0x7fa91c149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:53.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa920e00640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c15be10 0x7fa91c0c8140 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c400 tx=0x7fa90800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:53.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa919600640 1 -- 172.21.15.44:0/2814876753 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908017070 con 0x7fa91c15be10 2024-09-13T08:52:53.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa919600640 1 -- 172.21.15.44:0/2814876753 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa908005400 con 0x7fa91c15be10 2024-09-13T08:52:53.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa919600640 1 -- 172.21.15.44:0/2814876753 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908003860 con 0x7fa91c15be10 2024-09-13T08:52:53.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa924200640 1 -- 172.21.15.44:0/2814876753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c1f4c40 con 0x7fa91c15be10 2024-09-13T08:52:53.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa924200640 1 -- 172.21.15.44:0/2814876753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c1f5200 con 0x7fa91c15be10 2024-09-13T08:52:53.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa924200640 1 -- 172.21.15.44:0/2814876753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91c0cdca0 con 0x7fa91c15be10 2024-09-13T08:52:53.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.484+0000 7fa919600640 1 -- 172.21.15.44:0/2814876753 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 55) ==== 100140+0+0 (secure 0 0 0) 0x7fa90801a070 con 0x7fa91c15be10 2024-09-13T08:52:53.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.488+0000 7fa919600640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8fc075080 0x7fa8fc077540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:53.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.488+0000 7fa919600640 1 -- 172.21.15.44:0/2814876753 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7fa908092290 con 0x7fa91c15be10 2024-09-13T08:52:53.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.488+0000 7fa921800640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8fc075080 0x7fa8fc077540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:53.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.488+0000 7fa921800640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8fc075080 0x7fa8fc077540 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa9100b2000 tx=0x7fa91001d850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:53.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.492+0000 7fa919600640 1 -- 172.21.15.44:0/2814876753 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa90805e8f0 con 0x7fa91c15be10 2024-09-13T08:52:53.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:53.724+0000 7fa924200640 1 -- 172.21.15.44:0/2814876753 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa91c070350 con 0x7fa8fc075080 2024-09-13T08:52:54.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.036+0000 7fa919600640 1 -- 172.21.15.44:0/2814876753 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa91c070350 con 0x7fa8fc075080 2024-09-13T08:52:54.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 -- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8fc075080 msgr2=0x7fa8fc077540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:54.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8fc075080 0x7fa8fc077540 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa9100b2000 tx=0x7fa91001d850 comp rx=0 tx=0).stop 2024-09-13T08:52:54.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 -- 172.21.15.44:0/2814876753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c15be10 msgr2=0x7fa91c0c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:54.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c15be10 0x7fa91c0c8140 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c400 tx=0x7fa90800c8c0 comp rx=0 tx=0).stop 2024-09-13T08:52:54.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 -- 172.21.15.44:0/2814876753 shutdown_connections 2024-09-13T08:52:54.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8fc075080 0x7fa8fc077540 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:54.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c15c1f0 0x7fa91c0cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:54.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c15be10 0x7fa91c0c8140 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:54.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 --2- 172.21.15.44:0/2814876753 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa91c0b0b60 0x7fa91c149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:54.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 -- 172.21.15.44:0/2814876753 >> 172.21.15.44:0/2814876753 conn(0x7fa91c0d1da0 msgr2=0x7fa91c072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:54.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 -- 172.21.15.44:0/2814876753 shutdown_connections 2024-09-13T08:52:54.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54.040+0000 7fa924200640 1 -- 172.21.15.44:0/2814876753 wait complete. 2024-09-13T08:52:54.060 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-13T08:52:54.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:52:54.115 INFO:teuthology.orchestra.run.smithi044.stdout:3 2024-09-13T08:52:54.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:52:59.124 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-13T08:52:59.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:52:59.184 INFO:teuthology.orchestra.run.smithi044.stdout:2 2024-09-13T08:52:59.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:52:59.192 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:52:59.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 -- 172.21.15.44:0/3256872935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc004d50 msgr2=0x7f98bc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:59.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 --2- 172.21.15.44:0/3256872935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc004d50 0x7f98bc0051d0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f98d000c610 tx=0x7f98d001faa0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98d7400640 1 -- 172.21.15.44:0/3256872935 reap_dead start 2024-09-13T08:52:59.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 -- 172.21.15.44:0/3256872935 shutdown_connections 2024-09-13T08:52:59.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 -- 172.21.15.44:0/3256872935 >> 172.21.15.44:0/3256872935 conn(0x7f98d80d0cd0 msgr2=0x7f98d80d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:59.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 -- 172.21.15.44:0/3256872935 shutdown_connections 2024-09-13T08:52:59.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 -- 172.21.15.44:0/3256872935 wait complete. 2024-09-13T08:52:59.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 Processor -- start 2024-09-13T08:52:59.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 -- start start 2024-09-13T08:52:59.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc004d50 0x7f98d814e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:59.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d80b1c00 0x7f98d8160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:59.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98d80b2080 0x7f98d81612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80adc60 con 0x7f98bc004d50 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80adae0 con 0x7f98d80b1c00 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98dde00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f98d80adde0 con 0x7f98d80b2080 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.348+0000 7f98d6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc004d50 0x7f98d814e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98d6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc004d50 0x7f98d814e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54088/0 (socket says 172.21.15.44:54088) 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98d6a00640 1 -- 172.21.15.44:0/283773962 learned_addr learned my addr 172.21.15.44:0/283773962 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98cfe00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d80b1c00 0x7f98d8160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98d7400640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98d80b2080 0x7f98d81612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98d6a00640 1 -- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98d80b2080 msgr2=0x7f98d81612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98d6a00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98d80b2080 0x7f98d81612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98d6a00640 1 -- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d80b1c00 msgr2=0x7f98d8160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98d6a00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d80b1c00 0x7f98d8160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98d6a00640 1 -- 172.21.15.44:0/283773962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d81f5560 con 0x7f98bc004d50 2024-09-13T08:52:59.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98cfe00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d80b1c00 0x7f98d8160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:59.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98d7400640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98d80b2080 0x7f98d81612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:59.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98d6a00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc004d50 0x7f98d814e890 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f98c8011b40 tx=0x7f98c8009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:59.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98cf400640 1 -- 172.21.15.44:0/283773962 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c800fcd0 con 0x7f98bc004d50 2024-09-13T08:52:59.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98cf400640 1 -- 172.21.15.44:0/283773962 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98c800fe70 con 0x7f98bc004d50 2024-09-13T08:52:59.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98cf400640 1 -- 172.21.15.44:0/283773962 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8004260 con 0x7f98bc004d50 2024-09-13T08:52:59.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98dde00640 1 -- 172.21.15.44:0/283773962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d81f5280 con 0x7f98bc004d50 2024-09-13T08:52:59.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98dde00640 1 -- 172.21.15.44:0/283773962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d814e090 con 0x7f98bc004d50 2024-09-13T08:52:59.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98dde00640 1 -- 172.21.15.44:0/283773962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d8161b70 con 0x7f98bc004d50 2024-09-13T08:52:59.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.352+0000 7f98cf400640 1 -- 172.21.15.44:0/283773962 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f98c8003600 con 0x7f98bc004d50 2024-09-13T08:52:59.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.356+0000 7f98cf400640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b4075140 0x7f98b4077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:59.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.356+0000 7f98cf400640 1 -- 172.21.15.44:0/283773962 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5877+0+0 (secure 0 0 0) 0x7f98c80939c0 con 0x7f98bc004d50 2024-09-13T08:52:59.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.356+0000 7f98cfe00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b4075140 0x7f98b4077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:59.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.356+0000 7f98cfe00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b4075140 0x7f98b4077600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f98d80c6c90 tx=0x7f98c00091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:59.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.360+0000 7f98cf400640 1 -- 172.21.15.44:0/283773962 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f98c8060670 con 0x7f98bc004d50 2024-09-13T08:52:59.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 -- 172.21.15.44:0/283773962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f98d80cdda0 con 0x7f98bc004d50 2024-09-13T08:52:59.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98cf400640 1 -- 172.21.15.44:0/283773962 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v52)=0 v52) ==== 131+0+0 (secure 0 0 0) 0x7f98c805ff70 con 0x7f98bc004d50 2024-09-13T08:52:59.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 -- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b4075140 msgr2=0x7f98b4077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:59.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b4075140 0x7f98b4077600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f98d80c6c90 tx=0x7f98c00091e0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 -- 172.21.15.44:0/283773962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc004d50 msgr2=0x7f98d814e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:59.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc004d50 0x7f98d814e890 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f98c8011b40 tx=0x7f98c8009f20 comp rx=0 tx=0).stop 2024-09-13T08:52:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 -- 172.21.15.44:0/283773962 shutdown_connections 2024-09-13T08:52:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f98b4075140 0x7f98b4077600 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f98d80b2080 0x7f98d81612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98d80b1c00 0x7f98d8160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 --2- 172.21.15.44:0/283773962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc004d50 0x7f98d814e890 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 -- 172.21.15.44:0/283773962 >> 172.21.15.44:0/283773962 conn(0x7f98d80d0cd0 msgr2=0x7f98d80cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 -- 172.21.15.44:0/283773962 shutdown_connections 2024-09-13T08:52:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.536+0000 7f98dde00640 1 -- 172.21.15.44:0/283773962 wait complete. 2024-09-13T08:52:59.557 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:52:59.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:52:59.646 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:52:59.734 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:52:59.734 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:52:59.734 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:52:59.734 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:52:59.743 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:52:59.743 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:52:59.795 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:52:59.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 -- 172.21.15.44:0/3749029624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37241563e0 msgr2=0x7f372415d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:59.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 --2- 172.21.15.44:0/3749029624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37241563e0 0x7f372415d3d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f3714009840 tx=0x7f371401c7f0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 -- 172.21.15.44:0/3749029624 shutdown_connections 2024-09-13T08:52:59.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 --2- 172.21.15.44:0/3749029624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37241563e0 0x7f372415d3d0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 -- 172.21.15.44:0/3749029624 >> 172.21.15.44:0/3749029624 conn(0x7f372406dd30 msgr2=0x7f372406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:52:59.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 -- 172.21.15.44:0/3749029624 shutdown_connections 2024-09-13T08:52:59.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 -- 172.21.15.44:0/3749029624 wait complete. 2024-09-13T08:52:59.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 Processor -- start 2024-09-13T08:52:59.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 -- start start 2024-09-13T08:52:59.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f372406d330 0x7f3724071000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:59.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37241563e0 0x7f37241fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:59.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f372415cc90 0x7f372413e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:59.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3724169e90 con 0x7f37241563e0 2024-09-13T08:52:59.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3724169d10 con 0x7f372415cc90 2024-09-13T08:52:59.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3729600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f372416a010 con 0x7f372406d330 2024-09-13T08:52:59.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f371be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37241563e0 0x7f37241fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:59.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f371be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37241563e0 0x7f37241fb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54094/0 (socket says 172.21.15.44:54094) 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f371be00640 1 -- 172.21.15.44:0/776651766 learned_addr learned my addr 172.21.15.44:0/776651766 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3723400640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f372415cc90 0x7f372413e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3722a00640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f372406d330 0x7f3724071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f371be00640 1 -- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f372406d330 msgr2=0x7f3724071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f371be00640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f372406d330 0x7f3724071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f371be00640 1 -- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f372415cc90 msgr2=0x7f372413e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f371be00640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f372415cc90 0x7f372413e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f371be00640 1 -- 172.21.15.44:0/776651766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37241fbb30 con 0x7f37241563e0 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3723400640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f372415cc90 0x7f372413e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.988+0000 7f3722a00640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f372406d330 0x7f3724071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:52:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.992+0000 7f371be00640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37241563e0 0x7f37241fb5f0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f370c00c460 tx=0x7f370c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:59.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.992+0000 7f371aa00640 1 -- 172.21.15.44:0/776651766 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c016020 con 0x7f37241563e0 2024-09-13T08:52:59.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.992+0000 7f371aa00640 1 -- 172.21.15.44:0/776651766 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f370c012940 con 0x7f37241563e0 2024-09-13T08:52:59.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.992+0000 7f371aa00640 1 -- 172.21.15.44:0/776651766 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c012d90 con 0x7f37241563e0 2024-09-13T08:52:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.992+0000 7f3729600640 1 -- 172.21.15.44:0/776651766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372413eb60 con 0x7f37241563e0 2024-09-13T08:52:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.992+0000 7f3729600640 1 -- 172.21.15.44:0/776651766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372406fac0 con 0x7f37241563e0 2024-09-13T08:52:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.992+0000 7f3729600640 1 -- 172.21.15.44:0/776651766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372415d3d0 con 0x7f37241563e0 2024-09-13T08:52:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.992+0000 7f371aa00640 1 -- 172.21.15.44:0/776651766 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f370c019020 con 0x7f37241563e0 2024-09-13T08:52:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.992+0000 7f371aa00640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3700075110 0x7f37000775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:52:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.992+0000 7f371aa00640 1 -- 172.21.15.44:0/776651766 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5877+0+0 (secure 0 0 0) 0x7f370c092ca0 con 0x7f37241563e0 2024-09-13T08:52:59.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.996+0000 7f3722a00640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3700075110 0x7f37000775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:52:59.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.996+0000 7f3722a00640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3700075110 0x7f37000775d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f37140af000 tx=0x7f371401b030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:52:59.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:52:59.996+0000 7f371aa00640 1 -- 172.21.15.44:0/776651766 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f370c020030 con 0x7f37241563e0 2024-09-13T08:53:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.180+0000 7f3729600640 1 -- 172.21.15.44:0/776651766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3724073d90 con 0x7f37241563e0 2024-09-13T08:53:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f371aa00640 1 -- 172.21.15.44:0/776651766 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v132) ==== 80+0+0 (secure 0 0 0) 0x7f370c05fa10 con 0x7f37241563e0 2024-09-13T08:53:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:53:00.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 -- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3700075110 msgr2=0x7f37000775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:00.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3700075110 0x7f37000775d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f37140af000 tx=0x7f371401b030 comp rx=0 tx=0).stop 2024-09-13T08:53:00.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 -- 172.21.15.44:0/776651766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37241563e0 msgr2=0x7f37241fb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:00.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37241563e0 0x7f37241fb5f0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f370c00c460 tx=0x7f370c00c920 comp rx=0 tx=0).stop 2024-09-13T08:53:00.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 -- 172.21.15.44:0/776651766 shutdown_connections 2024-09-13T08:53:00.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3700075110 0x7f37000775d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f372415cc90 0x7f372413e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37241563e0 0x7f37241fb5f0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 --2- 172.21.15.44:0/776651766 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f372406d330 0x7f3724071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 -- 172.21.15.44:0/776651766 >> 172.21.15.44:0/776651766 conn(0x7f372406dd30 msgr2=0x7f37241510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:00.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 -- 172.21.15.44:0/776651766 shutdown_connections 2024-09-13T08:53:00.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.184+0000 7f3729600640 1 -- 172.21.15.44:0/776651766 wait complete. 2024-09-13T08:53:00.206 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T08:53:00.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:00.207 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T08:53:00.264 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:53:00.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:00.264 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T08:53:00.281 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:53:00.281 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:53:00.281 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:00.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:53:00.290 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:53:00.291 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:53:00.291 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:00.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:53:00.331 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:53:00.331 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:53:00.331 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:00.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:53:00.343 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:53:00.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.520+0000 7f5c07600640 1 -- 172.21.15.44:0/293033165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c0015bea0 msgr2=0x7f5c000ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:00.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.520+0000 7f5c07600640 1 --2- 172.21.15.44:0/293033165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c0015bea0 0x7f5c000ce670 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f5c00073d10 tx=0x7f5bf401c8c0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.520+0000 7f5c07600640 1 -- 172.21.15.44:0/293033165 shutdown_connections 2024-09-13T08:53:00.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.520+0000 7f5c07600640 1 --2- 172.21.15.44:0/293033165 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c000d28e0 0x7f5c000cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.520+0000 7f5c07600640 1 --2- 172.21.15.44:0/293033165 >> v2:172.21.15.129:3300/0 conn(0x7f5c000cdb30 0x7f5c000cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.520+0000 7f5c07600640 1 --2- 172.21.15.44:0/293033165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c0015bea0 0x7f5c000ce670 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.520+0000 7f5c07600640 1 -- 172.21.15.44:0/293033165 >> 172.21.15.44:0/293033165 conn(0x7f5c000d1dc0 msgr2=0x7f5c000d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:00.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.520+0000 7f5c07600640 1 -- 172.21.15.44:0/293033165 shutdown_connections 2024-09-13T08:53:00.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.520+0000 7f5c07600640 1 -- 172.21.15.44:0/293033165 wait complete. 2024-09-13T08:53:00.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c07600640 1 Processor -- start 2024-09-13T08:53:00.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c07600640 1 -- start start 2024-09-13T08:53:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c07600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c000cdb30 0x7f5c0014f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c07600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c000d28e0 0x7f5c00159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c07600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0015bea0 0x7f5c0014ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c07600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c000af070 con 0x7f5c000d28e0 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c07600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c000aeef0 con 0x7f5c0015bea0 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c07600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5c000af1f0 con 0x7f5c000cdb30 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c000d28e0 0x7f5c00159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c000d28e0 0x7f5c00159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54124/0 (socket says 172.21.15.44:54124) 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bffe00640 1 -- 172.21.15.44:0/3312374754 learned_addr learned my addr 172.21.15.44:0/3312374754 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c05600640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0015bea0 0x7f5c0014ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c04c00640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c000cdb30 0x7f5c0014f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bffe00640 1 -- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c000cdb30 msgr2=0x7f5c0014f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bffe00640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c000cdb30 0x7f5c0014f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bffe00640 1 -- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0015bea0 msgr2=0x7f5c0014ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bffe00640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0015bea0 0x7f5c0014ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bffe00640 1 -- 172.21.15.44:0/3312374754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0014ffa0 con 0x7f5c000d28e0 2024-09-13T08:53:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bffe00640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c000d28e0 0x7f5c00159ba0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00c940 tx=0x7f5bec00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bfcc00640 1 -- 172.21.15.44:0/3312374754 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec007bb0 con 0x7f5c000d28e0 2024-09-13T08:53:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bfcc00640 1 -- 172.21.15.44:0/3312374754 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bec00f450 con 0x7f5c000d28e0 2024-09-13T08:53:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5bfcc00640 1 -- 172.21.15.44:0/3312374754 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec007e60 con 0x7f5c000d28e0 2024-09-13T08:53:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c07600640 1 -- 172.21.15.44:0/3312374754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0015a0e0 con 0x7f5c000d28e0 2024-09-13T08:53:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.524+0000 7f5c07600640 1 -- 172.21.15.44:0/3312374754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c00158480 con 0x7f5c000d28e0 2024-09-13T08:53:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.528+0000 7f5c07600640 1 -- 172.21.15.44:0/3312374754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c000cc9d0 con 0x7f5c000d28e0 2024-09-13T08:53:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.528+0000 7f5bfcc00640 1 -- 172.21.15.44:0/3312374754 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f5bec00f8a0 con 0x7f5c000d28e0 2024-09-13T08:53:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.528+0000 7f5bfcc00640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5be0075110 0x7f5be00775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.528+0000 7f5bfcc00640 1 -- 172.21.15.44:0/3312374754 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5877+0+0 (secure 0 0 0) 0x7f5bec093850 con 0x7f5c000d28e0 2024-09-13T08:53:00.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.528+0000 7f5c04c00640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5be0075110 0x7f5be00775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:00.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.528+0000 7f5c04c00640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5be0075110 0x7f5be00775d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40af000 tx=0x7f5bf401b100 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:00.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.532+0000 7f5bfcc00640 1 -- 172.21.15.44:0/3312374754 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5c0014ffa0 con 0x7f5c000d28e0 2024-09-13T08:53:00.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.740+0000 7f5c07600640 1 -- 172.21.15.44:0/3312374754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c0015aa70 con 0x7f5c000d28e0 2024-09-13T08:53:00.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5bfcc00640 1 -- 172.21.15.44:0/3312374754 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5228 (secure 0 0 0) 0x7f5bec014030 con 0x7f5c000d28e0 2024-09-13T08:53:00.744 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:00.744 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":86,"btime":"2024-09-13T08:52:09:337186+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/771921648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":771921648},{"type":"v1","addr":"172.21.15.129:6836","nonce":771921648}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8104,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1464443005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1464443005},{"type":"v1","addr":"172.21.15.129:6837","nonce":1464443005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2601537201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2601537201},{"type":"v1","addr":"172.21.15.44:6833","nonce":2601537201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:52:00.698019+0000","modified":"2024-09-13T08:52:03.712285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8763":{"gid":8763,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2181881572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2181881572},{"type":"v1","addr":"172.21.15.44:6835","nonce":2181881572}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8763,"qdb_cluster":[8763]},"id":9}]} 2024-09-13T08:53:00.744 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 86 2024-09-13T08:53:00.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 -- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5be0075110 msgr2=0x7f5be00775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:00.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5be0075110 0x7f5be00775d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40af000 tx=0x7f5bf401b100 comp rx=0 tx=0).stop 2024-09-13T08:53:00.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 -- 172.21.15.44:0/3312374754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c000d28e0 msgr2=0x7f5c00159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:00.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c000d28e0 0x7f5c00159ba0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00c940 tx=0x7f5bec00ce00 comp rx=0 tx=0).stop 2024-09-13T08:53:00.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 -- 172.21.15.44:0/3312374754 shutdown_connections 2024-09-13T08:53:00.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5be0075110 0x7f5be00775d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0015bea0 0x7f5c0014ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c000d28e0 0x7f5c00159ba0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 --2- 172.21.15.44:0/3312374754 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5c000cdb30 0x7f5c0014f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 -- 172.21.15.44:0/3312374754 >> 172.21.15.44:0/3312374754 conn(0x7f5c000d1dc0 msgr2=0x7f5c0015b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 -- 172.21.15.44:0/3312374754 shutdown_connections 2024-09-13T08:53:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.744+0000 7f5c07600640 1 -- 172.21.15.44:0/3312374754 wait complete. 2024-09-13T08:53:00.765 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:53:00.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.912+0000 7f8da2600640 1 -- 172.21.15.44:0/4272604229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c0cdb30 msgr2=0x7f8d9c0af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:00.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.912+0000 7f8da2600640 1 --2- 172.21.15.44:0/4272604229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c0cdb30 0x7f8d9c0af0c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009900 tx=0x7f8d9401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.912+0000 7f8da2600640 1 -- 172.21.15.44:0/4272604229 shutdown_connections 2024-09-13T08:53:00.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.912+0000 7f8da2600640 1 --2- 172.21.15.44:0/4272604229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c0cdb30 0x7f8d9c0af0c0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.912+0000 7f8da2600640 1 -- 172.21.15.44:0/4272604229 >> 172.21.15.44:0/4272604229 conn(0x7f8d9c0d1dc0 msgr2=0x7f8d9c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:00.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.912+0000 7f8da2600640 1 -- 172.21.15.44:0/4272604229 shutdown_connections 2024-09-13T08:53:00.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.912+0000 7f8da2600640 1 -- 172.21.15.44:0/4272604229 wait complete. 2024-09-13T08:53:00.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 Processor -- start 2024-09-13T08:53:00.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 -- start start 2024-09-13T08:53:00.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d9c0cdb30 0x7f8d9c14f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:00.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c0ced80 0x7f8d9c0b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:00.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c15bea0 0x7f8d9c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:00.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c16a8a0 con 0x7f8d9c0ced80 2024-09-13T08:53:00.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c16a720 con 0x7f8d9c15bea0 2024-09-13T08:53:00.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8d9c16aa20 con 0x7f8d9c0cdb30 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d92a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c0ced80 0x7f8d9c0b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d92a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c0ced80 0x7f8d9c0b3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54140/0 (socket says 172.21.15.44:54140) 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d92a00640 1 -- 172.21.15.44:0/775087149 learned_addr learned my addr 172.21.15.44:0/775087149 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d9b400640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d9c0cdb30 0x7f8d9c14f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d9be00640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c15bea0 0x7f8d9c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d92a00640 1 -- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d9c0cdb30 msgr2=0x7f8d9c14f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d92a00640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d9c0cdb30 0x7f8d9c14f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d92a00640 1 -- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c15bea0 msgr2=0x7f8d9c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d92a00640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c15bea0 0x7f8d9c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d92a00640 1 -- 172.21.15.44:0/775087149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c14f7f0 con 0x7f8d9c0ced80 2024-09-13T08:53:00.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d92a00640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c0ced80 0x7f8d9c0b3770 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c480 tx=0x7f8d8400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:00.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d93e00640 1 -- 172.21.15.44:0/775087149 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84012830 con 0x7f8d9c0ced80 2024-09-13T08:53:00.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d93e00640 1 -- 172.21.15.44:0/775087149 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d84017070 con 0x7f8d9c0ced80 2024-09-13T08:53:00.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 -- 172.21.15.44:0/775087149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9c0b2ed0 con 0x7f8d9c0ced80 2024-09-13T08:53:00.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8d93e00640 1 -- 172.21.15.44:0/775087149 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84012d80 con 0x7f8d9c0ced80 2024-09-13T08:53:00.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 -- 172.21.15.44:0/775087149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9c0b3490 con 0x7f8d9c0ced80 2024-09-13T08:53:00.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.916+0000 7f8da2600640 1 -- 172.21.15.44:0/775087149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d9c0ce670 con 0x7f8d9c0ced80 2024-09-13T08:53:00.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.920+0000 7f8d93e00640 1 -- 172.21.15.44:0/775087149 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f8d8401c460 con 0x7f8d9c0ced80 2024-09-13T08:53:00.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.920+0000 7f8d93e00640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d78075110 0x7f8d780775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:00.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.920+0000 7f8d93e00640 1 -- 172.21.15.44:0/775087149 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5877+0+0 (secure 0 0 0) 0x7f8d84092ca0 con 0x7f8d9c0ced80 2024-09-13T08:53:00.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.920+0000 7f8d9b400640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d78075110 0x7f8d780775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:00.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.920+0000 7f8d9b400640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d78075110 0x7f8d780775d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c1fb790 tx=0x7f8d8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:00.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:00.924+0000 7f8d93e00640 1 -- 172.21.15.44:0/775087149 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8d8405f300 con 0x7f8d9c0ced80 2024-09-13T08:53:01.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.136+0000 7f8da2600640 1 -- 172.21.15.44:0/775087149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d9c072540 con 0x7f8d9c0ced80 2024-09-13T08:53:01.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.136+0000 7f8d93e00640 1 -- 172.21.15.44:0/775087149 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5228 (secure 0 0 0) 0x7f8d84063d30 con 0x7f8d9c0ced80 2024-09-13T08:53:01.137 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:01.138 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":86,"btime":"2024-09-13T08:52:09:337186+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/771921648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":771921648},{"type":"v1","addr":"172.21.15.129:6836","nonce":771921648}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8104,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1464443005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1464443005},{"type":"v1","addr":"172.21.15.129:6837","nonce":1464443005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2601537201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2601537201},{"type":"v1","addr":"172.21.15.44:6833","nonce":2601537201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:52:00.698019+0000","modified":"2024-09-13T08:52:03.712285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8763":{"gid":8763,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2181881572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2181881572},{"type":"v1","addr":"172.21.15.44:6835","nonce":2181881572}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8763,"qdb_cluster":[8763]},"id":9}]} 2024-09-13T08:53:01.138 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 86 2024-09-13T08:53:01.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 -- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d78075110 msgr2=0x7f8d780775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d78075110 0x7f8d780775d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c1fb790 tx=0x7f8d8c009040 comp rx=0 tx=0).stop 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 -- 172.21.15.44:0/775087149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c0ced80 msgr2=0x7f8d9c0b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c0ced80 0x7f8d9c0b3770 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c480 tx=0x7f8d8400c940 comp rx=0 tx=0).stop 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 -- 172.21.15.44:0/775087149 shutdown_connections 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8d78075110 0x7f8d780775d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c15bea0 0x7f8d9c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c0ced80 0x7f8d9c0b3770 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 --2- 172.21.15.44:0/775087149 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8d9c0cdb30 0x7f8d9c14f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 -- 172.21.15.44:0/775087149 >> 172.21.15.44:0/775087149 conn(0x7f8d9c0d1dc0 msgr2=0x7f8d9c0b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 -- 172.21.15.44:0/775087149 shutdown_connections 2024-09-13T08:53:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.140+0000 7f8da2600640 1 -- 172.21.15.44:0/775087149 wait complete. 2024-09-13T08:53:01.160 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:53:01.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.304+0000 7f83ebe00640 1 -- 172.21.15.44:0/4163987287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83ec0cefb0 msgr2=0x7f83ec0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.304+0000 7f83ebe00640 1 --2- 172.21.15.44:0/4163987287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83ec0cefb0 0x7f83ec0cc5b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0b1640 tx=0x7f83d401c830 comp rx=0 tx=0).stop 2024-09-13T08:53:01.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.304+0000 7f83ebe00640 1 -- 172.21.15.44:0/4163987287 shutdown_connections 2024-09-13T08:53:01.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.304+0000 7f83ebe00640 1 --2- 172.21.15.44:0/4163987287 >> v2:172.21.15.129:3300/0 conn(0x7f83ec0ccaf0 0x7f83ec0b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.304+0000 7f83ebe00640 1 --2- 172.21.15.44:0/4163987287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83ec0cefb0 0x7f83ec0cc5b0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.304+0000 7f83ebe00640 1 --2- 172.21.15.44:0/4163987287 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83ec15f000 0x7f83ec0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.304+0000 7f83ebe00640 1 -- 172.21.15.44:0/4163987287 >> 172.21.15.44:0/4163987287 conn(0x7f83ec09aa10 msgr2=0x7f83ec09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:01.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.304+0000 7f83ebe00640 1 -- 172.21.15.44:0/4163987287 shutdown_connections 2024-09-13T08:53:01.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.304+0000 7f83ebe00640 1 -- 172.21.15.44:0/4163987287 wait complete. 2024-09-13T08:53:01.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83ebe00640 1 Processor -- start 2024-09-13T08:53:01.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83ebe00640 1 -- start start 2024-09-13T08:53:01.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83ebe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83ec0ccaf0 0x7f83ec1f9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:01.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83ebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83ec0cefb0 0x7f83ec1f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:01.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83ebe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83ec15f000 0x7f83ec0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:01.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83ebe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec15ff80 con 0x7f83ec0cefb0 2024-09-13T08:53:01.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83ebe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec15fe00 con 0x7f83ec0ccaf0 2024-09-13T08:53:01.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83ebe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f83ec160100 con 0x7f83ec15f000 2024-09-13T08:53:01.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83ec0cefb0 0x7f83ec1f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:01.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83ec0cefb0 0x7f83ec1f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54162/0 (socket says 172.21.15.44:54162) 2024-09-13T08:53:01.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e2000640 1 -- 172.21.15.44:0/260224692 learned_addr learned my addr 172.21.15.44:0/260224692 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:01.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83eb400640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83ec15f000 0x7f83ec0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:01.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83eaa00640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83ec0ccaf0 0x7f83ec1f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:01.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e2000640 1 -- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83ec15f000 msgr2=0x7f83ec0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e2000640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83ec15f000 0x7f83ec0af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e2000640 1 -- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83ec0ccaf0 msgr2=0x7f83ec1f9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e2000640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83ec0ccaf0 0x7f83ec1f9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e2000640 1 -- 172.21.15.44:0/260224692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec1f9a10 con 0x7f83ec0cefb0 2024-09-13T08:53:01.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83eaa00640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83ec0ccaf0 0x7f83ec1f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:01.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e2000640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83ec0cefb0 0x7f83ec1f6430 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f83d40b0040 tx=0x7f83d4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:01.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e3400640 1 -- 172.21.15.44:0/260224692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4004430 con 0x7f83ec0cefb0 2024-09-13T08:53:01.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e3400640 1 -- 172.21.15.44:0/260224692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83d4005140 con 0x7f83ec0cefb0 2024-09-13T08:53:01.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83e3400640 1 -- 172.21.15.44:0/260224692 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4005590 con 0x7f83ec0cefb0 2024-09-13T08:53:01.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83ebe00640 1 -- 172.21.15.44:0/260224692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ec1f9cf0 con 0x7f83ec0cefb0 2024-09-13T08:53:01.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.308+0000 7f83ebe00640 1 -- 172.21.15.44:0/260224692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ec1f5c30 con 0x7f83ec0cefb0 2024-09-13T08:53:01.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.312+0000 7f83ebe00640 1 -- 172.21.15.44:0/260224692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ec0cc5b0 con 0x7f83ec0cefb0 2024-09-13T08:53:01.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.312+0000 7f83e3400640 1 -- 172.21.15.44:0/260224692 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f83d40046d0 con 0x7f83ec0cefb0 2024-09-13T08:53:01.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.312+0000 7f83e3400640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83c8075140 0x7f83c8077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:01.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.312+0000 7f83e3400640 1 -- 172.21.15.44:0/260224692 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5877+0+0 (secure 0 0 0) 0x7f83d412d2b0 con 0x7f83ec0cefb0 2024-09-13T08:53:01.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.312+0000 7f83eaa00640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83c8075140 0x7f83c8077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:01.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.312+0000 7f83eaa00640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83c8075140 0x7f83c8077600 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f83dc006fd0 tx=0x7f83dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:01.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.316+0000 7f83e3400640 1 -- 172.21.15.44:0/260224692 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f83d40164b0 con 0x7f83ec0cefb0 2024-09-13T08:53:01.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.492+0000 7f83ebe00640 1 -- 172.21.15.44:0/260224692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83ec15c9d0 con 0x7f83ec0cefb0 2024-09-13T08:53:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.492+0000 7f83e3400640 1 -- 172.21.15.44:0/260224692 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16988 (secure 0 0 0) 0x7f83d40f9880 con 0x7f83ec0cefb0 2024-09-13T08:53:01.494 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:01.495 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":132,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:52:56.489969+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:51:58.420485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-13T08:51:59.669296+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":132,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 -- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83c8075140 msgr2=0x7f83c8077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83c8075140 0x7f83c8077600 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f83dc006fd0 tx=0x7f83dc008040 comp rx=0 tx=0).stop 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 -- 172.21.15.44:0/260224692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83ec0cefb0 msgr2=0x7f83ec1f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83ec0cefb0 0x7f83ec1f6430 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f83d40b0040 tx=0x7f83d4004040 comp rx=0 tx=0).stop 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 -- 172.21.15.44:0/260224692 shutdown_connections 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f83c8075140 0x7f83c8077600 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f83ec15f000 0x7f83ec0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83ec0cefb0 0x7f83ec1f6430 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 --2- 172.21.15.44:0/260224692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83ec0ccaf0 0x7f83ec1f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 -- 172.21.15.44:0/260224692 >> 172.21.15.44:0/260224692 conn(0x7f83ec09aa10 msgr2=0x7f83ec15c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 -- 172.21.15.44:0/260224692 shutdown_connections 2024-09-13T08:53:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.496+0000 7f83ebe00640 1 -- 172.21.15.44:0/260224692 wait complete. 2024-09-13T08:53:01.518 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:53:01.518 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:53:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.664+0000 7f70d6c00640 1 -- 172.21.15.44:0/3448902517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70d00cdb30 msgr2=0x7f70d00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.664+0000 7f70d6c00640 1 --2- 172.21.15.44:0/3448902517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70d00cdb30 0x7f70d00cebb0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009900 tx=0x7f70bc01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.664+0000 7f70d6c00640 1 -- 172.21.15.44:0/3448902517 shutdown_connections 2024-09-13T08:53:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.664+0000 7f70d6c00640 1 --2- 172.21.15.44:0/3448902517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70d00cdb30 0x7f70d00cebb0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.664+0000 7f70d6c00640 1 --2- 172.21.15.44:0/3448902517 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70d015bea0 0x7f70d00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.664+0000 7f70d6c00640 1 -- 172.21.15.44:0/3448902517 >> 172.21.15.44:0/3448902517 conn(0x7f70d00d1dc0 msgr2=0x7f70d00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.664+0000 7f70d6c00640 1 -- 172.21.15.44:0/3448902517 shutdown_connections 2024-09-13T08:53:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.664+0000 7f70d6c00640 1 -- 172.21.15.44:0/3448902517 wait complete. 2024-09-13T08:53:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.664+0000 7f70d6c00640 1 Processor -- start 2024-09-13T08:53:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.664+0000 7f70d6c00640 1 -- start start 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70d6c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70d00cdb30 0x7f70d00b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70d6c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70d00d28e0 0x7f70d01fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70d6c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d015bea0 0x7f70d0148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70d6c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0169050 con 0x7f70d00cdb30 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70d6c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0168ed0 con 0x7f70d015bea0 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70d6c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f70d01691d0 con 0x7f70d00d28e0 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70d00cdb30 0x7f70d00b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70d00cdb30 0x7f70d00b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54180/0 (socket says 172.21.15.44:54180) 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cfe00640 1 -- 172.21.15.44:0/3290381938 learned_addr learned my addr 172.21.15.44:0/3290381938 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cf400640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70d00d28e0 0x7f70d01fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70d4c00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d015bea0 0x7f70d0148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cfe00640 1 -- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70d00d28e0 msgr2=0x7f70d01fbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cfe00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70d00d28e0 0x7f70d01fbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cfe00640 1 -- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d015bea0 msgr2=0x7f70d0148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cfe00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d015bea0 0x7f70d0148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cfe00640 1 -- 172.21.15.44:0/3290381938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d01fc040 con 0x7f70d00cdb30 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cf400640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70d00d28e0 0x7f70d01fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70cfe00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70d00cdb30 0x7f70d00b1340 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f70c00058d0 tx=0x7f70c000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70c7e00640 1 -- 172.21.15.44:0/3290381938 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0017070 con 0x7f70d00cdb30 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70c7e00640 1 -- 172.21.15.44:0/3290381938 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70c0009cc0 con 0x7f70d00cdb30 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70c7e00640 1 -- 172.21.15.44:0/3290381938 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c00106c0 con 0x7f70d00cdb30 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70d6c00640 1 -- 172.21.15.44:0/3290381938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d006f640 con 0x7f70d00cdb30 2024-09-13T08:53:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70d6c00640 1 -- 172.21.15.44:0/3290381938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d006fc00 con 0x7f70d00cdb30 2024-09-13T08:53:01.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70d6c00640 1 -- 172.21.15.44:0/3290381938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d00cc9d0 con 0x7f70d00cdb30 2024-09-13T08:53:01.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.668+0000 7f70c7e00640 1 -- 172.21.15.44:0/3290381938 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f70c001a070 con 0x7f70d00cdb30 2024-09-13T08:53:01.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.672+0000 7f70c7e00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70b0075110 0x7f70b00775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:01.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.672+0000 7f70c7e00640 1 -- 172.21.15.44:0/3290381938 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5877+0+0 (secure 0 0 0) 0x7f70c00930f0 con 0x7f70d00cdb30 2024-09-13T08:53:01.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.672+0000 7f70cf400640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70b0075110 0x7f70b00775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:01.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.672+0000 7f70cf400640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70b0075110 0x7f70b00775d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0af000 tx=0x7f70bc01b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:01.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.676+0000 7f70c7e00640 1 -- 172.21.15.44:0/3290381938 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f70d01fc040 con 0x7f70d00cdb30 2024-09-13T08:53:01.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.888+0000 7f70d6c00640 1 -- 172.21.15.44:0/3290381938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70d00cbd20 con 0x7f70d00cdb30 2024-09-13T08:53:01.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.888+0000 7f70c7e00640 1 -- 172.21.15.44:0/3290381938 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5228 (secure 0 0 0) 0x7f70c005fda0 con 0x7f70d00cdb30 2024-09-13T08:53:01.890 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:01.890 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":86,"btime":"2024-09-13T08:52:09:337186+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/771921648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":771921648},{"type":"v1","addr":"172.21.15.129:6836","nonce":771921648}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8104,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1464443005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1464443005},{"type":"v1","addr":"172.21.15.129:6837","nonce":1464443005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2601537201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2601537201},{"type":"v1","addr":"172.21.15.44:6833","nonce":2601537201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:52:00.698019+0000","modified":"2024-09-13T08:52:03.712285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8763":{"gid":8763,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2181881572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2181881572},{"type":"v1","addr":"172.21.15.44:6835","nonce":2181881572}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8763,"qdb_cluster":[8763]},"id":9}]} 2024-09-13T08:53:01.891 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 86 2024-09-13T08:53:01.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 -- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70b0075110 msgr2=0x7f70b00775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70b0075110 0x7f70b00775d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0af000 tx=0x7f70bc01b120 comp rx=0 tx=0).stop 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 -- 172.21.15.44:0/3290381938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70d00cdb30 msgr2=0x7f70d00b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70d00cdb30 0x7f70d00b1340 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f70c00058d0 tx=0x7f70c000dde0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 -- 172.21.15.44:0/3290381938 shutdown_connections 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f70b0075110 0x7f70b00775d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d015bea0 0x7f70d0148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70d00d28e0 0x7f70d01fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 --2- 172.21.15.44:0/3290381938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70d00cdb30 0x7f70d00b1340 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 -- 172.21.15.44:0/3290381938 >> 172.21.15.44:0/3290381938 conn(0x7f70d00d1dc0 msgr2=0x7f70d00af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 -- 172.21.15.44:0/3290381938 shutdown_connections 2024-09-13T08:53:01.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:01.892+0000 7f70d6c00640 1 -- 172.21.15.44:0/3290381938 wait complete. 2024-09-13T08:53:01.912 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:53:02.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.056+0000 7fa87a800640 1 -- 172.21.15.44:0/819556173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8740cdae0 msgr2=0x7fa8740cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.056+0000 7fa87a800640 1 --2- 172.21.15.44:0/819556173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8740cdae0 0x7fa8740cdf00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009900 tx=0x7fa85c01c850 comp rx=0 tx=0).stop 2024-09-13T08:53:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.056+0000 7fa87a800640 1 -- 172.21.15.44:0/819556173 shutdown_connections 2024-09-13T08:53:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.056+0000 7fa87a800640 1 --2- 172.21.15.44:0/819556173 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa87415f900 0x7fa87415fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.056+0000 7fa87a800640 1 --2- 172.21.15.44:0/819556173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8740cdae0 0x7fa8740cdf00 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.056+0000 7fa87a800640 1 -- 172.21.15.44:0/819556173 >> 172.21.15.44:0/819556173 conn(0x7fa8740b18f0 msgr2=0x7fa8740b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.056+0000 7fa87a800640 1 -- 172.21.15.44:0/819556173 shutdown_connections 2024-09-13T08:53:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.056+0000 7fa87a800640 1 -- 172.21.15.44:0/819556173 wait complete. 2024-09-13T08:53:02.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa87a800640 1 Processor -- start 2024-09-13T08:53:02.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa87a800640 1 -- start start 2024-09-13T08:53:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa87a800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8740cdae0 0x7fa874155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa87a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa87415f300 0x7fa8741664b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa87a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa87415f900 0x7fa8741443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa87a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa874169030 con 0x7fa87415f900 2024-09-13T08:53:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa87a800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa874168eb0 con 0x7fa87415f300 2024-09-13T08:53:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa87a800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa8741691b0 con 0x7fa8740cdae0 2024-09-13T08:53:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa873e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa87415f900 0x7fa8741443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa873e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa87415f900 0x7fa8741443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54196/0 (socket says 172.21.15.44:54196) 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa873e00640 1 -- 172.21.15.44:0/3437133855 learned_addr learned my addr 172.21.15.44:0/3437133855 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa873e00640 1 -- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8740cdae0 msgr2=0x7fa874155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa873e00640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8740cdae0 0x7fa874155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa873e00640 1 -- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa87415f300 msgr2=0x7fa8741664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa873e00640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa87415f300 0x7fa8741664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa873e00640 1 -- 172.21.15.44:0/3437133855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8741fb1a0 con 0x7fa87415f900 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa873e00640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa87415f900 0x7fa8741443f0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00e970 tx=0x7fa86c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa86be00640 1 -- 172.21.15.44:0/3437133855 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c004d30 con 0x7fa87415f900 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa86be00640 1 -- 172.21.15.44:0/3437133855 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa86c00cb20 con 0x7fa87415f900 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa86be00640 1 -- 172.21.15.44:0/3437133855 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c005020 con 0x7fa87415f900 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa87a800640 1 -- 172.21.15.44:0/3437133855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8741fb480 con 0x7fa87415f900 2024-09-13T08:53:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.060+0000 7fa87a800640 1 -- 172.21.15.44:0/3437133855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8741fba40 con 0x7fa87415f900 2024-09-13T08:53:02.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.064+0000 7fa87a800640 1 -- 172.21.15.44:0/3437133855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa874069410 con 0x7fa87415f900 2024-09-13T08:53:02.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.064+0000 7fa86be00640 1 -- 172.21.15.44:0/3437133855 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7fa86c0051c0 con 0x7fa87415f900 2024-09-13T08:53:02.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.064+0000 7fa86be00640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa850075110 0x7fa8500775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.064+0000 7fa86be00640 1 -- 172.21.15.44:0/3437133855 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5877+0+0 (secure 0 0 0) 0x7fa86c0939f0 con 0x7fa87415f900 2024-09-13T08:53:02.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.064+0000 7fa873400640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa850075110 0x7fa8500775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:02.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.068+0000 7fa873400640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa850075110 0x7fa8500775d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa864000f70 tx=0x7fa8640079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:02.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.068+0000 7fa86be00640 1 -- 172.21.15.44:0/3437133855 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa86c016030 con 0x7fa87415f900 2024-09-13T08:53:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 -- 172.21.15.44:0/3437133855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa87415fd60 con 0x7fa87415f900 2024-09-13T08:53:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa86be00640 1 -- 172.21.15.44:0/3437133855 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5228 (secure 0 0 0) 0x7fa86c0606a0 con 0x7fa87415f900 2024-09-13T08:53:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:02.280 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":86,"btime":"2024-09-13T08:52:09:337186+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/771921648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":771921648},{"type":"v1","addr":"172.21.15.129:6836","nonce":771921648}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8104,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1464443005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1464443005},{"type":"v1","addr":"172.21.15.129:6837","nonce":1464443005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2601537201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2601537201},{"type":"v1","addr":"172.21.15.44:6833","nonce":2601537201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:52:00.698019+0000","modified":"2024-09-13T08:52:03.712285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8763":{"gid":8763,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2181881572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2181881572},{"type":"v1","addr":"172.21.15.44:6835","nonce":2181881572}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8763,"qdb_cluster":[8763]},"id":9}]} 2024-09-13T08:53:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 86 2024-09-13T08:53:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 -- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa850075110 msgr2=0x7fa8500775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa850075110 0x7fa8500775d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa864000f70 tx=0x7fa8640079c0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 -- 172.21.15.44:0/3437133855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa87415f900 msgr2=0x7fa8741443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa87415f900 0x7fa8741443f0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00e970 tx=0x7fa86c00ee30 comp rx=0 tx=0).stop 2024-09-13T08:53:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 -- 172.21.15.44:0/3437133855 shutdown_connections 2024-09-13T08:53:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa850075110 0x7fa8500775d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa87415f900 0x7fa8741443f0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa87415f300 0x7fa8741664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 --2- 172.21.15.44:0/3437133855 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8740cdae0 0x7fa874155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 -- 172.21.15.44:0/3437133855 >> 172.21.15.44:0/3437133855 conn(0x7fa8740b18f0 msgr2=0x7fa8740b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 -- 172.21.15.44:0/3437133855 shutdown_connections 2024-09-13T08:53:02.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.280+0000 7fa87a800640 1 -- 172.21.15.44:0/3437133855 wait complete. 2024-09-13T08:53:02.303 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:53:02.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.448+0000 7f7626600640 1 -- 172.21.15.44:0/4276369077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762015bea0 msgr2=0x7f76200ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.448+0000 7f7626600640 1 --2- 172.21.15.44:0/4276369077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762015bea0 0x7f76200ce670 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f7610009900 tx=0x7f761001c8e0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.448+0000 7f7626600640 1 -- 172.21.15.44:0/4276369077 shutdown_connections 2024-09-13T08:53:02.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.448+0000 7f7626600640 1 --2- 172.21.15.44:0/4276369077 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f76200cdb30 0x7f76200cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.448+0000 7f7626600640 1 --2- 172.21.15.44:0/4276369077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762015bea0 0x7f76200ce670 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.448+0000 7f7626600640 1 -- 172.21.15.44:0/4276369077 >> 172.21.15.44:0/4276369077 conn(0x7f76200d1dc0 msgr2=0x7f76200d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:02.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.448+0000 7f7626600640 1 -- 172.21.15.44:0/4276369077 shutdown_connections 2024-09-13T08:53:02.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.448+0000 7f7626600640 1 -- 172.21.15.44:0/4276369077 wait complete. 2024-09-13T08:53:02.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.448+0000 7f7626600640 1 Processor -- start 2024-09-13T08:53:02.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7626600640 1 -- start start 2024-09-13T08:53:02.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7626600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f76200cdb30 0x7f76200cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7626600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76200d28e0 0x7f76201f8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7626600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762015bea0 0x7f762015a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7626600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7620168fc0 con 0x7f762015bea0 2024-09-13T08:53:02.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7626600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7620168e40 con 0x7f76200d28e0 2024-09-13T08:53:02.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7626600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7620169140 con 0x7f76200cdb30 2024-09-13T08:53:02.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f761fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762015bea0 0x7f762015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:02.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f761fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762015bea0 0x7f762015a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41886/0 (socket says 172.21.15.44:41886) 2024-09-13T08:53:02.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7616a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76200d28e0 0x7f76201f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:02.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f761f400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f76200cdb30 0x7f76200cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:02.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f761fe00640 1 -- 172.21.15.44:0/1683288229 learned_addr learned my addr 172.21.15.44:0/1683288229 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:02.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f761fe00640 1 -- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f76200cdb30 msgr2=0x7f76200cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f761fe00640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f76200cdb30 0x7f76200cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f761fe00640 1 -- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76200d28e0 msgr2=0x7f76201f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f761fe00640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76200d28e0 0x7f76201f8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f761fe00640 1 -- 172.21.15.44:0/1683288229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76200b6d80 con 0x7f762015bea0 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f761fe00640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762015bea0 0x7f762015a810 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f761800c7c0 tx=0x7f761800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7617e00640 1 -- 172.21.15.44:0/1683288229 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618003b80 con 0x7f762015bea0 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7617e00640 1 -- 172.21.15.44:0/1683288229 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7618003d20 con 0x7f762015bea0 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7617e00640 1 -- 172.21.15.44:0/1683288229 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761800aab0 con 0x7f762015bea0 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7626600640 1 -- 172.21.15.44:0/1683288229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76200b7060 con 0x7f762015bea0 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7626600640 1 -- 172.21.15.44:0/1683288229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76200b7620 con 0x7f762015bea0 2024-09-13T08:53:02.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.452+0000 7f7626600640 1 -- 172.21.15.44:0/1683288229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76200cc9d0 con 0x7f762015bea0 2024-09-13T08:53:02.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.456+0000 7f7617e00640 1 -- 172.21.15.44:0/1683288229 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f761800ac50 con 0x7f762015bea0 2024-09-13T08:53:02.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.456+0000 7f7617e00640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75fc075110 0x7f75fc0775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.456+0000 7f7617e00640 1 -- 172.21.15.44:0/1683288229 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5877+0+0 (secure 0 0 0) 0x7f7618092cf0 con 0x7f762015bea0 2024-09-13T08:53:02.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.456+0000 7f761f400640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75fc075110 0x7f75fc0775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:02.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.456+0000 7f761f400640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75fc075110 0x7f75fc0775d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f76100af000 tx=0x7f761001b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.460+0000 7f7617e00640 1 -- 172.21.15.44:0/1683288229 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f761805f350 con 0x7f762015bea0 2024-09-13T08:53:02.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.640+0000 7f7626600640 1 -- 172.21.15.44:0/1683288229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f762006e900 con 0x7f762015bea0 2024-09-13T08:53:02.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.640+0000 7f7617e00640 1 -- 172.21.15.44:0/1683288229 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16988 (secure 0 0 0) 0x7f7618063d80 con 0x7f762015bea0 2024-09-13T08:53:02.642 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:02.643 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":132,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:52:56.489969+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:51:58.420485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-13T08:51:59.669296+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":132,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:53:02.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 -- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75fc075110 msgr2=0x7f75fc0775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75fc075110 0x7f75fc0775d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f76100af000 tx=0x7f761001b120 comp rx=0 tx=0).stop 2024-09-13T08:53:02.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 -- 172.21.15.44:0/1683288229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762015bea0 msgr2=0x7f762015a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762015bea0 0x7f762015a810 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f761800c7c0 tx=0x7f761800cc80 comp rx=0 tx=0).stop 2024-09-13T08:53:02.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 -- 172.21.15.44:0/1683288229 shutdown_connections 2024-09-13T08:53:02.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f75fc075110 0x7f75fc0775d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762015bea0 0x7f762015a810 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76200d28e0 0x7f76201f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 --2- 172.21.15.44:0/1683288229 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f76200cdb30 0x7f76200cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 -- 172.21.15.44:0/1683288229 >> 172.21.15.44:0/1683288229 conn(0x7f76200d1dc0 msgr2=0x7f7620071490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:02.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 -- 172.21.15.44:0/1683288229 shutdown_connections 2024-09-13T08:53:02.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.644+0000 7f7626600640 1 -- 172.21.15.44:0/1683288229 wait complete. 2024-09-13T08:53:02.665 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T08:53:02.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 -- 172.21.15.44:0/3051037748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11580ce800 msgr2=0x7f115806cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 --2- 172.21.15.44:0/3051037748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11580ce800 0x7f115806cc20 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f114c009930 tx=0x7f114c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 -- 172.21.15.44:0/3051037748 shutdown_connections 2024-09-13T08:53:02.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 --2- 172.21.15.44:0/3051037748 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1158073170 0x7f115806d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 --2- 172.21.15.44:0/3051037748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11580ce800 0x7f115806cc20 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 -- 172.21.15.44:0/3051037748 >> 172.21.15.44:0/3051037748 conn(0x7f11580b2240 msgr2=0x7f11580b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:02.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 -- 172.21.15.44:0/3051037748 shutdown_connections 2024-09-13T08:53:02.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 -- 172.21.15.44:0/3051037748 wait complete. 2024-09-13T08:53:02.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 Processor -- start 2024-09-13T08:53:02.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 -- start start 2024-09-13T08:53:02.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1158073170 0x7f11581f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11580ce800 0x7f115815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1158143fe0 0x7f11581401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f11581682e0 con 0x7f11580ce800 2024-09-13T08:53:02.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1158168160 con 0x7f1158143fe0 2024-09-13T08:53:02.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f1160200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1158168460 con 0x7f1158073170 2024-09-13T08:53:02.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11580ce800 0x7f115815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:02.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11580ce800 0x7f115815dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41918/0 (socket says 172.21.15.44:41918) 2024-09-13T08:53:02.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115ce00640 1 -- 172.21.15.44:0/108867493 learned_addr learned my addr 172.21.15.44:0/108867493 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:02.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115d800640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1158073170 0x7f11581f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:02.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115e200640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1158143fe0 0x7f11581401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:02.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115ce00640 1 -- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1158073170 msgr2=0x7f11581f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115ce00640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1158073170 0x7f11581f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115ce00640 1 -- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1158143fe0 msgr2=0x7f11581401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:02.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115ce00640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1158143fe0 0x7f11581401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:02.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115ce00640 1 -- 172.21.15.44:0/108867493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115815e140 con 0x7f11580ce800 2024-09-13T08:53:02.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.812+0000 7f115e200640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1158143fe0 0x7f11581401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:02.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.816+0000 7f115ce00640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11580ce800 0x7f115815dc00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f114400b4d0 tx=0x7f114400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:02.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.816+0000 7f1155600640 1 -- 172.21.15.44:0/108867493 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144014070 con 0x7f11580ce800 2024-09-13T08:53:02.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.816+0000 7f1155600640 1 -- 172.21.15.44:0/108867493 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1144005a20 con 0x7f11580ce800 2024-09-13T08:53:02.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.816+0000 7f1155600640 1 -- 172.21.15.44:0/108867493 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144010610 con 0x7f11580ce800 2024-09-13T08:53:02.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.816+0000 7f1160200640 1 -- 172.21.15.44:0/108867493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11581fbac0 con 0x7f11580ce800 2024-09-13T08:53:02.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.816+0000 7f1160200640 1 -- 172.21.15.44:0/108867493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11581fc080 con 0x7f11580ce800 2024-09-13T08:53:02.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.816+0000 7f1160200640 1 -- 172.21.15.44:0/108867493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f115815e8b0 con 0x7f11580ce800 2024-09-13T08:53:02.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.816+0000 7f1155600640 1 -- 172.21.15.44:0/108867493 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f11440107b0 con 0x7f11580ce800 2024-09-13T08:53:02.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.816+0000 7f1155600640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1138075110 0x7f11380775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:02.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.816+0000 7f1155600640 1 -- 172.21.15.44:0/108867493 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5877+0+0 (secure 0 0 0) 0x7f11440928b0 con 0x7f11580ce800 2024-09-13T08:53:02.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.820+0000 7f115d800640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1138075110 0x7f11380775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:02.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.820+0000 7f115d800640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1138075110 0x7f11380775d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f114c0af000 tx=0x7f114c01b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:02.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:02.820+0000 7f1155600640 1 -- 172.21.15.44:0/108867493 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f114405f560 con 0x7f11580ce800 2024-09-13T08:53:03.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.032+0000 7f1160200640 1 -- 172.21.15.44:0/108867493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f11580af920 con 0x7f11580ce800 2024-09-13T08:53:03.033 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T08:53:03.032+0000 7f03b3c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T08:53:03.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1155600640 1 -- 172.21.15.44:0/108867493 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) ==== 161+0+0 (secure 0 0 0) 0x7f114405ee60 con 0x7f11580ce800 2024-09-13T08:53:03.035 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T08:53:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1160200640 1 -- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1138075110 msgr2=0x7f11380775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1160200640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1138075110 0x7f11380775d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f114c0af000 tx=0x7f114c01b120 comp rx=0 tx=0).stop 2024-09-13T08:53:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1160200640 1 -- 172.21.15.44:0/108867493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11580ce800 msgr2=0x7f115815dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1160200640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11580ce800 0x7f115815dc00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f114400b4d0 tx=0x7f114400b990 comp rx=0 tx=0).stop 2024-09-13T08:53:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1160200640 1 -- 172.21.15.44:0/108867493 shutdown_connections 2024-09-13T08:53:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1160200640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1138075110 0x7f11380775d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1160200640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1158143fe0 0x7f11581401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:03.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1160200640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11580ce800 0x7f115815dc00 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1160200640 1 --2- 172.21.15.44:0/108867493 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1158073170 0x7f11581f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.036+0000 7f1160200640 1 -- 172.21.15.44:0/108867493 >> 172.21.15.44:0/108867493 conn(0x7f11580b2240 msgr2=0x7f1158165000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.040+0000 7f1160200640 1 -- 172.21.15.44:0/108867493 shutdown_connections 2024-09-13T08:53:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.040+0000 7f1160200640 1 -- 172.21.15.44:0/108867493 wait complete. 2024-09-13T08:53:03.061 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T08:53:03.105 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:53:03.123 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:53:03.126 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:53:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.212+0000 7f1a69200640 1 -- 172.21.15.44:0/4015913739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a6415bf70 msgr2=0x7f1a640d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.212+0000 7f1a69200640 1 --2- 172.21.15.44:0/4015913739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a6415bf70 0x7f1a640d28f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009900 tx=0x7f1a5401c850 comp rx=0 tx=0).stop 2024-09-13T08:53:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.212+0000 7f1a69200640 1 -- 172.21.15.44:0/4015913739 shutdown_connections 2024-09-13T08:53:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.212+0000 7f1a69200640 1 --2- 172.21.15.44:0/4015913739 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a640d0950 0x7f1a640aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.212+0000 7f1a69200640 1 --2- 172.21.15.44:0/4015913739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a6415bf70 0x7f1a640d28f0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.212+0000 7f1a69200640 1 -- 172.21.15.44:0/4015913739 >> 172.21.15.44:0/4015913739 conn(0x7f1a640d1e00 msgr2=0x7f1a640d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.212+0000 7f1a69200640 1 -- 172.21.15.44:0/4015913739 shutdown_connections 2024-09-13T08:53:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.212+0000 7f1a69200640 1 -- 172.21.15.44:0/4015913739 wait complete. 2024-09-13T08:53:03.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a69200640 1 Processor -- start 2024-09-13T08:53:03.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a69200640 1 -- start start 2024-09-13T08:53:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a69200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a640b0f50 0x7f1a640b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a69200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a640d0950 0x7f1a64150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a69200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a6415bf70 0x7f1a64156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a69200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1a64168d40 con 0x7f1a640b0f50 2024-09-13T08:53:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a69200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a64168bc0 con 0x7f1a6415bf70 2024-09-13T08:53:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a69200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1a64168ec0 con 0x7f1a640d0950 2024-09-13T08:53:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a61e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a640b0f50 0x7f1a640b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a61e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a640b0f50 0x7f1a640b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41960/0 (socket says 172.21.15.44:41960) 2024-09-13T08:53:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a61e00640 1 -- 172.21.15.44:0/4195890689 learned_addr learned my addr 172.21.15.44:0/4195890689 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a5be00640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a640d0950 0x7f1a64150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a62800640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a6415bf70 0x7f1a64156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a61e00640 1 -- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a640d0950 msgr2=0x7f1a64150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a61e00640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a640d0950 0x7f1a64150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a61e00640 1 -- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a6415bf70 msgr2=0x7f1a64156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a61e00640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a6415bf70 0x7f1a64156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a61e00640 1 -- 172.21.15.44:0/4195890689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a641fb350 con 0x7f1a640b0f50 2024-09-13T08:53:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a62800640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a6415bf70 0x7f1a64156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a5be00640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a640d0950 0x7f1a64150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:03.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a61e00640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a640b0f50 0x7f1a640b58f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009850 tx=0x7f1a540a5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:03.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a5a000640 1 -- 172.21.15.44:0/4195890689 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a540b2040 con 0x7f1a640b0f50 2024-09-13T08:53:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a5a000640 1 -- 172.21.15.44:0/4195890689 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a540042c0 con 0x7f1a640b0f50 2024-09-13T08:53:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a5a000640 1 -- 172.21.15.44:0/4195890689 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54005030 con 0x7f1a640b0f50 2024-09-13T08:53:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a69200640 1 -- 172.21.15.44:0/4195890689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a641fb630 con 0x7f1a640b0f50 2024-09-13T08:53:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.216+0000 7f1a69200640 1 -- 172.21.15.44:0/4195890689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a641fbbf0 con 0x7f1a640b0f50 2024-09-13T08:53:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.220+0000 7f1a69200640 1 -- 172.21.15.44:0/4195890689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a640aeb50 con 0x7f1a640b0f50 2024-09-13T08:53:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.220+0000 7f1a5a000640 1 -- 172.21.15.44:0/4195890689 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f1a540051f0 con 0x7f1a640b0f50 2024-09-13T08:53:03.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.220+0000 7f1a5a000640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a40075140 0x7f1a40077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:03.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.220+0000 7f1a5a000640 1 -- 172.21.15.44:0/4195890689 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5935+0+0 (secure 0 0 0) 0x7f1a5412c9e0 con 0x7f1a640b0f50 2024-09-13T08:53:03.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.224+0000 7f1a5be00640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a40075140 0x7f1a40077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:03.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.224+0000 7f1a5be00640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a40075140 0x7f1a40077600 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1a640aee10 tx=0x7f1a4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:03.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.224+0000 7f1a5a000640 1 -- 172.21.15.44:0/4195890689 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1a540164d0 con 0x7f1a640b0f50 2024-09-13T08:53:03.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:03.440+0000 7f1a69200640 1 -- 172.21.15.44:0/4195890689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1a6415b710 con 0x7f1a640b0f50 2024-09-13T08:53:04.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.036+0000 7f1a5a000640 1 -- 172.21.15.44:0/4195890689 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v88) ==== 104+0+0 (secure 0 0 0) 0x7f1a540f9690 con 0x7f1a640b0f50 2024-09-13T08:53:04.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 -- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a40075140 msgr2=0x7f1a40077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:04.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a40075140 0x7f1a40077600 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1a640aee10 tx=0x7f1a4c009040 comp rx=0 tx=0).stop 2024-09-13T08:53:04.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 -- 172.21.15.44:0/4195890689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a640b0f50 msgr2=0x7f1a640b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:04.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a640b0f50 0x7f1a640b58f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009850 tx=0x7f1a540a5530 comp rx=0 tx=0).stop 2024-09-13T08:53:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 -- 172.21.15.44:0/4195890689 shutdown_connections 2024-09-13T08:53:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1a40075140 0x7f1a40077600 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a6415bf70 0x7f1a64156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1a640d0950 0x7f1a64150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 --2- 172.21.15.44:0/4195890689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a640b0f50 0x7f1a640b58f0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 -- 172.21.15.44:0/4195890689 >> 172.21.15.44:0/4195890689 conn(0x7f1a640d1e00 msgr2=0x7f1a640b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 -- 172.21.15.44:0/4195890689 shutdown_connections 2024-09-13T08:53:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.040+0000 7f1a69200640 1 -- 172.21.15.44:0/4195890689 wait complete. 2024-09-13T08:53:04.064 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T08:53:04.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 -- 172.21.15.44:0/946320465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed180734f0 msgr2=0x7fed1815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:04.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 --2- 172.21.15.44:0/946320465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed180734f0 0x7fed1815a9d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fed08009840 tx=0x7fed0801c760 comp rx=0 tx=0).stop 2024-09-13T08:53:04.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 -- 172.21.15.44:0/946320465 shutdown_connections 2024-09-13T08:53:04.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 --2- 172.21.15.44:0/946320465 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed180cea60 0x7fed1806eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:04.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 --2- 172.21.15.44:0/946320465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed180734f0 0x7fed1815a9d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:04.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 -- 172.21.15.44:0/946320465 >> 172.21.15.44:0/946320465 conn(0x7fed180cfbc0 msgr2=0x7fed180cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:04.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 -- 172.21.15.44:0/946320465 shutdown_connections 2024-09-13T08:53:04.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 -- 172.21.15.44:0/946320465 wait complete. 2024-09-13T08:53:04.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 Processor -- start 2024-09-13T08:53:04.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 -- start start 2024-09-13T08:53:04.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed180734f0 0x7fed18145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:04.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed180cea60 0x7fed1814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed180cf440 0x7fed1809cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed1806faf0 con 0x7fed180734f0 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed1806f970 con 0x7fed180cea60 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed1ca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fed1806fc70 con 0x7fed180cf440 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed15e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed180734f0 0x7fed18145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed15e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed180734f0 0x7fed18145060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41974/0 (socket says 172.21.15.44:41974) 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed15e00640 1 -- 172.21.15.44:0/596150389 learned_addr learned my addr 172.21.15.44:0/596150389 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed15e00640 1 -- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed180cf440 msgr2=0x7fed1809cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed15e00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed180cf440 0x7fed1809cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed15e00640 1 -- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed180cea60 msgr2=0x7fed1814dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed15e00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed180cea60 0x7fed1814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed15e00640 1 -- 172.21.15.44:0/596150389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed180adf10 con 0x7fed180734f0 2024-09-13T08:53:04.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.264+0000 7fed15e00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed180734f0 0x7fed18145060 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fed08009790 tx=0x7fed08005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:04.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.268+0000 7fed0e000640 1 -- 172.21.15.44:0/596150389 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed080b2040 con 0x7fed180734f0 2024-09-13T08:53:04.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.268+0000 7fed0e000640 1 -- 172.21.15.44:0/596150389 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed08004320 con 0x7fed180734f0 2024-09-13T08:53:04.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.268+0000 7fed0e000640 1 -- 172.21.15.44:0/596150389 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed080044f0 con 0x7fed180734f0 2024-09-13T08:53:04.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.268+0000 7fed1ca00640 1 -- 172.21.15.44:0/596150389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed18152f10 con 0x7fed180734f0 2024-09-13T08:53:04.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.268+0000 7fed1ca00640 1 -- 172.21.15.44:0/596150389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed181534d0 con 0x7fed180734f0 2024-09-13T08:53:04.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.268+0000 7fed1ca00640 1 -- 172.21.15.44:0/596150389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed18069410 con 0x7fed180734f0 2024-09-13T08:53:04.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.268+0000 7fed0e000640 1 -- 172.21.15.44:0/596150389 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7fed080bd060 con 0x7fed180734f0 2024-09-13T08:53:04.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.268+0000 7fed0e000640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf4075110 0x7fecf40775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:04.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.268+0000 7fed0e000640 1 -- 172.21.15.44:0/596150389 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5935+0+0 (secure 0 0 0) 0x7fed0812c720 con 0x7fed180734f0 2024-09-13T08:53:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.268+0000 7fed0fe00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf4075110 0x7fecf40775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.272+0000 7fed0fe00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf4075110 0x7fecf40775d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fed000046c0 tx=0x7fed00007910 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:04.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.272+0000 7fed0e000640 1 -- 172.21.15.44:0/596150389 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fed08016410 con 0x7fed180734f0 2024-09-13T08:53:04.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:04.452+0000 7fed1ca00640 1 -- 172.21.15.44:0/596150389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed1815a9d0 con 0x7fed180734f0 2024-09-13T08:53:05.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed0e000640 1 -- 172.21.15.44:0/596150389 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v135) ==== 181+0+0 (secure 0 0 0) 0x7fed080f93d0 con 0x7fed180734f0 2024-09-13T08:53:05.216 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T08:53:05.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 -- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf4075110 msgr2=0x7fecf40775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf4075110 0x7fecf40775d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fed000046c0 tx=0x7fed00007910 comp rx=0 tx=0).stop 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 -- 172.21.15.44:0/596150389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed180734f0 msgr2=0x7fed18145060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed180734f0 0x7fed18145060 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fed08009790 tx=0x7fed08005dd0 comp rx=0 tx=0).stop 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 -- 172.21.15.44:0/596150389 shutdown_connections 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fecf4075110 0x7fecf40775d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fed180cf440 0x7fed1809cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed180cea60 0x7fed1814dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 --2- 172.21.15.44:0/596150389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed180734f0 0x7fed18145060 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 -- 172.21.15.44:0/596150389 >> 172.21.15.44:0/596150389 conn(0x7fed180cfbc0 msgr2=0x7fed180ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 -- 172.21.15.44:0/596150389 shutdown_connections 2024-09-13T08:53:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.216+0000 7fed1ca00640 1 -- 172.21.15.44:0/596150389 wait complete. 2024-09-13T08:53:05.246 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T08:53:05.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 -- 172.21.15.44:0/3201647940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f721415c120 msgr2=0x7f721415c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:05.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 --2- 172.21.15.44:0/3201647940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f721415c120 0x7f721415c560 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009840 tx=0x7f71fc01c820 comp rx=0 tx=0).stop 2024-09-13T08:53:05.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 -- 172.21.15.44:0/3201647940 shutdown_connections 2024-09-13T08:53:05.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 --2- 172.21.15.44:0/3201647940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f721415c120 0x7f721415c560 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:05.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 --2- 172.21.15.44:0/3201647940 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f721415bd40 0x7f7214073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:05.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 -- 172.21.15.44:0/3201647940 >> 172.21.15.44:0/3201647940 conn(0x7f721406dd30 msgr2=0x7f721406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:05.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 -- 172.21.15.44:0/3201647940 shutdown_connections 2024-09-13T08:53:05.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 -- 172.21.15.44:0/3201647940 wait complete. 2024-09-13T08:53:05.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 Processor -- start 2024-09-13T08:53:05.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 -- start start 2024-09-13T08:53:05.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f721406cc20 0x7f7214150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721415bd40 0x7f72141f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f721415c120 0x7f72141f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72141695c0 con 0x7f721415bd40 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7214169440 con 0x7f721406cc20 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f7219000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7214169740 con 0x7f721415c120 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.404+0000 7f720be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721415bd40 0x7f72141f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721415bd40 0x7f72141f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41984/0 (socket says 172.21.15.44:41984) 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720be00640 1 -- 172.21.15.44:0/2655026637 learned_addr learned my addr 172.21.15.44:0/2655026637 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f7211e00640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f721406cc20 0x7f7214150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f7212800640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f721415c120 0x7f72141f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:05.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720be00640 1 -- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f721415c120 msgr2=0x7f72141f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:05.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720be00640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f721415c120 0x7f72141f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:05.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720be00640 1 -- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f721406cc20 msgr2=0x7f7214150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:05.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720be00640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f721406cc20 0x7f7214150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:05.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720be00640 1 -- 172.21.15.44:0/2655026637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721415aba0 con 0x7f721415bd40 2024-09-13T08:53:05.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720be00640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721415bd40 0x7f72141f6720 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009790 tx=0x7f71fc0a6620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:05.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720a000640 1 -- 172.21.15.44:0/2655026637 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc004220 con 0x7f721415bd40 2024-09-13T08:53:05.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720a000640 1 -- 172.21.15.44:0/2655026637 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71fc0044b0 con 0x7f721415bd40 2024-09-13T08:53:05.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720a000640 1 -- 172.21.15.44:0/2655026637 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc003860 con 0x7f721415bd40 2024-09-13T08:53:05.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f7219000640 1 -- 172.21.15.44:0/2655026637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7214151490 con 0x7f721415bd40 2024-09-13T08:53:05.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f7219000640 1 -- 172.21.15.44:0/2655026637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7214159de0 con 0x7f721415bd40 2024-09-13T08:53:05.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f7219000640 1 -- 172.21.15.44:0/2655026637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7214073f20 con 0x7f721415bd40 2024-09-13T08:53:05.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.408+0000 7f720a000640 1 -- 172.21.15.44:0/2655026637 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f71fc0c0060 con 0x7f721415bd40 2024-09-13T08:53:05.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.412+0000 7f720a000640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f71f0075110 0x7f71f00775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:05.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.412+0000 7f720a000640 1 -- 172.21.15.44:0/2655026637 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5469+0+0 (secure 0 0 0) 0x7f71fc12d2f0 con 0x7f721415bd40 2024-09-13T08:53:05.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.412+0000 7f7211e00640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f71f0075110 0x7f71f00775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:05.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.412+0000 7f7211e00640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f71f0075110 0x7f71f00775d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7204009be0 tx=0x7f72040091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:05.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.416+0000 7f720a000640 1 -- 172.21.15.44:0/2655026637 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f71fc016410 con 0x7f721415bd40 2024-09-13T08:53:05.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:05.592+0000 7f7219000640 1 -- 172.21.15.44:0/2655026637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f72140b5620 con 0x7f721415bd40 2024-09-13T08:53:06.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.532+0000 7f720a000640 1 -- 172.21.15.44:0/2655026637 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v136) ==== 169+0+0 (secure 0 0 0) 0x7f71fc0f9f20 con 0x7f721415bd40 2024-09-13T08:53:06.533 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T08:53:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 -- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f71f0075110 msgr2=0x7f71f00775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f71f0075110 0x7f71f00775d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7204009be0 tx=0x7f72040091e0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 -- 172.21.15.44:0/2655026637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721415bd40 msgr2=0x7f72141f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721415bd40 0x7f72141f6720 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009790 tx=0x7f71fc0a6620 comp rx=0 tx=0).stop 2024-09-13T08:53:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 -- 172.21.15.44:0/2655026637 shutdown_connections 2024-09-13T08:53:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f71f0075110 0x7f71f00775d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f721415c120 0x7f72141f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721415bd40 0x7f72141f6720 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 --2- 172.21.15.44:0/2655026637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f721406cc20 0x7f7214150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 -- 172.21.15.44:0/2655026637 >> 172.21.15.44:0/2655026637 conn(0x7f721406dd30 msgr2=0x7f72140b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 -- 172.21.15.44:0/2655026637 shutdown_connections 2024-09-13T08:53:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.536+0000 7f7219000640 1 -- 172.21.15.44:0/2655026637 wait complete. 2024-09-13T08:53:06.565 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-13T08:53:06.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 -- 172.21.15.44:0/4162570672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa60c158240 msgr2=0x7fa60c15ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 --2- 172.21.15.44:0/4162570672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa60c158240 0x7fa60c15ede0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa600009900 tx=0x7fa60001c8b0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 -- 172.21.15.44:0/4162570672 shutdown_connections 2024-09-13T08:53:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 --2- 172.21.15.44:0/4162570672 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa60c158620 0x7fa60c161a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 --2- 172.21.15.44:0/4162570672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa60c158240 0x7fa60c15ede0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 -- 172.21.15.44:0/4162570672 >> 172.21.15.44:0/4162570672 conn(0x7fa60c0d0c90 msgr2=0x7fa60c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 -- 172.21.15.44:0/4162570672 shutdown_connections 2024-09-13T08:53:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 -- 172.21.15.44:0/4162570672 wait complete. 2024-09-13T08:53:06.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 Processor -- start 2024-09-13T08:53:06.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 -- start start 2024-09-13T08:53:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa60c158240 0x7fa60c0b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c158620 0x7fa60c0c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa60c161f70 0x7fa60c154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c0cebf0 con 0x7fa60c161f70 2024-09-13T08:53:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c0cea70 con 0x7fa60c158620 2024-09-13T08:53:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa614200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa60c0ced70 con 0x7fa60c158240 2024-09-13T08:53:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa612200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa60c161f70 0x7fa60c154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa612200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa60c161f70 0x7fa60c154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42002/0 (socket says 172.21.15.44:42002) 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa612200640 1 -- 172.21.15.44:0/456178865 learned_addr learned my addr 172.21.15.44:0/456178865 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa611800640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa60c158240 0x7fa60c0b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa610e00640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c158620 0x7fa60c0c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa612200640 1 -- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa60c158240 msgr2=0x7fa60c0b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa612200640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa60c158240 0x7fa60c0b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa612200640 1 -- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c158620 msgr2=0x7fa60c0c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa612200640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c158620 0x7fa60c0c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa612200640 1 -- 172.21.15.44:0/456178865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c0b35c0 con 0x7fa60c161f70 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa610e00640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c158620 0x7fa60c0c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.724+0000 7fa612200640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa60c161f70 0x7fa60c154970 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa60400c760 tx=0x7fa60400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.728+0000 7fa609600640 1 -- 172.21.15.44:0/456178865 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604003b80 con 0x7fa60c161f70 2024-09-13T08:53:06.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.728+0000 7fa609600640 1 -- 172.21.15.44:0/456178865 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa604003d20 con 0x7fa60c161f70 2024-09-13T08:53:06.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.728+0000 7fa609600640 1 -- 172.21.15.44:0/456178865 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60400aab0 con 0x7fa60c161f70 2024-09-13T08:53:06.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.728+0000 7fa614200640 1 -- 172.21.15.44:0/456178865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60c153bb0 con 0x7fa60c161f70 2024-09-13T08:53:06.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.728+0000 7fa614200640 1 -- 172.21.15.44:0/456178865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60c154170 con 0x7fa60c161f70 2024-09-13T08:53:06.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.728+0000 7fa614200640 1 -- 172.21.15.44:0/456178865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60c0cf290 con 0x7fa60c161f70 2024-09-13T08:53:06.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.728+0000 7fa609600640 1 -- 172.21.15.44:0/456178865 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7fa60400ac50 con 0x7fa60c161f70 2024-09-13T08:53:06.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.728+0000 7fa609600640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5ec075110 0x7fa5ec0775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:06.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.728+0000 7fa609600640 1 -- 172.21.15.44:0/456178865 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5055+0+0 (secure 0 0 0) 0x7fa604092bc0 con 0x7fa60c161f70 2024-09-13T08:53:06.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.728+0000 7fa611800640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5ec075110 0x7fa5ec0775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:06.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.732+0000 7fa611800640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5ec075110 0x7fa5ec0775d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa6000af000 tx=0x7fa60001b0f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.732+0000 7fa609600640 1 -- 172.21.15.44:0/456178865 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa60405f8f0 con 0x7fa60c161f70 2024-09-13T08:53:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.912+0000 7fa614200640 1 -- 172.21.15.44:0/456178865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fa60c161580 con 0x7fa60c161f70 2024-09-13T08:53:06.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa609600640 1 -- 172.21.15.44:0/456178865 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v53) ==== 71+0+0 (secure 0 0 0) 0x7fa60405f1f0 con 0x7fa60c161f70 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 -- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5ec075110 msgr2=0x7fa5ec0775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5ec075110 0x7fa5ec0775d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa6000af000 tx=0x7fa60001b0f0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 -- 172.21.15.44:0/456178865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa60c161f70 msgr2=0x7fa60c154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa60c161f70 0x7fa60c154970 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa60400c760 tx=0x7fa60400cc20 comp rx=0 tx=0).stop 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 -- 172.21.15.44:0/456178865 shutdown_connections 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa5ec075110 0x7fa5ec0775d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa60c161f70 0x7fa60c154970 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c158620 0x7fa60c0c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 --2- 172.21.15.44:0/456178865 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa60c158240 0x7fa60c0b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 -- 172.21.15.44:0/456178865 >> 172.21.15.44:0/456178865 conn(0x7fa60c0d0c90 msgr2=0x7fa60c0cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 -- 172.21.15.44:0/456178865 shutdown_connections 2024-09-13T08:53:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:06.916+0000 7fa614200640 1 -- 172.21.15.44:0/456178865 wait complete. 2024-09-13T08:53:06.938 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-13T08:53:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.084+0000 7f6db2800640 1 -- 172.21.15.44:0/1057352092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dac15a240 msgr2=0x7f6dac15e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:07.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.084+0000 7f6db2800640 1 --2- 172.21.15.44:0/1057352092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dac15a240 0x7f6dac15e730 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f6da000a950 tx=0x7f6da001dae0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.084+0000 7f6db2800640 1 -- 172.21.15.44:0/1057352092 shutdown_connections 2024-09-13T08:53:07.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.084+0000 7f6db2800640 1 --2- 172.21.15.44:0/1057352092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dac15a240 0x7f6dac15e730 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.084+0000 7f6db2800640 1 -- 172.21.15.44:0/1057352092 >> 172.21.15.44:0/1057352092 conn(0x7f6dac073490 msgr2=0x7f6dac0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:07.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.084+0000 7f6db2800640 1 -- 172.21.15.44:0/1057352092 shutdown_connections 2024-09-13T08:53:07.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.084+0000 7f6db2800640 1 -- 172.21.15.44:0/1057352092 wait complete. 2024-09-13T08:53:07.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.084+0000 7f6db2800640 1 Processor -- start 2024-09-13T08:53:07.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.084+0000 7f6db2800640 1 -- start start 2024-09-13T08:53:07.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dac0cad60 0x7f6dac09dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:07.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db2800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dac0cca70 0x7f6dac09d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:07.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db2800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dac15a240 0x7f6dac0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:07.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db2800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac0cbf20 con 0x7f6dac0cad60 2024-09-13T08:53:07.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db2800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac168100 con 0x7f6dac15a240 2024-09-13T08:53:07.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db2800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6dac0c8c50 con 0x7f6dac0cca70 2024-09-13T08:53:07.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dac0cad60 0x7f6dac09dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dac0cad60 0x7f6dac09dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42030/0 (socket says 172.21.15.44:42030) 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1400640 1 -- 172.21.15.44:0/3469518575 learned_addr learned my addr 172.21.15.44:0/3469518575 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db0a00640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dac0cca70 0x7f6dac09d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1e00640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dac15a240 0x7f6dac0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1400640 1 -- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dac0cca70 msgr2=0x7f6dac09d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1400640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dac0cca70 0x7f6dac09d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1400640 1 -- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dac15a240 msgr2=0x7f6dac0ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1400640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dac15a240 0x7f6dac0ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1400640 1 -- 172.21.15.44:0/3469518575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac09d960 con 0x7f6dac0cad60 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db0a00640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dac0cca70 0x7f6dac09d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1e00640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dac15a240 0x7f6dac0ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db1400640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dac0cad60 0x7f6dac09dc40 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f6da000a8a0 tx=0x7f6da0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6da9600640 1 -- 172.21.15.44:0/3469518575 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00b3040 con 0x7f6dac0cad60 2024-09-13T08:53:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6da9600640 1 -- 172.21.15.44:0/3469518575 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6da00052f0 con 0x7f6dac0cad60 2024-09-13T08:53:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6da9600640 1 -- 172.21.15.44:0/3469518575 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00046d0 con 0x7f6dac0cad60 2024-09-13T08:53:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db2800640 1 -- 172.21.15.44:0/3469518575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dac0c8ea0 con 0x7f6dac0cad60 2024-09-13T08:53:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db2800640 1 -- 172.21.15.44:0/3469518575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dac0c9460 con 0x7f6dac0cad60 2024-09-13T08:53:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.088+0000 7f6db2800640 1 -- 172.21.15.44:0/3469518575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dac15e730 con 0x7f6dac0cad60 2024-09-13T08:53:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.092+0000 7f6da9600640 1 -- 172.21.15.44:0/3469518575 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f6da0004870 con 0x7f6dac0cad60 2024-09-13T08:53:07.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.092+0000 7f6da9600640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d8c075110 0x7f6d8c0775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:07.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.092+0000 7f6da9600640 1 -- 172.21.15.44:0/3469518575 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5055+0+0 (secure 0 0 0) 0x7f6da012d5c0 con 0x7f6dac0cad60 2024-09-13T08:53:07.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.092+0000 7f6db0a00640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d8c075110 0x7f6d8c0775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:07.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.092+0000 7f6db0a00640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d8c075110 0x7f6d8c0775d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0711d0 tx=0x7f6d98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:07.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.096+0000 7f6da9600640 1 -- 172.21.15.44:0/3469518575 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6da00c5060 con 0x7f6dac0cad60 2024-09-13T08:53:07.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.336+0000 7f6db2800640 1 -- 172.21.15.44:0/3469518575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7f6dac0d09d0 con 0x7f6dac0cad60 2024-09-13T08:53:07.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6da9600640 1 -- 172.21.15.44:0/3469518575 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v704) ==== 186+0+0 (secure 0 0 0) 0x7f6da0017760 con 0x7f6dac0cad60 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 -- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d8c075110 msgr2=0x7f6d8c0775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d8c075110 0x7f6d8c0775d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0711d0 tx=0x7f6d98009040 comp rx=0 tx=0).stop 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 -- 172.21.15.44:0/3469518575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dac0cad60 msgr2=0x7f6dac09dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dac0cad60 0x7f6dac09dc40 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f6da000a8a0 tx=0x7f6da0005db0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 -- 172.21.15.44:0/3469518575 shutdown_connections 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d8c075110 0x7f6d8c0775d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dac15a240 0x7f6dac0ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6dac0cca70 0x7f6dac09d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 --2- 172.21.15.44:0/3469518575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dac0cad60 0x7f6dac09dc40 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 -- 172.21.15.44:0/3469518575 >> 172.21.15.44:0/3469518575 conn(0x7f6dac073490 msgr2=0x7f6dac0c7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 -- 172.21.15.44:0/3469518575 shutdown_connections 2024-09-13T08:53:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.540+0000 7f6db2800640 1 -- 172.21.15.44:0/3469518575 wait complete. 2024-09-13T08:53:07.563 INFO:tasks.cephfs_test_runner:Test that the progress bar that is printed for 1 ongoing clone job is ... ok 2024-09-13T08:53:07.564 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that both the progress bars, that are printed for all 7 clone 2024-09-13T08:53:07.564 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-13T08:53:07.564 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:53:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.708+0000 7f4155c00640 1 -- 172.21.15.44:0/21386003 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4150073170 msgr2=0x7f415006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.708+0000 7f4155c00640 1 --2- 172.21.15.44:0/21386003 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4150073170 0x7f415006d160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4138009840 tx=0x7f413801c800 comp rx=0 tx=0).stop 2024-09-13T08:53:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 -- 172.21.15.44:0/21386003 shutdown_connections 2024-09-13T08:53:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 --2- 172.21.15.44:0/21386003 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4150073170 0x7f415006d160 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 --2- 172.21.15.44:0/21386003 >> v2:172.21.15.129:3300/0 conn(0x7f41500ce800 0x7f415006cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 -- 172.21.15.44:0/21386003 >> 172.21.15.44:0/21386003 conn(0x7f41500b2240 msgr2=0x7f41500b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 -- 172.21.15.44:0/21386003 shutdown_connections 2024-09-13T08:53:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 -- 172.21.15.44:0/21386003 wait complete. 2024-09-13T08:53:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 Processor -- start 2024-09-13T08:53:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 -- start start 2024-09-13T08:53:07.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4150073170 0x7f415015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:07.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41500ce800 0x7f41501fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:07.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4150143fe0 0x7f415015a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:07.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4150168240 con 0x7f4150143fe0 2024-09-13T08:53:07.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f41501680c0 con 0x7f41500ce800 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4155c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f41501683c0 con 0x7f4150073170 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f414f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4150143fe0 0x7f415015a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f414f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4150143fe0 0x7f415015a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42042/0 (socket says 172.21.15.44:42042) 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f414f400640 1 -- 172.21.15.44:0/2000753877 learned_addr learned my addr 172.21.15.44:0/2000753877 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f414ea00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4150073170 0x7f415015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f4147e00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41500ce800 0x7f41501fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f414f400640 1 -- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4150073170 msgr2=0x7f415015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f414f400640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4150073170 0x7f415015dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f414f400640 1 -- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41500ce800 msgr2=0x7f41501fbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f414f400640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41500ce800 0x7f41501fbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f414f400640 1 -- 172.21.15.44:0/2000753877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415015c550 con 0x7f4150143fe0 2024-09-13T08:53:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.712+0000 7f414f400640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4150143fe0 0x7f415015a280 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f414800c730 tx=0x7f414800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.716+0000 7f4146a00640 1 -- 172.21.15.44:0/2000753877 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148003b80 con 0x7f4150143fe0 2024-09-13T08:53:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.716+0000 7f4146a00640 1 -- 172.21.15.44:0/2000753877 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4148003d20 con 0x7f4150143fe0 2024-09-13T08:53:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.716+0000 7f4146a00640 1 -- 172.21.15.44:0/2000753877 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414800aab0 con 0x7f4150143fe0 2024-09-13T08:53:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.716+0000 7f4155c00640 1 -- 172.21.15.44:0/2000753877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415015c830 con 0x7f4150143fe0 2024-09-13T08:53:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.716+0000 7f4155c00640 1 -- 172.21.15.44:0/2000753877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415015cdf0 con 0x7f4150143fe0 2024-09-13T08:53:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.716+0000 7f4155c00640 1 -- 172.21.15.44:0/2000753877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415015e8b0 con 0x7f4150143fe0 2024-09-13T08:53:07.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.716+0000 7f4146a00640 1 -- 172.21.15.44:0/2000753877 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f414800ac50 con 0x7f4150143fe0 2024-09-13T08:53:07.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.716+0000 7f4146a00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f412c075140 0x7f412c077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:07.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.720+0000 7f4146a00640 1 -- 172.21.15.44:0/2000753877 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5055+0+0 (secure 0 0 0) 0x7f4148092b20 con 0x7f4150143fe0 2024-09-13T08:53:07.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.720+0000 7f414ea00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f412c075140 0x7f412c077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:07.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.720+0000 7f414ea00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f412c075140 0x7f412c077600 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f41400096c0 tx=0x7f4140009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:07.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.720+0000 7f4146a00640 1 -- 172.21.15.44:0/2000753877 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f414805f220 con 0x7f4150143fe0 2024-09-13T08:53:07.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.896+0000 7f4155c00640 1 -- 172.21.15.44:0/2000753877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41500b2a90 con 0x7f4150143fe0 2024-09-13T08:53:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4146a00640 1 -- 172.21.15.44:0/2000753877 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+13231 (secure 0 0 0) 0x7f4148063c50 con 0x7f4150143fe0 2024-09-13T08:53:07.899 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:07.900 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":136,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:53:06.533475+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/2181881572":"2024-09-14T08:53:03.036701+0000","172.21.15.44:6834/2181881572":"2024-09-14T08:53:03.036701+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:53:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 -- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f412c075140 msgr2=0x7f412c077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f412c075140 0x7f412c077600 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f41400096c0 tx=0x7f4140009260 comp rx=0 tx=0).stop 2024-09-13T08:53:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 -- 172.21.15.44:0/2000753877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4150143fe0 msgr2=0x7f415015a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4150143fe0 0x7f415015a280 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f414800c730 tx=0x7f414800cbf0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 -- 172.21.15.44:0/2000753877 shutdown_connections 2024-09-13T08:53:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f412c075140 0x7f412c077600 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4150143fe0 0x7f415015a280 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41500ce800 0x7f41501fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 --2- 172.21.15.44:0/2000753877 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4150073170 0x7f415015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 -- 172.21.15.44:0/2000753877 >> 172.21.15.44:0/2000753877 conn(0x7f41500b2240 msgr2=0x7f4150165220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 -- 172.21.15.44:0/2000753877 shutdown_connections 2024-09-13T08:53:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:07.900+0000 7f4155c00640 1 -- 172.21.15.44:0/2000753877 wait complete. 2024-09-13T08:53:07.921 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T08:37:24.563344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T08:53:07.921 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T08:53:08.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 -- 172.21.15.44:0/2447718693 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0700ce620 msgr2=0x7fa0700cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:08.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 --2- 172.21.15.44:0/2447718693 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0700ce620 0x7fa0700cfd30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa068009c80 tx=0x7fa06801cc80 comp rx=0 tx=0).stop 2024-09-13T08:53:08.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 -- 172.21.15.44:0/2447718693 shutdown_connections 2024-09-13T08:53:08.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 --2- 172.21.15.44:0/2447718693 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0700ce620 0x7fa0700cfd30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 --2- 172.21.15.44:0/2447718693 >> v2:172.21.15.129:3300/0 conn(0x7fa0700b27a0 0x7fa0700ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 -- 172.21.15.44:0/2447718693 >> 172.21.15.44:0/2447718693 conn(0x7fa0700d1e30 msgr2=0x7fa0700d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:08.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 -- 172.21.15.44:0/2447718693 shutdown_connections 2024-09-13T08:53:08.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 -- 172.21.15.44:0/2447718693 wait complete. 2024-09-13T08:53:08.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 Processor -- start 2024-09-13T08:53:08.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 -- start start 2024-09-13T08:53:08.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0700b27a0 0x7fa070152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:08.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0700ce620 0x7fa07015b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:08.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa07015f640 0x7fa07015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:08.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa0700b1010 con 0x7fa0700ce620 2024-09-13T08:53:08.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa0700b0e90 con 0x7fa07015f640 2024-09-13T08:53:08.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.072+0000 7fa075600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa070169390 con 0x7fa0700b27a0 2024-09-13T08:53:08.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa067e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0700ce620 0x7fa07015b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:08.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa067e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0700ce620 0x7fa07015b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42054/0 (socket says 172.21.15.44:42054) 2024-09-13T08:53:08.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa067e00640 1 -- 172.21.15.44:0/190258709 learned_addr learned my addr 172.21.15.44:0/190258709 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:08.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa06ea00640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0700b27a0 0x7fa070152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:08.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa06f400640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa07015f640 0x7fa07015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:08.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa067e00640 1 -- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0700b27a0 msgr2=0x7fa070152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:08.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa067e00640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0700b27a0 0x7fa070152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa067e00640 1 -- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa07015f640 msgr2=0x7fa07015bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:08.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa067e00640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa07015f640 0x7fa07015bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa067e00640 1 -- 172.21.15.44:0/190258709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa070153540 con 0x7fa0700ce620 2024-09-13T08:53:08.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa067e00640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0700ce620 0x7fa07015b820 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa05800e9f0 tx=0x7fa05800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:08.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa066a00640 1 -- 172.21.15.44:0/190258709 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05800cd80 con 0x7fa0700ce620 2024-09-13T08:53:08.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa066a00640 1 -- 172.21.15.44:0/190258709 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa058010430 con 0x7fa0700ce620 2024-09-13T08:53:08.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa066a00640 1 -- 172.21.15.44:0/190258709 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058010860 con 0x7fa0700ce620 2024-09-13T08:53:08.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa075600640 1 -- 172.21.15.44:0/190258709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa070152db0 con 0x7fa0700ce620 2024-09-13T08:53:08.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa075600640 1 -- 172.21.15.44:0/190258709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0701646f0 con 0x7fa0700ce620 2024-09-13T08:53:08.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa075600640 1 -- 172.21.15.44:0/190258709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0700ce0e0 con 0x7fa0700ce620 2024-09-13T08:53:08.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.076+0000 7fa066a00640 1 -- 172.21.15.44:0/190258709 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7fa058010a00 con 0x7fa0700ce620 2024-09-13T08:53:08.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.080+0000 7fa066a00640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa04c075140 0x7fa04c077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:08.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.080+0000 7fa066a00640 1 -- 172.21.15.44:0/190258709 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5055+0+0 (secure 0 0 0) 0x7fa058093a70 con 0x7fa0700ce620 2024-09-13T08:53:08.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.080+0000 7fa06ea00640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa04c075140 0x7fa04c077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:08.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.080+0000 7fa06ea00640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa04c075140 0x7fa04c077600 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa0700c8bf0 tx=0x7fa060009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:08.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.080+0000 7fa066a00640 1 -- 172.21.15.44:0/190258709 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa058060190 con 0x7fa0700ce620 2024-09-13T08:53:08.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.256+0000 7fa075600640 1 -- 172.21.15.44:0/190258709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa070156c90 con 0x7fa0700ce620 2024-09-13T08:53:08.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.256+0000 7fa066a00640 1 -- 172.21.15.44:0/190258709 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) ==== 93+0+10 (secure 0 0 0) 0x7fa058064bc0 con 0x7fa0700ce620 2024-09-13T08:53:08.258 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-09-13T08:53:08.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 -- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa04c075140 msgr2=0x7fa04c077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:08.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa04c075140 0x7fa04c077600 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa0700c8bf0 tx=0x7fa060009040 comp rx=0 tx=0).stop 2024-09-13T08:53:08.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 -- 172.21.15.44:0/190258709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0700ce620 msgr2=0x7fa07015b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:08.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0700ce620 0x7fa07015b820 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa05800e9f0 tx=0x7fa05800eeb0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 -- 172.21.15.44:0/190258709 shutdown_connections 2024-09-13T08:53:08.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa04c075140 0x7fa04c077600 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa07015f640 0x7fa07015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0700ce620 0x7fa07015b820 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 --2- 172.21.15.44:0/190258709 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa0700b27a0 0x7fa070152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 -- 172.21.15.44:0/190258709 >> 172.21.15.44:0/190258709 conn(0x7fa0700d1e30 msgr2=0x7fa0700b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 -- 172.21.15.44:0/190258709 shutdown_connections 2024-09-13T08:53:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.260+0000 7fa075600640 1 -- 172.21.15.44:0/190258709 wait complete. 2024-09-13T08:53:08.279 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-13T08:53:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.424+0000 7f6efe400640 1 -- 172.21.15.44:0/665833899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6edc003a60 msgr2=0x7f6edc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.424+0000 7f6efe400640 1 --2- 172.21.15.44:0/665833899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6edc003a60 0x7f6edc003e80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c2e0 tx=0x7f6ee801cf20 comp rx=0 tx=0).stop 2024-09-13T08:53:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.424+0000 7f6efe400640 1 -- 172.21.15.44:0/665833899 shutdown_connections 2024-09-13T08:53:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.424+0000 7f6efe400640 1 --2- 172.21.15.44:0/665833899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6edc003a60 0x7f6edc003e80 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.424+0000 7f6efe400640 1 -- 172.21.15.44:0/665833899 >> 172.21.15.44:0/665833899 conn(0x7f6ef80d1dc0 msgr2=0x7f6ef80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.424+0000 7f6efe400640 1 -- 172.21.15.44:0/665833899 shutdown_connections 2024-09-13T08:53:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.424+0000 7f6efe400640 1 -- 172.21.15.44:0/665833899 wait complete. 2024-09-13T08:53:08.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6efe400640 1 Processor -- start 2024-09-13T08:53:08.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6efe400640 1 -- start start 2024-09-13T08:53:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6efe400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6edc003a60 0x7f6ef80b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6efe400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8152570 0x7f6ef8158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6efe400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6ef81588d0 0x7f6ef81564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6efe400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef80cea80 con 0x7f6ef8152570 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6efe400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef80ce900 con 0x7f6edc003a60 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6efe400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef80cec00 con 0x7f6ef81588d0 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eefe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8152570 0x7f6ef8158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eefe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8152570 0x7f6ef8158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42076/0 (socket says 172.21.15.44:42076) 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eefe00640 1 -- 172.21.15.44:0/2585131589 learned_addr learned my addr 172.21.15.44:0/2585131589 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6ef7e00640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6ef81588d0 0x7f6ef81564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6ef7400640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6edc003a60 0x7f6ef80b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eefe00640 1 -- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6ef81588d0 msgr2=0x7f6ef81564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eefe00640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6ef81588d0 0x7f6ef81564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eefe00640 1 -- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6edc003a60 msgr2=0x7f6ef80b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eefe00640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6edc003a60 0x7f6ef80b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eefe00640 1 -- 172.21.15.44:0/2585131589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef81f69b0 con 0x7f6ef8152570 2024-09-13T08:53:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eefe00640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8152570 0x7f6ef8158390 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00069a0 tx=0x7f6ee0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eef400640 1 -- 172.21.15.44:0/2585131589 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0015020 con 0x7f6ef8152570 2024-09-13T08:53:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eef400640 1 -- 172.21.15.44:0/2585131589 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ee0011750 con 0x7f6ef8152570 2024-09-13T08:53:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6eef400640 1 -- 172.21.15.44:0/2585131589 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0011b80 con 0x7f6ef8152570 2024-09-13T08:53:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6efe400640 1 -- 172.21.15.44:0/2585131589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef81f5e30 con 0x7f6ef8152570 2024-09-13T08:53:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.428+0000 7f6efe400640 1 -- 172.21.15.44:0/2585131589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef81f63f0 con 0x7f6ef8152570 2024-09-13T08:53:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.432+0000 7f6efe400640 1 -- 172.21.15.44:0/2585131589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef8158ce0 con 0x7f6ef8152570 2024-09-13T08:53:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.432+0000 7f6eef400640 1 -- 172.21.15.44:0/2585131589 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f6ee0018020 con 0x7f6ef8152570 2024-09-13T08:53:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.432+0000 7f6eef400640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6ed4075110 0x7f6ed40775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.432+0000 7f6eef400640 1 -- 172.21.15.44:0/2585131589 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5055+0+0 (secure 0 0 0) 0x7f6ee0093570 con 0x7f6ef8152570 2024-09-13T08:53:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.432+0000 7f6ef7400640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6ed4075110 0x7f6ed40775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.432+0000 7f6ef7400640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6ed4075110 0x7f6ed40775d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80ac000 tx=0x7f6ee8019d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.436+0000 7f6eef400640 1 -- 172.21.15.44:0/2585131589 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6ef8158ce0 con 0x7f6ef8152570 2024-09-13T08:53:08.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:08.672+0000 7f6efe400640 1 -- 172.21.15.44:0/2585131589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7f6ef80b0850 con 0x7f6ef8152570 2024-09-13T08:53:09.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.548+0000 7f6eef400640 1 -- 172.21.15.44:0/2585131589 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v706) ==== 189+0+0 (secure 0 0 0) 0x7f6ee001f030 con 0x7f6ef8152570 2024-09-13T08:53:09.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 -- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6ed4075110 msgr2=0x7f6ed40775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:09.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6ed4075110 0x7f6ed40775d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80ac000 tx=0x7f6ee8019d60 comp rx=0 tx=0).stop 2024-09-13T08:53:09.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 -- 172.21.15.44:0/2585131589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8152570 msgr2=0x7f6ef8158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:09.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8152570 0x7f6ef8158390 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00069a0 tx=0x7f6ee0001640 comp rx=0 tx=0).stop 2024-09-13T08:53:09.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 -- 172.21.15.44:0/2585131589 shutdown_connections 2024-09-13T08:53:09.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6ed4075110 0x7f6ed40775d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6ef81588d0 0x7f6ef81564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ef8152570 0x7f6ef8158390 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 --2- 172.21.15.44:0/2585131589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6edc003a60 0x7f6ef80b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 -- 172.21.15.44:0/2585131589 >> 172.21.15.44:0/2585131589 conn(0x7f6ef80d1dc0 msgr2=0x7f6ef80c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:09.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 -- 172.21.15.44:0/2585131589 shutdown_connections 2024-09-13T08:53:09.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.552+0000 7f6efe400640 1 -- 172.21.15.44:0/2585131589 wait complete. 2024-09-13T08:53:09.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-13T08:53:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 -- 172.21.15.44:0/2420328393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11900cdb30 msgr2=0x7f11900cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 --2- 172.21.15.44:0/2420328393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11900cdb30 0x7f11900cebb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f117c009840 tx=0x7f117c01c820 comp rx=0 tx=0).stop 2024-09-13T08:53:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 -- 172.21.15.44:0/2420328393 shutdown_connections 2024-09-13T08:53:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 --2- 172.21.15.44:0/2420328393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11900cdb30 0x7f11900cebb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 --2- 172.21.15.44:0/2420328393 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f119015bea0 0x7f11900ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 -- 172.21.15.44:0/2420328393 >> 172.21.15.44:0/2420328393 conn(0x7f11900d1dc0 msgr2=0x7f11900d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 -- 172.21.15.44:0/2420328393 shutdown_connections 2024-09-13T08:53:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 -- 172.21.15.44:0/2420328393 wait complete. 2024-09-13T08:53:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 Processor -- start 2024-09-13T08:53:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 -- start start 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11900cdb30 0x7f119009f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11900d28e0 0x7f1190156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f119015bea0 0x7f1190158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1190168fc0 con 0x7f119015bea0 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1190168e40 con 0x7f11900cdb30 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1197600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1190169140 con 0x7f11900d28e0 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1195600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f119015bea0 0x7f1190158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1194c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11900cdb30 0x7f119009f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1195600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f119015bea0 0x7f1190158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42078/0 (socket says 172.21.15.44:42078) 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1195600640 1 -- 172.21.15.44:0/3553009084 learned_addr learned my addr 172.21.15.44:0/3553009084 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f118fe00640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11900d28e0 0x7f1190156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1194c00640 1 -- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11900d28e0 msgr2=0x7f1190156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1194c00640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11900d28e0 0x7f1190156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1194c00640 1 -- 172.21.15.44:0/3553009084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f119015bea0 msgr2=0x7f1190158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1194c00640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f119015bea0 0x7f1190158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.728+0000 7f1194c00640 1 -- 172.21.15.44:0/3553009084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119009fac0 con 0x7f11900cdb30 2024-09-13T08:53:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.732+0000 7f1194c00640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11900cdb30 0x7f119009f580 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f118400e9f0 tx=0x7f118400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.732+0000 7f118cc00640 1 -- 172.21.15.44:0/3553009084 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118400cd80 con 0x7f11900cdb30 2024-09-13T08:53:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.732+0000 7f118cc00640 1 -- 172.21.15.44:0/3553009084 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1184004020 con 0x7f11900cdb30 2024-09-13T08:53:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.732+0000 7f118cc00640 1 -- 172.21.15.44:0/3553009084 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1184004470 con 0x7f11900cdb30 2024-09-13T08:53:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.732+0000 7f1197600640 1 -- 172.21.15.44:0/3553009084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1190144b60 con 0x7f11900cdb30 2024-09-13T08:53:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.732+0000 7f1197600640 1 -- 172.21.15.44:0/3553009084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1190156350 con 0x7f11900cdb30 2024-09-13T08:53:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.732+0000 7f1197600640 1 -- 172.21.15.44:0/3553009084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11900cc9d0 con 0x7f11900cdb30 2024-09-13T08:53:09.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.732+0000 7f118cc00640 1 -- 172.21.15.44:0/3553009084 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f1184003600 con 0x7f11900cdb30 2024-09-13T08:53:09.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.736+0000 7f118cc00640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1170075110 0x7f11700775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:09.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.736+0000 7f118cc00640 1 -- 172.21.15.44:0/3553009084 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5055+0+0 (secure 0 0 0) 0x7f1184093950 con 0x7f11900cdb30 2024-09-13T08:53:09.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.736+0000 7f118fe00640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1170075110 0x7f11700775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:09.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.736+0000 7f118fe00640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1170075110 0x7f11700775d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f117c0af000 tx=0x7f117c01b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:09.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.736+0000 7f118cc00640 1 -- 172.21.15.44:0/3553009084 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1184060030 con 0x7f11900cdb30 2024-09-13T08:53:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.912+0000 7f1197600640 1 -- 172.21.15.44:0/3553009084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f11900b58c0 con 0x7f11900cdb30 2024-09-13T08:53:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.912+0000 7f118cc00640 1 -- 172.21.15.44:0/3553009084 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v53) ==== 59+0+1998 (secure 0 0 0) 0x7f1184064a60 con 0x7f11900cdb30 2024-09-13T08:53:09.915 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-13T08:53:09.915 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-09-13T08:53:09.915 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-13T08:53:09.915 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-13T08:53:09.915 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4094.620852 2024-09-13T08:53:09.915 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3964.014083 2024-09-13T08:53:09.915 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3978.263337 2024-09-13T08:53:09.915 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4002.298162 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4667.000623 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4664.046790 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6403.016405 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4696.908546 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 2/2 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-09-13T08:53:09.916 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-13T08:53:09.917 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-09-13T08:53:09.917 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-09-13T08:53:09.917 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-09-13T08:53:09.917 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-09-13T08:53:09.917 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-09-13T08:53:09.917 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-09-13T08:53:09.917 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-09-13T08:53:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 -- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1170075110 msgr2=0x7f11700775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1170075110 0x7f11700775d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f117c0af000 tx=0x7f117c01b060 comp rx=0 tx=0).stop 2024-09-13T08:53:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 -- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11900cdb30 msgr2=0x7f119009f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11900cdb30 0x7f119009f580 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f118400e9f0 tx=0x7f118400eeb0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 -- 172.21.15.44:0/3553009084 shutdown_connections 2024-09-13T08:53:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1170075110 0x7f11700775d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f119015bea0 0x7f1190158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11900d28e0 0x7f1190156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 --2- 172.21.15.44:0/3553009084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11900cdb30 0x7f119009f580 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:09.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 -- 172.21.15.44:0/3553009084 >> 172.21.15.44:0/3553009084 conn(0x7f11900d1dc0 msgr2=0x7f11900b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:09.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 -- 172.21.15.44:0/3553009084 shutdown_connections 2024-09-13T08:53:09.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:09.916+0000 7f1197600640 1 -- 172.21.15.44:0/3553009084 wait complete. 2024-09-13T08:53:09.936 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:53:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.084+0000 7f0d05a00640 1 -- 172.21.15.44:0/2985435132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d0015b5e0 msgr2=0x7f0d001650a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.084+0000 7f0d05a00640 1 --2- 172.21.15.44:0/2985435132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d0015b5e0 0x7f0d001650a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009c80 tx=0x7f0cf801cd70 comp rx=0 tx=0).stop 2024-09-13T08:53:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.084+0000 7f0d05a00640 1 -- 172.21.15.44:0/2985435132 shutdown_connections 2024-09-13T08:53:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.084+0000 7f0d05a00640 1 --2- 172.21.15.44:0/2985435132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d0015b5e0 0x7f0d001650a0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.084+0000 7f0d05a00640 1 --2- 172.21.15.44:0/2985435132 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d000cd100 0x7f0d000cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.084+0000 7f0d05a00640 1 -- 172.21.15.44:0/2985435132 >> 172.21.15.44:0/2985435132 conn(0x7f0d000b18f0 msgr2=0x7f0d000b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.084+0000 7f0d05a00640 1 -- 172.21.15.44:0/2985435132 shutdown_connections 2024-09-13T08:53:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.084+0000 7f0d05a00640 1 -- 172.21.15.44:0/2985435132 wait complete. 2024-09-13T08:53:10.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0d05a00640 1 Processor -- start 2024-09-13T08:53:10.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0d05a00640 1 -- start start 2024-09-13T08:53:10.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0d05a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d000b2bf0 0x7f0d00155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:10.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0d05a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000cd100 0x7f0d000ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0d05a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d001560e0 0x7f0d00164830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0d05a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001684c0 con 0x7f0d000cd100 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0d05a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00168340 con 0x7f0d000b2bf0 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0d05a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0d00168640 con 0x7f0d001560e0 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000cd100 0x7f0d000ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cfea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d000b2bf0 0x7f0d00155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000cd100 0x7f0d000ae0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42102/0 (socket says 172.21.15.44:42102) 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf6000640 1 -- 172.21.15.44:0/3163569237 learned_addr learned my addr 172.21.15.44:0/3163569237 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cfea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d000b2bf0 0x7f0d00155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:58030/0 (socket says 172.21.15.44:58030) 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cff400640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d001560e0 0x7f0d00164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf6000640 1 -- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d001560e0 msgr2=0x7f0d00164830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf6000640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d001560e0 0x7f0d00164830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf6000640 1 -- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d000b2bf0 msgr2=0x7f0d00155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf6000640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d000b2bf0 0x7f0d00155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf6000640 1 -- 172.21.15.44:0/3163569237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d00163790 con 0x7f0d000cd100 2024-09-13T08:53:10.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cff400640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d001560e0 0x7f0d00164830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:10.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf6000640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000cd100 0x7f0d000ae0f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800c400 tx=0x7f0ce800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:10.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf7400640 1 -- 172.21.15.44:0/3163569237 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8017070 con 0x7f0d000cd100 2024-09-13T08:53:10.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf7400640 1 -- 172.21.15.44:0/3163569237 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ce80053e0 con 0x7f0d000cd100 2024-09-13T08:53:10.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0cf7400640 1 -- 172.21.15.44:0/3163569237 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8003860 con 0x7f0d000cd100 2024-09-13T08:53:10.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0d05a00640 1 -- 172.21.15.44:0/3163569237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d00163a70 con 0x7f0d000cd100 2024-09-13T08:53:10.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.088+0000 7f0d05a00640 1 -- 172.21.15.44:0/3163569237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d00164030 con 0x7f0d000cd100 2024-09-13T08:53:10.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.092+0000 7f0d05a00640 1 -- 172.21.15.44:0/3163569237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d00069410 con 0x7f0d000cd100 2024-09-13T08:53:10.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.092+0000 7f0cf7400640 1 -- 172.21.15.44:0/3163569237 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f0ce801a070 con 0x7f0d000cd100 2024-09-13T08:53:10.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.092+0000 7f0cf7400640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cdc075110 0x7f0cdc0775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:10.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.092+0000 7f0cf7400640 1 -- 172.21.15.44:0/3163569237 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5055+0+0 (secure 0 0 0) 0x7f0ce8092300 con 0x7f0d000cd100 2024-09-13T08:53:10.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.092+0000 7f0cfea00640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cdc075110 0x7f0cdc0775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:10.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.092+0000 7f0cfea00640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cdc075110 0x7f0cdc0775d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0004630 tx=0x7f0cf000b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:10.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.096+0000 7f0cf7400640 1 -- 172.21.15.44:0/3163569237 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0ce8001130 con 0x7f0d000cd100 2024-09-13T08:53:10.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.272+0000 7f0d05a00640 1 -- 172.21.15.44:0/3163569237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0d00155e80 con 0x7f0d000cd100 2024-09-13T08:53:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0cf7400640 1 -- 172.21.15.44:0/3163569237 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7f0ce805e960 con 0x7f0d000cd100 2024-09-13T08:53:10.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 -- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cdc075110 msgr2=0x7f0cdc0775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:10.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cdc075110 0x7f0cdc0775d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0004630 tx=0x7f0cf000b390 comp rx=0 tx=0).stop 2024-09-13T08:53:10.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 -- 172.21.15.44:0/3163569237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000cd100 msgr2=0x7f0d000ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:10.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000cd100 0x7f0d000ae0f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800c400 tx=0x7f0ce800c8c0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 -- 172.21.15.44:0/3163569237 shutdown_connections 2024-09-13T08:53:10.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0cdc075110 0x7f0cdc0775d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0d001560e0 0x7f0d00164830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000cd100 0x7f0d000ae0f0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 --2- 172.21.15.44:0/3163569237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d000b2bf0 0x7f0d00155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 -- 172.21.15.44:0/3163569237 >> 172.21.15.44:0/3163569237 conn(0x7f0d000b18f0 msgr2=0x7f0d00072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:10.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 -- 172.21.15.44:0/3163569237 shutdown_connections 2024-09-13T08:53:10.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.276+0000 7f0d05a00640 1 -- 172.21.15.44:0/3163569237 wait complete. 2024-09-13T08:53:10.298 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:53:10.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:53:10.370 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:53:10.458 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:53:10.458 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:53:10.458 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:10.466 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:53:10.466 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:53:10.467 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:10.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:53:10.475 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:53:10.475 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:10.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:53:10.515 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:53:10.515 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:10.532 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:53:10.532 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T08:53:10.532 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:53:10.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.704+0000 7f9823400640 1 -- 172.21.15.44:0/3736063038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f981c0ce800 msgr2=0x7f981c06cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:10.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.704+0000 7f9823400640 1 --2- 172.21.15.44:0/3736063038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f981c0ce800 0x7f981c06cc20 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f981c068f50 tx=0x7f980c01c800 comp rx=0 tx=0).stop 2024-09-13T08:53:10.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.704+0000 7f9823400640 1 -- 172.21.15.44:0/3736063038 shutdown_connections 2024-09-13T08:53:10.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.704+0000 7f9823400640 1 --2- 172.21.15.44:0/3736063038 >> v2:172.21.15.129:3300/0 conn(0x7f981c073170 0x7f981c06d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.704+0000 7f9823400640 1 --2- 172.21.15.44:0/3736063038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f981c0ce800 0x7f981c06cc20 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.704+0000 7f9823400640 1 -- 172.21.15.44:0/3736063038 >> 172.21.15.44:0/3736063038 conn(0x7f981c0b2240 msgr2=0x7f981c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:10.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.704+0000 7f9823400640 1 -- 172.21.15.44:0/3736063038 shutdown_connections 2024-09-13T08:53:10.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.704+0000 7f9823400640 1 -- 172.21.15.44:0/3736063038 wait complete. 2024-09-13T08:53:10.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.704+0000 7f9823400640 1 Processor -- start 2024-09-13T08:53:10.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.704+0000 7f9823400640 1 -- start start 2024-09-13T08:53:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9823400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f981c073170 0x7f981c141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9823400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f981c0ce800 0x7f981c161f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9823400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f981c143fe0 0x7f981c0aa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9823400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1682e0 con 0x7f981c0ce800 2024-09-13T08:53:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9823400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f981c168160 con 0x7f981c143fe0 2024-09-13T08:53:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9823400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f981c168460 con 0x7f981c073170 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9813e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f981c0ce800 0x7f981c161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9813e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f981c0ce800 0x7f981c161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42116/0 (socket says 172.21.15.44:42116) 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9813e00640 1 -- 172.21.15.44:0/1077351330 learned_addr learned my addr 172.21.15.44:0/1077351330 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9821400640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f981c143fe0 0x7f981c0aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9820a00640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f981c073170 0x7f981c141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9813e00640 1 -- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f981c073170 msgr2=0x7f981c141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9813e00640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f981c073170 0x7f981c141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9813e00640 1 -- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f981c143fe0 msgr2=0x7f981c0aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9813e00640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f981c143fe0 0x7f981c0aa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9813e00640 1 -- 172.21.15.44:0/1077351330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981c141aa0 con 0x7f981c0ce800 2024-09-13T08:53:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9813e00640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f981c0ce800 0x7f981c161f20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f980400c920 tx=0x7f980400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9819600640 1 -- 172.21.15.44:0/1077351330 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804007c70 con 0x7f981c0ce800 2024-09-13T08:53:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9819600640 1 -- 172.21.15.44:0/1077351330 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9804005030 con 0x7f981c0ce800 2024-09-13T08:53:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9819600640 1 -- 172.21.15.44:0/1077351330 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804005480 con 0x7f981c0ce800 2024-09-13T08:53:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9823400640 1 -- 172.21.15.44:0/1077351330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981c160e80 con 0x7f981c0ce800 2024-09-13T08:53:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9823400640 1 -- 172.21.15.44:0/1077351330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981c161440 con 0x7f981c0ce800 2024-09-13T08:53:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.708+0000 7f9823400640 1 -- 172.21.15.44:0/1077351330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f981c15e8b0 con 0x7f981c0ce800 2024-09-13T08:53:10.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.712+0000 7f9819600640 1 -- 172.21.15.44:0/1077351330 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f98040047d0 con 0x7f981c0ce800 2024-09-13T08:53:10.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.712+0000 7f9819600640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f97f8075110 0x7f97f80775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:10.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.712+0000 7f9819600640 1 -- 172.21.15.44:0/1077351330 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5055+0+0 (secure 0 0 0) 0x7f98040935b0 con 0x7f981c0ce800 2024-09-13T08:53:10.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.712+0000 7f9820a00640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f97f8075110 0x7f97f80775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:10.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.712+0000 7f9820a00640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f97f8075110 0x7f97f80775d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f980c0af000 tx=0x7f980c01b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:10.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.716+0000 7f9819600640 1 -- 172.21.15.44:0/1077351330 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f981c141aa0 con 0x7f981c0ce800 2024-09-13T08:53:10.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 -- 172.21.15.44:0/1077351330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f981c068cb0 con 0x7f981c0ce800 2024-09-13T08:53:10.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9819600640 1 -- 172.21.15.44:0/1077351330 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+3911 (secure 0 0 0) 0x7f9804014070 con 0x7f981c0ce800 2024-09-13T08:53:10.932 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:10.932 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"btime":"2024-09-13T08:53:04:038762+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6836/771921648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":771921648},{"type":"v1","addr":"172.21.15.129:6836","nonce":771921648}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8104,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1464443005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6835","nonce":1464443005},{"type":"v1","addr":"172.21.15.129:6837","nonce":1464443005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2601537201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2601537201},{"type":"v1","addr":"172.21.15.44:6833","nonce":2601537201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/930254867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":930254867},{"type":"v1","addr":"172.21.15.44:6835","nonce":930254867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[]} 2024-09-13T08:53:10.933 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 88 2024-09-13T08:53:10.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 -- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f97f8075110 msgr2=0x7f97f80775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:10.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f97f8075110 0x7f97f80775d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f980c0af000 tx=0x7f980c01b040 comp rx=0 tx=0).stop 2024-09-13T08:53:10.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 -- 172.21.15.44:0/1077351330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f981c0ce800 msgr2=0x7f981c161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:10.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f981c0ce800 0x7f981c161f20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f980400c920 tx=0x7f980400cde0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 -- 172.21.15.44:0/1077351330 shutdown_connections 2024-09-13T08:53:10.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f97f8075110 0x7f97f80775d0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f981c143fe0 0x7f981c0aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f981c0ce800 0x7f981c161f20 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 --2- 172.21.15.44:0/1077351330 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f981c073170 0x7f981c141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:10.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 -- 172.21.15.44:0/1077351330 >> 172.21.15.44:0/1077351330 conn(0x7f981c0b2240 msgr2=0x7f981c0d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:10.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 -- 172.21.15.44:0/1077351330 shutdown_connections 2024-09-13T08:53:10.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:10.932+0000 7f9823400640 1 -- 172.21.15.44:0/1077351330 wait complete. 2024-09-13T08:53:10.955 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T08:53:10.956 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T08:53:10.956 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T08:53:10.956 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:53:10.956 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T08:53:10.957 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T08:53:10.957 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T08:53:10.957 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:53:10.957 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T08:53:10.957 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T08:53:10.957 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T08:53:10.958 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:53:10.958 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T08:53:10.958 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T08:53:10.958 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T08:53:10.958 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:53:10.968 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T08:53:10.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T08:53:10.969 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T08:53:10.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T08:53:10.971 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T08:53:10.971 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T08:53:10.972 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T08:53:10.972 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T08:53:11.018 INFO:tasks.ceph.mds.d:Started 2024-09-13T08:53:11.018 DEBUG:teuthology.parallel:result is None 2024-09-13T08:53:11.018 INFO:tasks.ceph.mds.a:Started 2024-09-13T08:53:11.019 DEBUG:teuthology.parallel:result is None 2024-09-13T08:53:11.019 INFO:tasks.ceph.mds.b:Started 2024-09-13T08:53:11.019 DEBUG:teuthology.parallel:result is None 2024-09-13T08:53:11.020 INFO:tasks.ceph.mds.c:Started 2024-09-13T08:53:11.020 DEBUG:teuthology.parallel:result is None 2024-09-13T08:53:11.020 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T08:53:11.106 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-09-13T08:53:11.107 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:53:11.112 INFO:tasks.ceph.mds.b.smithi129.stdout:starting mds.b at 2024-09-13T08:53:11.113 INFO:tasks.ceph.mds.d.smithi129.stdout:starting mds.d at 2024-09-13T08:53:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 -- 172.21.15.44:0/3271275321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f637015e330 msgr2=0x7f637006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 --2- 172.21.15.44:0/3271275321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f637015e330 0x7f637006ec80 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f6358009900 tx=0x7f635801c7c0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 -- 172.21.15.44:0/3271275321 shutdown_connections 2024-09-13T08:53:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 --2- 172.21.15.44:0/3271275321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f637015e330 0x7f637006ec80 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 --2- 172.21.15.44:0/3271275321 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f637015df50 0x7f637006e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 -- 172.21.15.44:0/3271275321 >> 172.21.15.44:0/3271275321 conn(0x7f63700d1dc0 msgr2=0x7f63700d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 -- 172.21.15.44:0/3271275321 shutdown_connections 2024-09-13T08:53:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 -- 172.21.15.44:0/3271275321 wait complete. 2024-09-13T08:53:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 Processor -- start 2024-09-13T08:53:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 -- start start 2024-09-13T08:53:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f63700b11d0 0x7f6370155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f637015df50 0x7f63700b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637015e330 0x7f6370155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f63700d0800 con 0x7f637015df50 2024-09-13T08:53:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63700d0680 con 0x7f637015e330 2024-09-13T08:53:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6376a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f63700d0980 con 0x7f63700b11d0 2024-09-13T08:53:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6367e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f637015df50 0x7f63700b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6367e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f637015df50 0x7f63700b68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42180/0 (socket says 172.21.15.44:42180) 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f6367e00640 1 -- 172.21.15.44:0/3534864997 learned_addr learned my addr 172.21.15.44:0/3534864997 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.172+0000 7f636fe00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f63700b11d0 0x7f6370155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6374a00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637015e330 0x7f6370155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6367e00640 1 -- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f63700b11d0 msgr2=0x7f6370155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6367e00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f63700b11d0 0x7f6370155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6367e00640 1 -- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637015e330 msgr2=0x7f6370155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6367e00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637015e330 0x7f6370155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6367e00640 1 -- 172.21.15.44:0/3534864997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63700b5ee0 con 0x7f637015df50 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6374a00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637015e330 0x7f6370155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6367e00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f637015df50 0x7f63700b68c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6358009850 tx=0x7f63580a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:11.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f636cc00640 1 -- 172.21.15.44:0/3534864997 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63580b2040 con 0x7f637015df50 2024-09-13T08:53:11.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f636cc00640 1 -- 172.21.15.44:0/3534864997 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6358004430 con 0x7f637015df50 2024-09-13T08:53:11.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f636cc00640 1 -- 172.21.15.44:0/3534864997 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358003860 con 0x7f637015df50 2024-09-13T08:53:11.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6376a00640 1 -- 172.21.15.44:0/3534864997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63700b61c0 con 0x7f637015df50 2024-09-13T08:53:11.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6376a00640 1 -- 172.21.15.44:0/3534864997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63700c5ad0 con 0x7f637015df50 2024-09-13T08:53:11.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f6376a00640 1 -- 172.21.15.44:0/3534864997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63700c58f0 con 0x7f637015df50 2024-09-13T08:53:11.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.176+0000 7f636cc00640 1 -- 172.21.15.44:0/3534864997 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f6358003a00 con 0x7f637015df50 2024-09-13T08:53:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.180+0000 7f636cc00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f634c075140 0x7f634c077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.180+0000 7f636cc00640 1 -- 172.21.15.44:0/3534864997 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5055+0+0 (secure 0 0 0) 0x7f635812cef0 con 0x7f637015df50 2024-09-13T08:53:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.180+0000 7f636fe00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f634c075140 0x7f634c077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:11.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.180+0000 7f636fe00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f634c075140 0x7f634c077600 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6360006fd0 tx=0x7f6360008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:11.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.180+0000 7f636cc00640 1 -- 172.21.15.44:0/3534864997 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f63700c58f0 con 0x7f637015df50 2024-09-13T08:53:11.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.368+0000 7f6376a00640 1 -- 172.21.15.44:0/3534864997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f63700b1f50 con 0x7f637015df50 2024-09-13T08:53:11.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f636cc00640 1 -- 172.21.15.44:0/3534864997 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v138) ==== 97+0+0 (secure 0 0 0) 0x7f63700b1f50 con 0x7f637015df50 2024-09-13T08:53:11.568 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-09-13T08:53:11.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f6376a00640 1 -- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f634c075140 msgr2=0x7f634c077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:11.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f6376a00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f634c075140 0x7f634c077600 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6360006fd0 tx=0x7f6360008040 comp rx=0 tx=0).stop 2024-09-13T08:53:11.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f6376a00640 1 -- 172.21.15.44:0/3534864997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f637015df50 msgr2=0x7f63700b68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:11.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f6376a00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f637015df50 0x7f63700b68c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6358009850 tx=0x7f63580a5710 comp rx=0 tx=0).stop 2024-09-13T08:53:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f6376a00640 1 -- 172.21.15.44:0/3534864997 shutdown_connections 2024-09-13T08:53:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f6376a00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f634c075140 0x7f634c077600 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f6376a00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637015e330 0x7f6370155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f6376a00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f637015df50 0x7f63700b68c0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f6376a00640 1 --2- 172.21.15.44:0/3534864997 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f63700b11d0 0x7f6370155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.567+0000 7f6376a00640 1 -- 172.21.15.44:0/3534864997 >> 172.21.15.44:0/3534864997 conn(0x7f63700d1dc0 msgr2=0x7f637015d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.571+0000 7f6376a00640 1 -- 172.21.15.44:0/3534864997 shutdown_connections 2024-09-13T08:53:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.571+0000 7f6376a00640 1 -- 172.21.15.44:0/3534864997 wait complete. 2024-09-13T08:53:11.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T08:53:11.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 -- 172.21.15.44:0/3085801714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ac00b2760 msgr2=0x7f8ac01605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:11.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 --2- 172.21.15.44:0/3085801714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ac00b2760 0x7f8ac01605d0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0074e40 tx=0x7f8ab801cab0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 -- 172.21.15.44:0/3085801714 shutdown_connections 2024-09-13T08:53:11.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 --2- 172.21.15.44:0/3085801714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ac00b2760 0x7f8ac01605d0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 --2- 172.21.15.44:0/3085801714 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ac00cfe80 0x7f8ac00d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 -- 172.21.15.44:0/3085801714 >> 172.21.15.44:0/3085801714 conn(0x7f8ac00d1e40 msgr2=0x7f8ac00d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:11.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 -- 172.21.15.44:0/3085801714 shutdown_connections 2024-09-13T08:53:11.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 -- 172.21.15.44:0/3085801714 wait complete. 2024-09-13T08:53:11.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 Processor -- start 2024-09-13T08:53:11.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 -- start start 2024-09-13T08:53:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ac00b2760 0x7f8ac0152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ac00b2d00 0x7f8ac00b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ac00cfe80 0x7f8ac00c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac00b0990 con 0x7f8ac00b2d00 2024-09-13T08:53:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac00b0810 con 0x7f8ac00b2760 2024-09-13T08:53:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ac5000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac00b0b10 con 0x7f8ac00cfe80 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.739+0000 7f8ab5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ac00b2d00 0x7f8ac00b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ac00b2d00 0x7f8ac00b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42202/0 (socket says 172.21.15.44:42202) 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab5400640 1 -- 172.21.15.44:0/3021011797 learned_addr learned my addr 172.21.15.44:0/3021011797 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8abde00640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ac00b2760 0x7f8ac0152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8abe800640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ac00cfe80 0x7f8ac00c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab5400640 1 -- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ac00cfe80 msgr2=0x7f8ac00c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab5400640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ac00cfe80 0x7f8ac00c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab5400640 1 -- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ac00b2760 msgr2=0x7f8ac0152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab5400640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ac00b2760 0x7f8ac0152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab5400640 1 -- 172.21.15.44:0/3021011797 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac00b51d0 con 0x7f8ac00b2d00 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8abde00640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ac00b2760 0x7f8ac0152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8abe800640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ac00cfe80 0x7f8ac00c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab5400640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ac00b2d00 0x7f8ac00b6270 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800c450 tx=0x7f8aa800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:11.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab6a00640 1 -- 172.21.15.44:0/3021011797 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8012830 con 0x7f8ac00b2d00 2024-09-13T08:53:11.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab6a00640 1 -- 172.21.15.44:0/3021011797 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8aa8017070 con 0x7f8ac00b2d00 2024-09-13T08:53:11.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab6a00640 1 -- 172.21.15.44:0/3021011797 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8012d80 con 0x7f8ac00b2d00 2024-09-13T08:53:11.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ac5000640 1 -- 172.21.15.44:0/3021011797 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac00b54b0 con 0x7f8ac00b2d00 2024-09-13T08:53:11.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ac5000640 1 -- 172.21.15.44:0/3021011797 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac00b5a70 con 0x7f8ac00b2d00 2024-09-13T08:53:11.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ac5000640 1 -- 172.21.15.44:0/3021011797 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac00d0260 con 0x7f8ac00b2d00 2024-09-13T08:53:11.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.743+0000 7f8ab6a00640 1 -- 172.21.15.44:0/3021011797 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f8aa801c460 con 0x7f8ac00b2d00 2024-09-13T08:53:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.747+0000 7f8ab6a00640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a9c075110 0x7f8a9c0775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.747+0000 7f8ab6a00640 1 -- 172.21.15.44:0/3021011797 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4969+0+0 (secure 0 0 0) 0x7f8aa8092dc0 con 0x7f8ac00b2d00 2024-09-13T08:53:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.747+0000 7f8abde00640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a9c075110 0x7f8a9c0775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:11.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.747+0000 7f8abde00640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a9c075110 0x7f8a9c0775d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0005fd0 tx=0x7f8ab0005f20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:11.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.747+0000 7f8ab6a00640 1 -- 172.21.15.44:0/3021011797 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8aa805fa70 con 0x7f8ac00b2d00 2024-09-13T08:53:11.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.983+0000 7f8ac5000640 1 -- 172.21.15.44:0/3021011797 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8ac006e1f0 con 0x7f8ac00b2d00 2024-09-13T08:53:11.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.983+0000 7f8ab6a00640 1 -- 172.21.15.44:0/3021011797 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7f8aa805f370 con 0x7f8ac00b2d00 2024-09-13T08:53:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:11.986 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:53:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-09-13T08:53:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNma5OgBxAAddTlYccqHxVHOUogDm4rUw==", 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmnscMFBAAN6Lh+/CSG/xXTYv1283wiQ==", 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmJ3rmDhAA1IJ5/smluI1PZV/nfbgTSA==", 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:53:11.988 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm6dpMGxAA07IudMNMyAxOEPC0B7ttMw==", 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCU+eNmOzumIhAAvRGMrdh9D+PqLpAwdZiVTg==", 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-09-13T08:53:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCX+eNmd6DIORAA1b/CU8+ju9sM3GJwUeySNg==", 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCb+eNmawgnExAATMo2fr20MiwaAgG3Pxxnxw==", 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:53:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCe+eNmw5aDJhAAB+1qhM/Jwu5SxJ9RbG1iig==", 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-09-13T08:53:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCk+eNmFRjuLRAAOvb5AyrSpGxziKddjKo4FQ==", 2024-09-13T08:53:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:53:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:53:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-09-13T08:53:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCo+eNmxLU3BhAA2uVmI9UY/N4YzOZy5y950g==", 2024-09-13T08:53:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:53:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCr+eNmhkq1GhAAtrTOEBTgv+hZPP5EIutCSg==", 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCv+eNmEZPRAhAA7+xjzx8itiw4bVX0FI5CKg==", 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA==", 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmaLcmKRAAS+ToABBtrnRnQIoBfvc47w==", 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmTs5yJRAA3wXz9e4578d6rv/EZHImBA==", 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-09-13T08:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNmvNEXLxAAXzZBM4y8+dGTp1po96fL3w==", 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmDD2LJxAA3s0jEb2EfPsLcu0w145G3Q==", 2024-09-13T08:53:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCR+eNm7qF4AxAADP/UTTWidfIF+oNLsl9AOQ==", 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQ+eNmTAMbOxAAX5cw4uYZ7cddYKNMwaGmIg==", 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T08:53:12.003 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:53:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 -- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a9c075110 msgr2=0x7f8a9c0775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a9c075110 0x7f8a9c0775d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0005fd0 tx=0x7f8ab0005f20 comp rx=0 tx=0).stop 2024-09-13T08:53:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 -- 172.21.15.44:0/3021011797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ac00b2d00 msgr2=0x7f8ac00b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ac00b2d00 0x7f8ac00b6270 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800c450 tx=0x7f8aa800c910 comp rx=0 tx=0).stop 2024-09-13T08:53:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 -- 172.21.15.44:0/3021011797 shutdown_connections 2024-09-13T08:53:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a9c075110 0x7f8a9c0775d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ac00cfe80 0x7f8ac00c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ac00b2d00 0x7f8ac00b6270 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 --2- 172.21.15.44:0/3021011797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ac00b2760 0x7f8ac0152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 -- 172.21.15.44:0/3021011797 >> 172.21.15.44:0/3021011797 conn(0x7f8ac00d1e40 msgr2=0x7f8ac0070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 -- 172.21.15.44:0/3021011797 shutdown_connections 2024-09-13T08:53:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:11.987+0000 7f8ac5000640 1 -- 172.21.15.44:0/3021011797 wait complete. 2024-09-13T08:53:12.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T08:53:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.155+0000 7f9e6ae00640 1 -- 172.21.15.44:0/1370847975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cecf0 msgr2=0x7f9e640cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.155+0000 7f9e6ae00640 1 --2- 172.21.15.44:0/1370847975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cecf0 0x7f9e640cc940 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009900 tx=0x7f9e4c01c7c0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.155+0000 7f9e6ae00640 1 -- 172.21.15.44:0/1370847975 shutdown_connections 2024-09-13T08:53:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.155+0000 7f9e6ae00640 1 --2- 172.21.15.44:0/1370847975 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e6415c130 0x7f9e640cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.155+0000 7f9e6ae00640 1 --2- 172.21.15.44:0/1370847975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cecf0 0x7f9e640cc940 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.155+0000 7f9e6ae00640 1 -- 172.21.15.44:0/1370847975 >> 172.21.15.44:0/1370847975 conn(0x7f9e640d1ea0 msgr2=0x7f9e640d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.155+0000 7f9e6ae00640 1 -- 172.21.15.44:0/1370847975 shutdown_connections 2024-09-13T08:53:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.155+0000 7f9e6ae00640 1 -- 172.21.15.44:0/1370847975 wait complete. 2024-09-13T08:53:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.155+0000 7f9e6ae00640 1 Processor -- start 2024-09-13T08:53:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e6ae00640 1 -- start start 2024-09-13T08:53:12.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e6ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cecf0 0x7f9e6415b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:12.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e6ae00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e6415bd50 0x7f9e640c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e6ae00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e6415c130 0x7f9e640ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e6ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e64169540 con 0x7f9e640cecf0 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e6ae00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641693c0 con 0x7f9e6415bd50 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e6ae00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9e641696c0 con 0x7f9e6415c130 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cecf0 0x7f9e6415b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cecf0 0x7f9e6415b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42962/0 (socket says 172.21.15.44:42962) 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e63e00640 1 -- 172.21.15.44:0/2912445224 learned_addr learned my addr 172.21.15.44:0/2912445224 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e5be00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e6415bd50 0x7f9e640c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e68e00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e6415c130 0x7f9e640ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e63e00640 1 -- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e6415c130 msgr2=0x7f9e640ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e63e00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e6415c130 0x7f9e640ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e63e00640 1 -- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e6415bd50 msgr2=0x7f9e640c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e63e00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e6415bd50 0x7f9e640c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e63e00640 1 -- 172.21.15.44:0/2912445224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e64159c80 con 0x7f9e640cecf0 2024-09-13T08:53:12.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e63e00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cecf0 0x7f9e6415b000 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400c530 tx=0x7f9e5400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:12.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e60c00640 1 -- 172.21.15.44:0/2912445224 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54017070 con 0x7f9e640cecf0 2024-09-13T08:53:12.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e60c00640 1 -- 172.21.15.44:0/2912445224 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e540053c0 con 0x7f9e640cecf0 2024-09-13T08:53:12.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e6ae00640 1 -- 172.21.15.44:0/2912445224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e64159f60 con 0x7f9e640cecf0 2024-09-13T08:53:12.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e60c00640 1 -- 172.21.15.44:0/2912445224 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54012890 con 0x7f9e640cecf0 2024-09-13T08:53:12.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e6ae00640 1 -- 172.21.15.44:0/2912445224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6415a520 con 0x7f9e640cecf0 2024-09-13T08:53:12.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.159+0000 7f9e6ae00640 1 -- 172.21.15.44:0/2912445224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e640ce5e0 con 0x7f9e640cecf0 2024-09-13T08:53:12.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.163+0000 7f9e60c00640 1 -- 172.21.15.44:0/2912445224 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f9e5401a070 con 0x7f9e640cecf0 2024-09-13T08:53:12.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.163+0000 7f9e60c00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e40075110 0x7f9e400775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:12.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.163+0000 7f9e60c00640 1 -- 172.21.15.44:0/2912445224 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4969+0+0 (secure 0 0 0) 0x7f9e54092320 con 0x7f9e640cecf0 2024-09-13T08:53:12.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.163+0000 7f9e5be00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e40075110 0x7f9e400775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:12.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.163+0000 7f9e5be00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e40075110 0x7f9e400775d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0af000 tx=0x7f9e4c01b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:12.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.167+0000 7f9e60c00640 1 -- 172.21.15.44:0/2912445224 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f9e5405e900 con 0x7f9e640cecf0 2024-09-13T08:53:12.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 -- 172.21.15.44:0/2912445224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9e640b2d50 con 0x7f9e640cecf0 2024-09-13T08:53:12.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e60c00640 1 -- 172.21.15.44:0/2912445224 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v140) ==== 84+0+65 (secure 0 0 0) 0x7f9e54063330 con 0x7f9e640cecf0 2024-09-13T08:53:12.348 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:12.348 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:53:12.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:12.348 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-09-13T08:53:12.348 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-09-13T08:53:12.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:12.348 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:53:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 -- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e40075110 msgr2=0x7f9e400775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e40075110 0x7f9e400775d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0af000 tx=0x7f9e4c01b000 comp rx=0 tx=0).stop 2024-09-13T08:53:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 -- 172.21.15.44:0/2912445224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cecf0 msgr2=0x7f9e6415b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cecf0 0x7f9e6415b000 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400c530 tx=0x7f9e5400c9f0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 -- 172.21.15.44:0/2912445224 shutdown_connections 2024-09-13T08:53:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9e40075110 0x7f9e400775d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9e6415c130 0x7f9e640ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e6415bd50 0x7f9e640c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 --2- 172.21.15.44:0/2912445224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cecf0 0x7f9e6415b000 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 -- 172.21.15.44:0/2912445224 >> 172.21.15.44:0/2912445224 conn(0x7f9e640d1ea0 msgr2=0x7f9e640d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 -- 172.21.15.44:0/2912445224 shutdown_connections 2024-09-13T08:53:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.347+0000 7f9e6ae00640 1 -- 172.21.15.44:0/2912445224 wait complete. 2024-09-13T08:53:12.369 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T08:53:12.370 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T08:53:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.519+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3363856207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40734f0 msgr2=0x7f8ef415a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.519+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/3363856207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40734f0 0x7f8ef415a9d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4072800 tx=0x7f8ee401c800 comp rx=0 tx=0).stop 2024-09-13T08:53:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.519+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3363856207 shutdown_connections 2024-09-13T08:53:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.519+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/3363856207 >> v2:172.21.15.129:3300/0 conn(0x7f8ef415e680 0x7f8ef4165350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.519+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/3363856207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40734f0 0x7f8ef415a9d0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.519+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3363856207 >> 172.21.15.44:0/3363856207 conn(0x7f8ef40cfbc0 msgr2=0x7f8ef40cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:12.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.519+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3363856207 shutdown_connections 2024-09-13T08:53:12.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.519+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3363856207 wait complete. 2024-09-13T08:53:12.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef9a00640 1 Processor -- start 2024-09-13T08:53:12.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef9a00640 1 -- start start 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40734f0 0x7f8ef41fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef9a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ef40cf440 0x7f8ef409d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef9a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ef415e680 0x7f8ef406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef9a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41682c0 con 0x7f8ef40734f0 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef9a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef406fe40 con 0x7f8ef40cf440 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef9a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef4168440 con 0x7f8ef415e680 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40734f0 0x7f8ef41fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40734f0 0x7f8ef41fbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42990/0 (socket says 172.21.15.44:42990) 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef2a00640 1 -- 172.21.15.44:0/3905757650 learned_addr learned my addr 172.21.15.44:0/3905757650 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8eebe00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ef40cf440 0x7f8ef409d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef3400640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ef415e680 0x7f8ef406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef2a00640 1 -- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ef415e680 msgr2=0x7f8ef406f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef2a00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ef415e680 0x7f8ef406f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef2a00640 1 -- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ef40cf440 msgr2=0x7f8ef409d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef2a00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ef40cf440 0x7f8ef409d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef2a00640 1 -- 172.21.15.44:0/3905757650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef409da10 con 0x7f8ef40734f0 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef3400640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ef415e680 0x7f8ef406f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef2a00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40734f0 0x7f8ef41fbde0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40b0040 tx=0x7f8ee40a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8eeaa00640 1 -- 172.21.15.44:0/3905757650 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40b2040 con 0x7f8ef40734f0 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8eeaa00640 1 -- 172.21.15.44:0/3905757650 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ee40043e0 con 0x7f8ef40734f0 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8eeaa00640 1 -- 172.21.15.44:0/3905757650 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4005290 con 0x7f8ef40734f0 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3905757650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef409dcf0 con 0x7f8ef40734f0 2024-09-13T08:53:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.523+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3905757650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef41fb8c0 con 0x7f8ef40734f0 2024-09-13T08:53:12.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.527+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3905757650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef4069410 con 0x7f8ef40734f0 2024-09-13T08:53:12.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.527+0000 7f8eeaa00640 1 -- 172.21.15.44:0/3905757650 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f8ee4005430 con 0x7f8ef40734f0 2024-09-13T08:53:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.527+0000 7f8eeaa00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ed0075110 0x7f8ed00775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.527+0000 7f8eeaa00640 1 -- 172.21.15.44:0/3905757650 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4969+0+0 (secure 0 0 0) 0x7f8ee412cb90 con 0x7f8ef40734f0 2024-09-13T08:53:12.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.527+0000 7f8eebe00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ed0075110 0x7f8ed00775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:12.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.527+0000 7f8eebe00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ed0075110 0x7f8ed00775d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8ef41f8f10 tx=0x7f8edc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:12.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.531+0000 7f8eeaa00640 1 -- 172.21.15.44:0/3905757650 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8ee40164b0 con 0x7f8ef40734f0 2024-09-13T08:53:12.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:12.711+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3905757650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8ef415a9d0 con 0x7f8ef40734f0 2024-09-13T08:53:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.575+0000 7f8eeaa00640 1 -- 172.21.15.44:0/3905757650 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v141) ==== 138+0+0 (secure 0 0 0) 0x7f8ee40f9840 con 0x7f8ef40734f0 2024-09-13T08:53:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-09-13T08:53:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ed0075110 msgr2=0x7f8ed00775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ed0075110 0x7f8ed00775d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8ef41f8f10 tx=0x7f8edc009040 comp rx=0 tx=0).stop 2024-09-13T08:53:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3905757650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40734f0 msgr2=0x7f8ef41fbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40734f0 0x7f8ef41fbde0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40b0040 tx=0x7f8ee40a56f0 comp rx=0 tx=0).stop 2024-09-13T08:53:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3905757650 shutdown_connections 2024-09-13T08:53:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ed0075110 0x7f8ed00775d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8ef415e680 0x7f8ef406f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ef40cf440 0x7f8ef409d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/3905757650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40734f0 0x7f8ef41fbde0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3905757650 >> 172.21.15.44:0/3905757650 conn(0x7f8ef40cfbc0 msgr2=0x7f8ef4154380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3905757650 shutdown_connections 2024-09-13T08:53:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.579+0000 7f8ef9a00640 1 -- 172.21.15.44:0/3905757650 wait complete. 2024-09-13T08:53:13.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T08:53:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3128588111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0cecf0 msgr2=0x7f5acc0cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 --2- 172.21.15.44:0/3128588111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0cecf0 0x7f5acc0cc940 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0d0780 tx=0x7f5ab401c7c0 comp rx=0 tx=0).stop 2024-09-13T08:53:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3128588111 shutdown_connections 2024-09-13T08:53:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 --2- 172.21.15.44:0/3128588111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0cecf0 0x7f5acc0cc940 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3128588111 >> 172.21.15.44:0/3128588111 conn(0x7f5acc0d1ea0 msgr2=0x7f5acc0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3128588111 shutdown_connections 2024-09-13T08:53:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3128588111 wait complete. 2024-09-13T08:53:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 Processor -- start 2024-09-13T08:53:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 -- start start 2024-09-13T08:53:13.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0cecf0 0x7f5acc1f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:13.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5acc15bd50 0x7f5acc0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:13.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5acc15c130 0x7f5acc14a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:13.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc16a5b0 con 0x7f5acc0cecf0 2024-09-13T08:53:13.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc16a430 con 0x7f5acc15bd50 2024-09-13T08:53:13.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5ad1a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5acc16a730 con 0x7f5acc15c130 2024-09-13T08:53:13.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5acaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0cecf0 0x7f5acc1f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:13.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5acaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0cecf0 0x7f5acc1f81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43000/0 (socket says 172.21.15.44:43000) 2024-09-13T08:53:13.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5acaa00640 1 -- 172.21.15.44:0/3513045168 learned_addr learned my addr 172.21.15.44:0/3513045168 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:13.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5acaa00640 1 -- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5acc15c130 msgr2=0x7f5acc14a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:13.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5acaa00640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5acc15c130 0x7f5acc14a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:13.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5acaa00640 1 -- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5acc15bd50 msgr2=0x7f5acc0cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:13.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5acaa00640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5acc15bd50 0x7f5acc0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:13.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.767+0000 7f5acaa00640 1 -- 172.21.15.44:0/3513045168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc14a980 con 0x7f5acc0cecf0 2024-09-13T08:53:13.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.771+0000 7f5acaa00640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0cecf0 0x7f5acc1f81d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00c500 tx=0x7f5abc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:13.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.771+0000 7f5ac3400640 1 -- 172.21.15.44:0/3513045168 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc017070 con 0x7f5acc0cecf0 2024-09-13T08:53:13.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.771+0000 7f5ac3400640 1 -- 172.21.15.44:0/3513045168 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5abc0053c0 con 0x7f5acc0cecf0 2024-09-13T08:53:13.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.771+0000 7f5ac3400640 1 -- 172.21.15.44:0/3513045168 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc012890 con 0x7f5acc0cecf0 2024-09-13T08:53:13.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.771+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3513045168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc1f8800 con 0x7f5acc0cecf0 2024-09-13T08:53:13.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.771+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3513045168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc1f8dc0 con 0x7f5acc0cecf0 2024-09-13T08:53:13.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.771+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3513045168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5acc0ce5e0 con 0x7f5acc0cecf0 2024-09-13T08:53:13.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.771+0000 7f5ac3400640 1 -- 172.21.15.44:0/3513045168 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f5abc01a070 con 0x7f5acc0cecf0 2024-09-13T08:53:13.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.771+0000 7f5ac3400640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8075110 0x7f5aa80775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:13.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.771+0000 7f5ac3400640 1 -- 172.21.15.44:0/3513045168 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5339+0+0 (secure 0 0 0) 0x7f5abc092480 con 0x7f5acc0cecf0 2024-09-13T08:53:13.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.775+0000 7f5ac2000640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8075110 0x7f5aa80775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.775+0000 7f5ac2000640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8075110 0x7f5aa80775d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40af000 tx=0x7f5ab4019560 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:13.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.775+0000 7f5ac3400640 1 -- 172.21.15.44:0/3513045168 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5abc021080 con 0x7f5acc0cecf0 2024-09-13T08:53:13.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:13.951+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3513045168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5acc0b2db0 con 0x7f5acc0cecf0 2024-09-13T08:53:14.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.583+0000 7f5ac3400640 1 -- 172.21.15.44:0/3513045168 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v142) ==== 170+0+0 (secure 0 0 0) 0x7f5abc05f1d0 con 0x7f5acc0cecf0 2024-09-13T08:53:14.584 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-09-13T08:53:14.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.583+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8075110 msgr2=0x7f5aa80775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:14.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.583+0000 7f5ad1a00640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8075110 0x7f5aa80775d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40af000 tx=0x7f5ab4019560 comp rx=0 tx=0).stop 2024-09-13T08:53:14.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.583+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3513045168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0cecf0 msgr2=0x7f5acc1f81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:14.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.583+0000 7f5ad1a00640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0cecf0 0x7f5acc1f81d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00c500 tx=0x7f5abc00c9c0 comp rx=0 tx=0).stop 2024-09-13T08:53:14.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.587+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3513045168 shutdown_connections 2024-09-13T08:53:14.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.587+0000 7f5ad1a00640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5aa8075110 0x7f5aa80775d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:14.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.587+0000 7f5ad1a00640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5acc15c130 0x7f5acc14a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:14.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.587+0000 7f5ad1a00640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5acc15bd50 0x7f5acc0cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:14.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.587+0000 7f5ad1a00640 1 --2- 172.21.15.44:0/3513045168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0cecf0 0x7f5acc1f81d0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:14.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.587+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3513045168 >> 172.21.15.44:0/3513045168 conn(0x7f5acc0d1ea0 msgr2=0x7f5acc0b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:14.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.587+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3513045168 shutdown_connections 2024-09-13T08:53:14.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.587+0000 7f5ad1a00640 1 -- 172.21.15.44:0/3513045168 wait complete. 2024-09-13T08:53:14.626 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T08:53:14.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.775+0000 7f6435200640 1 -- 172.21.15.44:0/1556720071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6430073980 msgr2=0x7f643006cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:14.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.775+0000 7f6435200640 1 --2- 172.21.15.44:0/1556720071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6430073980 0x7f643006cc20 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f64180098f0 tx=0x7f641801c8e0 comp rx=0 tx=0).stop 2024-09-13T08:53:14.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.775+0000 7f6435200640 1 -- 172.21.15.44:0/1556720071 shutdown_connections 2024-09-13T08:53:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.775+0000 7f6435200640 1 --2- 172.21.15.44:0/1556720071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6430073980 0x7f643006cc20 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.775+0000 7f6435200640 1 --2- 172.21.15.44:0/1556720071 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f643015ddd0 0x7f64300c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.775+0000 7f6435200640 1 -- 172.21.15.44:0/1556720071 >> 172.21.15.44:0/1556720071 conn(0x7f64300cce90 msgr2=0x7f64300cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 -- 172.21.15.44:0/1556720071 shutdown_connections 2024-09-13T08:53:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 -- 172.21.15.44:0/1556720071 wait complete. 2024-09-13T08:53:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 Processor -- start 2024-09-13T08:53:14.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 -- start start 2024-09-13T08:53:14.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643006d160 0x7f64300c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:14.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6430073980 0x7f64300b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:14.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f643015ddd0 0x7f643015a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:14.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f643016b020 con 0x7f6430073980 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f643016aea0 con 0x7f643006d160 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f643016b1a0 con 0x7f643015ddd0 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6427e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6430073980 0x7f64300b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6427e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6430073980 0x7f64300b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43020/0 (socket says 172.21.15.44:43020) 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6427e00640 1 -- 172.21.15.44:0/1837904104 learned_addr learned my addr 172.21.15.44:0/1837904104 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f642de00640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643006d160 0x7f64300c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f642e800640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f643015ddd0 0x7f643015a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6427e00640 1 -- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f643015ddd0 msgr2=0x7f643015a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6427e00640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f643015ddd0 0x7f643015a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6427e00640 1 -- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643006d160 msgr2=0x7f64300c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6427e00640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643006d160 0x7f64300c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:14.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6427e00640 1 -- 172.21.15.44:0/1837904104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6430150440 con 0x7f6430073980 2024-09-13T08:53:14.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6427e00640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6430073980 0x7f64300b5d70 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f6418000c10 tx=0x7f64180a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:14.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6426000640 1 -- 172.21.15.44:0/1837904104 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64180bb070 con 0x7f6430073980 2024-09-13T08:53:14.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6426000640 1 -- 172.21.15.44:0/1837904104 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6418004470 con 0x7f6430073980 2024-09-13T08:53:14.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6426000640 1 -- 172.21.15.44:0/1837904104 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64180b2040 con 0x7f6430073980 2024-09-13T08:53:14.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.779+0000 7f6435200640 1 -- 172.21.15.44:0/1837904104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6430150720 con 0x7f6430073980 2024-09-13T08:53:14.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.783+0000 7f6435200640 1 -- 172.21.15.44:0/1837904104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64300c7990 con 0x7f6430073980 2024-09-13T08:53:14.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.783+0000 7f6435200640 1 -- 172.21.15.44:0/1837904104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643006cc20 con 0x7f6430073980 2024-09-13T08:53:14.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.783+0000 7f6426000640 1 -- 172.21.15.44:0/1837904104 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f6418005290 con 0x7f6430073980 2024-09-13T08:53:14.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.787+0000 7f6426000640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f640c075110 0x7f640c0775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:14.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.787+0000 7f6426000640 1 -- 172.21.15.44:0/1837904104 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5721+0+0 (secure 0 0 0) 0x7f641812cd50 con 0x7f6430073980 2024-09-13T08:53:14.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.787+0000 7f642de00640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f640c075110 0x7f640c0775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:14.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.787+0000 7f642de00640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f640c075110 0x7f640c0775d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f642000b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:14.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:14.787+0000 7f6426000640 1 -- 172.21.15.44:0/1837904104 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f64180c4020 con 0x7f6430073980 2024-09-13T08:53:15.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.003+0000 7f6435200640 1 -- 172.21.15.44:0/1837904104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f643015e730 con 0x7f6430073980 2024-09-13T08:53:15.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.599+0000 7f6426000640 1 -- 172.21.15.44:0/1837904104 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v91) ==== 174+0+0 (secure 0 0 0) 0x7f64180164d0 con 0x7f6430073980 2024-09-13T08:53:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 20 and data pool 21 2024-09-13T08:53:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 -- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f640c075110 msgr2=0x7f640c0775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f640c075110 0x7f640c0775d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f642000b390 comp rx=0 tx=0).stop 2024-09-13T08:53:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 -- 172.21.15.44:0/1837904104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6430073980 msgr2=0x7f64300b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:15.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6430073980 0x7f64300b5d70 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f6418000c10 tx=0x7f64180a5690 comp rx=0 tx=0).stop 2024-09-13T08:53:15.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 -- 172.21.15.44:0/1837904104 shutdown_connections 2024-09-13T08:53:15.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f640c075110 0x7f640c0775d0 secure :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f642000b390 comp rx=0 tx=0).stop 2024-09-13T08:53:15.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f643015ddd0 0x7f643015a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6430073980 0x7f64300b5d70 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 --2- 172.21.15.44:0/1837904104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643006d160 0x7f64300c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 -- 172.21.15.44:0/1837904104 >> 172.21.15.44:0/1837904104 conn(0x7f64300cce90 msgr2=0x7f643006f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:15.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.615+0000 7f6435200640 1 -- 172.21.15.44:0/1837904104 shutdown_connections 2024-09-13T08:53:15.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.619+0000 7f6435200640 1 -- 172.21.15.44:0/1837904104 wait complete. 2024-09-13T08:53:15.648 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:53:15.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.795+0000 7f38a7a00640 1 -- 172.21.15.44:0/3448774777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38a00d28e0 msgr2=0x7f38a00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:15.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.795+0000 7f38a7a00640 1 --2- 172.21.15.44:0/3448774777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38a00d28e0 0x7f38a00cebb0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f38a00745d0 tx=0x7f389801ccf0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.795+0000 7f38a7a00640 1 -- 172.21.15.44:0/3448774777 shutdown_connections 2024-09-13T08:53:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.795+0000 7f38a7a00640 1 --2- 172.21.15.44:0/3448774777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38a00d28e0 0x7f38a00cebb0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.795+0000 7f38a7a00640 1 --2- 172.21.15.44:0/3448774777 >> v2:172.21.15.129:3300/0 conn(0x7f38a015bea0 0x7f38a00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.795+0000 7f38a7a00640 1 -- 172.21.15.44:0/3448774777 >> 172.21.15.44:0/3448774777 conn(0x7f38a00d1dc0 msgr2=0x7f38a00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.795+0000 7f38a7a00640 1 -- 172.21.15.44:0/3448774777 shutdown_connections 2024-09-13T08:53:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.795+0000 7f38a7a00640 1 -- 172.21.15.44:0/3448774777 wait complete. 2024-09-13T08:53:15.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.795+0000 7f38a7a00640 1 Processor -- start 2024-09-13T08:53:15.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.795+0000 7f38a7a00640 1 -- start start 2024-09-13T08:53:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a7a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38a00cdb30 0x7f38a00b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a7a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38a00d28e0 0x7f38a00b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a7a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38a015bea0 0x7f38a00cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a7a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0168fc0 con 0x7f38a00d28e0 2024-09-13T08:53:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a7a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0168e40 con 0x7f38a015bea0 2024-09-13T08:53:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a7a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f38a0169140 con 0x7f38a00cdb30 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38a00d28e0 0x7f38a00b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38a00d28e0 0x7f38a00b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43040/0 (socket says 172.21.15.44:43040) 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389fe00640 1 -- 172.21.15.44:0/470533080 learned_addr learned my addr 172.21.15.44:0/470533080 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a5000640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38a00cdb30 0x7f38a00b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a5a00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38a015bea0 0x7f38a00cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389fe00640 1 -- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38a00cdb30 msgr2=0x7f38a00b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389fe00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38a00cdb30 0x7f38a00b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389fe00640 1 -- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38a015bea0 msgr2=0x7f38a00cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389fe00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38a015bea0 0x7f38a00cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389fe00640 1 -- 172.21.15.44:0/470533080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a00b3210 con 0x7f38a00d28e0 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389fe00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38a00d28e0 0x7f38a00b1530 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c500 tx=0x7f388c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389cc00640 1 -- 172.21.15.44:0/470533080 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c017070 con 0x7f38a00d28e0 2024-09-13T08:53:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389cc00640 1 -- 172.21.15.44:0/470533080 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f388c0053c0 con 0x7f38a00d28e0 2024-09-13T08:53:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f389cc00640 1 -- 172.21.15.44:0/470533080 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c012890 con 0x7f38a00d28e0 2024-09-13T08:53:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a7a00640 1 -- 172.21.15.44:0/470533080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a00b3490 con 0x7f38a00d28e0 2024-09-13T08:53:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a7a00640 1 -- 172.21.15.44:0/470533080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a00b3a50 con 0x7f38a00d28e0 2024-09-13T08:53:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.799+0000 7f38a7a00640 1 -- 172.21.15.44:0/470533080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a00cc9d0 con 0x7f38a00d28e0 2024-09-13T08:53:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.803+0000 7f389cc00640 1 -- 172.21.15.44:0/470533080 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f388c01a070 con 0x7f38a00d28e0 2024-09-13T08:53:15.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.803+0000 7f389cc00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3880075110 0x7f38800775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:15.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.803+0000 7f389cc00640 1 -- 172.21.15.44:0/470533080 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5821+0+0 (secure 0 0 0) 0x7f388c0922a0 con 0x7f38a00d28e0 2024-09-13T08:53:15.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.803+0000 7f38a5000640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3880075110 0x7f38800775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:15.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.803+0000 7f38a5000640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3880075110 0x7f38800775d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f38940058d0 tx=0x7f3894005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:15.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.807+0000 7f389cc00640 1 -- 172.21.15.44:0/470533080 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f38a00b3210 con 0x7f38a00d28e0 2024-09-13T08:53:15.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 -- 172.21.15.44:0/470533080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38a006e6f0 con 0x7f38a00d28e0 2024-09-13T08:53:15.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f389cc00640 1 -- 172.21.15.44:0/470533080 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+16941 (secure 0 0 0) 0x7f388c05ef70 con 0x7f38a00d28e0 2024-09-13T08:53:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:15.989 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":143,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:53:15.007854+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:53:12.727898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-13T08:53:13.969013+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:53:15.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 -- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3880075110 msgr2=0x7f38800775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:15.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3880075110 0x7f38800775d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f38940058d0 tx=0x7f3894005860 comp rx=0 tx=0).stop 2024-09-13T08:53:15.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 -- 172.21.15.44:0/470533080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38a00d28e0 msgr2=0x7f38a00b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:15.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38a00d28e0 0x7f38a00b1530 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c500 tx=0x7f388c00c9c0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 -- 172.21.15.44:0/470533080 shutdown_connections 2024-09-13T08:53:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3880075110 0x7f38800775d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38a015bea0 0x7f38a00cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38a00d28e0 0x7f38a00b1530 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 --2- 172.21.15.44:0/470533080 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38a00cdb30 0x7f38a00b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 -- 172.21.15.44:0/470533080 >> 172.21.15.44:0/470533080 conn(0x7f38a00d1dc0 msgr2=0x7f38a00b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 -- 172.21.15.44:0/470533080 shutdown_connections 2024-09-13T08:53:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:15.987+0000 7f38a7a00640 1 -- 172.21.15.44:0/470533080 wait complete. 2024-09-13T08:53:16.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:53:16.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 -- 172.21.15.44:0/726453393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6de00cf660 msgr2=0x7f6de00b81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:16.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 --2- 172.21.15.44:0/726453393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6de00cf660 0x7f6de00b81c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009bc0 tx=0x7f6dd801cba0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 -- 172.21.15.44:0/726453393 shutdown_connections 2024-09-13T08:53:16.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 --2- 172.21.15.44:0/726453393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6de00cf660 0x7f6de00b81c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 -- 172.21.15.44:0/726453393 >> 172.21.15.44:0/726453393 conn(0x7f6de00d3fe0 msgr2=0x7f6de00d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:16.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 -- 172.21.15.44:0/726453393 shutdown_connections 2024-09-13T08:53:16.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 -- 172.21.15.44:0/726453393 wait complete. 2024-09-13T08:53:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 Processor -- start 2024-09-13T08:53:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 -- start start 2024-09-13T08:53:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de00cf660 0x7f6de01523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6de00d2060 0x7f6de01568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6de00d2b50 0x7f6de015c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6de016c460 con 0x7f6de00cf660 2024-09-13T08:53:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6de016c2e0 con 0x7f6de00d2060 2024-09-13T08:53:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de7000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6de016c5e0 con 0x7f6de00d2b50 2024-09-13T08:53:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de00cf660 0x7f6de01523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de00cf660 0x7f6de01523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43046/0 (socket says 172.21.15.44:43046) 2024-09-13T08:53:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6ddfe00640 1 -- 172.21.15.44:0/4229757882 learned_addr learned my addr 172.21.15.44:0/4229757882 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6ddfe00640 1 -- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6de00d2b50 msgr2=0x7f6de015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6ddf400640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6de00d2060 0x7f6de01568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de5000640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6de00d2b50 0x7f6de015c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6ddfe00640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6de00d2b50 0x7f6de015c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6ddfe00640 1 -- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6de00d2060 msgr2=0x7f6de01568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6ddfe00640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6de00d2060 0x7f6de01568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6ddfe00640 1 -- 172.21.15.44:0/4229757882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de0152d20 con 0x7f6de00cf660 2024-09-13T08:53:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6ddf400640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6de00d2060 0x7f6de01568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.159+0000 7f6de5000640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6de00d2b50 0x7f6de015c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6ddfe00640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de00cf660 0x7f6de01523c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000c490 tx=0x7f6dd000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6dd7e00640 1 -- 172.21.15.44:0/4229757882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00052d0 con 0x7f6de00cf660 2024-09-13T08:53:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6dd7e00640 1 -- 172.21.15.44:0/4229757882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dd0005490 con 0x7f6de00cf660 2024-09-13T08:53:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6dd7e00640 1 -- 172.21.15.44:0/4229757882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0012870 con 0x7f6de00cf660 2024-09-13T08:53:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6de7000640 1 -- 172.21.15.44:0/4229757882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de00cde90 con 0x7f6de00cf660 2024-09-13T08:53:16.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6de7000640 1 -- 172.21.15.44:0/4229757882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de0157f60 con 0x7f6de00cf660 2024-09-13T08:53:16.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6de7000640 1 -- 172.21.15.44:0/4229757882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de00d2440 con 0x7f6de00cf660 2024-09-13T08:53:16.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6dd7e00640 1 -- 172.21.15.44:0/4229757882 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f6dd0012a10 con 0x7f6de00cf660 2024-09-13T08:53:16.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6dd7e00640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dc0075140 0x7f6dc0077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:16.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6dd7e00640 1 -- 172.21.15.44:0/4229757882 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5821+0+0 (secure 0 0 0) 0x7f6dd0092b30 con 0x7f6de00cf660 2024-09-13T08:53:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.163+0000 7f6ddf400640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dc0075140 0x7f6dc0077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.167+0000 7f6ddf400640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dc0075140 0x7f6dc0077600 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000c10 tx=0x7f6dcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:16.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.167+0000 7f6dd7e00640 1 -- 172.21.15.44:0/4229757882 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6de0152d20 con 0x7f6de00cf660 2024-09-13T08:53:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.347+0000 7f6de7000640 1 -- 172.21.15.44:0/4229757882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6de00ce4a0 con 0x7f6de00cf660 2024-09-13T08:53:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6dd7e00640 1 -- 172.21.15.44:0/4229757882 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+16941 (secure 0 0 0) 0x7f6dd005f7e0 con 0x7f6de00cf660 2024-09-13T08:53:16.351 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:16.352 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":143,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:53:15.007854+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:53:12.727898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-13T08:53:13.969013+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 -- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dc0075140 msgr2=0x7f6dc0077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dc0075140 0x7f6dc0077600 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000c10 tx=0x7f6dcc009040 comp rx=0 tx=0).stop 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 -- 172.21.15.44:0/4229757882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de00cf660 msgr2=0x7f6de01523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de00cf660 0x7f6de01523c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000c490 tx=0x7f6dd000c950 comp rx=0 tx=0).stop 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 -- 172.21.15.44:0/4229757882 shutdown_connections 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6dc0075140 0x7f6dc0077600 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6de00d2b50 0x7f6de015c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6de00d2060 0x7f6de01568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 --2- 172.21.15.44:0/4229757882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de00cf660 0x7f6de01523c0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 -- 172.21.15.44:0/4229757882 >> 172.21.15.44:0/4229757882 conn(0x7f6de00d3fe0 msgr2=0x7f6de00d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:16.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 -- 172.21.15.44:0/4229757882 shutdown_connections 2024-09-13T08:53:16.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.351+0000 7f6de7000640 1 -- 172.21.15.44:0/4229757882 wait complete. 2024-09-13T08:53:16.375 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T08:53:16.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.523+0000 7f86f3800640 1 -- 172.21.15.44:0/3476784050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86ec1616f0 msgr2=0x7f86ec06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:16.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.523+0000 7f86f3800640 1 --2- 172.21.15.44:0/3476784050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86ec1616f0 0x7f86ec06dd30 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009900 tx=0x7f86d801c870 comp rx=0 tx=0).stop 2024-09-13T08:53:16.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.523+0000 7f86f3800640 1 -- 172.21.15.44:0/3476784050 shutdown_connections 2024-09-13T08:53:16.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.523+0000 7f86f3800640 1 --2- 172.21.15.44:0/3476784050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86ec1616f0 0x7f86ec06dd30 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.523+0000 7f86f3800640 1 --2- 172.21.15.44:0/3476784050 >> v2:172.21.15.129:3300/0 conn(0x7f86ec161310 0x7f86ec0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.523+0000 7f86f3800640 1 -- 172.21.15.44:0/3476784050 >> 172.21.15.44:0/3476784050 conn(0x7f86ec0d1e00 msgr2=0x7f86ec0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:16.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.523+0000 7f86f3800640 1 -- 172.21.15.44:0/3476784050 shutdown_connections 2024-09-13T08:53:16.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.523+0000 7f86f3800640 1 -- 172.21.15.44:0/3476784050 wait complete. 2024-09-13T08:53:16.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.523+0000 7f86f3800640 1 Processor -- start 2024-09-13T08:53:16.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.523+0000 7f86f3800640 1 -- start start 2024-09-13T08:53:16.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f3800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86ec06e270 0x7f86ec0c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:16.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f3800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86ec161310 0x7f86ec0cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f3800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86ec1616f0 0x7f86ec159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f3800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec16b3b0 con 0x7f86ec1616f0 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f3800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec16b230 con 0x7f86ec161310 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f3800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f86ec16b530 con 0x7f86ec06e270 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86ec1616f0 0x7f86ec159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86ec1616f0 0x7f86ec159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43072/0 (socket says 172.21.15.44:43072) 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f1800640 1 -- 172.21.15.44:0/1152669607 learned_addr learned my addr 172.21.15.44:0/1152669607 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f0e00640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86ec06e270 0x7f86ec0c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86ebe00640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86ec161310 0x7f86ec0cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f1800640 1 -- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86ec06e270 msgr2=0x7f86ec0c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f1800640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86ec06e270 0x7f86ec0c6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f1800640 1 -- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86ec161310 msgr2=0x7f86ec0cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f1800640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86ec161310 0x7f86ec0cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:16.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f1800640 1 -- 172.21.15.44:0/1152669607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec15d390 con 0x7f86ec1616f0 2024-09-13T08:53:16.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f1800640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86ec1616f0 0x7f86ec159270 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c7a0 tx=0x7f86e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:16.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86e8c00640 1 -- 172.21.15.44:0/1152669607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4003b80 con 0x7f86ec1616f0 2024-09-13T08:53:16.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86e8c00640 1 -- 172.21.15.44:0/1152669607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86e4003d40 con 0x7f86ec1616f0 2024-09-13T08:53:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86e8c00640 1 -- 172.21.15.44:0/1152669607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e400aab0 con 0x7f86ec1616f0 2024-09-13T08:53:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f3800640 1 -- 172.21.15.44:0/1152669607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ec15d630 con 0x7f86ec1616f0 2024-09-13T08:53:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f3800640 1 -- 172.21.15.44:0/1152669607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ec15dbf0 con 0x7f86ec1616f0 2024-09-13T08:53:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.527+0000 7f86f3800640 1 -- 172.21.15.44:0/1152669607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ec0d0a10 con 0x7f86ec1616f0 2024-09-13T08:53:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.531+0000 7f86e8c00640 1 -- 172.21.15.44:0/1152669607 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f86e400ac50 con 0x7f86ec1616f0 2024-09-13T08:53:16.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.531+0000 7f86e8c00640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86cc075140 0x7f86cc077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:16.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.531+0000 7f86e8c00640 1 -- 172.21.15.44:0/1152669607 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5821+0+0 (secure 0 0 0) 0x7f86e4092c30 con 0x7f86ec1616f0 2024-09-13T08:53:16.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.531+0000 7f86f0e00640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86cc075140 0x7f86cc077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:16.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.531+0000 7f86f0e00640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86cc075140 0x7f86cc077600 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f86ec1fc5d0 tx=0x7f86e000b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:16.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.535+0000 7f86e8c00640 1 -- 172.21.15.44:0/1152669607 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f86e405f290 con 0x7f86ec1616f0 2024-09-13T08:53:16.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:16.751+0000 7f86f3800640 1 -- 172.21.15.44:0/1152669607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f86ec0b36f0 con 0x7f86ec1616f0 2024-09-13T08:53:17.577 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-09-13T08:53:17.575+0000 7f3412c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T08:53:17.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.615+0000 7f86e8c00640 1 -- 172.21.15.44:0/1152669607 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v94) ==== 118+0+0 (secure 0 0 0) 0x7f86e4063cc0 con 0x7f86ec1616f0 2024-09-13T08:53:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.615+0000 7f86f3800640 1 -- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86cc075140 msgr2=0x7f86cc077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.615+0000 7f86f3800640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86cc075140 0x7f86cc077600 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f86ec1fc5d0 tx=0x7f86e000b390 comp rx=0 tx=0).stop 2024-09-13T08:53:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.615+0000 7f86f3800640 1 -- 172.21.15.44:0/1152669607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86ec1616f0 msgr2=0x7f86ec159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.615+0000 7f86f3800640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86ec1616f0 0x7f86ec159270 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c7a0 tx=0x7f86e400cc60 comp rx=0 tx=0).stop 2024-09-13T08:53:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.619+0000 7f86f3800640 1 -- 172.21.15.44:0/1152669607 shutdown_connections 2024-09-13T08:53:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.619+0000 7f86f3800640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86cc075140 0x7f86cc077600 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.619+0000 7f86f3800640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86ec1616f0 0x7f86ec159270 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.619+0000 7f86f3800640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86ec161310 0x7f86ec0cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.619+0000 7f86f3800640 1 --2- 172.21.15.44:0/1152669607 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86ec06e270 0x7f86ec0c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.619+0000 7f86f3800640 1 -- 172.21.15.44:0/1152669607 >> 172.21.15.44:0/1152669607 conn(0x7f86ec0d1e00 msgr2=0x7f86ec0b5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.619+0000 7f86f3800640 1 -- 172.21.15.44:0/1152669607 shutdown_connections 2024-09-13T08:53:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.619+0000 7f86f3800640 1 -- 172.21.15.44:0/1152669607 wait complete. 2024-09-13T08:53:17.644 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T08:53:17.644 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T08:53:17.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.795+0000 7fc4fcc00640 1 -- 172.21.15.44:0/2283753750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f80b2a00 msgr2=0x7fc4f80b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.795+0000 7fc4fcc00640 1 --2- 172.21.15.44:0/2283753750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f80b2a00 0x7fc4f80b39e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009840 tx=0x7fc4e001c7f0 comp rx=0 tx=0).stop 2024-09-13T08:53:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.795+0000 7fc4fcc00640 1 -- 172.21.15.44:0/2283753750 shutdown_connections 2024-09-13T08:53:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.795+0000 7fc4fcc00640 1 --2- 172.21.15.44:0/2283753750 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f80b2a00 0x7fc4f80b39e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.795+0000 7fc4fcc00640 1 --2- 172.21.15.44:0/2283753750 >> v2:172.21.15.129:3300/0 conn(0x7fc4f815f340 0x7fc4f80b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.795+0000 7fc4fcc00640 1 -- 172.21.15.44:0/2283753750 >> 172.21.15.44:0/2283753750 conn(0x7fc4f80d1e00 msgr2=0x7fc4f80d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:17.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.795+0000 7fc4fcc00640 1 -- 172.21.15.44:0/2283753750 shutdown_connections 2024-09-13T08:53:17.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.795+0000 7fc4fcc00640 1 -- 172.21.15.44:0/2283753750 wait complete. 2024-09-13T08:53:17.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4fcc00640 1 Processor -- start 2024-09-13T08:53:17.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4fcc00640 1 -- start start 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4fcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f80b2a00 0x7fc4f806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4fcc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4f80ccf10 0x7fc4f815a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4fcc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f815f340 0x7fc4f8166370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4fcc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8168fc0 con 0x7fc4f80b2a00 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4fcc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8168e40 con 0x7fc4f80ccf10 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4fcc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f8169140 con 0x7fc4f815f340 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4f5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f80b2a00 0x7fc4f806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4f5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f80b2a00 0x7fc4f806f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43086/0 (socket says 172.21.15.44:43086) 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4f5e00640 1 -- 172.21.15.44:0/1131320473 learned_addr learned my addr 172.21.15.44:0/1131320473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4efe00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4f80ccf10 0x7fc4f815a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4f6800640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f815f340 0x7fc4f8166370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:17.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4f5e00640 1 -- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f815f340 msgr2=0x7fc4f8166370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4f5e00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f815f340 0x7fc4f8166370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4f5e00640 1 -- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4f80ccf10 msgr2=0x7fc4f815a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4f5e00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4f80ccf10 0x7fc4f815a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4f5e00640 1 -- 172.21.15.44:0/1131320473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f806fba0 con 0x7fc4f80b2a00 2024-09-13T08:53:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4f5e00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f80b2a00 0x7fc4f806f5c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800e970 tx=0x7fc4e800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4ee000640 1 -- 172.21.15.44:0/1131320473 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e800cd80 con 0x7fc4f80b2a00 2024-09-13T08:53:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4ee000640 1 -- 172.21.15.44:0/1131320473 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4e8004020 con 0x7fc4f80b2a00 2024-09-13T08:53:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4ee000640 1 -- 172.21.15.44:0/1131320473 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8004470 con 0x7fc4f80b2a00 2024-09-13T08:53:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.799+0000 7fc4fcc00640 1 -- 172.21.15.44:0/1131320473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f81599c0 con 0x7fc4f80b2a00 2024-09-13T08:53:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.803+0000 7fc4fcc00640 1 -- 172.21.15.44:0/1131320473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f81668b0 con 0x7fc4f80b2a00 2024-09-13T08:53:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.803+0000 7fc4fcc00640 1 -- 172.21.15.44:0/1131320473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f80cd3a0 con 0x7fc4f80b2a00 2024-09-13T08:53:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.803+0000 7fc4ee000640 1 -- 172.21.15.44:0/1131320473 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7fc4e8003600 con 0x7fc4f80b2a00 2024-09-13T08:53:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.803+0000 7fc4ee000640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4d4075110 0x7fc4d40775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.803+0000 7fc4ee000640 1 -- 172.21.15.44:0/1131320473 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fc4e8093800 con 0x7fc4f80b2a00 2024-09-13T08:53:17.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.807+0000 7fc4efe00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4d4075110 0x7fc4d40775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:17.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.807+0000 7fc4efe00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4d4075110 0x7fc4d40775d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00af000 tx=0x7fc4e001b030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:17.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:17.807+0000 7fc4ee000640 1 -- 172.21.15.44:0/1131320473 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc4e805fe80 con 0x7fc4f80b2a00 2024-09-13T08:53:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.019+0000 7fc4fcc00640 1 -- 172.21.15.44:0/1131320473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc4f8072370 con 0x7fc4f80b2a00 2024-09-13T08:53:18.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.615+0000 7fc4ee000640 1 -- 172.21.15.44:0/1131320473 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v95) ==== 122+0+0 (secure 0 0 0) 0x7fc4e80648b0 con 0x7fc4f80b2a00 2024-09-13T08:53:18.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 -- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4d4075110 msgr2=0x7fc4d40775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4d4075110 0x7fc4d40775d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00af000 tx=0x7fc4e001b030 comp rx=0 tx=0).stop 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 -- 172.21.15.44:0/1131320473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f80b2a00 msgr2=0x7fc4f806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f80b2a00 0x7fc4f806f5c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800e970 tx=0x7fc4e800ee30 comp rx=0 tx=0).stop 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 -- 172.21.15.44:0/1131320473 shutdown_connections 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc4d4075110 0x7fc4d40775d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc4f815f340 0x7fc4f8166370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4f80ccf10 0x7fc4f815a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 --2- 172.21.15.44:0/1131320473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f80b2a00 0x7fc4f806f5c0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 -- 172.21.15.44:0/1131320473 >> 172.21.15.44:0/1131320473 conn(0x7fc4f80d1e00 msgr2=0x7fc4f80709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 -- 172.21.15.44:0/1131320473 shutdown_connections 2024-09-13T08:53:18.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.619+0000 7fc4fcc00640 1 -- 172.21.15.44:0/1131320473 wait complete. 2024-09-13T08:53:18.644 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T08:53:18.644 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:53:18.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.791+0000 7f4261c00640 1 -- 172.21.15.44:0/2833095438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0cfe80 msgr2=0x7f425c0d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:18.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.791+0000 7f4261c00640 1 --2- 172.21.15.44:0/2833095438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0cfe80 0x7f425c0d0260 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f425c0c5910 tx=0x7f424c01c830 comp rx=0 tx=0).stop 2024-09-13T08:53:18.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.791+0000 7f4261c00640 1 -- 172.21.15.44:0/2833095438 shutdown_connections 2024-09-13T08:53:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.791+0000 7f4261c00640 1 --2- 172.21.15.44:0/2833095438 >> v2:172.21.15.129:3300/0 conn(0x7f425c0b2d00 0x7f425c15fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.791+0000 7f4261c00640 1 --2- 172.21.15.44:0/2833095438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0cfe80 0x7f425c0d0260 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.791+0000 7f4261c00640 1 -- 172.21.15.44:0/2833095438 >> 172.21.15.44:0/2833095438 conn(0x7f425c0d1e40 msgr2=0x7f425c0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.791+0000 7f4261c00640 1 -- 172.21.15.44:0/2833095438 shutdown_connections 2024-09-13T08:53:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.791+0000 7f4261c00640 1 -- 172.21.15.44:0/2833095438 wait complete. 2024-09-13T08:53:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 Processor -- start 2024-09-13T08:53:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 -- start start 2024-09-13T08:53:18.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0b2d00 0x7f425c15f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:18.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c0cfe80 0x7f425c149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:18.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f425c160400 0x7f425c1497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:18.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f425c169520 con 0x7f425c0b2d00 2024-09-13T08:53:18.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1693a0 con 0x7f425c0cfe80 2024-09-13T08:53:18.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f425c1696a0 con 0x7f425c160400 2024-09-13T08:53:18.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f425aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0b2d00 0x7f425c15f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:18.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f425aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0b2d00 0x7f425c15f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43112/0 (socket says 172.21.15.44:43112) 2024-09-13T08:53:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f425aa00640 1 -- 172.21.15.44:0/1992027828 learned_addr learned my addr 172.21.15.44:0/1992027828 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4252000640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c0cfe80 0x7f425c149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f425b400640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f425c160400 0x7f425c1497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f425aa00640 1 -- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f425c160400 msgr2=0x7f425c1497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f425aa00640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f425c160400 0x7f425c1497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f425aa00640 1 -- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c0cfe80 msgr2=0x7f425c149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f425aa00640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c0cfe80 0x7f425c149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f425aa00640 1 -- 172.21.15.44:0/1992027828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c0c5910 con 0x7f425c0b2d00 2024-09-13T08:53:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4252000640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c0cfe80 0x7f425c149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f425aa00640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0b2d00 0x7f425c15f140 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f424c019440 tx=0x7f424c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:18.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4253400640 1 -- 172.21.15.44:0/1992027828 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c0bb020 con 0x7f425c0b2d00 2024-09-13T08:53:18.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4253400640 1 -- 172.21.15.44:0/1992027828 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f424c0044c0 con 0x7f425c0b2d00 2024-09-13T08:53:18.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4253400640 1 -- 172.21.15.44:0/1992027828 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c005290 con 0x7f425c0b2d00 2024-09-13T08:53:18.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 -- 172.21.15.44:0/1992027828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425c0c5bf0 con 0x7f425c0b2d00 2024-09-13T08:53:18.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 -- 172.21.15.44:0/1992027828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425c0c8780 con 0x7f425c0b2d00 2024-09-13T08:53:18.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.795+0000 7f4261c00640 1 -- 172.21.15.44:0/1992027828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f425c15fec0 con 0x7f425c0b2d00 2024-09-13T08:53:18.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.799+0000 7f4253400640 1 -- 172.21.15.44:0/1992027828 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f424c0be020 con 0x7f425c0b2d00 2024-09-13T08:53:18.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.799+0000 7f4253400640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4238075110 0x7f42380775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:18.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.799+0000 7f4253400640 1 -- 172.21.15.44:0/1992027828 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f424c12d430 con 0x7f425c0b2d00 2024-09-13T08:53:18.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.799+0000 7f4252000640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4238075110 0x7f42380775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:18.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.799+0000 7f4252000640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4238075110 0x7f42380775d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4244005fd0 tx=0x7f424400a5e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:18.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:18.803+0000 7f4253400640 1 -- 172.21.15.44:0/1992027828 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f424c0c6020 con 0x7f425c0b2d00 2024-09-13T08:53:19.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.015+0000 7f4261c00640 1 -- 172.21.15.44:0/1992027828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f425c15e700 con 0x7f425c0b2d00 2024-09-13T08:53:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.015+0000 7f4253400640 1 -- 172.21.15.44:0/1992027828 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5230 (secure 0 0 0) 0x7f424c0164b0 con 0x7f425c0b2d00 2024-09-13T08:53:19.018 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:19.018 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":95,"btime":"2024-09-13T08:53:18:619363+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/766502733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":766502733},{"type":"v1","addr":"172.21.15.129:6837","nonce":766502733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2074722286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2074722286},{"type":"v1","addr":"172.21.15.129:6835","nonce":2074722286}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3065267295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3065267295},{"type":"v1","addr":"172.21.15.44:6834","nonce":3065267295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:53:15.007787+0000","modified":"2024-09-13T08:53:18.026905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9219":{"gid":9219,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3418956320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3418956320},{"type":"v1","addr":"172.21.15.44:6835","nonce":3418956320}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9219,"qdb_cluster":[9219]},"id":10}]} 2024-09-13T08:53:19.018 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 95 2024-09-13T08:53:19.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 -- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4238075110 msgr2=0x7f42380775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4238075110 0x7f42380775d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4244005fd0 tx=0x7f424400a5e0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 -- 172.21.15.44:0/1992027828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0b2d00 msgr2=0x7f425c15f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0b2d00 0x7f425c15f140 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f424c019440 tx=0x7f424c005760 comp rx=0 tx=0).stop 2024-09-13T08:53:19.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 -- 172.21.15.44:0/1992027828 shutdown_connections 2024-09-13T08:53:19.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4238075110 0x7f42380775d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f425c160400 0x7f425c1497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c0cfe80 0x7f425c149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 --2- 172.21.15.44:0/1992027828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0b2d00 0x7f425c15f140 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 -- 172.21.15.44:0/1992027828 >> 172.21.15.44:0/1992027828 conn(0x7f425c0d1e40 msgr2=0x7f425c0ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:19.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 -- 172.21.15.44:0/1992027828 shutdown_connections 2024-09-13T08:53:19.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.019+0000 7f4261c00640 1 -- 172.21.15.44:0/1992027828 wait complete. 2024-09-13T08:53:19.039 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:53:19.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.183+0000 7f6847000640 1 -- 172.21.15.44:0/458009014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68400b2e80 msgr2=0x7f68400b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.183+0000 7f6847000640 1 --2- 172.21.15.44:0/458009014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68400b2e80 0x7f68400b1bd0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f6828009bc0 tx=0x7f682801cae0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.183+0000 7f6847000640 1 -- 172.21.15.44:0/458009014 shutdown_connections 2024-09-13T08:53:19.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.183+0000 7f6847000640 1 --2- 172.21.15.44:0/458009014 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68400afca0 0x7f68400b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.183+0000 7f6847000640 1 --2- 172.21.15.44:0/458009014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68400b2e80 0x7f68400b1bd0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.183+0000 7f6847000640 1 -- 172.21.15.44:0/458009014 >> 172.21.15.44:0/458009014 conn(0x7f68400d1e30 msgr2=0x7f68400d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:19.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.183+0000 7f6847000640 1 -- 172.21.15.44:0/458009014 shutdown_connections 2024-09-13T08:53:19.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.183+0000 7f6847000640 1 -- 172.21.15.44:0/458009014 wait complete. 2024-09-13T08:53:19.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6847000640 1 Processor -- start 2024-09-13T08:53:19.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6847000640 1 -- start start 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6847000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68400afca0 0x7f6840158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6847000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68400b2e80 0x7f684006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6847000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f684015f490 0x7f684006f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6847000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6840168e00 con 0x7f68400b2e80 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6847000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6840168c80 con 0x7f684015f490 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6847000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6840168f80 con 0x7f68400afca0 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6837400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68400b2e80 0x7f684006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6837400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68400b2e80 0x7f684006ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43132/0 (socket says 172.21.15.44:43132) 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6837400640 1 -- 172.21.15.44:0/172848476 learned_addr learned my addr 172.21.15.44:0/172848476 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f683fe00640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68400afca0 0x7f6840158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6845000640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f684015f490 0x7f684006f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6837400640 1 -- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68400afca0 msgr2=0x7f6840158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6837400640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68400afca0 0x7f6840158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6837400640 1 -- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f684015f490 msgr2=0x7f684006f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6837400640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f684015f490 0x7f684006f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6837400640 1 -- 172.21.15.44:0/172848476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68400b4fc0 con 0x7f68400b2e80 2024-09-13T08:53:19.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6845000640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f684015f490 0x7f684006f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:19.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f683fe00640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68400afca0 0x7f6840158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:19.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6837400640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68400b2e80 0x7f684006ee70 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f6828009b10 tx=0x7f68280a5a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:19.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f683cc00640 1 -- 172.21.15.44:0/172848476 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68280bb070 con 0x7f68400b2e80 2024-09-13T08:53:19.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f683cc00640 1 -- 172.21.15.44:0/172848476 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6828004e00 con 0x7f68400b2e80 2024-09-13T08:53:19.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f683cc00640 1 -- 172.21.15.44:0/172848476 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68280b2040 con 0x7f68400b2e80 2024-09-13T08:53:19.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6847000640 1 -- 172.21.15.44:0/172848476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68400b52a0 con 0x7f68400b2e80 2024-09-13T08:53:19.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.187+0000 7f6847000640 1 -- 172.21.15.44:0/172848476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68401502d0 con 0x7f68400b2e80 2024-09-13T08:53:19.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.191+0000 7f6847000640 1 -- 172.21.15.44:0/172848476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68400b6e80 con 0x7f68400b2e80 2024-09-13T08:53:19.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.191+0000 7f683cc00640 1 -- 172.21.15.44:0/172848476 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f68280a5a90 con 0x7f68400b2e80 2024-09-13T08:53:19.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.191+0000 7f683cc00640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f681c075110 0x7f681c0775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.191+0000 7f683cc00640 1 -- 172.21.15.44:0/172848476 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f682812d010 con 0x7f68400b2e80 2024-09-13T08:53:19.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.191+0000 7f683fe00640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f681c075110 0x7f681c0775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.191+0000 7f683fe00640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f681c075110 0x7f681c0775d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6830000f70 tx=0x7f6830009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:19.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.195+0000 7f683cc00640 1 -- 172.21.15.44:0/172848476 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f68400b6e80 con 0x7f68400b2e80 2024-09-13T08:53:19.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.371+0000 7f6847000640 1 -- 172.21.15.44:0/172848476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68400b0110 con 0x7f68400b2e80 2024-09-13T08:53:19.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.371+0000 7f683cc00640 1 -- 172.21.15.44:0/172848476 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16928 (secure 0 0 0) 0x7f6828016790 con 0x7f68400b2e80 2024-09-13T08:53:19.375 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:19.375 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":144,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:53:16.602486+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:53:12.727898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-13T08:53:13.969013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:53:19.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 -- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f681c075110 msgr2=0x7f681c0775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f681c075110 0x7f681c0775d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6830000f70 tx=0x7f6830009040 comp rx=0 tx=0).stop 2024-09-13T08:53:19.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 -- 172.21.15.44:0/172848476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68400b2e80 msgr2=0x7f684006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68400b2e80 0x7f684006ee70 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f6828009b10 tx=0x7f68280a5a20 comp rx=0 tx=0).stop 2024-09-13T08:53:19.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 -- 172.21.15.44:0/172848476 shutdown_connections 2024-09-13T08:53:19.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f681c075110 0x7f681c0775d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f684015f490 0x7f684006f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68400b2e80 0x7f684006ee70 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 --2- 172.21.15.44:0/172848476 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f68400afca0 0x7f6840158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 -- 172.21.15.44:0/172848476 >> 172.21.15.44:0/172848476 conn(0x7f68400d1e30 msgr2=0x7f684006e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:19.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 -- 172.21.15.44:0/172848476 shutdown_connections 2024-09-13T08:53:19.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.375+0000 7f6847000640 1 -- 172.21.15.44:0/172848476 wait complete. 2024-09-13T08:53:19.397 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:53:19.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:53:19.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 -- 172.21.15.44:0/2007673034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8580d28e0 msgr2=0x7fe8580cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 --2- 172.21.15.44:0/2007673034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8580d28e0 0x7fe8580cebb0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe8580745d0 tx=0x7fe85001cb80 comp rx=0 tx=0).stop 2024-09-13T08:53:19.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 -- 172.21.15.44:0/2007673034 shutdown_connections 2024-09-13T08:53:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 --2- 172.21.15.44:0/2007673034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8580d28e0 0x7fe8580cebb0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 --2- 172.21.15.44:0/2007673034 >> v2:172.21.15.129:3300/0 conn(0x7fe85815bea0 0x7fe8580ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 -- 172.21.15.44:0/2007673034 >> 172.21.15.44:0/2007673034 conn(0x7fe8580d1dc0 msgr2=0x7fe8580d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 -- 172.21.15.44:0/2007673034 shutdown_connections 2024-09-13T08:53:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 -- 172.21.15.44:0/2007673034 wait complete. 2024-09-13T08:53:19.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 Processor -- start 2024-09-13T08:53:19.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 -- start start 2024-09-13T08:53:19.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8580cdb30 0x7fe85815b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8580d28e0 0x7fe8580c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe85815bea0 0x7fe8580cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe85815c490 con 0x7fe85815bea0 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8580cfb50 con 0x7fe8580d28e0 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.543+0000 7fe85d200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe8580cd0e0 con 0x7fe8580cdb30 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe856800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe85815bea0 0x7fe8580cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe856800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe85815bea0 0x7fe8580cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43150/0 (socket says 172.21.15.44:43150) 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe856800640 1 -- 172.21.15.44:0/968588008 learned_addr learned my addr 172.21.15.44:0/968588008 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe855e00640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8580cdb30 0x7fe85815b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe84fe00640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8580d28e0 0x7fe8580c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe856800640 1 -- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8580cdb30 msgr2=0x7fe85815b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe856800640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8580cdb30 0x7fe85815b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe856800640 1 -- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8580d28e0 msgr2=0x7fe8580c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe856800640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8580d28e0 0x7fe8580c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe856800640 1 -- 172.21.15.44:0/968588008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8580b1a00 con 0x7fe85815bea0 2024-09-13T08:53:19.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe855e00640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8580cdb30 0x7fe85815b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:19.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe84fe00640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8580d28e0 0x7fe8580c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:19.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe856800640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe85815bea0 0x7fe8580cb770 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe85001a7b0 tx=0x7fe8500045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:19.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe84e000640 1 -- 172.21.15.44:0/968588008 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8500bb070 con 0x7fe85815bea0 2024-09-13T08:53:19.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe84e000640 1 -- 172.21.15.44:0/968588008 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8500047d0 con 0x7fe85815bea0 2024-09-13T08:53:19.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe85d200640 1 -- 172.21.15.44:0/968588008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8580b1ce0 con 0x7fe85815bea0 2024-09-13T08:53:19.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe84e000640 1 -- 172.21.15.44:0/968588008 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8500b2040 con 0x7fe85815bea0 2024-09-13T08:53:19.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe85d200640 1 -- 172.21.15.44:0/968588008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8580b22a0 con 0x7fe85815bea0 2024-09-13T08:53:19.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe85d200640 1 -- 172.21.15.44:0/968588008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8580cd670 con 0x7fe85815bea0 2024-09-13T08:53:19.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.547+0000 7fe84e000640 1 -- 172.21.15.44:0/968588008 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7fe8500be070 con 0x7fe85815bea0 2024-09-13T08:53:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.551+0000 7fe84e000640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe834075110 0x7fe8340775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.551+0000 7fe84e000640 1 -- 172.21.15.44:0/968588008 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fe85012d1a0 con 0x7fe85815bea0 2024-09-13T08:53:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.551+0000 7fe855e00640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe834075110 0x7fe8340775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.551+0000 7fe855e00640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe834075110 0x7fe8340775d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe84800bbd0 tx=0x7fe84800b5b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:19.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.555+0000 7fe84e000640 1 -- 172.21.15.44:0/968588008 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe850016890 con 0x7fe85815bea0 2024-09-13T08:53:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.735+0000 7fe85d200640 1 -- 172.21.15.44:0/968588008 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8580762f0 con 0x7fe834075110 2024-09-13T08:53:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.739+0000 7fe84e000640 1 -- 172.21.15.44:0/968588008 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241533 (secure 0 0 0) 0x7fe8580762f0 con 0x7fe834075110 2024-09-13T08:53:19.744 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:19.746 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:53:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 -- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe834075110 msgr2=0x7fe8340775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe834075110 0x7fe8340775d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe84800bbd0 tx=0x7fe84800b5b0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 -- 172.21.15.44:0/968588008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe85815bea0 msgr2=0x7fe8580cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe85815bea0 0x7fe8580cb770 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe85001a7b0 tx=0x7fe8500045e0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 -- 172.21.15.44:0/968588008 shutdown_connections 2024-09-13T08:53:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe834075110 0x7fe8340775d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe85815bea0 0x7fe8580cb770 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8580d28e0 0x7fe8580c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 --2- 172.21.15.44:0/968588008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8580cdb30 0x7fe85815b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 -- 172.21.15.44:0/968588008 >> 172.21.15.44:0/968588008 conn(0x7fe8580d1dc0 msgr2=0x7fe85806f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:19.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 -- 172.21.15.44:0/968588008 shutdown_connections 2024-09-13T08:53:19.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.743+0000 7fe85d200640 1 -- 172.21.15.44:0/968588008 wait complete. 2024-09-13T08:53:19.765 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2024-09-13T08:53:18.836558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":76,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1095440,"kb_used_data":14876,"kb_used_omap":12,"kb_used_meta":1080371,"kb_avail":753879280,"statfs":{"total":773094113280,"available":771972382720,"internally_reserved":0,"allocated":15233024,"data_stored":7377482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1106300465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252604"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129523+0000","last_change":"2024-09-13T08:53:15.613820+0000","last_active":"2024-09-13T08:53:17.129523+0000","last_peered":"2024-09-13T08:53:17.129523+0000","last_clean":"2024-09-13T08:53:17.129523+0000","last_became_active":"2024-09-13T08:53:15.613531+0000","last_became_peered":"2024-09-13T08:53:15.613531+0000","last_unstale":"2024-09-13T08:53:17.129523+0000","last_undegraded":"2024-09-13T08:53:17.129523+0000","last_fullsized":"2024-09-13T08:53:17.129523+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:16.193181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129499+0000","last_change":"2024-09-13T08:53:15.610999+0000","last_active":"2024-09-13T08:53:17.129499+0000","last_peered":"2024-09-13T08:53:17.129499+0000","last_clean":"2024-09-13T08:53:17.129499+0000","last_became_active":"2024-09-13T08:53:15.610709+0000","last_became_peered":"2024-09-13T08:53:15.610709+0000","last_unstale":"2024-09-13T08:53:17.129499+0000","last_undegraded":"2024-09-13T08:53:17.129499+0000","last_fullsized":"2024-09-13T08:53:17.129499+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:51.632629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129424+0000","last_change":"2024-09-13T08:53:15.608830+0000","last_active":"2024-09-13T08:53:17.129424+0000","last_peered":"2024-09-13T08:53:17.129424+0000","last_clean":"2024-09-13T08:53:17.129424+0000","last_became_active":"2024-09-13T08:53:15.608527+0000","last_became_peered":"2024-09-13T08:53:15.608527+0000","last_unstale":"2024-09-13T08:53:17.129424+0000","last_undegraded":"2024-09-13T08:53:17.129424+0000","last_fullsized":"2024-09-13T08:53:17.129424+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:06.563772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587498+0000","last_change":"2024-09-13T08:53:13.587498+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587498+0000","last_undegraded":"2024-09-13T08:53:13.587498+0000","last_fullsized":"2024-09-13T08:53:13.587498+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129266+0000","last_change":"2024-09-13T08:53:14.613339+0000","last_active":"2024-09-13T08:53:17.129266+0000","last_peered":"2024-09-13T08:53:17.129266+0000","last_clean":"2024-09-13T08:53:17.129266+0000","last_became_active":"2024-09-13T08:53:14.612799+0000","last_became_peered":"2024-09-13T08:53:14.612799+0000","last_unstale":"2024-09-13T08:53:17.129266+0000","last_undegraded":"2024-09-13T08:53:17.129266+0000","last_fullsized":"2024-09-13T08:53:17.129266+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:54.292680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129342+0000","last_change":"2024-09-13T08:53:15.616934+0000","last_active":"2024-09-13T08:53:17.129342+0000","last_peered":"2024-09-13T08:53:17.129342+0000","last_clean":"2024-09-13T08:53:17.129342+0000","last_became_active":"2024-09-13T08:53:15.616660+0000","last_became_peered":"2024-09-13T08:53:15.616660+0000","last_unstale":"2024-09-13T08:53:17.129342+0000","last_undegraded":"2024-09-13T08:53:17.129342+0000","last_fullsized":"2024-09-13T08:53:17.129342+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:52:39.024260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587950+0000","last_change":"2024-09-13T08:53:13.587950+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587950+0000","last_undegraded":"2024-09-13T08:53:13.587950+0000","last_fullsized":"2024-09-13T08:53:13.587950+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589740+0000","last_change":"2024-09-13T08:53:13.589740+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589740+0000","last_undegraded":"2024-09-13T08:53:13.589740+0000","last_fullsized":"2024-09-13T08:53:13.589740+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589133+0000","last_change":"2024-09-13T08:53:13.589133+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589133+0000","last_undegraded":"2024-09-13T08:53:13.589133+0000","last_fullsized":"2024-09-13T08:53:13.589133+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.586318+0000","last_change":"2024-09-13T08:53:13.586318+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.586318+0000","last_undegraded":"2024-09-13T08:53:13.586318+0000","last_fullsized":"2024-09-13T08:53:13.586318+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129340+0000","last_change":"2024-09-13T08:53:14.609106+0000","last_active":"2024-09-13T08:53:17.129340+0000","last_peered":"2024-09-13T08:53:17.129340+0000","last_clean":"2024-09-13T08:53:17.129340+0000","last_became_active":"2024-09-13T08:53:14.608810+0000","last_became_peered":"2024-09-13T08:53:14.608810+0000","last_unstale":"2024-09-13T08:53:17.129340+0000","last_undegraded":"2024-09-13T08:53:17.129340+0000","last_fullsized":"2024-09-13T08:53:17.129340+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:44.027745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587437+0000","last_change":"2024-09-13T08:53:13.587437+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587437+0000","last_undegraded":"2024-09-13T08:53:13.587437+0000","last_fullsized":"2024-09-13T08:53:13.587437+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129413+0000","last_change":"2024-09-13T08:53:14.611525+0000","last_active":"2024-09-13T08:53:17.129413+0000","last_peered":"2024-09-13T08:53:17.129413+0000","last_clean":"2024-09-13T08:53:17.129413+0000","last_became_active":"2024-09-13T08:53:14.611246+0000","last_became_peered":"2024-09-13T08:53:14.611246+0000","last_unstale":"2024-09-13T08:53:17.129413+0000","last_undegraded":"2024-09-13T08:53:17.129413+0000","last_fullsized":"2024-09-13T08:53:17.129413+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:51.437020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.586018+0000","last_change":"2024-09-13T08:53:13.586018+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.586018+0000","last_undegraded":"2024-09-13T08:53:13.586018+0000","last_fullsized":"2024-09-13T08:53:13.586018+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.586573+0000","last_change":"2024-09-13T08:53:13.586573+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.586573+0000","last_undegraded":"2024-09-13T08:53:13.586573+0000","last_fullsized":"2024-09-13T08:53:13.586573+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587990+0000","last_change":"2024-09-13T08:53:13.587990+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587990+0000","last_undegraded":"2024-09-13T08:53:13.587990+0000","last_fullsized":"2024-09-13T08:53:13.587990+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587409+0000","last_change":"2024-09-13T08:53:13.587409+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587409+0000","last_undegraded":"2024-09-13T08:53:13.587409+0000","last_fullsized":"2024-09-13T08:53:13.587409+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590758+0000","last_change":"2024-09-13T08:53:13.590758+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590758+0000","last_undegraded":"2024-09-13T08:53:13.590758+0000","last_fullsized":"2024-09-13T08:53:13.590758+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589385+0000","last_change":"2024-09-13T08:53:13.589385+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589385+0000","last_undegraded":"2024-09-13T08:53:13.589385+0000","last_fullsized":"2024-09-13T08:53:13.589385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592867+0000","last_change":"2024-09-13T08:53:13.592867+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592867+0000","last_undegraded":"2024-09-13T08:53:13.592867+0000","last_fullsized":"2024-09-13T08:53:13.592867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589224+0000","last_change":"2024-09-13T08:53:13.589224+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589224+0000","last_undegraded":"2024-09-13T08:53:13.589224+0000","last_fullsized":"2024-09-13T08:53:13.589224+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587069+0000","last_change":"2024-09-13T08:53:13.587069+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587069+0000","last_undegraded":"2024-09-13T08:53:13.587069+0000","last_fullsized":"2024-09-13T08:53:13.587069+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129489+0000","last_change":"2024-09-13T08:53:15.615099+0000","last_active":"2024-09-13T08:53:17.129489+0000","last_peered":"2024-09-13T08:53:17.129489+0000","last_clean":"2024-09-13T08:53:17.129489+0000","last_became_active":"2024-09-13T08:53:15.614834+0000","last_became_peered":"2024-09-13T08:53:15.614834+0000","last_unstale":"2024-09-13T08:53:17.129489+0000","last_undegraded":"2024-09-13T08:53:17.129489+0000","last_fullsized":"2024-09-13T08:53:17.129489+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:34.923374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592525+0000","last_change":"2024-09-13T08:53:13.592525+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592525+0000","last_undegraded":"2024-09-13T08:53:13.592525+0000","last_fullsized":"2024-09-13T08:53:13.592525+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129245+0000","last_change":"2024-09-13T08:53:14.613189+0000","last_active":"2024-09-13T08:53:17.129245+0000","last_peered":"2024-09-13T08:53:17.129245+0000","last_clean":"2024-09-13T08:53:17.129245+0000","last_became_active":"2024-09-13T08:53:14.612877+0000","last_became_peered":"2024-09-13T08:53:14.612877+0000","last_unstale":"2024-09-13T08:53:17.129245+0000","last_undegraded":"2024-09-13T08:53:17.129245+0000","last_fullsized":"2024-09-13T08:53:17.129245+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:12.557688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590616+0000","last_change":"2024-09-13T08:53:13.590616+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590616+0000","last_undegraded":"2024-09-13T08:53:13.590616+0000","last_fullsized":"2024-09-13T08:53:13.590616+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590716+0000","last_change":"2024-09-13T08:53:13.590716+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590716+0000","last_undegraded":"2024-09-13T08:53:13.590716+0000","last_fullsized":"2024-09-13T08:53:13.590716+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592095+0000","last_change":"2024-09-13T08:53:13.592095+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592095+0000","last_undegraded":"2024-09-13T08:53:13.592095+0000","last_fullsized":"2024-09-13T08:53:13.592095+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592071+0000","last_change":"2024-09-13T08:53:13.592071+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592071+0000","last_undegraded":"2024-09-13T08:53:13.592071+0000","last_fullsized":"2024-09-13T08:53:13.592071+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590826+0000","last_change":"2024-09-13T08:53:13.590826+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590826+0000","last_undegraded":"2024-09-13T08:53:13.590826+0000","last_fullsized":"2024-09-13T08:53:13.590826+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587534+0000","last_change":"2024-09-13T08:53:13.587534+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587534+0000","last_undegraded":"2024-09-13T08:53:13.587534+0000","last_fullsized":"2024-09-13T08:53:13.587534+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.585896+0000","last_change":"2024-09-13T08:53:13.585896+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.585896+0000","last_undegraded":"2024-09-13T08:53:13.585896+0000","last_fullsized":"2024-09-13T08:53:13.585896+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590718+0000","last_change":"2024-09-13T08:53:13.590718+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590718+0000","last_undegraded":"2024-09-13T08:53:13.590718+0000","last_fullsized":"2024-09-13T08:53:13.590718+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587554+0000","last_change":"2024-09-13T08:53:13.587554+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587554+0000","last_undegraded":"2024-09-13T08:53:13.587554+0000","last_fullsized":"2024-09-13T08:53:13.587554+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587486+0000","last_change":"2024-09-13T08:53:13.587486+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587486+0000","last_undegraded":"2024-09-13T08:53:13.587486+0000","last_fullsized":"2024-09-13T08:53:13.587486+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129123+0000","last_change":"2024-09-13T08:53:14.610914+0000","last_active":"2024-09-13T08:53:17.129123+0000","last_peered":"2024-09-13T08:53:17.129123+0000","last_clean":"2024-09-13T08:53:17.129123+0000","last_became_active":"2024-09-13T08:53:14.610635+0000","last_became_peered":"2024-09-13T08:53:14.610635+0000","last_unstale":"2024-09-13T08:53:17.129123+0000","last_undegraded":"2024-09-13T08:53:17.129123+0000","last_fullsized":"2024-09-13T08:53:17.129123+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:04.024112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587419+0000","last_change":"2024-09-13T08:53:13.587419+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587419+0000","last_undegraded":"2024-09-13T08:53:13.587419+0000","last_fullsized":"2024-09-13T08:53:13.587419+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":340,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T08:53:13.582720+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:53:13.582720+0000","last_peered":"2024-09-13T08:53:13.582720+0000","last_clean":"2024-09-13T08:53:13.582720+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:53:13.582720+0000","last_undegraded":"2024-09-13T08:53:13.582720+0000","last_fullsized":"2024-09-13T08:53:13.582720+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127228,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244612,"statfs":{"total":96636764160,"available":96506482688,"internally_reserved":0,"allocated":1744896,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509632,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119500,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252340,"statfs":{"total":96636764160,"available":96514396160,"internally_reserved":0,"allocated":2338816,"data_stored":1359536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509633,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153608,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218232,"statfs":{"total":96636764160,"available":96479469568,"internally_reserved":0,"allocated":1757184,"data_stored":775097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509633,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131392,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240448,"statfs":{"total":96636764160,"available":96502218752,"internally_reserved":0,"allocated":1744896,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131388,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240452,"statfs":{"total":96636764160,"available":96502222848,"internally_reserved":0,"allocated":1744896,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509634,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145868,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225972,"statfs":{"total":96636764160,"available":96487395328,"internally_reserved":0,"allocated":2351104,"data_stored":1365465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145288,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226552,"statfs":{"total":96636764160,"available":96487989248,"internally_reserved":0,"allocated":1757184,"data_stored":775097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542339,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141168,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230672,"statfs":{"total":96636764160,"available":96492208128,"internally_reserved":0,"allocated":1794048,"data_stored":794783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:53:19.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:53:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.951+0000 7f73b6400640 1 -- 172.21.15.44:0/2470915187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b015df70 msgr2=0x7f73b015e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.951+0000 7f73b6400640 1 --2- 172.21.15.44:0/2470915187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b015df70 0x7f73b015e3e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f739800aa10 tx=0x7f739801dc00 comp rx=0 tx=0).stop 2024-09-13T08:53:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.951+0000 7f73b6400640 1 -- 172.21.15.44:0/2470915187 shutdown_connections 2024-09-13T08:53:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.951+0000 7f73b6400640 1 --2- 172.21.15.44:0/2470915187 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b00b11e0 0x7f73b00b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.951+0000 7f73b6400640 1 --2- 172.21.15.44:0/2470915187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b015df70 0x7f73b015e3e0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.951+0000 7f73b6400640 1 -- 172.21.15.44:0/2470915187 >> 172.21.15.44:0/2470915187 conn(0x7f73b00d04d0 msgr2=0x7f73b00d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.951+0000 7f73b6400640 1 -- 172.21.15.44:0/2470915187 shutdown_connections 2024-09-13T08:53:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.951+0000 7f73b6400640 1 -- 172.21.15.44:0/2470915187 wait complete. 2024-09-13T08:53:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.951+0000 7f73b6400640 1 Processor -- start 2024-09-13T08:53:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.951+0000 7f73b6400640 1 -- start start 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73b6400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b00b11e0 0x7f73b0099500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73b6400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b015a250 0x7f73b00c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73b6400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b015df70 0x7f73b00c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73b6400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73b00b1fa0 con 0x7f73b00b11e0 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73b6400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73b00b1e20 con 0x7f73b015df70 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73b6400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f73b00b2120 con 0x7f73b015a250 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73af400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b00b11e0 0x7f73b0099500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73af400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b00b11e0 0x7f73b0099500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43178/0 (socket says 172.21.15.44:43178) 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73af400640 1 -- 172.21.15.44:0/931326201 learned_addr learned my addr 172.21.15.44:0/931326201 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73a7e00640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b015a250 0x7f73b00c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73afe00640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b015df70 0x7f73b00c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73af400640 1 -- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b015a250 msgr2=0x7f73b00c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73af400640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b015a250 0x7f73b00c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73af400640 1 -- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b015df70 msgr2=0x7f73b00c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73af400640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b015df70 0x7f73b00c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73af400640 1 -- 172.21.15.44:0/931326201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b014bd50 con 0x7f73b00b11e0 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73af400640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b00b11e0 0x7f73b0099500 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f73a000c500 tx=0x7f73a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73a7400640 1 -- 172.21.15.44:0/931326201 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0018070 con 0x7f73b00b11e0 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73a7400640 1 -- 172.21.15.44:0/931326201 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73a0003600 con 0x7f73b00b11e0 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73a7400640 1 -- 172.21.15.44:0/931326201 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0003ad0 con 0x7f73b00b11e0 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73b6400640 1 -- 172.21.15.44:0/931326201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b014c030 con 0x7f73b00b11e0 2024-09-13T08:53:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73b6400640 1 -- 172.21.15.44:0/931326201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b014c5f0 con 0x7f73b00b11e0 2024-09-13T08:53:19.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73b6400640 1 -- 172.21.15.44:0/931326201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b00b1650 con 0x7f73b00b11e0 2024-09-13T08:53:19.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.955+0000 7f73a7400640 1 -- 172.21.15.44:0/931326201 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f73a0003c70 con 0x7f73b00b11e0 2024-09-13T08:53:19.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.959+0000 7f73a7400640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f738c075110 0x7f738c0775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:19.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.959+0000 7f73a7400640 1 -- 172.21.15.44:0/931326201 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f73a0092390 con 0x7f73b00b11e0 2024-09-13T08:53:19.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.959+0000 7f73a7e00640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f738c075110 0x7f738c0775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:19.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.959+0000 7f73a7e00640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f738c075110 0x7f738c0775d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f73980b0000 tx=0x7f739801c440 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:19.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:19.963+0000 7f73a7400640 1 -- 172.21.15.44:0/931326201 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f73a005f080 con 0x7f73b00b11e0 2024-09-13T08:53:20.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.135+0000 7f73b6400640 1 -- 172.21.15.44:0/931326201 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73b00b6450 con 0x7f738c075110 2024-09-13T08:53:20.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.139+0000 7f73a7400640 1 -- 172.21.15.44:0/931326201 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241533 (secure 0 0 0) 0x7f73b00b6450 con 0x7f738c075110 2024-09-13T08:53:20.144 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:20.146 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:53:20.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 -- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f738c075110 msgr2=0x7f738c0775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f738c075110 0x7f738c0775d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f73980b0000 tx=0x7f739801c440 comp rx=0 tx=0).stop 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 -- 172.21.15.44:0/931326201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b00b11e0 msgr2=0x7f73b0099500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b00b11e0 0x7f73b0099500 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f73a000c500 tx=0x7f73a000c9c0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 -- 172.21.15.44:0/931326201 shutdown_connections 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f738c075110 0x7f738c0775d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b015df70 0x7f73b00c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f73b015a250 0x7f73b00c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 --2- 172.21.15.44:0/931326201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b00b11e0 0x7f73b0099500 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 -- 172.21.15.44:0/931326201 >> 172.21.15.44:0/931326201 conn(0x7f73b00d04d0 msgr2=0x7f73b0164990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 -- 172.21.15.44:0/931326201 shutdown_connections 2024-09-13T08:53:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.143+0000 7f73b6400640 1 -- 172.21.15.44:0/931326201 wait complete. 2024-09-13T08:53:20.166 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2024-09-13T08:53:18.836558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":76,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1095440,"kb_used_data":14876,"kb_used_omap":12,"kb_used_meta":1080371,"kb_avail":753879280,"statfs":{"total":773094113280,"available":771972382720,"internally_reserved":0,"allocated":15233024,"data_stored":7377482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1106300465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252604"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129523+0000","last_change":"2024-09-13T08:53:15.613820+0000","last_active":"2024-09-13T08:53:17.129523+0000","last_peered":"2024-09-13T08:53:17.129523+0000","last_clean":"2024-09-13T08:53:17.129523+0000","last_became_active":"2024-09-13T08:53:15.613531+0000","last_became_peered":"2024-09-13T08:53:15.613531+0000","last_unstale":"2024-09-13T08:53:17.129523+0000","last_undegraded":"2024-09-13T08:53:17.129523+0000","last_fullsized":"2024-09-13T08:53:17.129523+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:16.193181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129499+0000","last_change":"2024-09-13T08:53:15.610999+0000","last_active":"2024-09-13T08:53:17.129499+0000","last_peered":"2024-09-13T08:53:17.129499+0000","last_clean":"2024-09-13T08:53:17.129499+0000","last_became_active":"2024-09-13T08:53:15.610709+0000","last_became_peered":"2024-09-13T08:53:15.610709+0000","last_unstale":"2024-09-13T08:53:17.129499+0000","last_undegraded":"2024-09-13T08:53:17.129499+0000","last_fullsized":"2024-09-13T08:53:17.129499+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:51.632629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129424+0000","last_change":"2024-09-13T08:53:15.608830+0000","last_active":"2024-09-13T08:53:17.129424+0000","last_peered":"2024-09-13T08:53:17.129424+0000","last_clean":"2024-09-13T08:53:17.129424+0000","last_became_active":"2024-09-13T08:53:15.608527+0000","last_became_peered":"2024-09-13T08:53:15.608527+0000","last_unstale":"2024-09-13T08:53:17.129424+0000","last_undegraded":"2024-09-13T08:53:17.129424+0000","last_fullsized":"2024-09-13T08:53:17.129424+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:06.563772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587498+0000","last_change":"2024-09-13T08:53:13.587498+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587498+0000","last_undegraded":"2024-09-13T08:53:13.587498+0000","last_fullsized":"2024-09-13T08:53:13.587498+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129266+0000","last_change":"2024-09-13T08:53:14.613339+0000","last_active":"2024-09-13T08:53:17.129266+0000","last_peered":"2024-09-13T08:53:17.129266+0000","last_clean":"2024-09-13T08:53:17.129266+0000","last_became_active":"2024-09-13T08:53:14.612799+0000","last_became_peered":"2024-09-13T08:53:14.612799+0000","last_unstale":"2024-09-13T08:53:17.129266+0000","last_undegraded":"2024-09-13T08:53:17.129266+0000","last_fullsized":"2024-09-13T08:53:17.129266+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:54.292680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129342+0000","last_change":"2024-09-13T08:53:15.616934+0000","last_active":"2024-09-13T08:53:17.129342+0000","last_peered":"2024-09-13T08:53:17.129342+0000","last_clean":"2024-09-13T08:53:17.129342+0000","last_became_active":"2024-09-13T08:53:15.616660+0000","last_became_peered":"2024-09-13T08:53:15.616660+0000","last_unstale":"2024-09-13T08:53:17.129342+0000","last_undegraded":"2024-09-13T08:53:17.129342+0000","last_fullsized":"2024-09-13T08:53:17.129342+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:52:39.024260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587950+0000","last_change":"2024-09-13T08:53:13.587950+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587950+0000","last_undegraded":"2024-09-13T08:53:13.587950+0000","last_fullsized":"2024-09-13T08:53:13.587950+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589740+0000","last_change":"2024-09-13T08:53:13.589740+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589740+0000","last_undegraded":"2024-09-13T08:53:13.589740+0000","last_fullsized":"2024-09-13T08:53:13.589740+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589133+0000","last_change":"2024-09-13T08:53:13.589133+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589133+0000","last_undegraded":"2024-09-13T08:53:13.589133+0000","last_fullsized":"2024-09-13T08:53:13.589133+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.586318+0000","last_change":"2024-09-13T08:53:13.586318+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.586318+0000","last_undegraded":"2024-09-13T08:53:13.586318+0000","last_fullsized":"2024-09-13T08:53:13.586318+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129340+0000","last_change":"2024-09-13T08:53:14.609106+0000","last_active":"2024-09-13T08:53:17.129340+0000","last_peered":"2024-09-13T08:53:17.129340+0000","last_clean":"2024-09-13T08:53:17.129340+0000","last_became_active":"2024-09-13T08:53:14.608810+0000","last_became_peered":"2024-09-13T08:53:14.608810+0000","last_unstale":"2024-09-13T08:53:17.129340+0000","last_undegraded":"2024-09-13T08:53:17.129340+0000","last_fullsized":"2024-09-13T08:53:17.129340+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:44.027745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587437+0000","last_change":"2024-09-13T08:53:13.587437+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587437+0000","last_undegraded":"2024-09-13T08:53:13.587437+0000","last_fullsized":"2024-09-13T08:53:13.587437+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129413+0000","last_change":"2024-09-13T08:53:14.611525+0000","last_active":"2024-09-13T08:53:17.129413+0000","last_peered":"2024-09-13T08:53:17.129413+0000","last_clean":"2024-09-13T08:53:17.129413+0000","last_became_active":"2024-09-13T08:53:14.611246+0000","last_became_peered":"2024-09-13T08:53:14.611246+0000","last_unstale":"2024-09-13T08:53:17.129413+0000","last_undegraded":"2024-09-13T08:53:17.129413+0000","last_fullsized":"2024-09-13T08:53:17.129413+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:51.437020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.586018+0000","last_change":"2024-09-13T08:53:13.586018+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.586018+0000","last_undegraded":"2024-09-13T08:53:13.586018+0000","last_fullsized":"2024-09-13T08:53:13.586018+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.586573+0000","last_change":"2024-09-13T08:53:13.586573+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.586573+0000","last_undegraded":"2024-09-13T08:53:13.586573+0000","last_fullsized":"2024-09-13T08:53:13.586573+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587990+0000","last_change":"2024-09-13T08:53:13.587990+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587990+0000","last_undegraded":"2024-09-13T08:53:13.587990+0000","last_fullsized":"2024-09-13T08:53:13.587990+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587409+0000","last_change":"2024-09-13T08:53:13.587409+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587409+0000","last_undegraded":"2024-09-13T08:53:13.587409+0000","last_fullsized":"2024-09-13T08:53:13.587409+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590758+0000","last_change":"2024-09-13T08:53:13.590758+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590758+0000","last_undegraded":"2024-09-13T08:53:13.590758+0000","last_fullsized":"2024-09-13T08:53:13.590758+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589385+0000","last_change":"2024-09-13T08:53:13.589385+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589385+0000","last_undegraded":"2024-09-13T08:53:13.589385+0000","last_fullsized":"2024-09-13T08:53:13.589385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592867+0000","last_change":"2024-09-13T08:53:13.592867+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592867+0000","last_undegraded":"2024-09-13T08:53:13.592867+0000","last_fullsized":"2024-09-13T08:53:13.592867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589224+0000","last_change":"2024-09-13T08:53:13.589224+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589224+0000","last_undegraded":"2024-09-13T08:53:13.589224+0000","last_fullsized":"2024-09-13T08:53:13.589224+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587069+0000","last_change":"2024-09-13T08:53:13.587069+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587069+0000","last_undegraded":"2024-09-13T08:53:13.587069+0000","last_fullsized":"2024-09-13T08:53:13.587069+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129489+0000","last_change":"2024-09-13T08:53:15.615099+0000","last_active":"2024-09-13T08:53:17.129489+0000","last_peered":"2024-09-13T08:53:17.129489+0000","last_clean":"2024-09-13T08:53:17.129489+0000","last_became_active":"2024-09-13T08:53:15.614834+0000","last_became_peered":"2024-09-13T08:53:15.614834+0000","last_unstale":"2024-09-13T08:53:17.129489+0000","last_undegraded":"2024-09-13T08:53:17.129489+0000","last_fullsized":"2024-09-13T08:53:17.129489+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:34.923374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592525+0000","last_change":"2024-09-13T08:53:13.592525+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592525+0000","last_undegraded":"2024-09-13T08:53:13.592525+0000","last_fullsized":"2024-09-13T08:53:13.592525+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129245+0000","last_change":"2024-09-13T08:53:14.613189+0000","last_active":"2024-09-13T08:53:17.129245+0000","last_peered":"2024-09-13T08:53:17.129245+0000","last_clean":"2024-09-13T08:53:17.129245+0000","last_became_active":"2024-09-13T08:53:14.612877+0000","last_became_peered":"2024-09-13T08:53:14.612877+0000","last_unstale":"2024-09-13T08:53:17.129245+0000","last_undegraded":"2024-09-13T08:53:17.129245+0000","last_fullsized":"2024-09-13T08:53:17.129245+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:12.557688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590616+0000","last_change":"2024-09-13T08:53:13.590616+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590616+0000","last_undegraded":"2024-09-13T08:53:13.590616+0000","last_fullsized":"2024-09-13T08:53:13.590616+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590716+0000","last_change":"2024-09-13T08:53:13.590716+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590716+0000","last_undegraded":"2024-09-13T08:53:13.590716+0000","last_fullsized":"2024-09-13T08:53:13.590716+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592095+0000","last_change":"2024-09-13T08:53:13.592095+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592095+0000","last_undegraded":"2024-09-13T08:53:13.592095+0000","last_fullsized":"2024-09-13T08:53:13.592095+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592071+0000","last_change":"2024-09-13T08:53:13.592071+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592071+0000","last_undegraded":"2024-09-13T08:53:13.592071+0000","last_fullsized":"2024-09-13T08:53:13.592071+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590826+0000","last_change":"2024-09-13T08:53:13.590826+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590826+0000","last_undegraded":"2024-09-13T08:53:13.590826+0000","last_fullsized":"2024-09-13T08:53:13.590826+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587534+0000","last_change":"2024-09-13T08:53:13.587534+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587534+0000","last_undegraded":"2024-09-13T08:53:13.587534+0000","last_fullsized":"2024-09-13T08:53:13.587534+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.585896+0000","last_change":"2024-09-13T08:53:13.585896+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.585896+0000","last_undegraded":"2024-09-13T08:53:13.585896+0000","last_fullsized":"2024-09-13T08:53:13.585896+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590718+0000","last_change":"2024-09-13T08:53:13.590718+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590718+0000","last_undegraded":"2024-09-13T08:53:13.590718+0000","last_fullsized":"2024-09-13T08:53:13.590718+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587554+0000","last_change":"2024-09-13T08:53:13.587554+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587554+0000","last_undegraded":"2024-09-13T08:53:13.587554+0000","last_fullsized":"2024-09-13T08:53:13.587554+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587486+0000","last_change":"2024-09-13T08:53:13.587486+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587486+0000","last_undegraded":"2024-09-13T08:53:13.587486+0000","last_fullsized":"2024-09-13T08:53:13.587486+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129123+0000","last_change":"2024-09-13T08:53:14.610914+0000","last_active":"2024-09-13T08:53:17.129123+0000","last_peered":"2024-09-13T08:53:17.129123+0000","last_clean":"2024-09-13T08:53:17.129123+0000","last_became_active":"2024-09-13T08:53:14.610635+0000","last_became_peered":"2024-09-13T08:53:14.610635+0000","last_unstale":"2024-09-13T08:53:17.129123+0000","last_undegraded":"2024-09-13T08:53:17.129123+0000","last_fullsized":"2024-09-13T08:53:17.129123+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:04.024112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587419+0000","last_change":"2024-09-13T08:53:13.587419+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587419+0000","last_undegraded":"2024-09-13T08:53:13.587419+0000","last_fullsized":"2024-09-13T08:53:13.587419+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":340,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T08:53:13.582720+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:53:13.582720+0000","last_peered":"2024-09-13T08:53:13.582720+0000","last_clean":"2024-09-13T08:53:13.582720+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:53:13.582720+0000","last_undegraded":"2024-09-13T08:53:13.582720+0000","last_fullsized":"2024-09-13T08:53:13.582720+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127228,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244612,"statfs":{"total":96636764160,"available":96506482688,"internally_reserved":0,"allocated":1744896,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509632,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119500,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252340,"statfs":{"total":96636764160,"available":96514396160,"internally_reserved":0,"allocated":2338816,"data_stored":1359536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509633,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153608,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218232,"statfs":{"total":96636764160,"available":96479469568,"internally_reserved":0,"allocated":1757184,"data_stored":775097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509633,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131392,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240448,"statfs":{"total":96636764160,"available":96502218752,"internally_reserved":0,"allocated":1744896,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131388,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240452,"statfs":{"total":96636764160,"available":96502222848,"internally_reserved":0,"allocated":1744896,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509634,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145868,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225972,"statfs":{"total":96636764160,"available":96487395328,"internally_reserved":0,"allocated":2351104,"data_stored":1365465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145288,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226552,"statfs":{"total":96636764160,"available":96487989248,"internally_reserved":0,"allocated":1757184,"data_stored":775097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542339,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141168,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230672,"statfs":{"total":96636764160,"available":96492208128,"internally_reserved":0,"allocated":1794048,"data_stored":794783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:53:20.169 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:53:20.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.351+0000 7f6668800640 1 -- 172.21.15.44:0/2194961007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6648005920 msgr2=0x7f6648005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:20.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.351+0000 7f6668800640 1 --2- 172.21.15.44:0/2194961007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6648005920 0x7f6648005da0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f6650009b30 tx=0x7f665001d230 comp rx=0 tx=0).stop 2024-09-13T08:53:20.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.351+0000 7f6666800640 1 -- 172.21.15.44:0/2194961007 reap_dead start 2024-09-13T08:53:20.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.351+0000 7f6668800640 1 -- 172.21.15.44:0/2194961007 shutdown_connections 2024-09-13T08:53:20.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.351+0000 7f6668800640 1 -- 172.21.15.44:0/2194961007 >> 172.21.15.44:0/2194961007 conn(0x7f66600b18f0 msgr2=0x7f66600b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:20.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.351+0000 7f6668800640 1 -- 172.21.15.44:0/2194961007 shutdown_connections 2024-09-13T08:53:20.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.351+0000 7f6668800640 1 -- 172.21.15.44:0/2194961007 wait complete. 2024-09-13T08:53:20.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.351+0000 7f6668800640 1 Processor -- start 2024-09-13T08:53:20.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.351+0000 7f6668800640 1 -- start start 2024-09-13T08:53:20.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6668800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6648005920 0x7f66600ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:20.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6668800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6660164840 0x7f666009d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:20.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6668800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f666015e040 0x7f66601f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6668800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6660166120 con 0x7f6660164840 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6668800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66600ccc80 con 0x7f6648005920 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6668800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f66601662a0 con 0x7f666015e040 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6665400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6660164840 0x7f666009d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6665400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6660164840 0x7f666009d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43198/0 (socket says 172.21.15.44:43198) 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6665400640 1 -- 172.21.15.44:0/1311108203 learned_addr learned my addr 172.21.15.44:0/1311108203 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6665e00640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6648005920 0x7f66600ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6666800640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f666015e040 0x7f66601f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6665400640 1 -- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f666015e040 msgr2=0x7f66601f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6665400640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f666015e040 0x7f66601f94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6665400640 1 -- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6648005920 msgr2=0x7f66600ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6665400640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6648005920 0x7f66600ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6665400640 1 -- 172.21.15.44:0/1311108203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66601fa3b0 con 0x7f6660164840 2024-09-13T08:53:20.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6666800640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f666015e040 0x7f66601f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:20.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6665400640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6660164840 0x7f666009d280 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f665001d060 tx=0x7f66500acd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:20.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f665a000640 1 -- 172.21.15.44:0/1311108203 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500b1040 con 0x7f6660164840 2024-09-13T08:53:20.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f665a000640 1 -- 172.21.15.44:0/1311108203 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66500bc070 con 0x7f6660164840 2024-09-13T08:53:20.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f665a000640 1 -- 172.21.15.44:0/1311108203 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650005290 con 0x7f6660164840 2024-09-13T08:53:20.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6668800640 1 -- 172.21.15.44:0/1311108203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6660163ae0 con 0x7f6660164840 2024-09-13T08:53:20.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6668800640 1 -- 172.21.15.44:0/1311108203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6660163d60 con 0x7f6660164840 2024-09-13T08:53:20.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.355+0000 7f6668800640 1 -- 172.21.15.44:0/1311108203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6660069410 con 0x7f6660164840 2024-09-13T08:53:20.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.359+0000 7f665a000640 1 -- 172.21.15.44:0/1311108203 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f6650005430 con 0x7f6660164840 2024-09-13T08:53:20.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.359+0000 7f665a000640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6644075140 0x7f6644077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:20.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.359+0000 7f665a000640 1 -- 172.21.15.44:0/1311108203 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f665012e120 con 0x7f6660164840 2024-09-13T08:53:20.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.359+0000 7f6665e00640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6644075140 0x7f6644077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:20.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.359+0000 7f6665e00640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6644075140 0x7f6644077600 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6654000f70 tx=0x7f6654000f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:20.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.363+0000 7f665a000640 1 -- 172.21.15.44:0/1311108203 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6660163ae0 con 0x7f6660164840 2024-09-13T08:53:20.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.535+0000 7f6668800640 1 -- 172.21.15.44:0/1311108203 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6660158940 con 0x7f6644075140 2024-09-13T08:53:20.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f665a000640 1 -- 172.21.15.44:0/1311108203 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241533 (secure 0 0 0) 0x7f6660158940 con 0x7f6644075140 2024-09-13T08:53:20.547 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:20.548 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:53:20.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f6668800640 1 -- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6644075140 msgr2=0x7f6644077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:20.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f6668800640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6644075140 0x7f6644077600 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6654000f70 tx=0x7f6654000f00 comp rx=0 tx=0).stop 2024-09-13T08:53:20.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f6668800640 1 -- 172.21.15.44:0/1311108203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6660164840 msgr2=0x7f666009d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:20.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f6668800640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6660164840 0x7f666009d280 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f665001d060 tx=0x7f66500acd90 comp rx=0 tx=0).stop 2024-09-13T08:53:20.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f6668800640 1 -- 172.21.15.44:0/1311108203 shutdown_connections 2024-09-13T08:53:20.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f6668800640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6644075140 0x7f6644077600 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f6668800640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f666015e040 0x7f66601f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f6668800640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6660164840 0x7f666009d280 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f6668800640 1 --2- 172.21.15.44:0/1311108203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6648005920 0x7f66600ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.543+0000 7f6668800640 1 -- 172.21.15.44:0/1311108203 >> 172.21.15.44:0/1311108203 conn(0x7f66600b18f0 msgr2=0x7f666015a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:20.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.547+0000 7f6668800640 1 -- 172.21.15.44:0/1311108203 shutdown_connections 2024-09-13T08:53:20.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.547+0000 7f6668800640 1 -- 172.21.15.44:0/1311108203 wait complete. 2024-09-13T08:53:20.566 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2024-09-13T08:53:18.836558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":76,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1095440,"kb_used_data":14876,"kb_used_omap":12,"kb_used_meta":1080371,"kb_avail":753879280,"statfs":{"total":773094113280,"available":771972382720,"internally_reserved":0,"allocated":15233024,"data_stored":7377482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1106300465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252604"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129523+0000","last_change":"2024-09-13T08:53:15.613820+0000","last_active":"2024-09-13T08:53:17.129523+0000","last_peered":"2024-09-13T08:53:17.129523+0000","last_clean":"2024-09-13T08:53:17.129523+0000","last_became_active":"2024-09-13T08:53:15.613531+0000","last_became_peered":"2024-09-13T08:53:15.613531+0000","last_unstale":"2024-09-13T08:53:17.129523+0000","last_undegraded":"2024-09-13T08:53:17.129523+0000","last_fullsized":"2024-09-13T08:53:17.129523+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:16.193181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129499+0000","last_change":"2024-09-13T08:53:15.610999+0000","last_active":"2024-09-13T08:53:17.129499+0000","last_peered":"2024-09-13T08:53:17.129499+0000","last_clean":"2024-09-13T08:53:17.129499+0000","last_became_active":"2024-09-13T08:53:15.610709+0000","last_became_peered":"2024-09-13T08:53:15.610709+0000","last_unstale":"2024-09-13T08:53:17.129499+0000","last_undegraded":"2024-09-13T08:53:17.129499+0000","last_fullsized":"2024-09-13T08:53:17.129499+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:51.632629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129424+0000","last_change":"2024-09-13T08:53:15.608830+0000","last_active":"2024-09-13T08:53:17.129424+0000","last_peered":"2024-09-13T08:53:17.129424+0000","last_clean":"2024-09-13T08:53:17.129424+0000","last_became_active":"2024-09-13T08:53:15.608527+0000","last_became_peered":"2024-09-13T08:53:15.608527+0000","last_unstale":"2024-09-13T08:53:17.129424+0000","last_undegraded":"2024-09-13T08:53:17.129424+0000","last_fullsized":"2024-09-13T08:53:17.129424+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:06.563772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587498+0000","last_change":"2024-09-13T08:53:13.587498+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587498+0000","last_undegraded":"2024-09-13T08:53:13.587498+0000","last_fullsized":"2024-09-13T08:53:13.587498+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129266+0000","last_change":"2024-09-13T08:53:14.613339+0000","last_active":"2024-09-13T08:53:17.129266+0000","last_peered":"2024-09-13T08:53:17.129266+0000","last_clean":"2024-09-13T08:53:17.129266+0000","last_became_active":"2024-09-13T08:53:14.612799+0000","last_became_peered":"2024-09-13T08:53:14.612799+0000","last_unstale":"2024-09-13T08:53:17.129266+0000","last_undegraded":"2024-09-13T08:53:17.129266+0000","last_fullsized":"2024-09-13T08:53:17.129266+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:54.292680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129342+0000","last_change":"2024-09-13T08:53:15.616934+0000","last_active":"2024-09-13T08:53:17.129342+0000","last_peered":"2024-09-13T08:53:17.129342+0000","last_clean":"2024-09-13T08:53:17.129342+0000","last_became_active":"2024-09-13T08:53:15.616660+0000","last_became_peered":"2024-09-13T08:53:15.616660+0000","last_unstale":"2024-09-13T08:53:17.129342+0000","last_undegraded":"2024-09-13T08:53:17.129342+0000","last_fullsized":"2024-09-13T08:53:17.129342+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:52:39.024260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587950+0000","last_change":"2024-09-13T08:53:13.587950+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587950+0000","last_undegraded":"2024-09-13T08:53:13.587950+0000","last_fullsized":"2024-09-13T08:53:13.587950+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589740+0000","last_change":"2024-09-13T08:53:13.589740+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589740+0000","last_undegraded":"2024-09-13T08:53:13.589740+0000","last_fullsized":"2024-09-13T08:53:13.589740+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589133+0000","last_change":"2024-09-13T08:53:13.589133+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589133+0000","last_undegraded":"2024-09-13T08:53:13.589133+0000","last_fullsized":"2024-09-13T08:53:13.589133+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.586318+0000","last_change":"2024-09-13T08:53:13.586318+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.586318+0000","last_undegraded":"2024-09-13T08:53:13.586318+0000","last_fullsized":"2024-09-13T08:53:13.586318+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129340+0000","last_change":"2024-09-13T08:53:14.609106+0000","last_active":"2024-09-13T08:53:17.129340+0000","last_peered":"2024-09-13T08:53:17.129340+0000","last_clean":"2024-09-13T08:53:17.129340+0000","last_became_active":"2024-09-13T08:53:14.608810+0000","last_became_peered":"2024-09-13T08:53:14.608810+0000","last_unstale":"2024-09-13T08:53:17.129340+0000","last_undegraded":"2024-09-13T08:53:17.129340+0000","last_fullsized":"2024-09-13T08:53:17.129340+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:44.027745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587437+0000","last_change":"2024-09-13T08:53:13.587437+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587437+0000","last_undegraded":"2024-09-13T08:53:13.587437+0000","last_fullsized":"2024-09-13T08:53:13.587437+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129413+0000","last_change":"2024-09-13T08:53:14.611525+0000","last_active":"2024-09-13T08:53:17.129413+0000","last_peered":"2024-09-13T08:53:17.129413+0000","last_clean":"2024-09-13T08:53:17.129413+0000","last_became_active":"2024-09-13T08:53:14.611246+0000","last_became_peered":"2024-09-13T08:53:14.611246+0000","last_unstale":"2024-09-13T08:53:17.129413+0000","last_undegraded":"2024-09-13T08:53:17.129413+0000","last_fullsized":"2024-09-13T08:53:17.129413+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:51.437020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.586018+0000","last_change":"2024-09-13T08:53:13.586018+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.586018+0000","last_undegraded":"2024-09-13T08:53:13.586018+0000","last_fullsized":"2024-09-13T08:53:13.586018+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.586573+0000","last_change":"2024-09-13T08:53:13.586573+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.586573+0000","last_undegraded":"2024-09-13T08:53:13.586573+0000","last_fullsized":"2024-09-13T08:53:13.586573+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587990+0000","last_change":"2024-09-13T08:53:13.587990+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587990+0000","last_undegraded":"2024-09-13T08:53:13.587990+0000","last_fullsized":"2024-09-13T08:53:13.587990+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587409+0000","last_change":"2024-09-13T08:53:13.587409+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587409+0000","last_undegraded":"2024-09-13T08:53:13.587409+0000","last_fullsized":"2024-09-13T08:53:13.587409+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590758+0000","last_change":"2024-09-13T08:53:13.590758+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590758+0000","last_undegraded":"2024-09-13T08:53:13.590758+0000","last_fullsized":"2024-09-13T08:53:13.590758+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589385+0000","last_change":"2024-09-13T08:53:13.589385+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589385+0000","last_undegraded":"2024-09-13T08:53:13.589385+0000","last_fullsized":"2024-09-13T08:53:13.589385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592867+0000","last_change":"2024-09-13T08:53:13.592867+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592867+0000","last_undegraded":"2024-09-13T08:53:13.592867+0000","last_fullsized":"2024-09-13T08:53:13.592867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.589224+0000","last_change":"2024-09-13T08:53:13.589224+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.589224+0000","last_undegraded":"2024-09-13T08:53:13.589224+0000","last_fullsized":"2024-09-13T08:53:13.589224+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587069+0000","last_change":"2024-09-13T08:53:13.587069+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587069+0000","last_undegraded":"2024-09-13T08:53:13.587069+0000","last_fullsized":"2024-09-13T08:53:13.587069+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129489+0000","last_change":"2024-09-13T08:53:15.615099+0000","last_active":"2024-09-13T08:53:17.129489+0000","last_peered":"2024-09-13T08:53:17.129489+0000","last_clean":"2024-09-13T08:53:17.129489+0000","last_became_active":"2024-09-13T08:53:15.614834+0000","last_became_peered":"2024-09-13T08:53:15.614834+0000","last_unstale":"2024-09-13T08:53:17.129489+0000","last_undegraded":"2024-09-13T08:53:17.129489+0000","last_fullsized":"2024-09-13T08:53:17.129489+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:34.923374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592525+0000","last_change":"2024-09-13T08:53:13.592525+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592525+0000","last_undegraded":"2024-09-13T08:53:13.592525+0000","last_fullsized":"2024-09-13T08:53:13.592525+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129245+0000","last_change":"2024-09-13T08:53:14.613189+0000","last_active":"2024-09-13T08:53:17.129245+0000","last_peered":"2024-09-13T08:53:17.129245+0000","last_clean":"2024-09-13T08:53:17.129245+0000","last_became_active":"2024-09-13T08:53:14.612877+0000","last_became_peered":"2024-09-13T08:53:14.612877+0000","last_unstale":"2024-09-13T08:53:17.129245+0000","last_undegraded":"2024-09-13T08:53:17.129245+0000","last_fullsized":"2024-09-13T08:53:17.129245+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:12.557688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590616+0000","last_change":"2024-09-13T08:53:13.590616+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590616+0000","last_undegraded":"2024-09-13T08:53:13.590616+0000","last_fullsized":"2024-09-13T08:53:13.590616+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590716+0000","last_change":"2024-09-13T08:53:13.590716+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590716+0000","last_undegraded":"2024-09-13T08:53:13.590716+0000","last_fullsized":"2024-09-13T08:53:13.590716+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592095+0000","last_change":"2024-09-13T08:53:13.592095+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592095+0000","last_undegraded":"2024-09-13T08:53:13.592095+0000","last_fullsized":"2024-09-13T08:53:13.592095+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.592071+0000","last_change":"2024-09-13T08:53:13.592071+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.592071+0000","last_undegraded":"2024-09-13T08:53:13.592071+0000","last_fullsized":"2024-09-13T08:53:13.592071+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590826+0000","last_change":"2024-09-13T08:53:13.590826+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590826+0000","last_undegraded":"2024-09-13T08:53:13.590826+0000","last_fullsized":"2024-09-13T08:53:13.590826+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587534+0000","last_change":"2024-09-13T08:53:13.587534+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587534+0000","last_undegraded":"2024-09-13T08:53:13.587534+0000","last_fullsized":"2024-09-13T08:53:13.587534+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.585896+0000","last_change":"2024-09-13T08:53:13.585896+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.585896+0000","last_undegraded":"2024-09-13T08:53:13.585896+0000","last_fullsized":"2024-09-13T08:53:13.585896+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.590718+0000","last_change":"2024-09-13T08:53:13.590718+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.590718+0000","last_undegraded":"2024-09-13T08:53:13.590718+0000","last_fullsized":"2024-09-13T08:53:13.590718+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587554+0000","last_change":"2024-09-13T08:53:13.587554+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587554+0000","last_undegraded":"2024-09-13T08:53:13.587554+0000","last_fullsized":"2024-09-13T08:53:13.587554+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587486+0000","last_change":"2024-09-13T08:53:13.587486+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587486+0000","last_undegraded":"2024-09-13T08:53:13.587486+0000","last_fullsized":"2024-09-13T08:53:13.587486+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129123+0000","last_change":"2024-09-13T08:53:14.610914+0000","last_active":"2024-09-13T08:53:17.129123+0000","last_peered":"2024-09-13T08:53:17.129123+0000","last_clean":"2024-09-13T08:53:17.129123+0000","last_became_active":"2024-09-13T08:53:14.610635+0000","last_became_peered":"2024-09-13T08:53:14.610635+0000","last_unstale":"2024-09-13T08:53:17.129123+0000","last_undegraded":"2024-09-13T08:53:17.129123+0000","last_fullsized":"2024-09-13T08:53:17.129123+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:04.024112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T08:53:14.580538+0000","last_change":"2024-09-13T08:53:14.580538+0000","last_active":"2024-09-13T08:53:14.580538+0000","last_peered":"2024-09-13T08:53:14.580538+0000","last_clean":"2024-09-13T08:53:14.580538+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:14.580538+0000","last_undegraded":"2024-09-13T08:53:14.580538+0000","last_fullsized":"2024-09-13T08:53:14.580538+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-13T08:53:13.587419+0000","last_change":"2024-09-13T08:53:13.587419+0000","last_active":"2024-09-13T08:53:13.573742+0000","last_peered":"2024-09-13T08:53:13.573742+0000","last_clean":"2024-09-13T08:53:13.573742+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T08:53:13.587419+0000","last_undegraded":"2024-09-13T08:53:13.587419+0000","last_fullsized":"2024-09-13T08:53:13.587419+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":340,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T08:53:13.582720+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:53:13.582720+0000","last_peered":"2024-09-13T08:53:13.582720+0000","last_clean":"2024-09-13T08:53:13.582720+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:53:13.582720+0000","last_undegraded":"2024-09-13T08:53:13.582720+0000","last_fullsized":"2024-09-13T08:53:13.582720+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127228,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244612,"statfs":{"total":96636764160,"available":96506482688,"internally_reserved":0,"allocated":1744896,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509632,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119500,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252340,"statfs":{"total":96636764160,"available":96514396160,"internally_reserved":0,"allocated":2338816,"data_stored":1359536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509633,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153608,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218232,"statfs":{"total":96636764160,"available":96479469568,"internally_reserved":0,"allocated":1757184,"data_stored":775097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509633,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131392,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240448,"statfs":{"total":96636764160,"available":96502218752,"internally_reserved":0,"allocated":1744896,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131388,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240452,"statfs":{"total":96636764160,"available":96502222848,"internally_reserved":0,"allocated":1744896,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509634,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145868,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225972,"statfs":{"total":96636764160,"available":96487395328,"internally_reserved":0,"allocated":2351104,"data_stored":1365465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145288,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226552,"statfs":{"total":96636764160,"available":96487989248,"internally_reserved":0,"allocated":1757184,"data_stored":775097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542339,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141168,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230672,"statfs":{"total":96636764160,"available":96492208128,"internally_reserved":0,"allocated":1794048,"data_stored":794783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:53:20.570 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3f is not active+clean 2024-09-13T08:53:20.570 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3e is not active+clean 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3d is not active+clean 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3c is not active+clean 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3b is not active+clean 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:PG 21.39 is not active+clean 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.571 INFO:tasks.ceph.ceph_manager.ceph:PG 21.38 is not active+clean 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:PG 21.36 is not active+clean 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:PG 21.34 is not active+clean 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:PG 21.33 is not active+clean 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:PG 21.32 is not active+clean 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:PG 21.31 is not active+clean 2024-09-13T08:53:20.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.573 INFO:tasks.ceph.ceph_manager.ceph:PG 21.30 is not active+clean 2024-09-13T08:53:20.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.573 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2f is not active+clean 2024-09-13T08:53:20.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.573 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2e is not active+clean 2024-09-13T08:53:20.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.573 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2d is not active+clean 2024-09-13T08:53:20.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.573 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2c is not active+clean 2024-09-13T08:53:20.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2b is not active+clean 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2a is not active+clean 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:PG 21.29 is not active+clean 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:PG 21.28 is not active+clean 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:PG 21.27 is not active+clean 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:PG 21.26 is not active+clean 2024-09-13T08:53:20.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:PG 21.25 is not active+clean 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:PG 21.24 is not active+clean 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:PG 21.23 is not active+clean 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:PG 21.22 is not active+clean 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:PG 21.21 is not active+clean 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:PG 21.20 is not active+clean 2024-09-13T08:53:20.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.576 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1e is not active+clean 2024-09-13T08:53:20.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.576 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1f is not active+clean 2024-09-13T08:53:20.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.587498+0000', 'last_change': '2024-09-13T08:53:13.587498+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.587498+0000', 'last_undegraded': '2024-09-13T08:53:13.587498+0000', 'last_fullsized': '2024-09-13T08:53:13.587498+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:53:20.576 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1f is not active+clean 2024-09-13T08:53:20.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.576 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1d is not active+clean 2024-09-13T08:53:20.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.587950+0000', 'last_change': '2024-09-13T08:53:13.587950+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.587950+0000', 'last_undegraded': '2024-09-13T08:53:13.587950+0000', 'last_fullsized': '2024-09-13T08:53:13.587950+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:53:20.576 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1d is not active+clean 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1c is not active+clean 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.589740+0000', 'last_change': '2024-09-13T08:53:13.589740+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.589740+0000', 'last_undegraded': '2024-09-13T08:53:13.589740+0000', 'last_fullsized': '2024-09-13T08:53:13.589740+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1a is not active+clean 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1b is not active+clean 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.589133+0000', 'last_change': '2024-09-13T08:53:13.589133+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.589133+0000', 'last_undegraded': '2024-09-13T08:53:13.589133+0000', 'last_fullsized': '2024-09-13T08:53:13.589133+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1b is not active+clean 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1a is not active+clean 2024-09-13T08:53:20.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.586318+0000', 'last_change': '2024-09-13T08:53:13.586318+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.586318+0000', 'last_undegraded': '2024-09-13T08:53:13.586318+0000', 'last_fullsized': '2024-09-13T08:53:13.586318+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:PG 21.18 is not active+clean 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:PG 21.19 is not active+clean 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:PG 20.18 is not active+clean 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.587437+0000', 'last_change': '2024-09-13T08:53:13.587437+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.587437+0000', 'last_undegraded': '2024-09-13T08:53:13.587437+0000', 'last_fullsized': '2024-09-13T08:53:13.587437+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:PG 21.16 is not active+clean 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:PG 21.17 is not active+clean 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.578 INFO:tasks.ceph.ceph_manager.ceph:PG 20.16 is not active+clean 2024-09-13T08:53:20.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.586018+0000', 'last_change': '2024-09-13T08:53:13.586018+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.586018+0000', 'last_undegraded': '2024-09-13T08:53:13.586018+0000', 'last_fullsized': '2024-09-13T08:53:13.586018+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T08:53:20.579 INFO:tasks.ceph.ceph_manager.ceph:PG 21.15 is not active+clean 2024-09-13T08:53:20.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.579 INFO:tasks.ceph.ceph_manager.ceph:PG 20.14 is not active+clean 2024-09-13T08:53:20.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.586573+0000', 'last_change': '2024-09-13T08:53:13.586573+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.586573+0000', 'last_undegraded': '2024-09-13T08:53:13.586573+0000', 'last_fullsized': '2024-09-13T08:53:13.586573+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:53:20.579 INFO:tasks.ceph.ceph_manager.ceph:PG 21.12 is not active+clean 2024-09-13T08:53:20.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.579 INFO:tasks.ceph.ceph_manager.ceph:PG 20.13 is not active+clean 2024-09-13T08:53:20.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.587990+0000', 'last_change': '2024-09-13T08:53:13.587990+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.587990+0000', 'last_undegraded': '2024-09-13T08:53:13.587990+0000', 'last_fullsized': '2024-09-13T08:53:13.587990+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:53:20.579 INFO:tasks.ceph.ceph_manager.ceph:PG 21.13 is not active+clean 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:PG 20.12 is not active+clean 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.587409+0000', 'last_change': '2024-09-13T08:53:13.587409+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.587409+0000', 'last_undegraded': '2024-09-13T08:53:13.587409+0000', 'last_fullsized': '2024-09-13T08:53:13.587409+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:PG 21.10 is not active+clean 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:PG 20.11 is not active+clean 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.590758+0000', 'last_change': '2024-09-13T08:53:13.590758+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.590758+0000', 'last_undegraded': '2024-09-13T08:53:13.590758+0000', 'last_fullsized': '2024-09-13T08:53:13.590758+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:PG 21.11 is not active+clean 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:PG 20.10 is not active+clean 2024-09-13T08:53:20.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.589385+0000', 'last_change': '2024-09-13T08:53:13.589385+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.589385+0000', 'last_undegraded': '2024-09-13T08:53:13.589385+0000', 'last_fullsized': '2024-09-13T08:53:13.589385+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:PG 21.e is not active+clean 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:PG 20.f is not active+clean 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.592867+0000', 'last_change': '2024-09-13T08:53:13.592867+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.592867+0000', 'last_undegraded': '2024-09-13T08:53:13.592867+0000', 'last_fullsized': '2024-09-13T08:53:13.592867+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:PG 21.f is not active+clean 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:PG 20.e is not active+clean 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.589224+0000', 'last_change': '2024-09-13T08:53:13.589224+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.589224+0000', 'last_undegraded': '2024-09-13T08:53:13.589224+0000', 'last_fullsized': '2024-09-13T08:53:13.589224+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:PG 21.c is not active+clean 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.581 INFO:tasks.ceph.ceph_manager.ceph:PG 20.d is not active+clean 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.587069+0000', 'last_change': '2024-09-13T08:53:13.587069+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.587069+0000', 'last_undegraded': '2024-09-13T08:53:13.587069+0000', 'last_fullsized': '2024-09-13T08:53:13.587069+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:PG 20.c is not active+clean 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.592525+0000', 'last_change': '2024-09-13T08:53:13.592525+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.592525+0000', 'last_undegraded': '2024-09-13T08:53:13.592525+0000', 'last_fullsized': '2024-09-13T08:53:13.592525+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:PG 21.a is not active+clean 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:PG 21.b is not active+clean 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:PG 20.a is not active+clean 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.590616+0000', 'last_change': '2024-09-13T08:53:13.590616+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.590616+0000', 'last_undegraded': '2024-09-13T08:53:13.590616+0000', 'last_fullsized': '2024-09-13T08:53:13.590616+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:PG 21.8 is not active+clean 2024-09-13T08:53:20.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.583 INFO:tasks.ceph.ceph_manager.ceph:PG 20.9 is not active+clean 2024-09-13T08:53:20.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.590716+0000', 'last_change': '2024-09-13T08:53:13.590716+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.590716+0000', 'last_undegraded': '2024-09-13T08:53:13.590716+0000', 'last_fullsized': '2024-09-13T08:53:13.590716+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:53:20.583 INFO:tasks.ceph.ceph_manager.ceph:PG 21.9 is not active+clean 2024-09-13T08:53:20.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.583 INFO:tasks.ceph.ceph_manager.ceph:PG 20.8 is not active+clean 2024-09-13T08:53:20.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.592095+0000', 'last_change': '2024-09-13T08:53:13.592095+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.592095+0000', 'last_undegraded': '2024-09-13T08:53:13.592095+0000', 'last_fullsized': '2024-09-13T08:53:13.592095+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:53:20.583 INFO:tasks.ceph.ceph_manager.ceph:PG 21.6 is not active+clean 2024-09-13T08:53:20.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.583 INFO:tasks.ceph.ceph_manager.ceph:PG 20.7 is not active+clean 2024-09-13T08:53:20.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.592071+0000', 'last_change': '2024-09-13T08:53:13.592071+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.592071+0000', 'last_undegraded': '2024-09-13T08:53:13.592071+0000', 'last_fullsized': '2024-09-13T08:53:13.592071+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:PG 21.7 is not active+clean 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:PG 20.6 is not active+clean 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.590826+0000', 'last_change': '2024-09-13T08:53:13.590826+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.590826+0000', 'last_undegraded': '2024-09-13T08:53:13.590826+0000', 'last_fullsized': '2024-09-13T08:53:13.590826+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:PG 21.4 is not active+clean 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:PG 20.5 is not active+clean 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.587534+0000', 'last_change': '2024-09-13T08:53:13.587534+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.587534+0000', 'last_undegraded': '2024-09-13T08:53:13.587534+0000', 'last_fullsized': '2024-09-13T08:53:13.587534+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:PG 21.5 is not active+clean 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.584 INFO:tasks.ceph.ceph_manager.ceph:PG 20.4 is not active+clean 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.585896+0000', 'last_change': '2024-09-13T08:53:13.585896+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.585896+0000', 'last_undegraded': '2024-09-13T08:53:13.585896+0000', 'last_fullsized': '2024-09-13T08:53:13.585896+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2 is not active+clean 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:PG 20.3 is not active+clean 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.590718+0000', 'last_change': '2024-09-13T08:53:13.590718+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.590718+0000', 'last_undegraded': '2024-09-13T08:53:13.590718+0000', 'last_fullsized': '2024-09-13T08:53:13.590718+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3 is not active+clean 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:PG 20.2 is not active+clean 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.587554+0000', 'last_change': '2024-09-13T08:53:13.587554+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.587554+0000', 'last_undegraded': '2024-09-13T08:53:13.587554+0000', 'last_fullsized': '2024-09-13T08:53:13.587554+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:PG 21.0 is not active+clean 2024-09-13T08:53:20.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.586 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1 is not active+clean 2024-09-13T08:53:20.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.587486+0000', 'last_change': '2024-09-13T08:53:13.587486+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.587486+0000', 'last_undegraded': '2024-09-13T08:53:13.587486+0000', 'last_fullsized': '2024-09-13T08:53:13.587486+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:53:20.586 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1 is not active+clean 2024-09-13T08:53:20.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.586 INFO:tasks.ceph.ceph_manager.ceph:PG 21.14 is not active+clean 2024-09-13T08:53:20.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T08:53:14.580538+0000', 'last_change': '2024-09-13T08:53:14.580538+0000', 'last_active': '2024-09-13T08:53:14.580538+0000', 'last_peered': '2024-09-13T08:53:14.580538+0000', 'last_clean': '2024-09-13T08:53:14.580538+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:14.580538+0000', 'last_undegraded': '2024-09-13T08:53:14.580538+0000', 'last_fullsized': '2024-09-13T08:53:14.580538+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:14.580538+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T08:53:20.586 INFO:tasks.ceph.ceph_manager.ceph:PG 20.15 is not active+clean 2024-09-13T08:53:20.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-13T08:53:13.587419+0000', 'last_change': '2024-09-13T08:53:13.587419+0000', 'last_active': '2024-09-13T08:53:13.573742+0000', 'last_peered': '2024-09-13T08:53:13.573742+0000', 'last_clean': '2024-09-13T08:53:13.573742+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T08:53:13.587419+0000', 'last_undegraded': '2024-09-13T08:53:13.587419+0000', 'last_fullsized': '2024-09-13T08:53:13.587419+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'last_clean_scrub_stamp': '2024-09-13T08:53:13.573742+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T08:53:20.587 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T08:53:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 -- 172.21.15.44:0/3046713988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9058160400 msgr2=0x7f9058160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 --2- 172.21.15.44:0/3046713988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9058160400 0x7f9058160830 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f9058073ff0 tx=0x7f905001cc90 comp rx=0 tx=0).stop 2024-09-13T08:53:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 -- 172.21.15.44:0/3046713988 shutdown_connections 2024-09-13T08:53:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 --2- 172.21.15.44:0/3046713988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9058160400 0x7f9058160830 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 --2- 172.21.15.44:0/3046713988 >> v2:172.21.15.129:3300/0 conn(0x7f90580b2d00 0x7f905815fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 --2- 172.21.15.44:0/3046713988 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90580cfe80 0x7f90580d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 -- 172.21.15.44:0/3046713988 >> 172.21.15.44:0/3046713988 conn(0x7f90580d1e40 msgr2=0x7f90580d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 -- 172.21.15.44:0/3046713988 shutdown_connections 2024-09-13T08:53:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 -- 172.21.15.44:0/3046713988 wait complete. 2024-09-13T08:53:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 Processor -- start 2024-09-13T08:53:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.751+0000 7f905fa00640 1 -- start start 2024-09-13T08:53:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905fa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90580b2d00 0x7f90580b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905fa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90580cfe80 0x7f9058152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905fa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9058160400 0x7f9058152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905fa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90580b27e0 con 0x7f90580b2d00 2024-09-13T08:53:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905fa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90580b2660 con 0x7f90580cfe80 2024-09-13T08:53:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905fa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f90580b2960 con 0x7f9058160400 2024-09-13T08:53:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90580b2d00 0x7f90580b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90580b2d00 0x7f90580b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43204/0 (socket says 172.21.15.44:43204) 2024-09-13T08:53:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905d000640 1 -- 172.21.15.44:0/2587234120 learned_addr learned my addr 172.21.15.44:0/2587234120 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905d000640 1 -- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9058160400 msgr2=0x7f9058152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905d000640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9058160400 0x7f9058152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905d000640 1 -- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90580cfe80 msgr2=0x7f9058152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905d000640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90580cfe80 0x7f9058152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905d000640 1 -- 172.21.15.44:0/2587234120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905815c340 con 0x7f90580b2d00 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905d000640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90580b2d00 0x7f90580b5ee0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c970 tx=0x7f904c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f9054c00640 1 -- 172.21.15.44:0/2587234120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c007c70 con 0x7f90580b2d00 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f9054c00640 1 -- 172.21.15.44:0/2587234120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f904c00f450 con 0x7f90580b2d00 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905fa00640 1 -- 172.21.15.44:0/2587234120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905815c620 con 0x7f90580b2d00 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f9054c00640 1 -- 172.21.15.44:0/2587234120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c00f8a0 con 0x7f90580b2d00 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905fa00640 1 -- 172.21.15.44:0/2587234120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905815bf80 con 0x7f90580b2d00 2024-09-13T08:53:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.755+0000 7f905fa00640 1 -- 172.21.15.44:0/2587234120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905815ca10 con 0x7f90580b2d00 2024-09-13T08:53:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.759+0000 7f9054c00640 1 -- 172.21.15.44:0/2587234120 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f904c00fa40 con 0x7f90580b2d00 2024-09-13T08:53:20.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.759+0000 7f9054c00640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9038075110 0x7f90380775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:20.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.759+0000 7f9054c00640 1 -- 172.21.15.44:0/2587234120 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f904c093870 con 0x7f90580b2d00 2024-09-13T08:53:20.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.759+0000 7f9057e00640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9038075110 0x7f90380775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:20.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.759+0000 7f9057e00640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9038075110 0x7f90380775d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f90581539a0 tx=0x7f9044005c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:20.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:20.763+0000 7f9054c00640 1 -- 172.21.15.44:0/2587234120 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f904c014030 con 0x7f90580b2d00 2024-09-13T08:53:21.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 -- 172.21.15.44:0/2587234120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f905815c7f0 con 0x7f90580b2d00 2024-09-13T08:53:21.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f9054c00640 1 -- 172.21.15.44:0/2587234120 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2831 (secure 0 0 0) 0x7f904c060520 con 0x7f90580b2d00 2024-09-13T08:53:21.005 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:21.005 INFO:teuthology.orchestra.run.smithi044.stdout:{"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":971,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":144,"num_osds":8,"num_up_osds":8,"osd_up_since":1726216643,"num_in_osds":8,"osd_in_since":1726216632,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":59},{"state_name":"creating+peering","count":27},{"state_name":"active+clean","count":11}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":1121730560,"bytes_avail":771972382720,"bytes_total":773094113280,"unknown_pgs_ratio":0.60824739933013916,"inactive_pgs_ratio":0.27835050225257874,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":95,"btime":"2024-09-13T08:53:18:619363+0000","id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9219}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-09-13T08:53:16.836210+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-13T08:53:21.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 -- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9038075110 msgr2=0x7f90380775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:21.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9038075110 0x7f90380775d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f90581539a0 tx=0x7f9044005c10 comp rx=0 tx=0).stop 2024-09-13T08:53:21.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 -- 172.21.15.44:0/2587234120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90580b2d00 msgr2=0x7f90580b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:21.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90580b2d00 0x7f90580b5ee0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c970 tx=0x7f904c00ce30 comp rx=0 tx=0).stop 2024-09-13T08:53:21.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 -- 172.21.15.44:0/2587234120 shutdown_connections 2024-09-13T08:53:21.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9038075110 0x7f90380775d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f9058160400 0x7f9058152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90580cfe80 0x7f9058152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 --2- 172.21.15.44:0/2587234120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90580b2d00 0x7f90580b5ee0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 -- 172.21.15.44:0/2587234120 >> 172.21.15.44:0/2587234120 conn(0x7f90580d1e40 msgr2=0x7f9058159b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:21.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 -- 172.21.15.44:0/2587234120 shutdown_connections 2024-09-13T08:53:21.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.003+0000 7f905fa00640 1 -- 172.21.15.44:0/2587234120 wait complete. 2024-09-13T08:53:21.027 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-13T08:53:21.027 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:53:21.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.171+0000 7fb785600640 1 -- 172.21.15.44:0/2381385947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800b4cf0 msgr2=0x7fb7800b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:21.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.171+0000 7fb785600640 1 --2- 172.21.15.44:0/2381385947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800b4cf0 0x7fb7800b3130 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fb7800b2680 tx=0x7fb77801ccf0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.171+0000 7fb785600640 1 -- 172.21.15.44:0/2381385947 shutdown_connections 2024-09-13T08:53:21.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.171+0000 7fb785600640 1 --2- 172.21.15.44:0/2381385947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800b4cf0 0x7fb7800b3130 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.171+0000 7fb785600640 1 --2- 172.21.15.44:0/2381385947 >> v2:172.21.15.129:3300/0 conn(0x7fb78015e260 0x7fb7800b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.171+0000 7fb785600640 1 -- 172.21.15.44:0/2381385947 >> 172.21.15.44:0/2381385947 conn(0x7fb7800b18f0 msgr2=0x7fb7800b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:21.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.171+0000 7fb785600640 1 -- 172.21.15.44:0/2381385947 shutdown_connections 2024-09-13T08:53:21.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.171+0000 7fb785600640 1 -- 172.21.15.44:0/2381385947 wait complete. 2024-09-13T08:53:21.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb785600640 1 Processor -- start 2024-09-13T08:53:21.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb785600640 1 -- start start 2024-09-13T08:53:21.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb785600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7800b4cf0 0x7fb780159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:21.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb785600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb78015e260 0x7fb78014f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb785600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801595d0 0x7fb7801f9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb785600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7801682e0 con 0x7fb7801595d0 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb785600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb780168160 con 0x7fb7800b4cf0 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb785600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb780168460 con 0x7fb78015e260 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb77f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801595d0 0x7fb7801f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb77f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801595d0 0x7fb7801f9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43208/0 (socket says 172.21.15.44:43208) 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb77f400640 1 -- 172.21.15.44:0/2619988874 learned_addr learned my addr 172.21.15.44:0/2619988874 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb77ea00640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7800b4cf0 0x7fb780159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb777e00640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb78015e260 0x7fb78014f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb77f400640 1 -- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb78015e260 msgr2=0x7fb78014f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb77f400640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb78015e260 0x7fb78014f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb77f400640 1 -- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7800b4cf0 msgr2=0x7fb780159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb77f400640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7800b4cf0 0x7fb780159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb77f400640 1 -- 172.21.15.44:0/2619988874 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78015ca50 con 0x7fb7801595d0 2024-09-13T08:53:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb77f400640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801595d0 0x7fb7801f9420 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb778019900 tx=0x7fb778009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb776a00640 1 -- 172.21.15.44:0/2619988874 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7780bb070 con 0x7fb7801595d0 2024-09-13T08:53:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb776a00640 1 -- 172.21.15.44:0/2619988874 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb778004d40 con 0x7fb7801595d0 2024-09-13T08:53:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb776a00640 1 -- 172.21.15.44:0/2619988874 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7780b2040 con 0x7fb7801595d0 2024-09-13T08:53:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb785600640 1 -- 172.21.15.44:0/2619988874 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78015cd30 con 0x7fb7801595d0 2024-09-13T08:53:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.175+0000 7fb785600640 1 -- 172.21.15.44:0/2619988874 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78015d2f0 con 0x7fb7801595d0 2024-09-13T08:53:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.179+0000 7fb785600640 1 -- 172.21.15.44:0/2619988874 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb780069410 con 0x7fb7801595d0 2024-09-13T08:53:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.179+0000 7fb776a00640 1 -- 172.21.15.44:0/2619988874 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7fb778004200 con 0x7fb7801595d0 2024-09-13T08:53:21.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.179+0000 7fb776a00640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb75c075110 0x7fb75c0775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:21.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.179+0000 7fb776a00640 1 -- 172.21.15.44:0/2619988874 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fb77812ccb0 con 0x7fb7801595d0 2024-09-13T08:53:21.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.179+0000 7fb77ea00640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb75c075110 0x7fb75c0775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:21.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.179+0000 7fb77ea00640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb75c075110 0x7fb75c0775d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb78009e250 tx=0x7fb77000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:21.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.183+0000 7fb776a00640 1 -- 172.21.15.44:0/2619988874 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb7780f2060 con 0x7fb7801595d0 2024-09-13T08:53:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.359+0000 7fb785600640 1 -- 172.21.15.44:0/2619988874 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb7800cfef0 con 0x7fb75c075110 2024-09-13T08:53:21.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.363+0000 7fb776a00640 1 -- 172.21.15.44:0/2619988874 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252497 (secure 0 0 0) 0x7fb7800cfef0 con 0x7fb75c075110 2024-09-13T08:53:21.368 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:21.369 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:53:21.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 -- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb75c075110 msgr2=0x7fb75c0775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:21.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb75c075110 0x7fb75c0775d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb78009e250 tx=0x7fb77000b2e0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 -- 172.21.15.44:0/2619988874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801595d0 msgr2=0x7fb7801f9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801595d0 0x7fb7801f9420 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb778019900 tx=0x7fb778009370 comp rx=0 tx=0).stop 2024-09-13T08:53:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 -- 172.21.15.44:0/2619988874 shutdown_connections 2024-09-13T08:53:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb75c075110 0x7fb75c0775d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801595d0 0x7fb7801f9420 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb78015e260 0x7fb78014f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 --2- 172.21.15.44:0/2619988874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7800b4cf0 0x7fb780159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 -- 172.21.15.44:0/2619988874 >> 172.21.15.44:0/2619988874 conn(0x7fb7800b18f0 msgr2=0x7fb780159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 -- 172.21.15.44:0/2619988874 shutdown_connections 2024-09-13T08:53:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:21.367+0000 7fb785600640 1 -- 172.21.15.44:0/2619988874 wait complete. 2024-09-13T08:53:21.390 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":616,"stamp":"2024-09-13T08:53:20.837064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1083336,"kb_used_data":15240,"kb_used_omap":12,"kb_used_meta":1067891,"kb_avail":753891384,"statfs":{"total":773094113280,"available":771984777216,"internally_reserved":0,"allocated":15605760,"data_stored":7543820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1093520945},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.246938"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614737+0000","last_change":"2024-09-13T08:53:15.623539+0000","last_active":"2024-09-13T08:53:16.614737+0000","last_peered":"2024-09-13T08:53:16.614737+0000","last_clean":"2024-09-13T08:53:16.614737+0000","last_became_active":"2024-09-13T08:53:15.623156+0000","last_became_peered":"2024-09-13T08:53:15.623156+0000","last_unstale":"2024-09-13T08:53:16.614737+0000","last_undegraded":"2024-09-13T08:53:16.614737+0000","last_fullsized":"2024-09-13T08:53:16.614737+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:01.640017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623510+0000","last_change":"2024-09-13T08:53:15.618080+0000","last_active":"2024-09-13T08:53:16.623510+0000","last_peered":"2024-09-13T08:53:16.623510+0000","last_clean":"2024-09-13T08:53:16.623510+0000","last_became_active":"2024-09-13T08:53:15.617177+0000","last_became_peered":"2024-09-13T08:53:15.617177+0000","last_unstale":"2024-09-13T08:53:16.623510+0000","last_undegraded":"2024-09-13T08:53:16.623510+0000","last_fullsized":"2024-09-13T08:53:16.623510+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:02:49.554865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626931+0000","last_change":"2024-09-13T08:53:15.607705+0000","last_active":"2024-09-13T08:53:16.626931+0000","last_peered":"2024-09-13T08:53:16.626931+0000","last_clean":"2024-09-13T08:53:16.626931+0000","last_became_active":"2024-09-13T08:53:15.607426+0000","last_became_peered":"2024-09-13T08:53:15.607426+0000","last_unstale":"2024-09-13T08:53:16.626931+0000","last_undegraded":"2024-09-13T08:53:16.626931+0000","last_fullsized":"2024-09-13T08:53:16.626931+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:30.659179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624753+0000","last_change":"2024-09-13T08:53:15.612087+0000","last_active":"2024-09-13T08:53:16.624753+0000","last_peered":"2024-09-13T08:53:16.624753+0000","last_clean":"2024-09-13T08:53:16.624753+0000","last_became_active":"2024-09-13T08:53:15.611760+0000","last_became_peered":"2024-09-13T08:53:15.611760+0000","last_unstale":"2024-09-13T08:53:16.624753+0000","last_undegraded":"2024-09-13T08:53:16.624753+0000","last_fullsized":"2024-09-13T08:53:16.624753+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:41.555046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127127+0000","last_change":"2024-09-13T08:53:15.609980+0000","last_active":"2024-09-13T08:53:17.127127+0000","last_peered":"2024-09-13T08:53:17.127127+0000","last_clean":"2024-09-13T08:53:17.127127+0000","last_became_active":"2024-09-13T08:53:15.609388+0000","last_became_peered":"2024-09-13T08:53:15.609388+0000","last_unstale":"2024-09-13T08:53:17.127127+0000","last_undegraded":"2024-09-13T08:53:17.127127+0000","last_fullsized":"2024-09-13T08:53:17.127127+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:54.955446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129523+0000","last_change":"2024-09-13T08:53:15.613820+0000","last_active":"2024-09-13T08:53:17.129523+0000","last_peered":"2024-09-13T08:53:17.129523+0000","last_clean":"2024-09-13T08:53:17.129523+0000","last_became_active":"2024-09-13T08:53:15.613531+0000","last_became_peered":"2024-09-13T08:53:15.613531+0000","last_unstale":"2024-09-13T08:53:17.129523+0000","last_undegraded":"2024-09-13T08:53:17.129523+0000","last_fullsized":"2024-09-13T08:53:17.129523+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:16.193181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614649+0000","last_change":"2024-09-13T08:53:15.607043+0000","last_active":"2024-09-13T08:53:16.614649+0000","last_peered":"2024-09-13T08:53:16.614649+0000","last_clean":"2024-09-13T08:53:16.614649+0000","last_became_active":"2024-09-13T08:53:15.606686+0000","last_became_peered":"2024-09-13T08:53:15.606686+0000","last_unstale":"2024-09-13T08:53:16.614649+0000","last_undegraded":"2024-09-13T08:53:16.614649+0000","last_fullsized":"2024-09-13T08:53:16.614649+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:14.624303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614228+0000","last_change":"2024-09-13T08:53:15.625701+0000","last_active":"2024-09-13T08:53:16.614228+0000","last_peered":"2024-09-13T08:53:16.614228+0000","last_clean":"2024-09-13T08:53:16.614228+0000","last_became_active":"2024-09-13T08:53:15.625386+0000","last_became_peered":"2024-09-13T08:53:15.625386+0000","last_unstale":"2024-09-13T08:53:16.614228+0000","last_undegraded":"2024-09-13T08:53:16.614228+0000","last_fullsized":"2024-09-13T08:53:16.614228+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:35.675402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129499+0000","last_change":"2024-09-13T08:53:15.610999+0000","last_active":"2024-09-13T08:53:17.129499+0000","last_peered":"2024-09-13T08:53:17.129499+0000","last_clean":"2024-09-13T08:53:17.129499+0000","last_became_active":"2024-09-13T08:53:15.610709+0000","last_became_peered":"2024-09-13T08:53:15.610709+0000","last_unstale":"2024-09-13T08:53:17.129499+0000","last_undegraded":"2024-09-13T08:53:17.129499+0000","last_fullsized":"2024-09-13T08:53:17.129499+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:51.632629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.627107+0000","last_change":"2024-09-13T08:53:15.605883+0000","last_active":"2024-09-13T08:53:16.627107+0000","last_peered":"2024-09-13T08:53:16.627107+0000","last_clean":"2024-09-13T08:53:16.627107+0000","last_became_active":"2024-09-13T08:53:15.604354+0000","last_became_peered":"2024-09-13T08:53:15.604354+0000","last_unstale":"2024-09-13T08:53:16.627107+0000","last_undegraded":"2024-09-13T08:53:16.627107+0000","last_fullsized":"2024-09-13T08:53:16.627107+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:36.209208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129424+0000","last_change":"2024-09-13T08:53:15.608830+0000","last_active":"2024-09-13T08:53:17.129424+0000","last_peered":"2024-09-13T08:53:17.129424+0000","last_clean":"2024-09-13T08:53:17.129424+0000","last_became_active":"2024-09-13T08:53:15.608527+0000","last_became_peered":"2024-09-13T08:53:15.608527+0000","last_unstale":"2024-09-13T08:53:17.129424+0000","last_undegraded":"2024-09-13T08:53:17.129424+0000","last_fullsized":"2024-09-13T08:53:17.129424+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:06.563772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614175+0000","last_change":"2024-09-13T08:53:15.612419+0000","last_active":"2024-09-13T08:53:16.614175+0000","last_peered":"2024-09-13T08:53:16.614175+0000","last_clean":"2024-09-13T08:53:16.614175+0000","last_became_active":"2024-09-13T08:53:15.612145+0000","last_became_peered":"2024-09-13T08:53:15.612145+0000","last_unstale":"2024-09-13T08:53:16.614175+0000","last_undegraded":"2024-09-13T08:53:16.614175+0000","last_fullsized":"2024-09-13T08:53:16.614175+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:57:41.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127220+0000","last_change":"2024-09-13T08:53:15.609662+0000","last_active":"2024-09-13T08:53:17.127220+0000","last_peered":"2024-09-13T08:53:17.127220+0000","last_clean":"2024-09-13T08:53:17.127220+0000","last_became_active":"2024-09-13T08:53:15.609024+0000","last_became_peered":"2024-09-13T08:53:15.609024+0000","last_unstale":"2024-09-13T08:53:17.127220+0000","last_undegraded":"2024-09-13T08:53:17.127220+0000","last_fullsized":"2024-09-13T08:53:17.127220+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:09.589352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.613921+0000","last_change":"2024-09-13T08:53:15.615520+0000","last_active":"2024-09-13T08:53:16.613921+0000","last_peered":"2024-09-13T08:53:16.613921+0000","last_clean":"2024-09-13T08:53:16.613921+0000","last_became_active":"2024-09-13T08:53:15.614440+0000","last_became_peered":"2024-09-13T08:53:15.614440+0000","last_unstale":"2024-09-13T08:53:16.613921+0000","last_undegraded":"2024-09-13T08:53:16.613921+0000","last_fullsized":"2024-09-13T08:53:16.613921+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:41.555046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127125+0000","last_change":"2024-09-13T08:53:15.612875+0000","last_active":"2024-09-13T08:53:17.127125+0000","last_peered":"2024-09-13T08:53:17.127125+0000","last_clean":"2024-09-13T08:53:17.127125+0000","last_became_active":"2024-09-13T08:53:15.612614+0000","last_became_peered":"2024-09-13T08:53:15.612614+0000","last_unstale":"2024-09-13T08:53:17.127125+0000","last_undegraded":"2024-09-13T08:53:17.127125+0000","last_fullsized":"2024-09-13T08:53:17.127125+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:08.675301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614218+0000","last_change":"2024-09-13T08:53:15.601508+0000","last_active":"2024-09-13T08:53:16.614218+0000","last_peered":"2024-09-13T08:53:16.614218+0000","last_clean":"2024-09-13T08:53:16.614218+0000","last_became_active":"2024-09-13T08:53:15.601217+0000","last_became_peered":"2024-09-13T08:53:15.601217+0000","last_unstale":"2024-09-13T08:53:16.614218+0000","last_undegraded":"2024-09-13T08:53:16.614218+0000","last_fullsized":"2024-09-13T08:53:16.614218+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:08.001520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.627281+0000","last_change":"2024-09-13T08:53:15.604129+0000","last_active":"2024-09-13T08:53:16.627281+0000","last_peered":"2024-09-13T08:53:16.627281+0000","last_clean":"2024-09-13T08:53:16.627281+0000","last_became_active":"2024-09-13T08:53:15.603844+0000","last_became_peered":"2024-09-13T08:53:15.603844+0000","last_unstale":"2024-09-13T08:53:16.627281+0000","last_undegraded":"2024-09-13T08:53:16.627281+0000","last_fullsized":"2024-09-13T08:53:16.627281+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:53.045366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127225+0000","last_change":"2024-09-13T08:53:15.614058+0000","last_active":"2024-09-13T08:53:17.127225+0000","last_peered":"2024-09-13T08:53:17.127225+0000","last_clean":"2024-09-13T08:53:17.127225+0000","last_became_active":"2024-09-13T08:53:15.610159+0000","last_became_peered":"2024-09-13T08:53:15.610159+0000","last_unstale":"2024-09-13T08:53:17.127225+0000","last_undegraded":"2024-09-13T08:53:17.127225+0000","last_fullsized":"2024-09-13T08:53:17.127225+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:47:56.080386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614445+0000","last_change":"2024-09-13T08:53:15.616439+0000","last_active":"2024-09-13T08:53:16.614445+0000","last_peered":"2024-09-13T08:53:16.614445+0000","last_clean":"2024-09-13T08:53:16.614445+0000","last_became_active":"2024-09-13T08:53:15.616105+0000","last_became_peered":"2024-09-13T08:53:15.616105+0000","last_unstale":"2024-09-13T08:53:16.614445+0000","last_undegraded":"2024-09-13T08:53:16.614445+0000","last_fullsized":"2024-09-13T08:53:16.614445+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:45.954180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614686+0000","last_change":"2024-09-13T08:53:15.614259+0000","last_active":"2024-09-13T08:53:16.614686+0000","last_peered":"2024-09-13T08:53:16.614686+0000","last_clean":"2024-09-13T08:53:16.614686+0000","last_became_active":"2024-09-13T08:53:15.613790+0000","last_became_peered":"2024-09-13T08:53:15.613790+0000","last_unstale":"2024-09-13T08:53:16.614686+0000","last_undegraded":"2024-09-13T08:53:16.614686+0000","last_fullsized":"2024-09-13T08:53:16.614686+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:17:35.874448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626739+0000","last_change":"2024-09-13T08:53:15.604857+0000","last_active":"2024-09-13T08:53:16.626739+0000","last_peered":"2024-09-13T08:53:16.626739+0000","last_clean":"2024-09-13T08:53:16.626739+0000","last_became_active":"2024-09-13T08:53:15.604542+0000","last_became_peered":"2024-09-13T08:53:15.604542+0000","last_unstale":"2024-09-13T08:53:16.626739+0000","last_undegraded":"2024-09-13T08:53:16.626739+0000","last_fullsized":"2024-09-13T08:53:16.626739+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:26.046435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.613922+0000","last_change":"2024-09-13T08:53:15.614857+0000","last_active":"2024-09-13T08:53:16.613922+0000","last_peered":"2024-09-13T08:53:16.613922+0000","last_clean":"2024-09-13T08:53:16.613922+0000","last_became_active":"2024-09-13T08:53:15.610401+0000","last_became_peered":"2024-09-13T08:53:15.610401+0000","last_unstale":"2024-09-13T08:53:16.613922+0000","last_undegraded":"2024-09-13T08:53:16.613922+0000","last_fullsized":"2024-09-13T08:53:16.613922+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:52.689676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624144+0000","last_change":"2024-09-13T08:53:15.613034+0000","last_active":"2024-09-13T08:53:16.624144+0000","last_peered":"2024-09-13T08:53:16.624144+0000","last_clean":"2024-09-13T08:53:16.624144+0000","last_became_active":"2024-09-13T08:53:15.612492+0000","last_became_peered":"2024-09-13T08:53:15.612492+0000","last_unstale":"2024-09-13T08:53:16.624144+0000","last_undegraded":"2024-09-13T08:53:16.624144+0000","last_fullsized":"2024-09-13T08:53:16.624144+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:43.250681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.126801+0000","last_change":"2024-09-13T08:53:15.615654+0000","last_active":"2024-09-13T08:53:17.126801+0000","last_peered":"2024-09-13T08:53:17.126801+0000","last_clean":"2024-09-13T08:53:17.126801+0000","last_became_active":"2024-09-13T08:53:15.615357+0000","last_became_peered":"2024-09-13T08:53:15.615357+0000","last_unstale":"2024-09-13T08:53:17.126801+0000","last_undegraded":"2024-09-13T08:53:17.126801+0000","last_fullsized":"2024-09-13T08:53:17.126801+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:55.420448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623679+0000","last_change":"2024-09-13T08:53:15.603459+0000","last_active":"2024-09-13T08:53:16.623679+0000","last_peered":"2024-09-13T08:53:16.623679+0000","last_clean":"2024-09-13T08:53:16.623679+0000","last_became_active":"2024-09-13T08:53:15.603186+0000","last_became_peered":"2024-09-13T08:53:15.603186+0000","last_unstale":"2024-09-13T08:53:16.623679+0000","last_undegraded":"2024-09-13T08:53:16.623679+0000","last_fullsized":"2024-09-13T08:53:16.623679+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:03.529118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614520+0000","last_change":"2024-09-13T08:53:15.613234+0000","last_active":"2024-09-13T08:53:16.614520+0000","last_peered":"2024-09-13T08:53:16.614520+0000","last_clean":"2024-09-13T08:53:16.614520+0000","last_became_active":"2024-09-13T08:53:15.612853+0000","last_became_peered":"2024-09-13T08:53:15.612853+0000","last_unstale":"2024-09-13T08:53:16.614520+0000","last_undegraded":"2024-09-13T08:53:16.614520+0000","last_fullsized":"2024-09-13T08:53:16.614520+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:14.522632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623394+0000","last_change":"2024-09-13T08:53:15.604351+0000","last_active":"2024-09-13T08:53:16.623394+0000","last_peered":"2024-09-13T08:53:16.623394+0000","last_clean":"2024-09-13T08:53:16.623394+0000","last_became_active":"2024-09-13T08:53:15.603945+0000","last_became_peered":"2024-09-13T08:53:15.603945+0000","last_unstale":"2024-09-13T08:53:16.623394+0000","last_undegraded":"2024-09-13T08:53:16.623394+0000","last_fullsized":"2024-09-13T08:53:16.623394+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:27.087948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623492+0000","last_change":"2024-09-13T08:53:15.616379+0000","last_active":"2024-09-13T08:53:16.623492+0000","last_peered":"2024-09-13T08:53:16.623492+0000","last_clean":"2024-09-13T08:53:16.623492+0000","last_became_active":"2024-09-13T08:53:15.614571+0000","last_became_peered":"2024-09-13T08:53:15.614571+0000","last_unstale":"2024-09-13T08:53:16.623492+0000","last_undegraded":"2024-09-13T08:53:16.623492+0000","last_fullsized":"2024-09-13T08:53:16.623492+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:06.040200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614915+0000","last_change":"2024-09-13T08:53:15.625045+0000","last_active":"2024-09-13T08:53:16.614915+0000","last_peered":"2024-09-13T08:53:16.614915+0000","last_clean":"2024-09-13T08:53:16.614915+0000","last_became_active":"2024-09-13T08:53:15.624239+0000","last_became_peered":"2024-09-13T08:53:15.624239+0000","last_unstale":"2024-09-13T08:53:16.614915+0000","last_undegraded":"2024-09-13T08:53:16.614915+0000","last_fullsized":"2024-09-13T08:53:16.614915+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:37.874956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614194+0000","last_change":"2024-09-13T08:53:15.613649+0000","last_active":"2024-09-13T08:53:16.614194+0000","last_peered":"2024-09-13T08:53:16.614194+0000","last_clean":"2024-09-13T08:53:16.614194+0000","last_became_active":"2024-09-13T08:53:15.610200+0000","last_became_peered":"2024-09-13T08:53:15.610200+0000","last_unstale":"2024-09-13T08:53:16.614194+0000","last_undegraded":"2024-09-13T08:53:16.614194+0000","last_fullsized":"2024-09-13T08:53:16.614194+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:30.034701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623197+0000","last_change":"2024-09-13T08:53:15.607987+0000","last_active":"2024-09-13T08:53:16.623197+0000","last_peered":"2024-09-13T08:53:16.623197+0000","last_clean":"2024-09-13T08:53:16.623197+0000","last_became_active":"2024-09-13T08:53:15.607729+0000","last_became_peered":"2024-09-13T08:53:15.607729+0000","last_unstale":"2024-09-13T08:53:16.623197+0000","last_undegraded":"2024-09-13T08:53:16.623197+0000","last_fullsized":"2024-09-13T08:53:16.623197+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:45:25.782674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623363+0000","last_change":"2024-09-13T08:53:15.611359+0000","last_active":"2024-09-13T08:53:16.623363+0000","last_peered":"2024-09-13T08:53:16.623363+0000","last_clean":"2024-09-13T08:53:16.623363+0000","last_became_active":"2024-09-13T08:53:15.610868+0000","last_became_peered":"2024-09-13T08:53:15.610868+0000","last_unstale":"2024-09-13T08:53:16.623363+0000","last_undegraded":"2024-09-13T08:53:16.623363+0000","last_fullsized":"2024-09-13T08:53:16.623363+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:43.485104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127227+0000","last_change":"2024-09-13T08:53:15.620233+0000","last_active":"2024-09-13T08:53:17.127227+0000","last_peered":"2024-09-13T08:53:17.127227+0000","last_clean":"2024-09-13T08:53:17.127227+0000","last_became_active":"2024-09-13T08:53:15.619922+0000","last_became_peered":"2024-09-13T08:53:15.619922+0000","last_unstale":"2024-09-13T08:53:17.127227+0000","last_undegraded":"2024-09-13T08:53:17.127227+0000","last_fullsized":"2024-09-13T08:53:17.127227+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:05.522103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"143'4","reported_seq":19,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.627212+0000","last_change":"2024-09-13T08:53:14.609734+0000","last_active":"2024-09-13T08:53:16.627212+0000","last_peered":"2024-09-13T08:53:16.627212+0000","last_clean":"2024-09-13T08:53:16.627212+0000","last_became_active":"2024-09-13T08:53:14.609456+0000","last_became_peered":"2024-09-13T08:53:14.609456+0000","last_unstale":"2024-09-13T08:53:16.627212+0000","last_undegraded":"2024-09-13T08:53:16.627212+0000","last_fullsized":"2024-09-13T08:53:16.627212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:26.519784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615216+0000","last_change":"2024-09-13T08:53:15.616412+0000","last_active":"2024-09-13T08:53:16.615216+0000","last_peered":"2024-09-13T08:53:16.615216+0000","last_clean":"2024-09-13T08:53:16.615216+0000","last_became_active":"2024-09-13T08:53:15.616103+0000","last_became_peered":"2024-09-13T08:53:15.616103+0000","last_unstale":"2024-09-13T08:53:16.615216+0000","last_undegraded":"2024-09-13T08:53:16.615216+0000","last_fullsized":"2024-09-13T08:53:16.615216+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:07.581265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129266+0000","last_change":"2024-09-13T08:53:14.613339+0000","last_active":"2024-09-13T08:53:17.129266+0000","last_peered":"2024-09-13T08:53:17.129266+0000","last_clean":"2024-09-13T08:53:17.129266+0000","last_became_active":"2024-09-13T08:53:14.612799+0000","last_became_peered":"2024-09-13T08:53:14.612799+0000","last_unstale":"2024-09-13T08:53:17.129266+0000","last_undegraded":"2024-09-13T08:53:17.129266+0000","last_fullsized":"2024-09-13T08:53:17.129266+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:54.292680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129342+0000","last_change":"2024-09-13T08:53:15.616934+0000","last_active":"2024-09-13T08:53:17.129342+0000","last_peered":"2024-09-13T08:53:17.129342+0000","last_clean":"2024-09-13T08:53:17.129342+0000","last_became_active":"2024-09-13T08:53:15.616660+0000","last_became_peered":"2024-09-13T08:53:15.616660+0000","last_unstale":"2024-09-13T08:53:17.129342+0000","last_undegraded":"2024-09-13T08:53:17.129342+0000","last_fullsized":"2024-09-13T08:53:17.129342+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:52:39.024260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626829+0000","last_change":"2024-09-13T08:53:14.600079+0000","last_active":"2024-09-13T08:53:16.626829+0000","last_peered":"2024-09-13T08:53:16.626829+0000","last_clean":"2024-09-13T08:53:16.626829+0000","last_became_active":"2024-09-13T08:53:14.599758+0000","last_became_peered":"2024-09-13T08:53:14.599758+0000","last_unstale":"2024-09-13T08:53:16.626829+0000","last_undegraded":"2024-09-13T08:53:16.626829+0000","last_fullsized":"2024-09-13T08:53:16.626829+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:47.082774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625163+0000","last_change":"2024-09-13T08:53:15.613853+0000","last_active":"2024-09-13T08:53:16.625163+0000","last_peered":"2024-09-13T08:53:16.625163+0000","last_clean":"2024-09-13T08:53:16.625163+0000","last_became_active":"2024-09-13T08:53:15.610422+0000","last_became_peered":"2024-09-13T08:53:15.610422+0000","last_unstale":"2024-09-13T08:53:16.625163+0000","last_undegraded":"2024-09-13T08:53:16.625163+0000","last_fullsized":"2024-09-13T08:53:16.625163+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:57:41.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626714+0000","last_change":"2024-09-13T08:53:14.604187+0000","last_active":"2024-09-13T08:53:16.626714+0000","last_peered":"2024-09-13T08:53:16.626714+0000","last_clean":"2024-09-13T08:53:16.626714+0000","last_became_active":"2024-09-13T08:53:14.603875+0000","last_became_peered":"2024-09-13T08:53:14.603875+0000","last_unstale":"2024-09-13T08:53:16.626714+0000","last_undegraded":"2024-09-13T08:53:16.626714+0000","last_fullsized":"2024-09-13T08:53:16.626714+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:28.876441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623347+0000","last_change":"2024-09-13T08:53:15.604116+0000","last_active":"2024-09-13T08:53:16.623347+0000","last_peered":"2024-09-13T08:53:16.623347+0000","last_clean":"2024-09-13T08:53:16.623347+0000","last_became_active":"2024-09-13T08:53:15.603807+0000","last_became_peered":"2024-09-13T08:53:15.603807+0000","last_unstale":"2024-09-13T08:53:16.623347+0000","last_undegraded":"2024-09-13T08:53:16.623347+0000","last_fullsized":"2024-09-13T08:53:16.623347+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:31:32.926107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614982+0000","last_change":"2024-09-13T08:53:14.609131+0000","last_active":"2024-09-13T08:53:16.614982+0000","last_peered":"2024-09-13T08:53:16.614982+0000","last_clean":"2024-09-13T08:53:16.614982+0000","last_became_active":"2024-09-13T08:53:14.608869+0000","last_became_peered":"2024-09-13T08:53:14.608869+0000","last_unstale":"2024-09-13T08:53:16.614982+0000","last_undegraded":"2024-09-13T08:53:16.614982+0000","last_fullsized":"2024-09-13T08:53:16.614982+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:12.085049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624622+0000","last_change":"2024-09-13T08:53:15.629197+0000","last_active":"2024-09-13T08:53:16.624622+0000","last_peered":"2024-09-13T08:53:16.624622+0000","last_clean":"2024-09-13T08:53:16.624622+0000","last_became_active":"2024-09-13T08:53:15.626968+0000","last_became_peered":"2024-09-13T08:53:15.626968+0000","last_unstale":"2024-09-13T08:53:16.624622+0000","last_undegraded":"2024-09-13T08:53:16.624622+0000","last_fullsized":"2024-09-13T08:53:16.624622+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:03.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.126661+0000","last_change":"2024-09-13T08:53:14.613851+0000","last_active":"2024-09-13T08:53:17.126661+0000","last_peered":"2024-09-13T08:53:17.126661+0000","last_clean":"2024-09-13T08:53:17.126661+0000","last_became_active":"2024-09-13T08:53:14.613340+0000","last_became_peered":"2024-09-13T08:53:14.613340+0000","last_unstale":"2024-09-13T08:53:17.126661+0000","last_undegraded":"2024-09-13T08:53:17.126661+0000","last_fullsized":"2024-09-13T08:53:17.126661+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:40.082146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623144+0000","last_change":"2024-09-13T08:53:15.611980+0000","last_active":"2024-09-13T08:53:16.623144+0000","last_peered":"2024-09-13T08:53:16.623144+0000","last_clean":"2024-09-13T08:53:16.623144+0000","last_became_active":"2024-09-13T08:53:15.611737+0000","last_became_peered":"2024-09-13T08:53:15.611737+0000","last_unstale":"2024-09-13T08:53:16.623144+0000","last_undegraded":"2024-09-13T08:53:16.623144+0000","last_fullsized":"2024-09-13T08:53:16.623144+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:37.579887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129340+0000","last_change":"2024-09-13T08:53:14.609106+0000","last_active":"2024-09-13T08:53:17.129340+0000","last_peered":"2024-09-13T08:53:17.129340+0000","last_clean":"2024-09-13T08:53:17.129340+0000","last_became_active":"2024-09-13T08:53:14.608810+0000","last_became_peered":"2024-09-13T08:53:14.608810+0000","last_unstale":"2024-09-13T08:53:17.129340+0000","last_undegraded":"2024-09-13T08:53:17.129340+0000","last_fullsized":"2024-09-13T08:53:17.129340+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:44.027745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624213+0000","last_change":"2024-09-13T08:53:15.620180+0000","last_active":"2024-09-13T08:53:16.624213+0000","last_peered":"2024-09-13T08:53:16.624213+0000","last_clean":"2024-09-13T08:53:16.624213+0000","last_became_active":"2024-09-13T08:53:15.617653+0000","last_became_peered":"2024-09-13T08:53:15.617653+0000","last_unstale":"2024-09-13T08:53:16.624213+0000","last_undegraded":"2024-09-13T08:53:16.624213+0000","last_fullsized":"2024-09-13T08:53:16.624213+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:46.287335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626385+0000","last_change":"2024-09-13T08:53:14.612322+0000","last_active":"2024-09-13T08:53:16.626385+0000","last_peered":"2024-09-13T08:53:16.626385+0000","last_clean":"2024-09-13T08:53:16.626385+0000","last_became_active":"2024-09-13T08:53:14.612007+0000","last_became_peered":"2024-09-13T08:53:14.612007+0000","last_unstale":"2024-09-13T08:53:16.626385+0000","last_undegraded":"2024-09-13T08:53:16.626385+0000","last_fullsized":"2024-09-13T08:53:16.626385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:08.627533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625197+0000","last_change":"2024-09-13T08:53:15.618278+0000","last_active":"2024-09-13T08:53:16.625197+0000","last_peered":"2024-09-13T08:53:16.625197+0000","last_clean":"2024-09-13T08:53:16.625197+0000","last_became_active":"2024-09-13T08:53:15.617694+0000","last_became_peered":"2024-09-13T08:53:15.617694+0000","last_unstale":"2024-09-13T08:53:16.625197+0000","last_undegraded":"2024-09-13T08:53:16.625197+0000","last_fullsized":"2024-09-13T08:53:16.625197+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:23.022502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129413+0000","last_change":"2024-09-13T08:53:14.611525+0000","last_active":"2024-09-13T08:53:17.129413+0000","last_peered":"2024-09-13T08:53:17.129413+0000","last_clean":"2024-09-13T08:53:17.129413+0000","last_became_active":"2024-09-13T08:53:14.611246+0000","last_became_peered":"2024-09-13T08:53:14.611246+0000","last_unstale":"2024-09-13T08:53:17.129413+0000","last_undegraded":"2024-09-13T08:53:17.129413+0000","last_fullsized":"2024-09-13T08:53:17.129413+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:51.437020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614438+0000","last_change":"2024-09-13T08:53:15.623604+0000","last_active":"2024-09-13T08:53:16.614438+0000","last_peered":"2024-09-13T08:53:16.614438+0000","last_clean":"2024-09-13T08:53:16.614438+0000","last_became_active":"2024-09-13T08:53:15.623149+0000","last_became_peered":"2024-09-13T08:53:15.623149+0000","last_unstale":"2024-09-13T08:53:16.614438+0000","last_undegraded":"2024-09-13T08:53:16.614438+0000","last_fullsized":"2024-09-13T08:53:16.614438+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:58.827614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127307+0000","last_change":"2024-09-13T08:53:14.605714+0000","last_active":"2024-09-13T08:53:17.127307+0000","last_peered":"2024-09-13T08:53:17.127307+0000","last_clean":"2024-09-13T08:53:17.127307+0000","last_became_active":"2024-09-13T08:53:14.604522+0000","last_became_peered":"2024-09-13T08:53:14.604522+0000","last_unstale":"2024-09-13T08:53:17.127307+0000","last_undegraded":"2024-09-13T08:53:17.127307+0000","last_fullsized":"2024-09-13T08:53:17.127307+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:00:13.328557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615848+0000","last_change":"2024-09-13T08:53:15.602657+0000","last_active":"2024-09-13T08:53:16.615848+0000","last_peered":"2024-09-13T08:53:16.615848+0000","last_clean":"2024-09-13T08:53:16.615848+0000","last_became_active":"2024-09-13T08:53:15.602363+0000","last_became_peered":"2024-09-13T08:53:15.602363+0000","last_unstale":"2024-09-13T08:53:16.615848+0000","last_undegraded":"2024-09-13T08:53:16.615848+0000","last_fullsized":"2024-09-13T08:53:16.615848+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:14:11.309419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"143'6","reported_seq":21,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615631+0000","last_change":"2024-09-13T08:53:14.609450+0000","last_active":"2024-09-13T08:53:16.615631+0000","last_peered":"2024-09-13T08:53:16.615631+0000","last_clean":"2024-09-13T08:53:16.615631+0000","last_became_active":"2024-09-13T08:53:14.608508+0000","last_became_peered":"2024-09-13T08:53:14.608508+0000","last_unstale":"2024-09-13T08:53:16.615631+0000","last_undegraded":"2024-09-13T08:53:16.615631+0000","last_fullsized":"2024-09-13T08:53:16.615631+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:47.008225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624204+0000","last_change":"2024-09-13T08:53:15.615985+0000","last_active":"2024-09-13T08:53:16.624204+0000","last_peered":"2024-09-13T08:53:16.624204+0000","last_clean":"2024-09-13T08:53:16.624204+0000","last_became_active":"2024-09-13T08:53:15.615451+0000","last_became_peered":"2024-09-13T08:53:15.615451+0000","last_unstale":"2024-09-13T08:53:16.624204+0000","last_undegraded":"2024-09-13T08:53:16.624204+0000","last_fullsized":"2024-09-13T08:53:16.624204+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:20.002995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626608+0000","last_change":"2024-09-13T08:53:14.601073+0000","last_active":"2024-09-13T08:53:16.626608+0000","last_peered":"2024-09-13T08:53:16.626608+0000","last_clean":"2024-09-13T08:53:16.626608+0000","last_became_active":"2024-09-13T08:53:14.600773+0000","last_became_peered":"2024-09-13T08:53:14.600773+0000","last_unstale":"2024-09-13T08:53:16.626608+0000","last_undegraded":"2024-09-13T08:53:16.626608+0000","last_fullsized":"2024-09-13T08:53:16.626608+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:35.075345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615378+0000","last_change":"2024-09-13T08:53:15.625078+0000","last_active":"2024-09-13T08:53:16.615378+0000","last_peered":"2024-09-13T08:53:16.615378+0000","last_clean":"2024-09-13T08:53:16.615378+0000","last_became_active":"2024-09-13T08:53:15.624764+0000","last_became_peered":"2024-09-13T08:53:15.624764+0000","last_unstale":"2024-09-13T08:53:16.615378+0000","last_undegraded":"2024-09-13T08:53:16.615378+0000","last_fullsized":"2024-09-13T08:53:16.615378+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:16:53.839834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614260+0000","last_change":"2024-09-13T08:53:14.610271+0000","last_active":"2024-09-13T08:53:16.614260+0000","last_peered":"2024-09-13T08:53:16.614260+0000","last_clean":"2024-09-13T08:53:16.614260+0000","last_became_active":"2024-09-13T08:53:14.609955+0000","last_became_peered":"2024-09-13T08:53:14.609955+0000","last_unstale":"2024-09-13T08:53:16.614260+0000","last_undegraded":"2024-09-13T08:53:16.614260+0000","last_fullsized":"2024-09-13T08:53:16.614260+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:03:59.434188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624009+0000","last_change":"2024-09-13T08:53:15.611435+0000","last_active":"2024-09-13T08:53:16.624009+0000","last_peered":"2024-09-13T08:53:16.624009+0000","last_clean":"2024-09-13T08:53:16.624009+0000","last_became_active":"2024-09-13T08:53:15.611151+0000","last_became_peered":"2024-09-13T08:53:15.611151+0000","last_unstale":"2024-09-13T08:53:16.624009+0000","last_undegraded":"2024-09-13T08:53:16.624009+0000","last_fullsized":"2024-09-13T08:53:16.624009+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:01.835402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623216+0000","last_change":"2024-09-13T08:53:14.607433+0000","last_active":"2024-09-13T08:53:16.623216+0000","last_peered":"2024-09-13T08:53:16.623216+0000","last_clean":"2024-09-13T08:53:16.623216+0000","last_became_active":"2024-09-13T08:53:14.607139+0000","last_became_peered":"2024-09-13T08:53:14.607139+0000","last_unstale":"2024-09-13T08:53:16.623216+0000","last_undegraded":"2024-09-13T08:53:16.623216+0000","last_fullsized":"2024-09-13T08:53:16.623216+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:21:48.609279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624355+0000","last_change":"2024-09-13T08:53:15.612753+0000","last_active":"2024-09-13T08:53:16.624355+0000","last_peered":"2024-09-13T08:53:16.624355+0000","last_clean":"2024-09-13T08:53:16.624355+0000","last_became_active":"2024-09-13T08:53:15.612138+0000","last_became_peered":"2024-09-13T08:53:15.612138+0000","last_unstale":"2024-09-13T08:53:16.624355+0000","last_undegraded":"2024-09-13T08:53:16.624355+0000","last_fullsized":"2024-09-13T08:53:16.624355+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:48.015020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"143'8","reported_seq":23,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614230+0000","last_change":"2024-09-13T08:53:14.607820+0000","last_active":"2024-09-13T08:53:16.614230+0000","last_peered":"2024-09-13T08:53:16.614230+0000","last_clean":"2024-09-13T08:53:16.614230+0000","last_became_active":"2024-09-13T08:53:14.607522+0000","last_became_peered":"2024-09-13T08:53:14.607522+0000","last_unstale":"2024-09-13T08:53:16.614230+0000","last_undegraded":"2024-09-13T08:53:16.614230+0000","last_fullsized":"2024-09-13T08:53:16.614230+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:41.446670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127413+0000","last_change":"2024-09-13T08:53:15.617227+0000","last_active":"2024-09-13T08:53:17.127413+0000","last_peered":"2024-09-13T08:53:17.127413+0000","last_clean":"2024-09-13T08:53:17.127413+0000","last_became_active":"2024-09-13T08:53:15.616950+0000","last_became_peered":"2024-09-13T08:53:15.616950+0000","last_unstale":"2024-09-13T08:53:17.127413+0000","last_undegraded":"2024-09-13T08:53:17.127413+0000","last_fullsized":"2024-09-13T08:53:17.127413+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:17.577743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625495+0000","last_change":"2024-09-13T08:53:14.608024+0000","last_active":"2024-09-13T08:53:16.625495+0000","last_peered":"2024-09-13T08:53:16.625495+0000","last_clean":"2024-09-13T08:53:16.625495+0000","last_became_active":"2024-09-13T08:53:14.607741+0000","last_became_peered":"2024-09-13T08:53:14.607741+0000","last_unstale":"2024-09-13T08:53:16.625495+0000","last_undegraded":"2024-09-13T08:53:16.625495+0000","last_fullsized":"2024-09-13T08:53:16.625495+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:45:24.775879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623792+0000","last_change":"2024-09-13T08:53:15.607225+0000","last_active":"2024-09-13T08:53:16.623792+0000","last_peered":"2024-09-13T08:53:16.623792+0000","last_clean":"2024-09-13T08:53:16.623792+0000","last_became_active":"2024-09-13T08:53:15.606970+0000","last_became_peered":"2024-09-13T08:53:15.606970+0000","last_unstale":"2024-09-13T08:53:16.623792+0000","last_undegraded":"2024-09-13T08:53:16.623792+0000","last_fullsized":"2024-09-13T08:53:16.623792+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:01.304432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614584+0000","last_change":"2024-09-13T08:53:14.613217+0000","last_active":"2024-09-13T08:53:16.614584+0000","last_peered":"2024-09-13T08:53:16.614584+0000","last_clean":"2024-09-13T08:53:16.614584+0000","last_became_active":"2024-09-13T08:53:14.612795+0000","last_became_peered":"2024-09-13T08:53:14.612795+0000","last_unstale":"2024-09-13T08:53:16.614584+0000","last_undegraded":"2024-09-13T08:53:16.614584+0000","last_fullsized":"2024-09-13T08:53:16.614584+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:29:05.403502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.126914+0000","last_change":"2024-09-13T08:53:15.616529+0000","last_active":"2024-09-13T08:53:17.126914+0000","last_peered":"2024-09-13T08:53:17.126914+0000","last_clean":"2024-09-13T08:53:17.126914+0000","last_became_active":"2024-09-13T08:53:15.616238+0000","last_became_peered":"2024-09-13T08:53:15.616238+0000","last_unstale":"2024-09-13T08:53:17.126914+0000","last_undegraded":"2024-09-13T08:53:17.126914+0000","last_fullsized":"2024-09-13T08:53:17.126914+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:38.099277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615989+0000","last_change":"2024-09-13T08:53:14.607098+0000","last_active":"2024-09-13T08:53:16.615989+0000","last_peered":"2024-09-13T08:53:16.615989+0000","last_clean":"2024-09-13T08:53:16.615989+0000","last_became_active":"2024-09-13T08:53:14.606857+0000","last_became_peered":"2024-09-13T08:53:14.606857+0000","last_unstale":"2024-09-13T08:53:16.615989+0000","last_undegraded":"2024-09-13T08:53:16.615989+0000","last_fullsized":"2024-09-13T08:53:16.615989+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:40.548251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129489+0000","last_change":"2024-09-13T08:53:15.615099+0000","last_active":"2024-09-13T08:53:17.129489+0000","last_peered":"2024-09-13T08:53:17.129489+0000","last_clean":"2024-09-13T08:53:17.129489+0000","last_became_active":"2024-09-13T08:53:15.614834+0000","last_became_peered":"2024-09-13T08:53:15.614834+0000","last_unstale":"2024-09-13T08:53:17.129489+0000","last_undegraded":"2024-09-13T08:53:17.129489+0000","last_fullsized":"2024-09-13T08:53:17.129489+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:34.923374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623319+0000","last_change":"2024-09-13T08:53:14.600940+0000","last_active":"2024-09-13T08:53:16.623319+0000","last_peered":"2024-09-13T08:53:16.623319+0000","last_clean":"2024-09-13T08:53:16.623319+0000","last_became_active":"2024-09-13T08:53:14.600642+0000","last_became_peered":"2024-09-13T08:53:14.600642+0000","last_unstale":"2024-09-13T08:53:16.623319+0000","last_undegraded":"2024-09-13T08:53:16.623319+0000","last_fullsized":"2024-09-13T08:53:16.623319+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:07.668506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614193+0000","last_change":"2024-09-13T08:53:15.615878+0000","last_active":"2024-09-13T08:53:16.614193+0000","last_peered":"2024-09-13T08:53:16.614193+0000","last_clean":"2024-09-13T08:53:16.614193+0000","last_became_active":"2024-09-13T08:53:15.615238+0000","last_became_peered":"2024-09-13T08:53:15.615238+0000","last_unstale":"2024-09-13T08:53:16.614193+0000","last_undegraded":"2024-09-13T08:53:16.614193+0000","last_fullsized":"2024-09-13T08:53:16.614193+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:23.022502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129245+0000","last_change":"2024-09-13T08:53:14.613189+0000","last_active":"2024-09-13T08:53:17.129245+0000","last_peered":"2024-09-13T08:53:17.129245+0000","last_clean":"2024-09-13T08:53:17.129245+0000","last_became_active":"2024-09-13T08:53:14.612877+0000","last_became_peered":"2024-09-13T08:53:14.612877+0000","last_unstale":"2024-09-13T08:53:17.129245+0000","last_undegraded":"2024-09-13T08:53:17.129245+0000","last_fullsized":"2024-09-13T08:53:17.129245+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:12.557688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623444+0000","last_change":"2024-09-13T08:53:15.605208+0000","last_active":"2024-09-13T08:53:16.623444+0000","last_peered":"2024-09-13T08:53:16.623444+0000","last_clean":"2024-09-13T08:53:16.623444+0000","last_became_active":"2024-09-13T08:53:15.604912+0000","last_became_peered":"2024-09-13T08:53:15.604912+0000","last_unstale":"2024-09-13T08:53:16.623444+0000","last_undegraded":"2024-09-13T08:53:16.623444+0000","last_fullsized":"2024-09-13T08:53:16.623444+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:41.385047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624172+0000","last_change":"2024-09-13T08:53:14.601389+0000","last_active":"2024-09-13T08:53:16.624172+0000","last_peered":"2024-09-13T08:53:16.624172+0000","last_clean":"2024-09-13T08:53:16.624172+0000","last_became_active":"2024-09-13T08:53:14.601109+0000","last_became_peered":"2024-09-13T08:53:14.601109+0000","last_unstale":"2024-09-13T08:53:16.624172+0000","last_undegraded":"2024-09-13T08:53:16.624172+0000","last_fullsized":"2024-09-13T08:53:16.624172+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:42.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.613772+0000","last_change":"2024-09-13T08:53:15.612278+0000","last_active":"2024-09-13T08:53:16.613772+0000","last_peered":"2024-09-13T08:53:16.613772+0000","last_clean":"2024-09-13T08:53:16.613772+0000","last_became_active":"2024-09-13T08:53:15.609387+0000","last_became_peered":"2024-09-13T08:53:15.609387+0000","last_unstale":"2024-09-13T08:53:16.613772+0000","last_undegraded":"2024-09-13T08:53:16.613772+0000","last_fullsized":"2024-09-13T08:53:16.613772+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:03.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624283+0000","last_change":"2024-09-13T08:53:14.606466+0000","last_active":"2024-09-13T08:53:16.624283+0000","last_peered":"2024-09-13T08:53:16.624283+0000","last_clean":"2024-09-13T08:53:16.624283+0000","last_became_active":"2024-09-13T08:53:14.606205+0000","last_became_peered":"2024-09-13T08:53:14.606205+0000","last_unstale":"2024-09-13T08:53:16.624283+0000","last_undegraded":"2024-09-13T08:53:16.624283+0000","last_fullsized":"2024-09-13T08:53:16.624283+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:55:32.093615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623141+0000","last_change":"2024-09-13T08:53:15.609120+0000","last_active":"2024-09-13T08:53:16.623141+0000","last_peered":"2024-09-13T08:53:16.623141+0000","last_clean":"2024-09-13T08:53:16.623141+0000","last_became_active":"2024-09-13T08:53:15.607949+0000","last_became_peered":"2024-09-13T08:53:15.607949+0000","last_unstale":"2024-09-13T08:53:16.623141+0000","last_undegraded":"2024-09-13T08:53:16.623141+0000","last_fullsized":"2024-09-13T08:53:16.623141+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:45.407786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624140+0000","last_change":"2024-09-13T08:53:14.601703+0000","last_active":"2024-09-13T08:53:16.624140+0000","last_peered":"2024-09-13T08:53:16.624140+0000","last_clean":"2024-09-13T08:53:16.624140+0000","last_became_active":"2024-09-13T08:53:14.601345+0000","last_became_peered":"2024-09-13T08:53:14.601345+0000","last_unstale":"2024-09-13T08:53:16.624140+0000","last_undegraded":"2024-09-13T08:53:16.624140+0000","last_fullsized":"2024-09-13T08:53:16.624140+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:24:09.365454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625254+0000","last_change":"2024-09-13T08:53:15.611309+0000","last_active":"2024-09-13T08:53:16.625254+0000","last_peered":"2024-09-13T08:53:16.625254+0000","last_clean":"2024-09-13T08:53:16.625254+0000","last_became_active":"2024-09-13T08:53:15.611003+0000","last_became_peered":"2024-09-13T08:53:15.611003+0000","last_unstale":"2024-09-13T08:53:16.625254+0000","last_undegraded":"2024-09-13T08:53:16.625254+0000","last_fullsized":"2024-09-13T08:53:16.625254+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:54:39.468468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625587+0000","last_change":"2024-09-13T08:53:14.608168+0000","last_active":"2024-09-13T08:53:16.625587+0000","last_peered":"2024-09-13T08:53:16.625587+0000","last_clean":"2024-09-13T08:53:16.625587+0000","last_became_active":"2024-09-13T08:53:14.607901+0000","last_became_peered":"2024-09-13T08:53:14.607901+0000","last_unstale":"2024-09-13T08:53:16.625587+0000","last_undegraded":"2024-09-13T08:53:16.625587+0000","last_fullsized":"2024-09-13T08:53:16.625587+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:40.977746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625528+0000","last_change":"2024-09-13T08:53:15.627757+0000","last_active":"2024-09-13T08:53:16.625528+0000","last_peered":"2024-09-13T08:53:16.625528+0000","last_clean":"2024-09-13T08:53:16.625528+0000","last_became_active":"2024-09-13T08:53:15.627028+0000","last_became_peered":"2024-09-13T08:53:15.627028+0000","last_unstale":"2024-09-13T08:53:16.625528+0000","last_undegraded":"2024-09-13T08:53:16.625528+0000","last_fullsized":"2024-09-13T08:53:16.625528+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:58.827614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"144'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623510+0000","last_change":"2024-09-13T08:53:14.608469+0000","last_active":"2024-09-13T08:53:16.623510+0000","last_peered":"2024-09-13T08:53:16.623510+0000","last_clean":"2024-09-13T08:53:16.623510+0000","last_became_active":"2024-09-13T08:53:14.608207+0000","last_became_peered":"2024-09-13T08:53:14.608207+0000","last_unstale":"2024-09-13T08:53:16.623510+0000","last_undegraded":"2024-09-13T08:53:16.623510+0000","last_fullsized":"2024-09-13T08:53:16.623510+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:42.989669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614947+0000","last_change":"2024-09-13T08:53:15.614049+0000","last_active":"2024-09-13T08:53:16.614947+0000","last_peered":"2024-09-13T08:53:16.614947+0000","last_clean":"2024-09-13T08:53:16.614947+0000","last_became_active":"2024-09-13T08:53:15.613444+0000","last_became_peered":"2024-09-13T08:53:15.613444+0000","last_unstale":"2024-09-13T08:53:16.614947+0000","last_undegraded":"2024-09-13T08:53:16.614947+0000","last_fullsized":"2024-09-13T08:53:16.614947+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:06.092572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615986+0000","last_change":"2024-09-13T08:53:14.609225+0000","last_active":"2024-09-13T08:53:16.615986+0000","last_peered":"2024-09-13T08:53:16.615986+0000","last_clean":"2024-09-13T08:53:16.615986+0000","last_became_active":"2024-09-13T08:53:14.608429+0000","last_became_peered":"2024-09-13T08:53:14.608429+0000","last_unstale":"2024-09-13T08:53:16.615986+0000","last_undegraded":"2024-09-13T08:53:16.615986+0000","last_fullsized":"2024-09-13T08:53:16.615986+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:02.734300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614284+0000","last_change":"2024-09-13T08:53:15.619414+0000","last_active":"2024-09-13T08:53:16.614284+0000","last_peered":"2024-09-13T08:53:16.614284+0000","last_clean":"2024-09-13T08:53:16.614284+0000","last_became_active":"2024-09-13T08:53:15.619134+0000","last_became_peered":"2024-09-13T08:53:15.619134+0000","last_unstale":"2024-09-13T08:53:16.614284+0000","last_undegraded":"2024-09-13T08:53:16.614284+0000","last_fullsized":"2024-09-13T08:53:16.614284+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:42:20.746655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615760+0000","last_change":"2024-09-13T08:53:14.605915+0000","last_active":"2024-09-13T08:53:16.615760+0000","last_peered":"2024-09-13T08:53:16.615760+0000","last_clean":"2024-09-13T08:53:16.615760+0000","last_became_active":"2024-09-13T08:53:14.605660+0000","last_became_peered":"2024-09-13T08:53:14.605660+0000","last_unstale":"2024-09-13T08:53:16.615760+0000","last_undegraded":"2024-09-13T08:53:16.615760+0000","last_fullsized":"2024-09-13T08:53:16.615760+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:48.497980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615174+0000","last_change":"2024-09-13T08:53:15.619211+0000","last_active":"2024-09-13T08:53:16.615174+0000","last_peered":"2024-09-13T08:53:16.615174+0000","last_clean":"2024-09-13T08:53:16.615174+0000","last_became_active":"2024-09-13T08:53:15.617728+0000","last_became_peered":"2024-09-13T08:53:15.617728+0000","last_unstale":"2024-09-13T08:53:16.615174+0000","last_undegraded":"2024-09-13T08:53:16.615174+0000","last_fullsized":"2024-09-13T08:53:16.615174+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:19.945089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623003+0000","last_change":"2024-09-13T08:53:14.606764+0000","last_active":"2024-09-13T08:53:16.623003+0000","last_peered":"2024-09-13T08:53:16.623003+0000","last_clean":"2024-09-13T08:53:16.623003+0000","last_became_active":"2024-09-13T08:53:14.606338+0000","last_became_peered":"2024-09-13T08:53:14.606338+0000","last_unstale":"2024-09-13T08:53:16.623003+0000","last_undegraded":"2024-09-13T08:53:16.623003+0000","last_fullsized":"2024-09-13T08:53:16.623003+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:10.459562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624817+0000","last_change":"2024-09-13T08:53:15.611512+0000","last_active":"2024-09-13T08:53:16.624817+0000","last_peered":"2024-09-13T08:53:16.624817+0000","last_clean":"2024-09-13T08:53:16.624817+0000","last_became_active":"2024-09-13T08:53:15.611194+0000","last_became_peered":"2024-09-13T08:53:15.611194+0000","last_unstale":"2024-09-13T08:53:16.624817+0000","last_undegraded":"2024-09-13T08:53:16.624817+0000","last_fullsized":"2024-09-13T08:53:16.624817+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:49.504775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626562+0000","last_change":"2024-09-13T08:53:14.608787+0000","last_active":"2024-09-13T08:53:16.626562+0000","last_peered":"2024-09-13T08:53:16.626562+0000","last_clean":"2024-09-13T08:53:16.626562+0000","last_became_active":"2024-09-13T08:53:14.608491+0000","last_became_peered":"2024-09-13T08:53:14.608491+0000","last_unstale":"2024-09-13T08:53:16.626562+0000","last_undegraded":"2024-09-13T08:53:16.626562+0000","last_fullsized":"2024-09-13T08:53:16.626562+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:48.181981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624308+0000","last_change":"2024-09-13T08:53:15.618674+0000","last_active":"2024-09-13T08:53:16.624308+0000","last_peered":"2024-09-13T08:53:16.624308+0000","last_clean":"2024-09-13T08:53:16.624308+0000","last_became_active":"2024-09-13T08:53:15.618184+0000","last_became_peered":"2024-09-13T08:53:15.618184+0000","last_unstale":"2024-09-13T08:53:16.624308+0000","last_undegraded":"2024-09-13T08:53:16.624308+0000","last_fullsized":"2024-09-13T08:53:16.624308+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:14.950615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.613598+0000","last_change":"2024-09-13T08:53:14.600955+0000","last_active":"2024-09-13T08:53:16.613598+0000","last_peered":"2024-09-13T08:53:16.613598+0000","last_clean":"2024-09-13T08:53:16.613598+0000","last_became_active":"2024-09-13T08:53:14.600659+0000","last_became_peered":"2024-09-13T08:53:14.600659+0000","last_unstale":"2024-09-13T08:53:16.613598+0000","last_undegraded":"2024-09-13T08:53:16.613598+0000","last_fullsized":"2024-09-13T08:53:16.613598+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:17.207279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626577+0000","last_change":"2024-09-13T08:53:15.608954+0000","last_active":"2024-09-13T08:53:16.626577+0000","last_peered":"2024-09-13T08:53:16.626577+0000","last_clean":"2024-09-13T08:53:16.626577+0000","last_became_active":"2024-09-13T08:53:15.608686+0000","last_became_peered":"2024-09-13T08:53:15.608686+0000","last_unstale":"2024-09-13T08:53:16.626577+0000","last_undegraded":"2024-09-13T08:53:16.626577+0000","last_fullsized":"2024-09-13T08:53:16.626577+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:55.415999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129123+0000","last_change":"2024-09-13T08:53:14.610914+0000","last_active":"2024-09-13T08:53:17.129123+0000","last_peered":"2024-09-13T08:53:17.129123+0000","last_clean":"2024-09-13T08:53:17.129123+0000","last_became_active":"2024-09-13T08:53:14.610635+0000","last_became_peered":"2024-09-13T08:53:14.610635+0000","last_unstale":"2024-09-13T08:53:17.129123+0000","last_undegraded":"2024-09-13T08:53:17.129123+0000","last_fullsized":"2024-09-13T08:53:17.129123+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:04.024112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615796+0000","last_change":"2024-09-13T08:53:15.608925+0000","last_active":"2024-09-13T08:53:16.615796+0000","last_peered":"2024-09-13T08:53:16.615796+0000","last_clean":"2024-09-13T08:53:16.615796+0000","last_became_active":"2024-09-13T08:53:15.608642+0000","last_became_peered":"2024-09-13T08:53:15.608642+0000","last_unstale":"2024-09-13T08:53:16.615796+0000","last_undegraded":"2024-09-13T08:53:16.615796+0000","last_fullsized":"2024-09-13T08:53:16.615796+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:41:02.010094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"143'3","reported_seq":18,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614124+0000","last_change":"2024-09-13T08:53:14.605379+0000","last_active":"2024-09-13T08:53:16.614124+0000","last_peered":"2024-09-13T08:53:16.614124+0000","last_clean":"2024-09-13T08:53:16.614124+0000","last_became_active":"2024-09-13T08:53:14.605066+0000","last_became_peered":"2024-09-13T08:53:14.605066+0000","last_unstale":"2024-09-13T08:53:16.614124+0000","last_undegraded":"2024-09-13T08:53:16.614124+0000","last_fullsized":"2024-09-13T08:53:16.614124+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:47:18.799221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":346,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.126624+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:53:17.126624+0000","last_peered":"2024-09-13T08:53:17.126624+0000","last_clean":"2024-09-13T08:53:17.126624+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:53:17.126624+0000","last_undegraded":"2024-09-13T08:53:17.126624+0000","last_fullsized":"2024-09-13T08:53:17.126624+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509633,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127280,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244560,"statfs":{"total":96636764160,"available":96506429440,"internally_reserved":0,"allocated":1794048,"data_stored":794783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509633,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119568,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252272,"statfs":{"total":96636764160,"available":96514326528,"internally_reserved":0,"allocated":2412544,"data_stored":1386957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509634,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137028,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234812,"statfs":{"total":96636764160,"available":96496447488,"internally_reserved":0,"allocated":1814528,"data_stored":796999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509634,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131444,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240396,"statfs":{"total":96636764160,"available":96502165504,"internally_reserved":0,"allocated":1802240,"data_stored":795399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509633,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131444,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240396,"statfs":{"total":96636764160,"available":96502165504,"internally_reserved":0,"allocated":1798144,"data_stored":794805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509635,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145912,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225928,"statfs":{"total":96636764160,"available":96487350272,"internally_reserved":0,"allocated":2392064,"data_stored":1385221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509633,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149492,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222348,"statfs":{"total":96636764160,"available":96483684352,"internally_reserved":0,"allocated":1798144,"data_stored":794873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542339,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141168,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230672,"statfs":{"total":96636764160,"available":96492208128,"internally_reserved":0,"allocated":1794048,"data_stored":794783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:53:24.395 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:53:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.555+0000 7f6934a00640 1 -- 172.21.15.44:0/3945566828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6914004810 msgr2=0x7f6914004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.555+0000 7f6934a00640 1 --2- 172.21.15.44:0/3945566828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6914004810 0x7f6914004c90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f6918009b60 tx=0x7f691801d270 comp rx=0 tx=0).stop 2024-09-13T08:53:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.555+0000 7f692e800640 1 -- 172.21.15.44:0/3945566828 reap_dead start 2024-09-13T08:53:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.555+0000 7f6934a00640 1 -- 172.21.15.44:0/3945566828 shutdown_connections 2024-09-13T08:53:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.555+0000 7f6934a00640 1 -- 172.21.15.44:0/3945566828 >> 172.21.15.44:0/3945566828 conn(0x7f69300d1e30 msgr2=0x7f69300d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.555+0000 7f6934a00640 1 -- 172.21.15.44:0/3945566828 shutdown_connections 2024-09-13T08:53:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.555+0000 7f6934a00640 1 -- 172.21.15.44:0/3945566828 wait complete. 2024-09-13T08:53:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.555+0000 7f6934a00640 1 Processor -- start 2024-09-13T08:53:24.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.555+0000 7f6934a00640 1 -- start start 2024-09-13T08:53:24.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6934a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6914004810 0x7f69301f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:24.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6934a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69300c53e0 0x7f69301f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:24.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6934a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69301f6470 0x7f693014bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:24.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6934a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69300b1e10 con 0x7f6914004810 2024-09-13T08:53:24.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6934a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f69300b1c90 con 0x7f69300c53e0 2024-09-13T08:53:24.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6934a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f69300b1f90 con 0x7f69301f6470 2024-09-13T08:53:24.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f692de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6914004810 0x7f69301f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:24.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f692de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6914004810 0x7f69301f59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42114/0 (socket says 172.21.15.44:42114) 2024-09-13T08:53:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f692de00640 1 -- 172.21.15.44:0/2198460600 learned_addr learned my addr 172.21.15.44:0/2198460600 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6927e00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69300c53e0 0x7f69301f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f692e800640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69301f6470 0x7f693014bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f692de00640 1 -- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69301f6470 msgr2=0x7f693014bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f692de00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69301f6470 0x7f693014bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f692de00640 1 -- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69300c53e0 msgr2=0x7f69301f5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f692de00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69300c53e0 0x7f69301f5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f692de00640 1 -- 172.21.15.44:0/2198460600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69300b6670 con 0x7f6914004810 2024-09-13T08:53:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6927e00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69300c53e0 0x7f69301f5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:24.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f692de00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6914004810 0x7f69301f59f0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f6920011a80 tx=0x7f6920011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:24.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6926000640 1 -- 172.21.15.44:0/2198460600 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692001f070 con 0x7f6914004810 2024-09-13T08:53:24.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6926000640 1 -- 172.21.15.44:0/2198460600 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6920012450 con 0x7f6914004810 2024-09-13T08:53:24.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6926000640 1 -- 172.21.15.44:0/2198460600 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920012880 con 0x7f6914004810 2024-09-13T08:53:24.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6934a00640 1 -- 172.21.15.44:0/2198460600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69300b6910 con 0x7f6914004810 2024-09-13T08:53:24.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6934a00640 1 -- 172.21.15.44:0/2198460600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69300b6ed0 con 0x7f6914004810 2024-09-13T08:53:24.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6934a00640 1 -- 172.21.15.44:0/2198460600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69300b1370 con 0x7f6914004810 2024-09-13T08:53:24.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.559+0000 7f6926000640 1 -- 172.21.15.44:0/2198460600 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f6920022070 con 0x7f6914004810 2024-09-13T08:53:24.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.563+0000 7f6926000640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f690c075110 0x7f690c0775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:24.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.563+0000 7f6926000640 1 -- 172.21.15.44:0/2198460600 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f6920093c80 con 0x7f6914004810 2024-09-13T08:53:24.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.563+0000 7f6927e00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f690c075110 0x7f690c0775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:24.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.563+0000 7f6927e00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f690c075110 0x7f690c0775d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f69180ae000 tx=0x7f691801a010 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:24.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.567+0000 7f6926000640 1 -- 172.21.15.44:0/2198460600 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f69200602e0 con 0x7f6914004810 2024-09-13T08:53:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.739+0000 7f6934a00640 1 -- 172.21.15.44:0/2198460600 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f693015c050 con 0x7f690c075110 2024-09-13T08:53:24.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.743+0000 7f6926000640 1 -- 172.21.15.44:0/2198460600 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252497 (secure 0 0 0) 0x7f693015c050 con 0x7f690c075110 2024-09-13T08:53:24.747 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:24.748 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 -- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f690c075110 msgr2=0x7f690c0775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f690c075110 0x7f690c0775d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f69180ae000 tx=0x7f691801a010 comp rx=0 tx=0).stop 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 -- 172.21.15.44:0/2198460600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6914004810 msgr2=0x7f69301f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6914004810 0x7f69301f59f0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f6920011a80 tx=0x7f6920011f40 comp rx=0 tx=0).stop 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 -- 172.21.15.44:0/2198460600 shutdown_connections 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f690c075110 0x7f690c0775d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f69301f6470 0x7f693014bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69300c53e0 0x7f69301f5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 --2- 172.21.15.44:0/2198460600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6914004810 0x7f69301f59f0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 -- 172.21.15.44:0/2198460600 >> 172.21.15.44:0/2198460600 conn(0x7f69300d1e30 msgr2=0x7f693015cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 -- 172.21.15.44:0/2198460600 shutdown_connections 2024-09-13T08:53:24.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.747+0000 7f6934a00640 1 -- 172.21.15.44:0/2198460600 wait complete. 2024-09-13T08:53:24.768 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":617,"stamp":"2024-09-13T08:53:22.837644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1083336,"kb_used_data":15240,"kb_used_omap":12,"kb_used_meta":1067891,"kb_avail":753891384,"statfs":{"total":773094113280,"available":771984777216,"internally_reserved":0,"allocated":15605760,"data_stored":7543820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1093520945},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002124"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614737+0000","last_change":"2024-09-13T08:53:15.623539+0000","last_active":"2024-09-13T08:53:16.614737+0000","last_peered":"2024-09-13T08:53:16.614737+0000","last_clean":"2024-09-13T08:53:16.614737+0000","last_became_active":"2024-09-13T08:53:15.623156+0000","last_became_peered":"2024-09-13T08:53:15.623156+0000","last_unstale":"2024-09-13T08:53:16.614737+0000","last_undegraded":"2024-09-13T08:53:16.614737+0000","last_fullsized":"2024-09-13T08:53:16.614737+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:01.640017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623510+0000","last_change":"2024-09-13T08:53:15.618080+0000","last_active":"2024-09-13T08:53:16.623510+0000","last_peered":"2024-09-13T08:53:16.623510+0000","last_clean":"2024-09-13T08:53:16.623510+0000","last_became_active":"2024-09-13T08:53:15.617177+0000","last_became_peered":"2024-09-13T08:53:15.617177+0000","last_unstale":"2024-09-13T08:53:16.623510+0000","last_undegraded":"2024-09-13T08:53:16.623510+0000","last_fullsized":"2024-09-13T08:53:16.623510+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:02:49.554865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626931+0000","last_change":"2024-09-13T08:53:15.607705+0000","last_active":"2024-09-13T08:53:16.626931+0000","last_peered":"2024-09-13T08:53:16.626931+0000","last_clean":"2024-09-13T08:53:16.626931+0000","last_became_active":"2024-09-13T08:53:15.607426+0000","last_became_peered":"2024-09-13T08:53:15.607426+0000","last_unstale":"2024-09-13T08:53:16.626931+0000","last_undegraded":"2024-09-13T08:53:16.626931+0000","last_fullsized":"2024-09-13T08:53:16.626931+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:30.659179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624753+0000","last_change":"2024-09-13T08:53:15.612087+0000","last_active":"2024-09-13T08:53:16.624753+0000","last_peered":"2024-09-13T08:53:16.624753+0000","last_clean":"2024-09-13T08:53:16.624753+0000","last_became_active":"2024-09-13T08:53:15.611760+0000","last_became_peered":"2024-09-13T08:53:15.611760+0000","last_unstale":"2024-09-13T08:53:16.624753+0000","last_undegraded":"2024-09-13T08:53:16.624753+0000","last_fullsized":"2024-09-13T08:53:16.624753+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:41.555046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127127+0000","last_change":"2024-09-13T08:53:15.609980+0000","last_active":"2024-09-13T08:53:17.127127+0000","last_peered":"2024-09-13T08:53:17.127127+0000","last_clean":"2024-09-13T08:53:17.127127+0000","last_became_active":"2024-09-13T08:53:15.609388+0000","last_became_peered":"2024-09-13T08:53:15.609388+0000","last_unstale":"2024-09-13T08:53:17.127127+0000","last_undegraded":"2024-09-13T08:53:17.127127+0000","last_fullsized":"2024-09-13T08:53:17.127127+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:54.955446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129523+0000","last_change":"2024-09-13T08:53:15.613820+0000","last_active":"2024-09-13T08:53:17.129523+0000","last_peered":"2024-09-13T08:53:17.129523+0000","last_clean":"2024-09-13T08:53:17.129523+0000","last_became_active":"2024-09-13T08:53:15.613531+0000","last_became_peered":"2024-09-13T08:53:15.613531+0000","last_unstale":"2024-09-13T08:53:17.129523+0000","last_undegraded":"2024-09-13T08:53:17.129523+0000","last_fullsized":"2024-09-13T08:53:17.129523+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:16.193181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614649+0000","last_change":"2024-09-13T08:53:15.607043+0000","last_active":"2024-09-13T08:53:16.614649+0000","last_peered":"2024-09-13T08:53:16.614649+0000","last_clean":"2024-09-13T08:53:16.614649+0000","last_became_active":"2024-09-13T08:53:15.606686+0000","last_became_peered":"2024-09-13T08:53:15.606686+0000","last_unstale":"2024-09-13T08:53:16.614649+0000","last_undegraded":"2024-09-13T08:53:16.614649+0000","last_fullsized":"2024-09-13T08:53:16.614649+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:14.624303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614228+0000","last_change":"2024-09-13T08:53:15.625701+0000","last_active":"2024-09-13T08:53:16.614228+0000","last_peered":"2024-09-13T08:53:16.614228+0000","last_clean":"2024-09-13T08:53:16.614228+0000","last_became_active":"2024-09-13T08:53:15.625386+0000","last_became_peered":"2024-09-13T08:53:15.625386+0000","last_unstale":"2024-09-13T08:53:16.614228+0000","last_undegraded":"2024-09-13T08:53:16.614228+0000","last_fullsized":"2024-09-13T08:53:16.614228+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:35.675402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129499+0000","last_change":"2024-09-13T08:53:15.610999+0000","last_active":"2024-09-13T08:53:17.129499+0000","last_peered":"2024-09-13T08:53:17.129499+0000","last_clean":"2024-09-13T08:53:17.129499+0000","last_became_active":"2024-09-13T08:53:15.610709+0000","last_became_peered":"2024-09-13T08:53:15.610709+0000","last_unstale":"2024-09-13T08:53:17.129499+0000","last_undegraded":"2024-09-13T08:53:17.129499+0000","last_fullsized":"2024-09-13T08:53:17.129499+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:51.632629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.627107+0000","last_change":"2024-09-13T08:53:15.605883+0000","last_active":"2024-09-13T08:53:16.627107+0000","last_peered":"2024-09-13T08:53:16.627107+0000","last_clean":"2024-09-13T08:53:16.627107+0000","last_became_active":"2024-09-13T08:53:15.604354+0000","last_became_peered":"2024-09-13T08:53:15.604354+0000","last_unstale":"2024-09-13T08:53:16.627107+0000","last_undegraded":"2024-09-13T08:53:16.627107+0000","last_fullsized":"2024-09-13T08:53:16.627107+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:36.209208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129424+0000","last_change":"2024-09-13T08:53:15.608830+0000","last_active":"2024-09-13T08:53:17.129424+0000","last_peered":"2024-09-13T08:53:17.129424+0000","last_clean":"2024-09-13T08:53:17.129424+0000","last_became_active":"2024-09-13T08:53:15.608527+0000","last_became_peered":"2024-09-13T08:53:15.608527+0000","last_unstale":"2024-09-13T08:53:17.129424+0000","last_undegraded":"2024-09-13T08:53:17.129424+0000","last_fullsized":"2024-09-13T08:53:17.129424+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:06.563772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614175+0000","last_change":"2024-09-13T08:53:15.612419+0000","last_active":"2024-09-13T08:53:16.614175+0000","last_peered":"2024-09-13T08:53:16.614175+0000","last_clean":"2024-09-13T08:53:16.614175+0000","last_became_active":"2024-09-13T08:53:15.612145+0000","last_became_peered":"2024-09-13T08:53:15.612145+0000","last_unstale":"2024-09-13T08:53:16.614175+0000","last_undegraded":"2024-09-13T08:53:16.614175+0000","last_fullsized":"2024-09-13T08:53:16.614175+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:57:41.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127220+0000","last_change":"2024-09-13T08:53:15.609662+0000","last_active":"2024-09-13T08:53:17.127220+0000","last_peered":"2024-09-13T08:53:17.127220+0000","last_clean":"2024-09-13T08:53:17.127220+0000","last_became_active":"2024-09-13T08:53:15.609024+0000","last_became_peered":"2024-09-13T08:53:15.609024+0000","last_unstale":"2024-09-13T08:53:17.127220+0000","last_undegraded":"2024-09-13T08:53:17.127220+0000","last_fullsized":"2024-09-13T08:53:17.127220+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:09.589352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.613921+0000","last_change":"2024-09-13T08:53:15.615520+0000","last_active":"2024-09-13T08:53:16.613921+0000","last_peered":"2024-09-13T08:53:16.613921+0000","last_clean":"2024-09-13T08:53:16.613921+0000","last_became_active":"2024-09-13T08:53:15.614440+0000","last_became_peered":"2024-09-13T08:53:15.614440+0000","last_unstale":"2024-09-13T08:53:16.613921+0000","last_undegraded":"2024-09-13T08:53:16.613921+0000","last_fullsized":"2024-09-13T08:53:16.613921+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:41.555046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127125+0000","last_change":"2024-09-13T08:53:15.612875+0000","last_active":"2024-09-13T08:53:17.127125+0000","last_peered":"2024-09-13T08:53:17.127125+0000","last_clean":"2024-09-13T08:53:17.127125+0000","last_became_active":"2024-09-13T08:53:15.612614+0000","last_became_peered":"2024-09-13T08:53:15.612614+0000","last_unstale":"2024-09-13T08:53:17.127125+0000","last_undegraded":"2024-09-13T08:53:17.127125+0000","last_fullsized":"2024-09-13T08:53:17.127125+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:08.675301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614218+0000","last_change":"2024-09-13T08:53:15.601508+0000","last_active":"2024-09-13T08:53:16.614218+0000","last_peered":"2024-09-13T08:53:16.614218+0000","last_clean":"2024-09-13T08:53:16.614218+0000","last_became_active":"2024-09-13T08:53:15.601217+0000","last_became_peered":"2024-09-13T08:53:15.601217+0000","last_unstale":"2024-09-13T08:53:16.614218+0000","last_undegraded":"2024-09-13T08:53:16.614218+0000","last_fullsized":"2024-09-13T08:53:16.614218+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:08.001520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.627281+0000","last_change":"2024-09-13T08:53:15.604129+0000","last_active":"2024-09-13T08:53:16.627281+0000","last_peered":"2024-09-13T08:53:16.627281+0000","last_clean":"2024-09-13T08:53:16.627281+0000","last_became_active":"2024-09-13T08:53:15.603844+0000","last_became_peered":"2024-09-13T08:53:15.603844+0000","last_unstale":"2024-09-13T08:53:16.627281+0000","last_undegraded":"2024-09-13T08:53:16.627281+0000","last_fullsized":"2024-09-13T08:53:16.627281+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:53.045366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127225+0000","last_change":"2024-09-13T08:53:15.614058+0000","last_active":"2024-09-13T08:53:17.127225+0000","last_peered":"2024-09-13T08:53:17.127225+0000","last_clean":"2024-09-13T08:53:17.127225+0000","last_became_active":"2024-09-13T08:53:15.610159+0000","last_became_peered":"2024-09-13T08:53:15.610159+0000","last_unstale":"2024-09-13T08:53:17.127225+0000","last_undegraded":"2024-09-13T08:53:17.127225+0000","last_fullsized":"2024-09-13T08:53:17.127225+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:47:56.080386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614445+0000","last_change":"2024-09-13T08:53:15.616439+0000","last_active":"2024-09-13T08:53:16.614445+0000","last_peered":"2024-09-13T08:53:16.614445+0000","last_clean":"2024-09-13T08:53:16.614445+0000","last_became_active":"2024-09-13T08:53:15.616105+0000","last_became_peered":"2024-09-13T08:53:15.616105+0000","last_unstale":"2024-09-13T08:53:16.614445+0000","last_undegraded":"2024-09-13T08:53:16.614445+0000","last_fullsized":"2024-09-13T08:53:16.614445+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:45.954180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614686+0000","last_change":"2024-09-13T08:53:15.614259+0000","last_active":"2024-09-13T08:53:16.614686+0000","last_peered":"2024-09-13T08:53:16.614686+0000","last_clean":"2024-09-13T08:53:16.614686+0000","last_became_active":"2024-09-13T08:53:15.613790+0000","last_became_peered":"2024-09-13T08:53:15.613790+0000","last_unstale":"2024-09-13T08:53:16.614686+0000","last_undegraded":"2024-09-13T08:53:16.614686+0000","last_fullsized":"2024-09-13T08:53:16.614686+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:17:35.874448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626739+0000","last_change":"2024-09-13T08:53:15.604857+0000","last_active":"2024-09-13T08:53:16.626739+0000","last_peered":"2024-09-13T08:53:16.626739+0000","last_clean":"2024-09-13T08:53:16.626739+0000","last_became_active":"2024-09-13T08:53:15.604542+0000","last_became_peered":"2024-09-13T08:53:15.604542+0000","last_unstale":"2024-09-13T08:53:16.626739+0000","last_undegraded":"2024-09-13T08:53:16.626739+0000","last_fullsized":"2024-09-13T08:53:16.626739+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:26.046435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.613922+0000","last_change":"2024-09-13T08:53:15.614857+0000","last_active":"2024-09-13T08:53:16.613922+0000","last_peered":"2024-09-13T08:53:16.613922+0000","last_clean":"2024-09-13T08:53:16.613922+0000","last_became_active":"2024-09-13T08:53:15.610401+0000","last_became_peered":"2024-09-13T08:53:15.610401+0000","last_unstale":"2024-09-13T08:53:16.613922+0000","last_undegraded":"2024-09-13T08:53:16.613922+0000","last_fullsized":"2024-09-13T08:53:16.613922+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:52.689676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624144+0000","last_change":"2024-09-13T08:53:15.613034+0000","last_active":"2024-09-13T08:53:16.624144+0000","last_peered":"2024-09-13T08:53:16.624144+0000","last_clean":"2024-09-13T08:53:16.624144+0000","last_became_active":"2024-09-13T08:53:15.612492+0000","last_became_peered":"2024-09-13T08:53:15.612492+0000","last_unstale":"2024-09-13T08:53:16.624144+0000","last_undegraded":"2024-09-13T08:53:16.624144+0000","last_fullsized":"2024-09-13T08:53:16.624144+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:43.250681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.126801+0000","last_change":"2024-09-13T08:53:15.615654+0000","last_active":"2024-09-13T08:53:17.126801+0000","last_peered":"2024-09-13T08:53:17.126801+0000","last_clean":"2024-09-13T08:53:17.126801+0000","last_became_active":"2024-09-13T08:53:15.615357+0000","last_became_peered":"2024-09-13T08:53:15.615357+0000","last_unstale":"2024-09-13T08:53:17.126801+0000","last_undegraded":"2024-09-13T08:53:17.126801+0000","last_fullsized":"2024-09-13T08:53:17.126801+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:55.420448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623679+0000","last_change":"2024-09-13T08:53:15.603459+0000","last_active":"2024-09-13T08:53:16.623679+0000","last_peered":"2024-09-13T08:53:16.623679+0000","last_clean":"2024-09-13T08:53:16.623679+0000","last_became_active":"2024-09-13T08:53:15.603186+0000","last_became_peered":"2024-09-13T08:53:15.603186+0000","last_unstale":"2024-09-13T08:53:16.623679+0000","last_undegraded":"2024-09-13T08:53:16.623679+0000","last_fullsized":"2024-09-13T08:53:16.623679+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:03.529118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614520+0000","last_change":"2024-09-13T08:53:15.613234+0000","last_active":"2024-09-13T08:53:16.614520+0000","last_peered":"2024-09-13T08:53:16.614520+0000","last_clean":"2024-09-13T08:53:16.614520+0000","last_became_active":"2024-09-13T08:53:15.612853+0000","last_became_peered":"2024-09-13T08:53:15.612853+0000","last_unstale":"2024-09-13T08:53:16.614520+0000","last_undegraded":"2024-09-13T08:53:16.614520+0000","last_fullsized":"2024-09-13T08:53:16.614520+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:14.522632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623394+0000","last_change":"2024-09-13T08:53:15.604351+0000","last_active":"2024-09-13T08:53:16.623394+0000","last_peered":"2024-09-13T08:53:16.623394+0000","last_clean":"2024-09-13T08:53:16.623394+0000","last_became_active":"2024-09-13T08:53:15.603945+0000","last_became_peered":"2024-09-13T08:53:15.603945+0000","last_unstale":"2024-09-13T08:53:16.623394+0000","last_undegraded":"2024-09-13T08:53:16.623394+0000","last_fullsized":"2024-09-13T08:53:16.623394+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:27.087948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623492+0000","last_change":"2024-09-13T08:53:15.616379+0000","last_active":"2024-09-13T08:53:16.623492+0000","last_peered":"2024-09-13T08:53:16.623492+0000","last_clean":"2024-09-13T08:53:16.623492+0000","last_became_active":"2024-09-13T08:53:15.614571+0000","last_became_peered":"2024-09-13T08:53:15.614571+0000","last_unstale":"2024-09-13T08:53:16.623492+0000","last_undegraded":"2024-09-13T08:53:16.623492+0000","last_fullsized":"2024-09-13T08:53:16.623492+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:06.040200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614915+0000","last_change":"2024-09-13T08:53:15.625045+0000","last_active":"2024-09-13T08:53:16.614915+0000","last_peered":"2024-09-13T08:53:16.614915+0000","last_clean":"2024-09-13T08:53:16.614915+0000","last_became_active":"2024-09-13T08:53:15.624239+0000","last_became_peered":"2024-09-13T08:53:15.624239+0000","last_unstale":"2024-09-13T08:53:16.614915+0000","last_undegraded":"2024-09-13T08:53:16.614915+0000","last_fullsized":"2024-09-13T08:53:16.614915+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:37.874956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614194+0000","last_change":"2024-09-13T08:53:15.613649+0000","last_active":"2024-09-13T08:53:16.614194+0000","last_peered":"2024-09-13T08:53:16.614194+0000","last_clean":"2024-09-13T08:53:16.614194+0000","last_became_active":"2024-09-13T08:53:15.610200+0000","last_became_peered":"2024-09-13T08:53:15.610200+0000","last_unstale":"2024-09-13T08:53:16.614194+0000","last_undegraded":"2024-09-13T08:53:16.614194+0000","last_fullsized":"2024-09-13T08:53:16.614194+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:30.034701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623197+0000","last_change":"2024-09-13T08:53:15.607987+0000","last_active":"2024-09-13T08:53:16.623197+0000","last_peered":"2024-09-13T08:53:16.623197+0000","last_clean":"2024-09-13T08:53:16.623197+0000","last_became_active":"2024-09-13T08:53:15.607729+0000","last_became_peered":"2024-09-13T08:53:15.607729+0000","last_unstale":"2024-09-13T08:53:16.623197+0000","last_undegraded":"2024-09-13T08:53:16.623197+0000","last_fullsized":"2024-09-13T08:53:16.623197+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:45:25.782674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623363+0000","last_change":"2024-09-13T08:53:15.611359+0000","last_active":"2024-09-13T08:53:16.623363+0000","last_peered":"2024-09-13T08:53:16.623363+0000","last_clean":"2024-09-13T08:53:16.623363+0000","last_became_active":"2024-09-13T08:53:15.610868+0000","last_became_peered":"2024-09-13T08:53:15.610868+0000","last_unstale":"2024-09-13T08:53:16.623363+0000","last_undegraded":"2024-09-13T08:53:16.623363+0000","last_fullsized":"2024-09-13T08:53:16.623363+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:43.485104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127227+0000","last_change":"2024-09-13T08:53:15.620233+0000","last_active":"2024-09-13T08:53:17.127227+0000","last_peered":"2024-09-13T08:53:17.127227+0000","last_clean":"2024-09-13T08:53:17.127227+0000","last_became_active":"2024-09-13T08:53:15.619922+0000","last_became_peered":"2024-09-13T08:53:15.619922+0000","last_unstale":"2024-09-13T08:53:17.127227+0000","last_undegraded":"2024-09-13T08:53:17.127227+0000","last_fullsized":"2024-09-13T08:53:17.127227+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:05.522103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"143'4","reported_seq":19,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.627212+0000","last_change":"2024-09-13T08:53:14.609734+0000","last_active":"2024-09-13T08:53:16.627212+0000","last_peered":"2024-09-13T08:53:16.627212+0000","last_clean":"2024-09-13T08:53:16.627212+0000","last_became_active":"2024-09-13T08:53:14.609456+0000","last_became_peered":"2024-09-13T08:53:14.609456+0000","last_unstale":"2024-09-13T08:53:16.627212+0000","last_undegraded":"2024-09-13T08:53:16.627212+0000","last_fullsized":"2024-09-13T08:53:16.627212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:28:26.519784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615216+0000","last_change":"2024-09-13T08:53:15.616412+0000","last_active":"2024-09-13T08:53:16.615216+0000","last_peered":"2024-09-13T08:53:16.615216+0000","last_clean":"2024-09-13T08:53:16.615216+0000","last_became_active":"2024-09-13T08:53:15.616103+0000","last_became_peered":"2024-09-13T08:53:15.616103+0000","last_unstale":"2024-09-13T08:53:16.615216+0000","last_undegraded":"2024-09-13T08:53:16.615216+0000","last_fullsized":"2024-09-13T08:53:16.615216+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:07.581265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129266+0000","last_change":"2024-09-13T08:53:14.613339+0000","last_active":"2024-09-13T08:53:17.129266+0000","last_peered":"2024-09-13T08:53:17.129266+0000","last_clean":"2024-09-13T08:53:17.129266+0000","last_became_active":"2024-09-13T08:53:14.612799+0000","last_became_peered":"2024-09-13T08:53:14.612799+0000","last_unstale":"2024-09-13T08:53:17.129266+0000","last_undegraded":"2024-09-13T08:53:17.129266+0000","last_fullsized":"2024-09-13T08:53:17.129266+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:54.292680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129342+0000","last_change":"2024-09-13T08:53:15.616934+0000","last_active":"2024-09-13T08:53:17.129342+0000","last_peered":"2024-09-13T08:53:17.129342+0000","last_clean":"2024-09-13T08:53:17.129342+0000","last_became_active":"2024-09-13T08:53:15.616660+0000","last_became_peered":"2024-09-13T08:53:15.616660+0000","last_unstale":"2024-09-13T08:53:17.129342+0000","last_undegraded":"2024-09-13T08:53:17.129342+0000","last_fullsized":"2024-09-13T08:53:17.129342+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:52:39.024260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626829+0000","last_change":"2024-09-13T08:53:14.600079+0000","last_active":"2024-09-13T08:53:16.626829+0000","last_peered":"2024-09-13T08:53:16.626829+0000","last_clean":"2024-09-13T08:53:16.626829+0000","last_became_active":"2024-09-13T08:53:14.599758+0000","last_became_peered":"2024-09-13T08:53:14.599758+0000","last_unstale":"2024-09-13T08:53:16.626829+0000","last_undegraded":"2024-09-13T08:53:16.626829+0000","last_fullsized":"2024-09-13T08:53:16.626829+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:47.082774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625163+0000","last_change":"2024-09-13T08:53:15.613853+0000","last_active":"2024-09-13T08:53:16.625163+0000","last_peered":"2024-09-13T08:53:16.625163+0000","last_clean":"2024-09-13T08:53:16.625163+0000","last_became_active":"2024-09-13T08:53:15.610422+0000","last_became_peered":"2024-09-13T08:53:15.610422+0000","last_unstale":"2024-09-13T08:53:16.625163+0000","last_undegraded":"2024-09-13T08:53:16.625163+0000","last_fullsized":"2024-09-13T08:53:16.625163+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:57:41.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626714+0000","last_change":"2024-09-13T08:53:14.604187+0000","last_active":"2024-09-13T08:53:16.626714+0000","last_peered":"2024-09-13T08:53:16.626714+0000","last_clean":"2024-09-13T08:53:16.626714+0000","last_became_active":"2024-09-13T08:53:14.603875+0000","last_became_peered":"2024-09-13T08:53:14.603875+0000","last_unstale":"2024-09-13T08:53:16.626714+0000","last_undegraded":"2024-09-13T08:53:16.626714+0000","last_fullsized":"2024-09-13T08:53:16.626714+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:28.876441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623347+0000","last_change":"2024-09-13T08:53:15.604116+0000","last_active":"2024-09-13T08:53:16.623347+0000","last_peered":"2024-09-13T08:53:16.623347+0000","last_clean":"2024-09-13T08:53:16.623347+0000","last_became_active":"2024-09-13T08:53:15.603807+0000","last_became_peered":"2024-09-13T08:53:15.603807+0000","last_unstale":"2024-09-13T08:53:16.623347+0000","last_undegraded":"2024-09-13T08:53:16.623347+0000","last_fullsized":"2024-09-13T08:53:16.623347+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:31:32.926107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614982+0000","last_change":"2024-09-13T08:53:14.609131+0000","last_active":"2024-09-13T08:53:16.614982+0000","last_peered":"2024-09-13T08:53:16.614982+0000","last_clean":"2024-09-13T08:53:16.614982+0000","last_became_active":"2024-09-13T08:53:14.608869+0000","last_became_peered":"2024-09-13T08:53:14.608869+0000","last_unstale":"2024-09-13T08:53:16.614982+0000","last_undegraded":"2024-09-13T08:53:16.614982+0000","last_fullsized":"2024-09-13T08:53:16.614982+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:12.085049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624622+0000","last_change":"2024-09-13T08:53:15.629197+0000","last_active":"2024-09-13T08:53:16.624622+0000","last_peered":"2024-09-13T08:53:16.624622+0000","last_clean":"2024-09-13T08:53:16.624622+0000","last_became_active":"2024-09-13T08:53:15.626968+0000","last_became_peered":"2024-09-13T08:53:15.626968+0000","last_unstale":"2024-09-13T08:53:16.624622+0000","last_undegraded":"2024-09-13T08:53:16.624622+0000","last_fullsized":"2024-09-13T08:53:16.624622+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:03.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.126661+0000","last_change":"2024-09-13T08:53:14.613851+0000","last_active":"2024-09-13T08:53:17.126661+0000","last_peered":"2024-09-13T08:53:17.126661+0000","last_clean":"2024-09-13T08:53:17.126661+0000","last_became_active":"2024-09-13T08:53:14.613340+0000","last_became_peered":"2024-09-13T08:53:14.613340+0000","last_unstale":"2024-09-13T08:53:17.126661+0000","last_undegraded":"2024-09-13T08:53:17.126661+0000","last_fullsized":"2024-09-13T08:53:17.126661+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:40.082146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623144+0000","last_change":"2024-09-13T08:53:15.611980+0000","last_active":"2024-09-13T08:53:16.623144+0000","last_peered":"2024-09-13T08:53:16.623144+0000","last_clean":"2024-09-13T08:53:16.623144+0000","last_became_active":"2024-09-13T08:53:15.611737+0000","last_became_peered":"2024-09-13T08:53:15.611737+0000","last_unstale":"2024-09-13T08:53:16.623144+0000","last_undegraded":"2024-09-13T08:53:16.623144+0000","last_fullsized":"2024-09-13T08:53:16.623144+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:32:37.579887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129340+0000","last_change":"2024-09-13T08:53:14.609106+0000","last_active":"2024-09-13T08:53:17.129340+0000","last_peered":"2024-09-13T08:53:17.129340+0000","last_clean":"2024-09-13T08:53:17.129340+0000","last_became_active":"2024-09-13T08:53:14.608810+0000","last_became_peered":"2024-09-13T08:53:14.608810+0000","last_unstale":"2024-09-13T08:53:17.129340+0000","last_undegraded":"2024-09-13T08:53:17.129340+0000","last_fullsized":"2024-09-13T08:53:17.129340+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:44.027745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624213+0000","last_change":"2024-09-13T08:53:15.620180+0000","last_active":"2024-09-13T08:53:16.624213+0000","last_peered":"2024-09-13T08:53:16.624213+0000","last_clean":"2024-09-13T08:53:16.624213+0000","last_became_active":"2024-09-13T08:53:15.617653+0000","last_became_peered":"2024-09-13T08:53:15.617653+0000","last_unstale":"2024-09-13T08:53:16.624213+0000","last_undegraded":"2024-09-13T08:53:16.624213+0000","last_fullsized":"2024-09-13T08:53:16.624213+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:46.287335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626385+0000","last_change":"2024-09-13T08:53:14.612322+0000","last_active":"2024-09-13T08:53:16.626385+0000","last_peered":"2024-09-13T08:53:16.626385+0000","last_clean":"2024-09-13T08:53:16.626385+0000","last_became_active":"2024-09-13T08:53:14.612007+0000","last_became_peered":"2024-09-13T08:53:14.612007+0000","last_unstale":"2024-09-13T08:53:16.626385+0000","last_undegraded":"2024-09-13T08:53:16.626385+0000","last_fullsized":"2024-09-13T08:53:16.626385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:08.627533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625197+0000","last_change":"2024-09-13T08:53:15.618278+0000","last_active":"2024-09-13T08:53:16.625197+0000","last_peered":"2024-09-13T08:53:16.625197+0000","last_clean":"2024-09-13T08:53:16.625197+0000","last_became_active":"2024-09-13T08:53:15.617694+0000","last_became_peered":"2024-09-13T08:53:15.617694+0000","last_unstale":"2024-09-13T08:53:16.625197+0000","last_undegraded":"2024-09-13T08:53:16.625197+0000","last_fullsized":"2024-09-13T08:53:16.625197+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:23.022502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129413+0000","last_change":"2024-09-13T08:53:14.611525+0000","last_active":"2024-09-13T08:53:17.129413+0000","last_peered":"2024-09-13T08:53:17.129413+0000","last_clean":"2024-09-13T08:53:17.129413+0000","last_became_active":"2024-09-13T08:53:14.611246+0000","last_became_peered":"2024-09-13T08:53:14.611246+0000","last_unstale":"2024-09-13T08:53:17.129413+0000","last_undegraded":"2024-09-13T08:53:17.129413+0000","last_fullsized":"2024-09-13T08:53:17.129413+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:51.437020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614438+0000","last_change":"2024-09-13T08:53:15.623604+0000","last_active":"2024-09-13T08:53:16.614438+0000","last_peered":"2024-09-13T08:53:16.614438+0000","last_clean":"2024-09-13T08:53:16.614438+0000","last_became_active":"2024-09-13T08:53:15.623149+0000","last_became_peered":"2024-09-13T08:53:15.623149+0000","last_unstale":"2024-09-13T08:53:16.614438+0000","last_undegraded":"2024-09-13T08:53:16.614438+0000","last_fullsized":"2024-09-13T08:53:16.614438+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:58.827614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127307+0000","last_change":"2024-09-13T08:53:14.605714+0000","last_active":"2024-09-13T08:53:17.127307+0000","last_peered":"2024-09-13T08:53:17.127307+0000","last_clean":"2024-09-13T08:53:17.127307+0000","last_became_active":"2024-09-13T08:53:14.604522+0000","last_became_peered":"2024-09-13T08:53:14.604522+0000","last_unstale":"2024-09-13T08:53:17.127307+0000","last_undegraded":"2024-09-13T08:53:17.127307+0000","last_fullsized":"2024-09-13T08:53:17.127307+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:00:13.328557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615848+0000","last_change":"2024-09-13T08:53:15.602657+0000","last_active":"2024-09-13T08:53:16.615848+0000","last_peered":"2024-09-13T08:53:16.615848+0000","last_clean":"2024-09-13T08:53:16.615848+0000","last_became_active":"2024-09-13T08:53:15.602363+0000","last_became_peered":"2024-09-13T08:53:15.602363+0000","last_unstale":"2024-09-13T08:53:16.615848+0000","last_undegraded":"2024-09-13T08:53:16.615848+0000","last_fullsized":"2024-09-13T08:53:16.615848+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:14:11.309419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"143'6","reported_seq":21,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615631+0000","last_change":"2024-09-13T08:53:14.609450+0000","last_active":"2024-09-13T08:53:16.615631+0000","last_peered":"2024-09-13T08:53:16.615631+0000","last_clean":"2024-09-13T08:53:16.615631+0000","last_became_active":"2024-09-13T08:53:14.608508+0000","last_became_peered":"2024-09-13T08:53:14.608508+0000","last_unstale":"2024-09-13T08:53:16.615631+0000","last_undegraded":"2024-09-13T08:53:16.615631+0000","last_fullsized":"2024-09-13T08:53:16.615631+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:47.008225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624204+0000","last_change":"2024-09-13T08:53:15.615985+0000","last_active":"2024-09-13T08:53:16.624204+0000","last_peered":"2024-09-13T08:53:16.624204+0000","last_clean":"2024-09-13T08:53:16.624204+0000","last_became_active":"2024-09-13T08:53:15.615451+0000","last_became_peered":"2024-09-13T08:53:15.615451+0000","last_unstale":"2024-09-13T08:53:16.624204+0000","last_undegraded":"2024-09-13T08:53:16.624204+0000","last_fullsized":"2024-09-13T08:53:16.624204+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:20.002995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626608+0000","last_change":"2024-09-13T08:53:14.601073+0000","last_active":"2024-09-13T08:53:16.626608+0000","last_peered":"2024-09-13T08:53:16.626608+0000","last_clean":"2024-09-13T08:53:16.626608+0000","last_became_active":"2024-09-13T08:53:14.600773+0000","last_became_peered":"2024-09-13T08:53:14.600773+0000","last_unstale":"2024-09-13T08:53:16.626608+0000","last_undegraded":"2024-09-13T08:53:16.626608+0000","last_fullsized":"2024-09-13T08:53:16.626608+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:35.075345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615378+0000","last_change":"2024-09-13T08:53:15.625078+0000","last_active":"2024-09-13T08:53:16.615378+0000","last_peered":"2024-09-13T08:53:16.615378+0000","last_clean":"2024-09-13T08:53:16.615378+0000","last_became_active":"2024-09-13T08:53:15.624764+0000","last_became_peered":"2024-09-13T08:53:15.624764+0000","last_unstale":"2024-09-13T08:53:16.615378+0000","last_undegraded":"2024-09-13T08:53:16.615378+0000","last_fullsized":"2024-09-13T08:53:16.615378+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:16:53.839834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614260+0000","last_change":"2024-09-13T08:53:14.610271+0000","last_active":"2024-09-13T08:53:16.614260+0000","last_peered":"2024-09-13T08:53:16.614260+0000","last_clean":"2024-09-13T08:53:16.614260+0000","last_became_active":"2024-09-13T08:53:14.609955+0000","last_became_peered":"2024-09-13T08:53:14.609955+0000","last_unstale":"2024-09-13T08:53:16.614260+0000","last_undegraded":"2024-09-13T08:53:16.614260+0000","last_fullsized":"2024-09-13T08:53:16.614260+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:03:59.434188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624009+0000","last_change":"2024-09-13T08:53:15.611435+0000","last_active":"2024-09-13T08:53:16.624009+0000","last_peered":"2024-09-13T08:53:16.624009+0000","last_clean":"2024-09-13T08:53:16.624009+0000","last_became_active":"2024-09-13T08:53:15.611151+0000","last_became_peered":"2024-09-13T08:53:15.611151+0000","last_unstale":"2024-09-13T08:53:16.624009+0000","last_undegraded":"2024-09-13T08:53:16.624009+0000","last_fullsized":"2024-09-13T08:53:16.624009+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:01.835402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623216+0000","last_change":"2024-09-13T08:53:14.607433+0000","last_active":"2024-09-13T08:53:16.623216+0000","last_peered":"2024-09-13T08:53:16.623216+0000","last_clean":"2024-09-13T08:53:16.623216+0000","last_became_active":"2024-09-13T08:53:14.607139+0000","last_became_peered":"2024-09-13T08:53:14.607139+0000","last_unstale":"2024-09-13T08:53:16.623216+0000","last_undegraded":"2024-09-13T08:53:16.623216+0000","last_fullsized":"2024-09-13T08:53:16.623216+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:21:48.609279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624355+0000","last_change":"2024-09-13T08:53:15.612753+0000","last_active":"2024-09-13T08:53:16.624355+0000","last_peered":"2024-09-13T08:53:16.624355+0000","last_clean":"2024-09-13T08:53:16.624355+0000","last_became_active":"2024-09-13T08:53:15.612138+0000","last_became_peered":"2024-09-13T08:53:15.612138+0000","last_unstale":"2024-09-13T08:53:16.624355+0000","last_undegraded":"2024-09-13T08:53:16.624355+0000","last_fullsized":"2024-09-13T08:53:16.624355+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:48.015020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"143'8","reported_seq":23,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614230+0000","last_change":"2024-09-13T08:53:14.607820+0000","last_active":"2024-09-13T08:53:16.614230+0000","last_peered":"2024-09-13T08:53:16.614230+0000","last_clean":"2024-09-13T08:53:16.614230+0000","last_became_active":"2024-09-13T08:53:14.607522+0000","last_became_peered":"2024-09-13T08:53:14.607522+0000","last_unstale":"2024-09-13T08:53:16.614230+0000","last_undegraded":"2024-09-13T08:53:16.614230+0000","last_fullsized":"2024-09-13T08:53:16.614230+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:41.446670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.127413+0000","last_change":"2024-09-13T08:53:15.617227+0000","last_active":"2024-09-13T08:53:17.127413+0000","last_peered":"2024-09-13T08:53:17.127413+0000","last_clean":"2024-09-13T08:53:17.127413+0000","last_became_active":"2024-09-13T08:53:15.616950+0000","last_became_peered":"2024-09-13T08:53:15.616950+0000","last_unstale":"2024-09-13T08:53:17.127413+0000","last_undegraded":"2024-09-13T08:53:17.127413+0000","last_fullsized":"2024-09-13T08:53:17.127413+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:17.577743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625495+0000","last_change":"2024-09-13T08:53:14.608024+0000","last_active":"2024-09-13T08:53:16.625495+0000","last_peered":"2024-09-13T08:53:16.625495+0000","last_clean":"2024-09-13T08:53:16.625495+0000","last_became_active":"2024-09-13T08:53:14.607741+0000","last_became_peered":"2024-09-13T08:53:14.607741+0000","last_unstale":"2024-09-13T08:53:16.625495+0000","last_undegraded":"2024-09-13T08:53:16.625495+0000","last_fullsized":"2024-09-13T08:53:16.625495+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:45:24.775879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623792+0000","last_change":"2024-09-13T08:53:15.607225+0000","last_active":"2024-09-13T08:53:16.623792+0000","last_peered":"2024-09-13T08:53:16.623792+0000","last_clean":"2024-09-13T08:53:16.623792+0000","last_became_active":"2024-09-13T08:53:15.606970+0000","last_became_peered":"2024-09-13T08:53:15.606970+0000","last_unstale":"2024-09-13T08:53:16.623792+0000","last_undegraded":"2024-09-13T08:53:16.623792+0000","last_fullsized":"2024-09-13T08:53:16.623792+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:01.304432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614584+0000","last_change":"2024-09-13T08:53:14.613217+0000","last_active":"2024-09-13T08:53:16.614584+0000","last_peered":"2024-09-13T08:53:16.614584+0000","last_clean":"2024-09-13T08:53:16.614584+0000","last_became_active":"2024-09-13T08:53:14.612795+0000","last_became_peered":"2024-09-13T08:53:14.612795+0000","last_unstale":"2024-09-13T08:53:16.614584+0000","last_undegraded":"2024-09-13T08:53:16.614584+0000","last_fullsized":"2024-09-13T08:53:16.614584+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:29:05.403502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.126914+0000","last_change":"2024-09-13T08:53:15.616529+0000","last_active":"2024-09-13T08:53:17.126914+0000","last_peered":"2024-09-13T08:53:17.126914+0000","last_clean":"2024-09-13T08:53:17.126914+0000","last_became_active":"2024-09-13T08:53:15.616238+0000","last_became_peered":"2024-09-13T08:53:15.616238+0000","last_unstale":"2024-09-13T08:53:17.126914+0000","last_undegraded":"2024-09-13T08:53:17.126914+0000","last_fullsized":"2024-09-13T08:53:17.126914+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:38.099277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"143'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615989+0000","last_change":"2024-09-13T08:53:14.607098+0000","last_active":"2024-09-13T08:53:16.615989+0000","last_peered":"2024-09-13T08:53:16.615989+0000","last_clean":"2024-09-13T08:53:16.615989+0000","last_became_active":"2024-09-13T08:53:14.606857+0000","last_became_peered":"2024-09-13T08:53:14.606857+0000","last_unstale":"2024-09-13T08:53:16.615989+0000","last_undegraded":"2024-09-13T08:53:16.615989+0000","last_fullsized":"2024-09-13T08:53:16.615989+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:40.548251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129489+0000","last_change":"2024-09-13T08:53:15.615099+0000","last_active":"2024-09-13T08:53:17.129489+0000","last_peered":"2024-09-13T08:53:17.129489+0000","last_clean":"2024-09-13T08:53:17.129489+0000","last_became_active":"2024-09-13T08:53:15.614834+0000","last_became_peered":"2024-09-13T08:53:15.614834+0000","last_unstale":"2024-09-13T08:53:17.129489+0000","last_undegraded":"2024-09-13T08:53:17.129489+0000","last_fullsized":"2024-09-13T08:53:17.129489+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:07:34.923374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623319+0000","last_change":"2024-09-13T08:53:14.600940+0000","last_active":"2024-09-13T08:53:16.623319+0000","last_peered":"2024-09-13T08:53:16.623319+0000","last_clean":"2024-09-13T08:53:16.623319+0000","last_became_active":"2024-09-13T08:53:14.600642+0000","last_became_peered":"2024-09-13T08:53:14.600642+0000","last_unstale":"2024-09-13T08:53:16.623319+0000","last_undegraded":"2024-09-13T08:53:16.623319+0000","last_fullsized":"2024-09-13T08:53:16.623319+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:07.668506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614193+0000","last_change":"2024-09-13T08:53:15.615878+0000","last_active":"2024-09-13T08:53:16.614193+0000","last_peered":"2024-09-13T08:53:16.614193+0000","last_clean":"2024-09-13T08:53:16.614193+0000","last_became_active":"2024-09-13T08:53:15.615238+0000","last_became_peered":"2024-09-13T08:53:15.615238+0000","last_unstale":"2024-09-13T08:53:16.614193+0000","last_undegraded":"2024-09-13T08:53:16.614193+0000","last_fullsized":"2024-09-13T08:53:16.614193+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:23.022502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129245+0000","last_change":"2024-09-13T08:53:14.613189+0000","last_active":"2024-09-13T08:53:17.129245+0000","last_peered":"2024-09-13T08:53:17.129245+0000","last_clean":"2024-09-13T08:53:17.129245+0000","last_became_active":"2024-09-13T08:53:14.612877+0000","last_became_peered":"2024-09-13T08:53:14.612877+0000","last_unstale":"2024-09-13T08:53:17.129245+0000","last_undegraded":"2024-09-13T08:53:17.129245+0000","last_fullsized":"2024-09-13T08:53:17.129245+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:12.557688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623444+0000","last_change":"2024-09-13T08:53:15.605208+0000","last_active":"2024-09-13T08:53:16.623444+0000","last_peered":"2024-09-13T08:53:16.623444+0000","last_clean":"2024-09-13T08:53:16.623444+0000","last_became_active":"2024-09-13T08:53:15.604912+0000","last_became_peered":"2024-09-13T08:53:15.604912+0000","last_unstale":"2024-09-13T08:53:16.623444+0000","last_undegraded":"2024-09-13T08:53:16.623444+0000","last_fullsized":"2024-09-13T08:53:16.623444+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:41.385047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624172+0000","last_change":"2024-09-13T08:53:14.601389+0000","last_active":"2024-09-13T08:53:16.624172+0000","last_peered":"2024-09-13T08:53:16.624172+0000","last_clean":"2024-09-13T08:53:16.624172+0000","last_became_active":"2024-09-13T08:53:14.601109+0000","last_became_peered":"2024-09-13T08:53:14.601109+0000","last_unstale":"2024-09-13T08:53:16.624172+0000","last_undegraded":"2024-09-13T08:53:16.624172+0000","last_fullsized":"2024-09-13T08:53:16.624172+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:42.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.613772+0000","last_change":"2024-09-13T08:53:15.612278+0000","last_active":"2024-09-13T08:53:16.613772+0000","last_peered":"2024-09-13T08:53:16.613772+0000","last_clean":"2024-09-13T08:53:16.613772+0000","last_became_active":"2024-09-13T08:53:15.609387+0000","last_became_peered":"2024-09-13T08:53:15.609387+0000","last_unstale":"2024-09-13T08:53:16.613772+0000","last_undegraded":"2024-09-13T08:53:16.613772+0000","last_fullsized":"2024-09-13T08:53:16.613772+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:03.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624283+0000","last_change":"2024-09-13T08:53:14.606466+0000","last_active":"2024-09-13T08:53:16.624283+0000","last_peered":"2024-09-13T08:53:16.624283+0000","last_clean":"2024-09-13T08:53:16.624283+0000","last_became_active":"2024-09-13T08:53:14.606205+0000","last_became_peered":"2024-09-13T08:53:14.606205+0000","last_unstale":"2024-09-13T08:53:16.624283+0000","last_undegraded":"2024-09-13T08:53:16.624283+0000","last_fullsized":"2024-09-13T08:53:16.624283+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:55:32.093615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623141+0000","last_change":"2024-09-13T08:53:15.609120+0000","last_active":"2024-09-13T08:53:16.623141+0000","last_peered":"2024-09-13T08:53:16.623141+0000","last_clean":"2024-09-13T08:53:16.623141+0000","last_became_active":"2024-09-13T08:53:15.607949+0000","last_became_peered":"2024-09-13T08:53:15.607949+0000","last_unstale":"2024-09-13T08:53:16.623141+0000","last_undegraded":"2024-09-13T08:53:16.623141+0000","last_fullsized":"2024-09-13T08:53:16.623141+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:45.407786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624140+0000","last_change":"2024-09-13T08:53:14.601703+0000","last_active":"2024-09-13T08:53:16.624140+0000","last_peered":"2024-09-13T08:53:16.624140+0000","last_clean":"2024-09-13T08:53:16.624140+0000","last_became_active":"2024-09-13T08:53:14.601345+0000","last_became_peered":"2024-09-13T08:53:14.601345+0000","last_unstale":"2024-09-13T08:53:16.624140+0000","last_undegraded":"2024-09-13T08:53:16.624140+0000","last_fullsized":"2024-09-13T08:53:16.624140+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:24:09.365454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625254+0000","last_change":"2024-09-13T08:53:15.611309+0000","last_active":"2024-09-13T08:53:16.625254+0000","last_peered":"2024-09-13T08:53:16.625254+0000","last_clean":"2024-09-13T08:53:16.625254+0000","last_became_active":"2024-09-13T08:53:15.611003+0000","last_became_peered":"2024-09-13T08:53:15.611003+0000","last_unstale":"2024-09-13T08:53:16.625254+0000","last_undegraded":"2024-09-13T08:53:16.625254+0000","last_fullsized":"2024-09-13T08:53:16.625254+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:54:39.468468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"143'2","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625587+0000","last_change":"2024-09-13T08:53:14.608168+0000","last_active":"2024-09-13T08:53:16.625587+0000","last_peered":"2024-09-13T08:53:16.625587+0000","last_clean":"2024-09-13T08:53:16.625587+0000","last_became_active":"2024-09-13T08:53:14.607901+0000","last_became_peered":"2024-09-13T08:53:14.607901+0000","last_unstale":"2024-09-13T08:53:16.625587+0000","last_undegraded":"2024-09-13T08:53:16.625587+0000","last_fullsized":"2024-09-13T08:53:16.625587+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:40.977746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.625528+0000","last_change":"2024-09-13T08:53:15.627757+0000","last_active":"2024-09-13T08:53:16.625528+0000","last_peered":"2024-09-13T08:53:16.625528+0000","last_clean":"2024-09-13T08:53:16.625528+0000","last_became_active":"2024-09-13T08:53:15.627028+0000","last_became_peered":"2024-09-13T08:53:15.627028+0000","last_unstale":"2024-09-13T08:53:16.625528+0000","last_undegraded":"2024-09-13T08:53:16.625528+0000","last_fullsized":"2024-09-13T08:53:16.625528+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:58.827614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"144'1","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623510+0000","last_change":"2024-09-13T08:53:14.608469+0000","last_active":"2024-09-13T08:53:16.623510+0000","last_peered":"2024-09-13T08:53:16.623510+0000","last_clean":"2024-09-13T08:53:16.623510+0000","last_became_active":"2024-09-13T08:53:14.608207+0000","last_became_peered":"2024-09-13T08:53:14.608207+0000","last_unstale":"2024-09-13T08:53:16.623510+0000","last_undegraded":"2024-09-13T08:53:16.623510+0000","last_fullsized":"2024-09-13T08:53:16.623510+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:42.989669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614947+0000","last_change":"2024-09-13T08:53:15.614049+0000","last_active":"2024-09-13T08:53:16.614947+0000","last_peered":"2024-09-13T08:53:16.614947+0000","last_clean":"2024-09-13T08:53:16.614947+0000","last_became_active":"2024-09-13T08:53:15.613444+0000","last_became_peered":"2024-09-13T08:53:15.613444+0000","last_unstale":"2024-09-13T08:53:16.614947+0000","last_undegraded":"2024-09-13T08:53:16.614947+0000","last_fullsized":"2024-09-13T08:53:16.614947+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:06.092572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615986+0000","last_change":"2024-09-13T08:53:14.609225+0000","last_active":"2024-09-13T08:53:16.615986+0000","last_peered":"2024-09-13T08:53:16.615986+0000","last_clean":"2024-09-13T08:53:16.615986+0000","last_became_active":"2024-09-13T08:53:14.608429+0000","last_became_peered":"2024-09-13T08:53:14.608429+0000","last_unstale":"2024-09-13T08:53:16.615986+0000","last_undegraded":"2024-09-13T08:53:16.615986+0000","last_fullsized":"2024-09-13T08:53:16.615986+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:02.734300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614284+0000","last_change":"2024-09-13T08:53:15.619414+0000","last_active":"2024-09-13T08:53:16.614284+0000","last_peered":"2024-09-13T08:53:16.614284+0000","last_clean":"2024-09-13T08:53:16.614284+0000","last_became_active":"2024-09-13T08:53:15.619134+0000","last_became_peered":"2024-09-13T08:53:15.619134+0000","last_unstale":"2024-09-13T08:53:16.614284+0000","last_undegraded":"2024-09-13T08:53:16.614284+0000","last_fullsized":"2024-09-13T08:53:16.614284+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:42:20.746655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615760+0000","last_change":"2024-09-13T08:53:14.605915+0000","last_active":"2024-09-13T08:53:16.615760+0000","last_peered":"2024-09-13T08:53:16.615760+0000","last_clean":"2024-09-13T08:53:16.615760+0000","last_became_active":"2024-09-13T08:53:14.605660+0000","last_became_peered":"2024-09-13T08:53:14.605660+0000","last_unstale":"2024-09-13T08:53:16.615760+0000","last_undegraded":"2024-09-13T08:53:16.615760+0000","last_fullsized":"2024-09-13T08:53:16.615760+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:48.497980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615174+0000","last_change":"2024-09-13T08:53:15.619211+0000","last_active":"2024-09-13T08:53:16.615174+0000","last_peered":"2024-09-13T08:53:16.615174+0000","last_clean":"2024-09-13T08:53:16.615174+0000","last_became_active":"2024-09-13T08:53:15.617728+0000","last_became_peered":"2024-09-13T08:53:15.617728+0000","last_unstale":"2024-09-13T08:53:16.615174+0000","last_undegraded":"2024-09-13T08:53:16.615174+0000","last_fullsized":"2024-09-13T08:53:16.615174+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:19.945089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.623003+0000","last_change":"2024-09-13T08:53:14.606764+0000","last_active":"2024-09-13T08:53:16.623003+0000","last_peered":"2024-09-13T08:53:16.623003+0000","last_clean":"2024-09-13T08:53:16.623003+0000","last_became_active":"2024-09-13T08:53:14.606338+0000","last_became_peered":"2024-09-13T08:53:14.606338+0000","last_unstale":"2024-09-13T08:53:16.623003+0000","last_undegraded":"2024-09-13T08:53:16.623003+0000","last_fullsized":"2024-09-13T08:53:16.623003+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:10.459562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624817+0000","last_change":"2024-09-13T08:53:15.611512+0000","last_active":"2024-09-13T08:53:16.624817+0000","last_peered":"2024-09-13T08:53:16.624817+0000","last_clean":"2024-09-13T08:53:16.624817+0000","last_became_active":"2024-09-13T08:53:15.611194+0000","last_became_peered":"2024-09-13T08:53:15.611194+0000","last_unstale":"2024-09-13T08:53:16.624817+0000","last_undegraded":"2024-09-13T08:53:16.624817+0000","last_fullsized":"2024-09-13T08:53:16.624817+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:49.504775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626562+0000","last_change":"2024-09-13T08:53:14.608787+0000","last_active":"2024-09-13T08:53:16.626562+0000","last_peered":"2024-09-13T08:53:16.626562+0000","last_clean":"2024-09-13T08:53:16.626562+0000","last_became_active":"2024-09-13T08:53:14.608491+0000","last_became_peered":"2024-09-13T08:53:14.608491+0000","last_unstale":"2024-09-13T08:53:16.626562+0000","last_undegraded":"2024-09-13T08:53:16.626562+0000","last_fullsized":"2024-09-13T08:53:16.626562+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:48.181981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.624308+0000","last_change":"2024-09-13T08:53:15.618674+0000","last_active":"2024-09-13T08:53:16.624308+0000","last_peered":"2024-09-13T08:53:16.624308+0000","last_clean":"2024-09-13T08:53:16.624308+0000","last_became_active":"2024-09-13T08:53:15.618184+0000","last_became_peered":"2024-09-13T08:53:15.618184+0000","last_unstale":"2024-09-13T08:53:16.624308+0000","last_undegraded":"2024-09-13T08:53:16.624308+0000","last_fullsized":"2024-09-13T08:53:16.624308+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:14.950615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.613598+0000","last_change":"2024-09-13T08:53:14.600955+0000","last_active":"2024-09-13T08:53:16.613598+0000","last_peered":"2024-09-13T08:53:16.613598+0000","last_clean":"2024-09-13T08:53:16.613598+0000","last_became_active":"2024-09-13T08:53:14.600659+0000","last_became_peered":"2024-09-13T08:53:14.600659+0000","last_unstale":"2024-09-13T08:53:16.613598+0000","last_undegraded":"2024-09-13T08:53:16.613598+0000","last_fullsized":"2024-09-13T08:53:16.613598+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:17.207279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.626577+0000","last_change":"2024-09-13T08:53:15.608954+0000","last_active":"2024-09-13T08:53:16.626577+0000","last_peered":"2024-09-13T08:53:16.626577+0000","last_clean":"2024-09-13T08:53:16.626577+0000","last_became_active":"2024-09-13T08:53:15.608686+0000","last_became_peered":"2024-09-13T08:53:15.608686+0000","last_unstale":"2024-09-13T08:53:16.626577+0000","last_undegraded":"2024-09-13T08:53:16.626577+0000","last_fullsized":"2024-09-13T08:53:16.626577+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:55.415999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.129123+0000","last_change":"2024-09-13T08:53:14.610914+0000","last_active":"2024-09-13T08:53:17.129123+0000","last_peered":"2024-09-13T08:53:17.129123+0000","last_clean":"2024-09-13T08:53:17.129123+0000","last_became_active":"2024-09-13T08:53:14.610635+0000","last_became_peered":"2024-09-13T08:53:14.610635+0000","last_unstale":"2024-09-13T08:53:17.129123+0000","last_undegraded":"2024-09-13T08:53:17.129123+0000","last_fullsized":"2024-09-13T08:53:17.129123+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:04.024112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.615796+0000","last_change":"2024-09-13T08:53:15.608925+0000","last_active":"2024-09-13T08:53:16.615796+0000","last_peered":"2024-09-13T08:53:16.615796+0000","last_clean":"2024-09-13T08:53:16.615796+0000","last_became_active":"2024-09-13T08:53:15.608642+0000","last_became_peered":"2024-09-13T08:53:15.608642+0000","last_unstale":"2024-09-13T08:53:16.615796+0000","last_undegraded":"2024-09-13T08:53:16.615796+0000","last_fullsized":"2024-09-13T08:53:16.615796+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:14.580538+0000","last_clean_scrub_stamp":"2024-09-13T08:53:14.580538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:41:02.010094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"143'3","reported_seq":18,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:16.614124+0000","last_change":"2024-09-13T08:53:14.605379+0000","last_active":"2024-09-13T08:53:16.614124+0000","last_peered":"2024-09-13T08:53:16.614124+0000","last_clean":"2024-09-13T08:53:16.614124+0000","last_became_active":"2024-09-13T08:53:14.605066+0000","last_became_peered":"2024-09-13T08:53:14.605066+0000","last_unstale":"2024-09-13T08:53:16.614124+0000","last_undegraded":"2024-09-13T08:53:16.614124+0000","last_fullsized":"2024-09-13T08:53:16.614124+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:53:13.573742+0000","last_clean_scrub_stamp":"2024-09-13T08:53:13.573742+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:47:18.799221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":346,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-13T08:53:17.126624+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:53:17.126624+0000","last_peered":"2024-09-13T08:53:17.126624+0000","last_clean":"2024-09-13T08:53:17.126624+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:53:17.126624+0000","last_undegraded":"2024-09-13T08:53:17.126624+0000","last_fullsized":"2024-09-13T08:53:17.126624+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509633,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127280,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244560,"statfs":{"total":96636764160,"available":96506429440,"internally_reserved":0,"allocated":1794048,"data_stored":794783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509633,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119568,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252272,"statfs":{"total":96636764160,"available":96514326528,"internally_reserved":0,"allocated":2412544,"data_stored":1386957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509634,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137028,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234812,"statfs":{"total":96636764160,"available":96496447488,"internally_reserved":0,"allocated":1814528,"data_stored":796999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509634,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131444,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240396,"statfs":{"total":96636764160,"available":96502165504,"internally_reserved":0,"allocated":1802240,"data_stored":795399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509633,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131444,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240396,"statfs":{"total":96636764160,"available":96502165504,"internally_reserved":0,"allocated":1798144,"data_stored":794805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509635,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145912,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225928,"statfs":{"total":96636764160,"available":96487350272,"internally_reserved":0,"allocated":2392064,"data_stored":1385221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509633,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149492,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222348,"statfs":{"total":96636764160,"available":96483684352,"internally_reserved":0,"allocated":1798144,"data_stored":794873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542339,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141168,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230672,"statfs":{"total":96636764160,"available":96492208128,"internally_reserved":0,"allocated":1794048,"data_stored":794783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:53:24.771 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:53:24.772 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T08:53:24.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.963+0000 7f4e29800640 1 -- 172.21.15.44:0/2889338640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240b1650 msgr2=0x7f4e240b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:24.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.963+0000 7f4e29800640 1 --2- 172.21.15.44:0/2889338640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240b1650 0x7f4e240b1a30 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009900 tx=0x7f4e0c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.963+0000 7f4e29800640 1 -- 172.21.15.44:0/2889338640 shutdown_connections 2024-09-13T08:53:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.963+0000 7f4e29800640 1 --2- 172.21.15.44:0/2889338640 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e240afb10 0x7f4e240affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.963+0000 7f4e29800640 1 --2- 172.21.15.44:0/2889338640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240b1650 0x7f4e240b1a30 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.963+0000 7f4e29800640 1 -- 172.21.15.44:0/2889338640 >> 172.21.15.44:0/2889338640 conn(0x7f4e240d1dd0 msgr2=0x7f4e240d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.963+0000 7f4e29800640 1 -- 172.21.15.44:0/2889338640 shutdown_connections 2024-09-13T08:53:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.963+0000 7f4e29800640 1 -- 172.21.15.44:0/2889338640 wait complete. 2024-09-13T08:53:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.963+0000 7f4e29800640 1 Processor -- start 2024-09-13T08:53:24.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.963+0000 7f4e29800640 1 -- start start 2024-09-13T08:53:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e29800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e240afb10 0x7f4e24157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e29800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240b1650 0x7f4e240cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e29800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e241605d0 0x7f4e240ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e29800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e240b0ec0 con 0x7f4e240b1650 2024-09-13T08:53:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e29800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e240b1430 con 0x7f4e241605d0 2024-09-13T08:53:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e29800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4e24169540 con 0x7f4e240afb10 2024-09-13T08:53:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240b1650 0x7f4e240cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240b1650 0x7f4e240cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42124/0 (socket says 172.21.15.44:42124) 2024-09-13T08:53:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1be00640 1 -- 172.21.15.44:0/2395517285 learned_addr learned my addr 172.21.15.44:0/2395517285 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1be00640 1 -- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e240afb10 msgr2=0x7f4e24157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1be00640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e240afb10 0x7f4e24157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1be00640 1 -- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e241605d0 msgr2=0x7f4e240ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1be00640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e241605d0 0x7f4e240ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1be00640 1 -- 172.21.15.44:0/2395517285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e24164ed0 con 0x7f4e240b1650 2024-09-13T08:53:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1be00640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240b1650 0x7f4e240cca10 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009850 tx=0x7f4e0c0a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1aa00640 1 -- 172.21.15.44:0/2395517285 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c0b2040 con 0x7f4e240b1650 2024-09-13T08:53:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1aa00640 1 -- 172.21.15.44:0/2395517285 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e0c004340 con 0x7f4e240b1650 2024-09-13T08:53:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1aa00640 1 -- 172.21.15.44:0/2395517285 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c005280 con 0x7f4e240b1650 2024-09-13T08:53:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e29800640 1 -- 172.21.15.44:0/2395517285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e241f7d10 con 0x7f4e240b1650 2024-09-13T08:53:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e29800640 1 -- 172.21.15.44:0/2395517285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e241f82d0 con 0x7f4e240b1650 2024-09-13T08:53:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e29800640 1 -- 172.21.15.44:0/2395517285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e240affc0 con 0x7f4e240b1650 2024-09-13T08:53:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.967+0000 7f4e1aa00640 1 -- 172.21.15.44:0/2395517285 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f4e0c005420 con 0x7f4e240b1650 2024-09-13T08:53:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.971+0000 7f4e1aa00640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e00075110 0x7f4e000775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.971+0000 7f4e1aa00640 1 -- 172.21.15.44:0/2395517285 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f4e0c12cb40 con 0x7f4e240b1650 2024-09-13T08:53:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.971+0000 7f4e22a00640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e00075110 0x7f4e000775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.971+0000 7f4e22a00640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e00075110 0x7f4e000775d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4e2415bfa0 tx=0x7f4e14006bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:24.975+0000 7f4e1aa00640 1 -- 172.21.15.44:0/2395517285 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4e0c0164d0 con 0x7f4e240b1650 2024-09-13T08:53:25.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.219+0000 7f4e29800640 1 -- 172.21.15.44:0/2395517285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4e2415a860 con 0x7f4e240b1650 2024-09-13T08:53:25.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.219+0000 7f4e1aa00640 1 -- 172.21.15.44:0/2395517285 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f4e0c0f9890 con 0x7f4e240b1650 2024-09-13T08:53:25.223 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T08:53:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCR+eNm8340HxAAkMJCmCVqwBD+qY5VfXwnDA== 2024-09-13T08:53:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-09-13T08:53:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-09-13T08:53:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-13T08:53:25.223 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-09-13T08:53:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 -- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e00075110 msgr2=0x7f4e000775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e00075110 0x7f4e000775d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4e2415bfa0 tx=0x7f4e14006bd0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 -- 172.21.15.44:0/2395517285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240b1650 msgr2=0x7f4e240cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240b1650 0x7f4e240cca10 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009850 tx=0x7f4e0c0a5630 comp rx=0 tx=0).stop 2024-09-13T08:53:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 -- 172.21.15.44:0/2395517285 shutdown_connections 2024-09-13T08:53:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e00075110 0x7f4e000775d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e241605d0 0x7f4e240ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240b1650 0x7f4e240cca10 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 --2- 172.21.15.44:0/2395517285 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e240afb10 0x7f4e24157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 -- 172.21.15.44:0/2395517285 >> 172.21.15.44:0/2395517285 conn(0x7f4e240d1dd0 msgr2=0x7f4e24071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 -- 172.21.15.44:0/2395517285 shutdown_connections 2024-09-13T08:53:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.223+0000 7f4e29800640 1 -- 172.21.15.44:0/2395517285 wait complete. 2024-09-13T08:53:25.245 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:53:25.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 -- 172.21.15.44:0/1876079506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f705415bea0 msgr2=0x7f70540ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:25.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 --2- 172.21.15.44:0/1876079506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f705415bea0 0x7f70540ce670 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f70540745d0 tx=0x7f704401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 -- 172.21.15.44:0/1876079506 shutdown_connections 2024-09-13T08:53:25.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 --2- 172.21.15.44:0/1876079506 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70540d28e0 0x7f70540cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 --2- 172.21.15.44:0/1876079506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f705415bea0 0x7f70540ce670 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 -- 172.21.15.44:0/1876079506 >> 172.21.15.44:0/1876079506 conn(0x7f70540d1dc0 msgr2=0x7f70540d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:25.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 -- 172.21.15.44:0/1876079506 shutdown_connections 2024-09-13T08:53:25.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 -- 172.21.15.44:0/1876079506 wait complete. 2024-09-13T08:53:25.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 Processor -- start 2024-09-13T08:53:25.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 -- start start 2024-09-13T08:53:25.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70540cdb30 0x7f70541f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:25.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70540d28e0 0x7f70540cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:25.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f705415bea0 0x7f70540cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:25.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7054169050 con 0x7f70540cdb30 2024-09-13T08:53:25.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7054168ed0 con 0x7f705415bea0 2024-09-13T08:53:25.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f705ae00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f70541691d0 con 0x7f70540d28e0 2024-09-13T08:53:25.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f7053e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70540cdb30 0x7f70541f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f7053e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70540cdb30 0x7f70541f96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42146/0 (socket says 172.21.15.44:42146) 2024-09-13T08:53:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f7053e00640 1 -- 172.21.15.44:0/655823291 learned_addr learned my addr 172.21.15.44:0/655823291 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f704be00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70540d28e0 0x7f70540cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f7058e00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f705415bea0 0x7f70540cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f7053e00640 1 -- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70540d28e0 msgr2=0x7f70540cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f7053e00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70540d28e0 0x7f70540cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f7053e00640 1 -- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f705415bea0 msgr2=0x7f70540cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f7053e00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f705415bea0 0x7f70540cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.391+0000 7f7053e00640 1 -- 172.21.15.44:0/655823291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70541552a0 con 0x7f70540cdb30 2024-09-13T08:53:25.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.395+0000 7f7053e00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70540cdb30 0x7f70541f96e0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f70440b0040 tx=0x7f7044005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:25.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.395+0000 7f7050c00640 1 -- 172.21.15.44:0/655823291 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70440b2040 con 0x7f70540cdb30 2024-09-13T08:53:25.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.395+0000 7f7050c00640 1 -- 172.21.15.44:0/655823291 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70440044b0 con 0x7f70540cdb30 2024-09-13T08:53:25.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.395+0000 7f705ae00640 1 -- 172.21.15.44:0/655823291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70541433c0 con 0x7f70540cdb30 2024-09-13T08:53:25.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.395+0000 7f7050c00640 1 -- 172.21.15.44:0/655823291 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044003860 con 0x7f70540cdb30 2024-09-13T08:53:25.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.395+0000 7f705ae00640 1 -- 172.21.15.44:0/655823291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7054143980 con 0x7f70540cdb30 2024-09-13T08:53:25.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.395+0000 7f705ae00640 1 -- 172.21.15.44:0/655823291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70540cc9d0 con 0x7f70540cdb30 2024-09-13T08:53:25.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.395+0000 7f7050c00640 1 -- 172.21.15.44:0/655823291 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f7044003a00 con 0x7f70540cdb30 2024-09-13T08:53:25.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.399+0000 7f7050c00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7030075110 0x7f70300775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:25.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.399+0000 7f7050c00640 1 -- 172.21.15.44:0/655823291 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f704412cda0 con 0x7f70540cdb30 2024-09-13T08:53:25.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.399+0000 7f704be00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7030075110 0x7f70300775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:25.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.399+0000 7f704be00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7030075110 0x7f70300775d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7054159670 tx=0x7f703c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:25.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.403+0000 7f7050c00640 1 -- 172.21.15.44:0/655823291 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f70440bd060 con 0x7f70540cdb30 2024-09-13T08:53:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 -- 172.21.15.44:0/655823291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70540b7600 con 0x7f70540cdb30 2024-09-13T08:53:25.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f7050c00640 1 -- 172.21.15.44:0/655823291 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5230 (secure 0 0 0) 0x7f70440164d0 con 0x7f70540cdb30 2024-09-13T08:53:25.613 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:25.613 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":95,"btime":"2024-09-13T08:53:18:619363+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/766502733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":766502733},{"type":"v1","addr":"172.21.15.129:6837","nonce":766502733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2074722286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2074722286},{"type":"v1","addr":"172.21.15.129:6835","nonce":2074722286}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3065267295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3065267295},{"type":"v1","addr":"172.21.15.44:6834","nonce":3065267295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:53:15.007787+0000","modified":"2024-09-13T08:53:18.026905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9219":{"gid":9219,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3418956320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3418956320},{"type":"v1","addr":"172.21.15.44:6835","nonce":3418956320}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9219,"qdb_cluster":[9219]},"id":10}]} 2024-09-13T08:53:25.613 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 95 2024-09-13T08:53:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 -- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7030075110 msgr2=0x7f70300775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7030075110 0x7f70300775d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7054159670 tx=0x7f703c009040 comp rx=0 tx=0).stop 2024-09-13T08:53:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 -- 172.21.15.44:0/655823291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70540cdb30 msgr2=0x7f70541f96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70540cdb30 0x7f70541f96e0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f70440b0040 tx=0x7f7044005770 comp rx=0 tx=0).stop 2024-09-13T08:53:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 -- 172.21.15.44:0/655823291 shutdown_connections 2024-09-13T08:53:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7030075110 0x7f70300775d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f705415bea0 0x7f70540cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f70540d28e0 0x7f70540cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 --2- 172.21.15.44:0/655823291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70540cdb30 0x7f70541f96e0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 -- 172.21.15.44:0/655823291 >> 172.21.15.44:0/655823291 conn(0x7f70540d1dc0 msgr2=0x7f70540cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.611+0000 7f705ae00640 1 -- 172.21.15.44:0/655823291 shutdown_connections 2024-09-13T08:53:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:25.615+0000 7f705ae00640 1 -- 172.21.15.44:0/655823291 wait complete. 2024-09-13T08:53:26.038 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T08:53:15.007787+0000', 'modified': '2024-09-13T08:53:18.026905+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9219}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9219': {'gid': 9219, 'name': 'c', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/3418956320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 3418956320}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 3418956320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9219, 'qdb_cluster': [9219]} 2024-09-13T08:53:26.038 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T08:53:26.039 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T08:53:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.191+0000 7fd29b600640 1 -- 172.21.15.44:0/1183959323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd29415f160 msgr2=0x7fd2940cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.191+0000 7fd29b600640 1 --2- 172.21.15.44:0/1183959323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd29415f160 0x7fd2940cef20 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd28801c8e0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.191+0000 7fd29b600640 1 -- 172.21.15.44:0/1183959323 shutdown_connections 2024-09-13T08:53:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.191+0000 7fd29b600640 1 --2- 172.21.15.44:0/1183959323 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd29406f930 0x7fd29406fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.191+0000 7fd29b600640 1 --2- 172.21.15.44:0/1183959323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd29415f160 0x7fd2940cef20 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.191+0000 7fd29b600640 1 -- 172.21.15.44:0/1183959323 >> 172.21.15.44:0/1183959323 conn(0x7fd2940b2240 msgr2=0x7fd2940b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.191+0000 7fd29b600640 1 -- 172.21.15.44:0/1183959323 shutdown_connections 2024-09-13T08:53:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.191+0000 7fd29b600640 1 -- 172.21.15.44:0/1183959323 wait complete. 2024-09-13T08:53:26.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd29b600640 1 Processor -- start 2024-09-13T08:53:26.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd29b600640 1 -- start start 2024-09-13T08:53:26.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd29b600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd29406cc20 0x7fd294154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd29b600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd29406f930 0x7fd2941f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd29b600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd29415f160 0x7fd29415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd29b600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd29416b800 con 0x7fd29406cc20 2024-09-13T08:53:26.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd29b600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd29416b980 con 0x7fd29415f160 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd29b600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd29416bb00 con 0x7fd29406f930 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd298c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd29406cc20 0x7fd294154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd298c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd29406cc20 0x7fd294154ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42164/0 (socket says 172.21.15.44:42164) 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd298c00640 1 -- 172.21.15.44:0/2803482340 learned_addr learned my addr 172.21.15.44:0/2803482340 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd299600640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd29415f160 0x7fd29415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd298c00640 1 -- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd29406f930 msgr2=0x7fd2941f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd298c00640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd29406f930 0x7fd2941f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd298c00640 1 -- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd29415f160 msgr2=0x7fd29415fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd298c00640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd29415f160 0x7fd29415fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd298c00640 1 -- 172.21.15.44:0/2803482340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2941f8ec0 con 0x7fd29406cc20 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd299600640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd29415f160 0x7fd29415fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd298c00640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd29406cc20 0x7fd294154ee0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd2880b0040 tx=0x7fd2880a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd290c00640 1 -- 172.21.15.44:0/2803482340 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2880b2040 con 0x7fd29406cc20 2024-09-13T08:53:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd290c00640 1 -- 172.21.15.44:0/2803482340 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd288004430 con 0x7fd29406cc20 2024-09-13T08:53:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd29b600640 1 -- 172.21.15.44:0/2803482340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2941f91a0 con 0x7fd29406cc20 2024-09-13T08:53:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd290c00640 1 -- 172.21.15.44:0/2803482340 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288003860 con 0x7fd29406cc20 2024-09-13T08:53:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.195+0000 7fd29b600640 1 -- 172.21.15.44:0/2803482340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd294155420 con 0x7fd29406cc20 2024-09-13T08:53:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd290c00640 1 -- 172.21.15.44:0/2803482340 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7fd2880bf060 con 0x7fd29406cc20 2024-09-13T08:53:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd29ea2e000 1 Processor -- start 2024-09-13T08:53:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd290c00640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd274075110 0x7fd2740775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd29ea2e000 1 -- start start 2024-09-13T08:53:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd290c00640 1 -- 172.21.15.44:0/2803482340 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fd28812cad0 con 0x7fd29406cc20 2024-09-13T08:53:26.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd299600640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd274075110 0x7fd2740775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 0x55778cb31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb322f0 0x55778cae58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cae61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55778ca063f0 con 0x55778cae5e10 2024-09-13T08:53:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55778c943b20 con 0x55778cb319d0 2024-09-13T08:53:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55778c932f50 con 0x55778cb322f0 2024-09-13T08:53:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd299600640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd274075110 0x7fd2740775d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd28c007fd0 tx=0x7fd28c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd299600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cae61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd299600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cae61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42168/0 (socket says 172.21.15.44:42168) 2024-09-13T08:53:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd299600640 1 -- 172.21.15.44:0/2362002400 learned_addr learned my addr 172.21.15.44:0/2362002400 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd293e00640 1 --2- 172.21.15.44:0/2362002400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb322f0 0x55778cae58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.199+0000 7fd298c00640 1 --2- 172.21.15.44:0/2362002400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 0x55778cb31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd299600640 1 -- 172.21.15.44:0/2362002400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb322f0 msgr2=0x55778cae58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd299600640 1 --2- 172.21.15.44:0/2362002400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb322f0 0x55778cae58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd299600640 1 -- 172.21.15.44:0/2362002400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 msgr2=0x55778cb31db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd299600640 1 --2- 172.21.15.44:0/2362002400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 0x55778cb31db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd299600640 1 -- 172.21.15.44:0/2362002400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778c8fa590 con 0x55778cae5e10 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd299600640 1 --2- 172.21.15.44:0/2362002400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cae61f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00dc60 tx=0x7fd28c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1ccdb17397a40c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd285600640 1 -- 172.21.15.44:0/2362002400 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c024070 con 0x55778cae5e10 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd285600640 1 -- 172.21.15.44:0/2362002400 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd28c01f600 con 0x55778cae5e10 2024-09-13T08:53:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd285600640 1 -- 172.21.15.44:0/2362002400 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c01f9d0 con 0x55778cae5e10 2024-09-13T08:53:26.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2362002400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 msgr2=0x55778cae61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/2362002400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cae61f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00dc60 tx=0x7fd28c010000 comp rx=0 tx=0).stop 2024-09-13T08:53:26.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2362002400 shutdown_connections 2024-09-13T08:53:26.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/2362002400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cae61f0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/2362002400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb322f0 0x55778cae58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/2362002400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 0x55778cb31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2362002400 >> 172.21.15.44:0/2362002400 conn(0x55778c9cf130 msgr2=0x55778cb14e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2362002400 shutdown_connections 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2362002400 wait complete. 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 -- start start 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 Processor -- start 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cae5e10 0x55778cb126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 0x55778cb12e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cb133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55778c8cd5d0 con 0x55778cb322f0 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55778c936d60 con 0x55778cae5e10 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55778c8d9500 con 0x55778cb319d0 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd29ea2e000 10 client.0 _ll_register_callbacks cb 0x55778c8dd810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd299600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cb133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd299600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cb133b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42172/0 (socket says 172.21.15.44:42172) 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd299600640 1 -- 172.21.15.44:0/2074965514 learned_addr learned my addr 172.21.15.44:0/2074965514 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd293e00640 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 0x55778cb12e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.203+0000 7fd298c00640 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cae5e10 0x55778cb126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd299600640 1 -- 172.21.15.44:0/2074965514 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 msgr2=0x55778cb12e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd299600640 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 0x55778cb12e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd299600640 1 -- 172.21.15.44:0/2074965514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cae5e10 msgr2=0x55778cb126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd299600640 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cae5e10 0x55778cb126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd299600640 1 -- 172.21.15.44:0/2074965514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778c8d1170 con 0x55778cb322f0 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd299600640 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cb133b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x55778c8e4130 tx=0x7fd28c006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b928bd2ade967a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd27a000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd27a000640 1 -- 172.21.15.44:0/2074965514 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c02c020 con 0x55778cb322f0 2024-09-13T08:53:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd27a000640 1 -- 172.21.15.44:0/2074965514 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fd28c007240 con 0x55778cb322f0 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2074965514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55778c8cb0f0 con 0x55778cb322f0 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd27a000640 1 -- 172.21.15.44:0/2074965514 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fd28c036050 con 0x55778cb322f0 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 10 client.9372 fetch_fsmap learned FSMap version 95 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2074965514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x55778c9b7ef0 con 0x55778cb322f0 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd27a000640 1 -- 172.21.15.44:0/2074965514 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55778c8cb0f0 con 0x55778cb322f0 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd27a000640 1 -- 172.21.15.44:0/2074965514 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 95) ==== 3502+0+0 (secure 0 0 0) 0x7fd28c039020 con 0x55778cb322f0 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 10 client.9372 fetch_fsmap finished waiting for FSMap version 95 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 10 client.9372 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 20 client.9372 populate_metadata read hostname 'smithi044' 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.44:6833/3418956320,v1:172.21.15.44:6835/3418956320] conn(0x55778cb140f0 0x55778cb43c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 4 client.9372 mds_command: new command op to 9219 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2074965514 --> [v2:172.21.15.44:6833/3418956320,v1:172.21.15.44:6835/3418956320] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55778c945a70 con 0x55778cb140f0 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd298c00640 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.44:6833/3418956320,v1:172.21.15.44:6835/3418956320] conn(0x55778cb140f0 0x55778cb43c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd298c00640 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.44:6833/3418956320,v1:172.21.15.44:6835/3418956320] conn(0x55778cb140f0 0x55778cb43c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace66a0e62e2d23c server_cookie=baa7ef4d6402a18e in_seq=0 out_seq=0 2024-09-13T08:53:26.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd27a000640 10 client.9372 ms_handle_connect on v2:172.21.15.44:6833/3418956320 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd27a000640 1 -- 172.21.15.44:0/2074965514 <== mds.0 v2:172.21.15.44:6833/3418956320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55778c945a70 con 0x55778cb140f0 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd27a000640 10 client.9372 handle_command_reply: tid=0 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 client.9372 shutdown 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2074965514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 msgr2=0x55778cb133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cb133b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x55778c8e4130 tx=0x7fd28c006e10 comp rx=0 tx=0).stop 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2074965514 shutdown_connections 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.44:6833/3418956320,v1:172.21.15.44:6835/3418956320] conn(0x55778cb140f0 0x55778cb43c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cb133b0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 0x55778cb12e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/2074965514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cae5e10 0x55778cb126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2074965514 >> 172.21.15.44:0/2074965514 conn(0x55778c9cf130 msgr2=0x55778cae4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:26.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd27a000640 0 client.9372 ms_handle_reset on v2:172.21.15.44:6833/3418956320 2024-09-13T08:53:26.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2074965514 shutdown_connections 2024-09-13T08:53:26.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 1 -- 172.21.15.44:0/2074965514 wait complete. 2024-09-13T08:53:26.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.207+0000 7fd29ea2e000 20 client.9372 trim_cache size 0 max 16384 2024-09-13T08:53:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd29ea2e000 1 Processor -- start 2024-09-13T08:53:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd29ea2e000 1 -- start start 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cae5e10 0x55778cba39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 0x55778cba3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cba4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd294204710 con 0x55778cb322f0 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941578b0 con 0x55778cb319d0 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd294146480 con 0x55778cae5e10 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd299600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cba4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd299600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cba4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42174/0 (socket says 172.21.15.44:42174) 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd299600640 1 -- 172.21.15.44:0/367387684 learned_addr learned my addr 172.21.15.44:0/367387684 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd298c00640 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cae5e10 0x55778cba39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd293e00640 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 0x55778cba3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd299600640 1 -- 172.21.15.44:0/367387684 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cae5e10 msgr2=0x55778cba39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:53:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd299600640 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cae5e10 0x55778cba39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd299600640 1 -- 172.21.15.44:0/367387684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 msgr2=0x55778cba3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:53:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd299600640 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 0x55778cba3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd299600640 1 -- 172.21.15.44:0/367387684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778c98ed90 con 0x55778cb322f0 2024-09-13T08:53:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd293e00640 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 0x55778cba3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.227+0000 7fd298c00640 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cae5e10 0x55778cba39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd299600640 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cba4430 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00d8a0 tx=0x7fd28c00dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d00c0df85265f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd284c00640 1 -- 172.21.15.44:0/367387684 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c03f070 con 0x55778cb322f0 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd284c00640 1 -- 172.21.15.44:0/367387684 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd28c006e80 con 0x55778cb322f0 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd284c00640 1 -- 172.21.15.44:0/367387684 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c02c040 con 0x55778cb322f0 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 -- 172.21.15.44:0/367387684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 msgr2=0x55778cba4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cba4430 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00d8a0 tx=0x7fd28c00dea0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 -- 172.21.15.44:0/367387684 shutdown_connections 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cb322f0 0x55778cba4430 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb319d0 0x55778cba3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/367387684 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cae5e10 0x55778cba39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 -- 172.21.15.44:0/367387684 >> 172.21.15.44:0/367387684 conn(0x55778c9cf130 msgr2=0x55778caedad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 -- 172.21.15.44:0/367387684 shutdown_connections 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 -- 172.21.15.44:0/367387684 wait complete. 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 -- start start 2024-09-13T08:53:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 Processor -- start 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cba7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 0x55778cba7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb322f0 0x55778cba80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55778c8feec0 con 0x55778cae5e10 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd294157260 con 0x55778cb322f0 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x55778c8cd5d0 con 0x55778cb319d0 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd29ea2e000 10 client.0 _ll_register_callbacks cb 0x7fd28c022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd298c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cba7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd298c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cba7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42186/0 (socket says 172.21.15.44:42186) 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd298c00640 1 -- 172.21.15.44:0/1157191235 learned_addr learned my addr 172.21.15.44:0/1157191235 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd299600640 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb322f0 0x55778cba80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd293e00640 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 0x55778cba7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd298c00640 1 -- 172.21.15.44:0/1157191235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 msgr2=0x55778cba7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:53:26.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd298c00640 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 0x55778cba7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd298c00640 1 -- 172.21.15.44:0/1157191235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb322f0 msgr2=0x55778cba80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T08:53:26.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd298c00640 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb322f0 0x55778cba80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd298c00640 1 -- 172.21.15.44:0/1157191235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778c8f9a90 con 0x55778cae5e10 2024-09-13T08:53:26.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd293e00640 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 0x55778cba7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:26.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd298c00640 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cba7640 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd2880f9010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a148f8aaf7ff28 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:26.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd27a000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-09-13T08:53:26.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd27a000640 1 -- 172.21.15.44:0/1157191235 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2880f9120 con 0x55778cae5e10 2024-09-13T08:53:26.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.231+0000 7fd27a000640 1 -- 172.21.15.44:0/1157191235 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fd2880f5390 con 0x55778cae5e10 2024-09-13T08:53:26.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 -- 172.21.15.44:0/1157191235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd28c006e80 con 0x55778cae5e10 2024-09-13T08:53:26.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd27a000640 1 -- 172.21.15.44:0/1157191235 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fd2880fdc00 con 0x55778cae5e10 2024-09-13T08:53:26.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 10 client.9378 fetch_fsmap learned FSMap version 95 2024-09-13T08:53:26.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 -- 172.21.15.44:0/1157191235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x55778c8d8050 con 0x55778cae5e10 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd27a000640 1 -- 172.21.15.44:0/1157191235 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd28c006e80 con 0x55778cae5e10 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd27a000640 1 -- 172.21.15.44:0/1157191235 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 95) ==== 3502+0+0 (secure 0 0 0) 0x7fd288113050 con 0x55778cae5e10 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 10 client.9378 fetch_fsmap finished waiting for FSMap version 95 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 10 client.9378 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 20 client.9378 populate_metadata read hostname 'smithi044' 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.44:6833/3418956320,v1:172.21.15.44:6835/3418956320] conn(0x55778cb45640 0x55778cb45a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 4 client.9378 mds_command: new command op to 9219 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 -- 172.21.15.44:0/1157191235 --> [v2:172.21.15.44:6833/3418956320,v1:172.21.15.44:6835/3418956320] -- command(tid 0: {"prefix": "status"}) -- 0x55778c941520 con 0x55778cb45640 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd293e00640 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.44:6833/3418956320,v1:172.21.15.44:6835/3418956320] conn(0x55778cb45640 0x55778cb45a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd293e00640 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.44:6833/3418956320,v1:172.21.15.44:6835/3418956320] conn(0x55778cb45640 0x55778cb45a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db18ec9916514ea server_cookie=3d70e9b80a3dbb28 in_seq=0 out_seq=0 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd27a000640 10 client.9378 ms_handle_connect on v2:172.21.15.44:6833/3418956320 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd27a000640 1 -- 172.21.15.44:0/1157191235 <== mds.0 v2:172.21.15.44:6833/3418956320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55778c941520 con 0x55778cb45640 2024-09-13T08:53:26.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd27a000640 10 client.9378 handle_command_reply: tid=0 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 client.9378 shutdown 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 -- 172.21.15.44:0/1157191235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 msgr2=0x55778cba7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cba7640 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd2880f9010 comp rx=0 tx=0).stop 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 -- 172.21.15.44:0/1157191235 shutdown_connections 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.44:6833/3418956320,v1:172.21.15.44:6835/3418956320] conn(0x55778cb45640 0x55778cb45a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55778cb322f0 0x55778cba80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55778cb319d0 0x55778cba7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd27a000640 0 client.9378 ms_handle_reset on v2:172.21.15.44:6833/3418956320 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 --2- 172.21.15.44:0/1157191235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778cae5e10 0x55778cba7640 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 -- 172.21.15.44:0/1157191235 >> 172.21.15.44:0/1157191235 conn(0x55778c9cf130 msgr2=0x55778cb301d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 -- 172.21.15.44:0/1157191235 shutdown_connections 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 1 -- 172.21.15.44:0/1157191235 wait complete. 2024-09-13T08:53:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29ea2e000 20 client.9378 trim_cache size 0 max 16384 2024-09-13T08:53:26.242 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:53:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:53:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-09-13T08:53:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 9219, 2024-09-13T08:53:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-09-13T08:53:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-09-13T08:53:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-09-13T08:53:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 10.635407364000001, 2024-09-13T08:53:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 95, 2024-09-13T08:53:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 144, 2024-09-13T08:53:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 144, 2024-09-13T08:53:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 15.124899439 2024-09-13T08:53:26.243 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:53:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 -- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd274075110 msgr2=0x7fd2740775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd274075110 0x7fd2740775d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd28c007fd0 tx=0x7fd28c009040 comp rx=0 tx=0).stop 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 -- 172.21.15.44:0/2803482340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd29406cc20 msgr2=0x7fd294154ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd29406cc20 0x7fd294154ee0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd2880b0040 tx=0x7fd2880a56e0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 -- 172.21.15.44:0/2803482340 shutdown_connections 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd274075110 0x7fd2740775d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd29415f160 0x7fd29415fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd29406f930 0x7fd2941f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 --2- 172.21.15.44:0/2803482340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd29406cc20 0x7fd294154ee0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 -- 172.21.15.44:0/2803482340 >> 172.21.15.44:0/2803482340 conn(0x7fd2940b2240 msgr2=0x7fd2940b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 -- 172.21.15.44:0/2803482340 shutdown_connections 2024-09-13T08:53:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.235+0000 7fd29b600640 1 -- 172.21.15.44:0/2803482340 wait complete. 2024-09-13T08:53:26.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T08:53:26.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.399+0000 7f1b07a00640 1 -- 172.21.15.44:0/662885118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b0015dae0 msgr2=0x7f1b000acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.399+0000 7f1b07a00640 1 --2- 172.21.15.44:0/662885118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b0015dae0 0x7f1b000acd40 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009900 tx=0x7f1af401c7c0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 -- 172.21.15.44:0/662885118 shutdown_connections 2024-09-13T08:53:26.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 --2- 172.21.15.44:0/662885118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b0015dae0 0x7f1b000acd40 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 -- 172.21.15.44:0/662885118 >> 172.21.15.44:0/662885118 conn(0x7f1b000b18f0 msgr2=0x7f1b000b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:26.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 -- 172.21.15.44:0/662885118 shutdown_connections 2024-09-13T08:53:26.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 -- 172.21.15.44:0/662885118 wait complete. 2024-09-13T08:53:26.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 Processor -- start 2024-09-13T08:53:26.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 -- start start 2024-09-13T08:53:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b000b49e0 0x7f1b001f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b000cd100 0x7f1b0009d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b0015dae0 0x7f1b0009d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00168e90 con 0x7f1b000b49e0 2024-09-13T08:53:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00168d10 con 0x7f1b000cd100 2024-09-13T08:53:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b07a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1b00169010 con 0x7f1b0015dae0 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b000b49e0 0x7f1b001f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b000b49e0 0x7f1b001f7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42204/0 (socket says 172.21.15.44:42204) 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05000640 1 -- 172.21.15.44:0/734038768 learned_addr learned my addr 172.21.15.44:0/734038768 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05a00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b0015dae0 0x7f1b0009d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1affe00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b000cd100 0x7f1b0009d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05000640 1 -- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b0015dae0 msgr2=0x7f1b0009d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05000640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b0015dae0 0x7f1b0009d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05000640 1 -- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b000cd100 msgr2=0x7f1b0009d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05000640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b000cd100 0x7f1b0009d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05000640 1 -- 172.21.15.44:0/734038768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b000b0510 con 0x7f1b000b49e0 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05a00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b0015dae0 0x7f1b0009d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1affe00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b000cd100 0x7f1b0009d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.403+0000 7f1b05000640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b000b49e0 0x7f1b001f7850 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009850 tx=0x7f1af40a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.407+0000 7f1afcc00640 1 -- 172.21.15.44:0/734038768 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af40b2040 con 0x7f1b000b49e0 2024-09-13T08:53:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.407+0000 7f1afcc00640 1 -- 172.21.15.44:0/734038768 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1af4004430 con 0x7f1b000b49e0 2024-09-13T08:53:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.407+0000 7f1afcc00640 1 -- 172.21.15.44:0/734038768 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4003860 con 0x7f1b000b49e0 2024-09-13T08:53:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.407+0000 7f1b07a00640 1 -- 172.21.15.44:0/734038768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b000b07f0 con 0x7f1b000b49e0 2024-09-13T08:53:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.407+0000 7f1b07a00640 1 -- 172.21.15.44:0/734038768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b000ca800 con 0x7f1b000b49e0 2024-09-13T08:53:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.407+0000 7f1b07a00640 1 -- 172.21.15.44:0/734038768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b00069410 con 0x7f1b000b49e0 2024-09-13T08:53:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.407+0000 7f1afcc00640 1 -- 172.21.15.44:0/734038768 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f1af4003a00 con 0x7f1b000b49e0 2024-09-13T08:53:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.407+0000 7f1afcc00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1ae0075110 0x7f1ae00775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.407+0000 7f1afcc00640 1 -- 172.21.15.44:0/734038768 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f1af412c820 con 0x7f1b000b49e0 2024-09-13T08:53:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.407+0000 7f1affe00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1ae0075110 0x7f1ae00775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.411+0000 7f1affe00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1ae0075110 0x7f1ae00775d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1b0006ae80 tx=0x7f1aec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:26.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.411+0000 7f1afcc00640 1 -- 172.21.15.44:0/734038768 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1af40164d0 con 0x7f1b000b49e0 2024-09-13T08:53:26.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 -- 172.21.15.44:0/734038768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1b000acd40 con 0x7f1b000b49e0 2024-09-13T08:53:26.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1afcc00640 1 -- 172.21.15.44:0/734038768 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v95) ==== 53+0+73 (secure 0 0 0) 0x7f1af40f94d0 con 0x7f1b000b49e0 2024-09-13T08:53:26.628 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 -- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1ae0075110 msgr2=0x7f1ae00775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1ae0075110 0x7f1ae00775d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1b0006ae80 tx=0x7f1aec009040 comp rx=0 tx=0).stop 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 -- 172.21.15.44:0/734038768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b000b49e0 msgr2=0x7f1b001f7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b000b49e0 0x7f1b001f7850 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009850 tx=0x7f1af40a5630 comp rx=0 tx=0).stop 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 -- 172.21.15.44:0/734038768 shutdown_connections 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1ae0075110 0x7f1ae00775d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1b0015dae0 0x7f1b0009d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b000cd100 0x7f1b0009d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 --2- 172.21.15.44:0/734038768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b000b49e0 0x7f1b001f7850 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 -- 172.21.15.44:0/734038768 >> 172.21.15.44:0/734038768 conn(0x7f1b000b18f0 msgr2=0x7f1b000cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 -- 172.21.15.44:0/734038768 shutdown_connections 2024-09-13T08:53:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26.627+0000 7f1b07a00640 1 -- 172.21.15.44:0/734038768 wait complete. 2024-09-13T08:53:26.651 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T08:53:26.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T08:53:26.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-09-13T08:53:26.651 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T08:53:26.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T08:53:26.651 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T08:53:26.651 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-13T08:53:26.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T08:53:26.651 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:53:26.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:26.652 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-09-13T08:53:26.661 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T08:53:26.661 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T08:53:26.662 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41476sec preferred_lft 41476sec 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 02:42:8a:32:10:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:53:26.663 INFO:teuthology.orchestra.run.smithi044.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T08:53:26.664 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-09-13T08:53:26.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:26.667 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T08:53:26.720 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-09-13T08:53:26.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:26.723 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-09-13T08:53:26.777 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-09-13T08:53:26.778 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T08:53:26.778 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T08:53:26.778 INFO:teuthology.orchestra.run.smithi044.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T08:53:26.778 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T08:53:26.778 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T08:53:26.778 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-13T08:53:26.778 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-09-13T08:53:26.778 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T08:53:26.778 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T08:53:26.779 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T08:53:26.779 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-09-13T08:53:26.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-13T08:53:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:53:26.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:26.916 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:53:26.924 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:53:26.924 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T08:53:26.925 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:53:26.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:26.926 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-09-13T08:53:26.991 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:53:26.991 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T08:53:27.047 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T08:53:27.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T08:53:27.047 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:53:27.098 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-13T08:53:27.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:27.099 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:53:27.193 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:53:27.193 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:53:27.193 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:53:27.209 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:53:27.209 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:53:27.209 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:53:27.209 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:53:27.209 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:53:27.218 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-13T08:53:27.218 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-13T08:53:27.218 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-09-13T08:53:27.218 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.129:3300,172.21.15.129:3301" 2024-09-13T08:53:27.218 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-13T08:53:27.218 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790" 2024-09-13T08:53:27.218 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.129:6789,172.21.15.129:6790:/ 2024-09-13T08:53:27.219 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d85be7ac-239d-467b-bce8-dbc7fa81939e" will pass to kernel 2024-09-13T08:53:27.222 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-13T08:53:27.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:27.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T08:53:27.242 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-13T08:53:27.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:53:27.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:53:27.350 INFO:teuthology.orchestra.run.smithi044.stdout:client8605 2024-09-13T08:53:27.350 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:53:27.350 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:27.359 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-13T08:53:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.555+0000 7f9879400640 1 -- 172.21.15.44:0/3907028794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98740b0b60 msgr2=0x7f98740ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.555+0000 7f9879400640 1 --2- 172.21.15.44:0/3907028794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98740b0b60 0x7f98740ccbd0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f9874073e00 tx=0x7f986c01f580 comp rx=0 tx=0).stop 2024-09-13T08:53:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.555+0000 7f9879400640 1 -- 172.21.15.44:0/3907028794 shutdown_connections 2024-09-13T08:53:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.555+0000 7f9879400640 1 --2- 172.21.15.44:0/3907028794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98740b0b60 0x7f98740ccbd0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.555+0000 7f9879400640 1 --2- 172.21.15.44:0/3907028794 >> v2:172.21.15.129:3300/0 conn(0x7f987415be10 0x7f98740ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.555+0000 7f9879400640 1 -- 172.21.15.44:0/3907028794 >> 172.21.15.44:0/3907028794 conn(0x7f98740d1da0 msgr2=0x7f98740d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.555+0000 7f9879400640 1 -- 172.21.15.44:0/3907028794 shutdown_connections 2024-09-13T08:53:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.555+0000 7f9879400640 1 -- 172.21.15.44:0/3907028794 wait complete. 2024-09-13T08:53:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.555+0000 7f9879400640 1 Processor -- start 2024-09-13T08:53:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.555+0000 7f9879400640 1 -- start start 2024-09-13T08:53:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9879400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98740b0b60 0x7f98740ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9879400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f987415be10 0x7f98740c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9879400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f987415c1f0 0x7f98741f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9879400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98740cd290 con 0x7f987415c1f0 2024-09-13T08:53:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9879400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98740cd110 con 0x7f98740b0b60 2024-09-13T08:53:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9879400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f98740cd410 con 0x7f987415be10 2024-09-13T08:53:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9873200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f987415c1f0 0x7f98741f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9873200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f987415c1f0 0x7f98741f5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42208/0 (socket says 172.21.15.44:42208) 2024-09-13T08:53:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9873200640 1 -- 172.21.15.44:0/3491715967 learned_addr learned my addr 172.21.15.44:0/3491715967 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9872800640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98740b0b60 0x7f98740ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9869e00640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f987415be10 0x7f98740c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9873200640 1 -- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f987415be10 msgr2=0x7f98740c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9873200640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f987415be10 0x7f98740c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9873200640 1 -- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98740b0b60 msgr2=0x7f98740ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9873200640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98740b0b60 0x7f98740ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9873200640 1 -- 172.21.15.44:0/3491715967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98740cae70 con 0x7f987415c1f0 2024-09-13T08:53:27.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9873200640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f987415c1f0 0x7f98741f5e20 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f986c01c190 tx=0x7f986c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f986b400640 1 -- 172.21.15.44:0/3491715967 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c004e60 con 0x7f987415c1f0 2024-09-13T08:53:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f986b400640 1 -- 172.21.15.44:0/3491715967 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f986c0be030 con 0x7f987415c1f0 2024-09-13T08:53:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f986b400640 1 -- 172.21.15.44:0/3491715967 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c0afaa0 con 0x7f987415c1f0 2024-09-13T08:53:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9879400640 1 -- 172.21.15.44:0/3491715967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98740c5a00 con 0x7f987415c1f0 2024-09-13T08:53:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9879400640 1 -- 172.21.15.44:0/3491715967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98740c5fc0 con 0x7f987415c1f0 2024-09-13T08:53:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f9879400640 1 -- 172.21.15.44:0/3491715967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98740ccbd0 con 0x7f987415c1f0 2024-09-13T08:53:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.559+0000 7f986b400640 1 -- 172.21.15.44:0/3491715967 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f986c0afc40 con 0x7f987415c1f0 2024-09-13T08:53:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.563+0000 7f986b400640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9850075110 0x7f98500775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.563+0000 7f986b400640 1 -- 172.21.15.44:0/3491715967 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f986c131740 con 0x7f987415c1f0 2024-09-13T08:53:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.563+0000 7f9872800640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9850075110 0x7f98500775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:27.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.563+0000 7f9872800640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9850075110 0x7f98500775d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f98640058d0 tx=0x7f9864005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:27.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.567+0000 7f986b400640 1 -- 172.21.15.44:0/3491715967 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f986c0190e0 con 0x7f987415c1f0 2024-09-13T08:53:27.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.779+0000 7f9879400640 1 -- 172.21.15.44:0/3491715967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f9874070350 con 0x7f987415c1f0 2024-09-13T08:53:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.899+0000 7f986b400640 1 -- 172.21.15.44:0/3491715967 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v96) ==== 136+0+0 (secure 0 0 0) 0x7f986c0fe490 con 0x7f987415c1f0 2024-09-13T08:53:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 -- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9850075110 msgr2=0x7f98500775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9850075110 0x7f98500775d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f98640058d0 tx=0x7f9864005860 comp rx=0 tx=0).stop 2024-09-13T08:53:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 -- 172.21.15.44:0/3491715967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f987415c1f0 msgr2=0x7f98741f5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f987415c1f0 0x7f98741f5e20 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f986c01c190 tx=0x7f986c003f60 comp rx=0 tx=0).stop 2024-09-13T08:53:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 -- 172.21.15.44:0/3491715967 shutdown_connections 2024-09-13T08:53:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f9850075110 0x7f98500775d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f987415c1f0 0x7f98741f5e20 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:27.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f987415be10 0x7f98740c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:27.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 --2- 172.21.15.44:0/3491715967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98740b0b60 0x7f98740ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:27.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 -- 172.21.15.44:0/3491715967 >> 172.21.15.44:0/3491715967 conn(0x7f98740d1da0 msgr2=0x7f9874070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:27.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 -- 172.21.15.44:0/3491715967 shutdown_connections 2024-09-13T08:53:27.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:27.903+0000 7f9879400640 1 -- 172.21.15.44:0/3491715967 wait complete. 2024-09-13T08:53:27.925 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:53:28.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.071+0000 7f1fcb200640 1 -- 172.21.15.44:0/214884157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fc415e330 msgr2=0x7f1fc40b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.071+0000 7f1fcb200640 1 --2- 172.21.15.44:0/214884157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fc415e330 0x7f1fc40b0210 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4161c00 tx=0x7f1fbc01ccf0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.071+0000 7f1fcb200640 1 -- 172.21.15.44:0/214884157 shutdown_connections 2024-09-13T08:53:28.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.071+0000 7f1fcb200640 1 --2- 172.21.15.44:0/214884157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fc415e330 0x7f1fc40b0210 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.071+0000 7f1fcb200640 1 --2- 172.21.15.44:0/214884157 >> v2:172.21.15.129:3300/0 conn(0x7f1fc406ee50 0x7f1fc40afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.071+0000 7f1fcb200640 1 -- 172.21.15.44:0/214884157 >> 172.21.15.44:0/214884157 conn(0x7f1fc40d1dc0 msgr2=0x7f1fc40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:28.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.071+0000 7f1fcb200640 1 -- 172.21.15.44:0/214884157 shutdown_connections 2024-09-13T08:53:28.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.071+0000 7f1fcb200640 1 -- 172.21.15.44:0/214884157 wait complete. 2024-09-13T08:53:28.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.071+0000 7f1fcb200640 1 Processor -- start 2024-09-13T08:53:28.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fcb200640 1 -- start start 2024-09-13T08:53:28.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fcb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fc406ee50 0x7f1fc40c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fcb200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1fc415df50 0x7f1fc40b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fcb200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc415e330 0x7f1fc41fb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fcb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc40d0fd0 con 0x7f1fc406ee50 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fcb200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc40d0e50 con 0x7f1fc415e330 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fcb200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc40d1150 con 0x7f1fc415df50 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fc406ee50 0x7f1fc40c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fc406ee50 0x7f1fc40c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42216/0 (socket says 172.21.15.44:42216) 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc3e00640 1 -- 172.21.15.44:0/2519819655 learned_addr learned my addr 172.21.15.44:0/2519819655 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc3e00640 1 -- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1fc415df50 msgr2=0x7f1fc40b7220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc3e00640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1fc415df50 0x7f1fc40b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc3e00640 1 -- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc415e330 msgr2=0x7f1fc41fb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc3e00640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc415e330 0x7f1fc41fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc3e00640 1 -- 172.21.15.44:0/2519819655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc40c7f00 con 0x7f1fc406ee50 2024-09-13T08:53:28.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc3e00640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fc406ee50 0x7f1fc40c61a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c530 tx=0x7f1fb400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:28.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc0c00640 1 -- 172.21.15.44:0/2519819655 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4017070 con 0x7f1fc406ee50 2024-09-13T08:53:28.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc0c00640 1 -- 172.21.15.44:0/2519819655 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fb40053c0 con 0x7f1fc406ee50 2024-09-13T08:53:28.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fcb200640 1 -- 172.21.15.44:0/2519819655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc40c81e0 con 0x7f1fc406ee50 2024-09-13T08:53:28.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fc0c00640 1 -- 172.21.15.44:0/2519819655 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4012890 con 0x7f1fc406ee50 2024-09-13T08:53:28.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fcb200640 1 -- 172.21.15.44:0/2519819655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc41fb320 con 0x7f1fc406ee50 2024-09-13T08:53:28.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.075+0000 7f1fcb200640 1 -- 172.21.15.44:0/2519819655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc41fb4f0 con 0x7f1fc406ee50 2024-09-13T08:53:28.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.079+0000 7f1fc0c00640 1 -- 172.21.15.44:0/2519819655 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f1fb401a070 con 0x7f1fc406ee50 2024-09-13T08:53:28.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.079+0000 7f1fc0c00640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fa0075110 0x7f1fa00775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.079+0000 7f1fc0c00640 1 -- 172.21.15.44:0/2519819655 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f1fb4092370 con 0x7f1fc406ee50 2024-09-13T08:53:28.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.079+0000 7f1fbbe00640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fa0075110 0x7f1fa00775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:28.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.079+0000 7f1fbbe00640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fa0075110 0x7f1fa00775d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0058d0 tx=0x7f1fac005860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:28.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.083+0000 7f1fc0c00640 1 -- 172.21.15.44:0/2519819655 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1fb405efa0 con 0x7f1fc406ee50 2024-09-13T08:53:28.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.287+0000 7f1fcb200640 1 -- 172.21.15.44:0/2519819655 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1fc40b02a0 con 0x7f1fa0075110 2024-09-13T08:53:28.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fc0c00640 1 -- 172.21.15.44:0/2519819655 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1fc40b02a0 con 0x7f1fa0075110 2024-09-13T08:53:28.291 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:53:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:53:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:53:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:53:28.292 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 -- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fa0075110 msgr2=0x7f1fa00775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fa0075110 0x7f1fa00775d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0058d0 tx=0x7f1fac005860 comp rx=0 tx=0).stop 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 -- 172.21.15.44:0/2519819655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fc406ee50 msgr2=0x7f1fc40c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fc406ee50 0x7f1fc40c61a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c530 tx=0x7f1fb400c9f0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 -- 172.21.15.44:0/2519819655 shutdown_connections 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1fa0075110 0x7f1fa00775d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc415e330 0x7f1fc41fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1fc415df50 0x7f1fc40b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 --2- 172.21.15.44:0/2519819655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fc406ee50 0x7f1fc40c61a0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 -- 172.21.15.44:0/2519819655 >> 172.21.15.44:0/2519819655 conn(0x7f1fc40d1dc0 msgr2=0x7f1fc4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 -- 172.21.15.44:0/2519819655 shutdown_connections 2024-09-13T08:53:28.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.291+0000 7f1fcb200640 1 -- 172.21.15.44:0/2519819655 wait complete. 2024-09-13T08:53:28.314 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T08:53:28.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 -- 172.21.15.44:0/2709869010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd415e450 msgr2=0x7facd40ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 --2- 172.21.15.44:0/2709869010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd415e450 0x7facd40ca050 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7facd40c6a60 tx=0x7facc801c8c0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 -- 172.21.15.44:0/2709869010 shutdown_connections 2024-09-13T08:53:28.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 --2- 172.21.15.44:0/2709869010 >> v2:172.21.15.129:3300/0 conn(0x7facd4072f60 0x7facd40733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 --2- 172.21.15.44:0/2709869010 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facd40aea00 0x7facd40ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 --2- 172.21.15.44:0/2709869010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd415e450 0x7facd40ca050 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 -- 172.21.15.44:0/2709869010 >> 172.21.15.44:0/2709869010 conn(0x7facd40cfa00 msgr2=0x7facd40cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:28.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 -- 172.21.15.44:0/2709869010 shutdown_connections 2024-09-13T08:53:28.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 -- 172.21.15.44:0/2709869010 wait complete. 2024-09-13T08:53:28.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 Processor -- start 2024-09-13T08:53:28.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 -- start start 2024-09-13T08:53:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facd4072f60 0x7facd415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.463+0000 7facdaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40aea00 0x7facd4153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facdaa00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facd415e450 0x7facd4154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facdaa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7facd4073ec0 con 0x7facd40aea00 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facdaa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7facd4073d40 con 0x7facd4072f60 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facdaa00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7facd40aef70 con 0x7facd415e450 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40aea00 0x7facd4153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd9600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facd4072f60 0x7facd415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40aea00 0x7facd4153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42222/0 (socket says 172.21.15.44:42222) 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd8c00640 1 -- 172.21.15.44:0/2304906851 learned_addr learned my addr 172.21.15.44:0/2304906851 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facda000640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facd415e450 0x7facd4154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd8c00640 1 -- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facd415e450 msgr2=0x7facd4154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd8c00640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facd415e450 0x7facd4154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd8c00640 1 -- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facd4072f60 msgr2=0x7facd415c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd8c00640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facd4072f60 0x7facd415c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd8c00640 1 -- 172.21.15.44:0/2304906851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd41fb190 con 0x7facd40aea00 2024-09-13T08:53:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd8c00640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40aea00 0x7facd4153b40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7facc000b550 tx=0x7facc000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd1600640 1 -- 172.21.15.44:0/2304906851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0014070 con 0x7facd40aea00 2024-09-13T08:53:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd1600640 1 -- 172.21.15.44:0/2304906851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facc0010980 con 0x7facd40aea00 2024-09-13T08:53:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd1600640 1 -- 172.21.15.44:0/2304906851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0010dd0 con 0x7facd40aea00 2024-09-13T08:53:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facdaa00640 1 -- 172.21.15.44:0/2304906851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd41fb470 con 0x7facd40aea00 2024-09-13T08:53:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facdaa00640 1 -- 172.21.15.44:0/2304906851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd41fba30 con 0x7facd40aea00 2024-09-13T08:53:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facdaa00640 1 -- 172.21.15.44:0/2304906851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd40733f0 con 0x7facd40aea00 2024-09-13T08:53:28.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.467+0000 7facd1600640 1 -- 172.21.15.44:0/2304906851 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7facc0017070 con 0x7facd40aea00 2024-09-13T08:53:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.471+0000 7facd1600640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb4075110 0x7facb40775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.471+0000 7facd1600640 1 -- 172.21.15.44:0/2304906851 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7facc0092db0 con 0x7facd40aea00 2024-09-13T08:53:28.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.471+0000 7facd9600640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb4075110 0x7facb40775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:28.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.471+0000 7facd9600640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb4075110 0x7facb40775d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7facc80af000 tx=0x7facc801b100 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:28.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.475+0000 7facd1600640 1 -- 172.21.15.44:0/2304906851 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7facc005f410 con 0x7facd40aea00 2024-09-13T08:53:28.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.655+0000 7facdaa00640 1 -- 172.21.15.44:0/2304906851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7facd41fbc00 con 0x7facd40aea00 2024-09-13T08:53:28.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.655+0000 7facd1600640 1 -- 172.21.15.44:0/2304906851 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7facc0063e40 con 0x7facd40aea00 2024-09-13T08:53:28.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.655+0000 7facdaa00640 1 -- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb4075110 msgr2=0x7facb40775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb4075110 0x7facb40775d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7facc80af000 tx=0x7facc801b100 comp rx=0 tx=0).stop 2024-09-13T08:53:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 -- 172.21.15.44:0/2304906851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40aea00 msgr2=0x7facd4153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40aea00 0x7facd4153b40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7facc000b550 tx=0x7facc000ba10 comp rx=0 tx=0).stop 2024-09-13T08:53:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 -- 172.21.15.44:0/2304906851 shutdown_connections 2024-09-13T08:53:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7facb4075110 0x7facb40775d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7facd415e450 0x7facd4154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40aea00 0x7facd4153b40 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 --2- 172.21.15.44:0/2304906851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facd4072f60 0x7facd415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 -- 172.21.15.44:0/2304906851 >> 172.21.15.44:0/2304906851 conn(0x7facd40cfa00 msgr2=0x7facd4157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 -- 172.21.15.44:0/2304906851 shutdown_connections 2024-09-13T08:53:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.659+0000 7facdaa00640 1 -- 172.21.15.44:0/2304906851 wait complete. 2024-09-13T08:53:28.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-13T08:53:28.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.827+0000 7f0682600640 1 -- 172.21.15.44:0/3863400695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0660003a10 msgr2=0x7f0660003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.827+0000 7f0682600640 1 --2- 172.21.15.44:0/3863400695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0660003a10 0x7f0660003e30 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f066c009b60 tx=0x7f066c01d2b0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.827+0000 7f067be00640 1 -- 172.21.15.44:0/3863400695 reap_dead start 2024-09-13T08:53:28.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 -- 172.21.15.44:0/3863400695 shutdown_connections 2024-09-13T08:53:28.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 -- 172.21.15.44:0/3863400695 >> 172.21.15.44:0/3863400695 conn(0x7f067c0d1e90 msgr2=0x7f067c0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:28.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 -- 172.21.15.44:0/3863400695 shutdown_connections 2024-09-13T08:53:28.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 -- 172.21.15.44:0/3863400695 wait complete. 2024-09-13T08:53:28.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 Processor -- start 2024-09-13T08:53:28.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 -- start start 2024-09-13T08:53:28.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660003a10 0x7f067c1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f067c0cd5b0 0x7f067c0ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f067c0af740 0x7f067c0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f067c0b0ed0 con 0x7f067c0cd5b0 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f067c0b0d50 con 0x7f0660003a10 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0682600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f067c0af200 con 0x7f067c0af740 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0672a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f067c0cd5b0 0x7f067c0ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0672a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f067c0cd5b0 0x7f067c0ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42246/0 (socket says 172.21.15.44:42246) 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0672a00640 1 -- 172.21.15.44:0/3537279950 learned_addr learned my addr 172.21.15.44:0/3537279950 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f067b400640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660003a10 0x7f067c1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f067be00640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f067c0af740 0x7f067c0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0672a00640 1 -- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f067c0af740 msgr2=0x7f067c0afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0672a00640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f067c0af740 0x7f067c0afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0672a00640 1 -- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660003a10 msgr2=0x7f067c1522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:28.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0672a00640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660003a10 0x7f067c1522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:28.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0672a00640 1 -- 172.21.15.44:0/3537279950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c0cee90 con 0x7f067c0cd5b0 2024-09-13T08:53:28.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f067be00640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f067c0af740 0x7f067c0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:28.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0672a00640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f067c0cd5b0 0x7f067c0ce5f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f0664001800 tx=0x7f0664001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:28.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.831+0000 7f0673e00640 1 -- 172.21.15.44:0/3537279950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664012850 con 0x7f067c0cd5b0 2024-09-13T08:53:28.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.835+0000 7f0673e00640 1 -- 172.21.15.44:0/3537279950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0664012ae0 con 0x7f067c0cd5b0 2024-09-13T08:53:28.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.835+0000 7f0673e00640 1 -- 172.21.15.44:0/3537279950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066401a400 con 0x7f067c0cd5b0 2024-09-13T08:53:28.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.835+0000 7f0682600640 1 -- 172.21.15.44:0/3537279950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c0cf170 con 0x7f067c0cd5b0 2024-09-13T08:53:28.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.835+0000 7f0682600640 1 -- 172.21.15.44:0/3537279950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c0cf730 con 0x7f067c0cd5b0 2024-09-13T08:53:28.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.835+0000 7f0682600640 1 -- 172.21.15.44:0/3537279950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067c0b15e0 con 0x7f067c0cd5b0 2024-09-13T08:53:28.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.835+0000 7f0673e00640 1 -- 172.21.15.44:0/3537279950 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f066400d030 con 0x7f067c0cd5b0 2024-09-13T08:53:28.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.835+0000 7f0673e00640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658075140 0x7f0658077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:28.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.835+0000 7f0673e00640 1 -- 172.21.15.44:0/3537279950 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f0664096260 con 0x7f067c0cd5b0 2024-09-13T08:53:28.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.835+0000 7f067b400640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658075140 0x7f0658077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:28.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.839+0000 7f067b400640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658075140 0x7f0658077600 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f066c0ae000 tx=0x7f066c01a050 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:28.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:28.839+0000 7f0673e00640 1 -- 172.21.15.44:0/3537279950 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0664016030 con 0x7f067c0cd5b0 2024-09-13T08:53:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.023+0000 7f0682600640 1 -- 172.21.15.44:0/3537279950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f067c0d2770 con 0x7f067c0cd5b0 2024-09-13T08:53:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.023+0000 7f0673e00640 1 -- 172.21.15.44:0/3537279950 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v54) ==== 116+0+2 (secure 0 0 0) 0x7f0664062fb0 con 0x7f067c0cd5b0 2024-09-13T08:53:29.025 INFO:teuthology.orchestra.run.smithi044.stdout:4 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 -- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658075140 msgr2=0x7f0658077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658075140 0x7f0658077600 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f066c0ae000 tx=0x7f066c01a050 comp rx=0 tx=0).stop 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 -- 172.21.15.44:0/3537279950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f067c0cd5b0 msgr2=0x7f067c0ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f067c0cd5b0 0x7f067c0ce5f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f0664001800 tx=0x7f0664001830 comp rx=0 tx=0).stop 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 -- 172.21.15.44:0/3537279950 shutdown_connections 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0658075140 0x7f0658077600 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f067c0af740 0x7f067c0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f067c0cd5b0 0x7f067c0ce5f0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 --2- 172.21.15.44:0/3537279950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660003a10 0x7f067c1522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 -- 172.21.15.44:0/3537279950 >> 172.21.15.44:0/3537279950 conn(0x7f067c0d1e90 msgr2=0x7f067c152ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 -- 172.21.15.44:0/3537279950 shutdown_connections 2024-09-13T08:53:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.027+0000 7f0682600640 1 -- 172.21.15.44:0/3537279950 wait complete. 2024-09-13T08:53:29.048 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:53:29.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.195+0000 7f1222e00640 1 -- 172.21.15.44:0/4230038534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f121c0cd100 msgr2=0x7f121c0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.195+0000 7f1222e00640 1 --2- 172.21.15.44:0/4230038534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f121c0cd100 0x7f121c0ce810 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f121c070900 tx=0x7f121401ccf0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.195+0000 7f1222e00640 1 -- 172.21.15.44:0/4230038534 shutdown_connections 2024-09-13T08:53:29.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.195+0000 7f1222e00640 1 --2- 172.21.15.44:0/4230038534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f121c0cd100 0x7f121c0ce810 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.195+0000 7f1222e00640 1 --2- 172.21.15.44:0/4230038534 >> v2:172.21.15.129:3300/0 conn(0x7f121c0b4a50 0x7f121c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.195+0000 7f1222e00640 1 -- 172.21.15.44:0/4230038534 >> 172.21.15.44:0/4230038534 conn(0x7f121c0b18f0 msgr2=0x7f121c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:29.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.195+0000 7f1222e00640 1 -- 172.21.15.44:0/4230038534 shutdown_connections 2024-09-13T08:53:29.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.195+0000 7f1222e00640 1 -- 172.21.15.44:0/4230038534 wait complete. 2024-09-13T08:53:29.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 Processor -- start 2024-09-13T08:53:29.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 -- start start 2024-09-13T08:53:29.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f121c0b4a50 0x7f121c14d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f121c0cd100 0x7f121c1f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f121c15db50 0x7f121c1f8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f121c167120 con 0x7f121c0b4a50 2024-09-13T08:53:29.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f121c0cf5a0 con 0x7f121c15db50 2024-09-13T08:53:29.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f121c1672a0 con 0x7f121c0cd100 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f121be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f121c0b4a50 0x7f121c14d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f121be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f121c0b4a50 0x7f121c14d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42266/0 (socket says 172.21.15.44:42266) 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f121be00640 1 -- 172.21.15.44:0/244083108 learned_addr learned my addr 172.21.15.44:0/244083108 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f121b400640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f121c0cd100 0x7f121c1f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1220e00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f121c15db50 0x7f121c1f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f121be00640 1 -- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f121c0cd100 msgr2=0x7f121c1f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f121be00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f121c0cd100 0x7f121c1f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f121be00640 1 -- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f121c15db50 msgr2=0x7f121c1f8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f121be00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f121c15db50 0x7f121c1f8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f121be00640 1 -- 172.21.15.44:0/244083108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c1f8880 con 0x7f121c0b4a50 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f121be00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f121c0b4a50 0x7f121c14d010 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f120c00c3f0 tx=0x7f120c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1213e00640 1 -- 172.21.15.44:0/244083108 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c017070 con 0x7f121c0b4a50 2024-09-13T08:53:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1213e00640 1 -- 172.21.15.44:0/244083108 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f120c0053c0 con 0x7f121c0b4a50 2024-09-13T08:53:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 -- 172.21.15.44:0/244083108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121c1f8b60 con 0x7f121c0b4a50 2024-09-13T08:53:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1213e00640 1 -- 172.21.15.44:0/244083108 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c003860 con 0x7f121c0b4a50 2024-09-13T08:53:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 -- 172.21.15.44:0/244083108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121c09cdb0 con 0x7f121c0b4a50 2024-09-13T08:53:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.199+0000 7f1222e00640 1 -- 172.21.15.44:0/244083108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f121c0aee80 con 0x7f121c0b4a50 2024-09-13T08:53:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.203+0000 7f1213e00640 1 -- 172.21.15.44:0/244083108 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f120c01a070 con 0x7f121c0b4a50 2024-09-13T08:53:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.203+0000 7f1213e00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11fc075140 0x7f11fc077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.203+0000 7f1213e00640 1 -- 172.21.15.44:0/244083108 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f120c092250 con 0x7f121c0b4a50 2024-09-13T08:53:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.203+0000 7f121b400640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11fc075140 0x7f11fc077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.203+0000 7f121b400640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11fc075140 0x7f11fc077600 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f121c15d230 tx=0x7f120800b490 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.207+0000 7f1213e00640 1 -- 172.21.15.44:0/244083108 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f120c05e830 con 0x7f121c0b4a50 2024-09-13T08:53:29.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.383+0000 7f1222e00640 1 -- 172.21.15.44:0/244083108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f121c0adec0 con 0x7f121c0b4a50 2024-09-13T08:53:29.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.387+0000 7f1213e00640 1 -- 172.21.15.44:0/244083108 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v55)=0 v55) ==== 131+0+0 (secure 0 0 0) 0x7f121c0adec0 con 0x7f121c0b4a50 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 -- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11fc075140 msgr2=0x7f11fc077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11fc075140 0x7f11fc077600 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f121c15d230 tx=0x7f120800b490 comp rx=0 tx=0).stop 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 -- 172.21.15.44:0/244083108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f121c0b4a50 msgr2=0x7f121c14d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f121c0b4a50 0x7f121c14d010 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f120c00c3f0 tx=0x7f120c00c8b0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 -- 172.21.15.44:0/244083108 shutdown_connections 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11fc075140 0x7f11fc077600 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f121c15db50 0x7f121c1f8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f121c0cd100 0x7f121c1f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 --2- 172.21.15.44:0/244083108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f121c0b4a50 0x7f121c14d010 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 -- 172.21.15.44:0/244083108 >> 172.21.15.44:0/244083108 conn(0x7f121c0b18f0 msgr2=0x7f121c0cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 -- 172.21.15.44:0/244083108 shutdown_connections 2024-09-13T08:53:29.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.391+0000 7f1222e00640 1 -- 172.21.15.44:0/244083108 wait complete. 2024-09-13T08:53:29.412 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-13T08:53:29.414 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:53:29.412+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:53:29.414 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:53:29.412+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:53:29.414 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:53:29.412+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:53:29.414 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:53:29.412+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:53:29.414 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:53:29.412+0000 7f8ac5c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:53:29.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 -- 172.21.15.44:0/3590138396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd330005240 msgr2=0x7fd33000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 --2- 172.21.15.44:0/3590138396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd330005240 0x7fd33000b9d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd3480cfbb0 tx=0x7fd34401cbe0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 -- 172.21.15.44:0/3590138396 shutdown_connections 2024-09-13T08:53:29.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 --2- 172.21.15.44:0/3590138396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd330005240 0x7fd33000b9d0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 -- 172.21.15.44:0/3590138396 >> 172.21.15.44:0/3590138396 conn(0x7fd3480b18f0 msgr2=0x7fd3480b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:29.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 -- 172.21.15.44:0/3590138396 shutdown_connections 2024-09-13T08:53:29.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 -- 172.21.15.44:0/3590138396 wait complete. 2024-09-13T08:53:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 Processor -- start 2024-09-13T08:53:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 -- start start 2024-09-13T08:53:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd330005240 0x7fd3480b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3480ae3b0 0x7fd3481595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3481f5160 0x7fd34815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3480cd180 con 0x7fd3481f5160 2024-09-13T08:53:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3480cd000 con 0x7fd330005240 2024-09-13T08:53:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd350800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd3480cd300 con 0x7fd3480ae3b0 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd34e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3481f5160 0x7fd34815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd34e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3481f5160 0x7fd34815a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42280/0 (socket says 172.21.15.44:42280) 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd34e800640 1 -- 172.21.15.44:0/1215421392 learned_addr learned my addr 172.21.15.44:0/1215421392 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd34de00640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd330005240 0x7fd3480b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.563+0000 7fd34d400640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3480ae3b0 0x7fd3481595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd34e800640 1 -- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3480ae3b0 msgr2=0x7fd3481595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd34e800640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3480ae3b0 0x7fd3481595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd34e800640 1 -- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd330005240 msgr2=0x7fd3480b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd34e800640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd330005240 0x7fd3480b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd34e800640 1 -- 172.21.15.44:0/1215421392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3481f55e0 con 0x7fd3481f5160 2024-09-13T08:53:29.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd34e800640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3481f5160 0x7fd34815a020 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fd34401d480 tx=0x7fd344004fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd342000640 1 -- 172.21.15.44:0/1215421392 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344006c50 con 0x7fd3481f5160 2024-09-13T08:53:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd342000640 1 -- 172.21.15.44:0/1215421392 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3440b9070 con 0x7fd3481f5160 2024-09-13T08:53:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd350800640 1 -- 172.21.15.44:0/1215421392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd348159ca0 con 0x7fd3481f5160 2024-09-13T08:53:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd342000640 1 -- 172.21.15.44:0/1215421392 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344006490 con 0x7fd3481f5160 2024-09-13T08:53:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd350800640 1 -- 172.21.15.44:0/1215421392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3480b32f0 con 0x7fd3481f5160 2024-09-13T08:53:29.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd350800640 1 -- 172.21.15.44:0/1215421392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd348069410 con 0x7fd3481f5160 2024-09-13T08:53:29.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd342000640 1 -- 172.21.15.44:0/1215421392 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7fd344006630 con 0x7fd3481f5160 2024-09-13T08:53:29.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.567+0000 7fd342000640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd32c075140 0x7fd32c077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.571+0000 7fd342000640 1 -- 172.21.15.44:0/1215421392 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fd34412d5b0 con 0x7fd3481f5160 2024-09-13T08:53:29.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.571+0000 7fd34de00640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd32c075140 0x7fd32c077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.571+0000 7fd34de00640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd32c075140 0x7fd32c077600 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd33c001060 tx=0x7fd33c001e10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:29.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.575+0000 7fd342000640 1 -- 172.21.15.44:0/1215421392 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd344016990 con 0x7fd3481f5160 2024-09-13T08:53:29.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.751+0000 7fd350800640 1 -- 172.21.15.44:0/1215421392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7fd34815cb40 con 0x7fd3481f5160 2024-09-13T08:53:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.755+0000 7fd342000640 1 -- 172.21.15.44:0/1215421392 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v56)=0 v56) ==== 136+0+0 (secure 0 0 0) 0x7fd3440f9c10 con 0x7fd3481f5160 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 -- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd32c075140 msgr2=0x7fd32c077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd32c075140 0x7fd32c077600 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd33c001060 tx=0x7fd33c001e10 comp rx=0 tx=0).stop 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 -- 172.21.15.44:0/1215421392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3481f5160 msgr2=0x7fd34815a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3481f5160 0x7fd34815a020 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fd34401d480 tx=0x7fd344004fd0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 -- 172.21.15.44:0/1215421392 shutdown_connections 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd32c075140 0x7fd32c077600 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3481f5160 0x7fd34815a020 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd3480ae3b0 0x7fd3481595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 --2- 172.21.15.44:0/1215421392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd330005240 0x7fd3480b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 -- 172.21.15.44:0/1215421392 >> 172.21.15.44:0/1215421392 conn(0x7fd3480b18f0 msgr2=0x7fd34815d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 -- 172.21.15.44:0/1215421392 shutdown_connections 2024-09-13T08:53:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.759+0000 7fd350800640 1 -- 172.21.15.44:0/1215421392 wait complete. 2024-09-13T08:53:29.779 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-13T08:53:29.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.927+0000 7fb6cea00640 1 -- 172.21.15.44:0/2139830111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6c80b1950 msgr2=0x7fb6c80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.927+0000 7fb6cea00640 1 --2- 172.21.15.44:0/2139830111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6c80b1950 0x7fb6c80cd7c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009840 tx=0x7fb6bc01c700 comp rx=0 tx=0).stop 2024-09-13T08:53:29.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.927+0000 7fb6cea00640 1 -- 172.21.15.44:0/2139830111 shutdown_connections 2024-09-13T08:53:29.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.927+0000 7fb6cea00640 1 --2- 172.21.15.44:0/2139830111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6c80b1950 0x7fb6c80cd7c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.927+0000 7fb6cea00640 1 -- 172.21.15.44:0/2139830111 >> 172.21.15.44:0/2139830111 conn(0x7fb6c8073540 msgr2=0x7fb6c80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:29.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.927+0000 7fb6cea00640 1 -- 172.21.15.44:0/2139830111 shutdown_connections 2024-09-13T08:53:29.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.927+0000 7fb6cea00640 1 -- 172.21.15.44:0/2139830111 wait complete. 2024-09-13T08:53:29.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cea00640 1 Processor -- start 2024-09-13T08:53:29.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cea00640 1 -- start start 2024-09-13T08:53:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8068960 0x7fb6c8159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6c80b1950 0x7fb6c80c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cea00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6c815b100 0x7fb6c81589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8169280 con 0x7fb6c8068960 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cea00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8169100 con 0x7fb6c80b1950 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cea00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c8169400 con 0x7fb6c815b100 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cd600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8068960 0x7fb6c8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cd600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8068960 0x7fb6c8159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42284/0 (socket says 172.21.15.44:42284) 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cd600640 1 -- 172.21.15.44:0/2476328198 learned_addr learned my addr 172.21.15.44:0/2476328198 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6ccc00640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6c80b1950 0x7fb6c80c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6ce000640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6c815b100 0x7fb6c81589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cd600640 1 -- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6c815b100 msgr2=0x7fb6c81589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cd600640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6c815b100 0x7fb6c81589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cd600640 1 -- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6c80b1950 msgr2=0x7fb6c80c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cd600640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6c80b1950 0x7fb6c80c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cd600640 1 -- 172.21.15.44:0/2476328198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c80c75d0 con 0x7fb6c8068960 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cd600640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8068960 0x7fb6c8159070 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009790 tx=0x7fb6bc0a6640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6c5600640 1 -- 172.21.15.44:0/2476328198 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0bc070 con 0x7fb6c8068960 2024-09-13T08:53:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6c5600640 1 -- 172.21.15.44:0/2476328198 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6bc0043f0 con 0x7fb6c8068960 2024-09-13T08:53:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cea00640 1 -- 172.21.15.44:0/2476328198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c80ae850 con 0x7fb6c8068960 2024-09-13T08:53:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6c5600640 1 -- 172.21.15.44:0/2476328198 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc005250 con 0x7fb6c8068960 2024-09-13T08:53:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.931+0000 7fb6cea00640 1 -- 172.21.15.44:0/2476328198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c80c7b50 con 0x7fb6c8068960 2024-09-13T08:53:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.935+0000 7fb6cea00640 1 -- 172.21.15.44:0/2476328198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c80cd7c0 con 0x7fb6c8068960 2024-09-13T08:53:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.935+0000 7fb6c5600640 1 -- 172.21.15.44:0/2476328198 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7fb6bc0bf070 con 0x7fb6c8068960 2024-09-13T08:53:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.935+0000 7fb6c5600640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6a8075110 0x7fb6a80775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.935+0000 7fb6c5600640 1 -- 172.21.15.44:0/2476328198 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fb6bc12dd70 con 0x7fb6c8068960 2024-09-13T08:53:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.935+0000 7fb6ccc00640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6a8075110 0x7fb6a80775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.935+0000 7fb6ccc00640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6a8075110 0x7fb6a80775d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4007fd0 tx=0x7fb6b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:29.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:29.939+0000 7fb6c5600640 1 -- 172.21.15.44:0/2476328198 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb6bc016410 con 0x7fb6c8068960 2024-09-13T08:53:30.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.143+0000 7fb6cea00640 1 -- 172.21.15.44:0/2476328198 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fb6c80c7d90 con 0x7fb6a8075110 2024-09-13T08:53:30.154 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:53:30.152+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:53:30.154 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:53:30.152+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:53:30.154 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:53:30.152+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:53:30.154 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:53:30.152+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:53:30.154 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:53:30.152+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:53:30.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6c5600640 1 -- 172.21.15.44:0/2476328198 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb6c80c7d90 con 0x7fb6a8075110 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 -- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6a8075110 msgr2=0x7fb6a80775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6a8075110 0x7fb6a80775d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4007fd0 tx=0x7fb6b4009040 comp rx=0 tx=0).stop 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 -- 172.21.15.44:0/2476328198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8068960 msgr2=0x7fb6c8159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8068960 0x7fb6c8159070 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009790 tx=0x7fb6bc0a6640 comp rx=0 tx=0).stop 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 -- 172.21.15.44:0/2476328198 shutdown_connections 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb6a8075110 0x7fb6a80775d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6c815b100 0x7fb6c81589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6c80b1950 0x7fb6c80c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 --2- 172.21.15.44:0/2476328198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8068960 0x7fb6c8159070 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 -- 172.21.15.44:0/2476328198 >> 172.21.15.44:0/2476328198 conn(0x7fb6c8073540 msgr2=0x7fb6c806fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:30.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 -- 172.21.15.44:0/2476328198 shutdown_connections 2024-09-13T08:53:30.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.191+0000 7fb6cea00640 1 -- 172.21.15.44:0/2476328198 wait complete. 2024-09-13T08:53:30.213 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:53:30.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.359+0000 7f11d0a00640 1 -- 172.21.15.44:0/1104267696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc0b1650 msgr2=0x7f11cc0b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.359+0000 7f11d0a00640 1 --2- 172.21.15.44:0/1104267696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc0b1650 0x7f11cc0b1a30 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009840 tx=0x7f11b401c820 comp rx=0 tx=0).stop 2024-09-13T08:53:30.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.359+0000 7f11d0a00640 1 -- 172.21.15.44:0/1104267696 shutdown_connections 2024-09-13T08:53:30.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.359+0000 7f11d0a00640 1 --2- 172.21.15.44:0/1104267696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc0b1650 0x7f11cc0b1a30 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.359+0000 7f11d0a00640 1 -- 172.21.15.44:0/1104267696 >> 172.21.15.44:0/1104267696 conn(0x7f11cc0d1dd0 msgr2=0x7f11cc0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:30.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.359+0000 7f11d0a00640 1 -- 172.21.15.44:0/1104267696 shutdown_connections 2024-09-13T08:53:30.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.359+0000 7f11d0a00640 1 -- 172.21.15.44:0/1104267696 wait complete. 2024-09-13T08:53:30.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11d0a00640 1 Processor -- start 2024-09-13T08:53:30.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11d0a00640 1 -- start start 2024-09-13T08:53:30.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11d0a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11cc0afb10 0x7f11cc1f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11d0a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11cc0b1650 0x7f11cc15b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11d0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc1605d0 0x7f11cc0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11d0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc0b1220 con 0x7f11cc1605d0 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11d0a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc0b0ee0 con 0x7f11cc0afb10 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11d0a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc0b13a0 con 0x7f11cc0b1650 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11ca800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc1605d0 0x7f11cc0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11ca800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc1605d0 0x7f11cc0cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42304/0 (socket says 172.21.15.44:42304) 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11ca800640 1 -- 172.21.15.44:0/3012442426 learned_addr learned my addr 172.21.15.44:0/3012442426 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11c9e00640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11cc0afb10 0x7f11cc1f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11c1400640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11cc0b1650 0x7f11cc15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11ca800640 1 -- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11cc0b1650 msgr2=0x7f11cc15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11ca800640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11cc0b1650 0x7f11cc15b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11ca800640 1 -- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11cc0afb10 msgr2=0x7f11cc1f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11ca800640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11cc0afb10 0x7f11cc1f8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11ca800640 1 -- 172.21.15.44:0/3012442426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc15bb50 con 0x7f11cc1605d0 2024-09-13T08:53:30.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11ca800640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc1605d0 0x7f11cc0cded0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c780 tx=0x7f11c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:30.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11c2a00640 1 -- 172.21.15.44:0/3012442426 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4004b70 con 0x7f11cc1605d0 2024-09-13T08:53:30.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11c2a00640 1 -- 172.21.15.44:0/3012442426 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11c4004e00 con 0x7f11cc1605d0 2024-09-13T08:53:30.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11d0a00640 1 -- 172.21.15.44:0/3012442426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc06e8a0 con 0x7f11cc1605d0 2024-09-13T08:53:30.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11c2a00640 1 -- 172.21.15.44:0/3012442426 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c400aa90 con 0x7f11cc1605d0 2024-09-13T08:53:30.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.363+0000 7f11d0a00640 1 -- 172.21.15.44:0/3012442426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc1f8140 con 0x7f11cc1605d0 2024-09-13T08:53:30.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.367+0000 7f11d0a00640 1 -- 172.21.15.44:0/3012442426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11cc0affc0 con 0x7f11cc1605d0 2024-09-13T08:53:30.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.367+0000 7f11c2a00640 1 -- 172.21.15.44:0/3012442426 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f11c400ac30 con 0x7f11cc1605d0 2024-09-13T08:53:30.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.367+0000 7f11c2a00640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11a8075110 0x7f11a80775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:30.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.367+0000 7f11c2a00640 1 -- 172.21.15.44:0/3012442426 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f11c4093140 con 0x7f11cc1605d0 2024-09-13T08:53:30.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.367+0000 7f11c9e00640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11a8075110 0x7f11a80775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:30.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.367+0000 7f11c9e00640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11a8075110 0x7f11a80775d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f11bc007fd0 tx=0x7f11bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:30.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.371+0000 7f11c2a00640 1 -- 172.21.15.44:0/3012442426 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f11c4020070 con 0x7f11cc1605d0 2024-09-13T08:53:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.575+0000 7f11d0a00640 1 -- 172.21.15.44:0/3012442426 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f11cc0d1080 con 0x7f11a8075110 2024-09-13T08:53:30.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.579+0000 7f11c2a00640 1 -- 172.21.15.44:0/3012442426 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f11cc0d1080 con 0x7f11a8075110 2024-09-13T08:53:30.582 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883 2024-09-13T08:53:30.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 -- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11a8075110 msgr2=0x7f11a80775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11a8075110 0x7f11a80775d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f11bc007fd0 tx=0x7f11bc009040 comp rx=0 tx=0).stop 2024-09-13T08:53:30.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 -- 172.21.15.44:0/3012442426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc1605d0 msgr2=0x7f11cc0cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc1605d0 0x7f11cc0cded0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c780 tx=0x7f11c400cc40 comp rx=0 tx=0).stop 2024-09-13T08:53:30.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 -- 172.21.15.44:0/3012442426 shutdown_connections 2024-09-13T08:53:30.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f11a8075110 0x7f11a80775d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc1605d0 0x7f11cc0cded0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f11cc0b1650 0x7f11cc15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 --2- 172.21.15.44:0/3012442426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11cc0afb10 0x7f11cc1f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 -- 172.21.15.44:0/3012442426 >> 172.21.15.44:0/3012442426 conn(0x7f11cc0d1dd0 msgr2=0x7f11cc071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:30.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 -- 172.21.15.44:0/3012442426 shutdown_connections 2024-09-13T08:53:30.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.583+0000 7f11d0a00640 1 -- 172.21.15.44:0/3012442426 wait complete. 2024-09-13T08:53:30.604 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:53:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.751+0000 7f373ee00640 1 -- 172.21.15.44:0/3088504770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f373815c3e0 msgr2=0x7f373815c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.751+0000 7f373ee00640 1 --2- 172.21.15.44:0/3088504770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f373815c3e0 0x7f373815c7c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f3728009930 tx=0x7f372801f0d0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.751+0000 7f373ee00640 1 -- 172.21.15.44:0/3088504770 shutdown_connections 2024-09-13T08:53:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.751+0000 7f373ee00640 1 --2- 172.21.15.44:0/3088504770 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f373806e7a0 0x7f373806ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.751+0000 7f373ee00640 1 --2- 172.21.15.44:0/3088504770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f373815c3e0 0x7f373815c7c0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.751+0000 7f373ee00640 1 -- 172.21.15.44:0/3088504770 >> 172.21.15.44:0/3088504770 conn(0x7f37380b18f0 msgr2=0x7f37380b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.751+0000 7f373ee00640 1 -- 172.21.15.44:0/3088504770 shutdown_connections 2024-09-13T08:53:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.751+0000 7f373ee00640 1 -- 172.21.15.44:0/3088504770 wait complete. 2024-09-13T08:53:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ee00640 1 Processor -- start 2024-09-13T08:53:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ee00640 1 -- start start 2024-09-13T08:53:30.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f373806e7a0 0x7f373809d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:30.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f373815bfa0 0x7f373814c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:30.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ee00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f373815c3e0 0x7f37381e8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:30.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f373806f310 con 0x7f373806e7a0 2024-09-13T08:53:30.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f373806f190 con 0x7f373815bfa0 2024-09-13T08:53:30.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ee00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f373806f490 con 0x7f373815c3e0 2024-09-13T08:53:30.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3737e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f373806e7a0 0x7f373809d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:30.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3737e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f373806e7a0 0x7f373809d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42320/0 (socket says 172.21.15.44:42320) 2024-09-13T08:53:30.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3737e00640 1 -- 172.21.15.44:0/2702119593 learned_addr learned my addr 172.21.15.44:0/2702119593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:30.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ce00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f373815c3e0 0x7f37381e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:30.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f372fe00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f373815bfa0 0x7f373814c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:30.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3737e00640 1 -- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f373815c3e0 msgr2=0x7f37381e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3737e00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f373815c3e0 0x7f37381e8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3737e00640 1 -- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f373815bfa0 msgr2=0x7f373814c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3737e00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f373815bfa0 0x7f373814c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3737e00640 1 -- 172.21.15.44:0/2702119593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37381f4d10 con 0x7f373806e7a0 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f372fe00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f373815bfa0 0x7f373814c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ce00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f373815c3e0 0x7f37381e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3737e00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f373806e7a0 0x7f373809d290 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f3728009850 tx=0x7f37280a8ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3734c00640 1 -- 172.21.15.44:0/2702119593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37280b5040 con 0x7f373806e7a0 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3734c00640 1 -- 172.21.15.44:0/2702119593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3728003970 con 0x7f373806e7a0 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f3734c00640 1 -- 172.21.15.44:0/2702119593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728003d80 con 0x7f373806e7a0 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ee00640 1 -- 172.21.15.44:0/2702119593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37380b4d00 con 0x7f373806e7a0 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.755+0000 7f373ee00640 1 -- 172.21.15.44:0/2702119593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373809bdd0 con 0x7f373806e7a0 2024-09-13T08:53:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.759+0000 7f373ee00640 1 -- 172.21.15.44:0/2702119593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f373806ec50 con 0x7f373806e7a0 2024-09-13T08:53:30.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.759+0000 7f3734c00640 1 -- 172.21.15.44:0/2702119593 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 100191+0+0 (secure 0 0 0) 0x7f37280bb470 con 0x7f373806e7a0 2024-09-13T08:53:30.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.759+0000 7f3734c00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3714075110 0x7f37140775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:30.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.759+0000 7f3734c00640 1 -- 172.21.15.44:0/2702119593 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f37281305c0 con 0x7f373806e7a0 2024-09-13T08:53:30.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.759+0000 7f372fe00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3714075110 0x7f37140775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:30.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.763+0000 7f372fe00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3714075110 0x7f37140775d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f37381e93b0 tx=0x7f3720009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:30.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.763+0000 7f3734c00640 1 -- 172.21.15.44:0/2702119593 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f37280c0060 con 0x7f373806e7a0 2024-09-13T08:53:30.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.967+0000 7f373ee00640 1 -- 172.21.15.44:0/2702119593 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f373809da90 con 0x7f3714075110 2024-09-13T08:53:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.971+0000 7f3734c00640 1 -- 172.21.15.44:0/2702119593 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f373809da90 con 0x7f3714075110 2024-09-13T08:53:30.974 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883 2024-09-13T08:53:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 -- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3714075110 msgr2=0x7f37140775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3714075110 0x7f37140775d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f37381e93b0 tx=0x7f3720009040 comp rx=0 tx=0).stop 2024-09-13T08:53:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 -- 172.21.15.44:0/2702119593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f373806e7a0 msgr2=0x7f373809d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f373806e7a0 0x7f373809d290 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f3728009850 tx=0x7f37280a8ce0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 -- 172.21.15.44:0/2702119593 shutdown_connections 2024-09-13T08:53:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3714075110 0x7f37140775d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f373815c3e0 0x7f37381e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f373815bfa0 0x7f373814c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 --2- 172.21.15.44:0/2702119593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f373806e7a0 0x7f373809d290 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 -- 172.21.15.44:0/2702119593 >> 172.21.15.44:0/2702119593 conn(0x7f37380b18f0 msgr2=0x7f37380af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 -- 172.21.15.44:0/2702119593 shutdown_connections 2024-09-13T08:53:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:30.975+0000 7f373ee00640 1 -- 172.21.15.44:0/2702119593 wait complete. 2024-09-13T08:53:30.995 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883 2024-09-13T08:53:30.996 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:53:30.996 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:31.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:31.005 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:53:33.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:53:33.192 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:53:40.369 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:53:40.369 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:53:40.369 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.30712 s, 115 MB/s 2024-09-13T08:53:40.370 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:53:40.370 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:40.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:40.379 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:53:49.578 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:53:49.578 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:53:49.579 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.14585 s, 117 MB/s 2024-09-13T08:53:49.579 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:53:49.580 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:53:49.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:49.588 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-13T08:53:58.949 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records in 2024-09-13T08:53:58.949 INFO:teuthology.orchestra.run.smithi044.stderr:1024+0 records out 2024-09-13T08:53:58.949 INFO:teuthology.orchestra.run.smithi044.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.30871 s, 115 MB/s 2024-09-13T08:53:58.952 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-13T08:53:59.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.103+0000 7f6172600640 1 -- 172.21.15.44:0/697907269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f616c15e360 msgr2=0x7f616c0adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:59.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.103+0000 7f6172600640 1 --2- 172.21.15.44:0/697907269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f616c15e360 0x7f616c0adf50 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f615c009900 tx=0x7f615c01c850 comp rx=0 tx=0).stop 2024-09-13T08:53:59.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.103+0000 7f6172600640 1 -- 172.21.15.44:0/697907269 shutdown_connections 2024-09-13T08:53:59.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.103+0000 7f6172600640 1 --2- 172.21.15.44:0/697907269 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f616c15e740 0x7f616c15ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.103+0000 7f6172600640 1 --2- 172.21.15.44:0/697907269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f616c15e360 0x7f616c0adf50 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.103+0000 7f6172600640 1 -- 172.21.15.44:0/697907269 >> 172.21.15.44:0/697907269 conn(0x7f616c0cfbc0 msgr2=0x7f616c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:59.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.103+0000 7f6172600640 1 -- 172.21.15.44:0/697907269 shutdown_connections 2024-09-13T08:53:59.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.103+0000 7f6172600640 1 -- 172.21.15.44:0/697907269 wait complete. 2024-09-13T08:53:59.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6172600640 1 Processor -- start 2024-09-13T08:53:59.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6172600640 1 -- start start 2024-09-13T08:53:59.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6172600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f616c0aef60 0x7f616c1fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:59.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6172600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f616c15e360 0x7f616c159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:59.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6172600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f616c15e740 0x7f616c141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:59.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6172600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f616c1680b0 con 0x7f616c15e740 2024-09-13T08:53:59.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6172600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f616c167f30 con 0x7f616c0aef60 2024-09-13T08:53:59.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6172600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f616c168230 con 0x7f616c15e360 2024-09-13T08:53:59.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f616be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f616c15e740 0x7f616c141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:59.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f616be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f616c15e740 0x7f616c141380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60240/0 (socket says 172.21.15.44:60240) 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f616be00640 1 -- 172.21.15.44:0/2893367594 learned_addr learned my addr 172.21.15.44:0/2893367594 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f616b400640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f616c0aef60 0x7f616c1fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6162a00640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f616c15e360 0x7f616c159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f616be00640 1 -- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f616c15e360 msgr2=0x7f616c159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f616be00640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f616c15e360 0x7f616c159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f616be00640 1 -- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f616c0aef60 msgr2=0x7f616c1fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f616be00640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f616c0aef60 0x7f616c1fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f616be00640 1 -- 172.21.15.44:0/2893367594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c1fc200 con 0x7f616c15e740 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f616be00640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f616c15e740 0x7f616c141380 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f616400b9a0 tx=0x7f616400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6163e00640 1 -- 172.21.15.44:0/2893367594 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164019070 con 0x7f616c15e740 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6163e00640 1 -- 172.21.15.44:0/2893367594 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f616400cc50 con 0x7f616c15e740 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6163e00640 1 -- 172.21.15.44:0/2893367594 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61640146a0 con 0x7f616c15e740 2024-09-13T08:53:59.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6172600640 1 -- 172.21.15.44:0/2893367594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616c1f80d0 con 0x7f616c15e740 2024-09-13T08:53:59.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.107+0000 7f6172600640 1 -- 172.21.15.44:0/2893367594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616c1f8690 con 0x7f616c15e740 2024-09-13T08:53:59.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.111+0000 7f6172600640 1 -- 172.21.15.44:0/2893367594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616c15ebd0 con 0x7f616c15e740 2024-09-13T08:53:59.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.111+0000 7f6163e00640 1 -- 172.21.15.44:0/2893367594 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 57) ==== 100293+0+0 (secure 0 0 0) 0x7f616401c070 con 0x7f616c15e740 2024-09-13T08:53:59.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.111+0000 7f6163e00640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6148075230 0x7f61480776f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:59.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.111+0000 7f6163e00640 1 -- 172.21.15.44:0/2893367594 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f6164092bf0 con 0x7f616c15e740 2024-09-13T08:53:59.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.111+0000 7f616b400640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6148075230 0x7f61480776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:59.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.111+0000 7f616b400640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6148075230 0x7f61480776f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f615c0af000 tx=0x7f615c01b090 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:59.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.115+0000 7f6163e00640 1 -- 172.21.15.44:0/2893367594 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f616c1fc200 con 0x7f616c15e740 2024-09-13T08:53:59.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.319+0000 7f6172600640 1 -- 172.21.15.44:0/2893367594 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f616c0b4230 con 0x7f6148075230 2024-09-13T08:53:59.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6163e00640 1 -- 172.21.15.44:0/2893367594 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f616c0b4230 con 0x7f6148075230 2024-09-13T08:53:59.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6172600640 1 -- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6148075230 msgr2=0x7f61480776f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:59.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6172600640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6148075230 0x7f61480776f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f615c0af000 tx=0x7f615c01b090 comp rx=0 tx=0).stop 2024-09-13T08:53:59.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6172600640 1 -- 172.21.15.44:0/2893367594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f616c15e740 msgr2=0x7f616c141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:59.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6172600640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f616c15e740 0x7f616c141380 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f616400b9a0 tx=0x7f616400be60 comp rx=0 tx=0).stop 2024-09-13T08:53:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6172600640 1 -- 172.21.15.44:0/2893367594 shutdown_connections 2024-09-13T08:53:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6172600640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6148075230 0x7f61480776f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6172600640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f616c15e740 0x7f616c141380 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6172600640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f616c15e360 0x7f616c159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6172600640 1 --2- 172.21.15.44:0/2893367594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f616c0aef60 0x7f616c1fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.335+0000 7f6172600640 1 -- 172.21.15.44:0/2893367594 >> 172.21.15.44:0/2893367594 conn(0x7f616c0cfbc0 msgr2=0x7f616c164f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.339+0000 7f6172600640 1 -- 172.21.15.44:0/2893367594 shutdown_connections 2024-09-13T08:53:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.339+0000 7f6172600640 1 -- 172.21.15.44:0/2893367594 wait complete. 2024-09-13T08:53:59.358 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-13T08:53:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.507+0000 7f5908a00640 1 -- 172.21.15.44:0/1167564649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59040cd100 msgr2=0x7f59040b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.507+0000 7f5908a00640 1 --2- 172.21.15.44:0/1167564649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59040cd100 0x7f59040b42d0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5904070d30 tx=0x7f58ec01db60 comp rx=0 tx=0).stop 2024-09-13T08:53:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.507+0000 7f5908a00640 1 -- 172.21.15.44:0/1167564649 shutdown_connections 2024-09-13T08:53:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.507+0000 7f5908a00640 1 --2- 172.21.15.44:0/1167564649 >> v2:172.21.15.129:3300/0 conn(0x7f59040b4810 0x7f59040ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.507+0000 7f5908a00640 1 --2- 172.21.15.44:0/1167564649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59040cd100 0x7f59040b42d0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.507+0000 7f5908a00640 1 --2- 172.21.15.44:0/1167564649 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f590415dae0 0x7f59040acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.507+0000 7f5908a00640 1 -- 172.21.15.44:0/1167564649 >> 172.21.15.44:0/1167564649 conn(0x7f59040b18f0 msgr2=0x7f59040b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.507+0000 7f5908a00640 1 -- 172.21.15.44:0/1167564649 shutdown_connections 2024-09-13T08:53:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.507+0000 7f5908a00640 1 -- 172.21.15.44:0/1167564649 wait complete. 2024-09-13T08:53:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 Processor -- start 2024-09-13T08:53:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 -- start start 2024-09-13T08:53:59.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59040b4810 0x7f5904147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:59.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59040cd100 0x7f590406fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:59.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f590415dae0 0x7f590406ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:59.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59040cf440 con 0x7f59040b4810 2024-09-13T08:53:59.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f59040cf2c0 con 0x7f59040cd100 2024-09-13T08:53:59.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f590406e7a0 con 0x7f590415dae0 2024-09-13T08:53:59.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5901e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59040b4810 0x7f5904147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:59.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5901e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59040b4810 0x7f5904147d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60244/0 (socket says 172.21.15.44:60244) 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5901e00640 1 -- 172.21.15.44:0/2338819508 learned_addr learned my addr 172.21.15.44:0/2338819508 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f58fbe00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59040cd100 0x7f590406fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5902800640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f590415dae0 0x7f590406ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5901e00640 1 -- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f590415dae0 msgr2=0x7f590406ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5901e00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f590415dae0 0x7f590406ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5901e00640 1 -- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59040cd100 msgr2=0x7f590406fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5901e00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59040cd100 0x7f590406fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5901e00640 1 -- 172.21.15.44:0/2338819508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59041fadb0 con 0x7f59040b4810 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f58fbe00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59040cd100 0x7f590406fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5901e00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59040b4810 0x7f5904147d90 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f58f400e7c0 tx=0x7f58f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f58fa000640 1 -- 172.21.15.44:0/2338819508 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f400ccc0 con 0x7f59040b4810 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f58fa000640 1 -- 172.21.15.44:0/2338819508 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58f400ce80 con 0x7f59040b4810 2024-09-13T08:53:59.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f58fa000640 1 -- 172.21.15.44:0/2338819508 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4005280 con 0x7f59040b4810 2024-09-13T08:53:59.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 -- 172.21.15.44:0/2338819508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59041fb090 con 0x7f59040b4810 2024-09-13T08:53:59.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 -- 172.21.15.44:0/2338819508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59041fb650 con 0x7f59040b4810 2024-09-13T08:53:59.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.511+0000 7f5908a00640 1 -- 172.21.15.44:0/2338819508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5904069410 con 0x7f59040b4810 2024-09-13T08:53:59.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.515+0000 7f58fa000640 1 -- 172.21.15.44:0/2338819508 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 57) ==== 100293+0+0 (secure 0 0 0) 0x7f58f4005420 con 0x7f59040b4810 2024-09-13T08:53:59.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.515+0000 7f58fa000640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58e0075230 0x7f58e00776f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:53:59.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.515+0000 7f58fa000640 1 -- 172.21.15.44:0/2338819508 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f58f40939e0 con 0x7f59040b4810 2024-09-13T08:53:59.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.515+0000 7f58fbe00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58e0075230 0x7f58e00776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:53:59.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.515+0000 7f58fbe00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58e0075230 0x7f58e00776f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f58ec0b0000 tx=0x7f58ec01c3a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:53:59.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.519+0000 7f58fa000640 1 -- 172.21.15.44:0/2338819508 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f58f4060690 con 0x7f59040b4810 2024-09-13T08:53:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.739+0000 7f5908a00640 1 -- 172.21.15.44:0/2338819508 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f59040b42d0 con 0x7f58e0075230 2024-09-13T08:53:59.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.743+0000 7f58fa000640 1 -- 172.21.15.44:0/2338819508 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f59040b42d0 con 0x7f58e0075230 2024-09-13T08:53:59.746 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883 2024-09-13T08:53:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 -- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58e0075230 msgr2=0x7f58e00776f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58e0075230 0x7f58e00776f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f58ec0b0000 tx=0x7f58ec01c3a0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 -- 172.21.15.44:0/2338819508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59040b4810 msgr2=0x7f5904147d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:53:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59040b4810 0x7f5904147d90 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f58f400e7c0 tx=0x7f58f400ec80 comp rx=0 tx=0).stop 2024-09-13T08:53:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 -- 172.21.15.44:0/2338819508 shutdown_connections 2024-09-13T08:53:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f58e0075230 0x7f58e00776f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f590415dae0 0x7f590406ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59040cd100 0x7f590406fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 --2- 172.21.15.44:0/2338819508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59040b4810 0x7f5904147d90 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:53:59.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 -- 172.21.15.44:0/2338819508 >> 172.21.15.44:0/2338819508 conn(0x7f59040b18f0 msgr2=0x7f590406ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:53:59.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 -- 172.21.15.44:0/2338819508 shutdown_connections 2024-09-13T08:53:59.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:53:59.747+0000 7f5908a00640 1 -- 172.21.15.44:0/2338819508 wait complete. 2024-09-13T08:53:59.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:53:59.767 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883 2024-09-13T08:53:59.767 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:53:59.777 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883 2024-09-13T08:53:59.777 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3055190016" 2024-09-13T08:53:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:53:59.778 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-13T08:54:01.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:01.779 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883 2024-09-13T08:54:01.779 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:54:01.790 INFO:teuthology.orchestra.run.smithi044.stdout:# file: volumes/_nogroup/sv1/b374d673-38a2-4c98-947a-0213e3f39883 2024-09-13T08:54:01.790 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.rbytes="3221225472" 2024-09-13T08:54:01.790 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:01.791 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-13T08:54:01.791 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone48080 2024-09-13T08:54:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:01.999+0000 7f9008000640 1 -- 172.21.15.44:0/1385223603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90000b1650 msgr2=0x7f90000b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:01.999+0000 7f9008000640 1 --2- 172.21.15.44:0/1385223603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90000b1650 0x7f90000b1a30 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009900 tx=0x7f8fec01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:01.999+0000 7f9008000640 1 -- 172.21.15.44:0/1385223603 shutdown_connections 2024-09-13T08:54:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:01.999+0000 7f9008000640 1 --2- 172.21.15.44:0/1385223603 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90000afb10 0x7f90000affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:01.999+0000 7f9008000640 1 --2- 172.21.15.44:0/1385223603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90000b1650 0x7f90000b1a30 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:01.999+0000 7f9008000640 1 -- 172.21.15.44:0/1385223603 >> 172.21.15.44:0/1385223603 conn(0x7f90000d1dd0 msgr2=0x7f90000d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:01.999+0000 7f9008000640 1 -- 172.21.15.44:0/1385223603 shutdown_connections 2024-09-13T08:54:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:01.999+0000 7f9008000640 1 -- 172.21.15.44:0/1385223603 wait complete. 2024-09-13T08:54:02.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9008000640 1 Processor -- start 2024-09-13T08:54:02.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9008000640 1 -- start start 2024-09-13T08:54:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9008000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90000afb10 0x7f9000149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9008000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90000b1650 0x7f90000c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9008000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001605d0 0x7f90000cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9008000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90000b0ec0 con 0x7f90001605d0 2024-09-13T08:54:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9008000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90000b1430 con 0x7f90000b1650 2024-09-13T08:54:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9008000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9000169540 con 0x7f90000afb10 2024-09-13T08:54:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9006000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001605d0 0x7f90000cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9006000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001605d0 0x7f90000cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60260/0 (socket says 172.21.15.44:60260) 2024-09-13T08:54:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9006000640 1 -- 172.21.15.44:0/2365183685 learned_addr learned my addr 172.21.15.44:0/2365183685 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9005600640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90000afb10 0x7f9000149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9004c00640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90000b1650 0x7f90000c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9006000640 1 -- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90000afb10 msgr2=0x7f9000149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9006000640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90000afb10 0x7f9000149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9006000640 1 -- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90000b1650 msgr2=0x7f90000c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9006000640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90000b1650 0x7f90000c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9006000640 1 -- 172.21.15.44:0/2365183685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90000c87d0 con 0x7f90001605d0 2024-09-13T08:54:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9006000640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001605d0 0x7f90000cd680 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800ea20 tx=0x7f8ff800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f8ffd600640 1 -- 172.21.15.44:0/2365183685 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800cde0 con 0x7f90001605d0 2024-09-13T08:54:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f8ffd600640 1 -- 172.21.15.44:0/2365183685 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ff8004c40 con 0x7f90001605d0 2024-09-13T08:54:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f8ffd600640 1 -- 172.21.15.44:0/2365183685 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8005280 con 0x7f90001605d0 2024-09-13T08:54:02.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9008000640 1 -- 172.21.15.44:0/2365183685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90000c8210 con 0x7f90001605d0 2024-09-13T08:54:02.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.003+0000 7f9008000640 1 -- 172.21.15.44:0/2365183685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900015cb40 con 0x7f90001605d0 2024-09-13T08:54:02.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.007+0000 7f9008000640 1 -- 172.21.15.44:0/2365183685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90000affc0 con 0x7f90001605d0 2024-09-13T08:54:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.007+0000 7f8ffd600640 1 -- 172.21.15.44:0/2365183685 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 57) ==== 100293+0+0 (secure 0 0 0) 0x7f8ff8005420 con 0x7f90001605d0 2024-09-13T08:54:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.007+0000 7f8ffd600640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fe0075230 0x7f8fe00776f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.007+0000 7f8ffd600640 1 -- 172.21.15.44:0/2365183685 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f8ff8093990 con 0x7f90001605d0 2024-09-13T08:54:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.007+0000 7f9005600640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fe0075230 0x7f8fe00776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:02.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.011+0000 7f9005600640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fe0075230 0x7f8fe00776f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4007fd0 tx=0x7f8ff4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:02.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.011+0000 7f8ffd600640 1 -- 172.21.15.44:0/2365183685 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8ff8016030 con 0x7f90001605d0 2024-09-13T08:54:02.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.219+0000 7f9008000640 1 -- 172.21.15.44:0/2365183685 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone48080", "target": ["mon-mgr", ""]}) -- 0x7f90000cc290 con 0x7f8fe0075230 2024-09-13T08:54:02.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f8ffd600640 1 -- 172.21.15.44:0/2365183685 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90000cc290 con 0x7f8fe0075230 2024-09-13T08:54:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 -- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fe0075230 msgr2=0x7f8fe00776f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fe0075230 0x7f8fe00776f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4007fd0 tx=0x7f8ff4009040 comp rx=0 tx=0).stop 2024-09-13T08:54:02.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 -- 172.21.15.44:0/2365183685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001605d0 msgr2=0x7f90000cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:02.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001605d0 0x7f90000cd680 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800ea20 tx=0x7f8ff800eee0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 -- 172.21.15.44:0/2365183685 shutdown_connections 2024-09-13T08:54:02.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8fe0075230 0x7f8fe00776f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001605d0 0x7f90000cd680 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90000b1650 0x7f90000c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 --2- 172.21.15.44:0/2365183685 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90000afb10 0x7f9000149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 -- 172.21.15.44:0/2365183685 >> 172.21.15.44:0/2365183685 conn(0x7f90000d1dd0 msgr2=0x7f900015b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:02.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 -- 172.21.15.44:0/2365183685 shutdown_connections 2024-09-13T08:54:02.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.283+0000 7f9008000640 1 -- 172.21.15.44:0/2365183685 wait complete. 2024-09-13T08:54:02.289 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:02.288+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:02.289 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:02.288+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:02.289 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:02.288+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:02.290 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:02.288+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:02.290 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:02.288+0000 7f8aca200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:02.308 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone60481 2024-09-13T08:54:02.309 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:02.308+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:02.310 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:02.308+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:02.310 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:02.308+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:02.310 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:02.308+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:02.310 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:02.308+0000 7f8acac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:02.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.459+0000 7f8dd1800640 1 -- 172.21.15.44:0/3848086957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dcc1605d0 msgr2=0x7f8dcc0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:02.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.459+0000 7f8dd1800640 1 --2- 172.21.15.44:0/3848086957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dcc1605d0 0x7f8dcc0d0690 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009900 tx=0x7f8dbc01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.459+0000 7f8dd1800640 1 -- 172.21.15.44:0/3848086957 shutdown_connections 2024-09-13T08:54:02.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.459+0000 7f8dd1800640 1 --2- 172.21.15.44:0/3848086957 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8dcc0b1650 0x7f8dcc0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.459+0000 7f8dd1800640 1 --2- 172.21.15.44:0/3848086957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dcc1605d0 0x7f8dcc0d0690 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.459+0000 7f8dd1800640 1 -- 172.21.15.44:0/3848086957 >> 172.21.15.44:0/3848086957 conn(0x7f8dcc0d1dd0 msgr2=0x7f8dcc0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:02.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 -- 172.21.15.44:0/3848086957 shutdown_connections 2024-09-13T08:54:02.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 -- 172.21.15.44:0/3848086957 wait complete. 2024-09-13T08:54:02.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 Processor -- start 2024-09-13T08:54:02.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 -- start start 2024-09-13T08:54:02.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dcc0aeb00 0x7f8dcc0b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:02.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dcc0b1650 0x7f8dcc154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:02.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8dcc1605d0 0x7f8dcc0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:02.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc0b0810 con 0x7f8dcc0b1650 2024-09-13T08:54:02.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc0b0ee0 con 0x7f8dcc0aeb00 2024-09-13T08:54:02.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8dcc0b1220 con 0x7f8dcc1605d0 2024-09-13T08:54:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dcc0b1650 0x7f8dcc154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dcc0b1650 0x7f8dcc154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57576/0 (socket says 172.21.15.44:57576) 2024-09-13T08:54:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dca000640 1 -- 172.21.15.44:0/1003474720 learned_addr learned my addr 172.21.15.44:0/1003474720 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dcb400640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8dcc1605d0 0x7f8dcc0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dcaa00640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dcc0aeb00 0x7f8dcc0b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dca000640 1 -- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8dcc1605d0 msgr2=0x7f8dcc0b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dca000640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8dcc1605d0 0x7f8dcc0b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dca000640 1 -- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dcc0aeb00 msgr2=0x7f8dcc0b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dca000640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dcc0aeb00 0x7f8dcc0b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dca000640 1 -- 172.21.15.44:0/1003474720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc1570d0 con 0x7f8dcc0b1650 2024-09-13T08:54:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dca000640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dcc0b1650 0x7f8dcc154c80 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f8db800d980 tx=0x7f8db800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dc2a00640 1 -- 172.21.15.44:0/1003474720 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8017070 con 0x7f8dcc0b1650 2024-09-13T08:54:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dc2a00640 1 -- 172.21.15.44:0/1003474720 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8db8009d20 con 0x7f8dcc0b1650 2024-09-13T08:54:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.463+0000 7f8dd1800640 1 -- 172.21.15.44:0/1003474720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dcc0b5290 con 0x7f8dcc0b1650 2024-09-13T08:54:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.467+0000 7f8dc2a00640 1 -- 172.21.15.44:0/1003474720 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db80106c0 con 0x7f8dcc0b1650 2024-09-13T08:54:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.467+0000 7f8dd1800640 1 -- 172.21.15.44:0/1003474720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dcc0b5850 con 0x7f8dcc0b1650 2024-09-13T08:54:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.467+0000 7f8dd1800640 1 -- 172.21.15.44:0/1003474720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dcc0b1a30 con 0x7f8dcc0b1650 2024-09-13T08:54:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.467+0000 7f8dc2a00640 1 -- 172.21.15.44:0/1003474720 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 57) ==== 100293+0+0 (secure 0 0 0) 0x7f8db8010860 con 0x7f8dcc0b1650 2024-09-13T08:54:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.467+0000 7f8dc2a00640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8dac075230 0x7f8dac0776f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.467+0000 7f8dc2a00640 1 -- 172.21.15.44:0/1003474720 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f8db8093380 con 0x7f8dcc0b1650 2024-09-13T08:54:02.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.467+0000 7f8dcaa00640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8dac075230 0x7f8dac0776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:02.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.471+0000 7f8dcaa00640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8dac075230 0x7f8dac0776f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0af000 tx=0x7f8dbc01b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:02.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.471+0000 7f8dc2a00640 1 -- 172.21.15.44:0/1003474720 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8db805f9e0 con 0x7f8dcc0b1650 2024-09-13T08:54:02.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:02.683+0000 7f8dd1800640 1 -- 172.21.15.44:0/1003474720 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone60481", "target": ["mon-mgr", ""]}) -- 0x7f8dcc0cbff0 con 0x7f8dac075230 2024-09-13T08:54:03.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.043+0000 7f8dc2a00640 1 -- 172.21.15.44:0/1003474720 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 58) ==== 100395+0+0 (secure 0 0 0) 0x7f8db805bc50 con 0x7f8dcc0b1650 2024-09-13T08:54:03.250 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:54:03.254 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:54:03.289 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:03.288+0000 7f8a36a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:03.289 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:03.288+0000 7f8a36a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:03.289 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:03.288+0000 7f8a36a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:03.289 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:03.288+0000 7f8a36a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:03.289 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:03.288+0000 7f8a36a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:03.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.375+0000 7f8dc2a00640 1 -- 172.21.15.44:0/1003474720 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8dcc0cbff0 con 0x7f8dac075230 2024-09-13T08:54:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 -- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8dac075230 msgr2=0x7f8dac0776f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8dac075230 0x7f8dac0776f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0af000 tx=0x7f8dbc01b120 comp rx=0 tx=0).stop 2024-09-13T08:54:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 -- 172.21.15.44:0/1003474720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dcc0b1650 msgr2=0x7f8dcc154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dcc0b1650 0x7f8dcc154c80 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f8db800d980 tx=0x7f8db800de40 comp rx=0 tx=0).stop 2024-09-13T08:54:03.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 -- 172.21.15.44:0/1003474720 shutdown_connections 2024-09-13T08:54:03.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8dac075230 0x7f8dac0776f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:03.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8dcc1605d0 0x7f8dcc0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:03.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dcc0b1650 0x7f8dcc154c80 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:03.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 --2- 172.21.15.44:0/1003474720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dcc0aeb00 0x7f8dcc0b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:03.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 -- 172.21.15.44:0/1003474720 >> 172.21.15.44:0/1003474720 conn(0x7f8dcc0d1dd0 msgr2=0x7f8dcc15bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:03.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 -- 172.21.15.44:0/1003474720 shutdown_connections 2024-09-13T08:54:03.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.379+0000 7f8dd1800640 1 -- 172.21.15.44:0/1003474720 wait complete. 2024-09-13T08:54:03.417 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone10682 2024-09-13T08:54:03.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.663+0000 7f7fc8000640 1 -- 172.21.15.44:0/686945791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc015dda0 msgr2=0x7f7fc006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.663+0000 7f7fc8000640 1 --2- 172.21.15.44:0/686945791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc015dda0 0x7f7fc006fca0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0070d80 tx=0x7f7fb401f0b0 comp rx=0 tx=0).stop 2024-09-13T08:54:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.663+0000 7f7fc8000640 1 -- 172.21.15.44:0/686945791 shutdown_connections 2024-09-13T08:54:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.663+0000 7f7fc8000640 1 --2- 172.21.15.44:0/686945791 >> v2:172.21.15.129:3300/0 conn(0x7f7fc00b4590 0x7f7fc00b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.663+0000 7f7fc8000640 1 --2- 172.21.15.44:0/686945791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc015dda0 0x7f7fc006fca0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.663+0000 7f7fc8000640 1 -- 172.21.15.44:0/686945791 >> 172.21.15.44:0/686945791 conn(0x7f7fc00b18f0 msgr2=0x7f7fc00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.663+0000 7f7fc8000640 1 -- 172.21.15.44:0/686945791 shutdown_connections 2024-09-13T08:54:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.667+0000 7f7fc8000640 1 -- 172.21.15.44:0/686945791 wait complete. 2024-09-13T08:54:03.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.667+0000 7f7fc8000640 1 Processor -- start 2024-09-13T08:54:03.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.667+0000 7f7fc8000640 1 -- start start 2024-09-13T08:54:03.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc8000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc00b4590 0x7f7fc01f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:03.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc8000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7fc015dda0 0x7f7fc01f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:03.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc8000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc0161b60 0x7f7fc015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:03.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc8000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc0161fb0 con 0x7f7fc0161b60 2024-09-13T08:54:03.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc8000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc015f450 con 0x7f7fc00b4590 2024-09-13T08:54:03.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc8000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc0162130 con 0x7f7fc015dda0 2024-09-13T08:54:03.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc0161b60 0x7f7fc015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:03.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc0161b60 0x7f7fc015cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57608/0 (socket says 172.21.15.44:57608) 2024-09-13T08:54:03.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc6000640 1 -- 172.21.15.44:0/2588673477 learned_addr learned my addr 172.21.15.44:0/2588673477 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:03.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc6000640 1 -- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7fc015dda0 msgr2=0x7f7fc01f4a40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:54:03.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc6000640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7fc015dda0 0x7f7fc01f4a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:03.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc5600640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc00b4590 0x7f7fc01f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:03.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc6000640 1 -- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc00b4590 msgr2=0x7f7fc01f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:03.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc6000640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc00b4590 0x7f7fc01f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:03.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc6000640 1 -- 172.21.15.44:0/2588673477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc01f4f80 con 0x7f7fc0161b60 2024-09-13T08:54:03.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.671+0000 7f7fc5600640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc00b4590 0x7f7fc01f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:03.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.675+0000 7f7fc6000640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc0161b60 0x7f7fc015cfc0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800c6b0 tx=0x7f7fb800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:03.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.675+0000 7f7fbd600640 1 -- 172.21.15.44:0/2588673477 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8003b80 con 0x7f7fc0161b60 2024-09-13T08:54:03.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.675+0000 7f7fbd600640 1 -- 172.21.15.44:0/2588673477 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fb8003d40 con 0x7f7fc0161b60 2024-09-13T08:54:03.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.675+0000 7f7fbd600640 1 -- 172.21.15.44:0/2588673477 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8005330 con 0x7f7fc0161b60 2024-09-13T08:54:03.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.675+0000 7f7fc8000640 1 -- 172.21.15.44:0/2588673477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc01f5200 con 0x7f7fc0161b60 2024-09-13T08:54:03.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.675+0000 7f7fc8000640 1 -- 172.21.15.44:0/2588673477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc01f57c0 con 0x7f7fc0161b60 2024-09-13T08:54:03.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.675+0000 7f7fc8000640 1 -- 172.21.15.44:0/2588673477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc0069410 con 0x7f7fc0161b60 2024-09-13T08:54:03.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.679+0000 7f7fbd600640 1 -- 172.21.15.44:0/2588673477 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 100395+0+0 (secure 0 0 0) 0x7f7fb80054d0 con 0x7f7fc0161b60 2024-09-13T08:54:03.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.679+0000 7f7fbd600640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fa00753a0 0x7f7fa0077860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:03.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.679+0000 7f7fbd600640 1 -- 172.21.15.44:0/2588673477 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f7fb8092cd0 con 0x7f7fc0161b60 2024-09-13T08:54:03.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.683+0000 7f7fc5600640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fa00753a0 0x7f7fa0077860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:03.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.687+0000 7f7fc5600640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fa00753a0 0x7f7fa0077860 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40b2000 tx=0x7f7fb401d8f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:03.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:03.695+0000 7f7fbd600640 1 -- 172.21.15.44:0/2588673477 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7fc01f4f80 con 0x7f7fc0161b60 2024-09-13T08:54:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.043+0000 7f7fc8000640 1 -- 172.21.15.44:0/2588673477 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone10682", "target": ["mon-mgr", ""]}) -- 0x7f7fc0144ed0 con 0x7f7fa00753a0 2024-09-13T08:54:04.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fbd600640 1 -- 172.21.15.44:0/2588673477 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7fc0144ed0 con 0x7f7fa00753a0 2024-09-13T08:54:04.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 -- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fa00753a0 msgr2=0x7f7fa0077860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:04.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fa00753a0 0x7f7fa0077860 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40b2000 tx=0x7f7fb401d8f0 comp rx=0 tx=0).stop 2024-09-13T08:54:04.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 -- 172.21.15.44:0/2588673477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc0161b60 msgr2=0x7f7fc015cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:04.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc0161b60 0x7f7fc015cfc0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800c6b0 tx=0x7f7fb800cb70 comp rx=0 tx=0).stop 2024-09-13T08:54:04.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 -- 172.21.15.44:0/2588673477 shutdown_connections 2024-09-13T08:54:04.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7fa00753a0 0x7f7fa0077860 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:04.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc0161b60 0x7f7fc015cfc0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7fc015dda0 0x7f7fc01f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 --2- 172.21.15.44:0/2588673477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc00b4590 0x7f7fc01f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 -- 172.21.15.44:0/2588673477 >> 172.21.15.44:0/2588673477 conn(0x7f7fc00b18f0 msgr2=0x7f7fc01607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 -- 172.21.15.44:0/2588673477 shutdown_connections 2024-09-13T08:54:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:04.855+0000 7f7fc8000640 1 -- 172.21.15.44:0/2588673477 wait complete. 2024-09-13T08:54:04.894 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone19493 2024-09-13T08:54:04.913 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:04.912+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:04.913 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:04.912+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:04.913 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:04.912+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:04.913 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:04.912+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:04.913 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:04.912+0000 7f8acb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:05.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.167+0000 7f7c02a00640 1 -- 172.21.15.44:0/1580276453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be00053d0 msgr2=0x7f7be000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:05.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.167+0000 7f7c02a00640 1 --2- 172.21.15.44:0/1580276453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be00053d0 0x7f7be000ba30 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400c4a0 tx=0x7f7bf401f6c0 comp rx=0 tx=0).stop 2024-09-13T08:54:05.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.167+0000 7f7c02a00640 1 -- 172.21.15.44:0/1580276453 shutdown_connections 2024-09-13T08:54:05.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.167+0000 7f7c02a00640 1 --2- 172.21.15.44:0/1580276453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be00053d0 0x7f7be000ba30 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:05.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.167+0000 7f7c02a00640 1 -- 172.21.15.44:0/1580276453 >> 172.21.15.44:0/1580276453 conn(0x7f7bfc0d3f90 msgr2=0x7f7bfc0d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:05.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.167+0000 7f7c02a00640 1 -- 172.21.15.44:0/1580276453 shutdown_connections 2024-09-13T08:54:05.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.167+0000 7f7c02a00640 1 -- 172.21.15.44:0/1580276453 wait complete. 2024-09-13T08:54:05.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c02a00640 1 Processor -- start 2024-09-13T08:54:05.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c02a00640 1 -- start start 2024-09-13T08:54:05.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c02a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7be00053d0 0x7f7bfc1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c02a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bfc0b3480 0x7f7bfc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c02a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0cd090 0x7f7bfc1f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c02a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc0d2ed0 con 0x7f7bfc0cd090 2024-09-13T08:54:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c02a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc0d2d50 con 0x7f7bfc0b3480 2024-09-13T08:54:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c02a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc162530 con 0x7f7be00053d0 2024-09-13T08:54:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c00a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0cd090 0x7f7bfc1f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c00a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0cd090 0x7f7bfc1f4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57634/0 (socket says 172.21.15.44:57634) 2024-09-13T08:54:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c00a00640 1 -- 172.21.15.44:0/3968152009 learned_addr learned my addr 172.21.15.44:0/3968152009 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:05.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7bfbe00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7be00053d0 0x7f7bfc1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:05.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7bfb400640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bfc0b3480 0x7f7bfc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c00a00640 1 -- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7be00053d0 msgr2=0x7f7bfc1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c00a00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7be00053d0 0x7f7bfc1556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c00a00640 1 -- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bfc0b3480 msgr2=0x7f7bfc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c00a00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bfc0b3480 0x7f7bfc0ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c00a00640 1 -- 172.21.15.44:0/3968152009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc155c20 con 0x7f7bfc0cd090 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.171+0000 7f7c00a00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0cd090 0x7f7bfc1f4c30 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f7bf409bfc0 tx=0x7f7bf4006e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.175+0000 7f7bf3e00640 1 -- 172.21.15.44:0/3968152009 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4006290 con 0x7f7bfc0cd090 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.175+0000 7f7bf3e00640 1 -- 172.21.15.44:0/3968152009 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bf4006520 con 0x7f7bfc0cd090 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.175+0000 7f7bf3e00640 1 -- 172.21.15.44:0/3968152009 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4003860 con 0x7f7bfc0cd090 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.175+0000 7f7c02a00640 1 -- 172.21.15.44:0/3968152009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bfc1f5810 con 0x7f7bfc0cd090 2024-09-13T08:54:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.175+0000 7f7c02a00640 1 -- 172.21.15.44:0/3968152009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bfc0cb060 con 0x7f7bfc0cd090 2024-09-13T08:54:05.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.175+0000 7f7c02a00640 1 -- 172.21.15.44:0/3968152009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bfc162a40 con 0x7f7bfc0cd090 2024-09-13T08:54:05.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.175+0000 7f7bf3e00640 1 -- 172.21.15.44:0/3968152009 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 59) ==== 100497+0+0 (secure 0 0 0) 0x7f7bf4003a00 con 0x7f7bfc0cd090 2024-09-13T08:54:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.179+0000 7f7bf3e00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7bd8075470 0x7f7bd8077930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.179+0000 7f7bf3e00640 1 -- 172.21.15.44:0/3968152009 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f7bf4130040 con 0x7f7bfc0cd090 2024-09-13T08:54:05.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.183+0000 7f7bf3e00640 1 -- 172.21.15.44:0/3968152009 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f7bf40c0030 con 0x7f7bfc0cd090 2024-09-13T08:54:05.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.207+0000 7f7bfbe00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7bd8075470 0x7f7bd8077930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.223+0000 7f7bfbe00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7bd8075470 0x7f7bd8077930 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7bec007fd0 tx=0x7f7bec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:05.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:05.543+0000 7f7c02a00640 1 -- 172.21.15.44:0/3968152009 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone19493", "target": ["mon-mgr", ""]}) -- 0x7f7bfc0b9f10 con 0x7f7bd8075470 2024-09-13T08:54:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.507+0000 7f7bf3e00640 1 -- 172.21.15.44:0/3968152009 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7bfc0b9f10 con 0x7f7bd8075470 2024-09-13T08:54:06.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 -- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7bd8075470 msgr2=0x7f7bd8077930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:06.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7bd8075470 0x7f7bd8077930 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7bec007fd0 tx=0x7f7bec009040 comp rx=0 tx=0).stop 2024-09-13T08:54:06.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 -- 172.21.15.44:0/3968152009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0cd090 msgr2=0x7f7bfc1f4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:06.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0cd090 0x7f7bfc1f4c30 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f7bf409bfc0 tx=0x7f7bf4006e50 comp rx=0 tx=0).stop 2024-09-13T08:54:06.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 -- 172.21.15.44:0/3968152009 shutdown_connections 2024-09-13T08:54:06.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7bd8075470 0x7f7bd8077930 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:06.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0cd090 0x7f7bfc1f4c30 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:06.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bfc0b3480 0x7f7bfc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:06.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 --2- 172.21.15.44:0/3968152009 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7be00053d0 0x7f7bfc1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:06.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 -- 172.21.15.44:0/3968152009 >> 172.21.15.44:0/3968152009 conn(0x7f7bfc0d3f90 msgr2=0x7f7bfc0b93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:06.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 -- 172.21.15.44:0/3968152009 shutdown_connections 2024-09-13T08:54:06.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.511+0000 7f7c02a00640 1 -- 172.21.15.44:0/3968152009 wait complete. 2024-09-13T08:54:06.532 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:06.532+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:06.532 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:06.532+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:06.532 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:06.532+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:06.532 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:06.532+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:06.532 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:06.532+0000 7f8ac9800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:06.553 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone43314 2024-09-13T08:54:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.783+0000 7f0756e00640 1 -- 172.21.15.44:0/3768601453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07500cf1d0 msgr2=0x7f0750160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.783+0000 7f0756e00640 1 --2- 172.21.15.44:0/3768601453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07500cf1d0 0x7f0750160490 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f073c009900 tx=0x7f073c01c8b0 comp rx=0 tx=0).stop 2024-09-13T08:54:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.783+0000 7f0756e00640 1 -- 172.21.15.44:0/3768601453 shutdown_connections 2024-09-13T08:54:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.783+0000 7f0756e00640 1 --2- 172.21.15.44:0/3768601453 >> v2:172.21.15.129:3300/0 conn(0x7f075006e7a0 0x7f07500ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.783+0000 7f0756e00640 1 --2- 172.21.15.44:0/3768601453 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f07500cf1d0 0x7f0750160490 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.783+0000 7f0756e00640 1 -- 172.21.15.44:0/3768601453 >> 172.21.15.44:0/3768601453 conn(0x7f07500b18f0 msgr2=0x7f07500b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.783+0000 7f0756e00640 1 -- 172.21.15.44:0/3768601453 shutdown_connections 2024-09-13T08:54:06.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.783+0000 7f0756e00640 1 -- 172.21.15.44:0/3768601453 wait complete. 2024-09-13T08:54:06.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.787+0000 7f0756e00640 1 Processor -- start 2024-09-13T08:54:06.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.787+0000 7f0756e00640 1 -- start start 2024-09-13T08:54:06.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.787+0000 7f0756e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f075006e7a0 0x7f075015bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:06.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.787+0000 7f0756e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07500cf1d0 0x7f0750152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:06.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.787+0000 7f0756e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075015f9a0 0x7f0750158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:06.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.787+0000 7f0756e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0750169200 con 0x7f07500cf1d0 2024-09-13T08:54:06.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.787+0000 7f0756e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0750169080 con 0x7f075015f9a0 2024-09-13T08:54:06.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.787+0000 7f0756e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0750169380 con 0x7f075006e7a0 2024-09-13T08:54:06.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f074f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07500cf1d0 0x7f0750152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:06.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f074f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07500cf1d0 0x7f0750152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57646/0 (socket says 172.21.15.44:57646) 2024-09-13T08:54:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f074f400640 1 -- 172.21.15.44:0/1829482993 learned_addr learned my addr 172.21.15.44:0/1829482993 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f074f400640 1 -- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f075006e7a0 msgr2=0x7f075015bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f074f400640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f075006e7a0 0x7f075015bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f074f400640 1 -- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075015f9a0 msgr2=0x7f0750158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f074f400640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075015f9a0 0x7f0750158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f074f400640 1 -- 172.21.15.44:0/1829482993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0750157260 con 0x7f07500cf1d0 2024-09-13T08:54:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f074f400640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07500cf1d0 0x7f0750152610 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f073c0b0040 tx=0x7f073c0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f0747e00640 1 -- 172.21.15.44:0/1829482993 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c0b2040 con 0x7f07500cf1d0 2024-09-13T08:54:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f0756e00640 1 -- 172.21.15.44:0/1829482993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0750157540 con 0x7f07500cf1d0 2024-09-13T08:54:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f0747e00640 1 -- 172.21.15.44:0/1829482993 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f073c003600 con 0x7f07500cf1d0 2024-09-13T08:54:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f0747e00640 1 -- 172.21.15.44:0/1829482993 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c003a50 con 0x7f07500cf1d0 2024-09-13T08:54:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.791+0000 7f0756e00640 1 -- 172.21.15.44:0/1829482993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0750157b00 con 0x7f07500cf1d0 2024-09-13T08:54:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.795+0000 7f0756e00640 1 -- 172.21.15.44:0/1829482993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0750069410 con 0x7f07500cf1d0 2024-09-13T08:54:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.795+0000 7f0747e00640 1 -- 172.21.15.44:0/1829482993 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 59) ==== 100497+0+0 (secure 0 0 0) 0x7f073c0bf060 con 0x7f07500cf1d0 2024-09-13T08:54:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.795+0000 7f0747e00640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0730075470 0x7f0730077930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.795+0000 7f0747e00640 1 -- 172.21.15.44:0/1829482993 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f073c12d1c0 con 0x7f07500cf1d0 2024-09-13T08:54:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.799+0000 7f074fe00640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0730075470 0x7f0730077930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:06.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.803+0000 7f0747e00640 1 -- 172.21.15.44:0/1829482993 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f073c0164d0 con 0x7f07500cf1d0 2024-09-13T08:54:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:06.803+0000 7f074fe00640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0730075470 0x7f0730077930 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0740009650 tx=0x7f0740006bb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:07.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:07.195+0000 7f0756e00640 1 -- 172.21.15.44:0/1829482993 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone43314", "target": ["mon-mgr", ""]}) -- 0x7f0750070a60 con 0x7f0730075470 2024-09-13T08:54:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:07.231+0000 7f0747e00640 1 -- 172.21.15.44:0/1829482993 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 60) ==== 100548+0+0 (secure 0 0 0) 0x7f073c0f5a20 con 0x7f07500cf1d0 2024-09-13T08:54:08.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.575+0000 7f0747e00640 1 -- 172.21.15.44:0/1829482993 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0750070a60 con 0x7f0730075470 2024-09-13T08:54:08.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 -- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0730075470 msgr2=0x7f0730077930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:08.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0730075470 0x7f0730077930 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0740009650 tx=0x7f0740006bb0 comp rx=0 tx=0).stop 2024-09-13T08:54:08.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 -- 172.21.15.44:0/1829482993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07500cf1d0 msgr2=0x7f0750152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07500cf1d0 0x7f0750152610 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f073c0b0040 tx=0x7f073c0040e0 comp rx=0 tx=0).stop 2024-09-13T08:54:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 -- 172.21.15.44:0/1829482993 shutdown_connections 2024-09-13T08:54:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0730075470 0x7f0730077930 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075015f9a0 0x7f0750158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07500cf1d0 0x7f0750152610 secure :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f073c0b0040 tx=0x7f073c0040e0 comp rx=0 tx=0).stop 2024-09-13T08:54:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 --2- 172.21.15.44:0/1829482993 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f075006e7a0 0x7f075015bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 -- 172.21.15.44:0/1829482993 >> 172.21.15.44:0/1829482993 conn(0x7f07500b18f0 msgr2=0x7f0750152ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 -- 172.21.15.44:0/1829482993 shutdown_connections 2024-09-13T08:54:08.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.583+0000 7f0756e00640 1 -- 172.21.15.44:0/1829482993 wait complete. 2024-09-13T08:54:08.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone67425 2024-09-13T08:54:08.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.883+0000 7f4379800640 1 -- 172.21.15.44:0/1499360964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43741614d0 msgr2=0x7f43740d1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.883+0000 7f4379800640 1 --2- 172.21.15.44:0/1499360964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43741614d0 0x7f43740d1710 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f43740cfbe0 tx=0x7f436401c650 comp rx=0 tx=0).stop 2024-09-13T08:54:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.883+0000 7f4379800640 1 -- 172.21.15.44:0/1499360964 shutdown_connections 2024-09-13T08:54:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.883+0000 7f4379800640 1 --2- 172.21.15.44:0/1499360964 >> v2:172.21.15.129:3300/0 conn(0x7f43741618b0 0x7f43740aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.883+0000 7f4379800640 1 --2- 172.21.15.44:0/1499360964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43741614d0 0x7f43740d1710 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.883+0000 7f4379800640 1 -- 172.21.15.44:0/1499360964 >> 172.21.15.44:0/1499360964 conn(0x7f43740d1dc0 msgr2=0x7f43740d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.883+0000 7f4379800640 1 -- 172.21.15.44:0/1499360964 shutdown_connections 2024-09-13T08:54:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.883+0000 7f4379800640 1 -- 172.21.15.44:0/1499360964 wait complete. 2024-09-13T08:54:08.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4379800640 1 Processor -- start 2024-09-13T08:54:08.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4379800640 1 -- start start 2024-09-13T08:54:08.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4379800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437406dd30 0x7f437414ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:08.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4379800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43741614d0 0x7f4374158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:08.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4379800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43741618b0 0x7f437414f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:08.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4379800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f437416b140 con 0x7f43741618b0 2024-09-13T08:54:08.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4379800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f437416afc0 con 0x7f437406dd30 2024-09-13T08:54:08.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4379800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f437416b2c0 con 0x7f43741614d0 2024-09-13T08:54:08.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4373400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43741618b0 0x7f437414f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:08.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4373400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43741618b0 0x7f437414f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57672/0 (socket says 172.21.15.44:57672) 2024-09-13T08:54:08.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4373400640 1 -- 172.21.15.44:0/1527722253 learned_addr learned my addr 172.21.15.44:0/1527722253 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4373400640 1 -- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43741614d0 msgr2=0x7f4374158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4373400640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43741614d0 0x7f4374158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4373400640 1 -- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437406dd30 msgr2=0x7f437414ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4373400640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437406dd30 0x7f437414ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4373400640 1 -- 172.21.15.44:0/1527722253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43741552b0 con 0x7f43741618b0 2024-09-13T08:54:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4373400640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43741618b0 0x7f437414f500 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c7c0 tx=0x7f436c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f436aa00640 1 -- 172.21.15.44:0/1527722253 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c003b80 con 0x7f43741618b0 2024-09-13T08:54:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f436aa00640 1 -- 172.21.15.44:0/1527722253 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f436c003d20 con 0x7f43741618b0 2024-09-13T08:54:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f436aa00640 1 -- 172.21.15.44:0/1527722253 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c00aab0 con 0x7f43741618b0 2024-09-13T08:54:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4379800640 1 -- 172.21.15.44:0/1527722253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43741585e0 con 0x7f43741618b0 2024-09-13T08:54:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.887+0000 7f4379800640 1 -- 172.21.15.44:0/1527722253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4374155830 con 0x7f43741618b0 2024-09-13T08:54:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.891+0000 7f4379800640 1 -- 172.21.15.44:0/1527722253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43740d1710 con 0x7f43741618b0 2024-09-13T08:54:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.891+0000 7f436aa00640 1 -- 172.21.15.44:0/1527722253 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 60) ==== 100548+0+0 (secure 0 0 0) 0x7f436c00ac50 con 0x7f43741618b0 2024-09-13T08:54:08.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.895+0000 7f436aa00640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4354075530 0x7f43540779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:08.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.895+0000 7f436aa00640 1 -- 172.21.15.44:0/1527722253 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f436c092fd0 con 0x7f43741618b0 2024-09-13T08:54:08.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.899+0000 7f436aa00640 1 -- 172.21.15.44:0/1527722253 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f436c05fc80 con 0x7f43741618b0 2024-09-13T08:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:08.951+0000 7f4372a00640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4354075530 0x7f43540779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:09.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:09.023+0000 7f4372a00640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4354075530 0x7f43540779f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f43640af000 tx=0x7f43640b2040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:09.287+0000 7f4379800640 1 -- 172.21.15.44:0/1527722253 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone67425", "target": ["mon-mgr", ""]}) -- 0x7f4374072860 con 0x7f4354075530 2024-09-13T08:54:10.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.283+0000 7f436aa00640 1 -- 172.21.15.44:0/1527722253 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4374072860 con 0x7f4354075530 2024-09-13T08:54:10.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.283+0000 7f4379800640 1 -- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4354075530 msgr2=0x7f43540779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:10.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4354075530 0x7f43540779f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f43640af000 tx=0x7f43640b2040 comp rx=0 tx=0).stop 2024-09-13T08:54:10.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 -- 172.21.15.44:0/1527722253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43741618b0 msgr2=0x7f437414f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:10.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43741618b0 0x7f437414f500 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c7c0 tx=0x7f436c00cc80 comp rx=0 tx=0).stop 2024-09-13T08:54:10.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 -- 172.21.15.44:0/1527722253 shutdown_connections 2024-09-13T08:54:10.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4354075530 0x7f43540779f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:10.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43741618b0 0x7f437414f500 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:10.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f43741614d0 0x7f4374158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:10.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 --2- 172.21.15.44:0/1527722253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437406dd30 0x7f437414ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:10.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 -- 172.21.15.44:0/1527722253 >> 172.21.15.44:0/1527722253 conn(0x7f43740d1dc0 msgr2=0x7f4374157990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:10.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 -- 172.21.15.44:0/1527722253 shutdown_connections 2024-09-13T08:54:10.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.287+0000 7f4379800640 1 -- 172.21.15.44:0/1527722253 wait complete. 2024-09-13T08:54:10.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone73666 2024-09-13T08:54:10.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.615+0000 7fc98a800640 1 -- 172.21.15.44:0/2225273723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc968003a60 msgr2=0x7fc968003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:10.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.615+0000 7fc98a800640 1 --2- 172.21.15.44:0/2225273723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc968003a60 0x7fc968003e60 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fc97400c6e0 tx=0x7fc97401c680 comp rx=0 tx=0).stop 2024-09-13T08:54:10.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.615+0000 7fc98a800640 1 -- 172.21.15.44:0/2225273723 shutdown_connections 2024-09-13T08:54:10.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.615+0000 7fc98a800640 1 --2- 172.21.15.44:0/2225273723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc968003a60 0x7fc968003e60 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:10.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.615+0000 7fc98a800640 1 -- 172.21.15.44:0/2225273723 >> 172.21.15.44:0/2225273723 conn(0x7fc9840b18f0 msgr2=0x7fc9840b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:10.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.615+0000 7fc98a800640 1 -- 172.21.15.44:0/2225273723 shutdown_connections 2024-09-13T08:54:10.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.615+0000 7fc98a800640 1 -- 172.21.15.44:0/2225273723 wait complete. 2024-09-13T08:54:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.623+0000 7fc98a800640 1 Processor -- start 2024-09-13T08:54:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.623+0000 7fc98a800640 1 -- start start 2024-09-13T08:54:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc98a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc968003a60 0x7fc98409c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc98a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98415ebf0 0x7fc9840b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc98a800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc9840b0740 0x7fc9841f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc98a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc98406fbd0 con 0x7fc98415ebf0 2024-09-13T08:54:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc98a800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841622e0 con 0x7fc968003a60 2024-09-13T08:54:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc98a800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc98406fd50 con 0x7fc9840b0740 2024-09-13T08:54:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc983400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc968003a60 0x7fc98409c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc983400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc968003a60 0x7fc98409c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:54310/0 (socket says 172.21.15.44:54310) 2024-09-13T08:54:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc983400640 1 -- 172.21.15.44:0/3070603910 learned_addr learned my addr 172.21.15.44:0/3070603910 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:10.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc983400640 1 -- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc9840b0740 msgr2=0x7fc9841f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:10.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc983400640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc9840b0740 0x7fc9841f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:10.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc983400640 1 -- 172.21.15.44:0/3070603910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98415ebf0 msgr2=0x7fc9840b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:10.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.627+0000 7fc983400640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98415ebf0 0x7fc9840b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:10.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.631+0000 7fc983400640 1 -- 172.21.15.44:0/3070603910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9841f5010 con 0x7fc968003a60 2024-09-13T08:54:10.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.639+0000 7fc983400640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc968003a60 0x7fc98409c6f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc97400c630 tx=0x7fc974003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:10.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.639+0000 7fc97b400640 1 -- 172.21.15.44:0/3070603910 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9740ad040 con 0x7fc968003a60 2024-09-13T08:54:10.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.639+0000 7fc97b400640 1 -- 172.21.15.44:0/3070603910 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc974003e20 con 0x7fc968003a60 2024-09-13T08:54:10.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.639+0000 7fc98a800640 1 -- 172.21.15.44:0/3070603910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841f52f0 con 0x7fc968003a60 2024-09-13T08:54:10.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.643+0000 7fc97b400640 1 -- 172.21.15.44:0/3070603910 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974006910 con 0x7fc968003a60 2024-09-13T08:54:10.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.643+0000 7fc98a800640 1 -- 172.21.15.44:0/3070603910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841f58b0 con 0x7fc968003a60 2024-09-13T08:54:10.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.643+0000 7fc98a800640 1 -- 172.21.15.44:0/3070603910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9840b0b40 con 0x7fc968003a60 2024-09-13T08:54:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.655+0000 7fc97b400640 1 -- 172.21.15.44:0/3070603910 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 60) ==== 100548+0+0 (secure 0 0 0) 0x7fc9740ba060 con 0x7fc968003a60 2024-09-13T08:54:10.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.655+0000 7fc97b400640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc960075500 0x7fc9600779c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:10.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.655+0000 7fc97b400640 1 -- 172.21.15.44:0/3070603910 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fc97412c6f0 con 0x7fc968003a60 2024-09-13T08:54:10.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.667+0000 7fc97b400640 1 -- 172.21.15.44:0/3070603910 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc974016390 con 0x7fc968003a60 2024-09-13T08:54:10.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.711+0000 7fc97be00640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc960075500 0x7fc9600779c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:10.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:10.723+0000 7fc97be00640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc960075500 0x7fc9600779c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0046e0 tx=0x7fc96c000f60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:11.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:11.091+0000 7fc98a800640 1 -- 172.21.15.44:0/3070603910 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone73666", "target": ["mon-mgr", ""]}) -- 0x7fc9840b1d70 con 0x7fc960075500 2024-09-13T08:54:11.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:11.995+0000 7fc97b400640 1 -- 172.21.15.44:0/3070603910 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9840b1d70 con 0x7fc960075500 2024-09-13T08:54:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 -- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc960075500 msgr2=0x7fc9600779c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc960075500 0x7fc9600779c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0046e0 tx=0x7fc96c000f60 comp rx=0 tx=0).stop 2024-09-13T08:54:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 -- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc968003a60 msgr2=0x7fc98409c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc968003a60 0x7fc98409c6f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc97400c630 tx=0x7fc974003c30 comp rx=0 tx=0).stop 2024-09-13T08:54:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 -- 172.21.15.44:0/3070603910 shutdown_connections 2024-09-13T08:54:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc960075500 0x7fc9600779c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc9840b0740 0x7fc9841f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98415ebf0 0x7fc9840b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 --2- 172.21.15.44:0/3070603910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc968003a60 0x7fc98409c6f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 -- 172.21.15.44:0/3070603910 >> 172.21.15.44:0/3070603910 conn(0x7fc9840b18f0 msgr2=0x7fc9840b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 -- 172.21.15.44:0/3070603910 shutdown_connections 2024-09-13T08:54:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:12.003+0000 7fc98a800640 1 -- 172.21.15.44:0/3070603910 wait complete. 2024-09-13T08:54:13.051 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone48080 2024-09-13T08:54:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.327+0000 7fc8c7e00640 1 -- 172.21.15.44:0/3955061683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8ac004ea0 msgr2=0x7fc8ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.327+0000 7fc8c7e00640 1 --2- 172.21.15.44:0/3955061683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8ac004ea0 0x7fc8ac00b9d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80d1b70 tx=0x7fc8c001c8d0 comp rx=0 tx=0).stop 2024-09-13T08:54:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.327+0000 7fc8c7400640 1 -- 172.21.15.44:0/3955061683 reap_dead start 2024-09-13T08:54:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.327+0000 7fc8c7e00640 1 -- 172.21.15.44:0/3955061683 shutdown_connections 2024-09-13T08:54:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.327+0000 7fc8c7e00640 1 -- 172.21.15.44:0/3955061683 >> 172.21.15.44:0/3955061683 conn(0x7fc8c80b2680 msgr2=0x7fc8c80b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.327+0000 7fc8c7e00640 1 -- 172.21.15.44:0/3955061683 shutdown_connections 2024-09-13T08:54:13.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.327+0000 7fc8c7e00640 1 -- 172.21.15.44:0/3955061683 wait complete. 2024-09-13T08:54:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.331+0000 7fc8c7e00640 1 Processor -- start 2024-09-13T08:54:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.331+0000 7fc8c7e00640 1 -- start start 2024-09-13T08:54:13.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8ac004ea0 0x7fc8c80c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:13.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c80c7b80 0x7fc8c80c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:13.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c81511d0 0x7fc8c80ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:13.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c8074880 con 0x7fc8c81511d0 2024-09-13T08:54:13.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c8074700 con 0x7fc8ac004ea0 2024-09-13T08:54:13.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c80cd970 con 0x7fc8c80c7b80 2024-09-13T08:54:13.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c81511d0 0x7fc8c80ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:13.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c81511d0 0x7fc8c80ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54540/0 (socket says 172.21.15.44:54540) 2024-09-13T08:54:13.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7400640 1 -- 172.21.15.44:0/37858869 learned_addr learned my addr 172.21.15.44:0/37858869 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c6000640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c80c7b80 0x7fc8c80c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7400640 1 -- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c80c7b80 msgr2=0x7fc8c80c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7400640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c80c7b80 0x7fc8c80c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7400640 1 -- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8ac004ea0 msgr2=0x7fc8c80c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7400640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8ac004ea0 0x7fc8c80c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7400640 1 -- 172.21.15.44:0/37858869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c8071110 con 0x7fc8c81511d0 2024-09-13T08:54:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7400640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c81511d0 0x7fc8c80ca270 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00b0040 tx=0x7fc8c0005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8bea00640 1 -- 172.21.15.44:0/37858869 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00b2040 con 0x7fc8c81511d0 2024-09-13T08:54:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7e00640 1 -- 172.21.15.44:0/37858869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c80c9700 con 0x7fc8c81511d0 2024-09-13T08:54:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8c7e00640 1 -- 172.21.15.44:0/37858869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c80c8400 con 0x7fc8c81511d0 2024-09-13T08:54:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8bea00640 1 -- 172.21.15.44:0/37858869 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8c00a5950 con 0x7fc8c81511d0 2024-09-13T08:54:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.335+0000 7fc8bea00640 1 -- 172.21.15.44:0/37858869 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00be030 con 0x7fc8c81511d0 2024-09-13T08:54:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.339+0000 7fc8c7e00640 1 -- 172.21.15.44:0/37858869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a00052a0 con 0x7fc8c81511d0 2024-09-13T08:54:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.343+0000 7fc8bea00640 1 -- 172.21.15.44:0/37858869 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 60) ==== 100548+0+0 (secure 0 0 0) 0x7fc8c00a5b20 con 0x7fc8c81511d0 2024-09-13T08:54:13.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.343+0000 7fc8bea00640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc8a8075530 0x7fc8a80779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:13.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.343+0000 7fc8bea00640 1 -- 172.21.15.44:0/37858869 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fc8c012cd30 con 0x7fc8c81511d0 2024-09-13T08:54:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.351+0000 7fc8bea00640 1 -- 172.21.15.44:0/37858869 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc8c0016640 con 0x7fc8c81511d0 2024-09-13T08:54:13.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.383+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc8a8075530 0x7fc8a80779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:13.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.411+0000 7fc8c6a00640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc8a8075530 0x7fc8a80779f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8007fd0 tx=0x7fc8b80079a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:13.787+0000 7fc8c7e00640 1 -- 172.21.15.44:0/37858869 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone48080", "target": ["mon-mgr", ""]}) -- 0x7fc8a0000bd0 con 0x7fc8a8075530 2024-09-13T08:54:13.895 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:13.895+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:13.895 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:13.895+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:13.895 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:13.895+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:13.895 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:13.895+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:13.896 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:13.895+0000 7f8ac3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:15.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.282+0000 7fc8bea00640 1 -- 172.21.15.44:0/37858869 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc8a0000bd0 con 0x7fc8a8075530 2024-09-13T08:54:15.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 -- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc8a8075530 msgr2=0x7fc8a80779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:15.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc8a8075530 0x7fc8a80779f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8007fd0 tx=0x7fc8b80079a0 comp rx=0 tx=0).stop 2024-09-13T08:54:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 -- 172.21.15.44:0/37858869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c81511d0 msgr2=0x7fc8c80ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c81511d0 0x7fc8c80ca270 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00b0040 tx=0x7fc8c0005b70 comp rx=0 tx=0).stop 2024-09-13T08:54:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 -- 172.21.15.44:0/37858869 shutdown_connections 2024-09-13T08:54:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc8a8075530 0x7fc8a80779f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8c81511d0 0x7fc8c80ca270 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:15.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc8c80c7b80 0x7fc8c80c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 --2- 172.21.15.44:0/37858869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8ac004ea0 0x7fc8c80c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 -- 172.21.15.44:0/37858869 >> 172.21.15.44:0/37858869 conn(0x7fc8c80b2680 msgr2=0x7fc8c815e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 -- 172.21.15.44:0/37858869 shutdown_connections 2024-09-13T08:54:15.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.286+0000 7fc8c7e00640 1 -- 172.21.15.44:0/37858869 wait complete. 2024-09-13T08:54:15.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone60481 2024-09-13T08:54:15.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.578+0000 7feea5600640 1 -- 172.21.15.44:0/3819391259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0159ff0 msgr2=0x7feea006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:15.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.578+0000 7feea5600640 1 --2- 172.21.15.44:0/3819391259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0159ff0 0x7feea006f130 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fee980093a0 tx=0x7fee9801dcd0 comp rx=0 tx=0).stop 2024-09-13T08:54:15.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 -- 172.21.15.44:0/3819391259 shutdown_connections 2024-09-13T08:54:15.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 --2- 172.21.15.44:0/3819391259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0159ff0 0x7feea006f130 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:15.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 --2- 172.21.15.44:0/3819391259 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feea0159c10 0x7feea006ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:15.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 -- 172.21.15.44:0/3819391259 >> 172.21.15.44:0/3819391259 conn(0x7feea006cc20 msgr2=0x7feea006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:15.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 -- 172.21.15.44:0/3819391259 shutdown_connections 2024-09-13T08:54:15.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 -- 172.21.15.44:0/3819391259 wait complete. 2024-09-13T08:54:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 Processor -- start 2024-09-13T08:54:15.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 -- start start 2024-09-13T08:54:15.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea00705d0 0x7feea014e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feea0159c10 0x7feea013fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feea014ade0 0x7feea0151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feea01671f0 con 0x7feea00705d0 2024-09-13T08:54:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feea0167070 con 0x7feea0159c10 2024-09-13T08:54:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7feea5600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7feea0167370 con 0x7feea014ade0 2024-09-13T08:54:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7fee9fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea00705d0 0x7feea014e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7fee9fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea00705d0 0x7feea014e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54554/0 (socket says 172.21.15.44:54554) 2024-09-13T08:54:15.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.582+0000 7fee9fe00640 1 -- 172.21.15.44:0/1146606077 learned_addr learned my addr 172.21.15.44:0/1146606077 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:15.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7fee9fe00640 1 -- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feea014ade0 msgr2=0x7feea0151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:15.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7fee9fe00640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feea014ade0 0x7feea0151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:15.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7fee9fe00640 1 -- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feea0159c10 msgr2=0x7feea013fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:15.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7fee9fe00640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feea0159c10 0x7feea013fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:15.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7fee9fe00640 1 -- 172.21.15.44:0/1146606077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea014ef10 con 0x7feea00705d0 2024-09-13T08:54:15.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7fee9fe00640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea00705d0 0x7feea014e9d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00b580 tx=0x7fee8c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:15.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7fee97e00640 1 -- 172.21.15.44:0/1146606077 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c014070 con 0x7feea00705d0 2024-09-13T08:54:15.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7fee97e00640 1 -- 172.21.15.44:0/1146606077 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee8c010980 con 0x7feea00705d0 2024-09-13T08:54:15.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7feea5600640 1 -- 172.21.15.44:0/1146606077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea012fa20 con 0x7feea00705d0 2024-09-13T08:54:15.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7fee97e00640 1 -- 172.21.15.44:0/1146606077 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c010dd0 con 0x7feea00705d0 2024-09-13T08:54:15.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7feea5600640 1 -- 172.21.15.44:0/1146606077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea0130260 con 0x7feea00705d0 2024-09-13T08:54:15.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.586+0000 7feea5600640 1 -- 172.21.15.44:0/1146606077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea00b2f00 con 0x7feea00705d0 2024-09-13T08:54:15.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.590+0000 7fee97e00640 1 -- 172.21.15.44:0/1146606077 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fee8c017070 con 0x7feea00705d0 2024-09-13T08:54:15.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.590+0000 7fee97e00640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fee80075590 0x7fee80077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:15.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.590+0000 7fee97e00640 1 -- 172.21.15.44:0/1146606077 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7fee8c093430 con 0x7feea00705d0 2024-09-13T08:54:15.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.594+0000 7fee97e00640 1 -- 172.21.15.44:0/1146606077 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7feea012fa20 con 0x7feea00705d0 2024-09-13T08:54:15.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.602+0000 7fee9f400640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fee80075590 0x7fee80077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:15.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.854+0000 7fee9f400640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fee80075590 0x7fee80077a50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7feea00b4a70 tx=0x7fee9801aa70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:15.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:15.966+0000 7feea5600640 1 -- 172.21.15.44:0/1146606077 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone60481", "target": ["mon-mgr", ""]}) -- 0x7feea006f210 con 0x7fee80075590 2024-09-13T08:54:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.178+0000 7fee97e00640 1 -- 172.21.15.44:0/1146606077 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feea006f210 con 0x7fee80075590 2024-09-13T08:54:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 -- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fee80075590 msgr2=0x7fee80077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fee80075590 0x7fee80077a50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7feea00b4a70 tx=0x7fee9801aa70 comp rx=0 tx=0).stop 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 -- 172.21.15.44:0/1146606077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea00705d0 msgr2=0x7feea014e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea00705d0 0x7feea014e9d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00b580 tx=0x7fee8c00ba40 comp rx=0 tx=0).stop 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 -- 172.21.15.44:0/1146606077 shutdown_connections 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fee80075590 0x7fee80077a50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7feea014ade0 0x7feea0151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feea0159c10 0x7feea013fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 --2- 172.21.15.44:0/1146606077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea00705d0 0x7feea014e9d0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 -- 172.21.15.44:0/1146606077 >> 172.21.15.44:0/1146606077 conn(0x7feea006cc20 msgr2=0x7feea014a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 -- 172.21.15.44:0/1146606077 shutdown_connections 2024-09-13T08:54:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.182+0000 7feea5600640 1 -- 172.21.15.44:0/1146606077 wait complete. 2024-09-13T08:54:17.233 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone10682 2024-09-13T08:54:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.494+0000 7f2513e00640 1 -- 172.21.15.44:0/1363599892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c1605d0 msgr2=0x7f250c0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.494+0000 7f2513e00640 1 --2- 172.21.15.44:0/1363599892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c1605d0 0x7f250c0d0690 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f250c073990 tx=0x7f250001c8c0 comp rx=0 tx=0).stop 2024-09-13T08:54:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.494+0000 7f2513e00640 1 -- 172.21.15.44:0/1363599892 shutdown_connections 2024-09-13T08:54:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.494+0000 7f2513e00640 1 --2- 172.21.15.44:0/1363599892 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f250c0aeb00 0x7f250c0aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.494+0000 7f2513e00640 1 --2- 172.21.15.44:0/1363599892 >> v2:172.21.15.129:3300/0 conn(0x7f250c0b1650 0x7f250c0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.494+0000 7f2513e00640 1 --2- 172.21.15.44:0/1363599892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c1605d0 0x7f250c0d0690 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.494+0000 7f2513e00640 1 -- 172.21.15.44:0/1363599892 >> 172.21.15.44:0/1363599892 conn(0x7f250c0d1dd0 msgr2=0x7f250c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.494+0000 7f2513e00640 1 -- 172.21.15.44:0/1363599892 shutdown_connections 2024-09-13T08:54:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.494+0000 7f2513e00640 1 -- 172.21.15.44:0/1363599892 wait complete. 2024-09-13T08:54:17.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2513e00640 1 Processor -- start 2024-09-13T08:54:17.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2513e00640 1 -- start start 2024-09-13T08:54:17.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2513e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f250c0aeb00 0x7f250c0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:17.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2513e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0b1650 0x7f250c0cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:17.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2513e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c1552b0 0x7f250c0c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:17.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2513e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f250c0b1220 con 0x7f250c0b1650 2024-09-13T08:54:17.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2513e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f250c0b0ee0 con 0x7f250c1552b0 2024-09-13T08:54:17.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2513e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f250c0b13a0 con 0x7f250c0aeb00 2024-09-13T08:54:17.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2510a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0b1650 0x7f250c0cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2510a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0b1650 0x7f250c0cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54564/0 (socket says 172.21.15.44:54564) 2024-09-13T08:54:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2510a00640 1 -- 172.21.15.44:0/3018702051 learned_addr learned my addr 172.21.15.44:0/3018702051 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.498+0000 7f2511400640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f250c0aeb00 0x7f250c0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2510a00640 1 -- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f250c0aeb00 msgr2=0x7f250c0b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2510a00640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f250c0aeb00 0x7f250c0b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2510a00640 1 -- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c1552b0 msgr2=0x7f250c0c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2510a00640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c1552b0 0x7f250c0c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2510a00640 1 -- 172.21.15.44:0/3018702051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c15f8b0 con 0x7f250c0b1650 2024-09-13T08:54:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2510a00640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0b1650 0x7f250c0cca40 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f24f800b410 tx=0x7f24f800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2509600640 1 -- 172.21.15.44:0/3018702051 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8015020 con 0x7f250c0b1650 2024-09-13T08:54:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2509600640 1 -- 172.21.15.44:0/3018702051 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24f8011920 con 0x7f250c0b1650 2024-09-13T08:54:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2509600640 1 -- 172.21.15.44:0/3018702051 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8011d70 con 0x7f250c0b1650 2024-09-13T08:54:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2513e00640 1 -- 172.21.15.44:0/3018702051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c0b4dc0 con 0x7f250c0b1650 2024-09-13T08:54:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2513e00640 1 -- 172.21.15.44:0/3018702051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c0b5380 con 0x7f250c0b1650 2024-09-13T08:54:17.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.502+0000 7f2513e00640 1 -- 172.21.15.44:0/3018702051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250c0aefb0 con 0x7f250c0b1650 2024-09-13T08:54:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.506+0000 7f2509600640 1 -- 172.21.15.44:0/3018702051 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7f24f8018020 con 0x7f250c0b1650 2024-09-13T08:54:17.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.506+0000 7f2509600640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24ec075590 0x7f24ec077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:17.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.506+0000 7f2509600640 1 -- 172.21.15.44:0/3018702051 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5821+0+0 (secure 0 0 0) 0x7f24f8093090 con 0x7f250c0b1650 2024-09-13T08:54:17.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.514+0000 7f2509600640 1 -- 172.21.15.44:0/3018702051 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f24f801f020 con 0x7f250c0b1650 2024-09-13T08:54:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.518+0000 7f2511400640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24ec075590 0x7f24ec077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.526+0000 7f2511400640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24ec075590 0x7f24ec077a50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f25000af000 tx=0x7f250001b100 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:17.942+0000 7f2513e00640 1 -- 172.21.15.44:0/3018702051 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone10682", "target": ["mon-mgr", ""]}) -- 0x7f250c0d10b0 con 0x7f24ec075590 2024-09-13T08:54:19.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.974+0000 7f2509600640 1 -- 172.21.15.44:0/3018702051 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f250c0d10b0 con 0x7f24ec075590 2024-09-13T08:54:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.982+0000 7f2513e00640 1 -- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24ec075590 msgr2=0x7f24ec077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.982+0000 7f2513e00640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24ec075590 0x7f24ec077a50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f25000af000 tx=0x7f250001b100 comp rx=0 tx=0).stop 2024-09-13T08:54:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.982+0000 7f2513e00640 1 -- 172.21.15.44:0/3018702051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0b1650 msgr2=0x7f250c0cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.982+0000 7f2513e00640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0b1650 0x7f250c0cca40 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f24f800b410 tx=0x7f24f800b8d0 comp rx=0 tx=0).stop 2024-09-13T08:54:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.986+0000 7f2513e00640 1 -- 172.21.15.44:0/3018702051 shutdown_connections 2024-09-13T08:54:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.986+0000 7f2513e00640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f24ec075590 0x7f24ec077a50 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.986+0000 7f2513e00640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c1552b0 0x7f250c0c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.986+0000 7f2513e00640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0b1650 0x7f250c0cca40 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.986+0000 7f2513e00640 1 --2- 172.21.15.44:0/3018702051 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f250c0aeb00 0x7f250c0b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.986+0000 7f2513e00640 1 -- 172.21.15.44:0/3018702051 >> 172.21.15.44:0/3018702051 conn(0x7f250c0d1dd0 msgr2=0x7f250c0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:19.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.986+0000 7f2513e00640 1 -- 172.21.15.44:0/3018702051 shutdown_connections 2024-09-13T08:54:19.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:19.986+0000 7f2513e00640 1 -- 172.21.15.44:0/3018702051 wait complete. 2024-09-13T08:54:20.031 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone19493 2024-09-13T08:54:20.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.294+0000 7f26b9e00640 1 -- 172.21.15.44:0/3593398780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b40cc650 msgr2=0x7f26b406cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:20.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.294+0000 7f26b9e00640 1 --2- 172.21.15.44:0/3593398780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b40cc650 0x7f26b406cc20 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f26a00098f0 tx=0x7f26a001c850 comp rx=0 tx=0).stop 2024-09-13T08:54:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.294+0000 7f26b9e00640 1 -- 172.21.15.44:0/3593398780 shutdown_connections 2024-09-13T08:54:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.294+0000 7f26b9e00640 1 --2- 172.21.15.44:0/3593398780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b40cc650 0x7f26b406cc20 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.294+0000 7f26b9e00640 1 --2- 172.21.15.44:0/3593398780 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26b415dd70 0x7f26b40ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.294+0000 7f26b9e00640 1 -- 172.21.15.44:0/3593398780 >> 172.21.15.44:0/3593398780 conn(0x7f26b406dd30 msgr2=0x7f26b406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.294+0000 7f26b9e00640 1 -- 172.21.15.44:0/3593398780 shutdown_connections 2024-09-13T08:54:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.294+0000 7f26b9e00640 1 -- 172.21.15.44:0/3593398780 wait complete. 2024-09-13T08:54:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b9e00640 1 Processor -- start 2024-09-13T08:54:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b9e00640 1 -- start start 2024-09-13T08:54:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26b40cc650 0x7f26b4159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b415dd70 0x7f26b40ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b9e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26b415f440 0x7f26b4146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b9e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f26b416b160 con 0x7f26b415dd70 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b9e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f26b416afe0 con 0x7f26b40cc650 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b9e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f26b416b2e0 con 0x7f26b415f440 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b415dd70 0x7f26b40ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b415dd70 0x7f26b40ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54588/0 (socket says 172.21.15.44:54588) 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b3e00640 1 -- 172.21.15.44:0/3586992103 learned_addr learned my addr 172.21.15.44:0/3586992103 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b8a00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26b40cc650 0x7f26b4159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b3e00640 1 -- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26b415f440 msgr2=0x7f26b4146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b3e00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26b415f440 0x7f26b4146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b3e00640 1 -- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26b40cc650 msgr2=0x7f26b4159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b3e00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26b40cc650 0x7f26b4159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b3e00640 1 -- 172.21.15.44:0/3586992103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b40caac0 con 0x7f26b415dd70 2024-09-13T08:54:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b3e00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b415dd70 0x7f26b40ca360 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f26a0000c10 tx=0x7f26a0005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b0c00640 1 -- 172.21.15.44:0/3586992103 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a00b2040 con 0x7f26b415dd70 2024-09-13T08:54:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b9e00640 1 -- 172.21.15.44:0/3586992103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b4158dc0 con 0x7f26b415dd70 2024-09-13T08:54:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b0c00640 1 -- 172.21.15.44:0/3586992103 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26a00a5a80 con 0x7f26b415dd70 2024-09-13T08:54:20.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b0c00640 1 -- 172.21.15.44:0/3586992103 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0003600 con 0x7f26b415dd70 2024-09-13T08:54:20.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.298+0000 7f26b9e00640 1 -- 172.21.15.44:0/3586992103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b4159380 con 0x7f26b415dd70 2024-09-13T08:54:20.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.302+0000 7f26b9e00640 1 -- 172.21.15.44:0/3586992103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b406cc20 con 0x7f26b415dd70 2024-09-13T08:54:20.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.302+0000 7f26b0c00640 1 -- 172.21.15.44:0/3586992103 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7f26a00bd030 con 0x7f26b415dd70 2024-09-13T08:54:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.302+0000 7f26b0c00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f26940755c0 0x7f2694077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.306+0000 7f26b0c00640 1 -- 172.21.15.44:0/3586992103 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7f26a012cd60 con 0x7f26b415dd70 2024-09-13T08:54:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.310+0000 7f26b0c00640 1 -- 172.21.15.44:0/3586992103 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f26a00c4060 con 0x7f26b415dd70 2024-09-13T08:54:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.314+0000 7f26b8a00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f26940755c0 0x7f2694077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:20.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.322+0000 7f26b8a00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f26940755c0 0x7f2694077a80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f26a8006fd0 tx=0x7f26a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:20.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:20.650+0000 7f26b9e00640 1 -- 172.21.15.44:0/3586992103 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone19493", "target": ["mon-mgr", ""]}) -- 0x7f26b40d00f0 con 0x7f26940755c0 2024-09-13T08:54:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.822+0000 7f26b0c00640 1 -- 172.21.15.44:0/3586992103 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26b40d00f0 con 0x7f26940755c0 2024-09-13T08:54:21.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.822+0000 7f26b9e00640 1 -- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f26940755c0 msgr2=0x7f2694077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:21.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f26940755c0 0x7f2694077a80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f26a8006fd0 tx=0x7f26a8008040 comp rx=0 tx=0).stop 2024-09-13T08:54:21.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 -- 172.21.15.44:0/3586992103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b415dd70 msgr2=0x7f26b40ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:21.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b415dd70 0x7f26b40ca360 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f26a0000c10 tx=0x7f26a0005e30 comp rx=0 tx=0).stop 2024-09-13T08:54:21.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 -- 172.21.15.44:0/3586992103 shutdown_connections 2024-09-13T08:54:21.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f26940755c0 0x7f2694077a80 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:21.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26b415f440 0x7f26b4146fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:21.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b415dd70 0x7f26b40ca360 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:21.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 --2- 172.21.15.44:0/3586992103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26b40cc650 0x7f26b4159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:21.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 -- 172.21.15.44:0/3586992103 >> 172.21.15.44:0/3586992103 conn(0x7f26b406dd30 msgr2=0x7f26b40ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:21.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 -- 172.21.15.44:0/3586992103 shutdown_connections 2024-09-13T08:54:21.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:21.826+0000 7f26b9e00640 1 -- 172.21.15.44:0/3586992103 wait complete. 2024-09-13T08:54:21.864 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone43314 2024-09-13T08:54:22.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.126+0000 7fb9cb200640 1 -- 172.21.15.44:0/763619877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c406fef0 msgr2=0x7fb9c40d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:22.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.126+0000 7fb9cb200640 1 --2- 172.21.15.44:0/763619877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c406fef0 0x7fb9c40d03c0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009c80 tx=0x7fb9c001cda0 comp rx=0 tx=0).stop 2024-09-13T08:54:22.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.126+0000 7fb9cb200640 1 -- 172.21.15.44:0/763619877 shutdown_connections 2024-09-13T08:54:22.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.126+0000 7fb9cb200640 1 --2- 172.21.15.44:0/763619877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c406fef0 0x7fb9c40d03c0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:22.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.126+0000 7fb9cb200640 1 --2- 172.21.15.44:0/763619877 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9c415be80 0x7fb9c40cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:22.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.126+0000 7fb9cb200640 1 -- 172.21.15.44:0/763619877 >> 172.21.15.44:0/763619877 conn(0x7fb9c406cc20 msgr2=0x7fb9c406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:22.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.126+0000 7fb9cb200640 1 -- 172.21.15.44:0/763619877 shutdown_connections 2024-09-13T08:54:22.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.126+0000 7fb9cb200640 1 -- 172.21.15.44:0/763619877 wait complete. 2024-09-13T08:54:22.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.126+0000 7fb9cb200640 1 Processor -- start 2024-09-13T08:54:22.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.126+0000 7fb9cb200640 1 -- start start 2024-09-13T08:54:22.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9cb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c406fef0 0x7fb9c414d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:22.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9cb200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9c415baa0 0x7fb9c4151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:22.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9cb200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9c415be80 0x7fb9c41fb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:22.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9cb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4168b80 con 0x7fb9c406fef0 2024-09-13T08:54:22.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9cb200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4168a00 con 0x7fb9c415be80 2024-09-13T08:54:22.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9cb200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c4168d00 con 0x7fb9c415baa0 2024-09-13T08:54:22.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9c9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c406fef0 0x7fb9c414d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:22.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9c9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c406fef0 0x7fb9c414d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57614/0 (socket says 172.21.15.44:57614) 2024-09-13T08:54:22.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9c9e00640 1 -- 172.21.15.44:0/1468676963 learned_addr learned my addr 172.21.15.44:0/1468676963 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:22.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9c9e00640 1 -- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9c415baa0 msgr2=0x7fb9c4151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:22.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9c9e00640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9c415baa0 0x7fb9c4151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:22.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9c9e00640 1 -- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9c415be80 msgr2=0x7fb9c41fb150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T08:54:22.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9c9e00640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9c415be80 0x7fb9c41fb150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:22.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9c9e00640 1 -- 172.21.15.44:0/1468676963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c414dad0 con 0x7fb9c406fef0 2024-09-13T08:54:22.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9c9e00640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c406fef0 0x7fb9c414d590 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800c500 tx=0x7fb9b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:22.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9be000640 1 -- 172.21.15.44:0/1468676963 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8017070 con 0x7fb9c406fef0 2024-09-13T08:54:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9be000640 1 -- 172.21.15.44:0/1468676963 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9b80053c0 con 0x7fb9c406fef0 2024-09-13T08:54:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9be000640 1 -- 172.21.15.44:0/1468676963 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8012890 con 0x7fb9c406fef0 2024-09-13T08:54:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9cb200640 1 -- 172.21.15.44:0/1468676963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c415efa0 con 0x7fb9c406fef0 2024-09-13T08:54:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.130+0000 7fb9cb200640 1 -- 172.21.15.44:0/1468676963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c415e9e0 con 0x7fb9c406fef0 2024-09-13T08:54:22.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.134+0000 7fb9cb200640 1 -- 172.21.15.44:0/1468676963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c40b2f00 con 0x7fb9c406fef0 2024-09-13T08:54:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.134+0000 7fb9be000640 1 -- 172.21.15.44:0/1468676963 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fb9b801a070 con 0x7fb9c406fef0 2024-09-13T08:54:22.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.134+0000 7fb9be000640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9a8075590 0x7fb9a8077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:22.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.138+0000 7fb9be000640 1 -- 172.21.15.44:0/1468676963 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7fb9b8092580 con 0x7fb9c406fef0 2024-09-13T08:54:22.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.142+0000 7fb9be000640 1 -- 172.21.15.44:0/1468676963 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb9b805f1b0 con 0x7fb9c406fef0 2024-09-13T08:54:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.154+0000 7fb9c9400640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9a8075590 0x7fb9a8077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:22.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.190+0000 7fb9c9400640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9a8075590 0x7fb9a8077a50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4151dc0 tx=0x7fb9b4006c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:22.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:22.542+0000 7fb9cb200640 1 -- 172.21.15.44:0/1468676963 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone43314", "target": ["mon-mgr", ""]}) -- 0x7fb9c406d340 con 0x7fb9a8075590 2024-09-13T08:54:23.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.414+0000 7fb9be000640 1 -- 172.21.15.44:0/1468676963 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb9c406d340 con 0x7fb9a8075590 2024-09-13T08:54:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.422+0000 7fb9cb200640 1 -- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9a8075590 msgr2=0x7fb9a8077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.422+0000 7fb9cb200640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9a8075590 0x7fb9a8077a50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4151dc0 tx=0x7fb9b4006c00 comp rx=0 tx=0).stop 2024-09-13T08:54:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.422+0000 7fb9cb200640 1 -- 172.21.15.44:0/1468676963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c406fef0 msgr2=0x7fb9c414d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.422+0000 7fb9cb200640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c406fef0 0x7fb9c414d590 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800c500 tx=0x7fb9b800c9c0 comp rx=0 tx=0).stop 2024-09-13T08:54:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.426+0000 7fb9cb200640 1 -- 172.21.15.44:0/1468676963 shutdown_connections 2024-09-13T08:54:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.426+0000 7fb9cb200640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb9a8075590 0x7fb9a8077a50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.426+0000 7fb9cb200640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9c415be80 0x7fb9c41fb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.426+0000 7fb9cb200640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb9c415baa0 0x7fb9c4151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.426+0000 7fb9cb200640 1 --2- 172.21.15.44:0/1468676963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c406fef0 0x7fb9c414d590 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.426+0000 7fb9cb200640 1 -- 172.21.15.44:0/1468676963 >> 172.21.15.44:0/1468676963 conn(0x7fb9c406cc20 msgr2=0x7fb9c414e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.426+0000 7fb9cb200640 1 -- 172.21.15.44:0/1468676963 shutdown_connections 2024-09-13T08:54:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.426+0000 7fb9cb200640 1 -- 172.21.15.44:0/1468676963 wait complete. 2024-09-13T08:54:23.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone67425 2024-09-13T08:54:23.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.698+0000 7faecfe00640 1 -- 172.21.15.44:0/3917467809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faed00ca050 msgr2=0x7faed00ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:23.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.698+0000 7faecfe00640 1 --2- 172.21.15.44:0/3917467809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faed00ca050 0x7faed00ca4c0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7faed00cd490 tx=0x7faec801cd20 comp rx=0 tx=0).stop 2024-09-13T08:54:23.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.698+0000 7faecfe00640 1 -- 172.21.15.44:0/3917467809 shutdown_connections 2024-09-13T08:54:23.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.698+0000 7faecfe00640 1 --2- 172.21.15.44:0/3917467809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faed00ca050 0x7faed00ca4c0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:23.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.698+0000 7faecfe00640 1 --2- 172.21.15.44:0/3917467809 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faed00d0520 0x7faed0157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:23.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.698+0000 7faecfe00640 1 --2- 172.21.15.44:0/3917467809 >> v2:172.21.15.129:3300/0 conn(0x7faed0070700 0x7faed015e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.698+0000 7faecfe00640 1 -- 172.21.15.44:0/3917467809 >> 172.21.15.44:0/3917467809 conn(0x7faed00cfa00 msgr2=0x7faed00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.698+0000 7faecfe00640 1 -- 172.21.15.44:0/3917467809 shutdown_connections 2024-09-13T08:54:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.698+0000 7faecfe00640 1 -- 172.21.15.44:0/3917467809 wait complete. 2024-09-13T08:54:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecfe00640 1 Processor -- start 2024-09-13T08:54:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecfe00640 1 -- start start 2024-09-13T08:54:23.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecfe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faed0070700 0x7faed01fb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecfe00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faed00ca050 0x7faed0151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faed00d0520 0x7faed00725d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faed00ce860 con 0x7faed00d0520 2024-09-13T08:54:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecfe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faed00ce6e0 con 0x7faed0070700 2024-09-13T08:54:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecfe00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7faed006e650 con 0x7faed00ca050 2024-09-13T08:54:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faed00d0520 0x7faed00725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faed00d0520 0x7faed00725d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57636/0 (socket says 172.21.15.44:57636) 2024-09-13T08:54:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecf400640 1 -- 172.21.15.44:0/4167947991 learned_addr learned my addr 172.21.15.44:0/4167947991 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecf400640 1 -- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faed00ca050 msgr2=0x7faed0151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecf400640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faed00ca050 0x7faed0151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecf400640 1 -- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faed0070700 msgr2=0x7faed01fb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecf400640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faed0070700 0x7faed01fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecf400640 1 -- 172.21.15.44:0/4167947991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed0155940 con 0x7faed00d0520 2024-09-13T08:54:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecf400640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faed00d0520 0x7faed00725d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7faec8009bd0 tx=0x7faec8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:23.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faec6a00640 1 -- 172.21.15.44:0/4167947991 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec80bb070 con 0x7faed00d0520 2024-09-13T08:54:23.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecfe00640 1 -- 172.21.15.44:0/4167947991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed01fb6d0 con 0x7faed00d0520 2024-09-13T08:54:23.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faec6a00640 1 -- 172.21.15.44:0/4167947991 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faec8004e00 con 0x7faed00d0520 2024-09-13T08:54:23.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faec6a00640 1 -- 172.21.15.44:0/4167947991 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec80b2040 con 0x7faed00d0520 2024-09-13T08:54:23.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.702+0000 7faecfe00640 1 -- 172.21.15.44:0/4167947991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed01fbc90 con 0x7faed00d0520 2024-09-13T08:54:23.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.706+0000 7faecfe00640 1 -- 172.21.15.44:0/4167947991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed00ca4c0 con 0x7faed00d0520 2024-09-13T08:54:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.706+0000 7faec6a00640 1 -- 172.21.15.44:0/4167947991 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7faec80be070 con 0x7faed00d0520 2024-09-13T08:54:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.710+0000 7faec6a00640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faeb0075590 0x7faeb0077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.710+0000 7faec6a00640 1 -- 172.21.15.44:0/4167947991 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7faec812d210 con 0x7faed00d0520 2024-09-13T08:54:23.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.714+0000 7faecea00640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faeb0075590 0x7faeb0077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:23.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.714+0000 7faec6a00640 1 -- 172.21.15.44:0/4167947991 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7faec80c7080 con 0x7faed00d0520 2024-09-13T08:54:23.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:23.734+0000 7faecea00640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faeb0075590 0x7faeb0077a50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7faec0009730 tx=0x7faec0006ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:24.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.114+0000 7faecfe00640 1 -- 172.21.15.44:0/4167947991 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone67425", "target": ["mon-mgr", ""]}) -- 0x7faed0166240 con 0x7faeb0075590 2024-09-13T08:54:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.762+0000 7faec6a00640 1 -- 172.21.15.44:0/4167947991 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faed0166240 con 0x7faeb0075590 2024-09-13T08:54:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.766+0000 7faecfe00640 1 -- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faeb0075590 msgr2=0x7faeb0077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.766+0000 7faecfe00640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faeb0075590 0x7faeb0077a50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7faec0009730 tx=0x7faec0006ca0 comp rx=0 tx=0).stop 2024-09-13T08:54:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.766+0000 7faecfe00640 1 -- 172.21.15.44:0/4167947991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faed00d0520 msgr2=0x7faed00725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.766+0000 7faecfe00640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faed00d0520 0x7faed00725d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7faec8009bd0 tx=0x7faec8003f60 comp rx=0 tx=0).stop 2024-09-13T08:54:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.766+0000 7faecfe00640 1 -- 172.21.15.44:0/4167947991 shutdown_connections 2024-09-13T08:54:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.766+0000 7faecfe00640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7faeb0075590 0x7faeb0077a50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.766+0000 7faecfe00640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faed00d0520 0x7faed00725d0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.766+0000 7faecfe00640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7faed00ca050 0x7faed0151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.766+0000 7faecfe00640 1 --2- 172.21.15.44:0/4167947991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faed0070700 0x7faed01fb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.766+0000 7faecfe00640 1 -- 172.21.15.44:0/4167947991 >> 172.21.15.44:0/4167947991 conn(0x7faed00cfa00 msgr2=0x7faed015b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.770+0000 7faecfe00640 1 -- 172.21.15.44:0/4167947991 shutdown_connections 2024-09-13T08:54:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:24.770+0000 7faecfe00640 1 -- 172.21.15.44:0/4167947991 wait complete. 2024-09-13T08:54:24.810 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone73666 2024-09-13T08:54:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.006+0000 7fcbb7e00640 1 -- 172.21.15.44:0/2020276606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80b0ce0 msgr2=0x7fcbb80b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.006+0000 7fcbb7e00640 1 --2- 172.21.15.44:0/2020276606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80b0ce0 0x7fcbb80b5160 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8069da0 tx=0x7fcba401c8c0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.006+0000 7fcbb7e00640 1 -- 172.21.15.44:0/2020276606 shutdown_connections 2024-09-13T08:54:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.006+0000 7fcbb7e00640 1 --2- 172.21.15.44:0/2020276606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80b0ce0 0x7fcbb80b5160 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.006+0000 7fcbb7e00640 1 --2- 172.21.15.44:0/2020276606 >> v2:172.21.15.129:3300/0 conn(0x7fcbb80b4d80 0x7fcbb80b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.006+0000 7fcbb7e00640 1 -- 172.21.15.44:0/2020276606 >> 172.21.15.44:0/2020276606 conn(0x7fcbb809aa10 msgr2=0x7fcbb809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.006+0000 7fcbb7e00640 1 -- 172.21.15.44:0/2020276606 shutdown_connections 2024-09-13T08:54:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.006+0000 7fcbb7e00640 1 -- 172.21.15.44:0/2020276606 wait complete. 2024-09-13T08:54:25.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb7e00640 1 Processor -- start 2024-09-13T08:54:25.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb7e00640 1 -- start start 2024-09-13T08:54:25.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbb80b0ce0 0x7fcbb815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:25.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80b4d80 0x7fcbb81f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:25.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb7e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbb80b56a0 0x7fcbb81533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb8168a20 con 0x7fcbb80b4d80 2024-09-13T08:54:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb7e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81688a0 con 0x7fcbb80b0ce0 2024-09-13T08:54:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb7e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb8168ba0 con 0x7fcbb80b56a0 2024-09-13T08:54:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80b4d80 0x7fcbb81f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80b4d80 0x7fcbb81f91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57650/0 (socket says 172.21.15.44:57650) 2024-09-13T08:54:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb6000640 1 -- 172.21.15.44:0/1096249479 learned_addr learned my addr 172.21.15.44:0/1096249479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:25.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb6000640 1 -- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbb80b56a0 msgr2=0x7fcbb81533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:25.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb6000640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbb80b56a0 0x7fcbb81533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb6000640 1 -- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbb80b0ce0 msgr2=0x7fcbb815e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:25.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb6000640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbb80b0ce0 0x7fcbb815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.010+0000 7fcbb6000640 1 -- 172.21.15.44:0/1096249479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb815e810 con 0x7fcbb80b4d80 2024-09-13T08:54:25.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.014+0000 7fcbb6000640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80b4d80 0x7fcbb81f91d0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fcba40194d0 tx=0x7fcba40a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:25.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.014+0000 7fcbaea00640 1 -- 172.21.15.44:0/1096249479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40bc070 con 0x7fcbb80b4d80 2024-09-13T08:54:25.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.014+0000 7fcbaea00640 1 -- 172.21.15.44:0/1096249479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcba4005030 con 0x7fcbb80b4d80 2024-09-13T08:54:25.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.014+0000 7fcbaea00640 1 -- 172.21.15.44:0/1096249479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40b2040 con 0x7fcbb80b4d80 2024-09-13T08:54:25.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.014+0000 7fcbb7e00640 1 -- 172.21.15.44:0/1096249479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb809f6a0 con 0x7fcbb80b4d80 2024-09-13T08:54:25.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.014+0000 7fcbb7e00640 1 -- 172.21.15.44:0/1096249479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb81f8cb0 con 0x7fcbb80b4d80 2024-09-13T08:54:25.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.014+0000 7fcbb7e00640 1 -- 172.21.15.44:0/1096249479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb81f97d0 con 0x7fcbb80b4d80 2024-09-13T08:54:25.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.014+0000 7fcbaea00640 1 -- 172.21.15.44:0/1096249479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fcba4005480 con 0x7fcbb80b4d80 2024-09-13T08:54:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.014+0000 7fcbaea00640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcb98075590 0x7fcb98077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.014+0000 7fcbaea00640 1 -- 172.21.15.44:0/1096249479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7fcba412c880 con 0x7fcbb80b4d80 2024-09-13T08:54:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.018+0000 7fcbb6a00640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcb98075590 0x7fcb98077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.018+0000 7fcbb6a00640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcb98075590 0x7fcb98077a50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0005c50 tx=0x7fcbb0005be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.018+0000 7fcbaea00640 1 -- 172.21.15.44:0/1096249479 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcba40164b0 con 0x7fcbb80b4d80 2024-09-13T08:54:25.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.322+0000 7fcbb7e00640 1 -- 172.21.15.44:0/1096249479 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone73666", "target": ["mon-mgr", ""]}) -- 0x7fcbb80b5160 con 0x7fcb98075590 2024-09-13T08:54:25.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbaea00640 1 -- 172.21.15.44:0/1096249479 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcbb80b5160 con 0x7fcb98075590 2024-09-13T08:54:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 -- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcb98075590 msgr2=0x7fcb98077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcb98075590 0x7fcb98077a50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0005c50 tx=0x7fcbb0005be0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 -- 172.21.15.44:0/1096249479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80b4d80 msgr2=0x7fcbb81f91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80b4d80 0x7fcbb81f91d0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fcba40194d0 tx=0x7fcba40a57b0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 -- 172.21.15.44:0/1096249479 shutdown_connections 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcb98075590 0x7fcb98077a50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcbb80b56a0 0x7fcbb81533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbb80b4d80 0x7fcbb81f91d0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 --2- 172.21.15.44:0/1096249479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbb80b0ce0 0x7fcbb815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 -- 172.21.15.44:0/1096249479 >> 172.21.15.44:0/1096249479 conn(0x7fcbb809aa10 msgr2=0x7fcbb814f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 -- 172.21.15.44:0/1096249479 shutdown_connections 2024-09-13T08:54:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.478+0000 7fcbb7e00640 1 -- 172.21.15.44:0/1096249479 wait complete. 2024-09-13T08:54:25.502 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone48080 2024-09-13T08:54:25.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 -- 172.21.15.44:0/4123345052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32240b4810 msgr2=0x7f32240ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:25.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 --2- 172.21.15.44:0/4123345052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32240b4810 0x7f32240ce810 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f322406aab0 tx=0x7f321c01ccf0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 -- 172.21.15.44:0/4123345052 shutdown_connections 2024-09-13T08:54:25.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 --2- 172.21.15.44:0/4123345052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32240b4810 0x7f32240ce810 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 --2- 172.21.15.44:0/4123345052 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f32240cd100 0x7f32240b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 --2- 172.21.15.44:0/4123345052 >> v2:172.21.15.129:3300/0 conn(0x7f322415dae0 0x7f32240acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 -- 172.21.15.44:0/4123345052 >> 172.21.15.44:0/4123345052 conn(0x7f32240b18f0 msgr2=0x7f32240b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:25.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 -- 172.21.15.44:0/4123345052 shutdown_connections 2024-09-13T08:54:25.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 -- 172.21.15.44:0/4123345052 wait complete. 2024-09-13T08:54:25.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 Processor -- start 2024-09-13T08:54:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.650+0000 7f322ba00640 1 -- start start 2024-09-13T08:54:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f322ba00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f32240b4810 0x7f322414d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f322ba00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32240cd100 0x7f32240af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f322ba00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f322415dae0 0x7f322415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f322ba00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32240cf470 con 0x7f32240cd100 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f322ba00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f32240cf2f0 con 0x7f322415dae0 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f322ba00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f322406e7a0 con 0x7f32240b4810 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3223e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32240cd100 0x7f32240af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3223e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32240cd100 0x7f32240af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57674/0 (socket says 172.21.15.44:57674) 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3223e00640 1 -- 172.21.15.44:0/581839374 learned_addr learned my addr 172.21.15.44:0/581839374 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3229a00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f322415dae0 0x7f322415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3229000640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f32240b4810 0x7f322414d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3223e00640 1 -- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f32240b4810 msgr2=0x7f322414d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3223e00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f32240b4810 0x7f322414d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3223e00640 1 -- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f322415dae0 msgr2=0x7f322415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3223e00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f322415dae0 0x7f322415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3223e00640 1 -- 172.21.15.44:0/581839374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322414da30 con 0x7f32240cd100 2024-09-13T08:54:25.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3223e00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32240cd100 0x7f32240af860 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f321000e9e0 tx=0x7f321000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:25.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3220c00640 1 -- 172.21.15.44:0/581839374 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321000ccb0 con 0x7f32240cd100 2024-09-13T08:54:25.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3220c00640 1 -- 172.21.15.44:0/581839374 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3210004030 con 0x7f32240cd100 2024-09-13T08:54:25.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f322ba00640 1 -- 172.21.15.44:0/581839374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3224159090 con 0x7f32240cd100 2024-09-13T08:54:25.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f3220c00640 1 -- 172.21.15.44:0/581839374 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210004480 con 0x7f32240cd100 2024-09-13T08:54:25.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f322ba00640 1 -- 172.21.15.44:0/581839374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3224159650 con 0x7f32240cd100 2024-09-13T08:54:25.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.654+0000 7f322ba00640 1 -- 172.21.15.44:0/581839374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3224069410 con 0x7f32240cd100 2024-09-13T08:54:25.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.658+0000 7f3220c00640 1 -- 172.21.15.44:0/581839374 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7f3210003600 con 0x7f32240cd100 2024-09-13T08:54:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.658+0000 7f3220c00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f32040755c0 0x7f3204077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.658+0000 7f3220c00640 1 -- 172.21.15.44:0/581839374 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7f3210093640 con 0x7f32240cd100 2024-09-13T08:54:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.658+0000 7f3229000640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f32040755c0 0x7f3204077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:25.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.658+0000 7f3229000640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f32040755c0 0x7f3204077a80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3218009ac0 tx=0x7f32180091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:25.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.662+0000 7f3220c00640 1 -- 172.21.15.44:0/581839374 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f321009f080 con 0x7f32240cd100 2024-09-13T08:54:25.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.874+0000 7f322ba00640 1 -- 172.21.15.44:0/581839374 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone48080", "target": ["mon-mgr", ""]}) -- 0x7f32240b42d0 con 0x7f32040755c0 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.882+0000 7f3220c00640 1 -- 172.21.15.44:0/581839374 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f32240b42d0 con 0x7f32040755c0 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:54:25.886 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:54:25.887 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:54:25.887 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:25.887 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:25.887 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:25.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 -- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f32040755c0 msgr2=0x7f3204077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:25.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f32040755c0 0x7f3204077a80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3218009ac0 tx=0x7f32180091e0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 -- 172.21.15.44:0/581839374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32240cd100 msgr2=0x7f32240af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:25.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32240cd100 0x7f32240af860 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f321000e9e0 tx=0x7f321000eea0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 -- 172.21.15.44:0/581839374 shutdown_connections 2024-09-13T08:54:25.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f32040755c0 0x7f3204077a80 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f322415dae0 0x7f322415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32240cd100 0x7f32240af860 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 --2- 172.21.15.44:0/581839374 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f32240b4810 0x7f322414d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:25.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 -- 172.21.15.44:0/581839374 >> 172.21.15.44:0/581839374 conn(0x7f32240b18f0 msgr2=0x7f32240b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:25.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 -- 172.21.15.44:0/581839374 shutdown_connections 2024-09-13T08:54:25.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:25.886+0000 7f322ba00640 1 -- 172.21.15.44:0/581839374 wait complete. 2024-09-13T08:54:25.908 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:54:25.908 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone60481 2024-09-13T08:54:26.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.054+0000 7fbfc8800640 1 -- 172.21.15.44:0/1615613297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00b1650 msgr2=0x7fbfc00b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.054+0000 7fbfc8800640 1 --2- 172.21.15.44:0/1615613297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00b1650 0x7fbfc00b1a30 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0074840 tx=0x7fbfb001c8a0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.054+0000 7fbfc8800640 1 -- 172.21.15.44:0/1615613297 shutdown_connections 2024-09-13T08:54:26.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.054+0000 7fbfc8800640 1 --2- 172.21.15.44:0/1615613297 >> v2:172.21.15.129:3300/0 conn(0x7fbfc00aeb00 0x7fbfc00aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.054+0000 7fbfc8800640 1 --2- 172.21.15.44:0/1615613297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00b1650 0x7fbfc00b1a30 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.054+0000 7fbfc8800640 1 --2- 172.21.15.44:0/1615613297 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbfc01605d0 0x7fbfc00d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.054+0000 7fbfc8800640 1 -- 172.21.15.44:0/1615613297 >> 172.21.15.44:0/1615613297 conn(0x7fbfc00d1dd0 msgr2=0x7fbfc00d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:26.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.054+0000 7fbfc8800640 1 -- 172.21.15.44:0/1615613297 shutdown_connections 2024-09-13T08:54:26.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.054+0000 7fbfc8800640 1 -- 172.21.15.44:0/1615613297 wait complete. 2024-09-13T08:54:26.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 Processor -- start 2024-09-13T08:54:26.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 -- start start 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00aeb00 0x7fbfc01fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfc00b1650 0x7fbfc0164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbfc01605d0 0x7fbfc006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc00b13a0 con 0x7fbfc00aeb00 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc00b1220 con 0x7fbfc00b1650 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc00afd00 con 0x7fbfc01605d0 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00aeb00 0x7fbfc01fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00aeb00 0x7fbfc01fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57698/0 (socket says 172.21.15.44:57698) 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc5e00640 1 -- 172.21.15.44:0/1658830278 learned_addr learned my addr 172.21.15.44:0/1658830278 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc6800640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbfc01605d0 0x7fbfc006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc5400640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfc00b1650 0x7fbfc0164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc5e00640 1 -- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbfc01605d0 msgr2=0x7fbfc006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc5e00640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbfc01605d0 0x7fbfc006ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc5e00640 1 -- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfc00b1650 msgr2=0x7fbfc0164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc5e00640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfc00b1650 0x7fbfc0164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc5e00640 1 -- 172.21.15.44:0/1658830278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc006e210 con 0x7fbfc00aeb00 2024-09-13T08:54:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc5e00640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00aeb00 0x7fbfc01fb760 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400d750 tx=0x7fbfb400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfba000640 1 -- 172.21.15.44:0/1658830278 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4017070 con 0x7fbfc00aeb00 2024-09-13T08:54:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfba000640 1 -- 172.21.15.44:0/1658830278 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfb4009d20 con 0x7fbfc00aeb00 2024-09-13T08:54:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfba000640 1 -- 172.21.15.44:0/1658830278 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400de60 con 0x7fbfc00aeb00 2024-09-13T08:54:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 -- 172.21.15.44:0/1658830278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc00cbba0 con 0x7fbfc00aeb00 2024-09-13T08:54:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 -- 172.21.15.44:0/1658830278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc006e760 con 0x7fbfc00aeb00 2024-09-13T08:54:26.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.058+0000 7fbfc8800640 1 -- 172.21.15.44:0/1658830278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc00b1a30 con 0x7fbfc00aeb00 2024-09-13T08:54:26.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.062+0000 7fbfba000640 1 -- 172.21.15.44:0/1658830278 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fbfb4004280 con 0x7fbfc00aeb00 2024-09-13T08:54:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.062+0000 7fbfba000640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbfa4075590 0x7fbfa4077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.062+0000 7fbfba000640 1 -- 172.21.15.44:0/1658830278 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7fbfb4093440 con 0x7fbfc00aeb00 2024-09-13T08:54:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.062+0000 7fbfc5400640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbfa4075590 0x7fbfa4077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.062+0000 7fbfc5400640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbfa4075590 0x7fbfa4077a50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00af000 tx=0x7fbfb001b0e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.066+0000 7fbfba000640 1 -- 172.21.15.44:0/1658830278 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fbfb4020020 con 0x7fbfc00aeb00 2024-09-13T08:54:26.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.286+0000 7fbfc8800640 1 -- 172.21.15.44:0/1658830278 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone60481", "target": ["mon-mgr", ""]}) -- 0x7fbfc00d1080 con 0x7fbfa4075590 2024-09-13T08:54:26.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.294+0000 7fbfba000640 1 -- 172.21.15.44:0/1658830278 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fbfc00d1080 con 0x7fbfa4075590 2024-09-13T08:54:26.298 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:26.299 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:54:26.299 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:54:26.299 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:54:26.300 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:54:26.300 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:54:26.300 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:54:26.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:26.300 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:54:26.300 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:54:26.300 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:54:26.300 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:26.300 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:26.301 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:26.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 -- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbfa4075590 msgr2=0x7fbfa4077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbfa4075590 0x7fbfa4077a50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00af000 tx=0x7fbfb001b0e0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 -- 172.21.15.44:0/1658830278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00aeb00 msgr2=0x7fbfc01fb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00aeb00 0x7fbfc01fb760 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400d750 tx=0x7fbfb400dc10 comp rx=0 tx=0).stop 2024-09-13T08:54:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 -- 172.21.15.44:0/1658830278 shutdown_connections 2024-09-13T08:54:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fbfa4075590 0x7fbfa4077a50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fbfc01605d0 0x7fbfc006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfc00b1650 0x7fbfc0164ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 --2- 172.21.15.44:0/1658830278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00aeb00 0x7fbfc01fb760 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 -- 172.21.15.44:0/1658830278 >> 172.21.15.44:0/1658830278 conn(0x7fbfc00d1dd0 msgr2=0x7fbfc00b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 -- 172.21.15.44:0/1658830278 shutdown_connections 2024-09-13T08:54:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.298+0000 7fbfc8800640 1 -- 172.21.15.44:0/1658830278 wait complete. 2024-09-13T08:54:26.321 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:54:26.321 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone10682 2024-09-13T08:54:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.466+0000 7f6b22e00640 1 -- 172.21.15.44:0/1203926956 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1c1605d0 msgr2=0x7f6b1c0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.466+0000 7f6b22e00640 1 --2- 172.21.15.44:0/1203926956 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1c1605d0 0x7f6b1c0d0690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009900 tx=0x7f6b0c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.466+0000 7f6b22e00640 1 -- 172.21.15.44:0/1203926956 shutdown_connections 2024-09-13T08:54:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.466+0000 7f6b22e00640 1 --2- 172.21.15.44:0/1203926956 >> v2:172.21.15.129:3300/0 conn(0x7f6b1c0afb10 0x7f6b1c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.466+0000 7f6b22e00640 1 --2- 172.21.15.44:0/1203926956 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1c1605d0 0x7f6b1c0d0690 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.466+0000 7f6b22e00640 1 -- 172.21.15.44:0/1203926956 >> 172.21.15.44:0/1203926956 conn(0x7f6b1c0d1dd0 msgr2=0x7f6b1c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.466+0000 7f6b22e00640 1 -- 172.21.15.44:0/1203926956 shutdown_connections 2024-09-13T08:54:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.466+0000 7f6b22e00640 1 -- 172.21.15.44:0/1203926956 wait complete. 2024-09-13T08:54:26.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b22e00640 1 Processor -- start 2024-09-13T08:54:26.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b22e00640 1 -- start start 2024-09-13T08:54:26.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b22e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b1c0afb10 0x7f6b1c155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b22e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1c0b1650 0x7f6b1c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b22e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c1605d0 0x7f6b1c0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b22e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0b13a0 con 0x7f6b1c0afb10 2024-09-13T08:54:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b22e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0b1220 con 0x7f6b1c1605d0 2024-09-13T08:54:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b22e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c169540 con 0x7f6b1c0b1650 2024-09-13T08:54:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b1c0afb10 0x7f6b1c155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b1c0afb10 0x7f6b1c155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57714/0 (socket says 172.21.15.44:57714) 2024-09-13T08:54:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b1be00640 1 -- 172.21.15.44:0/4244404682 learned_addr learned my addr 172.21.15.44:0/4244404682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b13e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1c0b1650 0x7f6b1c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b20e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c1605d0 0x7f6b1c0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b1be00640 1 -- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1c0b1650 msgr2=0x7f6b1c1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b1be00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1c0b1650 0x7f6b1c1522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b1be00640 1 -- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c1605d0 msgr2=0x7f6b1c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b1be00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c1605d0 0x7f6b1c0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b1be00640 1 -- 172.21.15.44:0/4244404682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c156680 con 0x7f6b1c0afb10 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b13e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1c0b1650 0x7f6b1c1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b1be00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b1c0afb10 0x7f6b1c155fa0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c01a510 tx=0x7f6b0c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b18c00640 1 -- 172.21.15.44:0/4244404682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0b2040 con 0x7f6b1c0afb10 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b18c00640 1 -- 172.21.15.44:0/4244404682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b0c0bd070 con 0x7f6b1c0afb10 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b18c00640 1 -- 172.21.15.44:0/4244404682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c003600 con 0x7f6b1c0afb10 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b22e00640 1 -- 172.21.15.44:0/4244404682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c156960 con 0x7f6b1c0afb10 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.470+0000 7f6b22e00640 1 -- 172.21.15.44:0/4244404682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c0cc7e0 con 0x7f6b1c0afb10 2024-09-13T08:54:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.474+0000 7f6b22e00640 1 -- 172.21.15.44:0/4244404682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c0affc0 con 0x7f6b1c0afb10 2024-09-13T08:54:26.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.474+0000 7f6b18c00640 1 -- 172.21.15.44:0/4244404682 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7f6b0c0037a0 con 0x7f6b1c0afb10 2024-09-13T08:54:26.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.474+0000 7f6b18c00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af8075590 0x7f6af8077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.474+0000 7f6b18c00640 1 -- 172.21.15.44:0/4244404682 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7f6b0c12d8c0 con 0x7f6b1c0afb10 2024-09-13T08:54:26.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.474+0000 7f6b13e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af8075590 0x7f6af8077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.474+0000 7f6b13e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af8075590 0x7f6af8077a50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c157740 tx=0x7f6b04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:26.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.478+0000 7f6b18c00640 1 -- 172.21.15.44:0/4244404682 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6b0c0164d0 con 0x7f6b1c0afb10 2024-09-13T08:54:26.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.690+0000 7f6b22e00640 1 -- 172.21.15.44:0/4244404682 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone10682", "target": ["mon-mgr", ""]}) -- 0x7f6b1c0cecb0 con 0x7f6af8075590 2024-09-13T08:54:26.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b18c00640 1 -- 172.21.15.44:0/4244404682 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f6b040018d0 con 0x7f6af8075590 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:54:26.705 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:54:26.706 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:26.706 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:26.706 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 -- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af8075590 msgr2=0x7f6af8077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af8075590 0x7f6af8077a50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c157740 tx=0x7f6b04009040 comp rx=0 tx=0).stop 2024-09-13T08:54:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 -- 172.21.15.44:0/4244404682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b1c0afb10 msgr2=0x7f6b1c155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b1c0afb10 0x7f6b1c155fa0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c01a510 tx=0x7f6b0c005d60 comp rx=0 tx=0).stop 2024-09-13T08:54:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 -- 172.21.15.44:0/4244404682 shutdown_connections 2024-09-13T08:54:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6af8075590 0x7f6af8077a50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c1605d0 0x7f6b1c0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6b1c0b1650 0x7f6b1c1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 --2- 172.21.15.44:0/4244404682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b1c0afb10 0x7f6b1c155fa0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 -- 172.21.15.44:0/4244404682 >> 172.21.15.44:0/4244404682 conn(0x7f6b1c0d1dd0 msgr2=0x7f6b1c15bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:26.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 -- 172.21.15.44:0/4244404682 shutdown_connections 2024-09-13T08:54:26.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.702+0000 7f6b22e00640 1 -- 172.21.15.44:0/4244404682 wait complete. 2024-09-13T08:54:26.726 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:54:26.727 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone19493 2024-09-13T08:54:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.874+0000 7fd13e600640 1 -- 172.21.15.44:0/1637247734 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1380cdf20 msgr2=0x7fd1380b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.874+0000 7fd13e600640 1 --2- 172.21.15.44:0/1637247734 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1380cdf20 0x7fd1380b1a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd130009bf0 tx=0x7fd13001cad0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.874+0000 7fd13e600640 1 -- 172.21.15.44:0/1637247734 shutdown_connections 2024-09-13T08:54:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.874+0000 7fd13e600640 1 --2- 172.21.15.44:0/1637247734 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1380cdf20 0x7fd1380b1a30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.874+0000 7fd13e600640 1 -- 172.21.15.44:0/1637247734 >> 172.21.15.44:0/1637247734 conn(0x7fd1380d1e00 msgr2=0x7fd1380d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.874+0000 7fd13e600640 1 -- 172.21.15.44:0/1637247734 shutdown_connections 2024-09-13T08:54:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.874+0000 7fd13e600640 1 -- 172.21.15.44:0/1637247734 wait complete. 2024-09-13T08:54:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.874+0000 7fd13e600640 1 Processor -- start 2024-09-13T08:54:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.874+0000 7fd13e600640 1 -- start start 2024-09-13T08:54:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd13e600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1380b3bb0 0x7fd1380cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd13e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1380cdf20 0x7fd1380b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd13e600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13815f340 0x7fd13814fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd13e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd13816a450 con 0x7fd1380cdf20 2024-09-13T08:54:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd13e600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd13816a2d0 con 0x7fd13815f340 2024-09-13T08:54:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd13e600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd13816a5d0 con 0x7fd1380b3bb0 2024-09-13T08:54:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1380cdf20 0x7fd1380b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1380cdf20 0x7fd1380b6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57724/0 (socket says 172.21.15.44:57724) 2024-09-13T08:54:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12fe00640 1 -- 172.21.15.44:0/586092962 learned_addr learned my addr 172.21.15.44:0/586092962 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd137400640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1380b3bb0 0x7fd1380cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd137e00640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13815f340 0x7fd13814fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12fe00640 1 -- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1380b3bb0 msgr2=0x7fd1380cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12fe00640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1380b3bb0 0x7fd1380cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12fe00640 1 -- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13815f340 msgr2=0x7fd13814fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12fe00640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13815f340 0x7fd13814fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12fe00640 1 -- 172.21.15.44:0/586092962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13815e4e0 con 0x7fd1380cdf20 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd137e00640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13815f340 0x7fd13814fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12fe00640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1380cdf20 0x7fd1380b6bb0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd12000c490 tx=0x7fd12000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12f400640 1 -- 172.21.15.44:0/586092962 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1200052d0 con 0x7fd1380cdf20 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12f400640 1 -- 172.21.15.44:0/586092962 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd120005490 con 0x7fd1380cdf20 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12f400640 1 -- 172.21.15.44:0/586092962 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120012870 con 0x7fd1380cdf20 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd13e600640 1 -- 172.21.15.44:0/586092962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1380cbeb0 con 0x7fd1380cdf20 2024-09-13T08:54:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd13e600640 1 -- 172.21.15.44:0/586092962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13815ea60 con 0x7fd1380cdf20 2024-09-13T08:54:26.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd13e600640 1 -- 172.21.15.44:0/586092962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1380b34a0 con 0x7fd1380cdf20 2024-09-13T08:54:26.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.878+0000 7fd12f400640 1 -- 172.21.15.44:0/586092962 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fd120012a10 con 0x7fd1380cdf20 2024-09-13T08:54:26.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.882+0000 7fd12f400640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd114075590 0x7fd114077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:26.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.882+0000 7fd12f400640 1 -- 172.21.15.44:0/586092962 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7fd120092c30 con 0x7fd1380cdf20 2024-09-13T08:54:26.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.882+0000 7fd137400640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd114075590 0x7fd114077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:26.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.882+0000 7fd137400640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd114075590 0x7fd114077a50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd128001400 tx=0x7fd128009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:26.886+0000 7fd12f400640 1 -- 172.21.15.44:0/586092962 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd12005f980 con 0x7fd1380cdf20 2024-09-13T08:54:27.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.094+0000 7fd13e600640 1 -- 172.21.15.44:0/586092962 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone19493", "target": ["mon-mgr", ""]}) -- 0x7fd1380cc350 con 0x7fd114075590 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.106+0000 7fd12f400640 1 -- 172.21.15.44:0/586092962 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fd1380cc350 con 0x7fd114075590 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:54:27.109 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:54:27.110 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:54:27.110 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:27.110 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:27.110 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:27.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 -- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd114075590 msgr2=0x7fd114077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:27.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd114075590 0x7fd114077a50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd128001400 tx=0x7fd128009040 comp rx=0 tx=0).stop 2024-09-13T08:54:27.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 -- 172.21.15.44:0/586092962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1380cdf20 msgr2=0x7fd1380b6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:27.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1380cdf20 0x7fd1380b6bb0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd12000c490 tx=0x7fd12000c950 comp rx=0 tx=0).stop 2024-09-13T08:54:27.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 -- 172.21.15.44:0/586092962 shutdown_connections 2024-09-13T08:54:27.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd114075590 0x7fd114077a50 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:27.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd13815f340 0x7fd13814fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:27.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1380cdf20 0x7fd1380b6bb0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:27.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 --2- 172.21.15.44:0/586092962 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd1380b3bb0 0x7fd1380cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:27.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 -- 172.21.15.44:0/586092962 >> 172.21.15.44:0/586092962 conn(0x7fd1380d1e00 msgr2=0x7fd1380b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:27.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 -- 172.21.15.44:0/586092962 shutdown_connections 2024-09-13T08:54:27.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.110+0000 7fd13e600640 1 -- 172.21.15.44:0/586092962 wait complete. 2024-09-13T08:54:27.133 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:54:27.133 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone43314 2024-09-13T08:54:27.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.278+0000 7fb626a00640 1 -- 172.21.15.44:0/3141374765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604004d50 msgr2=0x7fb6040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:27.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.278+0000 7fb626a00640 1 --2- 172.21.15.44:0/3141374765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604004d50 0x7fb6040051d0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fb62015c540 tx=0x7fb61801caf0 comp rx=0 tx=0).stop 2024-09-13T08:54:27.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.278+0000 7fb624a00640 1 -- 172.21.15.44:0/3141374765 reap_dead start 2024-09-13T08:54:27.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.278+0000 7fb626a00640 1 -- 172.21.15.44:0/3141374765 shutdown_connections 2024-09-13T08:54:27.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.278+0000 7fb626a00640 1 -- 172.21.15.44:0/3141374765 >> 172.21.15.44:0/3141374765 conn(0x7fb6200d1ea0 msgr2=0x7fb6200d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:27.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.278+0000 7fb626a00640 1 -- 172.21.15.44:0/3141374765 shutdown_connections 2024-09-13T08:54:27.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.278+0000 7fb626a00640 1 -- 172.21.15.44:0/3141374765 wait complete. 2024-09-13T08:54:27.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb626a00640 1 Processor -- start 2024-09-13T08:54:27.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb626a00640 1 -- start start 2024-09-13T08:54:27.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb626a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604004d50 0x7fb6200cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:27.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb626a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6200c8f70 0x7fb6200b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:27.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb626a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6200b7350 0x7fb6200b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:27.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb626a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb6200cc940 con 0x7fb604004d50 2024-09-13T08:54:27.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb626a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6200cf7e0 con 0x7fb6200c8f70 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb626a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb6200ccac0 con 0x7fb6200b7350 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604004d50 0x7fb6200cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604004d50 0x7fb6200cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57748/0 (socket says 172.21.15.44:57748) 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61fe00640 1 -- 172.21.15.44:0/759106771 learned_addr learned my addr 172.21.15.44:0/759106771 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb624a00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6200b7350 0x7fb6200b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb617e00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6200c8f70 0x7fb6200b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61fe00640 1 -- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6200b7350 msgr2=0x7fb6200b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61fe00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6200b7350 0x7fb6200b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61fe00640 1 -- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6200c8f70 msgr2=0x7fb6200b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61fe00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6200c8f70 0x7fb6200b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61fe00640 1 -- 172.21.15.44:0/759106771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6200cb900 con 0x7fb604004d50 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb624a00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6200b7350 0x7fb6200b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb617e00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6200c8f70 0x7fb6200b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:27.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61fe00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604004d50 0x7fb6200cb3c0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fb61000e310 tx=0x7fb61000e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:27.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61cc00640 1 -- 172.21.15.44:0/759106771 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610012850 con 0x7fb604004d50 2024-09-13T08:54:27.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61cc00640 1 -- 172.21.15.44:0/759106771 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb610012b00 con 0x7fb604004d50 2024-09-13T08:54:27.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb626a00640 1 -- 172.21.15.44:0/759106771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62009efa0 con 0x7fb604004d50 2024-09-13T08:54:27.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb61cc00640 1 -- 172.21.15.44:0/759106771 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61001a400 con 0x7fb604004d50 2024-09-13T08:54:27.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.282+0000 7fb626a00640 1 -- 172.21.15.44:0/759106771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62009f560 con 0x7fb604004d50 2024-09-13T08:54:27.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.286+0000 7fb626a00640 1 -- 172.21.15.44:0/759106771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6200ce5e0 con 0x7fb604004d50 2024-09-13T08:54:27.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.286+0000 7fb61cc00640 1 -- 172.21.15.44:0/759106771 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fb61000d030 con 0x7fb604004d50 2024-09-13T08:54:27.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.286+0000 7fb61cc00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5fc0755c0 0x7fb5fc077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:27.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.286+0000 7fb61cc00640 1 -- 172.21.15.44:0/759106771 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7fb610096380 con 0x7fb604004d50 2024-09-13T08:54:27.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.290+0000 7fb617e00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5fc0755c0 0x7fb5fc077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:27.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.290+0000 7fb617e00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5fc0755c0 0x7fb5fc077a80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb6200c4e60 tx=0x7fb6080091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.290+0000 7fb61cc00640 1 -- 172.21.15.44:0/759106771 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb610016030 con 0x7fb604004d50 2024-09-13T08:54:27.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:27.498+0000 7fb626a00640 1 -- 172.21.15.44:0/759106771 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone43314", "target": ["mon-mgr", ""]}) -- 0x7fb6201f4880 con 0x7fb5fc0755c0 2024-09-13T08:54:31.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.126+0000 7fb61cc00640 1 -- 172.21.15.44:0/759106771 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fb6201f4880 con 0x7fb5fc0755c0 2024-09-13T08:54:31.129 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:54:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:54:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:54:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:54:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:54:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:54:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:54:31.129 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:54:31.130 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:54:31.130 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:31.130 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:31.130 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:31.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.126+0000 7fb626a00640 1 -- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5fc0755c0 msgr2=0x7fb5fc077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.126+0000 7fb626a00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5fc0755c0 0x7fb5fc077a80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb6200c4e60 tx=0x7fb6080091e0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.126+0000 7fb626a00640 1 -- 172.21.15.44:0/759106771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604004d50 msgr2=0x7fb6200cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.126+0000 7fb626a00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604004d50 0x7fb6200cb3c0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fb61000e310 tx=0x7fb61000e7d0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.126+0000 7fb626a00640 1 -- 172.21.15.44:0/759106771 shutdown_connections 2024-09-13T08:54:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.130+0000 7fb626a00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb5fc0755c0 0x7fb5fc077a80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.130+0000 7fb626a00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb6200b7350 0x7fb6200b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.130+0000 7fb626a00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6200c8f70 0x7fb6200b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.130+0000 7fb626a00640 1 --2- 172.21.15.44:0/759106771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604004d50 0x7fb6200cb3c0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.130+0000 7fb626a00640 1 -- 172.21.15.44:0/759106771 >> 172.21.15.44:0/759106771 conn(0x7fb6200d1ea0 msgr2=0x7fb6200d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.130+0000 7fb626a00640 1 -- 172.21.15.44:0/759106771 shutdown_connections 2024-09-13T08:54:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.130+0000 7fb626a00640 1 -- 172.21.15.44:0/759106771 wait complete. 2024-09-13T08:54:31.153 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:54:31.153 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone67425 2024-09-13T08:54:31.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.306+0000 7f5fcb200640 1 -- 172.21.15.44:0/2165514637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fcc15bd40 msgr2=0x7f5fb0003a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.306+0000 7f5fcb200640 1 --2- 172.21.15.44:0/2165514637 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fcc15bd40 0x7f5fb0003a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009b60 tx=0x7f5fbc01d2b0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.306+0000 7f5fca800640 1 -- 172.21.15.44:0/2165514637 reap_dead start 2024-09-13T08:54:31.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.306+0000 7f5fcb200640 1 -- 172.21.15.44:0/2165514637 shutdown_connections 2024-09-13T08:54:31.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.306+0000 7f5fcb200640 1 -- 172.21.15.44:0/2165514637 >> 172.21.15.44:0/2165514637 conn(0x7f5fcc09aa10 msgr2=0x7f5fcc09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:31.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.306+0000 7f5fcb200640 1 -- 172.21.15.44:0/2165514637 shutdown_connections 2024-09-13T08:54:31.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.306+0000 7f5fcb200640 1 -- 172.21.15.44:0/2165514637 wait complete. 2024-09-13T08:54:31.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.306+0000 7f5fcb200640 1 Processor -- start 2024-09-13T08:54:31.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.306+0000 7f5fcb200640 1 -- start start 2024-09-13T08:54:31.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fcb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15bd40 0x7f5fcc1f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:31.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fcb200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fcc0c9740 0x7f5fcc15eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:31.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fcb200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fcc0c8da0 0x7f5fcc0aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:31.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fcb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc0d0450 con 0x7f5fcc15bd40 2024-09-13T08:54:31.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fcb200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc0cd820 con 0x7f5fcc0c8da0 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fcb200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5fcc0d05d0 con 0x7f5fcc0c9740 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15bd40 0x7f5fcc1f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15bd40 0x7f5fcc1f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57764/0 (socket says 172.21.15.44:57764) 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc9e00640 1 -- 172.21.15.44:0/1448189085 learned_addr learned my addr 172.21.15.44:0/1448189085 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc1400640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fcc0c9740 0x7f5fcc15eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fca800640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fcc0c8da0 0x7f5fcc0aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc9e00640 1 -- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fcc0c9740 msgr2=0x7f5fcc15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc9e00640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fcc0c9740 0x7f5fcc15eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc9e00640 1 -- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fcc0c8da0 msgr2=0x7f5fcc0aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc9e00640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fcc0c8da0 0x7f5fcc0aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc9e00640 1 -- 172.21.15.44:0/1448189085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc1f5270 con 0x7f5fcc15bd40 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fca800640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fcc0c8da0 0x7f5fcc0aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc1400640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fcc0c9740 0x7f5fcc15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc9e00640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15bd40 0x7f5fcc1f4d30 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc01cd20 tx=0x7f5fbc01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc2a00640 1 -- 172.21.15.44:0/1448189085 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0bd070 con 0x7f5fcc15bd40 2024-09-13T08:54:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc2a00640 1 -- 172.21.15.44:0/1448189085 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fbc0b3040 con 0x7f5fcc15bd40 2024-09-13T08:54:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fc2a00640 1 -- 172.21.15.44:0/1448189085 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0b2070 con 0x7f5fcc15bd40 2024-09-13T08:54:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fcb200640 1 -- 172.21.15.44:0/1448189085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fcc0ca200 con 0x7f5fcc15bd40 2024-09-13T08:54:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fcb200640 1 -- 172.21.15.44:0/1448189085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fcc0c9f20 con 0x7f5fcc15bd40 2024-09-13T08:54:31.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.310+0000 7f5fcb200640 1 -- 172.21.15.44:0/1448189085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fcc0d1bb0 con 0x7f5fcc15bd40 2024-09-13T08:54:31.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.314+0000 7f5fc2a00640 1 -- 172.21.15.44:0/1448189085 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7f5fbc0c0070 con 0x7f5fcc15bd40 2024-09-13T08:54:31.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.314+0000 7f5fc2a00640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8075590 0x7f5fa8077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:31.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.314+0000 7f5fc2a00640 1 -- 172.21.15.44:0/1448189085 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7f5fbc12d810 con 0x7f5fcc15bd40 2024-09-13T08:54:31.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.314+0000 7f5fc1400640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8075590 0x7f5fa8077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:31.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.314+0000 7f5fc1400640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8075590 0x7f5fa8077a50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0c8310 tx=0x7f5fb40091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:31.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.318+0000 7f5fc2a00640 1 -- 172.21.15.44:0/1448189085 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5fcc1f5270 con 0x7f5fcc15bd40 2024-09-13T08:54:31.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.526+0000 7f5fcb200640 1 -- 172.21.15.44:0/1448189085 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone67425", "target": ["mon-mgr", ""]}) -- 0x7f5fcc06f0a0 con 0x7f5fa8075590 2024-09-13T08:54:31.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fc2a00640 1 -- 172.21.15.44:0/1448189085 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f5fcc06f0a0 con 0x7f5fa8075590 2024-09-13T08:54:31.540 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:31.540 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:54:31.540 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:54:31.540 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:54:31.540 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:54:31.540 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:54:31.540 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:54:31.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:31.541 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:54:31.541 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:54:31.541 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:54:31.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:31.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:31.541 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:31.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 -- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8075590 msgr2=0x7f5fa8077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8075590 0x7f5fa8077a50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0c8310 tx=0x7f5fb40091e0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 -- 172.21.15.44:0/1448189085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15bd40 msgr2=0x7f5fcc1f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15bd40 0x7f5fcc1f4d30 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc01cd20 tx=0x7f5fbc01baf0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 -- 172.21.15.44:0/1448189085 shutdown_connections 2024-09-13T08:54:31.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5fa8075590 0x7f5fa8077a50 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fcc0c8da0 0x7f5fcc0aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5fcc0c9740 0x7f5fcc15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 --2- 172.21.15.44:0/1448189085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15bd40 0x7f5fcc1f4d30 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 -- 172.21.15.44:0/1448189085 >> 172.21.15.44:0/1448189085 conn(0x7f5fcc09aa10 msgr2=0x7f5fcc0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:31.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 -- 172.21.15.44:0/1448189085 shutdown_connections 2024-09-13T08:54:31.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.538+0000 7f5fcb200640 1 -- 172.21.15.44:0/1448189085 wait complete. 2024-09-13T08:54:31.562 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:54:31.563 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone73666 2024-09-13T08:54:31.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.710+0000 7f3928200640 1 -- 172.21.15.44:0/2188296227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39080051d0 msgr2=0x7f390800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.710+0000 7f3928200640 1 --2- 172.21.15.44:0/2188296227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39080051d0 0x7f390800b9d0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f3918009eb0 tx=0x7f391801d5f0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.710+0000 7f3928200640 1 -- 172.21.15.44:0/2188296227 shutdown_connections 2024-09-13T08:54:31.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.710+0000 7f3928200640 1 --2- 172.21.15.44:0/2188296227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39080051d0 0x7f390800b9d0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.710+0000 7f3928200640 1 -- 172.21.15.44:0/2188296227 >> 172.21.15.44:0/2188296227 conn(0x7f39200d1d70 msgr2=0x7f39200d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:31.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.710+0000 7f3928200640 1 -- 172.21.15.44:0/2188296227 shutdown_connections 2024-09-13T08:54:31.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.710+0000 7f3928200640 1 -- 172.21.15.44:0/2188296227 wait complete. 2024-09-13T08:54:31.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 Processor -- start 2024-09-13T08:54:31.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 -- start start 2024-09-13T08:54:31.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39080051d0 0x7f39200ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:31.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f392015b6b0 0x7f39200c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:31.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39200c6640 0x7f39200b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:31.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39200d2c20 con 0x7f39080051d0 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39200d2aa0 con 0x7f39200c6640 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f39200d06a0 con 0x7f392015b6b0 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3925800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39080051d0 0x7f39200ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3925800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39080051d0 0x7f39200ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57784/0 (socket says 172.21.15.44:57784) 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3925800640 1 -- 172.21.15.44:0/2371804099 learned_addr learned my addr 172.21.15.44:0/2371804099 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3924e00640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f392015b6b0 0x7f39200c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3926200640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39200c6640 0x7f39200b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3925800640 1 -- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f392015b6b0 msgr2=0x7f39200c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3925800640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f392015b6b0 0x7f39200c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3925800640 1 -- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39200c6640 msgr2=0x7f39200b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3925800640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39200c6640 0x7f39200b6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3925800640 1 -- 172.21.15.44:0/2371804099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39201f5c40 con 0x7f39080051d0 2024-09-13T08:54:31.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3924e00640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f392015b6b0 0x7f39200c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:31.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3925800640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39080051d0 0x7f39200ca150 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f391400e590 tx=0x7f3914004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:31.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f391d600640 1 -- 172.21.15.44:0/2371804099 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914015070 con 0x7f39080051d0 2024-09-13T08:54:31.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f391d600640 1 -- 172.21.15.44:0/2371804099 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39140109a0 con 0x7f39080051d0 2024-09-13T08:54:31.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f391d600640 1 -- 172.21.15.44:0/2371804099 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914010df0 con 0x7f39080051d0 2024-09-13T08:54:31.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 -- 172.21.15.44:0/2371804099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39201f5f20 con 0x7f39080051d0 2024-09-13T08:54:31.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 -- 172.21.15.44:0/2371804099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39201f64e0 con 0x7f39080051d0 2024-09-13T08:54:31.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.714+0000 7f3928200640 1 -- 172.21.15.44:0/2371804099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39200d0b90 con 0x7f39080051d0 2024-09-13T08:54:31.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.718+0000 7f391d600640 1 -- 172.21.15.44:0/2371804099 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7f3914018070 con 0x7f39080051d0 2024-09-13T08:54:31.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.718+0000 7f391d600640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3900075590 0x7f3900077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:31.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.718+0000 7f391d600640 1 -- 172.21.15.44:0/2371804099 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7f39140934b0 con 0x7f39080051d0 2024-09-13T08:54:31.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.718+0000 7f3924e00640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3900075590 0x7f3900077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:31.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.718+0000 7f3924e00640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3900075590 0x7f3900077a50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c10 tx=0x7f390c006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:31.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.722+0000 7f391d600640 1 -- 172.21.15.44:0/2371804099 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f39200cd360 con 0x7f39080051d0 2024-09-13T08:54:31.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.930+0000 7f3928200640 1 -- 172.21.15.44:0/2371804099 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone73666", "target": ["mon-mgr", ""]}) -- 0x7f392015e970 con 0x7f3900075590 2024-09-13T08:54:31.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f391d600640 1 -- 172.21.15.44:0/2371804099 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f392015e970 con 0x7f3900075590 2024-09-13T08:54:31.947 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "status": { 2024-09-13T08:54:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "canceled", 2024-09-13T08:54:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "source": { 2024-09-13T08:54:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "volume": "cephfs", 2024-09-13T08:54:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "subvolume": "sv1", 2024-09-13T08:54:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot": "ss1" 2024-09-13T08:54:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "failure": { 2024-09-13T08:54:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "errno": "4", 2024-09-13T08:54:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "error_msg": "user interrupted clone operation" 2024-09-13T08:54:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:31.948 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:31.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 -- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3900075590 msgr2=0x7f3900077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3900075590 0x7f3900077a50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c10 tx=0x7f390c006b80 comp rx=0 tx=0).stop 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 -- 172.21.15.44:0/2371804099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39080051d0 msgr2=0x7f39200ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39080051d0 0x7f39200ca150 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f391400e590 tx=0x7f3914004f90 comp rx=0 tx=0).stop 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 -- 172.21.15.44:0/2371804099 shutdown_connections 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f3900075590 0x7f3900077a50 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39200c6640 0x7f39200b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f392015b6b0 0x7f39200c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 --2- 172.21.15.44:0/2371804099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39080051d0 0x7f39200ca150 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 -- 172.21.15.44:0/2371804099 >> 172.21.15.44:0/2371804099 conn(0x7f39200d1d70 msgr2=0x7f39200b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 -- 172.21.15.44:0/2371804099 shutdown_connections 2024-09-13T08:54:31.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:31.946+0000 7f3928200640 1 -- 172.21.15.44:0/2371804099 wait complete. 2024-09-13T08:54:31.969 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-13T08:54:31.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-13T08:54:32.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.110+0000 7f65b7000640 1 -- 172.21.15.44:0/3219200578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b00d0530 msgr2=0x7f65b0072a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.110+0000 7f65b7000640 1 --2- 172.21.15.44:0/3219200578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b00d0530 0x7f65b0072a70 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009c80 tx=0x7f65a801cd70 comp rx=0 tx=0).stop 2024-09-13T08:54:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.110+0000 7f65b7000640 1 -- 172.21.15.44:0/3219200578 shutdown_connections 2024-09-13T08:54:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.110+0000 7f65b7000640 1 --2- 172.21.15.44:0/3219200578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b00d0530 0x7f65b0072a70 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.110+0000 7f65b7000640 1 --2- 172.21.15.44:0/3219200578 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65b00cc7d0 0x7f65b00cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.110+0000 7f65b7000640 1 -- 172.21.15.44:0/3219200578 >> 172.21.15.44:0/3219200578 conn(0x7f65b006cdf0 msgr2=0x7f65b006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.110+0000 7f65b7000640 1 -- 172.21.15.44:0/3219200578 shutdown_connections 2024-09-13T08:54:32.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.110+0000 7f65b7000640 1 -- 172.21.15.44:0/3219200578 wait complete. 2024-09-13T08:54:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b7000640 1 Processor -- start 2024-09-13T08:54:32.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b7000640 1 -- start start 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b7000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65b00cc7d0 0x7f65b01f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b7000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b00d0530 0x7f65b0151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b7000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65b015fc20 0x7f65b013b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b7000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65b0168c20 con 0x7f65b00d0530 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b7000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65b0073a60 con 0x7f65b015fc20 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b7000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f65b0168da0 con 0x7f65b00cc7d0 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b00d0530 0x7f65b0151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b00d0530 0x7f65b0151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54900/0 (socket says 172.21.15.44:54900) 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65a7e00640 1 -- 172.21.15.44:0/1992233270 learned_addr learned my addr 172.21.15.44:0/1992233270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b5000640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65b015fc20 0x7f65b013b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65afe00640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65b00cc7d0 0x7f65b01f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65a7e00640 1 -- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65b00cc7d0 msgr2=0x7f65b01f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65a7e00640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65b00cc7d0 0x7f65b01f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65a7e00640 1 -- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65b015fc20 msgr2=0x7f65b013b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65a7e00640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65b015fc20 0x7f65b013b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65a7e00640 1 -- 172.21.15.44:0/1992233270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b0156250 con 0x7f65b00d0530 2024-09-13T08:54:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65a7e00640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b00d0530 0x7f65b0151ff0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f65980058d0 tx=0x7f659800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65acc00640 1 -- 172.21.15.44:0/1992233270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598017070 con 0x7f65b00d0530 2024-09-13T08:54:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65acc00640 1 -- 172.21.15.44:0/1992233270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6598009d20 con 0x7f65b00d0530 2024-09-13T08:54:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65acc00640 1 -- 172.21.15.44:0/1992233270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65980106c0 con 0x7f65b00d0530 2024-09-13T08:54:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b7000640 1 -- 172.21.15.44:0/1992233270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b0156530 con 0x7f65b00d0530 2024-09-13T08:54:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.114+0000 7f65b7000640 1 -- 172.21.15.44:0/1992233270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b0152530 con 0x7f65b00d0530 2024-09-13T08:54:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.118+0000 7f65b7000640 1 -- 172.21.15.44:0/1992233270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b00cfff0 con 0x7f65b00d0530 2024-09-13T08:54:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.118+0000 7f65acc00640 1 -- 172.21.15.44:0/1992233270 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7f659801a070 con 0x7f65b00d0530 2024-09-13T08:54:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.118+0000 7f65acc00640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f658c0755c0 0x7f658c077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.118+0000 7f65acc00640 1 -- 172.21.15.44:0/1992233270 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7f6598093190 con 0x7f65b00d0530 2024-09-13T08:54:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.118+0000 7f65afe00640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f658c0755c0 0x7f658c077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.118+0000 7f65afe00640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f658c0755c0 0x7f658c077a80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f65b01f32f0 tx=0x7f65a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:32.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.122+0000 7f65acc00640 1 -- 172.21.15.44:0/1992233270 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f659805f7f0 con 0x7f65b00d0530 2024-09-13T08:54:32.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 -- 172.21.15.44:0/1992233270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f65b0075300 con 0x7f65b00d0530 2024-09-13T08:54:32.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65acc00640 1 -- 172.21.15.44:0/1992233270 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2400 (secure 0 0 0) 0x7f6598064220 con 0x7f65b00d0530 2024-09-13T08:54:32.364 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:32.364 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "d85be7ac-239d-467b-bce8-dbc7fa81939e", 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: "health": { 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "HEALTH_OK", 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: "checks": {}, 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: "mutes": [] 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2024-09-13T08:54:32.365 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_names": [ 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "a", 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "b", 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "c" 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1043, 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "squid", 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "num_mons": 3 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap": { 2024-09-13T08:54:32.366 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 146, 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: "num_osds": 8, 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: "num_up_osds": 8, 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_up_since": 1726216643, 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: "num_in_osds": 8, 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_in_since": 1726216632, 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: "num_remapped_pgs": 0 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap": { 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: "pgs_by_state": [ 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: "state_name": "active+clean", 2024-09-13T08:54:32.367 INFO:teuthology.orchestra.run.smithi044.stdout: "count": 97 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pgs": 97, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "num_pools": 3, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects": 4267, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "data_bytes": 17752477561, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_used": 36701458432, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_avail": 736392654848, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "bytes_total": 773094113280, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_objects_per_sec": 7, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_bytes_per_sec": 32621074, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_keys_per_sec": 0, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "num_objects_recovered": 95, 2024-09-13T08:54:32.368 INFO:teuthology.orchestra.run.smithi044.stdout: "num_bytes_recovered": 394264576, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "num_keys_recovered": 0, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "read_bytes_sec": 560116125, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "write_bytes_sec": 570243447, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "read_op_per_sec": 138, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "write_op_per_sec": 157 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "fsmap": { 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 96, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "btime": "2024-09-13T08:53:27:902711+0000", 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 10, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1, 2024-09-13T08:54:32.369 INFO:teuthology.orchestra.run.smithi044.stdout: "by_rank": [ 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: "filesystem_id": 10, 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "up:active", 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 9219 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: "up:standby": 3 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: "mgrmap": { 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-09-13T08:54:32.370 INFO:teuthology.orchestra.run.smithi044.stdout: "num_standbys": 1, 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "restful" 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "servicemap": { 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 27, 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T08:54:22.949671+0000", 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "services": {} 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "progress_events": { 2024-09-13T08:54:32.371 INFO:teuthology.orchestra.run.smithi044.stdout: "f28edad2-5e09-4e69-ad16-b30099a37c1c": { 2024-09-13T08:54:32.372 INFO:teuthology.orchestra.run.smithi044.stdout: "message": "Global Recovery Event (5s)\n [===========================.] ", 2024-09-13T08:54:32.372 INFO:teuthology.orchestra.run.smithi044.stdout: "progress": 0.98969072103500366, 2024-09-13T08:54:32.372 INFO:teuthology.orchestra.run.smithi044.stdout: "add_to_ceph_s": true 2024-09-13T08:54:32.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:32.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:32.372 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:32.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 -- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f658c0755c0 msgr2=0x7f658c077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f658c0755c0 0x7f658c077a80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f65b01f32f0 tx=0x7f65a0009040 comp rx=0 tx=0).stop 2024-09-13T08:54:32.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 -- 172.21.15.44:0/1992233270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b00d0530 msgr2=0x7f65b0151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b00d0530 0x7f65b0151ff0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f65980058d0 tx=0x7f659800dde0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 -- 172.21.15.44:0/1992233270 shutdown_connections 2024-09-13T08:54:32.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f658c0755c0 0x7f658c077a80 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65b015fc20 0x7f65b013b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b00d0530 0x7f65b0151ff0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 --2- 172.21.15.44:0/1992233270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f65b00cc7d0 0x7f65b01f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 -- 172.21.15.44:0/1992233270 >> 172.21.15.44:0/1992233270 conn(0x7f65b006cdf0 msgr2=0x7f65b006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:32.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 -- 172.21.15.44:0/1992233270 shutdown_connections 2024-09-13T08:54:32.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.362+0000 7f65b7000640 1 -- 172.21.15.44:0/1992233270 wait complete. 2024-09-13T08:54:32.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:32.385 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone48080/b374d673-38a2-4c98-947a-0213e3f39883 2024-09-13T08:54:32.385 DEBUG:teuthology.orchestra.run.smithi044:> ) 2024-09-13T08:54:32.395 INFO:teuthology.orchestra.run.smithi044.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone48080/b374d673-38a2-4c98-947a-0213e3f39883': No such file or directory 2024-09-13T08:54:32.396 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-13T08:54:32.397 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-13T08:54:32.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.582+0000 7f26a8e00640 1 -- 172.21.15.44:0/809413166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a415bea0 msgr2=0x7f26a40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.582+0000 7f26a8e00640 1 --2- 172.21.15.44:0/809413166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a415bea0 0x7f26a40ce670 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f26a4073d10 tx=0x7f269401c8c0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.582+0000 7f26a8e00640 1 -- 172.21.15.44:0/809413166 shutdown_connections 2024-09-13T08:54:32.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.582+0000 7f26a8e00640 1 --2- 172.21.15.44:0/809413166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26a40d28e0 0x7f26a40cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.582+0000 7f26a8e00640 1 --2- 172.21.15.44:0/809413166 >> v2:172.21.15.129:3300/0 conn(0x7f26a40cdb30 0x7f26a40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.582+0000 7f26a8e00640 1 --2- 172.21.15.44:0/809413166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a415bea0 0x7f26a40ce670 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.582+0000 7f26a8e00640 1 -- 172.21.15.44:0/809413166 >> 172.21.15.44:0/809413166 conn(0x7f26a40d1dc0 msgr2=0x7f26a40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:32.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.582+0000 7f26a8e00640 1 -- 172.21.15.44:0/809413166 shutdown_connections 2024-09-13T08:54:32.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.582+0000 7f26a8e00640 1 -- 172.21.15.44:0/809413166 wait complete. 2024-09-13T08:54:32.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a8e00640 1 Processor -- start 2024-09-13T08:54:32.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a8e00640 1 -- start start 2024-09-13T08:54:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a8e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26a40cdb30 0x7f26a40b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a40d28e0 0x7f26a414bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a8e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26a415bea0 0x7f26a414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a8e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f26a40af070 con 0x7f26a40d28e0 2024-09-13T08:54:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a8e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f26a40aeef0 con 0x7f26a40cdb30 2024-09-13T08:54:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a8e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f26a40af1f0 con 0x7f26a415bea0 2024-09-13T08:54:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f2699400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a40d28e0 0x7f26a414bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f2699400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a40d28e0 0x7f26a414bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54922/0 (socket says 172.21.15.44:54922) 2024-09-13T08:54:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f2699400640 1 -- 172.21.15.44:0/1134469858 learned_addr learned my addr 172.21.15.44:0/1134469858 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:32.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a1e00640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26a40cdb30 0x7f26a40b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a2800640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26a415bea0 0x7f26a414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f2699400640 1 -- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26a415bea0 msgr2=0x7f26a414b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f2699400640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26a415bea0 0x7f26a414b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f2699400640 1 -- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26a40cdb30 msgr2=0x7f26a40b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f2699400640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26a40cdb30 0x7f26a40b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f2699400640 1 -- 172.21.15.44:0/1134469858 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a414bc20 con 0x7f26a40d28e0 2024-09-13T08:54:32.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f2699400640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a40d28e0 0x7f26a414bf00 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f268c00e9f0 tx=0x7f268c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:32.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f269aa00640 1 -- 172.21.15.44:0/1134469858 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c00ccc0 con 0x7f26a40d28e0 2024-09-13T08:54:32.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f269aa00640 1 -- 172.21.15.44:0/1134469858 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f268c010430 con 0x7f26a40d28e0 2024-09-13T08:54:32.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f269aa00640 1 -- 172.21.15.44:0/1134469858 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c010840 con 0x7f26a40d28e0 2024-09-13T08:54:32.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a8e00640 1 -- 172.21.15.44:0/1134469858 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a40b25f0 con 0x7f26a40d28e0 2024-09-13T08:54:32.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.586+0000 7f26a8e00640 1 -- 172.21.15.44:0/1134469858 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a40b2bb0 con 0x7f26a40d28e0 2024-09-13T08:54:32.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.590+0000 7f26a8e00640 1 -- 172.21.15.44:0/1134469858 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a40cc9d0 con 0x7f26a40d28e0 2024-09-13T08:54:32.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.590+0000 7f269aa00640 1 -- 172.21.15.44:0/1134469858 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7f268c0109e0 con 0x7f26a40d28e0 2024-09-13T08:54:32.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.590+0000 7f269aa00640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2680075590 0x7f2680077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.590+0000 7f269aa00640 1 -- 172.21.15.44:0/1134469858 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7f268c093b30 con 0x7f26a40d28e0 2024-09-13T08:54:32.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.590+0000 7f26a1e00640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2680075590 0x7f2680077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.590+0000 7f26a1e00640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2680075590 0x7f2680077a50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f26940af000 tx=0x7f26940a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:32.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.594+0000 7f269aa00640 1 -- 172.21.15.44:0/1134469858 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f268c060230 con 0x7f26a40d28e0 2024-09-13T08:54:32.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.794+0000 7f26a8e00640 1 -- 172.21.15.44:0/1134469858 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f26a40b00e0 con 0x7f2680075590 2024-09-13T08:54:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f269aa00640 1 -- 172.21.15.44:0/1134469858 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f26a40b00e0 con 0x7f2680075590 2024-09-13T08:54:32.837 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:54:32.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:32.838 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-09-13T08:54:32.838 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:32.838 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:54:32.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 -- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2680075590 msgr2=0x7f2680077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2680075590 0x7f2680077a50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f26940af000 tx=0x7f26940a7040 comp rx=0 tx=0).stop 2024-09-13T08:54:32.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 -- 172.21.15.44:0/1134469858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a40d28e0 msgr2=0x7f26a414bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a40d28e0 0x7f26a414bf00 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f268c00e9f0 tx=0x7f268c00eeb0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 -- 172.21.15.44:0/1134469858 shutdown_connections 2024-09-13T08:54:32.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2680075590 0x7f2680077a50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f26a415bea0 0x7f26a414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a40d28e0 0x7f26a414bf00 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 --2- 172.21.15.44:0/1134469858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26a40cdb30 0x7f26a40b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 -- 172.21.15.44:0/1134469858 >> 172.21.15.44:0/1134469858 conn(0x7f26a40d1dc0 msgr2=0x7f26a40d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:32.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 -- 172.21.15.44:0/1134469858 shutdown_connections 2024-09-13T08:54:32.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.798+0000 7f26a8e00640 1 -- 172.21.15.44:0/1134469858 wait complete. 2024-09-13T08:54:32.840 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-13T08:54:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.986+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3588141157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0d1990 msgr2=0x7fd7ac161340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.986+0000 7fd7b3e00640 1 --2- 172.21.15.44:0/3588141157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0d1990 0x7fd7ac161340 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fd798009900 tx=0x7fd79801c790 comp rx=0 tx=0).stop 2024-09-13T08:54:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.986+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3588141157 shutdown_connections 2024-09-13T08:54:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.986+0000 7fd7b3e00640 1 --2- 172.21.15.44:0/3588141157 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd7ac0ced50 0x7fd7ac161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.986+0000 7fd7b3e00640 1 --2- 172.21.15.44:0/3588141157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0d1990 0x7fd7ac161340 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.986+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3588141157 >> 172.21.15.44:0/3588141157 conn(0x7fd7ac0d1e90 msgr2=0x7fd7ac0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.986+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3588141157 shutdown_connections 2024-09-13T08:54:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.986+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3588141157 wait complete. 2024-09-13T08:54:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 Processor -- start 2024-09-13T08:54:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 -- start start 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0ce970 0x7fd7ac0b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd7ac0ced50 0x7fd7ac1fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7ac0d1990 0x7fd7ac0af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac16b190 con 0x7fd7ac0ce970 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac0b2270 con 0x7fd7ac0d1990 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd7ac16b310 con 0x7fd7ac0ced50 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0ce970 0x7fd7ac0b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0ce970 0x7fd7ac0b7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54944/0 (socket says 172.21.15.44:54944) 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b1400640 1 -- 172.21.15.44:0/3612420708 learned_addr learned my addr 172.21.15.44:0/3612420708 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b0a00640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd7ac0ced50 0x7fd7ac1fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b1e00640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7ac0d1990 0x7fd7ac0af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b1400640 1 -- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd7ac0ced50 msgr2=0x7fd7ac1fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b1400640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd7ac0ced50 0x7fd7ac1fbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b1400640 1 -- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7ac0d1990 msgr2=0x7fd7ac0af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b1400640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7ac0d1990 0x7fd7ac0af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b1400640 1 -- 172.21.15.44:0/3612420708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac0b7800 con 0x7fd7ac0ce970 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b1400640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0ce970 0x7fd7ac0b7170 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c530 tx=0x7fd7a000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7a9600640 1 -- 172.21.15.44:0/3612420708 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0017070 con 0x7fd7ac0ce970 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7a9600640 1 -- 172.21.15.44:0/3612420708 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7a00053c0 con 0x7fd7ac0ce970 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3612420708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ac0b7ae0 con 0x7fd7ac0ce970 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7a9600640 1 -- 172.21.15.44:0/3612420708 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0012890 con 0x7fd7ac0ce970 2024-09-13T08:54:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3612420708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ac1fb740 con 0x7fd7ac0ce970 2024-09-13T08:54:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.990+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3612420708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ac1fb280 con 0x7fd7ac0ce970 2024-09-13T08:54:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.994+0000 7fd7a9600640 1 -- 172.21.15.44:0/3612420708 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fd7a001a070 con 0x7fd7ac0ce970 2024-09-13T08:54:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.994+0000 7fd7a9600640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd78c0755c0 0x7fd78c077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.994+0000 7fd7a9600640 1 -- 172.21.15.44:0/3612420708 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7fd7a00922f0 con 0x7fd7ac0ce970 2024-09-13T08:54:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.994+0000 7fd7b0a00640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd78c0755c0 0x7fd78c077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.994+0000 7fd7b0a00640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd78c0755c0 0x7fd78c077a80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd7980af000 tx=0x7fd79801afd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:33.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:32.998+0000 7fd7a9600640 1 -- 172.21.15.44:0/3612420708 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd7ac1fb280 con 0x7fd7ac0ce970 2024-09-13T08:54:33.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.202+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3612420708 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7ac0af6a0 con 0x7fd78c0755c0 2024-09-13T08:54:33.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7a9600640 1 -- 172.21.15.44:0/3612420708 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7fd7ac0af6a0 con 0x7fd78c0755c0 2024-09-13T08:54:33.212 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:33.212 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:54:33.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:33.212 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone60481" 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sv1" 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone10682" 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone43314" 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone19493" 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:33.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:33.214 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone48080" 2024-09-13T08:54:33.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:33.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:33.214 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone67425" 2024-09-13T08:54:33.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T08:54:33.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:33.214 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subvol_clone73666" 2024-09-13T08:54:33.214 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:33.214 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:54:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd78c0755c0 msgr2=0x7fd78c077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd78c0755c0 0x7fd78c077a80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd7980af000 tx=0x7fd79801afd0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3612420708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0ce970 msgr2=0x7fd7ac0b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0ce970 0x7fd7ac0b7170 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c530 tx=0x7fd7a000c9f0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3612420708 shutdown_connections 2024-09-13T08:54:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd78c0755c0 0x7fd78c077a80 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7ac0d1990 0x7fd7ac0af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd7ac0ced50 0x7fd7ac1fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 --2- 172.21.15.44:0/3612420708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ac0ce970 0x7fd7ac0b7170 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3612420708 >> 172.21.15.44:0/3612420708 conn(0x7fd7ac0d1e90 msgr2=0x7fd7ac072170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3612420708 shutdown_connections 2024-09-13T08:54:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.210+0000 7fd7b3e00640 1 -- 172.21.15.44:0/3612420708 wait complete. 2024-09-13T08:54:33.237 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone60481 2024-09-13T08:54:33.308 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:54:33.312 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:54:33.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.386+0000 7fcc7c000640 1 -- 172.21.15.44:0/1946281034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc5c004d50 msgr2=0x7fcc5c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:33.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.386+0000 7fcc7c000640 1 --2- 172.21.15.44:0/1946281034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc5c004d50 0x7fcc5c0051d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009ee0 tx=0x7fcc6c01d5f0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.386+0000 7fcc7a000640 1 -- 172.21.15.44:0/1946281034 reap_dead start 2024-09-13T08:54:33.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- 172.21.15.44:0/1946281034 shutdown_connections 2024-09-13T08:54:33.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- 172.21.15.44:0/1946281034 >> 172.21.15.44:0/1946281034 conn(0x7fcc740d1da0 msgr2=0x7fcc740d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:33.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- 172.21.15.44:0/1946281034 shutdown_connections 2024-09-13T08:54:33.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- 172.21.15.44:0/1946281034 wait complete. 2024-09-13T08:54:33.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 Processor -- start 2024-09-13T08:54:33.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- start start 2024-09-13T08:54:33.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc5c004d50 0x7fcc740cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:33.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc74154390 0x7fcc741f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcc741f5000 0x7fcc7415c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc740cc570 con 0x7fcc5c004d50 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcc740cc3f0 con 0x7fcc74154390 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fcc740cc6f0 con 0x7fcc741f5000 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc79600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc5c004d50 0x7fcc740cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc79600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc5c004d50 0x7fcc740cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54974/0 (socket says 172.21.15.44:54974) 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc79600640 1 -- 172.21.15.44:0/2109923161 learned_addr learned my addr 172.21.15.44:0/2109923161 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7a000640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcc741f5000 0x7fcc7415c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc78c00640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc74154390 0x7fcc741f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc79600640 1 -- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcc741f5000 msgr2=0x7fcc7415c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc79600640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcc741f5000 0x7fcc7415c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc79600640 1 -- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc74154390 msgr2=0x7fcc741f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:33.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc79600640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc74154390 0x7fcc741f8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc79600640 1 -- 172.21.15.44:0/2109923161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc740cba70 con 0x7fcc5c004d50 2024-09-13T08:54:33.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc79600640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc5c004d50 0x7fcc740cb530 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fcc68001240 tx=0x7fcc680039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:33.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc71600640 1 -- 172.21.15.44:0/2109923161 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68011700 con 0x7fcc5c004d50 2024-09-13T08:54:33.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc71600640 1 -- 172.21.15.44:0/2109923161 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc68016020 con 0x7fcc5c004d50 2024-09-13T08:54:33.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- 172.21.15.44:0/2109923161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc741f5900 con 0x7fcc5c004d50 2024-09-13T08:54:33.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc71600640 1 -- 172.21.15.44:0/2109923161 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68011b60 con 0x7fcc5c004d50 2024-09-13T08:54:33.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- 172.21.15.44:0/2109923161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc741f8670 con 0x7fcc5c004d50 2024-09-13T08:54:33.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.394+0000 7fcc7c000640 1 -- 172.21.15.44:0/2109923161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc740cddd0 con 0x7fcc5c004d50 2024-09-13T08:54:33.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.398+0000 7fcc71600640 1 -- 172.21.15.44:0/2109923161 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fcc68011d00 con 0x7fcc5c004d50 2024-09-13T08:54:33.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.398+0000 7fcc71600640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcc540755c0 0x7fcc54077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:33.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.398+0000 7fcc71600640 1 -- 172.21.15.44:0/2109923161 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7fcc680939b0 con 0x7fcc5c004d50 2024-09-13T08:54:33.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.398+0000 7fcc78c00640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcc540755c0 0x7fcc54077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:33.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.398+0000 7fcc78c00640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcc540755c0 0x7fcc54077a80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcc740b1a30 tx=0x7fcc60006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:33.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.402+0000 7fcc71600640 1 -- 172.21.15.44:0/2109923161 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fcc68020020 con 0x7fcc5c004d50 2024-09-13T08:54:33.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.606+0000 7fcc7c000640 1 -- 172.21.15.44:0/2109923161 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone60481", "target": ["mon-mgr", ""]}) -- 0x7fcc740c8620 con 0x7fcc540755c0 2024-09-13T08:54:33.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.622+0000 7fcc71600640 1 -- 172.21.15.44:0/2109923161 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcc740c8620 con 0x7fcc540755c0 2024-09-13T08:54:33.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 -- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcc540755c0 msgr2=0x7fcc54077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:33.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcc540755c0 0x7fcc54077a80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcc740b1a30 tx=0x7fcc60006b80 comp rx=0 tx=0).stop 2024-09-13T08:54:33.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 -- 172.21.15.44:0/2109923161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc5c004d50 msgr2=0x7fcc740cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:33.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc5c004d50 0x7fcc740cb530 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fcc68001240 tx=0x7fcc680039d0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 -- 172.21.15.44:0/2109923161 shutdown_connections 2024-09-13T08:54:33.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fcc540755c0 0x7fcc54077a80 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fcc741f5000 0x7fcc7415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc74154390 0x7fcc741f8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 --2- 172.21.15.44:0/2109923161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc5c004d50 0x7fcc740cb530 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 -- 172.21.15.44:0/2109923161 >> 172.21.15.44:0/2109923161 conn(0x7fcc740d1da0 msgr2=0x7fcc740b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:33.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 -- 172.21.15.44:0/2109923161 shutdown_connections 2024-09-13T08:54:33.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.626+0000 7fcc7c000640 1 -- 172.21.15.44:0/2109923161 wait complete. 2024-09-13T08:54:33.643 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:33.643+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:33.643 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:33.643+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:33.644 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:33.643+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:33.644 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:33.643+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:33.644 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T08:54:33.643+0000 7f8ac6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T08:54:33.650 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-13T08:54:33.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.842+0000 7fe995e00640 1 -- 172.21.15.44:0/2718985976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe974003a90 msgr2=0x7fe974003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:33.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.842+0000 7fe995e00640 1 --2- 172.21.15.44:0/2718985976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe974003a90 0x7fe974003eb0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fe980009b60 tx=0x7fe98001d270 comp rx=0 tx=0).stop 2024-09-13T08:54:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.842+0000 7fe98f400640 1 -- 172.21.15.44:0/2718985976 reap_dead start 2024-09-13T08:54:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.842+0000 7fe995e00640 1 -- 172.21.15.44:0/2718985976 shutdown_connections 2024-09-13T08:54:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.842+0000 7fe995e00640 1 -- 172.21.15.44:0/2718985976 >> 172.21.15.44:0/2718985976 conn(0x7fe9900d1da0 msgr2=0x7fe9900d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.842+0000 7fe995e00640 1 -- 172.21.15.44:0/2718985976 shutdown_connections 2024-09-13T08:54:33.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.842+0000 7fe995e00640 1 -- 172.21.15.44:0/2718985976 wait complete. 2024-09-13T08:54:33.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe995e00640 1 Processor -- start 2024-09-13T08:54:33.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe995e00640 1 -- start start 2024-09-13T08:54:33.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe995e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe974003a90 0x7fe9900b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:33.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe995e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9900b7a40 0x7fe9900b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:33.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe995e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9901f8040 0x7fe9901f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:33.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe995e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900cea90 con 0x7fe974003a90 2024-09-13T08:54:33.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe995e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900ce910 con 0x7fe9901f8040 2024-09-13T08:54:33.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe995e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe9900cd2e0 con 0x7fe9900b7a40 2024-09-13T08:54:33.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe974003a90 0x7fe9900b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:33.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe974003a90 0x7fe9900b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54990/0 (socket says 172.21.15.44:54990) 2024-09-13T08:54:33.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98ea00640 1 -- 172.21.15.44:0/2157022950 learned_addr learned my addr 172.21.15.44:0/2157022950 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:33.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98e000640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9900b7a40 0x7fe9900b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:33.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98f400640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9901f8040 0x7fe9901f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:33.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98ea00640 1 -- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9900b7a40 msgr2=0x7fe9900b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:33.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98ea00640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9900b7a40 0x7fe9900b5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98ea00640 1 -- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9901f8040 msgr2=0x7fe9901f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98ea00640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9901f8040 0x7fe9901f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98ea00640 1 -- 172.21.15.44:0/2157022950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9900d0130 con 0x7fe974003a90 2024-09-13T08:54:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe98ea00640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe974003a90 0x7fe9900b4bd0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fe980009a80 tx=0x7fe980005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe987400640 1 -- 172.21.15.44:0/2157022950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9800b1040 con 0x7fe974003a90 2024-09-13T08:54:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe987400640 1 -- 172.21.15.44:0/2157022950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9800bc070 con 0x7fe974003a90 2024-09-13T08:54:33.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe987400640 1 -- 172.21.15.44:0/2157022950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980004c20 con 0x7fe974003a90 2024-09-13T08:54:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe995e00640 1 -- 172.21.15.44:0/2157022950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9901f8db0 con 0x7fe974003a90 2024-09-13T08:54:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.846+0000 7fe995e00640 1 -- 172.21.15.44:0/2157022950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9900b6f60 con 0x7fe974003a90 2024-09-13T08:54:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.850+0000 7fe995e00640 1 -- 172.21.15.44:0/2157022950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9900ccbd0 con 0x7fe974003a90 2024-09-13T08:54:33.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.850+0000 7fe987400640 1 -- 172.21.15.44:0/2157022950 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fe9800c1460 con 0x7fe974003a90 2024-09-13T08:54:33.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.850+0000 7fe987400640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe970075590 0x7fe970077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:33.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.850+0000 7fe987400640 1 -- 172.21.15.44:0/2157022950 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7fe98012de40 con 0x7fe974003a90 2024-09-13T08:54:33.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.850+0000 7fe98e000640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe970075590 0x7fe970077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:33.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.850+0000 7fe98e000640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe970075590 0x7fe970077a50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe9900b68d0 tx=0x7fe97c0049e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:33.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:33.854+0000 7fe987400640 1 -- 172.21.15.44:0/2157022950 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe980016f80 con 0x7fe974003a90 2024-09-13T08:54:34.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.058+0000 7fe995e00640 1 -- 172.21.15.44:0/2157022950 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe990070db0 con 0x7fe970075590 2024-09-13T08:54:34.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe987400640 1 -- 172.21.15.44:0/2157022950 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe990070db0 con 0x7fe970075590 2024-09-13T08:54:34.076 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:34.076 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-09-13T08:54:34.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T08:54:34.077 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ss1" 2024-09-13T08:54:34.077 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T08:54:34.077 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-09-13T08:54:34.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 -- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe970075590 msgr2=0x7fe970077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe970075590 0x7fe970077a50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe9900b68d0 tx=0x7fe97c0049e0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 -- 172.21.15.44:0/2157022950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe974003a90 msgr2=0x7fe9900b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe974003a90 0x7fe9900b4bd0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fe980009a80 tx=0x7fe980005620 comp rx=0 tx=0).stop 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 -- 172.21.15.44:0/2157022950 shutdown_connections 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe970075590 0x7fe970077a50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9901f8040 0x7fe9901f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe9900b7a40 0x7fe9900b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 --2- 172.21.15.44:0/2157022950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe974003a90 0x7fe9900b4bd0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 -- 172.21.15.44:0/2157022950 >> 172.21.15.44:0/2157022950 conn(0x7fe9900d1da0 msgr2=0x7fe9900720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 -- 172.21.15.44:0/2157022950 shutdown_connections 2024-09-13T08:54:34.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.074+0000 7fe995e00640 1 -- 172.21.15.44:0/2157022950 wait complete. 2024-09-13T08:54:34.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-13T08:54:34.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.246+0000 7fdc7d800640 1 -- 172.21.15.44:0/995285603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc781605d0 msgr2=0x7fdc780d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.246+0000 7fdc7d800640 1 --2- 172.21.15.44:0/995285603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc781605d0 0x7fdc780d0690 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009840 tx=0x7fdc6801c820 comp rx=0 tx=0).stop 2024-09-13T08:54:34.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.246+0000 7fdc7d800640 1 -- 172.21.15.44:0/995285603 shutdown_connections 2024-09-13T08:54:34.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.246+0000 7fdc7d800640 1 --2- 172.21.15.44:0/995285603 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc780b1650 0x7fdc780b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.246+0000 7fdc7d800640 1 --2- 172.21.15.44:0/995285603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc781605d0 0x7fdc780d0690 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.246+0000 7fdc7d800640 1 -- 172.21.15.44:0/995285603 >> 172.21.15.44:0/995285603 conn(0x7fdc780d1dd0 msgr2=0x7fdc780d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:34.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.246+0000 7fdc7d800640 1 -- 172.21.15.44:0/995285603 shutdown_connections 2024-09-13T08:54:34.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.246+0000 7fdc7d800640 1 -- 172.21.15.44:0/995285603 wait complete. 2024-09-13T08:54:34.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 Processor -- start 2024-09-13T08:54:34.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 -- start start 2024-09-13T08:54:34.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc780aeb00 0x7fdc78158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:34.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc780b1650 0x7fdc780b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:34.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc781605d0 0x7fdc780cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:34.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdc780b0810 con 0x7fdc780b1650 2024-09-13T08:54:34.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc780b0ee0 con 0x7fdc781605d0 2024-09-13T08:54:34.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fdc780b1220 con 0x7fdc780aeb00 2024-09-13T08:54:34.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc780b1650 0x7fdc780b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc780b1650 0x7fdc780b4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55004/0 (socket says 172.21.15.44:55004) 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6fe00640 1 -- 172.21.15.44:0/4241897845 learned_addr learned my addr 172.21.15.44:0/4241897845 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc76a00640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc780aeb00 0x7fdc78158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc77400640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc781605d0 0x7fdc780cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6fe00640 1 -- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc780aeb00 msgr2=0x7fdc78158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6fe00640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc780aeb00 0x7fdc78158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6fe00640 1 -- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc781605d0 msgr2=0x7fdc780cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6fe00640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc781605d0 0x7fdc780cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6fe00640 1 -- 172.21.15.44:0/4241897845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc780ceab0 con 0x7fdc780b1650 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6fe00640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc780b1650 0x7fdc780b4ef0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c970 tx=0x7fdc6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6ea00640 1 -- 172.21.15.44:0/4241897845 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60007c70 con 0x7fdc780b1650 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6ea00640 1 -- 172.21.15.44:0/4241897845 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc6000f450 con 0x7fdc780b1650 2024-09-13T08:54:34.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 -- 172.21.15.44:0/4241897845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc781fc1d0 con 0x7fdc780b1650 2024-09-13T08:54:34.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc6ea00640 1 -- 172.21.15.44:0/4241897845 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000f8a0 con 0x7fdc780b1650 2024-09-13T08:54:34.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 -- 172.21.15.44:0/4241897845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc781fb6f0 con 0x7fdc780b1650 2024-09-13T08:54:34.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.250+0000 7fdc7d800640 1 -- 172.21.15.44:0/4241897845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc780b1a30 con 0x7fdc780b1650 2024-09-13T08:54:34.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.254+0000 7fdc6ea00640 1 -- 172.21.15.44:0/4241897845 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fdc6000fa40 con 0x7fdc780b1650 2024-09-13T08:54:34.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.254+0000 7fdc6ea00640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc54075590 0x7fdc54077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:34.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.254+0000 7fdc6ea00640 1 -- 172.21.15.44:0/4241897845 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7fdc60093a30 con 0x7fdc780b1650 2024-09-13T08:54:34.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.254+0000 7fdc76a00640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc54075590 0x7fdc54077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:34.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.254+0000 7fdc76a00640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc54075590 0x7fdc54077a50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdc680af000 tx=0x7fdc6801b060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:34.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.258+0000 7fdc6ea00640 1 -- 172.21.15.44:0/4241897845 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fdc780ceab0 con 0x7fdc780b1650 2024-09-13T08:54:34.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.466+0000 7fdc7d800640 1 -- 172.21.15.44:0/4241897845 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc780cd960 con 0x7fdc54075590 2024-09-13T08:54:34.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc6ea00640 1 -- 172.21.15.44:0/4241897845 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdc780cd960 con 0x7fdc54075590 2024-09-13T08:54:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:34.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 -- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc54075590 msgr2=0x7fdc54077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc54075590 0x7fdc54077a50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdc680af000 tx=0x7fdc6801b060 comp rx=0 tx=0).stop 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 -- 172.21.15.44:0/4241897845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc780b1650 msgr2=0x7fdc780b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc780b1650 0x7fdc780b4ef0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c970 tx=0x7fdc6000ce30 comp rx=0 tx=0).stop 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 -- 172.21.15.44:0/4241897845 shutdown_connections 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fdc54075590 0x7fdc54077a50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc781605d0 0x7fdc780cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc780b1650 0x7fdc780b4ef0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 --2- 172.21.15.44:0/4241897845 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fdc780aeb00 0x7fdc78158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 -- 172.21.15.44:0/4241897845 >> 172.21.15.44:0/4241897845 conn(0x7fdc780d1dd0 msgr2=0x7fdc7815c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 -- 172.21.15.44:0/4241897845 shutdown_connections 2024-09-13T08:54:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.498+0000 7fdc7d800640 1 -- 172.21.15.44:0/4241897845 wait complete. 2024-09-13T08:54:34.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-13T08:54:34.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.670+0000 7fa722400640 1 -- 172.21.15.44:0/3030074695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa71c15be80 msgr2=0x7fa71c0ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.670+0000 7fa722400640 1 --2- 172.21.15.44:0/3030074695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa71c15be80 0x7fa71c0ce4e0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009900 tx=0x7fa70c01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 -- 172.21.15.44:0/3030074695 shutdown_connections 2024-09-13T08:54:34.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 --2- 172.21.15.44:0/3030074695 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa71c0d28c0 0x7fa71c0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 --2- 172.21.15.44:0/3030074695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa71c15be80 0x7fa71c0ce4e0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 -- 172.21.15.44:0/3030074695 >> 172.21.15.44:0/3030074695 conn(0x7fa71c0d1da0 msgr2=0x7fa71c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:34.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 -- 172.21.15.44:0/3030074695 shutdown_connections 2024-09-13T08:54:34.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 -- 172.21.15.44:0/3030074695 wait complete. 2024-09-13T08:54:34.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 Processor -- start 2024-09-13T08:54:34.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 -- start start 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa71c0b53f0 0x7fa71c155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa71c0d28c0 0x7fa71c155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa71c15be80 0x7fa71c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c169160 con 0x7fa71c15be80 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c168fe0 con 0x7fa71c0b53f0 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa722400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa71c1692e0 con 0x7fa71c0d28c0 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa71be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa71c15be80 0x7fa71c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa71be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa71c15be80 0x7fa71c155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55016/0 (socket says 172.21.15.44:55016) 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa71be00640 1 -- 172.21.15.44:0/2412114141 learned_addr learned my addr 172.21.15.44:0/2412114141 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa713e00640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa71c0d28c0 0x7fa71c155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa71b400640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa71c0b53f0 0x7fa71c155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa71be00640 1 -- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa71c0d28c0 msgr2=0x7fa71c155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa71be00640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa71c0d28c0 0x7fa71c155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa71be00640 1 -- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa71c0b53f0 msgr2=0x7fa71c155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa71be00640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa71c0b53f0 0x7fa71c155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa71be00640 1 -- 172.21.15.44:0/2412114141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c0b0830 con 0x7fa71c15be80 2024-09-13T08:54:34.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.674+0000 7fa71be00640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa71c15be80 0x7fa71c155cc0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c780 tx=0x7fa71400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:34.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.678+0000 7fa713400640 1 -- 172.21.15.44:0/2412114141 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714003ba0 con 0x7fa71c15be80 2024-09-13T08:54:34.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.678+0000 7fa722400640 1 -- 172.21.15.44:0/2412114141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71c0b0b10 con 0x7fa71c15be80 2024-09-13T08:54:34.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.678+0000 7fa713400640 1 -- 172.21.15.44:0/2412114141 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa714003d60 con 0x7fa71c15be80 2024-09-13T08:54:34.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.678+0000 7fa713400640 1 -- 172.21.15.44:0/2412114141 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71400aab0 con 0x7fa71c15be80 2024-09-13T08:54:34.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.678+0000 7fa722400640 1 -- 172.21.15.44:0/2412114141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71c156200 con 0x7fa71c15be80 2024-09-13T08:54:34.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.678+0000 7fa722400640 1 -- 172.21.15.44:0/2412114141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71c0b5860 con 0x7fa71c15be80 2024-09-13T08:54:34.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.678+0000 7fa713400640 1 -- 172.21.15.44:0/2412114141 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7fa71400ac50 con 0x7fa71c15be80 2024-09-13T08:54:34.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.678+0000 7fa713400640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6f8075590 0x7fa6f8077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:34.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.678+0000 7fa713400640 1 -- 172.21.15.44:0/2412114141 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7fa714092e30 con 0x7fa71c15be80 2024-09-13T08:54:34.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.678+0000 7fa71b400640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6f8075590 0x7fa6f8077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:34.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.682+0000 7fa71b400640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6f8075590 0x7fa6f8077a50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0af000 tx=0x7fa70c01b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:34.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.682+0000 7fa713400640 1 -- 172.21.15.44:0/2412114141 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa71c0d0350 con 0x7fa71c15be80 2024-09-13T08:54:34.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.886+0000 7fa722400640 1 -- 172.21.15.44:0/2412114141 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa71c0c85f0 con 0x7fa6f8075590 2024-09-13T08:54:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.918+0000 7fa713400640 1 -- 172.21.15.44:0/2412114141 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa71c0c85f0 con 0x7fa6f8075590 2024-09-13T08:54:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 -- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6f8075590 msgr2=0x7fa6f8077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6f8075590 0x7fa6f8077a50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0af000 tx=0x7fa70c01b120 comp rx=0 tx=0).stop 2024-09-13T08:54:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 -- 172.21.15.44:0/2412114141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa71c15be80 msgr2=0x7fa71c155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa71c15be80 0x7fa71c155cc0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c780 tx=0x7fa71400cc40 comp rx=0 tx=0).stop 2024-09-13T08:54:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 -- 172.21.15.44:0/2412114141 shutdown_connections 2024-09-13T08:54:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa6f8075590 0x7fa6f8077a50 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa71c15be80 0x7fa71c155cc0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa71c0d28c0 0x7fa71c155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 --2- 172.21.15.44:0/2412114141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa71c0b53f0 0x7fa71c155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 -- 172.21.15.44:0/2412114141 >> 172.21.15.44:0/2412114141 conn(0x7fa71c0d1da0 msgr2=0x7fa71c0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 -- 172.21.15.44:0/2412114141 shutdown_connections 2024-09-13T08:54:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:34.922+0000 7fa722400640 1 -- 172.21.15.44:0/2412114141 wait complete. 2024-09-13T08:54:34.946 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone10682 2024-09-13T08:54:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.094+0000 7ff773200640 1 -- 172.21.15.44:0/1127574352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c15afc0 msgr2=0x7ff76c157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.094+0000 7ff773200640 1 --2- 172.21.15.44:0/1127574352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c15afc0 0x7ff76c157aa0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ff76c075440 tx=0x7ff76401cd50 comp rx=0 tx=0).stop 2024-09-13T08:54:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.094+0000 7ff773200640 1 -- 172.21.15.44:0/1127574352 shutdown_connections 2024-09-13T08:54:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.094+0000 7ff773200640 1 --2- 172.21.15.44:0/1127574352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c15afc0 0x7ff76c157aa0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.094+0000 7ff773200640 1 --2- 172.21.15.44:0/1127574352 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff76c06d260 0x7ff76c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.094+0000 7ff773200640 1 --2- 172.21.15.44:0/1127574352 >> v2:172.21.15.129:3300/0 conn(0x7ff76c06ce80 0x7ff76c15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.094+0000 7ff773200640 1 -- 172.21.15.44:0/1127574352 >> 172.21.15.44:0/1127574352 conn(0x7ff76c0cce90 msgr2=0x7ff76c0cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.094+0000 7ff773200640 1 -- 172.21.15.44:0/1127574352 shutdown_connections 2024-09-13T08:54:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.094+0000 7ff773200640 1 -- 172.21.15.44:0/1127574352 wait complete. 2024-09-13T08:54:35.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 Processor -- start 2024-09-13T08:54:35.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 -- start start 2024-09-13T08:54:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff76c06ce80 0x7ff76c1f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff76c06d260 0x7ff76c142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c15afc0 0x7ff76c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c073890 con 0x7ff76c15afc0 2024-09-13T08:54:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c073710 con 0x7ff76c06d260 2024-09-13T08:54:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7ff76c073a10 con 0x7ff76c06ce80 2024-09-13T08:54:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff771200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c15afc0 0x7ff76c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff771200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c15afc0 0x7ff76c15d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55042/0 (socket says 172.21.15.44:55042) 2024-09-13T08:54:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff771200640 1 -- 172.21.15.44:0/2213207739 learned_addr learned my addr 172.21.15.44:0/2213207739 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff763400640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff76c06d260 0x7ff76c142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff771200640 1 -- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff76c06ce80 msgr2=0x7ff76c1f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff76be00640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff76c06ce80 0x7ff76c1f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff771200640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff76c06ce80 0x7ff76c1f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff771200640 1 -- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff76c06d260 msgr2=0x7ff76c142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff771200640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff76c06d260 0x7ff76c142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff771200640 1 -- 172.21.15.44:0/2213207739 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c1f8a10 con 0x7ff76c15afc0 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff763400640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff76c06d260 0x7ff76c142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff76be00640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff76c06ce80 0x7ff76c1f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff771200640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c15afc0 0x7ff76c15d0a0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff76401b400 tx=0x7ff764004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff768c00640 1 -- 172.21.15.44:0/2213207739 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7640bb070 con 0x7ff76c15afc0 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff768c00640 1 -- 172.21.15.44:0/2213207739 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff764004a50 con 0x7ff76c15afc0 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff768c00640 1 -- 172.21.15.44:0/2213207739 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7640b2040 con 0x7ff76c15afc0 2024-09-13T08:54:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 -- 172.21.15.44:0/2213207739 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76c1514b0 con 0x7ff76c15afc0 2024-09-13T08:54:35.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 -- 172.21.15.44:0/2213207739 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76c15c2e0 con 0x7ff76c15afc0 2024-09-13T08:54:35.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.098+0000 7ff773200640 1 -- 172.21.15.44:0/2213207739 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff76c15aa80 con 0x7ff76c15afc0 2024-09-13T08:54:35.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.102+0000 7ff768c00640 1 -- 172.21.15.44:0/2213207739 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 100599+0+0 (secure 0 0 0) 0x7ff7640be070 con 0x7ff76c15afc0 2024-09-13T08:54:35.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.102+0000 7ff768c00640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff748075590 0x7ff748077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:35.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.102+0000 7ff768c00640 1 -- 172.21.15.44:0/2213207739 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7ff76412cd80 con 0x7ff76c15afc0 2024-09-13T08:54:35.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.102+0000 7ff76be00640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff748075590 0x7ff748077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.102+0000 7ff76be00640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff748075590 0x7ff748077a50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00ab50 tx=0x7ff75c00a2f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.106+0000 7ff768c00640 1 -- 172.21.15.44:0/2213207739 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7ff764016970 con 0x7ff76c15afc0 2024-09-13T08:54:35.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.154+0000 7ff768c00640 1 -- 172.21.15.44:0/2213207739 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7ff7640f5640 con 0x7ff76c15afc0 2024-09-13T08:54:35.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.310+0000 7ff773200640 1 -- 172.21.15.44:0/2213207739 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone10682", "target": ["mon-mgr", ""]}) -- 0x7ff76c076160 con 0x7ff748075590 2024-09-13T08:54:35.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.326+0000 7ff768c00640 1 -- 172.21.15.44:0/2213207739 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff76c076160 con 0x7ff748075590 2024-09-13T08:54:35.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 -- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff748075590 msgr2=0x7ff748077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:35.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff748075590 0x7ff748077a50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00ab50 tx=0x7ff75c00a2f0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 -- 172.21.15.44:0/2213207739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c15afc0 msgr2=0x7ff76c15d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:35.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c15afc0 0x7ff76c15d0a0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff76401b400 tx=0x7ff764004270 comp rx=0 tx=0).stop 2024-09-13T08:54:35.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 -- 172.21.15.44:0/2213207739 shutdown_connections 2024-09-13T08:54:35.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7ff748075590 0x7ff748077a50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c15afc0 0x7ff76c15d0a0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff76c06d260 0x7ff76c142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 --2- 172.21.15.44:0/2213207739 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7ff76c06ce80 0x7ff76c1f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 -- 172.21.15.44:0/2213207739 >> 172.21.15.44:0/2213207739 conn(0x7ff76c0cce90 msgr2=0x7ff76c0cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:35.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 -- 172.21.15.44:0/2213207739 shutdown_connections 2024-09-13T08:54:35.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.330+0000 7ff773200640 1 -- 172.21.15.44:0/2213207739 wait complete. 2024-09-13T08:54:35.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone43314 2024-09-13T08:54:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 -- 172.21.15.44:0/2154797464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381015f390 msgr2=0x7f38100cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 --2- 172.21.15.44:0/2154797464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381015f390 0x7f38100cfc30 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f38100cc4a0 tx=0x7f380401c890 comp rx=0 tx=0).stop 2024-09-13T08:54:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 -- 172.21.15.44:0/2154797464 shutdown_connections 2024-09-13T08:54:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 --2- 172.21.15.44:0/2154797464 >> v2:172.21.15.129:3300/0 conn(0x7f38100d28e0 0x7f38100d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 --2- 172.21.15.44:0/2154797464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381015f390 0x7f38100cfc30 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 -- 172.21.15.44:0/2154797464 >> 172.21.15.44:0/2154797464 conn(0x7f38100d1dc0 msgr2=0x7f38100d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:35.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 -- 172.21.15.44:0/2154797464 shutdown_connections 2024-09-13T08:54:35.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 -- 172.21.15.44:0/2154797464 wait complete. 2024-09-13T08:54:35.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 Processor -- start 2024-09-13T08:54:35.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 -- start start 2024-09-13T08:54:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38100d06b0 0x7f38100b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38100d28e0 0x7f38100cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381015f390 0x7f38100ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3810168f50 con 0x7f381015f390 2024-09-13T08:54:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3810168dd0 con 0x7f38100d06b0 2024-09-13T08:54:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3817e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f38101690d0 con 0x7f38100d28e0 2024-09-13T08:54:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3815e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381015f390 0x7f38100ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3815e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381015f390 0x7f38100ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55054/0 (socket says 172.21.15.44:55054) 2024-09-13T08:54:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3815e00640 1 -- 172.21.15.44:0/3734693050 learned_addr learned my addr 172.21.15.44:0/3734693050 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3815400640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38100d06b0 0x7f38100b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3814a00640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38100d28e0 0x7f38100cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3815e00640 1 -- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38100d28e0 msgr2=0x7f38100cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3815e00640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38100d28e0 0x7f38100cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3815e00640 1 -- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38100d06b0 msgr2=0x7f38100b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3815e00640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38100d06b0 0x7f38100b5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.506+0000 7f3815e00640 1 -- 172.21.15.44:0/3734693050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3810152760 con 0x7f381015f390 2024-09-13T08:54:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.510+0000 7f3815e00640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381015f390 0x7f38100ccd10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f380800c7c0 tx=0x7f380800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.510+0000 7f380d600640 1 -- 172.21.15.44:0/3734693050 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808003b80 con 0x7f381015f390 2024-09-13T08:54:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.510+0000 7f3817e00640 1 -- 172.21.15.44:0/3734693050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38101501e0 con 0x7f381015f390 2024-09-13T08:54:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.510+0000 7f380d600640 1 -- 172.21.15.44:0/3734693050 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3808003d20 con 0x7f381015f390 2024-09-13T08:54:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.510+0000 7f380d600640 1 -- 172.21.15.44:0/3734693050 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380800aa50 con 0x7f381015f390 2024-09-13T08:54:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.510+0000 7f3817e00640 1 -- 172.21.15.44:0/3734693050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38101529d0 con 0x7f381015f390 2024-09-13T08:54:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.510+0000 7f3817e00640 1 -- 172.21.15.44:0/3734693050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38100b1350 con 0x7f381015f390 2024-09-13T08:54:35.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.510+0000 7f380d600640 1 -- 172.21.15.44:0/3734693050 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f380800abf0 con 0x7f381015f390 2024-09-13T08:54:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.510+0000 7f380d600640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f37f0075620 0x7f37f0077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.510+0000 7f380d600640 1 -- 172.21.15.44:0/3734693050 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7f3808092f50 con 0x7f381015f390 2024-09-13T08:54:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.514+0000 7f3815400640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f37f0075620 0x7f37f0077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:35.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.514+0000 7f3815400640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f37f0075620 0x7f37f0077ae0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f38040af000 tx=0x7f38040a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:35.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.514+0000 7f380d600640 1 -- 172.21.15.44:0/3734693050 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f380805f530 con 0x7f381015f390 2024-09-13T08:54:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.742+0000 7f3817e00640 1 -- 172.21.15.44:0/3734693050 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone43314", "target": ["mon-mgr", ""]}) -- 0x7f3810151110 con 0x7f37f0075620 2024-09-13T08:54:35.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.782+0000 7f380d600640 1 -- 172.21.15.44:0/3734693050 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3810151110 con 0x7f37f0075620 2024-09-13T08:54:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.786+0000 7f3817e00640 1 -- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f37f0075620 msgr2=0x7f37f0077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.786+0000 7f3817e00640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f37f0075620 0x7f37f0077ae0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f38040af000 tx=0x7f38040a7040 comp rx=0 tx=0).stop 2024-09-13T08:54:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.786+0000 7f3817e00640 1 -- 172.21.15.44:0/3734693050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381015f390 msgr2=0x7f38100ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.786+0000 7f3817e00640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381015f390 0x7f38100ccd10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f380800c7c0 tx=0x7f380800cc80 comp rx=0 tx=0).stop 2024-09-13T08:54:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.786+0000 7f3817e00640 1 -- 172.21.15.44:0/3734693050 shutdown_connections 2024-09-13T08:54:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.786+0000 7f3817e00640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f37f0075620 0x7f37f0077ae0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.786+0000 7f3817e00640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381015f390 0x7f38100ccd10 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.786+0000 7f3817e00640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f38100d28e0 0x7f38100cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.786+0000 7f3817e00640 1 --2- 172.21.15.44:0/3734693050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38100d06b0 0x7f38100b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:35.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.786+0000 7f3817e00640 1 -- 172.21.15.44:0/3734693050 >> 172.21.15.44:0/3734693050 conn(0x7f38100d1dc0 msgr2=0x7f38100b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:35.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.790+0000 7f3817e00640 1 -- 172.21.15.44:0/3734693050 shutdown_connections 2024-09-13T08:54:35.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:35.790+0000 7f3817e00640 1 -- 172.21.15.44:0/3734693050 wait complete. 2024-09-13T08:54:35.825 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone19493 2024-09-13T08:54:36.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.074+0000 7f4e4ce00640 1 -- 172.21.15.44:0/702364203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e480d1000 msgr2=0x7f4e4806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:36.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.074+0000 7f4e4ce00640 1 --2- 172.21.15.44:0/702364203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e480d1000 0x7f4e4806ecb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009840 tx=0x7f4e3401f010 comp rx=0 tx=0).stop 2024-09-13T08:54:36.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.074+0000 7f4e4ce00640 1 -- 172.21.15.44:0/702364203 shutdown_connections 2024-09-13T08:54:36.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.074+0000 7f4e4ce00640 1 --2- 172.21.15.44:0/702364203 >> v2:172.21.15.129:3300/0 conn(0x7f4e480d00c0 0x7f4e480ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.074+0000 7f4e4ce00640 1 --2- 172.21.15.44:0/702364203 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e480d1000 0x7f4e4806ecb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.074+0000 7f4e4ce00640 1 -- 172.21.15.44:0/702364203 >> 172.21.15.44:0/702364203 conn(0x7f4e480d1da0 msgr2=0x7f4e480d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:36.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.074+0000 7f4e4ce00640 1 -- 172.21.15.44:0/702364203 shutdown_connections 2024-09-13T08:54:36.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.074+0000 7f4e4ce00640 1 -- 172.21.15.44:0/702364203 wait complete. 2024-09-13T08:54:36.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e4ce00640 1 Processor -- start 2024-09-13T08:54:36.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e4ce00640 1 -- start start 2024-09-13T08:54:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e4ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e480d00c0 0x7f4e48154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e4ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4815e030 0x7f4e4814c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e4ce00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e480c8e70 0x7f4e480ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e4ce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e480cc8f0 con 0x7f4e4815e030 2024-09-13T08:54:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e4ce00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e480cc770 con 0x7f4e480d00c0 2024-09-13T08:54:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e4ce00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4e480cca70 con 0x7f4e480c8e70 2024-09-13T08:54:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e45400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4815e030 0x7f4e4814c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e45400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4815e030 0x7f4e4814c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55070/0 (socket says 172.21.15.44:55070) 2024-09-13T08:54:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e45400640 1 -- 172.21.15.44:0/274976933 learned_addr learned my addr 172.21.15.44:0/274976933 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e45e00640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e480d00c0 0x7f4e48154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e46800640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e480c8e70 0x7f4e480ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e45400640 1 -- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e480c8e70 msgr2=0x7f4e480ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e45400640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e480c8e70 0x7f4e480ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e45400640 1 -- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e480d00c0 msgr2=0x7f4e48154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e45400640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e480d00c0 0x7f4e48154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e45400640 1 -- 172.21.15.44:0/274976933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e480b6cf0 con 0x7f4e4815e030 2024-09-13T08:54:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e45400640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4815e030 0x7f4e4814c160 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009790 tx=0x7f4e34004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e3e000640 1 -- 172.21.15.44:0/274976933 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e340b5040 con 0x7f4e4815e030 2024-09-13T08:54:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e3e000640 1 -- 172.21.15.44:0/274976933 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e34005220 con 0x7f4e4815e030 2024-09-13T08:54:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e3e000640 1 -- 172.21.15.44:0/274976933 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e340046d0 con 0x7f4e4815e030 2024-09-13T08:54:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e4ce00640 1 -- 172.21.15.44:0/274976933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e480b6f90 con 0x7f4e4815e030 2024-09-13T08:54:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.078+0000 7f4e4ce00640 1 -- 172.21.15.44:0/274976933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4814c6a0 con 0x7f4e4815e030 2024-09-13T08:54:36.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.082+0000 7f4e4ce00640 1 -- 172.21.15.44:0/274976933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e480ccf20 con 0x7f4e4815e030 2024-09-13T08:54:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.082+0000 7f4e3e000640 1 -- 172.21.15.44:0/274976933 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f4e340c1060 con 0x7f4e4815e030 2024-09-13T08:54:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.082+0000 7f4e3e000640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e28075620 0x7f4e28077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.082+0000 7f4e3e000640 1 -- 172.21.15.44:0/274976933 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5879+0+0 (secure 0 0 0) 0x7f4e3412fd60 con 0x7f4e4815e030 2024-09-13T08:54:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.082+0000 7f4e45e00640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e28075620 0x7f4e28077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:36.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.082+0000 7f4e45e00640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e28075620 0x7f4e28077ae0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4e38001870 tx=0x7f4e38001380 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:36.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.086+0000 7f4e3e000640 1 -- 172.21.15.44:0/274976933 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4e34018b70 con 0x7f4e4815e030 2024-09-13T08:54:36.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.302+0000 7f4e4ce00640 1 -- 172.21.15.44:0/274976933 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone19493", "target": ["mon-mgr", ""]}) -- 0x7f4e4815c070 con 0x7f4e28075620 2024-09-13T08:54:36.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e3e000640 1 -- 172.21.15.44:0/274976933 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e4815c070 con 0x7f4e28075620 2024-09-13T08:54:36.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 -- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e28075620 msgr2=0x7f4e28077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:36.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e28075620 0x7f4e28077ae0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4e38001870 tx=0x7f4e38001380 comp rx=0 tx=0).stop 2024-09-13T08:54:36.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 -- 172.21.15.44:0/274976933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4815e030 msgr2=0x7f4e4814c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:36.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4815e030 0x7f4e4814c160 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009790 tx=0x7f4e34004620 comp rx=0 tx=0).stop 2024-09-13T08:54:36.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 -- 172.21.15.44:0/274976933 shutdown_connections 2024-09-13T08:54:36.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4e28075620 0x7f4e28077ae0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4e480c8e70 0x7f4e480ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4815e030 0x7f4e4814c160 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 --2- 172.21.15.44:0/274976933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e480d00c0 0x7f4e48154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 -- 172.21.15.44:0/274976933 >> 172.21.15.44:0/274976933 conn(0x7f4e480d1da0 msgr2=0x7f4e4815c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:36.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 -- 172.21.15.44:0/274976933 shutdown_connections 2024-09-13T08:54:36.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.318+0000 7f4e4ce00640 1 -- 172.21.15.44:0/274976933 wait complete. 2024-09-13T08:54:36.344 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone48080 2024-09-13T08:54:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 -- 172.21.15.44:0/1238804717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25815be10 msgr2=0x7fe2580ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 --2- 172.21.15.44:0/1238804717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25815be10 0x7fe2580ce700 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fe258074e00 tx=0x7fe24801f580 comp rx=0 tx=0).stop 2024-09-13T08:54:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 -- 172.21.15.44:0/1238804717 shutdown_connections 2024-09-13T08:54:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 --2- 172.21.15.44:0/1238804717 >> v2:172.21.15.129:3300/0 conn(0x7fe2580b0b60 0x7fe2580cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 --2- 172.21.15.44:0/1238804717 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe25815c1f0 0x7fe2580cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 --2- 172.21.15.44:0/1238804717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25815be10 0x7fe2580ce700 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 -- 172.21.15.44:0/1238804717 >> 172.21.15.44:0/1238804717 conn(0x7fe2580d1da0 msgr2=0x7fe2580d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 -- 172.21.15.44:0/1238804717 shutdown_connections 2024-09-13T08:54:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 -- 172.21.15.44:0/1238804717 wait complete. 2024-09-13T08:54:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 Processor -- start 2024-09-13T08:54:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 -- start start 2024-09-13T08:54:36.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2580b0b60 0x7fe2581f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:36.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25815be10 0x7fe2581512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:36.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.494+0000 7fe25d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25815c1f0 0x7fe25806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:36.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe25d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe2581600a0 con 0x7fe25815c1f0 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe25d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe25815ff20 con 0x7fe25815be10 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe25d800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe258160220 con 0x7fe2580b0b60 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe257400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25815c1f0 0x7fe25806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe257400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25815c1f0 0x7fe25806dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55086/0 (socket says 172.21.15.44:55086) 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe257400640 1 -- 172.21.15.44:0/3896182400 learned_addr learned my addr 172.21.15.44:0/3896182400 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe24e000640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25815be10 0x7fe2581512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe256a00640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2580b0b60 0x7fe2581f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe257400640 1 -- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2580b0b60 msgr2=0x7fe2581f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe257400640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2580b0b60 0x7fe2581f8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe257400640 1 -- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25815be10 msgr2=0x7fe2581512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe257400640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25815be10 0x7fe2581512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe257400640 1 -- 172.21.15.44:0/3896182400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25806eb70 con 0x7fe25815c1f0 2024-09-13T08:54:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe256a00640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2580b0b60 0x7fe2581f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe257400640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25815c1f0 0x7fe25806dd30 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fe25000d750 tx=0x7fe25000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe24f400640 1 -- 172.21.15.44:0/3896182400 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250017070 con 0x7fe25815c1f0 2024-09-13T08:54:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe24f400640 1 -- 172.21.15.44:0/3896182400 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe250004310 con 0x7fe25815c1f0 2024-09-13T08:54:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe24f400640 1 -- 172.21.15.44:0/3896182400 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250005280 con 0x7fe25815c1f0 2024-09-13T08:54:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe25d800640 1 -- 172.21.15.44:0/3896182400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25806fcf0 con 0x7fe25815c1f0 2024-09-13T08:54:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe25d800640 1 -- 172.21.15.44:0/3896182400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25806fa10 con 0x7fe25815c1f0 2024-09-13T08:54:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe25d800640 1 -- 172.21.15.44:0/3896182400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2581606d0 con 0x7fe25815c1f0 2024-09-13T08:54:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.498+0000 7fe24f400640 1 -- 172.21.15.44:0/3896182400 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fe250005420 con 0x7fe25815c1f0 2024-09-13T08:54:36.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.502+0000 7fe24f400640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe234075650 0x7fe234077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:36.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.502+0000 7fe24f400640 1 -- 172.21.15.44:0/3896182400 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7fe250093560 con 0x7fe25815c1f0 2024-09-13T08:54:36.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.502+0000 7fe256a00640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe234075650 0x7fe234077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:36.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.502+0000 7fe256a00640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe234075650 0x7fe234077b10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe2480b2000 tx=0x7fe24801ddc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:36.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.506+0000 7fe24f400640 1 -- 172.21.15.44:0/3896182400 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe250020020 con 0x7fe25815c1f0 2024-09-13T08:54:36.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.714+0000 7fe25d800640 1 -- 172.21.15.44:0/3896182400 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone48080", "target": ["mon-mgr", ""]}) -- 0x7fe258070350 con 0x7fe234075650 2024-09-13T08:54:36.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.746+0000 7fe24f400640 1 -- 172.21.15.44:0/3896182400 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe258070350 con 0x7fe234075650 2024-09-13T08:54:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 -- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe234075650 msgr2=0x7fe234077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe234075650 0x7fe234077b10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe2480b2000 tx=0x7fe24801ddc0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 -- 172.21.15.44:0/3896182400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25815c1f0 msgr2=0x7fe25806dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25815c1f0 0x7fe25806dd30 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fe25000d750 tx=0x7fe25000dc10 comp rx=0 tx=0).stop 2024-09-13T08:54:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 -- 172.21.15.44:0/3896182400 shutdown_connections 2024-09-13T08:54:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe234075650 0x7fe234077b10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25815c1f0 0x7fe25806dd30 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25815be10 0x7fe2581512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 --2- 172.21.15.44:0/3896182400 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe2580b0b60 0x7fe2581f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 -- 172.21.15.44:0/3896182400 >> 172.21.15.44:0/3896182400 conn(0x7fe2580d1da0 msgr2=0x7fe2580af890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 -- 172.21.15.44:0/3896182400 shutdown_connections 2024-09-13T08:54:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:36.750+0000 7fe25d800640 1 -- 172.21.15.44:0/3896182400 wait complete. 2024-09-13T08:54:36.797 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone67425 2024-09-13T08:54:37.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.050+0000 7f27a8400640 1 -- 172.21.15.44:0/840292166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27a015be10 msgr2=0x7f27a00ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:37.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.050+0000 7f27a8400640 1 --2- 172.21.15.44:0/840292166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27a015be10 0x7f27a00ce700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2794009bc0 tx=0x7f279401f3b0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.050+0000 7f27a8400640 1 -- 172.21.15.44:0/840292166 shutdown_connections 2024-09-13T08:54:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.050+0000 7f27a8400640 1 --2- 172.21.15.44:0/840292166 >> v2:172.21.15.44:3300/0 conn(0x7f27a00b0b60 0x7f27a00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.050+0000 7f27a8400640 1 --2- 172.21.15.44:0/840292166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27a015be10 0x7f27a00ce700 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.050+0000 7f27a8400640 1 -- 172.21.15.44:0/840292166 >> 172.21.15.44:0/840292166 conn(0x7f27a00d1da0 msgr2=0x7f27a00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:37.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.050+0000 7f27a8400640 1 -- 172.21.15.44:0/840292166 shutdown_connections 2024-09-13T08:54:37.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.054+0000 7f27a8400640 1 -- 172.21.15.44:0/840292166 wait complete. 2024-09-13T08:54:37.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.054+0000 7f27a8400640 1 Processor -- start 2024-09-13T08:54:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.054+0000 7f27a8400640 1 -- start start 2024-09-13T08:54:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a8400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27a00b0b60 0x7f27a00b6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a8400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27a015be10 0x7f27a00b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a8400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27a015c1f0 0x7f27a00b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a8400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00cd320 con 0x7f27a015be10 2024-09-13T08:54:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a8400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00cf590 con 0x7f27a015c1f0 2024-09-13T08:54:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a8400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f27a00cd4a0 con 0x7f27a00b0b60 2024-09-13T08:54:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a6400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27a015c1f0 0x7f27a00b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27a015be10 0x7f27a00b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a6400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27a015c1f0 0x7f27a00b1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.44:34228/0 (socket says 172.21.15.44:34228) 2024-09-13T08:54:37.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a6400640 1 -- 172.21.15.44:0/2828626166 learned_addr learned my addr 172.21.15.44:0/2828626166 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:37.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a5a00640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27a00b0b60 0x7f27a00b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:37.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a5000640 1 -- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27a00b0b60 msgr2=0x7f27a00b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:37.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a5000640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27a00b0b60 0x7f27a00b6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a5000640 1 -- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27a015c1f0 msgr2=0x7f27a00b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:37.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a5000640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27a015c1f0 0x7f27a00b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.058+0000 7f27a5000640 1 -- 172.21.15.44:0/2828626166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a01fd310 con 0x7f27a015be10 2024-09-13T08:54:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.062+0000 7f27a5000640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27a015be10 0x7f27a00b19b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c390 tx=0x7f278c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.062+0000 7f279d600640 1 -- 172.21.15.44:0/2828626166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c011790 con 0x7f27a015be10 2024-09-13T08:54:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.062+0000 7f279d600640 1 -- 172.21.15.44:0/2828626166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f278c017070 con 0x7f27a015be10 2024-09-13T08:54:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.062+0000 7f279d600640 1 -- 172.21.15.44:0/2828626166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c011d20 con 0x7f27a015be10 2024-09-13T08:54:37.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.062+0000 7f27a8400640 1 -- 172.21.15.44:0/2828626166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a006df80 con 0x7f27a015be10 2024-09-13T08:54:37.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.062+0000 7f27a8400640 1 -- 172.21.15.44:0/2828626166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a006e540 con 0x7f27a015be10 2024-09-13T08:54:37.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.062+0000 7f27a8400640 1 -- 172.21.15.44:0/2828626166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a006e740 con 0x7f27a015be10 2024-09-13T08:54:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.066+0000 7f279d600640 1 -- 172.21.15.44:0/2828626166 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f278c01c460 con 0x7f27a015be10 2024-09-13T08:54:37.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.066+0000 7f279d600640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2780075650 0x7f2780077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:37.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.066+0000 7f279d600640 1 -- 172.21.15.44:0/2828626166 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7f278c0937c0 con 0x7f27a015be10 2024-09-13T08:54:37.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.066+0000 7f27a5a00640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2780075650 0x7f2780077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.070+0000 7f27a5a00640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2780075650 0x7f2780077b10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f27940b1000 tx=0x7f279401dbf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.070+0000 7f279d600640 1 -- 172.21.15.44:0/2828626166 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f27a006e740 con 0x7f27a015be10 2024-09-13T08:54:37.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.286+0000 7f27a8400640 1 -- 172.21.15.44:0/2828626166 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone67425", "target": ["mon-mgr", ""]}) -- 0x7f27a0070350 con 0x7f2780075650 2024-09-13T08:54:37.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f279d600640 1 -- 172.21.15.44:0/2828626166 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f27a0070350 con 0x7f2780075650 2024-09-13T08:54:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 -- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2780075650 msgr2=0x7f2780077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2780075650 0x7f2780077b10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f27940b1000 tx=0x7f279401dbf0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 -- 172.21.15.44:0/2828626166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27a015be10 msgr2=0x7f27a00b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27a015be10 0x7f27a00b19b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c390 tx=0x7f278c00c850 comp rx=0 tx=0).stop 2024-09-13T08:54:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 -- 172.21.15.44:0/2828626166 shutdown_connections 2024-09-13T08:54:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2780075650 0x7f2780077b10 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27a015c1f0 0x7f27a00b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27a015be10 0x7f27a00b19b0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 --2- 172.21.15.44:0/2828626166 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f27a00b0b60 0x7f27a00b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 -- 172.21.15.44:0/2828626166 >> 172.21.15.44:0/2828626166 conn(0x7f27a00d1da0 msgr2=0x7f27a00ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 -- 172.21.15.44:0/2828626166 shutdown_connections 2024-09-13T08:54:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.318+0000 7f27a8400640 1 -- 172.21.15.44:0/2828626166 wait complete. 2024-09-13T08:54:37.344 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone73666 2024-09-13T08:54:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 -- 172.21.15.44:0/2644976021 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e9c15abc0 msgr2=0x7f5e9c0d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 --2- 172.21.15.44:0/2644976021 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e9c15abc0 0x7f5e9c0d0c40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009870 tx=0x7f5e8801c860 comp rx=0 tx=0).stop 2024-09-13T08:54:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 -- 172.21.15.44:0/2644976021 shutdown_connections 2024-09-13T08:54:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 --2- 172.21.15.44:0/2644976021 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e9c15abc0 0x7f5e9c0d0c40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 -- 172.21.15.44:0/2644976021 >> 172.21.15.44:0/2644976021 conn(0x7f5e9c06dd30 msgr2=0x7f5e9c06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 -- 172.21.15.44:0/2644976021 shutdown_connections 2024-09-13T08:54:37.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 -- 172.21.15.44:0/2644976021 wait complete. 2024-09-13T08:54:37.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 Processor -- start 2024-09-13T08:54:37.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 -- start start 2024-09-13T08:54:37.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e9c0d1180 0x7f5e9c1496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:37.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e9c15a7e0 0x7f5e9c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:37.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e9c15abc0 0x7f5e9c162530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:37.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c168e00 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c168c80 con 0x7f5e9c0d1180 2024-09-13T08:54:37.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea3c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9c168f80 con 0x7f5e9c15abc0 2024-09-13T08:54:37.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5e9be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e9c15a7e0 0x7f5e9c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:37.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5e9be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e9c15a7e0 0x7f5e9c161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55116/0 (socket says 172.21.15.44:55116) 2024-09-13T08:54:37.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5e9be00640 1 -- 172.21.15.44:0/74370217 learned_addr learned my addr 172.21.15.44:0/74370217 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:37.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea1200640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e9c0d1180 0x7f5e9c1496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:37.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea1c00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e9c15abc0 0x7f5e9c162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:37.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5e9be00640 1 -- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e9c15abc0 msgr2=0x7f5e9c162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:37.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5e9be00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e9c15abc0 0x7f5e9c162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5e9be00640 1 -- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e9c0d1180 msgr2=0x7f5e9c1496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:37.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5e9be00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e9c0d1180 0x7f5e9c1496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5e9be00640 1 -- 172.21.15.44:0/74370217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c0af840 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5ea1c00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e9c15abc0 0x7f5e9c162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:37.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.494+0000 7f5e9be00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e9c15a7e0 0x7f5e9c161ff0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f5e88000c10 tx=0x7f5e88005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:37.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.498+0000 7f5e98c00640 1 -- 172.21.15.44:0/74370217 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880bb070 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.498+0000 7f5e98c00640 1 -- 172.21.15.44:0/74370217 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e88004330 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.498+0000 7f5e98c00640 1 -- 172.21.15.44:0/74370217 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880b2040 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.498+0000 7f5ea3c00640 1 -- 172.21.15.44:0/74370217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9c0afb10 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.498+0000 7f5ea3c00640 1 -- 172.21.15.44:0/74370217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9c0b00d0 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.498+0000 7f5ea3c00640 1 -- 172.21.15.44:0/74370217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e9c0d0c40 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.498+0000 7f5e98c00640 1 -- 172.21.15.44:0/74370217 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f5e880be070 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.498+0000 7f5e98c00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5e7c075620 0x7f5e7c077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:37.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.498+0000 7f5e98c00640 1 -- 172.21.15.44:0/74370217 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7f5e8812c760 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.498+0000 7f5ea1200640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5e7c075620 0x7f5e7c077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:37.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.502+0000 7f5ea1200640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5e7c075620 0x7f5e7c077ae0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5e90007fd0 tx=0x7f5e90001440 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:37.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.502+0000 7f5e98c00640 1 -- 172.21.15.44:0/74370217 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f5e88016450 con 0x7f5e9c15a7e0 2024-09-13T08:54:37.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.706+0000 7f5ea3c00640 1 -- 172.21.15.44:0/74370217 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone73666", "target": ["mon-mgr", ""]}) -- 0x7f5e9c070e30 con 0x7f5e7c075620 2024-09-13T08:54:37.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.738+0000 7f5e98c00640 1 -- 172.21.15.44:0/74370217 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e9c070e30 con 0x7f5e7c075620 2024-09-13T08:54:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 -- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5e7c075620 msgr2=0x7f5e7c077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5e7c075620 0x7f5e7c077ae0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5e90007fd0 tx=0x7f5e90001440 comp rx=0 tx=0).stop 2024-09-13T08:54:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 -- 172.21.15.44:0/74370217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e9c15a7e0 msgr2=0x7f5e9c161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e9c15a7e0 0x7f5e9c161ff0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f5e88000c10 tx=0x7f5e88005d40 comp rx=0 tx=0).stop 2024-09-13T08:54:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 -- 172.21.15.44:0/74370217 shutdown_connections 2024-09-13T08:54:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5e7c075620 0x7f5e7c077ae0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5e9c15abc0 0x7f5e9c162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e9c15a7e0 0x7f5e9c161ff0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 --2- 172.21.15.44:0/74370217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e9c0d1180 0x7f5e9c1496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:37.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 -- 172.21.15.44:0/74370217 >> 172.21.15.44:0/74370217 conn(0x7f5e9c06dd30 msgr2=0x7f5e9c1585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:37.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 -- 172.21.15.44:0/74370217 shutdown_connections 2024-09-13T08:54:37.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37.742+0000 7f5ea3c00640 1 -- 172.21.15.44:0/74370217 wait complete. 2024-09-13T08:54:37.791 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-13T08:54:37.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-13T08:54:37.874 INFO:teuthology.orchestra.run.smithi044.stdout:2 2024-09-13T08:54:37.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-13T08:54:37.888 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-13T08:54:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.142+0000 7f4b05400640 1 -- 172.21.15.44:0/1023498547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ae4005340 msgr2=0x7f4ae400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.142+0000 7f4b05400640 1 --2- 172.21.15.44:0/1023498547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ae4005340 0x7f4ae400b9d0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f4af8001960 tx=0x7f4af801ef70 comp rx=0 tx=0).stop 2024-09-13T08:54:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.146+0000 7f4b05400640 1 -- 172.21.15.44:0/1023498547 shutdown_connections 2024-09-13T08:54:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.146+0000 7f4b05400640 1 --2- 172.21.15.44:0/1023498547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ae4005340 0x7f4ae400b9d0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.146+0000 7f4b05400640 1 -- 172.21.15.44:0/1023498547 >> 172.21.15.44:0/1023498547 conn(0x7f4b000d3f90 msgr2=0x7f4b000d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.146+0000 7f4b05400640 1 -- 172.21.15.44:0/1023498547 shutdown_connections 2024-09-13T08:54:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.146+0000 7f4b05400640 1 -- 172.21.15.44:0/1023498547 wait complete. 2024-09-13T08:54:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4b05400640 1 Processor -- start 2024-09-13T08:54:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4b05400640 1 -- start start 2024-09-13T08:54:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4b05400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4ae4005340 0x7f4b0014b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4b05400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0014b560 0x7f4b0015ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4b05400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b0015b180 0x7f4b000cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4b05400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b000d2e80 con 0x7f4b0014b560 2024-09-13T08:54:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4b05400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b000d2d00 con 0x7f4b0015b180 2024-09-13T08:54:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4b05400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4b00162530 con 0x7f4ae4005340 2024-09-13T08:54:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4af5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0014b560 0x7f4b0015ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4afe800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4ae4005340 0x7f4b0014b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4af5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0014b560 0x7f4b0015ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55134/0 (socket says 172.21.15.44:55134) 2024-09-13T08:54:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4af5e00640 1 -- 172.21.15.44:0/3996193735 learned_addr learned my addr 172.21.15.44:0/3996193735 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.150+0000 7f4aff200640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b0015b180 0x7f4b000cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4afe800640 1 -- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b0015b180 msgr2=0x7f4b000cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4afe800640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b0015b180 0x7f4b000cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4afe800640 1 -- 172.21.15.44:0/3996193735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0014b560 msgr2=0x7f4b0015ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4afe800640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0014b560 0x7f4b0015ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4afe800640 1 -- 172.21.15.44:0/3996193735 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b000cc250 con 0x7f4ae4005340 2024-09-13T08:54:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4af5e00640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0014b560 0x7f4b0015ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T08:54:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4afe800640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4ae4005340 0x7f4b0014b020 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4af00012a0 tx=0x7f4af000ea20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4af7400640 1 -- 172.21.15.44:0/3996193735 <== mon.2 v2:172.21.15.129:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0015020 con 0x7f4ae4005340 2024-09-13T08:54:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4af7400640 1 -- 172.21.15.44:0/3996193735 <== mon.2 v2:172.21.15.129:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4af00109a0 con 0x7f4ae4005340 2024-09-13T08:54:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4af7400640 1 -- 172.21.15.44:0/3996193735 <== mon.2 v2:172.21.15.129:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0010df0 con 0x7f4ae4005340 2024-09-13T08:54:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4b05400640 1 -- 172.21.15.44:0/3996193735 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b000cc530 con 0x7f4ae4005340 2024-09-13T08:54:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.154+0000 7f4b05400640 1 -- 172.21.15.44:0/3996193735 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b000ccaf0 con 0x7f4ae4005340 2024-09-13T08:54:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.158+0000 7f4b05400640 1 -- 172.21.15.44:0/3996193735 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0015b590 con 0x7f4ae4005340 2024-09-13T08:54:38.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.162+0000 7f4af7400640 1 -- 172.21.15.44:0/3996193735 <== mon.2 v2:172.21.15.129:3301/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f4af0018020 con 0x7f4ae4005340 2024-09-13T08:54:38.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.162+0000 7f4af7400640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4adc075720 0x7f4adc077be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:38.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.162+0000 7f4af7400640 1 -- 172.21.15.44:0/3996193735 <== mon.2 v2:172.21.15.129:3301/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7f4af0093850 con 0x7f4ae4005340 2024-09-13T08:54:38.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.162+0000 7f4af5e00640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4adc075720 0x7f4adc077be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:38.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.166+0000 7f4af5e00640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4adc075720 0x7f4adc077be0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4b000caf00 tx=0x7f4ae8009f50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:38.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.166+0000 7f4af7400640 1 -- 172.21.15.44:0/3996193735 <== mon.2 v2:172.21.15.129:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4af00c4060 con 0x7f4ae4005340 2024-09-13T08:54:38.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.478+0000 7f4b05400640 1 -- 172.21.15.44:0/3996193735 --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f4b000b0ae0 con 0x7f4ae4005340 2024-09-13T08:54:38.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4af7400640 1 -- 172.21.15.44:0/3996193735 <== mon.2 v2:172.21.15.129:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v56)=0 v56) ==== 131+0+0 (secure 0 0 0) 0x7f4b0015b590 con 0x7f4ae4005340 2024-09-13T08:54:38.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 -- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4adc075720 msgr2=0x7f4adc077be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:38.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4adc075720 0x7f4adc077be0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4b000caf00 tx=0x7f4ae8009f50 comp rx=0 tx=0).stop 2024-09-13T08:54:38.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 -- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4ae4005340 msgr2=0x7f4b0014b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:38.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4ae4005340 0x7f4b0014b020 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4af00012a0 tx=0x7f4af000ea20 comp rx=0 tx=0).stop 2024-09-13T08:54:38.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 -- 172.21.15.44:0/3996193735 shutdown_connections 2024-09-13T08:54:38.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4adc075720 0x7f4adc077be0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:38.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b0015b180 0x7f4b000cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:38.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0014b560 0x7f4b0015ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:38.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 --2- 172.21.15.44:0/3996193735 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4ae4005340 0x7f4b0014b020 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:38.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 -- 172.21.15.44:0/3996193735 >> 172.21.15.44:0/3996193735 conn(0x7f4b000d3f90 msgr2=0x7f4b000b7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:38.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 -- 172.21.15.44:0/3996193735 shutdown_connections 2024-09-13T08:54:38.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:38.482+0000 7f4b05400640 1 -- 172.21.15.44:0/3996193735 wait complete. 2024-09-13T08:54:38.506 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T08:54:38.506 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:54:38.585 DEBUG:teuthology.orchestra.run.smithi129:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T08:54:38.701 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T08:54:38.702 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:54:38.702 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:54:38.702 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:54:38.714 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:54:38.714 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:54:38.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T08:54:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.026+0000 7f4f09000640 1 -- 172.21.15.44:0/1455151759 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f040cdb30 msgr2=0x7f4f040aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.026+0000 7f4f09000640 1 --2- 172.21.15.44:0/1455151759 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f040cdb30 0x7f4f040aeef0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009bc0 tx=0x7f4efc01cba0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.026+0000 7f4f09000640 1 -- 172.21.15.44:0/1455151759 shutdown_connections 2024-09-13T08:54:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.026+0000 7f4f09000640 1 --2- 172.21.15.44:0/1455151759 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f040cdb30 0x7f4f040aeef0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.026+0000 7f4f09000640 1 --2- 172.21.15.44:0/1455151759 >> v2:172.21.15.129:3300/0 conn(0x7f4f040ced80 0x7f4f040ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.026+0000 7f4f09000640 1 -- 172.21.15.44:0/1455151759 >> 172.21.15.44:0/1455151759 conn(0x7f4f040d1dc0 msgr2=0x7f4f040d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.026+0000 7f4f09000640 1 -- 172.21.15.44:0/1455151759 shutdown_connections 2024-09-13T08:54:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.026+0000 7f4f09000640 1 -- 172.21.15.44:0/1455151759 wait complete. 2024-09-13T08:54:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 Processor -- start 2024-09-13T08:54:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 -- start start 2024-09-13T08:54:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f040cdb30 0x7f4f04152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f040ced80 0x7f4f040cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f0415bea0 0x7f4f040cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04169290 con 0x7f4f040ced80 2024-09-13T08:54:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04169110 con 0x7f4f040cdb30 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4f04169410 con 0x7f4f0415bea0 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f040ced80 0x7f4f040cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f040cdb30 0x7f4f04152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f040ced80 0x7f4f040cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55148/0 (socket says 172.21.15.44:55148) 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efbe00640 1 -- 172.21.15.44:0/990602491 learned_addr learned my addr 172.21.15.44:0/990602491 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f02800640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f0415bea0 0x7f4f040cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efbe00640 1 -- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f0415bea0 msgr2=0x7f4f040cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efbe00640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f0415bea0 0x7f4f040cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efbe00640 1 -- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f040cdb30 msgr2=0x7f4f04152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efbe00640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f040cdb30 0x7f4f04152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efbe00640 1 -- 172.21.15.44:0/990602491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f040b3150 con 0x7f4f040ced80 2024-09-13T08:54:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efbe00640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f040ced80 0x7f4f040cb960 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00e9f0 tx=0x7f4eec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efa000640 1 -- 172.21.15.44:0/990602491 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec00cd80 con 0x7f4f040ced80 2024-09-13T08:54:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efa000640 1 -- 172.21.15.44:0/990602491 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4eec010430 con 0x7f4f040ced80 2024-09-13T08:54:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4efa000640 1 -- 172.21.15.44:0/990602491 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec010860 con 0x7f4f040ced80 2024-09-13T08:54:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 -- 172.21.15.44:0/990602491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f040b3430 con 0x7f4f040ced80 2024-09-13T08:54:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 -- 172.21.15.44:0/990602491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f040b39f0 con 0x7f4f040ced80 2024-09-13T08:54:39.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.030+0000 7f4f09000640 1 -- 172.21.15.44:0/990602491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f040ce670 con 0x7f4f040ced80 2024-09-13T08:54:39.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.034+0000 7f4efa000640 1 -- 172.21.15.44:0/990602491 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f4eec010a00 con 0x7f4f040ced80 2024-09-13T08:54:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.034+0000 7f4efa000640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ee0075650 0x7f4ee0077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.034+0000 7f4efa000640 1 -- 172.21.15.44:0/990602491 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7f4eec093d90 con 0x7f4f040ced80 2024-09-13T08:54:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.034+0000 7f4f01e00640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ee0075650 0x7f4ee0077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.034+0000 7f4f01e00640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ee0075650 0x7f4ee0077b10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4001870 tx=0x7f4ef4001380 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:39.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.038+0000 7f4efa000640 1 -- 172.21.15.44:0/990602491 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f4eec0603a0 con 0x7f4f040ced80 2024-09-13T08:54:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.226+0000 7f4f09000640 1 -- 172.21.15.44:0/990602491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4f040008e0 con 0x7f4f040ced80 2024-09-13T08:54:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.226+0000 7f4efa000640 1 -- 172.21.15.44:0/990602491 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v147) ==== 80+0+0 (secure 0 0 0) 0x7f4eec064dd0 con 0x7f4f040ced80 2024-09-13T08:54:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-09-13T08:54:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 -- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ee0075650 msgr2=0x7f4ee0077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ee0075650 0x7f4ee0077b10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4001870 tx=0x7f4ef4001380 comp rx=0 tx=0).stop 2024-09-13T08:54:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 -- 172.21.15.44:0/990602491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f040ced80 msgr2=0x7f4f040cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f040ced80 0x7f4f040cb960 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00e9f0 tx=0x7f4eec00eeb0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 -- 172.21.15.44:0/990602491 shutdown_connections 2024-09-13T08:54:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4ee0075650 0x7f4ee0077b10 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4f0415bea0 0x7f4f040cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f040ced80 0x7f4f040cb960 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 --2- 172.21.15.44:0/990602491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f040cdb30 0x7f4f04152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 -- 172.21.15.44:0/990602491 >> 172.21.15.44:0/990602491 conn(0x7f4f040d1dc0 msgr2=0x7f4f040c8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 -- 172.21.15.44:0/990602491 shutdown_connections 2024-09-13T08:54:39.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.230+0000 7f4f09000640 1 -- 172.21.15.44:0/990602491 wait complete. 2024-09-13T08:54:39.252 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-13T08:54:39.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:39.252 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-13T08:54:39.316 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:54:39.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:39.316 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T08:54:39.332 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:54:39.332 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:54:39.332 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:54:39.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:54:39.341 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:54:39.341 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:54:39.341 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:54:39.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:54:39.383 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T08:54:39.383 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:54:39.384 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:54:39.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:54:39.392 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:54:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 -- 172.21.15.44:0/3272311089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8af415bea0 msgr2=0x7f8af40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 --2- 172.21.15.44:0/3272311089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8af415bea0 0x7f8af40ce670 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009900 tx=0x7f8ae401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 -- 172.21.15.44:0/3272311089 shutdown_connections 2024-09-13T08:54:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 --2- 172.21.15.44:0/3272311089 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8af40cdb30 0x7f8af40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 --2- 172.21.15.44:0/3272311089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8af415bea0 0x7f8af40ce670 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 -- 172.21.15.44:0/3272311089 >> 172.21.15.44:0/3272311089 conn(0x7f8af40d1dc0 msgr2=0x7f8af40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:39.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 -- 172.21.15.44:0/3272311089 shutdown_connections 2024-09-13T08:54:39.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 -- 172.21.15.44:0/3272311089 wait complete. 2024-09-13T08:54:39.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 Processor -- start 2024-09-13T08:54:39.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 -- start start 2024-09-13T08:54:39.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8af40cdb30 0x7f8af4158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:39.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.574+0000 7f8afbc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8af40d28e0 0x7f8af40c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:39.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8afbc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8af415bea0 0x7f8af406f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:39.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8afbc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4169020 con 0x7f8af40cdb30 2024-09-13T08:54:39.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8afbc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4168ea0 con 0x7f8af40d28e0 2024-09-13T08:54:39.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8afbc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8af41691a0 con 0x7f8af415bea0 2024-09-13T08:54:39.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8af40cdb30 0x7f8af4158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:39.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8af40cdb30 0x7f8af4158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55172/0 (socket says 172.21.15.44:55172) 2024-09-13T08:54:39.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9200640 1 -- 172.21.15.44:0/3037758442 learned_addr learned my addr 172.21.15.44:0/3037758442 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:39.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9200640 1 -- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8af415bea0 msgr2=0x7f8af406f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:39.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9c00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8af415bea0 0x7f8af406f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:39.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9200640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8af415bea0 0x7f8af406f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8aebe00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8af40d28e0 0x7f8af40c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:39.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9200640 1 -- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8af40d28e0 msgr2=0x7f8af40c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:39.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9200640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8af40d28e0 0x7f8af40c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9200640 1 -- 172.21.15.44:0/3037758442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af40c84a0 con 0x7f8af40cdb30 2024-09-13T08:54:39.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9c00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8af415bea0 0x7f8af406f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:39.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8aebe00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8af40d28e0 0x7f8af40c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:39.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af9200640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8af40cdb30 0x7f8af4158910 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009850 tx=0x7f8ae4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:39.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af1600640 1 -- 172.21.15.44:0/3037758442 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40b2040 con 0x7f8af40cdb30 2024-09-13T08:54:39.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af1600640 1 -- 172.21.15.44:0/3037758442 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ae4003600 con 0x7f8af40cdb30 2024-09-13T08:54:39.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8afbc00640 1 -- 172.21.15.44:0/3037758442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af40c9200 con 0x7f8af40cdb30 2024-09-13T08:54:39.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af1600640 1 -- 172.21.15.44:0/3037758442 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40039f0 con 0x7f8af40cdb30 2024-09-13T08:54:39.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8afbc00640 1 -- 172.21.15.44:0/3037758442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af4157e30 con 0x7f8af40cdb30 2024-09-13T08:54:39.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8afbc00640 1 -- 172.21.15.44:0/3037758442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af40cc9d0 con 0x7f8af40cdb30 2024-09-13T08:54:39.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.578+0000 7f8af1600640 1 -- 172.21.15.44:0/3037758442 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f8ae4003b90 con 0x7f8af40cdb30 2024-09-13T08:54:39.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.582+0000 7f8af1600640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ad0075620 0x7f8ad0077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:39.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.582+0000 7f8af1600640 1 -- 172.21.15.44:0/3037758442 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7f8ae412d020 con 0x7f8af40cdb30 2024-09-13T08:54:39.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.582+0000 7f8aebe00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ad0075620 0x7f8ad0077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:39.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.582+0000 7f8aebe00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ad0075620 0x7f8ad0077ae0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8af4159b40 tx=0x7f8adc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:39.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.586+0000 7f8af1600640 1 -- 172.21.15.44:0/3037758442 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8ae40bd060 con 0x7f8af40cdb30 2024-09-13T08:54:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.902+0000 7f8afbc00640 1 -- 172.21.15.44:0/3037758442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8af40762f0 con 0x7f8af40cdb30 2024-09-13T08:54:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.906+0000 7f8af1600640 1 -- 172.21.15.44:0/3037758442 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5230 (secure 0 0 0) 0x7f8ae40164d0 con 0x7f8af40cdb30 2024-09-13T08:54:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:39.908 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":96,"btime":"2024-09-13T08:53:27:902711+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/766502733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":766502733},{"type":"v1","addr":"172.21.15.129:6837","nonce":766502733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2074722286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2074722286},{"type":"v1","addr":"172.21.15.129:6835","nonce":2074722286}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3065267295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3065267295},{"type":"v1","addr":"172.21.15.44:6834","nonce":3065267295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:53:15.007787+0000","modified":"2024-09-13T08:53:18.026905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9219":{"gid":9219,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3418956320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3418956320},{"type":"v1","addr":"172.21.15.44:6835","nonce":3418956320}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9219,"qdb_cluster":[9219]},"id":10}]} 2024-09-13T08:54:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 96 2024-09-13T08:54:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 -- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ad0075620 msgr2=0x7f8ad0077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ad0075620 0x7f8ad0077ae0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8af4159b40 tx=0x7f8adc008040 comp rx=0 tx=0).stop 2024-09-13T08:54:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 -- 172.21.15.44:0/3037758442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8af40cdb30 msgr2=0x7f8af4158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8af40cdb30 0x7f8af4158910 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009850 tx=0x7f8ae4004040 comp rx=0 tx=0).stop 2024-09-13T08:54:39.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 -- 172.21.15.44:0/3037758442 shutdown_connections 2024-09-13T08:54:39.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8ad0075620 0x7f8ad0077ae0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8af415bea0 0x7f8af406f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8af40d28e0 0x7f8af40c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 --2- 172.21.15.44:0/3037758442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8af40cdb30 0x7f8af4158910 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:39.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 -- 172.21.15.44:0/3037758442 >> 172.21.15.44:0/3037758442 conn(0x7f8af40d1dc0 msgr2=0x7f8af40b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:39.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 -- 172.21.15.44:0/3037758442 shutdown_connections 2024-09-13T08:54:39.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:39.910+0000 7f8afbc00640 1 -- 172.21.15.44:0/3037758442 wait complete. 2024-09-13T08:54:39.948 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:54:40.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.122+0000 7fd97ee00640 1 -- 172.21.15.44:0/3711477641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9780b3300 msgr2=0x7fd97815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.122+0000 7fd97ee00640 1 --2- 172.21.15.44:0/3711477641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9780b3300 0x7fd97815f4e0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fd978069800 tx=0x7fd96401c890 comp rx=0 tx=0).stop 2024-09-13T08:54:40.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.122+0000 7fd97ee00640 1 -- 172.21.15.44:0/3711477641 shutdown_connections 2024-09-13T08:54:40.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.122+0000 7fd97ee00640 1 --2- 172.21.15.44:0/3711477641 >> v2:172.21.15.129:3300/0 conn(0x7fd9780cd0d0 0x7fd9780cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.122+0000 7fd97ee00640 1 --2- 172.21.15.44:0/3711477641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9780b3300 0x7fd97815f4e0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.122+0000 7fd97ee00640 1 -- 172.21.15.44:0/3711477641 >> 172.21.15.44:0/3711477641 conn(0x7fd9780b18f0 msgr2=0x7fd9780b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:40.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.122+0000 7fd97ee00640 1 -- 172.21.15.44:0/3711477641 shutdown_connections 2024-09-13T08:54:40.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.122+0000 7fd97ee00640 1 -- 172.21.15.44:0/3711477641 wait complete. 2024-09-13T08:54:40.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 Processor -- start 2024-09-13T08:54:40.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 -- start start 2024-09-13T08:54:40.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9780b3300 0x7fd97814baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9780cd0d0 0x7fd97815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd97815af90 0x7fd97815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780cdc60 con 0x7fd9780cd0d0 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780cdae0 con 0x7fd9780b3300 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd9780cdde0 con 0x7fd97815af90 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9780cd0d0 0x7fd97815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9780cd0d0 0x7fd97815c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55188/0 (socket says 172.21.15.44:55188) 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977400640 1 -- 172.21.15.44:0/819341927 learned_addr learned my addr 172.21.15.44:0/819341927 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977e00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9780b3300 0x7fd97814baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ce00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd97815af90 0x7fd97815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977400640 1 -- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd97815af90 msgr2=0x7fd97815caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977400640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd97815af90 0x7fd97815caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977400640 1 -- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9780b3300 msgr2=0x7fd97814baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977400640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9780b3300 0x7fd97814baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977400640 1 -- 172.21.15.44:0/819341927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97815d1c0 con 0x7fd9780cd0d0 2024-09-13T08:54:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977e00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9780b3300 0x7fd97814baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ce00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd97815af90 0x7fd97815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd977400640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9780cd0d0 0x7fd97815c560 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd96401af40 tx=0x7fd964004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd96fe00640 1 -- 172.21.15.44:0/819341927 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640ba020 con 0x7fd9780cd0d0 2024-09-13T08:54:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd96fe00640 1 -- 172.21.15.44:0/819341927 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd964005030 con 0x7fd9780cd0d0 2024-09-13T08:54:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 -- 172.21.15.44:0/819341927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97815d4a0 con 0x7fd9780cd0d0 2024-09-13T08:54:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd96fe00640 1 -- 172.21.15.44:0/819341927 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b2040 con 0x7fd9780cd0d0 2024-09-13T08:54:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 -- 172.21.15.44:0/819341927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd978150860 con 0x7fd9780cd0d0 2024-09-13T08:54:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.126+0000 7fd97ee00640 1 -- 172.21.15.44:0/819341927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd978069410 con 0x7fd9780cd0d0 2024-09-13T08:54:40.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.130+0000 7fd96fe00640 1 -- 172.21.15.44:0/819341927 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fd964005480 con 0x7fd9780cd0d0 2024-09-13T08:54:40.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.130+0000 7fd96fe00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd958075620 0x7fd958077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.130+0000 7fd96fe00640 1 -- 172.21.15.44:0/819341927 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7fd96412cfb0 con 0x7fd9780cd0d0 2024-09-13T08:54:40.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.130+0000 7fd977e00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd958075620 0x7fd958077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.130+0000 7fd977e00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd958075620 0x7fd958077ae0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd9680098b0 tx=0x7fd968008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:40.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.134+0000 7fd96fe00640 1 -- 172.21.15.44:0/819341927 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd9640164b0 con 0x7fd9780cd0d0 2024-09-13T08:54:40.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.350+0000 7fd97ee00640 1 -- 172.21.15.44:0/819341927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd97815f4e0 con 0x7fd9780cd0d0 2024-09-13T08:54:40.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.350+0000 7fd96fe00640 1 -- 172.21.15.44:0/819341927 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5230 (secure 0 0 0) 0x7fd9640f9d00 con 0x7fd9780cd0d0 2024-09-13T08:54:40.354 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:40.354 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":96,"btime":"2024-09-13T08:53:27:902711+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/766502733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":766502733},{"type":"v1","addr":"172.21.15.129:6837","nonce":766502733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2074722286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2074722286},{"type":"v1","addr":"172.21.15.129:6835","nonce":2074722286}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3065267295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3065267295},{"type":"v1","addr":"172.21.15.44:6834","nonce":3065267295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:53:15.007787+0000","modified":"2024-09-13T08:53:18.026905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9219":{"gid":9219,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3418956320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3418956320},{"type":"v1","addr":"172.21.15.44:6835","nonce":3418956320}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9219,"qdb_cluster":[9219]},"id":10}]} 2024-09-13T08:54:40.354 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 96 2024-09-13T08:54:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 -- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd958075620 msgr2=0x7fd958077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd958075620 0x7fd958077ae0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd9680098b0 tx=0x7fd968008040 comp rx=0 tx=0).stop 2024-09-13T08:54:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 -- 172.21.15.44:0/819341927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9780cd0d0 msgr2=0x7fd97815c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9780cd0d0 0x7fd97815c560 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd96401af40 tx=0x7fd964004040 comp rx=0 tx=0).stop 2024-09-13T08:54:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 -- 172.21.15.44:0/819341927 shutdown_connections 2024-09-13T08:54:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd958075620 0x7fd958077ae0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd97815af90 0x7fd97815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9780cd0d0 0x7fd97815c560 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 --2- 172.21.15.44:0/819341927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9780b3300 0x7fd97814baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 -- 172.21.15.44:0/819341927 >> 172.21.15.44:0/819341927 conn(0x7fd9780b18f0 msgr2=0x7fd9780b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:40.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 -- 172.21.15.44:0/819341927 shutdown_connections 2024-09-13T08:54:40.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.354+0000 7fd97ee00640 1 -- 172.21.15.44:0/819341927 wait complete. 2024-09-13T08:54:40.375 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:54:40.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.518+0000 7f3402200640 1 -- 172.21.15.44:0/3060400002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33fc0d2060 msgr2=0x7f33fc0cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.518+0000 7f3402200640 1 --2- 172.21.15.44:0/3060400002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33fc0d2060 0x7f33fc0cf660 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009900 tx=0x7f33ec01c8e0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.518+0000 7f3402200640 1 -- 172.21.15.44:0/3060400002 shutdown_connections 2024-09-13T08:54:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.518+0000 7f3402200640 1 --2- 172.21.15.44:0/3060400002 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33fc0b7ab0 0x7f33fc0b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.518+0000 7f3402200640 1 --2- 172.21.15.44:0/3060400002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33fc0d2060 0x7f33fc0cf660 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.518+0000 7f3402200640 1 -- 172.21.15.44:0/3060400002 >> 172.21.15.44:0/3060400002 conn(0x7f33fc0d3fe0 msgr2=0x7f33fc0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.518+0000 7f3402200640 1 -- 172.21.15.44:0/3060400002 shutdown_connections 2024-09-13T08:54:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.518+0000 7f3402200640 1 -- 172.21.15.44:0/3060400002 wait complete. 2024-09-13T08:54:40.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.518+0000 7f3402200640 1 Processor -- start 2024-09-13T08:54:40.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f3402200640 1 -- start start 2024-09-13T08:54:40.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f3402200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc0b7ab0 0x7f33fc159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f3402200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33fc0d2060 0x7f33fc1fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f3402200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33fc0d2440 0x7f33fc0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f3402200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc16b690 con 0x7f33fc0d2440 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f3402200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc16b510 con 0x7f33fc0b7ab0 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f3402200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f33fc16b810 con 0x7f33fc0d2060 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33fc0d2440 0x7f33fc0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33fc0d2440 0x7f33fc0cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55198/0 (socket says 172.21.15.44:55198) 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33fbe00640 1 -- 172.21.15.44:0/2770413611 learned_addr learned my addr 172.21.15.44:0/2770413611 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33fb400640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc0b7ab0 0x7f33fc159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33faa00640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33fc0d2060 0x7f33fc1fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33fbe00640 1 -- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33fc0d2060 msgr2=0x7f33fc1fb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33fbe00640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33fc0d2060 0x7f33fc1fb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33fbe00640 1 -- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc0b7ab0 msgr2=0x7f33fc159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33fbe00640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc0b7ab0 0x7f33fc159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33fbe00640 1 -- 172.21.15.44:0/2770413611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc0c7af0 con 0x7f33fc0d2440 2024-09-13T08:54:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33fbe00640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33fc0d2440 0x7f33fc0cbce0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f33f400ea50 tx=0x7f33f400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33f3400640 1 -- 172.21.15.44:0/2770413611 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f400cdc0 con 0x7f33fc0d2440 2024-09-13T08:54:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33f3400640 1 -- 172.21.15.44:0/2770413611 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33f4004c40 con 0x7f33fc0d2440 2024-09-13T08:54:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f33f3400640 1 -- 172.21.15.44:0/2770413611 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4005280 con 0x7f33fc0d2440 2024-09-13T08:54:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f3402200640 1 -- 172.21.15.44:0/2770413611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc1fbc20 con 0x7f33fc0d2440 2024-09-13T08:54:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f3402200640 1 -- 172.21.15.44:0/2770413611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc1fc1e0 con 0x7f33fc0d2440 2024-09-13T08:54:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.522+0000 7f3402200640 1 -- 172.21.15.44:0/2770413611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33fc0d07d0 con 0x7f33fc0d2440 2024-09-13T08:54:40.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.526+0000 7f33f3400640 1 -- 172.21.15.44:0/2770413611 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f33f4005420 con 0x7f33fc0d2440 2024-09-13T08:54:40.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.526+0000 7f33f3400640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f33dc075620 0x7f33dc077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.526+0000 7f33f3400640 1 -- 172.21.15.44:0/2770413611 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7f33f4093a10 con 0x7f33fc0d2440 2024-09-13T08:54:40.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.526+0000 7f33fb400640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f33dc075620 0x7f33dc077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.526+0000 7f33fb400640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f33dc075620 0x7f33dc077ae0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0af000 tx=0x7f33ec01b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:40.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.530+0000 7f33f3400640 1 -- 172.21.15.44:0/2770413611 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f33f4016030 con 0x7f33fc0d2440 2024-09-13T08:54:40.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.706+0000 7f3402200640 1 -- 172.21.15.44:0/2770413611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33fc0c7cc0 con 0x7f33fc0d2440 2024-09-13T08:54:40.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f33f3400640 1 -- 172.21.15.44:0/2770413611 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) ==== 74+0+17078 (secure 0 0 0) 0x7f33f4060070 con 0x7f33fc0d2440 2024-09-13T08:54:40.711 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:40.712 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":147,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:54:36.151241+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:53:12.727898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-13T08:53:13.969013+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":147,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.10","mappings":[{"from":2,"to":5}]},{"pgid":"21.2b","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:54:40.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 -- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f33dc075620 msgr2=0x7f33dc077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f33dc075620 0x7f33dc077ae0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0af000 tx=0x7f33ec01b120 comp rx=0 tx=0).stop 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 -- 172.21.15.44:0/2770413611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33fc0d2440 msgr2=0x7f33fc0cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33fc0d2440 0x7f33fc0cbce0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f33f400ea50 tx=0x7f33f400ef10 comp rx=0 tx=0).stop 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 -- 172.21.15.44:0/2770413611 shutdown_connections 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f33dc075620 0x7f33dc077ae0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33fc0d2440 0x7f33fc0cbce0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f33fc0d2060 0x7f33fc1fb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 --2- 172.21.15.44:0/2770413611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc0b7ab0 0x7f33fc159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 -- 172.21.15.44:0/2770413611 >> 172.21.15.44:0/2770413611 conn(0x7f33fc0d3fe0 msgr2=0x7f33fc0d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 -- 172.21.15.44:0/2770413611 shutdown_connections 2024-09-13T08:54:40.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.710+0000 7f3402200640 1 -- 172.21.15.44:0/2770413611 wait complete. 2024-09-13T08:54:40.733 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:54:40.734 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:54:40.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 -- 172.21.15.44:0/3639735544 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa64c0703d0 msgr2=0x7fa64c15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 --2- 172.21.15.44:0/3639735544 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa64c0703d0 0x7fa64c15ea30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa64400ad00 tx=0x7fa64401dda0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 -- 172.21.15.44:0/3639735544 shutdown_connections 2024-09-13T08:54:40.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 --2- 172.21.15.44:0/3639735544 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa64c0703d0 0x7fa64c15ea30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 -- 172.21.15.44:0/3639735544 >> 172.21.15.44:0/3639735544 conn(0x7fa64c0b18f0 msgr2=0x7fa64c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:40.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 -- 172.21.15.44:0/3639735544 shutdown_connections 2024-09-13T08:54:40.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 -- 172.21.15.44:0/3639735544 wait complete. 2024-09-13T08:54:40.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 Processor -- start 2024-09-13T08:54:40.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 -- start start 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa64c0703d0 0x7fa64c15c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa64c0b49b0 0x7fa64c142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa64c15a4d0 0x7fa64c158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c0ccfe0 con 0x7fa64c15a4d0 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c0cce60 con 0x7fa64c0703d0 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa652800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa64c168180 con 0x7fa64c0b49b0 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa64be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa64c15a4d0 0x7fa64c158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa64be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa64c15a4d0 0x7fa64c158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55210/0 (socket says 172.21.15.44:55210) 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa64be00640 1 -- 172.21.15.44:0/299782881 learned_addr learned my addr 172.21.15.44:0/299782881 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa64b400640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa64c0703d0 0x7fa64c15c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.878+0000 7fa642a00640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa64c0b49b0 0x7fa64c142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa64be00640 1 -- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa64c0b49b0 msgr2=0x7fa64c142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa64be00640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa64c0b49b0 0x7fa64c142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa64be00640 1 -- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa64c0703d0 msgr2=0x7fa64c15c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa64be00640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa64c0703d0 0x7fa64c15c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa64be00640 1 -- 172.21.15.44:0/299782881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c15cb10 con 0x7fa64c15a4d0 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa64be00640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa64c15a4d0 0x7fa64c158a90 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa64401c450 tx=0x7fa64400a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa643e00640 1 -- 172.21.15.44:0/299782881 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440bb020 con 0x7fa64c15a4d0 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa643e00640 1 -- 172.21.15.44:0/299782881 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa644004e00 con 0x7fa64c15a4d0 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa643e00640 1 -- 172.21.15.44:0/299782881 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440b3040 con 0x7fa64c15a4d0 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa652800640 1 -- 172.21.15.44:0/299782881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c15cdf0 con 0x7fa64c15a4d0 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa652800640 1 -- 172.21.15.44:0/299782881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c15d3b0 con 0x7fa64c15a4d0 2024-09-13T08:54:40.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa652800640 1 -- 172.21.15.44:0/299782881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64c069410 con 0x7fa64c15a4d0 2024-09-13T08:54:40.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa643e00640 1 -- 172.21.15.44:0/299782881 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fa644004320 con 0x7fa64c15a4d0 2024-09-13T08:54:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.882+0000 7fa643e00640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa628075650 0x7fa628077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.886+0000 7fa643e00640 1 -- 172.21.15.44:0/299782881 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7fa64412d680 con 0x7fa64c15a4d0 2024-09-13T08:54:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.886+0000 7fa64b400640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa628075650 0x7fa628077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.886+0000 7fa64b400640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa628075650 0x7fa628077b10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa63c001ec0 tx=0x7fa63c001760 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:40.890+0000 7fa643e00640 1 -- 172.21.15.44:0/299782881 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa6440c5060 con 0x7fa64c15a4d0 2024-09-13T08:54:41.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.098+0000 7fa652800640 1 -- 172.21.15.44:0/299782881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa64c0cbf90 con 0x7fa64c15a4d0 2024-09-13T08:54:41.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.098+0000 7fa643e00640 1 -- 172.21.15.44:0/299782881 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5230 (secure 0 0 0) 0x7fa644017a20 con 0x7fa64c15a4d0 2024-09-13T08:54:41.102 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:41.102 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":96,"btime":"2024-09-13T08:53:27:902711+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/766502733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":766502733},{"type":"v1","addr":"172.21.15.129:6837","nonce":766502733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2074722286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2074722286},{"type":"v1","addr":"172.21.15.129:6835","nonce":2074722286}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3065267295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3065267295},{"type":"v1","addr":"172.21.15.44:6834","nonce":3065267295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:53:15.007787+0000","modified":"2024-09-13T08:53:18.026905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9219":{"gid":9219,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3418956320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3418956320},{"type":"v1","addr":"172.21.15.44:6835","nonce":3418956320}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9219,"qdb_cluster":[9219]},"id":10}]} 2024-09-13T08:54:41.103 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 96 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 -- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa628075650 msgr2=0x7fa628077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa628075650 0x7fa628077b10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa63c001ec0 tx=0x7fa63c001760 comp rx=0 tx=0).stop 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 -- 172.21.15.44:0/299782881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa64c15a4d0 msgr2=0x7fa64c158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa64c15a4d0 0x7fa64c158a90 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa64401c450 tx=0x7fa64400a310 comp rx=0 tx=0).stop 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 -- 172.21.15.44:0/299782881 shutdown_connections 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa628075650 0x7fa628077b10 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa64c15a4d0 0x7fa64c158a90 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa64c0b49b0 0x7fa64c142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 --2- 172.21.15.44:0/299782881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa64c0703d0 0x7fa64c15c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 -- 172.21.15.44:0/299782881 >> 172.21.15.44:0/299782881 conn(0x7fa64c0b18f0 msgr2=0x7fa64c15a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:41.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 -- 172.21.15.44:0/299782881 shutdown_connections 2024-09-13T08:54:41.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.102+0000 7fa652800640 1 -- 172.21.15.44:0/299782881 wait complete. 2024-09-13T08:54:41.126 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:54:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.270+0000 7fa588200640 1 -- 172.21.15.44:0/2545372953 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa58015bea0 msgr2=0x7fa5800ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.270+0000 7fa588200640 1 --2- 172.21.15.44:0/2545372953 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa58015bea0 0x7fa5800ce670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa574009900 tx=0x7fa57401c8e0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.270+0000 7fa588200640 1 -- 172.21.15.44:0/2545372953 shutdown_connections 2024-09-13T08:54:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.270+0000 7fa588200640 1 --2- 172.21.15.44:0/2545372953 >> v2:172.21.15.129:3300/0 conn(0x7fa5800d28e0 0x7fa5800cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.270+0000 7fa588200640 1 --2- 172.21.15.44:0/2545372953 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa58015bea0 0x7fa5800ce670 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.270+0000 7fa588200640 1 -- 172.21.15.44:0/2545372953 >> 172.21.15.44:0/2545372953 conn(0x7fa5800d1dc0 msgr2=0x7fa5800d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.270+0000 7fa588200640 1 -- 172.21.15.44:0/2545372953 shutdown_connections 2024-09-13T08:54:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.270+0000 7fa588200640 1 -- 172.21.15.44:0/2545372953 wait complete. 2024-09-13T08:54:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 Processor -- start 2024-09-13T08:54:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 -- start start 2024-09-13T08:54:41.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5800cdb30 0x7fa5800c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:41.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5800d28e0 0x7fa58009db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:41.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58015bea0 0x7fa580159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:41.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa580168fc0 con 0x7fa58015bea0 2024-09-13T08:54:41.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa580168e40 con 0x7fa5800d28e0 2024-09-13T08:54:41.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa580169140 con 0x7fa5800cdb30 2024-09-13T08:54:41.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa586200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58015bea0 0x7fa580159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa586200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58015bea0 0x7fa580159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55226/0 (socket says 172.21.15.44:55226) 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa586200640 1 -- 172.21.15.44:0/4131776446 learned_addr learned my addr 172.21.15.44:0/4131776446 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa585800640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5800cdb30 0x7fa5800c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa584e00640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5800d28e0 0x7fa58009db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa586200640 1 -- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5800cdb30 msgr2=0x7fa5800c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa586200640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5800cdb30 0x7fa5800c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa586200640 1 -- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5800d28e0 msgr2=0x7fa58009db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa586200640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5800d28e0 0x7fa58009db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa586200640 1 -- 172.21.15.44:0/4131776446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa580146f40 con 0x7fa58015bea0 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa584e00640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5800d28e0 0x7fa58009db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa586200640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58015bea0 0x7fa580159cf0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa578009d30 tx=0x7fa57800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa57d600640 1 -- 172.21.15.44:0/4131776446 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578003b80 con 0x7fa58015bea0 2024-09-13T08:54:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa57d600640 1 -- 172.21.15.44:0/4131776446 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa578003d20 con 0x7fa58015bea0 2024-09-13T08:54:41.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa57d600640 1 -- 172.21.15.44:0/4131776446 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578004280 con 0x7fa58015bea0 2024-09-13T08:54:41.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 -- 172.21.15.44:0/4131776446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa580147220 con 0x7fa58015bea0 2024-09-13T08:54:41.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 -- 172.21.15.44:0/4131776446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5801477e0 con 0x7fa58015bea0 2024-09-13T08:54:41.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.274+0000 7fa588200640 1 -- 172.21.15.44:0/4131776446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5800cc9d0 con 0x7fa58015bea0 2024-09-13T08:54:41.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.278+0000 7fa57d600640 1 -- 172.21.15.44:0/4131776446 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fa578004420 con 0x7fa58015bea0 2024-09-13T08:54:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.278+0000 7fa57d600640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa560075620 0x7fa560077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.278+0000 7fa57d600640 1 -- 172.21.15.44:0/4131776446 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7fa578093e70 con 0x7fa58015bea0 2024-09-13T08:54:41.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.278+0000 7fa585800640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa560075620 0x7fa560077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:41.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.278+0000 7fa585800640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa560075620 0x7fa560077ae0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa5740af000 tx=0x7fa57401b120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:41.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.282+0000 7fa57d600640 1 -- 172.21.15.44:0/4131776446 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa578060b20 con 0x7fa58015bea0 2024-09-13T08:54:41.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.490+0000 7fa588200640 1 -- 172.21.15.44:0/4131776446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5800762f0 con 0x7fa58015bea0 2024-09-13T08:54:41.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.490+0000 7fa57d600640 1 -- 172.21.15.44:0/4131776446 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5230 (secure 0 0 0) 0x7fa578060420 con 0x7fa58015bea0 2024-09-13T08:54:41.495 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:41.495 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":96,"btime":"2024-09-13T08:53:27:902711+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/766502733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":766502733},{"type":"v1","addr":"172.21.15.129:6837","nonce":766502733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2074722286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2074722286},{"type":"v1","addr":"172.21.15.129:6835","nonce":2074722286}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3065267295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3065267295},{"type":"v1","addr":"172.21.15.44:6834","nonce":3065267295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T08:53:15.007787+0000","modified":"2024-09-13T08:53:18.026905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9219":{"gid":9219,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3418956320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3418956320},{"type":"v1","addr":"172.21.15.44:6835","nonce":3418956320}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9219,"qdb_cluster":[9219]},"id":10}]} 2024-09-13T08:54:41.495 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 96 2024-09-13T08:54:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 -- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa560075620 msgr2=0x7fa560077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa560075620 0x7fa560077ae0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa5740af000 tx=0x7fa57401b120 comp rx=0 tx=0).stop 2024-09-13T08:54:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 -- 172.21.15.44:0/4131776446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58015bea0 msgr2=0x7fa580159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58015bea0 0x7fa580159cf0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa578009d30 tx=0x7fa57800c6a0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 -- 172.21.15.44:0/4131776446 shutdown_connections 2024-09-13T08:54:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa560075620 0x7fa560077ae0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58015bea0 0x7fa580159cf0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5800d28e0 0x7fa58009db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 --2- 172.21.15.44:0/4131776446 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa5800cdb30 0x7fa5800c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 -- 172.21.15.44:0/4131776446 >> 172.21.15.44:0/4131776446 conn(0x7fa5800d1dc0 msgr2=0x7fa5800b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 -- 172.21.15.44:0/4131776446 shutdown_connections 2024-09-13T08:54:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.494+0000 7fa588200640 1 -- 172.21.15.44:0/4131776446 wait complete. 2024-09-13T08:54:41.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:54:41.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.658+0000 7f3956a00640 1 -- 172.21.15.44:0/56803914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39500ced80 msgr2=0x7f39500ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.658+0000 7f3956a00640 1 --2- 172.21.15.44:0/56803914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39500ced80 0x7f39500ae9b0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f3938009900 tx=0x7f393801c8e0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.658+0000 7f3956a00640 1 -- 172.21.15.44:0/56803914 shutdown_connections 2024-09-13T08:54:41.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.658+0000 7f3956a00640 1 --2- 172.21.15.44:0/56803914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39500ced80 0x7f39500ae9b0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.658+0000 7f3956a00640 1 -- 172.21.15.44:0/56803914 >> 172.21.15.44:0/56803914 conn(0x7f39500d1dc0 msgr2=0x7f39500d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:41.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.658+0000 7f3956a00640 1 -- 172.21.15.44:0/56803914 shutdown_connections 2024-09-13T08:54:41.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.658+0000 7f3956a00640 1 -- 172.21.15.44:0/56803914 wait complete. 2024-09-13T08:54:41.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3956a00640 1 Processor -- start 2024-09-13T08:54:41.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3956a00640 1 -- start start 2024-09-13T08:54:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3956a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39500cdb30 0x7f3950157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3956a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39500ced80 0x7f39501f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3956a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f395015bea0 0x7f39500c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3956a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f395016a8a0 con 0x7f39500ced80 2024-09-13T08:54:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3956a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f395016a720 con 0x7f39500cdb30 2024-09-13T08:54:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3956a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f395016aa20 con 0x7f395015bea0 2024-09-13T08:54:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3947e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39500ced80 0x7f39501f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3947e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39500ced80 0x7f39501f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55236/0 (socket says 172.21.15.44:55236) 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3947e00640 1 -- 172.21.15.44:0/2494161487 learned_addr learned my addr 172.21.15.44:0/2494161487 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3954a00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f395015bea0 0x7f39500c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f394fe00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39500cdb30 0x7f3950157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3947e00640 1 -- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f395015bea0 msgr2=0x7f39500c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3947e00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f395015bea0 0x7f39500c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3947e00640 1 -- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39500cdb30 msgr2=0x7f3950157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3947e00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39500cdb30 0x7f3950157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3947e00640 1 -- 172.21.15.44:0/2494161487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39500cbb30 con 0x7f39500ced80 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f394fe00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39500cdb30 0x7f3950157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3954a00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f395015bea0 0x7f39500c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3947e00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39500ced80 0x7f39501f89f0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f3938009850 tx=0x7f39380040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f394cc00640 1 -- 172.21.15.44:0/2494161487 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39380b2040 con 0x7f39500ced80 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f394cc00640 1 -- 172.21.15.44:0/2494161487 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3938005030 con 0x7f39500ced80 2024-09-13T08:54:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f394cc00640 1 -- 172.21.15.44:0/2494161487 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938005460 con 0x7f39500ced80 2024-09-13T08:54:41.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3956a00640 1 -- 172.21.15.44:0/2494161487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39501f8f30 con 0x7f39500ced80 2024-09-13T08:54:41.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.662+0000 7f3956a00640 1 -- 172.21.15.44:0/2494161487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395015b6a0 con 0x7f39500ced80 2024-09-13T08:54:41.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.666+0000 7f3956a00640 1 -- 172.21.15.44:0/2494161487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39500ce670 con 0x7f39500ced80 2024-09-13T08:54:41.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.666+0000 7f394cc00640 1 -- 172.21.15.44:0/2494161487 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f39380046d0 con 0x7f39500ced80 2024-09-13T08:54:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.666+0000 7f394cc00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f392c075620 0x7f392c077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.666+0000 7f394cc00640 1 -- 172.21.15.44:0/2494161487 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7f393812cde0 con 0x7f39500ced80 2024-09-13T08:54:41.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.666+0000 7f394fe00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f392c075620 0x7f392c077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:41.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.666+0000 7f394fe00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f392c075620 0x7f392c077ae0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f395006fcd0 tx=0x7f3940008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:41.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.670+0000 7f394cc00640 1 -- 172.21.15.44:0/2494161487 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f39501f8f30 con 0x7f39500ced80 2024-09-13T08:54:41.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.846+0000 7f3956a00640 1 -- 172.21.15.44:0/2494161487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39500762f0 con 0x7f39500ced80 2024-09-13T08:54:41.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.846+0000 7f394cc00640 1 -- 172.21.15.44:0/2494161487 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) ==== 74+0+17078 (secure 0 0 0) 0x7f39380164d0 con 0x7f39500ced80 2024-09-13T08:54:41.850 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:41.850 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":147,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:54:36.151241+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-13T08:53:12.727898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-13T08:53:13.969013+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":147,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.10","mappings":[{"from":2,"to":5}]},{"pgid":"21.2b","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:54:41.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 -- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f392c075620 msgr2=0x7f392c077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f392c075620 0x7f392c077ae0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f395006fcd0 tx=0x7f3940008040 comp rx=0 tx=0).stop 2024-09-13T08:54:41.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 -- 172.21.15.44:0/2494161487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39500ced80 msgr2=0x7f39501f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:41.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39500ced80 0x7f39501f89f0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f3938009850 tx=0x7f39380040c0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 -- 172.21.15.44:0/2494161487 shutdown_connections 2024-09-13T08:54:41.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f392c075620 0x7f392c077ae0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f395015bea0 0x7f39500c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39500ced80 0x7f39501f89f0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 --2- 172.21.15.44:0/2494161487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39500cdb30 0x7f3950157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:41.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 -- 172.21.15.44:0/2494161487 >> 172.21.15.44:0/2494161487 conn(0x7f39500d1dc0 msgr2=0x7f39500d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:41.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 -- 172.21.15.44:0/2494161487 shutdown_connections 2024-09-13T08:54:41.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:41.850+0000 7f3956a00640 1 -- 172.21.15.44:0/2494161487 wait complete. 2024-09-13T08:54:41.871 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T08:54:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 -- 172.21.15.44:0/1268767774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b4003c40 msgr2=0x7f87b4004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 --2- 172.21.15.44:0/1268767774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b4003c40 0x7f87b4004040 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f87c0001860 tx=0x7f87c001c880 comp rx=0 tx=0).stop 2024-09-13T08:54:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 -- 172.21.15.44:0/1268767774 shutdown_connections 2024-09-13T08:54:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 --2- 172.21.15.44:0/1268767774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b4003c40 0x7f87b4004040 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 -- 172.21.15.44:0/1268767774 >> 172.21.15.44:0/1268767774 conn(0x7f87d00b18f0 msgr2=0x7f87d00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:42.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 -- 172.21.15.44:0/1268767774 shutdown_connections 2024-09-13T08:54:42.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 -- 172.21.15.44:0/1268767774 wait complete. 2024-09-13T08:54:42.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 Processor -- start 2024-09-13T08:54:42.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 -- start start 2024-09-13T08:54:42.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87b4003c40 0x7f87d0072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:42.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87d0150ad0 0x7f87d01fb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:42.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f87d01424e0 0x7f87d015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:42.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87d00cc8e0 con 0x7f87d0150ad0 2024-09-13T08:54:42.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87d00cc760 con 0x7f87b4003c40 2024-09-13T08:54:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87d5200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f87d00cca60 con 0x7f87d01424e0 2024-09-13T08:54:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87d0150ad0 0x7f87d01fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87d0150ad0 0x7f87d01fb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55254/0 (socket says 172.21.15.44:55254) 2024-09-13T08:54:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87c7e00640 1 -- 172.21.15.44:0/1168005810 learned_addr learned my addr 172.21.15.44:0/1168005810 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.018+0000 7f87ce800640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f87d01424e0 0x7f87d015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87cde00640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87b4003c40 0x7f87d0072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87c7e00640 1 -- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f87d01424e0 msgr2=0x7f87d015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87c7e00640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f87d01424e0 0x7f87d015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87c7e00640 1 -- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87b4003c40 msgr2=0x7f87d0072870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:42.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87c7e00640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87b4003c40 0x7f87d0072870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87c7e00640 1 -- 172.21.15.44:0/1168005810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d01fbba0 con 0x7f87d0150ad0 2024-09-13T08:54:42.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87c7e00640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87d0150ad0 0x7f87d01fb660 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f87b800e590 tx=0x7f87b800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:42.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87c6000640 1 -- 172.21.15.44:0/1168005810 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8015070 con 0x7f87d0150ad0 2024-09-13T08:54:42.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87c6000640 1 -- 172.21.15.44:0/1168005810 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87b8010980 con 0x7f87d0150ad0 2024-09-13T08:54:42.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87c6000640 1 -- 172.21.15.44:0/1168005810 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8010dd0 con 0x7f87d0150ad0 2024-09-13T08:54:42.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87d5200640 1 -- 172.21.15.44:0/1168005810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d009bee0 con 0x7f87d0150ad0 2024-09-13T08:54:42.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87d5200640 1 -- 172.21.15.44:0/1168005810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d01fae60 con 0x7f87d0150ad0 2024-09-13T08:54:42.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87d5200640 1 -- 172.21.15.44:0/1168005810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d00ccfc0 con 0x7f87d0150ad0 2024-09-13T08:54:42.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.022+0000 7f87c6000640 1 -- 172.21.15.44:0/1168005810 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f87b8018070 con 0x7f87d0150ad0 2024-09-13T08:54:42.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.026+0000 7f87c6000640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f87ac075620 0x7f87ac077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:42.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.026+0000 7f87c6000640 1 -- 172.21.15.44:0/1168005810 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5935+0+0 (secure 0 0 0) 0x7f87b80933f0 con 0x7f87d0150ad0 2024-09-13T08:54:42.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.026+0000 7f87cde00640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f87ac075620 0x7f87ac077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:42.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.026+0000 7f87cde00640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f87ac075620 0x7f87ac077ae0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f87c0001830 tx=0x7f87c00196c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:42.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.030+0000 7f87c6000640 1 -- 172.21.15.44:0/1168005810 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f87b805fa50 con 0x7f87d0150ad0 2024-09-13T08:54:42.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.242+0000 7f87d5200640 1 -- 172.21.15.44:0/1168005810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f87d0069b00 con 0x7f87d0150ad0 2024-09-13T08:54:42.246 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T08:54:42.242+0000 7f03b3c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T08:54:42.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.246+0000 7f87c6000640 1 -- 172.21.15.44:0/1168005810 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) ==== 161+0+0 (secure 0 0 0) 0x7f87b8064480 con 0x7f87d0150ad0 2024-09-13T08:54:42.248 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.246+0000 7f87d5200640 1 -- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f87ac075620 msgr2=0x7f87ac077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.246+0000 7f87d5200640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f87ac075620 0x7f87ac077ae0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f87c0001830 tx=0x7f87c00196c0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.246+0000 7f87d5200640 1 -- 172.21.15.44:0/1168005810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87d0150ad0 msgr2=0x7f87d01fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.246+0000 7f87d5200640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87d0150ad0 0x7f87d01fb660 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f87b800e590 tx=0x7f87b800e990 comp rx=0 tx=0).stop 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.250+0000 7f87d5200640 1 -- 172.21.15.44:0/1168005810 shutdown_connections 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.250+0000 7f87d5200640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f87ac075620 0x7f87ac077ae0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.250+0000 7f87d5200640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f87d01424e0 0x7f87d015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.250+0000 7f87d5200640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87d0150ad0 0x7f87d01fb660 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.250+0000 7f87d5200640 1 --2- 172.21.15.44:0/1168005810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87b4003c40 0x7f87d0072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.250+0000 7f87d5200640 1 -- 172.21.15.44:0/1168005810 >> 172.21.15.44:0/1168005810 conn(0x7f87d00b18f0 msgr2=0x7f87d00d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.250+0000 7f87d5200640 1 -- 172.21.15.44:0/1168005810 shutdown_connections 2024-09-13T08:54:42.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.250+0000 7f87d5200640 1 -- 172.21.15.44:0/1168005810 wait complete. 2024-09-13T08:54:42.273 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T08:54:42.319 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-09-13T08:54:42.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.422+0000 7f50df800640 1 -- 172.21.15.44:0/17250066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d80d1f80 msgr2=0x7f50d815c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:42.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.422+0000 7f50df800640 1 --2- 172.21.15.44:0/17250066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d80d1f80 0x7f50d815c800 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009c80 tx=0x7f50d001cce0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.422+0000 7f50df800640 1 -- 172.21.15.44:0/17250066 shutdown_connections 2024-09-13T08:54:42.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.422+0000 7f50df800640 1 --2- 172.21.15.44:0/17250066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d80d1f80 0x7f50d815c800 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.422+0000 7f50df800640 1 --2- 172.21.15.44:0/17250066 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50d815c060 0x7f50d80d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.422+0000 7f50df800640 1 -- 172.21.15.44:0/17250066 >> 172.21.15.44:0/17250066 conn(0x7f50d809aa10 msgr2=0x7f50d809adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:42.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.422+0000 7f50df800640 1 -- 172.21.15.44:0/17250066 shutdown_connections 2024-09-13T08:54:42.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.422+0000 7f50df800640 1 -- 172.21.15.44:0/17250066 wait complete. 2024-09-13T08:54:42.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.422+0000 7f50df800640 1 Processor -- start 2024-09-13T08:54:42.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.422+0000 7f50df800640 1 -- start start 2024-09-13T08:54:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50df800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50d80d1f80 0x7f50d8152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50df800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50d815bc80 0x7f50d81f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50df800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d815c060 0x7f50d806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50df800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50d80ccff0 con 0x7f50d815c060 2024-09-13T08:54:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50df800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50d80cce70 con 0x7f50d815bc80 2024-09-13T08:54:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50df800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f50d80cd170 con 0x7f50d80d1f80 2024-09-13T08:54:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50dd800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d815c060 0x7f50d806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50dd800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d815c060 0x7f50d806dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50632/0 (socket says 172.21.15.44:50632) 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50dd800640 1 -- 172.21.15.44:0/3219983107 learned_addr learned my addr 172.21.15.44:0/3219983107 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50dce00640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50d80d1f80 0x7f50d8152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50d7e00640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50d815bc80 0x7f50d81f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50dd800640 1 -- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50d80d1f80 msgr2=0x7f50d8152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50dd800640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50d80d1f80 0x7f50d8152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50dd800640 1 -- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50d815bc80 msgr2=0x7f50d81f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50dd800640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50d815bc80 0x7f50d81f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50dd800640 1 -- 172.21.15.44:0/3219983107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d815a240 con 0x7f50d815c060 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50dd800640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d815c060 0x7f50d806dd30 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f50d001b390 tx=0x7f50d0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:42.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50d4c00640 1 -- 172.21.15.44:0/3219983107 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d00bb070 con 0x7f50d815c060 2024-09-13T08:54:42.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50d4c00640 1 -- 172.21.15.44:0/3219983107 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50d0004b40 con 0x7f50d815c060 2024-09-13T08:54:42.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50d4c00640 1 -- 172.21.15.44:0/3219983107 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d00b2040 con 0x7f50d815c060 2024-09-13T08:54:42.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50df800640 1 -- 172.21.15.44:0/3219983107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d8154790 con 0x7f50d815c060 2024-09-13T08:54:42.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50df800640 1 -- 172.21.15.44:0/3219983107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d8152ca0 con 0x7f50d815c060 2024-09-13T08:54:42.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.426+0000 7f50df800640 1 -- 172.21.15.44:0/3219983107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d815c800 con 0x7f50d815c060 2024-09-13T08:54:42.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.430+0000 7f50d4c00640 1 -- 172.21.15.44:0/3219983107 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f50d0004000 con 0x7f50d815c060 2024-09-13T08:54:42.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.430+0000 7f50d4c00640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50b8075620 0x7f50b8077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:42.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.430+0000 7f50d4c00640 1 -- 172.21.15.44:0/3219983107 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5993+0+0 (secure 0 0 0) 0x7f50d012cf90 con 0x7f50d815c060 2024-09-13T08:54:42.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.430+0000 7f50dce00640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50b8075620 0x7f50b8077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:42.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.430+0000 7f50dce00640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50b8075620 0x7f50b8077ae0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f50cc007fd0 tx=0x7f50cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:42.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.434+0000 7f50d4c00640 1 -- 172.21.15.44:0/3219983107 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f50d0016990 con 0x7f50d815c060 2024-09-13T08:54:42.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:42.646+0000 7f50df800640 1 -- 172.21.15.44:0/3219983107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f50d8070cb0 con 0x7f50d815c060 2024-09-13T08:54:43.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.250+0000 7f50d4c00640 1 -- 172.21.15.44:0/3219983107 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) ==== 104+0+0 (secure 0 0 0) 0x7f50d00f9690 con 0x7f50d815c060 2024-09-13T08:54:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 -- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50b8075620 msgr2=0x7f50b8077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50b8075620 0x7f50b8077ae0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f50cc007fd0 tx=0x7f50cc009040 comp rx=0 tx=0).stop 2024-09-13T08:54:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 -- 172.21.15.44:0/3219983107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d815c060 msgr2=0x7f50d806dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d815c060 0x7f50d806dd30 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f50d001b390 tx=0x7f50d0005b00 comp rx=0 tx=0).stop 2024-09-13T08:54:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 -- 172.21.15.44:0/3219983107 shutdown_connections 2024-09-13T08:54:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f50b8075620 0x7f50b8077ae0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d815c060 0x7f50d806dd30 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50d815bc80 0x7f50d81f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 --2- 172.21.15.44:0/3219983107 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f50d80d1f80 0x7f50d8152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:43.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 -- 172.21.15.44:0/3219983107 >> 172.21.15.44:0/3219983107 conn(0x7f50d809aa10 msgr2=0x7f50d80cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:43.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 -- 172.21.15.44:0/3219983107 shutdown_connections 2024-09-13T08:54:43.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.254+0000 7f50df800640 1 -- 172.21.15.44:0/3219983107 wait complete. 2024-09-13T08:54:43.283 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T08:54:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.438+0000 7f7333200640 1 -- 172.21.15.44:0/2322405475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73100051d0 msgr2=0x7f731000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.438+0000 7f7333200640 1 --2- 172.21.15.44:0/2322405475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73100051d0 0x7f731000b9d0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f732400c640 tx=0x7f732401d2c0 comp rx=0 tx=0).stop 2024-09-13T08:54:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.438+0000 7f7333200640 1 -- 172.21.15.44:0/2322405475 shutdown_connections 2024-09-13T08:54:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.438+0000 7f7333200640 1 --2- 172.21.15.44:0/2322405475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73100051d0 0x7f731000b9d0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.438+0000 7f7333200640 1 -- 172.21.15.44:0/2322405475 >> 172.21.15.44:0/2322405475 conn(0x7f732c0d1da0 msgr2=0x7f732c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.438+0000 7f7333200640 1 -- 172.21.15.44:0/2322405475 shutdown_connections 2024-09-13T08:54:43.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.438+0000 7f7333200640 1 -- 172.21.15.44:0/2322405475 wait complete. 2024-09-13T08:54:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.438+0000 7f7333200640 1 Processor -- start 2024-09-13T08:54:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.438+0000 7f7333200640 1 -- start start 2024-09-13T08:54:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7333200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73100051d0 0x7f732c152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7333200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c0b5500 0x7f732c152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7333200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f732c0c58a0 0x7f732c0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7333200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f732c0ceba0 con 0x7f73100051d0 2024-09-13T08:54:43.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7333200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f732c0cea20 con 0x7f732c0b5500 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7333200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f732c0ced20 con 0x7f732c0c58a0 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f732be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73100051d0 0x7f732c152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f732be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73100051d0 0x7f732c152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50648/0 (socket says 172.21.15.44:50648) 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f732be00640 1 -- 172.21.15.44:0/2308841431 learned_addr learned my addr 172.21.15.44:0/2308841431 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7323e00640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c0b5500 0x7f732c152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7331200640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f732c0c58a0 0x7f732c0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f732be00640 1 -- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f732c0c58a0 msgr2=0x7f732c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f732be00640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f732c0c58a0 0x7f732c0c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f732be00640 1 -- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c0b5500 msgr2=0x7f732c152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f732be00640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c0b5500 0x7f732c152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f732be00640 1 -- 172.21.15.44:0/2308841431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c0b7cb0 con 0x7f73100051d0 2024-09-13T08:54:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f732be00640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73100051d0 0x7f732c152c30 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f731c003f10 tx=0x7f731c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7328c00640 1 -- 172.21.15.44:0/2308841431 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c011640 con 0x7f73100051d0 2024-09-13T08:54:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7328c00640 1 -- 172.21.15.44:0/2308841431 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f731c016020 con 0x7f73100051d0 2024-09-13T08:54:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7328c00640 1 -- 172.21.15.44:0/2308841431 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c011a80 con 0x7f73100051d0 2024-09-13T08:54:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7333200640 1 -- 172.21.15.44:0/2308841431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732c1584c0 con 0x7f73100051d0 2024-09-13T08:54:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7333200640 1 -- 172.21.15.44:0/2308841431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732c158780 con 0x7f73100051d0 2024-09-13T08:54:43.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7333200640 1 -- 172.21.15.44:0/2308841431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732c0cf280 con 0x7f73100051d0 2024-09-13T08:54:43.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.442+0000 7f7328c00640 1 -- 172.21.15.44:0/2308841431 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f731c011c20 con 0x7f73100051d0 2024-09-13T08:54:43.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.446+0000 7f7328c00640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7308075650 0x7f7308077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:43.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.446+0000 7f7328c00640 1 -- 172.21.15.44:0/2308841431 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5993+0+0 (secure 0 0 0) 0x7f731c093c70 con 0x7f73100051d0 2024-09-13T08:54:43.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.446+0000 7f7323e00640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7308075650 0x7f7308077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:43.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.446+0000 7f7323e00640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7308075650 0x7f7308077b10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f732c0b2ae0 tx=0x7f73140091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:43.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.454+0000 7f7328c00640 1 -- 172.21.15.44:0/2308841431 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f731c020020 con 0x7f73100051d0 2024-09-13T08:54:43.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:43.634+0000 7f7333200640 1 -- 172.21.15.44:0/2308841431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f732c0c55e0 con 0x7f73100051d0 2024-09-13T08:54:44.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.258+0000 7f7328c00640 1 -- 172.21.15.44:0/2308841431 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v149) ==== 181+0+0 (secure 0 0 0) 0x7f731c060920 con 0x7f73100051d0 2024-09-13T08:54:44.257 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-09-13T08:54:44.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 -- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7308075650 msgr2=0x7f7308077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7308075650 0x7f7308077b10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f732c0b2ae0 tx=0x7f73140091e0 comp rx=0 tx=0).stop 2024-09-13T08:54:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 -- 172.21.15.44:0/2308841431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73100051d0 msgr2=0x7f732c152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73100051d0 0x7f732c152c30 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f731c003f10 tx=0x7f731c001640 comp rx=0 tx=0).stop 2024-09-13T08:54:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 -- 172.21.15.44:0/2308841431 shutdown_connections 2024-09-13T08:54:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f7308075650 0x7f7308077b10 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f732c0c58a0 0x7f732c0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c0b5500 0x7f732c152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 --2- 172.21.15.44:0/2308841431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73100051d0 0x7f732c152c30 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:44.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 -- 172.21.15.44:0/2308841431 >> 172.21.15.44:0/2308841431 conn(0x7f732c0d1da0 msgr2=0x7f732c071640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 -- 172.21.15.44:0/2308841431 shutdown_connections 2024-09-13T08:54:44.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.262+0000 7f7333200640 1 -- 172.21.15.44:0/2308841431 wait complete. 2024-09-13T08:54:44.285 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T08:54:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.450+0000 7fb729400640 1 -- 172.21.15.44:0/2012840382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7080051d0 msgr2=0x7fb70800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.450+0000 7fb729400640 1 --2- 172.21.15.44:0/2012840382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7080051d0 0x7fb70800b9d0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fb71c001e60 tx=0x7fb71c01cbe0 comp rx=0 tx=0).stop 2024-09-13T08:54:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 -- 172.21.15.44:0/2012840382 shutdown_connections 2024-09-13T08:54:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 --2- 172.21.15.44:0/2012840382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7080051d0 0x7fb70800b9d0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 -- 172.21.15.44:0/2012840382 >> 172.21.15.44:0/2012840382 conn(0x7fb7240d1da0 msgr2=0x7fb7240d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 -- 172.21.15.44:0/2012840382 shutdown_connections 2024-09-13T08:54:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 -- 172.21.15.44:0/2012840382 wait complete. 2024-09-13T08:54:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 Processor -- start 2024-09-13T08:54:44.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 -- start start 2024-09-13T08:54:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7080051d0 0x7fb7240c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7241f4490 0x7fb7241f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb7240cd2b0 0x7fb7240cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7240ceba0 con 0x7fb7241f4490 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7240cea20 con 0x7fb7080051d0 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fb7240ced20 con 0x7fb7240cd2b0 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7241f4490 0x7fb7241f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7241f4490 0x7fb7241f4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50672/0 (socket says 172.21.15.44:50672) 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71be00640 1 -- 172.21.15.44:0/3891735452 learned_addr learned my addr 172.21.15.44:0/3891735452 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb722800640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7080051d0 0x7fb7240c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb723200640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb7240cd2b0 0x7fb7240cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71be00640 1 -- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb7240cd2b0 msgr2=0x7fb7240cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71be00640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb7240cd2b0 0x7fb7240cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71be00640 1 -- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7080051d0 msgr2=0x7fb7240c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71be00640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7080051d0 0x7fb7240c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71be00640 1 -- 172.21.15.44:0/3891735452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7241f0a00 con 0x7fb7241f4490 2024-09-13T08:54:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb723200640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb7240cd2b0 0x7fb7240cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T08:54:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71be00640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7241f4490 0x7fb7241f4910 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fb70c006660 tx=0x7fb70c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71aa00640 1 -- 172.21.15.44:0/3891735452 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c015070 con 0x7fb7241f4490 2024-09-13T08:54:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71aa00640 1 -- 172.21.15.44:0/3891735452 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb70c011920 con 0x7fb7241f4490 2024-09-13T08:54:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb71aa00640 1 -- 172.21.15.44:0/3891735452 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c011d70 con 0x7fb7241f4490 2024-09-13T08:54:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.454+0000 7fb729400640 1 -- 172.21.15.44:0/3891735452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7241f0ce0 con 0x7fb7241f4490 2024-09-13T08:54:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.458+0000 7fb729400640 1 -- 172.21.15.44:0/3891735452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7241f12a0 con 0x7fb7241f4490 2024-09-13T08:54:44.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.458+0000 7fb729400640 1 -- 172.21.15.44:0/3891735452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7240cf280 con 0x7fb7241f4490 2024-09-13T08:54:44.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.458+0000 7fb71aa00640 1 -- 172.21.15.44:0/3891735452 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fb70c018070 con 0x7fb7241f4490 2024-09-13T08:54:44.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.458+0000 7fb71aa00640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb700075620 0x7fb700077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:44.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.458+0000 7fb71aa00640 1 -- 172.21.15.44:0/3891735452 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5555+0+0 (secure 0 0 0) 0x7fb70c093ae0 con 0x7fb7241f4490 2024-09-13T08:54:44.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.458+0000 7fb722800640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb700075620 0x7fb700077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:44.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.462+0000 7fb722800640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb700075620 0x7fb700077ae0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb7241549d0 tx=0x7fb7140091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.462+0000 7fb71aa00640 1 -- 172.21.15.44:0/3891735452 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fb70c060790 con 0x7fb7241f4490 2024-09-13T08:54:44.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:44.638+0000 7fb729400640 1 -- 172.21.15.44:0/3891735452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb7240b0940 con 0x7fb7241f4490 2024-09-13T08:54:45.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.306+0000 7fb71aa00640 1 -- 172.21.15.44:0/3891735452 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v151) ==== 169+0+0 (secure 0 0 0) 0x7fb70c060090 con 0x7fb7241f4490 2024-09-13T08:54:45.303 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-09-13T08:54:45.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 -- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb700075620 msgr2=0x7fb700077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:45.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb700075620 0x7fb700077ae0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb7241549d0 tx=0x7fb7140091e0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 -- 172.21.15.44:0/3891735452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7241f4490 msgr2=0x7fb7241f4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:45.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7241f4490 0x7fb7241f4910 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fb70c006660 tx=0x7fb70c0039d0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 -- 172.21.15.44:0/3891735452 shutdown_connections 2024-09-13T08:54:45.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fb700075620 0x7fb700077ae0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fb7240cd2b0 0x7fb7240cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7241f4490 0x7fb7241f4910 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 --2- 172.21.15.44:0/3891735452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7080051d0 0x7fb7240c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 -- 172.21.15.44:0/3891735452 >> 172.21.15.44:0/3891735452 conn(0x7fb7240d1da0 msgr2=0x7fb7240b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:45.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 -- 172.21.15.44:0/3891735452 shutdown_connections 2024-09-13T08:54:45.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.310+0000 7fb729400640 1 -- 172.21.15.44:0/3891735452 wait complete. 2024-09-13T08:54:45.337 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-13T08:54:45.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.506+0000 7fa8cec00640 1 -- 172.21.15.44:0/99942660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8ac004030 msgr2=0x7fa8ac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:45.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.506+0000 7fa8cec00640 1 --2- 172.21.15.44:0/99942660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8ac004030 0x7fa8ac004810 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009b60 tx=0x7fa8b001d270 comp rx=0 tx=0).stop 2024-09-13T08:54:45.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.506+0000 7fa8ccc00640 1 -- 172.21.15.44:0/99942660 reap_dead start 2024-09-13T08:54:45.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.506+0000 7fa8cec00640 1 -- 172.21.15.44:0/99942660 shutdown_connections 2024-09-13T08:54:45.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.506+0000 7fa8cec00640 1 -- 172.21.15.44:0/99942660 >> 172.21.15.44:0/99942660 conn(0x7fa8c80d1dc0 msgr2=0x7fa8c80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:45.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.506+0000 7fa8cec00640 1 -- 172.21.15.44:0/99942660 shutdown_connections 2024-09-13T08:54:45.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.506+0000 7fa8cec00640 1 -- 172.21.15.44:0/99942660 wait complete. 2024-09-13T08:54:45.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.506+0000 7fa8cec00640 1 Processor -- start 2024-09-13T08:54:45.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.506+0000 7fa8cec00640 1 -- start start 2024-09-13T08:54:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8cec00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac004030 0x7fa8c80b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8cec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81f3040 0x7fa8c80cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8cec00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8c815dbe0 0x7fa8c815e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8cec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c815c480 con 0x7fa8c81f3040 2024-09-13T08:54:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8cec00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c80d2af0 con 0x7fa8ac004030 2024-09-13T08:54:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8cec00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c80ae9b0 con 0x7fa8c815dbe0 2024-09-13T08:54:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8bf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81f3040 0x7fa8c80cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8bf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81f3040 0x7fa8c80cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50692/0 (socket says 172.21.15.44:50692) 2024-09-13T08:54:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8bf400640 1 -- 172.21.15.44:0/2585543515 learned_addr learned my addr 172.21.15.44:0/2585543515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8c7e00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac004030 0x7fa8c80b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8ccc00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8c815dbe0 0x7fa8c815e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8bf400640 1 -- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8c815dbe0 msgr2=0x7fa8c815e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8bf400640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8c815dbe0 0x7fa8c815e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8bf400640 1 -- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac004030 msgr2=0x7fa8c80b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8bf400640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac004030 0x7fa8c80b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8bf400640 1 -- 172.21.15.44:0/2585543515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c80b1780 con 0x7fa8c81f3040 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8c7e00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac004030 0x7fa8c80b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8bf400640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81f3040 0x7fa8c80cb2c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fa8b001d0a0 tx=0x7fa8b0005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8c4c00640 1 -- 172.21.15.44:0/2585543515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b00b1040 con 0x7fa8c81f3040 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8c4c00640 1 -- 172.21.15.44:0/2585543515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8b0004310 con 0x7fa8c81f3040 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8c4c00640 1 -- 172.21.15.44:0/2585543515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0009730 con 0x7fa8c81f3040 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8cec00640 1 -- 172.21.15.44:0/2585543515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c80b1a60 con 0x7fa8c81f3040 2024-09-13T08:54:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8cec00640 1 -- 172.21.15.44:0/2585543515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c80b2020 con 0x7fa8c81f3040 2024-09-13T08:54:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8cec00640 1 -- 172.21.15.44:0/2585543515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c80ce670 con 0x7fa8c81f3040 2024-09-13T08:54:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.510+0000 7fa8c4c00640 1 -- 172.21.15.44:0/2585543515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fa8b00bf060 con 0x7fa8c81f3040 2024-09-13T08:54:45.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.514+0000 7fa8c4c00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8a4075620 0x7fa8a4077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:45.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.514+0000 7fa8c4c00640 1 -- 172.21.15.44:0/2585543515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fa8b012d140 con 0x7fa8c81f3040 2024-09-13T08:54:45.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.514+0000 7fa8c7e00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8a4075620 0x7fa8a4077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:45.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.514+0000 7fa8c7e00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8a4075620 0x7fa8a4077ae0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8001bf0 tx=0x7fa8b8001b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:45.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.518+0000 7fa8c4c00640 1 -- 172.21.15.44:0/2585543515 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fa8b0016f80 con 0x7fa8c81f3040 2024-09-13T08:54:45.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.694+0000 7fa8cec00640 1 -- 172.21.15.44:0/2585543515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fa8c815b8a0 con 0x7fa8c81f3040 2024-09-13T08:54:45.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.698+0000 7fa8c4c00640 1 -- 172.21.15.44:0/2585543515 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v57) ==== 71+0+0 (secure 0 0 0) 0x7fa8b00f9840 con 0x7fa8c81f3040 2024-09-13T08:54:45.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 -- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8a4075620 msgr2=0x7fa8a4077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:45.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8a4075620 0x7fa8a4077ae0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8001bf0 tx=0x7fa8b8001b80 comp rx=0 tx=0).stop 2024-09-13T08:54:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 -- 172.21.15.44:0/2585543515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81f3040 msgr2=0x7fa8c80cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81f3040 0x7fa8c80cb2c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fa8b001d0a0 tx=0x7fa8b0005520 comp rx=0 tx=0).stop 2024-09-13T08:54:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 -- 172.21.15.44:0/2585543515 shutdown_connections 2024-09-13T08:54:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fa8a4075620 0x7fa8a4077ae0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fa8c815dbe0 0x7fa8c815e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81f3040 0x7fa8c80cb2c0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 --2- 172.21.15.44:0/2585543515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac004030 0x7fa8c80b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 -- 172.21.15.44:0/2585543515 >> 172.21.15.44:0/2585543515 conn(0x7fa8c80d1dc0 msgr2=0x7fa8c80b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 -- 172.21.15.44:0/2585543515 shutdown_connections 2024-09-13T08:54:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.702+0000 7fa8cec00640 1 -- 172.21.15.44:0/2585543515 wait complete. 2024-09-13T08:54:45.721 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-13T08:54:45.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 -- 172.21.15.44:0/3945927853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b240d01b0 msgr2=0x7f8b240cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 --2- 172.21.15.44:0/3945927853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b240d01b0 0x7f8b240cefb0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009840 tx=0x7f8b0c01c790 comp rx=0 tx=0).stop 2024-09-13T08:54:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 -- 172.21.15.44:0/3945927853 shutdown_connections 2024-09-13T08:54:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 --2- 172.21.15.44:0/3945927853 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b240b2590 0x7f8b240b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 --2- 172.21.15.44:0/3945927853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b240d01b0 0x7f8b240cefb0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 -- 172.21.15.44:0/3945927853 >> 172.21.15.44:0/3945927853 conn(0x7f8b2409aa10 msgr2=0x7f8b2409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 -- 172.21.15.44:0/3945927853 shutdown_connections 2024-09-13T08:54:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 -- 172.21.15.44:0/3945927853 wait complete. 2024-09-13T08:54:45.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 Processor -- start 2024-09-13T08:54:45.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 -- start start 2024-09-13T08:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b240b2590 0x7f8b24166230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b240d01b0 0x7f8b240c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b2415f000 0x7f8b240c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b24168ea0 con 0x7f8b240b2590 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8b24168d20 con 0x7f8b2415f000 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.870+0000 7f8b23200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8b24169020 con 0x7f8b240d01b0 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b21e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b240b2590 0x7f8b24166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b1be00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b240d01b0 0x7f8b240c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b21e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b240b2590 0x7f8b24166230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50710/0 (socket says 172.21.15.44:50710) 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b21e00640 1 -- 172.21.15.44:0/3606866587 learned_addr learned my addr 172.21.15.44:0/3606866587 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b22800640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b2415f000 0x7f8b240c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b21e00640 1 -- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b240d01b0 msgr2=0x7f8b240c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b21e00640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b240d01b0 0x7f8b240c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b21e00640 1 -- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b2415f000 msgr2=0x7f8b240c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b21e00640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b2415f000 0x7f8b240c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b21e00640 1 -- 172.21.15.44:0/3606866587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b240b6960 con 0x7f8b240b2590 2024-09-13T08:54:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b21e00640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b240b2590 0x7f8b24166230 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c400 tx=0x7f8b1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b1a000640 1 -- 172.21.15.44:0/3606866587 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14017070 con 0x7f8b240b2590 2024-09-13T08:54:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b1a000640 1 -- 172.21.15.44:0/3606866587 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b14003600 con 0x7f8b240b2590 2024-09-13T08:54:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b1a000640 1 -- 172.21.15.44:0/3606866587 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14003a50 con 0x7f8b240b2590 2024-09-13T08:54:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b23200640 1 -- 172.21.15.44:0/3606866587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b240c6d80 con 0x7f8b240b2590 2024-09-13T08:54:45.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b23200640 1 -- 172.21.15.44:0/3606866587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b240c7340 con 0x7f8b240b2590 2024-09-13T08:54:45.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b23200640 1 -- 172.21.15.44:0/3606866587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b240b74f0 con 0x7f8b240b2590 2024-09-13T08:54:45.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.874+0000 7f8b1a000640 1 -- 172.21.15.44:0/3606866587 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f8b1401a070 con 0x7f8b240b2590 2024-09-13T08:54:45.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.878+0000 7f8b1a000640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b00075650 0x7f8b00077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:45.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.878+0000 7f8b1a000640 1 -- 172.21.15.44:0/3606866587 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f8b14092440 con 0x7f8b240b2590 2024-09-13T08:54:45.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.878+0000 7f8b1be00640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b00075650 0x7f8b00077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:45.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.878+0000 7f8b1be00640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b00075650 0x7f8b00077b10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0ae000 tx=0x7f8b0c01afd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:45.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:45.882+0000 7f8b1a000640 1 -- 172.21.15.44:0/3606866587 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8b1405ea10 con 0x7f8b240b2590 2024-09-13T08:54:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.118+0000 7f8b23200640 1 -- 172.21.15.44:0/3606866587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7f8b240b0af0 con 0x7f8b240b2590 2024-09-13T08:54:46.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.306+0000 7f8b1a000640 1 -- 172.21.15.44:0/3606866587 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v779) ==== 186+0+0 (secure 0 0 0) 0x7f8b14063440 con 0x7f8b240b2590 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.306+0000 7f8b23200640 1 -- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b00075650 msgr2=0x7f8b00077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.306+0000 7f8b23200640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b00075650 0x7f8b00077b10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0ae000 tx=0x7f8b0c01afd0 comp rx=0 tx=0).stop 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.306+0000 7f8b23200640 1 -- 172.21.15.44:0/3606866587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b240b2590 msgr2=0x7f8b24166230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.306+0000 7f8b23200640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b240b2590 0x7f8b24166230 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c400 tx=0x7f8b1400c8c0 comp rx=0 tx=0).stop 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.310+0000 7f8b23200640 1 -- 172.21.15.44:0/3606866587 shutdown_connections 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.310+0000 7f8b23200640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8b00075650 0x7f8b00077b10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.310+0000 7f8b23200640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b2415f000 0x7f8b240c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.310+0000 7f8b23200640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8b240d01b0 0x7f8b240c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.310+0000 7f8b23200640 1 --2- 172.21.15.44:0/3606866587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b240b2590 0x7f8b24166230 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.310+0000 7f8b23200640 1 -- 172.21.15.44:0/3606866587 >> 172.21.15.44:0/3606866587 conn(0x7f8b2409aa10 msgr2=0x7f8b240ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.310+0000 7f8b23200640 1 -- 172.21.15.44:0/3606866587 shutdown_connections 2024-09-13T08:54:46.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:46.310+0000 7f8b23200640 1 -- 172.21.15.44:0/3606866587 wait complete. 2024-09-13T08:54:46.329 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone ... ok 2024-09-13T08:54:46.329 INFO:tasks.cephfs_test_runner: 2024-09-13T08:54:46.329 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-13T08:54:46.329 INFO:tasks.cephfs_test_runner:Ran 9 tests in 1005.122s 2024-09-13T08:54:46.329 INFO:tasks.cephfs_test_runner: 2024-09-13T08:54:46.329 INFO:tasks.cephfs_test_runner:OK 2024-09-13T08:54:46.330 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-13T08:54:46.343 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-13T08:54:46.409 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-13T08:54:46.409 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:54:46.409 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:54:46.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:54:46.418 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T08:54:46.419 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:54:46.466 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T08:54:46.467 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:54:46.467 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:54:46.476 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:54:46.476 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T08:54:46.476 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T08:54:46.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350232k,nr_inodes=4087558,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5616 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T08:54:46.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:46.531 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:54:46.539 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T08:54:46.540 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T08:54:46.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:46.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T08:54:46.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:46.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T08:54:47.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:47.021 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-09-13T08:54:47.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:47.191 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-09-13T08:54:47.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:47.201 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-09-13T08:54:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2024-09-13T08:54:47.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:54:47.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:47.648 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-09-13T08:54:47.658 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-13T08:54:47.658 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-13T08:54:47.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:47.658 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T08:54:47.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:47.724 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-13T08:54:48.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:48.215 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-09-13T08:54:48.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:48.362 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-09-13T08:54:48.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T08:54:48.372 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-09-13T08:54:48.796 INFO:teuthology.orchestra.run.smithi129.stderr:Cannot find device "ceph-brx" 2024-09-13T08:54:48.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:54:48.798 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T08:54:48.810 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T08:54:48.810 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:54:48.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 -- 172.21.15.44:0/3399161995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c0d28e0 msgr2=0x7f245c0cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 --2- 172.21.15.44:0/3399161995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c0d28e0 0x7f245c0cc9d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f245c0745d0 tx=0x7f245401cd80 comp rx=0 tx=0).stop 2024-09-13T08:54:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 -- 172.21.15.44:0/3399161995 shutdown_connections 2024-09-13T08:54:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 --2- 172.21.15.44:0/3399161995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c0d28e0 0x7f245c0cc9d0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 --2- 172.21.15.44:0/3399161995 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f245c0cdb30 0x7f245c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 --2- 172.21.15.44:0/3399161995 >> v2:172.21.15.129:3300/0 conn(0x7f245c15bea0 0x7f245c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 -- 172.21.15.44:0/3399161995 >> 172.21.15.44:0/3399161995 conn(0x7f245c0d1dc0 msgr2=0x7f245c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 -- 172.21.15.44:0/3399161995 shutdown_connections 2024-09-13T08:54:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 -- 172.21.15.44:0/3399161995 wait complete. 2024-09-13T08:54:48.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 Processor -- start 2024-09-13T08:54:48.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.970+0000 7f2460c00640 1 -- start start 2024-09-13T08:54:48.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2460c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c0cdb30 0x7f245c1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2460c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f245c0d28e0 0x7f245c0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2460c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c15bea0 0x7f245c157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2460c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f245c0af120 con 0x7f245c0cdb30 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2460c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f245c0aefa0 con 0x7f245c15bea0 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2460c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f245c0af2a0 con 0x7f245c0d28e0 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c0cdb30 0x7f245c1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c0cdb30 0x7f245c1524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50726/0 (socket says 172.21.15.44:50726) 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459e00640 1 -- 172.21.15.44:0/4282696694 learned_addr learned my addr 172.21.15.44:0/4282696694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459400640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f245c0d28e0 0x7f245c0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f245a800640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c15bea0 0x7f245c157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459e00640 1 -- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f245c0d28e0 msgr2=0x7f245c0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459e00640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f245c0d28e0 0x7f245c0ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459e00640 1 -- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c15bea0 msgr2=0x7f245c157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:48.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459e00640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c15bea0 0x7f245c157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:48.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459e00640 1 -- 172.21.15.44:0/4282696694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c0cac20 con 0x7f245c0cdb30 2024-09-13T08:54:48.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f245a800640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c15bea0 0x7f245c157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:48.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459400640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f245c0d28e0 0x7f245c0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:48.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2459e00640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c0cdb30 0x7f245c1524e0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f244c00d750 tx=0x7f244c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:48.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2452000640 1 -- 172.21.15.44:0/4282696694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c017070 con 0x7f245c0cdb30 2024-09-13T08:54:48.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2452000640 1 -- 172.21.15.44:0/4282696694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f244c004310 con 0x7f245c0cdb30 2024-09-13T08:54:48.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2460c00640 1 -- 172.21.15.44:0/4282696694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c1558c0 con 0x7f245c0cdb30 2024-09-13T08:54:48.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2452000640 1 -- 172.21.15.44:0/4282696694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c00de60 con 0x7f245c0cdb30 2024-09-13T08:54:48.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2460c00640 1 -- 172.21.15.44:0/4282696694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c155300 con 0x7f245c0cdb30 2024-09-13T08:54:48.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.974+0000 7f2460c00640 1 -- 172.21.15.44:0/4282696694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c0af830 con 0x7f245c0cdb30 2024-09-13T08:54:48.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.978+0000 7f2452000640 1 -- 172.21.15.44:0/4282696694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f244c009c30 con 0x7f245c0cdb30 2024-09-13T08:54:48.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.978+0000 7f2452000640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f243c075620 0x7f243c077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:48.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.978+0000 7f2452000640 1 -- 172.21.15.44:0/4282696694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f244c093b60 con 0x7f245c0cdb30 2024-09-13T08:54:48.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.978+0000 7f2459400640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f243c075620 0x7f243c077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:48.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.978+0000 7f2459400640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f243c075620 0x7f243c077ae0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f245c154c70 tx=0x7f24480091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:48.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:48.982+0000 7f2452000640 1 -- 172.21.15.44:0/4282696694 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f244c020020 con 0x7f245c0cdb30 2024-09-13T08:54:49.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.154+0000 7f2460c00640 1 -- 172.21.15.44:0/4282696694 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f245c156980 con 0x7f243c075620 2024-09-13T08:54:49.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.154+0000 7f2452000640 1 -- 172.21.15.44:0/4282696694 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13564 (secure 0 0 0) 0x7f245c156980 con 0x7f243c075620 2024-09-13T08:54:49.153 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:54:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 -- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f243c075620 msgr2=0x7f243c077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f243c075620 0x7f243c077ae0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f245c154c70 tx=0x7f24480091e0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 -- 172.21.15.44:0/4282696694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c0cdb30 msgr2=0x7f245c1524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c0cdb30 0x7f245c1524e0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f244c00d750 tx=0x7f244c00dc10 comp rx=0 tx=0).stop 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 -- 172.21.15.44:0/4282696694 shutdown_connections 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f243c075620 0x7f243c077ae0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c15bea0 0x7f245c157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f245c0d28e0 0x7f245c0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 --2- 172.21.15.44:0/4282696694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c0cdb30 0x7f245c1524e0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 -- 172.21.15.44:0/4282696694 >> 172.21.15.44:0/4282696694 conn(0x7f245c0d1dc0 msgr2=0x7f245c159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 -- 172.21.15.44:0/4282696694 shutdown_connections 2024-09-13T08:54:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.158+0000 7f2460c00640 1 -- 172.21.15.44:0/4282696694 wait complete. 2024-09-13T08:54:49.176 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":667,"stamp":"2024-09-13T08:54:48.956817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":1157072,"kb_used_data":19564,"kb_used_omap":12,"kb_used_meta":1137331,"kb_avail":753817648,"statfs":{"total":773094113280,"available":771909271552,"internally_reserved":0,"allocated":20033536,"data_stored":11753503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1164627505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3189631551,"num_objects":-786,"num_object_clones":0,"num_object_copies":-1572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-786,"num_whiteouts":0,"num_read":-3599,"num_read_kb":-14327909,"num_write":-8435,"num_write_kb":-17336486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-95,"num_bytes_recovered":-394264576,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001212"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":355,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-13T08:54:42.253541+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:54:42.253541+0000","last_peered":"2024-09-13T08:54:42.253541+0000","last_clean":"2024-09-13T08:54:42.253541+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:54:42.253541+0000","last_undegraded":"2024-09-13T08:54:42.253541+0000","last_fullsized":"2024-09-13T08:54:42.253541+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509650,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119004,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252836,"statfs":{"total":96636764160,"available":96514904064,"internally_reserved":0,"allocated":1843200,"data_stored":819848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147924,"kb_used_data":4352,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94223916,"statfs":{"total":96636764160,"available":96485289984,"internally_reserved":0,"allocated":4456448,"data_stored":3407756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":165380,"kb_used_data":3760,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94206460,"statfs":{"total":96636764160,"available":96467415040,"internally_reserved":0,"allocated":3850240,"data_stored":2809811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509651,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141220,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230620,"statfs":{"total":96636764160,"available":96492154880,"internally_reserved":0,"allocated":1851392,"data_stored":820464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509650,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141216,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230624,"statfs":{"total":96636764160,"available":96492158976,"internally_reserved":0,"allocated":1847296,"data_stored":819870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509652,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137636,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234204,"statfs":{"total":96636764160,"available":96495824896,"internally_reserved":0,"allocated":2441216,"data_stored":1410286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509650,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141224,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230616,"statfs":{"total":96636764160,"available":96492150784,"internally_reserved":0,"allocated":1855488,"data_stored":827925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542357,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163468,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208372,"statfs":{"total":96636764160,"available":96469372928,"internally_reserved":0,"allocated":1888256,"data_stored":837543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165476806},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:54:49.176 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:54:49.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 -- 172.21.15.44:0/3104284045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640051d0 msgr2=0x7fc36400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 --2- 172.21.15.44:0/3104284045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640051d0 0x7fc36400b9d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fc378009ee0 tx=0x7fc37801d630 comp rx=0 tx=0).stop 2024-09-13T08:54:49.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 -- 172.21.15.44:0/3104284045 shutdown_connections 2024-09-13T08:54:49.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 --2- 172.21.15.44:0/3104284045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640051d0 0x7fc36400b9d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 -- 172.21.15.44:0/3104284045 >> 172.21.15.44:0/3104284045 conn(0x7fc3800d1dc0 msgr2=0x7fc3800d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:49.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 -- 172.21.15.44:0/3104284045 shutdown_connections 2024-09-13T08:54:49.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 -- 172.21.15.44:0/3104284045 wait complete. 2024-09-13T08:54:49.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 Processor -- start 2024-09-13T08:54:49.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 -- start start 2024-09-13T08:54:49.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640051d0 0x7fc3800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:49.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc38015cdb0 0x7fc3800b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:49.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3800b6270 0x7fc3800b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:49.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3800d2a60 con 0x7fc3640051d0 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3800d28e0 con 0x7fc3800b6270 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.326+0000 7fc386400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc3800d2be0 con 0x7fc38015cdb0 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640051d0 0x7fc3800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640051d0 0x7fc3800b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50746/0 (socket says 172.21.15.44:50746) 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37f400640 1 -- 172.21.15.44:0/1541654409 learned_addr learned my addr 172.21.15.44:0/1541654409 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc376a00640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc38015cdb0 0x7fc3800b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37fe00640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3800b6270 0x7fc3800b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37f400640 1 -- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc38015cdb0 msgr2=0x7fc3800b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37f400640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc38015cdb0 0x7fc3800b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37f400640 1 -- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3800b6270 msgr2=0x7fc3800b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37f400640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3800b6270 0x7fc3800b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37f400640 1 -- 172.21.15.44:0/1541654409 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38015d230 con 0x7fc3640051d0 2024-09-13T08:54:49.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc376a00640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc38015cdb0 0x7fc3800b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:49.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37fe00640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3800b6270 0x7fc3800b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:49.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc37f400640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640051d0 0x7fc3800b4bf0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc3700066b0 tx=0x7fc3700039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:49.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc377e00640 1 -- 172.21.15.44:0/1541654409 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370015070 con 0x7fc3640051d0 2024-09-13T08:54:49.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc377e00640 1 -- 172.21.15.44:0/1541654409 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc370011920 con 0x7fc3640051d0 2024-09-13T08:54:49.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc377e00640 1 -- 172.21.15.44:0/1541654409 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370011d70 con 0x7fc3640051d0 2024-09-13T08:54:49.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc386400640 1 -- 172.21.15.44:0/1541654409 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3800b51d0 con 0x7fc3640051d0 2024-09-13T08:54:49.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc386400640 1 -- 172.21.15.44:0/1541654409 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3800b5790 con 0x7fc3640051d0 2024-09-13T08:54:49.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc386400640 1 -- 172.21.15.44:0/1541654409 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3800ce910 con 0x7fc3640051d0 2024-09-13T08:54:49.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.330+0000 7fc377e00640 1 -- 172.21.15.44:0/1541654409 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fc370018070 con 0x7fc3640051d0 2024-09-13T08:54:49.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.334+0000 7fc377e00640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc35c075650 0x7fc35c077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:49.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.334+0000 7fc377e00640 1 -- 172.21.15.44:0/1541654409 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fc370093de0 con 0x7fc3640051d0 2024-09-13T08:54:49.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.334+0000 7fc376a00640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc35c075650 0x7fc35c077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:49.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.334+0000 7fc376a00640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc35c075650 0x7fc35c077b10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc368006fd0 tx=0x7fc368006f60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:49.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.338+0000 7fc377e00640 1 -- 172.21.15.44:0/1541654409 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc370060b10 con 0x7fc3640051d0 2024-09-13T08:54:49.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.514+0000 7fc386400640 1 -- 172.21.15.44:0/1541654409 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3800afbd0 con 0x7fc35c075650 2024-09-13T08:54:49.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.514+0000 7fc377e00640 1 -- 172.21.15.44:0/1541654409 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13564 (secure 0 0 0) 0x7fc3800afbd0 con 0x7fc35c075650 2024-09-13T08:54:49.513 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:49.514 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:54:49.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 -- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc35c075650 msgr2=0x7fc35c077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc35c075650 0x7fc35c077b10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc368006fd0 tx=0x7fc368006f60 comp rx=0 tx=0).stop 2024-09-13T08:54:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 -- 172.21.15.44:0/1541654409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640051d0 msgr2=0x7fc3800b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640051d0 0x7fc3800b4bf0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc3700066b0 tx=0x7fc3700039d0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 -- 172.21.15.44:0/1541654409 shutdown_connections 2024-09-13T08:54:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc35c075650 0x7fc35c077b10 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3800b6270 0x7fc3800b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc38015cdb0 0x7fc3800b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 --2- 172.21.15.44:0/1541654409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640051d0 0x7fc3800b4bf0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 -- 172.21.15.44:0/1541654409 >> 172.21.15.44:0/1541654409 conn(0x7fc3800d1dc0 msgr2=0x7fc38006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 -- 172.21.15.44:0/1541654409 shutdown_connections 2024-09-13T08:54:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.518+0000 7fc386400640 1 -- 172.21.15.44:0/1541654409 wait complete. 2024-09-13T08:54:49.535 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":667,"stamp":"2024-09-13T08:54:48.956817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":1157072,"kb_used_data":19564,"kb_used_omap":12,"kb_used_meta":1137331,"kb_avail":753817648,"statfs":{"total":773094113280,"available":771909271552,"internally_reserved":0,"allocated":20033536,"data_stored":11753503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1164627505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3189631551,"num_objects":-786,"num_object_clones":0,"num_object_copies":-1572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-786,"num_whiteouts":0,"num_read":-3599,"num_read_kb":-14327909,"num_write":-8435,"num_write_kb":-17336486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-95,"num_bytes_recovered":-394264576,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001212"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":355,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-13T08:54:42.253541+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:54:42.253541+0000","last_peered":"2024-09-13T08:54:42.253541+0000","last_clean":"2024-09-13T08:54:42.253541+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:54:42.253541+0000","last_undegraded":"2024-09-13T08:54:42.253541+0000","last_fullsized":"2024-09-13T08:54:42.253541+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509650,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119004,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252836,"statfs":{"total":96636764160,"available":96514904064,"internally_reserved":0,"allocated":1843200,"data_stored":819848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147924,"kb_used_data":4352,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94223916,"statfs":{"total":96636764160,"available":96485289984,"internally_reserved":0,"allocated":4456448,"data_stored":3407756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":165380,"kb_used_data":3760,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94206460,"statfs":{"total":96636764160,"available":96467415040,"internally_reserved":0,"allocated":3850240,"data_stored":2809811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509651,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141220,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230620,"statfs":{"total":96636764160,"available":96492154880,"internally_reserved":0,"allocated":1851392,"data_stored":820464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509650,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141216,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230624,"statfs":{"total":96636764160,"available":96492158976,"internally_reserved":0,"allocated":1847296,"data_stored":819870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509652,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137636,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234204,"statfs":{"total":96636764160,"available":96495824896,"internally_reserved":0,"allocated":2441216,"data_stored":1410286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509650,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141224,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230616,"statfs":{"total":96636764160,"available":96492150784,"internally_reserved":0,"allocated":1855488,"data_stored":827925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542357,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163468,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208372,"statfs":{"total":96636764160,"available":96469372928,"internally_reserved":0,"allocated":1888256,"data_stored":837543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165476806},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:54:49.536 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T08:54:49.536 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:54:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 -- 172.21.15.44:0/1485554876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7980cdb30 msgr2=0x7fd7980af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 --2- 172.21.15.44:0/1485554876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7980cdb30 0x7fd7980af0c0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fd790009bc0 tx=0x7fd79001cba0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 -- 172.21.15.44:0/1485554876 shutdown_connections 2024-09-13T08:54:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 --2- 172.21.15.44:0/1485554876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7980cdb30 0x7fd7980af0c0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 -- 172.21.15.44:0/1485554876 >> 172.21.15.44:0/1485554876 conn(0x7fd7980d1dc0 msgr2=0x7fd7980d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 -- 172.21.15.44:0/1485554876 shutdown_connections 2024-09-13T08:54:49.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 -- 172.21.15.44:0/1485554876 wait complete. 2024-09-13T08:54:49.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 Processor -- start 2024-09-13T08:54:49.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 -- start start 2024-09-13T08:54:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7980cdb30 0x7fd79806ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7980ced80 0x7fd79806ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd79815bea0 0x7fd79806f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd79816a840 con 0x7fd7980ced80 2024-09-13T08:54:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd79816a6c0 con 0x7fd7980cdb30 2024-09-13T08:54:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd79e200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd79816a9c0 con 0x7fd79815bea0 2024-09-13T08:54:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd78ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7980ced80 0x7fd79806ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:49.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd78ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7980ced80 0x7fd79806ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50750/0 (socket says 172.21.15.44:50750) 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd78ea00640 1 -- 172.21.15.44:0/444898461 learned_addr learned my addr 172.21.15.44:0/444898461 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd797e00640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd79815bea0 0x7fd79806f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd797400640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7980cdb30 0x7fd79806ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd78ea00640 1 -- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd79815bea0 msgr2=0x7fd79806f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd78ea00640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd79815bea0 0x7fd79806f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd78ea00640 1 -- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7980cdb30 msgr2=0x7fd79806ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd78ea00640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7980cdb30 0x7fd79806ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd78ea00640 1 -- 172.21.15.44:0/444898461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79815e060 con 0x7fd7980ced80 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.686+0000 7fd797e00640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd79815bea0 0x7fd79806f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd78ea00640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7980ced80 0x7fd79806ef40 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fd78000c500 tx=0x7fd78000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd78fe00640 1 -- 172.21.15.44:0/444898461 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780017070 con 0x7fd7980ced80 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd78fe00640 1 -- 172.21.15.44:0/444898461 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7800053c0 con 0x7fd7980ced80 2024-09-13T08:54:49.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd78fe00640 1 -- 172.21.15.44:0/444898461 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780012890 con 0x7fd7980ced80 2024-09-13T08:54:49.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd79e200640 1 -- 172.21.15.44:0/444898461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79815e2e0 con 0x7fd7980ced80 2024-09-13T08:54:49.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd79e200640 1 -- 172.21.15.44:0/444898461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79815e8a0 con 0x7fd7980ced80 2024-09-13T08:54:49.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd79e200640 1 -- 172.21.15.44:0/444898461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7980b0eb0 con 0x7fd7980ced80 2024-09-13T08:54:49.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd78fe00640 1 -- 172.21.15.44:0/444898461 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fd78001a070 con 0x7fd7980ced80 2024-09-13T08:54:49.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd78fe00640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd774075620 0x7fd774077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:49.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd78fe00640 1 -- 172.21.15.44:0/444898461 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fd780092510 con 0x7fd7980ced80 2024-09-13T08:54:49.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.690+0000 7fd797400640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd774075620 0x7fd774077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:49.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.694+0000 7fd797400640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd774075620 0x7fd774077ae0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd788001640 tx=0x7fd788009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:49.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.694+0000 7fd78fe00640 1 -- 172.21.15.44:0/444898461 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd79815e060 con 0x7fd7980ced80 2024-09-13T08:54:49.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.866+0000 7fd79e200640 1 -- 172.21.15.44:0/444898461 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd798157c40 con 0x7fd774075620 2024-09-13T08:54:49.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.866+0000 7fd78fe00640 1 -- 172.21.15.44:0/444898461 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13564 (secure 0 0 0) 0x7fd788015430 con 0x7fd774075620 2024-09-13T08:54:49.866 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:49.867 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:54:49.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 -- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd774075620 msgr2=0x7fd774077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd774075620 0x7fd774077ae0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd788001640 tx=0x7fd788009040 comp rx=0 tx=0).stop 2024-09-13T08:54:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 -- 172.21.15.44:0/444898461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7980ced80 msgr2=0x7fd79806ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7980ced80 0x7fd79806ef40 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fd78000c500 tx=0x7fd78000c9c0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 -- 172.21.15.44:0/444898461 shutdown_connections 2024-09-13T08:54:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd774075620 0x7fd774077ae0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd79815bea0 0x7fd79806f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7980ced80 0x7fd79806ef40 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 --2- 172.21.15.44:0/444898461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7980cdb30 0x7fd79806ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 -- 172.21.15.44:0/444898461 >> 172.21.15.44:0/444898461 conn(0x7fd7980d1dc0 msgr2=0x7fd7980d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 -- 172.21.15.44:0/444898461 shutdown_connections 2024-09-13T08:54:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:49.870+0000 7fd79e200640 1 -- 172.21.15.44:0/444898461 wait complete. 2024-09-13T08:54:49.888 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":667,"stamp":"2024-09-13T08:54:48.956817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":1157072,"kb_used_data":19564,"kb_used_omap":12,"kb_used_meta":1137331,"kb_avail":753817648,"statfs":{"total":773094113280,"available":771909271552,"internally_reserved":0,"allocated":20033536,"data_stored":11753503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1164627505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3189631551,"num_objects":-786,"num_object_clones":0,"num_object_copies":-1572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-786,"num_whiteouts":0,"num_read":-3599,"num_read_kb":-14327909,"num_write":-8435,"num_write_kb":-17336486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-95,"num_bytes_recovered":-394264576,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001212"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":355,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-13T08:54:42.253541+0000","last_change":"2024-09-13T08:37:25.827847+0000","last_active":"2024-09-13T08:54:42.253541+0000","last_peered":"2024-09-13T08:54:42.253541+0000","last_clean":"2024-09-13T08:54:42.253541+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:54:42.253541+0000","last_undegraded":"2024-09-13T08:54:42.253541+0000","last_fullsized":"2024-09-13T08:54:42.253541+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T08:37:24.815875+0000","last_clean_scrub_stamp":"2024-09-13T08:37:24.815875+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:14:06.151629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509650,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119004,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252836,"statfs":{"total":96636764160,"available":96514904064,"internally_reserved":0,"allocated":1843200,"data_stored":819848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147924,"kb_used_data":4352,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94223916,"statfs":{"total":96636764160,"available":96485289984,"internally_reserved":0,"allocated":4456448,"data_stored":3407756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":165380,"kb_used_data":3760,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94206460,"statfs":{"total":96636764160,"available":96467415040,"internally_reserved":0,"allocated":3850240,"data_stored":2809811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509651,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141220,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230620,"statfs":{"total":96636764160,"available":96492154880,"internally_reserved":0,"allocated":1851392,"data_stored":820464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509650,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141216,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230624,"statfs":{"total":96636764160,"available":96492158976,"internally_reserved":0,"allocated":1847296,"data_stored":819870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509652,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137636,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234204,"statfs":{"total":96636764160,"available":96495824896,"internally_reserved":0,"allocated":2441216,"data_stored":1410286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509650,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141224,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230616,"statfs":{"total":96636764160,"available":96492150784,"internally_reserved":0,"allocated":1855488,"data_stored":827925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542357,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163468,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208372,"statfs":{"total":96636764160,"available":96469372928,"internally_reserved":0,"allocated":1888256,"data_stored":837543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165476806},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:54:49.889 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T08:54:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 -- 172.21.15.44:0/446504993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5d0004bc0 msgr2=0x7fd5d0004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 --2- 172.21.15.44:0/446504993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5d0004bc0 0x7fd5d0004fe0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006f90 tx=0x7fd5dc01cea0 comp rx=0 tx=0).stop 2024-09-13T08:54:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 -- 172.21.15.44:0/446504993 shutdown_connections 2024-09-13T08:54:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 --2- 172.21.15.44:0/446504993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5d0004bc0 0x7fd5d0004fe0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 -- 172.21.15.44:0/446504993 >> 172.21.15.44:0/446504993 conn(0x7fd5e80b18f0 msgr2=0x7fd5e80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 -- 172.21.15.44:0/446504993 shutdown_connections 2024-09-13T08:54:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 -- 172.21.15.44:0/446504993 wait complete. 2024-09-13T08:54:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 Processor -- start 2024-09-13T08:54:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 -- start start 2024-09-13T08:54:50.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5d0004bc0 0x7fd5e81535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:50.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd5e81f9e50 0x7fd5e815c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:50.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5e81512b0 0x7fd5e81e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:50.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e80ced10 con 0x7fd5d0004bc0 2024-09-13T08:54:50.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e80ceb90 con 0x7fd5e81512b0 2024-09-13T08:54:50.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5f0400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e80cee90 con 0x7fd5e81f9e50 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5eda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5d0004bc0 0x7fd5e81535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5eda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5d0004bc0 0x7fd5e81535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50776/0 (socket says 172.21.15.44:50776) 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5eda00640 1 -- 172.21.15.44:0/2475208192 learned_addr learned my addr 172.21.15.44:0/2475208192 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5ed000640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd5e81f9e50 0x7fd5e815c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5ee400640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5e81512b0 0x7fd5e81e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5eda00640 1 -- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd5e81f9e50 msgr2=0x7fd5e815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5eda00640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd5e81f9e50 0x7fd5e815c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5eda00640 1 -- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5e81512b0 msgr2=0x7fd5e81e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5eda00640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5e81512b0 0x7fd5e81e7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5eda00640 1 -- 172.21.15.44:0/2475208192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e815cba0 con 0x7fd5d0004bc0 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.042+0000 7fd5ee400640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5e81512b0 0x7fd5e81e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:50.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.046+0000 7fd5eda00640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5d0004bc0 0x7fd5e81535f0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006f90 tx=0x7fd5dc001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:50.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.046+0000 7fd5e5600640 1 -- 172.21.15.44:0/2475208192 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0a5b30 con 0x7fd5d0004bc0 2024-09-13T08:54:50.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.046+0000 7fd5e5600640 1 -- 172.21.15.44:0/2475208192 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5dc001500 con 0x7fd5d0004bc0 2024-09-13T08:54:50.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.046+0000 7fd5e5600640 1 -- 172.21.15.44:0/2475208192 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc006330 con 0x7fd5d0004bc0 2024-09-13T08:54:50.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.046+0000 7fd5f0400640 1 -- 172.21.15.44:0/2475208192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e815ce70 con 0x7fd5d0004bc0 2024-09-13T08:54:50.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.046+0000 7fd5f0400640 1 -- 172.21.15.44:0/2475208192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e81f9650 con 0x7fd5d0004bc0 2024-09-13T08:54:50.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.046+0000 7fd5f0400640 1 -- 172.21.15.44:0/2475208192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e8069410 con 0x7fd5d0004bc0 2024-09-13T08:54:50.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.046+0000 7fd5e5600640 1 -- 172.21.15.44:0/2475208192 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fd5dc003600 con 0x7fd5d0004bc0 2024-09-13T08:54:50.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.046+0000 7fd5e5600640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5c8075650 0x7fd5c8077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:50.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.050+0000 7fd5e5600640 1 -- 172.21.15.44:0/2475208192 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fd5dc12c260 con 0x7fd5d0004bc0 2024-09-13T08:54:50.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.050+0000 7fd5ed000640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5c8075650 0x7fd5c8077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:50.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.050+0000 7fd5ed000640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5c8075650 0x7fd5c8077b10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd5e815c5f0 tx=0x7fd5d4006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:50.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.050+0000 7fd5e5600640 1 -- 172.21.15.44:0/2475208192 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fd5dc016c50 con 0x7fd5d0004bc0 2024-09-13T08:54:50.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.234+0000 7fd5f0400640 1 -- 172.21.15.44:0/2475208192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5e815d390 con 0x7fd5d0004bc0 2024-09-13T08:54:50.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.234+0000 7fd5e5600640 1 -- 172.21.15.44:0/2475208192 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+13231 (secure 0 0 0) 0x7fd5e815d390 con 0x7fd5d0004bc0 2024-09-13T08:54:50.234 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:50.234 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":151,"fsid":"d85be7ac-239d-467b-bce8-dbc7fa81939e","created":"2024-09-13T08:37:09.257844+0000","modified":"2024-09-13T08:54:45.299600+0000","last_up_change":"2024-09-13T08:37:23.813189+0000","last_in_change":"2024-09-13T08:37:12.953327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T08:37:24.563344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"552c332f-dfc0-4d0c-a685-8d43312f4cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6801","nonce":1438138297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6803","nonce":1438138297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6807","nonce":1438138297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1438138297},{"type":"v1","addr":"172.21.15.44:6805","nonce":1438138297}]},"public_addr":"172.21.15.44:6801/1438138297","cluster_addr":"172.21.15.44:6803/1438138297","heartbeat_back_addr":"172.21.15.44:6807/1438138297","heartbeat_front_addr":"172.21.15.44:6805/1438138297","state":["exists","up"]},{"osd":1,"uuid":"df0f39fa-83f5-4276-a3ab-ab685aaf8c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6817","nonce":3258852146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6819","nonce":3258852146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6823","nonce":3258852146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3258852146},{"type":"v1","addr":"172.21.15.44:6821","nonce":3258852146}]},"public_addr":"172.21.15.44:6817/3258852146","cluster_addr":"172.21.15.44:6819/3258852146","heartbeat_back_addr":"172.21.15.44:6823/3258852146","heartbeat_front_addr":"172.21.15.44:6821/3258852146","state":["exists","up"]},{"osd":2,"uuid":"bcb344c5-4c69-4f64-8990-956e8d477ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6809","nonce":231873797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6811","nonce":231873797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6815","nonce":231873797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":231873797},{"type":"v1","addr":"172.21.15.44:6813","nonce":231873797}]},"public_addr":"172.21.15.44:6809/231873797","cluster_addr":"172.21.15.44:6811/231873797","heartbeat_back_addr":"172.21.15.44:6815/231873797","heartbeat_front_addr":"172.21.15.44:6813/231873797","state":["exists","up"]},{"osd":3,"uuid":"e8dff9fa-c0a5-40f5-995d-190975f1c6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6825","nonce":3593887774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6827","nonce":3593887774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6831","nonce":3593887774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3593887774},{"type":"v1","addr":"172.21.15.44:6829","nonce":3593887774}]},"public_addr":"172.21.15.44:6825/3593887774","cluster_addr":"172.21.15.44:6827/3593887774","heartbeat_back_addr":"172.21.15.44:6831/3593887774","heartbeat_front_addr":"172.21.15.44:6829/3593887774","state":["exists","up"]},{"osd":4,"uuid":"daf4f548-002d-444f-bdb0-b0c58efe6e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6827","nonce":3123318411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6829","nonce":3123318411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6833","nonce":3123318411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3123318411},{"type":"v1","addr":"172.21.15.129:6831","nonce":3123318411}]},"public_addr":"172.21.15.129:6827/3123318411","cluster_addr":"172.21.15.129:6829/3123318411","heartbeat_back_addr":"172.21.15.129:6833/3123318411","heartbeat_front_addr":"172.21.15.129:6831/3123318411","state":["exists","up"]},{"osd":5,"uuid":"7363ce33-d055-4b66-96ce-a9d3c2e7ad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6813","nonce":3446737198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6817","nonce":3446737198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6824","nonce":3446737198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3446737198},{"type":"v1","addr":"172.21.15.129:6820","nonce":3446737198}]},"public_addr":"172.21.15.129:6813/3446737198","cluster_addr":"172.21.15.129:6817/3446737198","heartbeat_back_addr":"172.21.15.129:6824/3446737198","heartbeat_front_addr":"172.21.15.129:6820/3446737198","state":["exists","up"]},{"osd":6,"uuid":"4af3eb16-8c24-4b52-a409-d34dafb11a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6803","nonce":2280849631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6805","nonce":2280849631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6809","nonce":2280849631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2280849631},{"type":"v1","addr":"172.21.15.129:6807","nonce":2280849631}]},"public_addr":"172.21.15.129:6803/2280849631","cluster_addr":"172.21.15.129:6805/2280849631","heartbeat_back_addr":"172.21.15.129:6809/2280849631","heartbeat_front_addr":"172.21.15.129:6807/2280849631","state":["exists","up"]},{"osd":7,"uuid":"9315c934-f5fb-4bd4-bd3c-cc01accf02d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6812","nonce":1299717984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6816","nonce":1299717984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6823","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6825","nonce":1299717984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6819","nonce":1299717984},{"type":"v1","addr":"172.21.15.129:6821","nonce":1299717984}]},"public_addr":"172.21.15.129:6812/1299717984","cluster_addr":"172.21.15.129:6816/1299717984","heartbeat_back_addr":"172.21.15.129:6825/1299717984","heartbeat_front_addr":"172.21.15.129:6821/1299717984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.099010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.401577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.062762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.175654+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.595347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:20.015547+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.904431+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T08:37:19.984348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/3418956320":"2024-09-14T08:54:42.249339+0000","172.21.15.44:6833/3418956320":"2024-09-14T08:54:42.249339+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T08:54:50.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 -- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5c8075650 msgr2=0x7fd5c8077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:50.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5c8075650 0x7fd5c8077b10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd5e815c5f0 tx=0x7fd5d4006b80 comp rx=0 tx=0).stop 2024-09-13T08:54:50.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 -- 172.21.15.44:0/2475208192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5d0004bc0 msgr2=0x7fd5e81535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:50.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5d0004bc0 0x7fd5e81535f0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006f90 tx=0x7fd5dc001c70 comp rx=0 tx=0).stop 2024-09-13T08:54:50.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 -- 172.21.15.44:0/2475208192 shutdown_connections 2024-09-13T08:54:50.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fd5c8075650 0x7fd5c8077b10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:50.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5e81512b0 0x7fd5e81e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:50.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fd5e81f9e50 0x7fd5e815c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:50.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 --2- 172.21.15.44:0/2475208192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5d0004bc0 0x7fd5e81535f0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:50.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 -- 172.21.15.44:0/2475208192 >> 172.21.15.44:0/2475208192 conn(0x7fd5e80b18f0 msgr2=0x7fd5e806f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:50.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 -- 172.21.15.44:0/2475208192 shutdown_connections 2024-09-13T08:54:50.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:50.238+0000 7fd5f0400640 1 -- 172.21.15.44:0/2475208192 wait complete. 2024-09-13T08:54:51.257 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T08:54:51.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T08:54:51.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 -- 172.21.15.44:0/3298514970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c280b0b60 msgr2=0x7f4c280ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 --2- 172.21.15.44:0/3298514970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c280b0b60 0x7f4c280ccbd0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f4c28073e00 tx=0x7f4c2001f580 comp rx=0 tx=0).stop 2024-09-13T08:54:51.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 -- 172.21.15.44:0/3298514970 shutdown_connections 2024-09-13T08:54:51.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 --2- 172.21.15.44:0/3298514970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c280b0b60 0x7f4c280ccbd0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 --2- 172.21.15.44:0/3298514970 >> v2:172.21.15.129:3300/0 conn(0x7f4c2815be10 0x7f4c280ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 -- 172.21.15.44:0/3298514970 >> 172.21.15.44:0/3298514970 conn(0x7f4c280d1da0 msgr2=0x7f4c280d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:51.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 -- 172.21.15.44:0/3298514970 shutdown_connections 2024-09-13T08:54:51.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 -- 172.21.15.44:0/3298514970 wait complete. 2024-09-13T08:54:51.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 Processor -- start 2024-09-13T08:54:51.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 -- start start 2024-09-13T08:54:51.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c280b0b60 0x7f4c2815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c2815be10 0x7f4c2815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c2815c1f0 0x7f4c280c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4c280cd290 con 0x7f4c280b0b60 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c280cd110 con 0x7f4c2815be10 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.422+0000 7f4c30600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4c280cd410 con 0x7f4c2815c1f0 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c280b0b60 0x7f4c2815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c280b0b60 0x7f4c2815a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50796/0 (socket says 172.21.15.44:50796) 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2dc00640 1 -- 172.21.15.44:0/2873140992 learned_addr learned my addr 172.21.15.44:0/2873140992 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2d200640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c2815be10 0x7f4c2815d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2e600640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c2815c1f0 0x7f4c280c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2dc00640 1 -- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c2815c1f0 msgr2=0x7f4c280c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2dc00640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f4c2815c1f0 0x7f4c280c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2dc00640 1 -- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c2815be10 msgr2=0x7f4c2815d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2dc00640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c2815be10 0x7f4c2815d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2dc00640 1 -- 172.21.15.44:0/2873140992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c280cffb0 con 0x7f4c280b0b60 2024-09-13T08:54:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c2dc00640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c280b0b60 0x7f4c2815a3e0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00e9f0 tx=0x7f4c1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c25600640 1 -- 172.21.15.44:0/2873140992 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c00cd80 con 0x7f4c280b0b60 2024-09-13T08:54:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c25600640 1 -- 172.21.15.44:0/2873140992 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c1c010430 con 0x7f4c280b0b60 2024-09-13T08:54:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c25600640 1 -- 172.21.15.44:0/2873140992 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c010860 con 0x7f4c280b0b60 2024-09-13T08:54:51.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c280d2180 con 0x7f4c280b0b60 2024-09-13T08:54:51.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c28158470 con 0x7f4c280b0b60 2024-09-13T08:54:51.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c280ccbd0 con 0x7f4c280b0b60 2024-09-13T08:54:51.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.426+0000 7f4c25600640 1 -- 172.21.15.44:0/2873140992 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f4c1c010a00 con 0x7f4c280b0b60 2024-09-13T08:54:51.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.430+0000 7f4c25600640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c08075650 0x7f4c08077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.430+0000 7f4c25600640 1 -- 172.21.15.44:0/2873140992 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f4c1c0947b0 con 0x7f4c280b0b60 2024-09-13T08:54:51.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.430+0000 7f4c25600640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] conn(0x7f4c0807b5d0 0x7f4c0807da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.430+0000 7f4c25600640 1 -- 172.21.15.44:0/2873140992 --> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c0807e120 con 0x7f4c0807b5d0 2024-09-13T08:54:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.430+0000 7f4c25600640 1 -- 172.21.15.44:0/2873140992 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f4c1c094f00 con 0x7f4c280b0b60 2024-09-13T08:54:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.430+0000 7f4c2d200640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c08075650 0x7f4c08077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.430+0000 7f4c2e600640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] conn(0x7f4c0807b5d0 0x7f4c0807da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.430+0000 7f4c2e600640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] conn(0x7f4c0807b5d0 0x7f4c0807da50 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.430+0000 7f4c2d200640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c08075650 0x7f4c08077b10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4c280cf2a0 tx=0x7f4c14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.430+0000 7f4c25600640 1 -- 172.21.15.44:0/2873140992 <== osd.0 v2:172.21.15.44:6800/1438138297 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c0807e120 con 0x7f4c0807b5d0 2024-09-13T08:54:51.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.442+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 --> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4c28070350 con 0x7f4c0807b5d0 2024-09-13T08:54:51.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.442+0000 7f4c25600640 1 -- 172.21.15.44:0/2873140992 <== osd.0 v2:172.21.15.44:6800/1438138297 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4c28070350 con 0x7f4c0807b5d0 2024-09-13T08:54:51.443 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:51.443 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T08:54:51.443 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:51.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.442+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 >> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] conn(0x7f4c0807b5d0 msgr2=0x7f4c0807da50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.442+0000 7f4c30600640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.44:6800/1438138297,v1:172.21.15.44:6801/1438138297] conn(0x7f4c0807b5d0 0x7f4c0807da50 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.446+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c08075650 msgr2=0x7f4c08077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.446+0000 7f4c30600640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4c08075650 0x7f4c08077b10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4c280cf2a0 tx=0x7f4c14009040 comp rx=0 tx=0).stop 2024-09-13T08:54:51.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.446+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c280b0b60 msgr2=0x7f4c2815a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.446+0000 7f4c30600640 1 --2- 172.21.15.44:0/2873140992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c280b0b60 0x7f4c2815a3e0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00e9f0 tx=0x7f4c1c00eeb0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.446+0000 7f4c2e600640 1 -- 172.21.15.44:0/2873140992 reap_dead start 2024-09-13T08:54:51.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.446+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 shutdown_connections 2024-09-13T08:54:51.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.446+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 >> 172.21.15.44:0/2873140992 conn(0x7f4c280d1da0 msgr2=0x7f4c28071090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:51.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.446+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 shutdown_connections 2024-09-13T08:54:51.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.446+0000 7f4c30600640 1 -- 172.21.15.44:0/2873140992 wait complete. 2024-09-13T08:54:51.461 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T08:54:51.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.610+0000 7f6da6c00640 1 -- 172.21.15.44:0/1478717868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6da00cdb30 msgr2=0x7f6da00aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.610+0000 7f6da6c00640 1 --2- 172.21.15.44:0/1478717868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6da00cdb30 0x7f6da00aeef0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009bc0 tx=0x7f6d9801cba0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.610+0000 7f6da6c00640 1 -- 172.21.15.44:0/1478717868 shutdown_connections 2024-09-13T08:54:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.610+0000 7f6da6c00640 1 --2- 172.21.15.44:0/1478717868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6da00cdb30 0x7f6da00aeef0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.610+0000 7f6da6c00640 1 --2- 172.21.15.44:0/1478717868 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6da00ced80 0x7f6da00ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.610+0000 7f6da6c00640 1 -- 172.21.15.44:0/1478717868 >> 172.21.15.44:0/1478717868 conn(0x7f6da00d1dc0 msgr2=0x7f6da00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.610+0000 7f6da6c00640 1 -- 172.21.15.44:0/1478717868 shutdown_connections 2024-09-13T08:54:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.610+0000 7f6da6c00640 1 -- 172.21.15.44:0/1478717868 wait complete. 2024-09-13T08:54:51.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 Processor -- start 2024-09-13T08:54:51.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 -- start start 2024-09-13T08:54:51.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6da00cdb30 0x7f6da00c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6da00ced80 0x7f6da014d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6da015bea0 0x7f6da00cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01692a0 con 0x7f6da015bea0 2024-09-13T08:54:51.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6da0169120 con 0x7f6da00cdb30 2024-09-13T08:54:51.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f6da0169420 con 0x7f6da00ced80 2024-09-13T08:54:51.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6da015bea0 0x7f6da00cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6da015bea0 0x7f6da00cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50822/0 (socket says 172.21.15.44:50822) 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da4c00640 1 -- 172.21.15.44:0/2711892665 learned_addr learned my addr 172.21.15.44:0/2711892665 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da4c00640 1 -- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6da00ced80 msgr2=0x7f6da014d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da4c00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6da00ced80 0x7f6da014d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da4c00640 1 -- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6da00cdb30 msgr2=0x7f6da00c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da4c00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6da00cdb30 0x7f6da00c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da4c00640 1 -- 172.21.15.44:0/2711892665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da00c5950 con 0x7f6da015bea0 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da4c00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6da015bea0 0x7f6da00cb060 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f6d9801c610 tx=0x7f6d98005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6d9cc00640 1 -- 172.21.15.44:0/2711892665 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d980b2040 con 0x7f6da015bea0 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6d9cc00640 1 -- 172.21.15.44:0/2711892665 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d980bb070 con 0x7f6da015bea0 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6d9cc00640 1 -- 172.21.15.44:0/2711892665 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98004370 con 0x7f6da015bea0 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 -- 172.21.15.44:0/2711892665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da00c88a0 con 0x7f6da015bea0 2024-09-13T08:54:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 -- 172.21.15.44:0/2711892665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da014d560 con 0x7f6da015bea0 2024-09-13T08:54:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.614+0000 7f6da6c00640 1 -- 172.21.15.44:0/2711892665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da00b3230 con 0x7f6da015bea0 2024-09-13T08:54:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.618+0000 7f6d9cc00640 1 -- 172.21.15.44:0/2711892665 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f6d98004510 con 0x7f6da015bea0 2024-09-13T08:54:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.618+0000 7f6d9cc00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d7c075620 0x7f6d7c077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.618+0000 7f6d9cc00640 1 -- 172.21.15.44:0/2711892665 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f6d9812dae0 con 0x7f6da015bea0 2024-09-13T08:54:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.618+0000 7f6d9fe00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d7c075620 0x7f6d7c077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.618+0000 7f6d9fe00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d7c075620 0x7f6d7c077ae0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6d900046c0 tx=0x7f6d90007910 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:51.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.622+0000 7f6d9cc00640 1 -- 172.21.15.44:0/2711892665 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f6d98016790 con 0x7f6da015bea0 2024-09-13T08:54:51.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 -- 172.21.15.44:0/2711892665 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f6da00ce670 con 0x7f6d7c075620 2024-09-13T08:54:51.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6d9cc00640 1 -- 172.21.15.44:0/2711892665 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6da00ce670 con 0x7f6d7c075620 2024-09-13T08:54:51.797 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T08:54:51.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 -- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d7c075620 msgr2=0x7f6d7c077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d7c075620 0x7f6d7c077ae0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6d900046c0 tx=0x7f6d90007910 comp rx=0 tx=0).stop 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 -- 172.21.15.44:0/2711892665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6da015bea0 msgr2=0x7f6da00cb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6da015bea0 0x7f6da00cb060 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f6d9801c610 tx=0x7f6d98005b00 comp rx=0 tx=0).stop 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 -- 172.21.15.44:0/2711892665 shutdown_connections 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f6d7c075620 0x7f6d7c077ae0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6da015bea0 0x7f6da00cb060 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f6da00ced80 0x7f6da014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 --2- 172.21.15.44:0/2711892665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6da00cdb30 0x7f6da00c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 -- 172.21.15.44:0/2711892665 >> 172.21.15.44:0/2711892665 conn(0x7f6da00d1dc0 msgr2=0x7f6da015aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 -- 172.21.15.44:0/2711892665 shutdown_connections 2024-09-13T08:54:51.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.798+0000 7f6da6c00640 1 -- 172.21.15.44:0/2711892665 wait complete. 2024-09-13T08:54:51.818 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T08:54:51.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T08:54:51.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 -- 172.21.15.44:0/3717125330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90c015c1f0 msgr2=0x7f90c00cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 --2- 172.21.15.44:0/3717125330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90c015c1f0 0x7f90c00cee10 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f90c00b0fb0 tx=0x7f90a801f0b0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 -- 172.21.15.44:0/3717125330 shutdown_connections 2024-09-13T08:54:51.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 --2- 172.21.15.44:0/3717125330 >> v2:172.21.15.129:3300/0 conn(0x7f90c00b0b60 0x7f90c00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 --2- 172.21.15.44:0/3717125330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90c015c1f0 0x7f90c00cee10 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 -- 172.21.15.44:0/3717125330 >> 172.21.15.44:0/3717125330 conn(0x7f90c00d1da0 msgr2=0x7f90c00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:51.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 -- 172.21.15.44:0/3717125330 shutdown_connections 2024-09-13T08:54:51.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 -- 172.21.15.44:0/3717125330 wait complete. 2024-09-13T08:54:51.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 Processor -- start 2024-09-13T08:54:51.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 -- start start 2024-09-13T08:54:51.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90c00b0b60 0x7f90c00c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90c015be10 0x7f90c01f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90c015c1f0 0x7f90c0156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90c00cd4a0 con 0x7f90c00b0b60 2024-09-13T08:54:51.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90c00cd320 con 0x7f90c015c1f0 2024-09-13T08:54:51.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90c7200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f90c00cd620 con 0x7f90c015be10 2024-09-13T08:54:51.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90c00b0b60 0x7f90c00c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90c00b0b60 0x7f90c00c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50836/0 (socket says 172.21.15.44:50836) 2024-09-13T08:54:51.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.970+0000 7f90bfe00640 1 -- 172.21.15.44:0/2250195457 learned_addr learned my addr 172.21.15.44:0/2250195457 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90b7e00640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90c015be10 0x7f90c01f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90c5200640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90c015c1f0 0x7f90c0156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90bfe00640 1 -- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90c015be10 msgr2=0x7f90c01f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90bfe00640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f90c015be10 0x7f90c01f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90bfe00640 1 -- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90c015c1f0 msgr2=0x7f90c0156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90bfe00640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90c015c1f0 0x7f90c0156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90bfe00640 1 -- 172.21.15.44:0/2250195457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c00c9780 con 0x7f90c00b0b60 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90bfe00640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90c00b0b60 0x7f90c00c83e0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c500 tx=0x7f90b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90bcc00640 1 -- 172.21.15.44:0/2250195457 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0017070 con 0x7f90c00b0b60 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90bcc00640 1 -- 172.21.15.44:0/2250195457 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90b00053c0 con 0x7f90c00b0b60 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c00c9a60 con 0x7f90c00b0b60 2024-09-13T08:54:51.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90bcc00640 1 -- 172.21.15.44:0/2250195457 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0012890 con 0x7f90c00b0b60 2024-09-13T08:54:51.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c01f6160 con 0x7f90c00b0b60 2024-09-13T08:54:51.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90c00cdb40 con 0x7f90c00b0b60 2024-09-13T08:54:51.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.974+0000 7f90bcc00640 1 -- 172.21.15.44:0/2250195457 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f90b001a070 con 0x7f90c00b0b60 2024-09-13T08:54:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.978+0000 7f90bcc00640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c075620 0x7f909c077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.978+0000 7f90bcc00640 1 -- 172.21.15.44:0/2250195457 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f90b0092520 con 0x7f90c00b0b60 2024-09-13T08:54:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.978+0000 7f90bcc00640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] conn(0x7f909c07b540 0x7f909c07d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.978+0000 7f90bcc00640 1 -- 172.21.15.44:0/2250195457 --> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90b0011ec0 con 0x7f909c07b540 2024-09-13T08:54:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.978+0000 7f90bcc00640 1 -- 172.21.15.44:0/2250195457 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f90b009e050 con 0x7f90c00b0b60 2024-09-13T08:54:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.978+0000 7f90b7e00640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c075620 0x7f909c077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.978+0000 7f90c5200640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] conn(0x7f909c07b540 0x7f909c07d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:51.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.978+0000 7f90c5200640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] conn(0x7f909c07b540 0x7f909c07d9c0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:51.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.978+0000 7f90b7e00640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c075620 0x7f909c077ae0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f90a80b2000 tx=0x7f90a801d8f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:51.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.978+0000 7f90bcc00640 1 -- 172.21.15.44:0/2250195457 <== osd.1 v2:172.21.15.44:6816/3258852146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90b0011ec0 con 0x7f909c07b540 2024-09-13T08:54:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.990+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 --> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f90c0070350 con 0x7f909c07b540 2024-09-13T08:54:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.990+0000 7f90bcc00640 1 -- 172.21.15.44:0/2250195457 <== osd.1 v2:172.21.15.44:6816/3258852146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f90c0070350 con 0x7f909c07b540 2024-09-13T08:54:51.991 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:51.991 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T08:54:51.991 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.990+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 >> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] conn(0x7f909c07b540 msgr2=0x7f909c07d9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.990+0000 7f90c7200640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.44:6816/3258852146,v1:172.21.15.44:6817/3258852146] conn(0x7f909c07b540 0x7f909c07d9c0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.990+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c075620 msgr2=0x7f909c077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.990+0000 7f90c7200640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f909c075620 0x7f909c077ae0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f90a80b2000 tx=0x7f90a801d8f0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.990+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90c00b0b60 msgr2=0x7f90c00c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.990+0000 7f90c7200640 1 --2- 172.21.15.44:0/2250195457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90c00b0b60 0x7f90c00c83e0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c500 tx=0x7f90b000c9c0 comp rx=0 tx=0).stop 2024-09-13T08:54:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.990+0000 7f90c5200640 1 -- 172.21.15.44:0/2250195457 reap_dead start 2024-09-13T08:54:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.994+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 shutdown_connections 2024-09-13T08:54:51.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.994+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 >> 172.21.15.44:0/2250195457 conn(0x7f90c00d1da0 msgr2=0x7f90c00afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:51.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.994+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 shutdown_connections 2024-09-13T08:54:51.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:51.994+0000 7f90c7200640 1 -- 172.21.15.44:0/2250195457 wait complete. 2024-09-13T08:54:52.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T08:54:52.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 -- 172.21.15.44:0/4158436488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7415db50 msgr2=0x7f2a740acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 --2- 172.21.15.44:0/4158436488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7415db50 0x7f2a740acd40 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f2a74070900 tx=0x7f2a6401db60 comp rx=0 tx=0).stop 2024-09-13T08:54:52.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 -- 172.21.15.44:0/4158436488 shutdown_connections 2024-09-13T08:54:52.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 --2- 172.21.15.44:0/4158436488 >> v2:172.21.15.129:3300/0 conn(0x7f2a740cd100 0x7f2a740b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 --2- 172.21.15.44:0/4158436488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7415db50 0x7f2a740acd40 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 -- 172.21.15.44:0/4158436488 >> 172.21.15.44:0/4158436488 conn(0x7f2a740b18f0 msgr2=0x7f2a740b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:52.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 -- 172.21.15.44:0/4158436488 shutdown_connections 2024-09-13T08:54:52.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 -- 172.21.15.44:0/4158436488 wait complete. 2024-09-13T08:54:52.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 Processor -- start 2024-09-13T08:54:52.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 -- start start 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2a740b4880 0x7f2a7413df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a740cd100 0x7f2a7409c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a7415db50 0x7f2a74140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a740ccc90 con 0x7f2a740cd100 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a740ccb10 con 0x7f2a7415db50 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a7a400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f2a74167120 con 0x7f2a740b4880 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a6aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a740cd100 0x7f2a7409c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a6aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a740cd100 0x7f2a7409c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42818/0 (socket says 172.21.15.44:42818) 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a6aa00640 1 -- 172.21.15.44:0/688056213 learned_addr learned my addr 172.21.15.44:0/688056213 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a73400640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2a740b4880 0x7f2a7413df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.158+0000 7f2a73e00640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a7415db50 0x7f2a74140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6aa00640 1 -- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2a740b4880 msgr2=0x7f2a7413df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6aa00640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2a740b4880 0x7f2a7413df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6aa00640 1 -- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a7415db50 msgr2=0x7f2a74140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6aa00640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a7415db50 0x7f2a74140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6aa00640 1 -- 172.21.15.44:0/688056213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7409cf10 con 0x7f2a740cd100 2024-09-13T08:54:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6aa00640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a740cd100 0x7f2a7409c7a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00d950 tx=0x7f2a5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6be00640 1 -- 172.21.15.44:0/688056213 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c017070 con 0x7f2a740cd100 2024-09-13T08:54:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6be00640 1 -- 172.21.15.44:0/688056213 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a5c009d20 con 0x7f2a740cd100 2024-09-13T08:54:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6be00640 1 -- 172.21.15.44:0/688056213 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0106c0 con 0x7f2a740cd100 2024-09-13T08:54:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a7a400640 1 -- 172.21.15.44:0/688056213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7409d1f0 con 0x7f2a740cd100 2024-09-13T08:54:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a7a400640 1 -- 172.21.15.44:0/688056213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7409d7b0 con 0x7f2a740cd100 2024-09-13T08:54:52.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a7a400640 1 -- 172.21.15.44:0/688056213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a74069410 con 0x7f2a740cd100 2024-09-13T08:54:52.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6be00640 1 -- 172.21.15.44:0/688056213 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f2a5c010860 con 0x7f2a740cd100 2024-09-13T08:54:52.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.162+0000 7f2a6be00640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2a50075620 0x7f2a50077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.166+0000 7f2a6be00640 1 -- 172.21.15.44:0/688056213 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f2a5c0934d0 con 0x7f2a740cd100 2024-09-13T08:54:52.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.166+0000 7f2a73400640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2a50075620 0x7f2a50077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.166+0000 7f2a73400640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2a50075620 0x7f2a50077ae0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2a640b0000 tx=0x7f2a640a8040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:52.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.166+0000 7f2a6be00640 1 -- 172.21.15.44:0/688056213 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f2a5c05fb30 con 0x7f2a740cd100 2024-09-13T08:54:52.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 -- 172.21.15.44:0/688056213 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f2a740b4cd0 con 0x7f2a50075620 2024-09-13T08:54:52.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a6be00640 1 -- 172.21.15.44:0/688056213 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2a740b4cd0 con 0x7f2a50075620 2024-09-13T08:54:52.344 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 -- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2a50075620 msgr2=0x7f2a50077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2a50075620 0x7f2a50077ae0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2a640b0000 tx=0x7f2a640a8040 comp rx=0 tx=0).stop 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 -- 172.21.15.44:0/688056213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a740cd100 msgr2=0x7f2a7409c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a740cd100 0x7f2a7409c7a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00d950 tx=0x7f2a5c00de10 comp rx=0 tx=0).stop 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 -- 172.21.15.44:0/688056213 shutdown_connections 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f2a50075620 0x7f2a50077ae0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a7415db50 0x7f2a74140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a740cd100 0x7f2a7409c7a0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 --2- 172.21.15.44:0/688056213 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f2a740b4880 0x7f2a7413df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 -- 172.21.15.44:0/688056213 >> 172.21.15.44:0/688056213 conn(0x7f2a740b18f0 msgr2=0x7f2a7415b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 -- 172.21.15.44:0/688056213 shutdown_connections 2024-09-13T08:54:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.346+0000 7f2a7a400640 1 -- 172.21.15.44:0/688056213 wait complete. 2024-09-13T08:54:52.366 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T08:54:52.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T08:54:52.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.518+0000 7f414ca00640 1 -- 172.21.15.44:0/2797092589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41480cff00 msgr2=0x7f41480d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.518+0000 7f414ca00640 1 --2- 172.21.15.44:0/2797092589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41480cff00 0x7f41480d28c0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f4138009900 tx=0x7f413801c8b0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.518+0000 7f414ca00640 1 -- 172.21.15.44:0/2797092589 shutdown_connections 2024-09-13T08:54:52.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.518+0000 7f414ca00640 1 --2- 172.21.15.44:0/2797092589 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414815f700 0x7f41480d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.518+0000 7f414ca00640 1 --2- 172.21.15.44:0/2797092589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41480cff00 0x7f41480d28c0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.518+0000 7f414ca00640 1 -- 172.21.15.44:0/2797092589 >> 172.21.15.44:0/2797092589 conn(0x7f41480d1da0 msgr2=0x7f41480d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:52.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.518+0000 7f414ca00640 1 -- 172.21.15.44:0/2797092589 shutdown_connections 2024-09-13T08:54:52.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.518+0000 7f414ca00640 1 -- 172.21.15.44:0/2797092589 wait complete. 2024-09-13T08:54:52.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.518+0000 7f414ca00640 1 Processor -- start 2024-09-13T08:54:52.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.518+0000 7f414ca00640 1 -- start start 2024-09-13T08:54:52.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f414ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41480cff00 0x7f41480c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f414ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41480d02e0 0x7f41480c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f414ca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414815f700 0x7f41480c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f414ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41480b0c50 con 0x7f41480cff00 2024-09-13T08:54:52.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f414ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f41480b0ad0 con 0x7f41480d02e0 2024-09-13T08:54:52.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f414ca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f4148168bf0 con 0x7f414815f700 2024-09-13T08:54:52.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4146800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41480cff00 0x7f41480c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4146800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41480cff00 0x7f41480c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42830/0 (socket says 172.21.15.44:42830) 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4146800640 1 -- 172.21.15.44:0/1259724259 learned_addr learned my addr 172.21.15.44:0/1259724259 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4147200640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414815f700 0x7f41480c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f413fe00640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41480d02e0 0x7f41480c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4146800640 1 -- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414815f700 msgr2=0x7f41480c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4146800640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414815f700 0x7f41480c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4146800640 1 -- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41480d02e0 msgr2=0x7f41480c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4146800640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41480d02e0 0x7f41480c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4146800640 1 -- 172.21.15.44:0/1259724259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41480b7730 con 0x7f41480cff00 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4147200640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f414815f700 0x7f41480c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f4146800640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41480cff00 0x7f41480c9bf0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f4138009850 tx=0x7f41380a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f413ea00640 1 -- 172.21.15.44:0/1259724259 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41380b2040 con 0x7f41480cff00 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f413ea00640 1 -- 172.21.15.44:0/1259724259 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4138003970 con 0x7f41480cff00 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f413ea00640 1 -- 172.21.15.44:0/1259724259 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138003da0 con 0x7f41480cff00 2024-09-13T08:54:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41480b5b00 con 0x7f41480cff00 2024-09-13T08:54:52.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41480cf290 con 0x7f41480cff00 2024-09-13T08:54:52.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.522+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41480d28c0 con 0x7f41480cff00 2024-09-13T08:54:52.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f413ea00640 1 -- 172.21.15.44:0/1259724259 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f41380b8470 con 0x7f41480cff00 2024-09-13T08:54:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f413ea00640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4124075620 0x7f4124077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f413ea00640 1 -- 172.21.15.44:0/1259724259 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f413812d7f0 con 0x7f41480cff00 2024-09-13T08:54:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f413ea00640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] conn(0x7f412407b5a0 0x7f412407da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f413ea00640 1 -- 172.21.15.44:0/1259724259 --> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4138007e80 con 0x7f412407b5a0 2024-09-13T08:54:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f413ea00640 1 -- 172.21.15.44:0/1259724259 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f413812dba0 con 0x7f41480cff00 2024-09-13T08:54:52.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f4147200640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] conn(0x7f412407b5a0 0x7f412407da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f413fe00640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4124075620 0x7f4124077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f4147200640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] conn(0x7f412407b5a0 0x7f412407da20 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:52.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f413fe00640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4124075620 0x7f4124077ae0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f414815bf80 tx=0x7f4130009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:52.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.526+0000 7f413ea00640 1 -- 172.21.15.44:0/1259724259 <== osd.2 v2:172.21.15.44:6808/231873797 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4138007e80 con 0x7f412407b5a0 2024-09-13T08:54:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 --> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f41480d2500 con 0x7f412407b5a0 2024-09-13T08:54:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f413ea00640 1 -- 172.21.15.44:0/1259724259 <== osd.2 v2:172.21.15.44:6808/231873797 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f41480d2500 con 0x7f412407b5a0 2024-09-13T08:54:52.540 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:52.540 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T08:54:52.540 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 >> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] conn(0x7f412407b5a0 msgr2=0x7f412407da20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.44:6808/231873797,v1:172.21.15.44:6809/231873797] conn(0x7f412407b5a0 0x7f412407da20 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4124075620 msgr2=0x7f4124077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f4124075620 0x7f4124077ae0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f414815bf80 tx=0x7f4130009040 comp rx=0 tx=0).stop 2024-09-13T08:54:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41480cff00 msgr2=0x7f41480c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 --2- 172.21.15.44:0/1259724259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41480cff00 0x7f41480c9bf0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f4138009850 tx=0x7f41380a57f0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f4147200640 1 -- 172.21.15.44:0/1259724259 reap_dead start 2024-09-13T08:54:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 shutdown_connections 2024-09-13T08:54:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 >> 172.21.15.44:0/1259724259 conn(0x7f41480d1da0 msgr2=0x7f41480b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 shutdown_connections 2024-09-13T08:54:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.542+0000 7f414ca00640 1 -- 172.21.15.44:0/1259724259 wait complete. 2024-09-13T08:54:52.558 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T08:54:52.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.706+0000 7f8986e00640 1 -- 172.21.15.44:0/371587981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89800cdb30 msgr2=0x7f89800af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.706+0000 7f8986e00640 1 --2- 172.21.15.44:0/371587981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89800cdb30 0x7f89800af0c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8978009bc0 tx=0x7f897801cba0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.706+0000 7f8986e00640 1 -- 172.21.15.44:0/371587981 shutdown_connections 2024-09-13T08:54:52.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.706+0000 7f8986e00640 1 --2- 172.21.15.44:0/371587981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89800cdb30 0x7f89800af0c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.706+0000 7f8986e00640 1 -- 172.21.15.44:0/371587981 >> 172.21.15.44:0/371587981 conn(0x7f89800d1dc0 msgr2=0x7f89800d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:52.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.706+0000 7f8986e00640 1 -- 172.21.15.44:0/371587981 shutdown_connections 2024-09-13T08:54:52.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.706+0000 7f8986e00640 1 -- 172.21.15.44:0/371587981 wait complete. 2024-09-13T08:54:52.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8986e00640 1 Processor -- start 2024-09-13T08:54:52.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8986e00640 1 -- start start 2024-09-13T08:54:52.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8986e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89800cdb30 0x7f89800719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8986e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89800ced80 0x7f8980071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8986e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898015bea0 0x7f898009eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8986e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f898016a840 con 0x7f89800cdb30 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8986e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f898016a6c0 con 0x7f89800ced80 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8986e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f898016a9c0 con 0x7f898015bea0 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89800cdb30 0x7f89800719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89800cdb30 0x7f89800719d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42832/0 (socket says 172.21.15.44:42832) 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897fe00640 1 -- 172.21.15.44:0/2902757009 learned_addr learned my addr 172.21.15.44:0/2902757009 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8977e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89800ced80 0x7f8980071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8984e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898015bea0 0x7f898009eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897fe00640 1 -- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898015bea0 msgr2=0x7f898009eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897fe00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898015bea0 0x7f898009eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897fe00640 1 -- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89800ced80 msgr2=0x7f8980071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897fe00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89800ced80 0x7f8980071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897fe00640 1 -- 172.21.15.44:0/2902757009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898015b8c0 con 0x7f89800cdb30 2024-09-13T08:54:52.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8984e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898015bea0 0x7f898009eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897fe00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89800cdb30 0x7f89800719d0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f897000c450 tx=0x7f897000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897cc00640 1 -- 172.21.15.44:0/2902757009 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89700052d0 con 0x7f89800cdb30 2024-09-13T08:54:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897cc00640 1 -- 172.21.15.44:0/2902757009 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8970005470 con 0x7f89800cdb30 2024-09-13T08:54:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f897cc00640 1 -- 172.21.15.44:0/2902757009 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970012870 con 0x7f89800cdb30 2024-09-13T08:54:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8986e00640 1 -- 172.21.15.44:0/2902757009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898015af70 con 0x7f89800cdb30 2024-09-13T08:54:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.710+0000 7f8986e00640 1 -- 172.21.15.44:0/2902757009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89800b1370 con 0x7f89800cdb30 2024-09-13T08:54:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.714+0000 7f8986e00640 1 -- 172.21.15.44:0/2902757009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89800ce670 con 0x7f89800cdb30 2024-09-13T08:54:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.714+0000 7f897cc00640 1 -- 172.21.15.44:0/2902757009 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f8970003600 con 0x7f89800cdb30 2024-09-13T08:54:52.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.714+0000 7f897cc00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f895c075620 0x7f895c077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:52.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.714+0000 7f897cc00640 1 -- 172.21.15.44:0/2902757009 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f8970092c50 con 0x7f89800cdb30 2024-09-13T08:54:52.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.714+0000 7f8977e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f895c075620 0x7f895c077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:52.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.714+0000 7f8977e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f895c075620 0x7f895c077ae0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f89800b2890 tx=0x7f8968007cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:52.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.718+0000 7f897cc00640 1 -- 172.21.15.44:0/2902757009 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f898015b8c0 con 0x7f89800cdb30 2024-09-13T08:54:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 -- 172.21.15.44:0/2902757009 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f89800762f0 con 0x7f895c075620 2024-09-13T08:54:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f897cc00640 1 -- 172.21.15.44:0/2902757009 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f89800762f0 con 0x7f895c075620 2024-09-13T08:54:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T08:54:52.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 -- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f895c075620 msgr2=0x7f895c077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f895c075620 0x7f895c077ae0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f89800b2890 tx=0x7f8968007cd0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 -- 172.21.15.44:0/2902757009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89800cdb30 msgr2=0x7f89800719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:52.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89800cdb30 0x7f89800719d0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f897000c450 tx=0x7f897000c910 comp rx=0 tx=0).stop 2024-09-13T08:54:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 -- 172.21.15.44:0/2902757009 shutdown_connections 2024-09-13T08:54:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f895c075620 0x7f895c077ae0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f898015bea0 0x7f898009eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89800ced80 0x7f8980071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 --2- 172.21.15.44:0/2902757009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89800cdb30 0x7f89800719d0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 -- 172.21.15.44:0/2902757009 >> 172.21.15.44:0/2902757009 conn(0x7f89800d1dc0 msgr2=0x7f89800b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 -- 172.21.15.44:0/2902757009 shutdown_connections 2024-09-13T08:54:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:52.894+0000 7f8986e00640 1 -- 172.21.15.44:0/2902757009 wait complete. 2024-09-13T08:54:52.914 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T08:54:52.914 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T08:54:53.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 -- 172.21.15.44:0/916748007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f933015be10 msgr2=0x7f93300ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 --2- 172.21.15.44:0/916748007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f933015be10 0x7f93300ce700 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f9330074e00 tx=0x7f932001f0b0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 -- 172.21.15.44:0/916748007 shutdown_connections 2024-09-13T08:54:53.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 --2- 172.21.15.44:0/916748007 >> v2:172.21.15.129:3300/0 conn(0x7f93300b0b60 0x7f93300cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 --2- 172.21.15.44:0/916748007 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f933015c1f0 0x7f93300cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 --2- 172.21.15.44:0/916748007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f933015be10 0x7f93300ce700 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 -- 172.21.15.44:0/916748007 >> 172.21.15.44:0/916748007 conn(0x7f93300d1da0 msgr2=0x7f93300d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:53.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 -- 172.21.15.44:0/916748007 shutdown_connections 2024-09-13T08:54:53.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 -- 172.21.15.44:0/916748007 wait complete. 2024-09-13T08:54:53.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 Processor -- start 2024-09-13T08:54:53.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.066+0000 7f9336400640 1 -- start start 2024-09-13T08:54:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9336400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93300b0b60 0x7f93300c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9336400640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f933015be10 0x7f93301f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9336400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f933015c1f0 0x7f93301f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9336400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f93301600a0 con 0x7f93300b0b60 2024-09-13T08:54:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9336400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f933015ff20 con 0x7f933015c1f0 2024-09-13T08:54:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9336400640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f9330160220 con 0x7f933015be10 2024-09-13T08:54:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f932f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93300b0b60 0x7f93300c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f932f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93300b0b60 0x7f93300c62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42850/0 (socket says 172.21.15.44:42850) 2024-09-13T08:54:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f932f400640 1 -- 172.21.15.44:0/57912270 learned_addr learned my addr 172.21.15.44:0/57912270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:53.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9327e00640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f933015be10 0x7f93301f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f932fe00640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f933015c1f0 0x7f93301f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f932f400640 1 -- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f933015be10 msgr2=0x7f93301f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f932f400640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f933015be10 0x7f93301f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f932f400640 1 -- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f933015c1f0 msgr2=0x7f93301f3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f932f400640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f933015c1f0 0x7f93301f3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f932f400640 1 -- 172.21.15.44:0/57912270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93300c6d10 con 0x7f93300b0b60 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9327e00640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f933015be10 0x7f93301f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f932f400640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93300b0b60 0x7f93300c62e0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f93200b3040 tx=0x7f9320005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9327400640 1 -- 172.21.15.44:0/57912270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93200be070 con 0x7f93300b0b60 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9327400640 1 -- 172.21.15.44:0/57912270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93200053e0 con 0x7f93300b0b60 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9327400640 1 -- 172.21.15.44:0/57912270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93200b5040 con 0x7f93300b0b60 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93300c6fb0 con 0x7f93300b0b60 2024-09-13T08:54:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93300c7570 con 0x7f93300b0b60 2024-09-13T08:54:53.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93301606d0 con 0x7f93300b0b60 2024-09-13T08:54:53.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.070+0000 7f9327400640 1 -- 172.21.15.44:0/57912270 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f93200c1070 con 0x7f93300b0b60 2024-09-13T08:54:53.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.074+0000 7f9327400640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f930c075620 0x7f930c077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.074+0000 7f9327400640 1 -- 172.21.15.44:0/57912270 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f9320130560 con 0x7f93300b0b60 2024-09-13T08:54:53.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.074+0000 7f9327400640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] conn(0x7f930c07b540 0x7f930c07d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.074+0000 7f9327400640 1 -- 172.21.15.44:0/57912270 --> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93200c4e50 con 0x7f930c07b540 2024-09-13T08:54:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.074+0000 7f9327e00640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f930c075620 0x7f930c077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.074+0000 7f9327400640 1 -- 172.21.15.44:0/57912270 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f9320130910 con 0x7f93300b0b60 2024-09-13T08:54:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.074+0000 7f932fe00640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] conn(0x7f930c07b540 0x7f930c07d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.074+0000 7f9327e00640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f930c075620 0x7f930c077ae0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9330158280 tx=0x7f931800b5b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:53.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.074+0000 7f932fe00640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] conn(0x7f930c07b540 0x7f930c07d9c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:53.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.074+0000 7f9327400640 1 -- 172.21.15.44:0/57912270 <== osd.3 v2:172.21.15.44:6824/3593887774 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93200c4e50 con 0x7f930c07b540 2024-09-13T08:54:53.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.086+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 --> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f93300c5220 con 0x7f930c07b540 2024-09-13T08:54:53.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9327400640 1 -- 172.21.15.44:0/57912270 <== osd.3 v2:172.21.15.44:6824/3593887774 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f93300c5220 con 0x7f930c07b540 2024-09-13T08:54:53.087 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:53.087 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T08:54:53.087 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:53.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 >> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] conn(0x7f930c07b540 msgr2=0x7f930c07d9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9336400640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.44:6824/3593887774,v1:172.21.15.44:6825/3593887774] conn(0x7f930c07b540 0x7f930c07d9c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f930c075620 msgr2=0x7f930c077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9336400640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f930c075620 0x7f930c077ae0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9330158280 tx=0x7f931800b5b0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93300b0b60 msgr2=0x7f93300c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9336400640 1 --2- 172.21.15.44:0/57912270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93300b0b60 0x7f93300c62e0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f93200b3040 tx=0x7f9320005760 comp rx=0 tx=0).stop 2024-09-13T08:54:53.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f932fe00640 1 -- 172.21.15.44:0/57912270 reap_dead start 2024-09-13T08:54:53.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 shutdown_connections 2024-09-13T08:54:53.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 >> 172.21.15.44:0/57912270 conn(0x7f93300d1da0 msgr2=0x7f93300b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:53.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 shutdown_connections 2024-09-13T08:54:53.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.090+0000 7f9336400640 1 -- 172.21.15.44:0/57912270 wait complete. 2024-09-13T08:54:53.106 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T08:54:53.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.254+0000 7fe65a600640 1 -- 172.21.15.44:0/3764430248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6380052f0 msgr2=0x7fe63800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.254+0000 7fe65a600640 1 --2- 172.21.15.44:0/3764430248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6380052f0 0x7fe63800b9d0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fe6540b2850 tx=0x7fe64c01caf0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.254+0000 7fe65a600640 1 -- 172.21.15.44:0/3764430248 shutdown_connections 2024-09-13T08:54:53.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.254+0000 7fe65a600640 1 --2- 172.21.15.44:0/3764430248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6380052f0 0x7fe63800b9d0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.254+0000 7fe65a600640 1 -- 172.21.15.44:0/3764430248 >> 172.21.15.44:0/3764430248 conn(0x7fe6540b18f0 msgr2=0x7fe6540b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.254+0000 7fe65a600640 1 -- 172.21.15.44:0/3764430248 shutdown_connections 2024-09-13T08:54:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.254+0000 7fe65a600640 1 -- 172.21.15.44:0/3764430248 wait complete. 2024-09-13T08:54:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe65a600640 1 Processor -- start 2024-09-13T08:54:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe65a600640 1 -- start start 2024-09-13T08:54:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe65a600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe6380052f0 0x7fe654074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe65a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6541f5430 0x7fe6541f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe65a600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6541f5890 0x7fe6541586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe65a600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6540b2bf0 con 0x7fe6541f5430 2024-09-13T08:54:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe65a600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6540cef10 con 0x7fe6541f5890 2024-09-13T08:54:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe65a600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe6540b2d70 con 0x7fe6380052f0 2024-09-13T08:54:53.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6541f5430 0x7fe6541f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6541f5430 0x7fe6541f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42868/0 (socket says 172.21.15.44:42868) 2024-09-13T08:54:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64aa00640 1 -- 172.21.15.44:0/282461923 learned_addr learned my addr 172.21.15.44:0/282461923 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe653400640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe6380052f0 0x7fe654074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe653e00640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6541f5890 0x7fe6541586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64aa00640 1 -- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe6380052f0 msgr2=0x7fe654074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64aa00640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe6380052f0 0x7fe654074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64aa00640 1 -- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6541f5890 msgr2=0x7fe6541586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64aa00640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6541f5890 0x7fe6541586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64aa00640 1 -- 172.21.15.44:0/282461923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6541f9020 con 0x7fe6541f5430 2024-09-13T08:54:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64aa00640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6541f5430 0x7fe6541f8110 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fe63c010fb0 tx=0x7fe63c00e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64be00640 1 -- 172.21.15.44:0/282461923 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c012850 con 0x7fe6541f5430 2024-09-13T08:54:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64be00640 1 -- 172.21.15.44:0/282461923 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe63c012ae0 con 0x7fe6541f5430 2024-09-13T08:54:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe65a600640 1 -- 172.21.15.44:0/282461923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6541f8a60 con 0x7fe6541f5430 2024-09-13T08:54:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.258+0000 7fe64be00640 1 -- 172.21.15.44:0/282461923 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c01a400 con 0x7fe6541f5430 2024-09-13T08:54:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.262+0000 7fe65a600640 1 -- 172.21.15.44:0/282461923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6541f4c30 con 0x7fe6541f5430 2024-09-13T08:54:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.262+0000 7fe65a600640 1 -- 172.21.15.44:0/282461923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe654069410 con 0x7fe6541f5430 2024-09-13T08:54:53.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.262+0000 7fe64be00640 1 -- 172.21.15.44:0/282461923 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fe63c00d030 con 0x7fe6541f5430 2024-09-13T08:54:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.262+0000 7fe64be00640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe630075620 0x7fe630077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.262+0000 7fe64be00640 1 -- 172.21.15.44:0/282461923 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fe63c096210 con 0x7fe6541f5430 2024-09-13T08:54:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.262+0000 7fe653400640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe630075620 0x7fe630077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.266+0000 7fe653400640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe630075620 0x7fe630077ae0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe644009be0 tx=0x7fe6440091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:53.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.266+0000 7fe64be00640 1 -- 172.21.15.44:0/282461923 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe63c016030 con 0x7fe6541f5430 2024-09-13T08:54:53.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.442+0000 7fe65a600640 1 -- 172.21.15.44:0/282461923 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe6540b3260 con 0x7fe630075620 2024-09-13T08:54:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.442+0000 7fe64be00640 1 -- 172.21.15.44:0/282461923 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe6540b3260 con 0x7fe630075620 2024-09-13T08:54:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T08:54:53.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 -- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe630075620 msgr2=0x7fe630077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe630075620 0x7fe630077ae0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe644009be0 tx=0x7fe6440091e0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 -- 172.21.15.44:0/282461923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6541f5430 msgr2=0x7fe6541f8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6541f5430 0x7fe6541f8110 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fe63c010fb0 tx=0x7fe63c00e6b0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 -- 172.21.15.44:0/282461923 shutdown_connections 2024-09-13T08:54:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe630075620 0x7fe630077ae0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6541f5890 0x7fe6541586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6541f5430 0x7fe6541f8110 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 --2- 172.21.15.44:0/282461923 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe6380052f0 0x7fe654074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 -- 172.21.15.44:0/282461923 >> 172.21.15.44:0/282461923 conn(0x7fe6540b18f0 msgr2=0x7fe6540afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 -- 172.21.15.44:0/282461923 shutdown_connections 2024-09-13T08:54:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.446+0000 7fe65a600640 1 -- 172.21.15.44:0/282461923 wait complete. 2024-09-13T08:54:53.466 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T08:54:53.466 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T08:54:53.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.618+0000 7f5f3e800640 1 -- 172.21.15.44:0/1828097265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f380cff00 msgr2=0x7f5f380d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.618+0000 7f5f3e800640 1 --2- 172.21.15.44:0/1828097265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f380cff00 0x7f5f380d28c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5f3806dc00 tx=0x7f5f2c01c7d0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.618+0000 7f5f3e800640 1 -- 172.21.15.44:0/1828097265 shutdown_connections 2024-09-13T08:54:53.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.618+0000 7f5f3e800640 1 --2- 172.21.15.44:0/1828097265 >> v2:172.21.15.129:3300/0 conn(0x7f5f380d02e0 0x7f5f380d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.618+0000 7f5f3e800640 1 --2- 172.21.15.44:0/1828097265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f380cff00 0x7f5f380d28c0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.618+0000 7f5f3e800640 1 -- 172.21.15.44:0/1828097265 >> 172.21.15.44:0/1828097265 conn(0x7f5f380d1da0 msgr2=0x7f5f380d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:53.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.618+0000 7f5f3e800640 1 -- 172.21.15.44:0/1828097265 shutdown_connections 2024-09-13T08:54:53.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.618+0000 7f5f3e800640 1 -- 172.21.15.44:0/1828097265 wait complete. 2024-09-13T08:54:53.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.618+0000 7f5f3e800640 1 Processor -- start 2024-09-13T08:54:53.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.618+0000 7f5f3e800640 1 -- start start 2024-09-13T08:54:53.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f380cff00 0x7f5f380b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3e800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f380d02e0 0x7f5f381fa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3e800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f3815eff0 0x7f5f380ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f380b0b30 con 0x7f5f380cff00 2024-09-13T08:54:53.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3e800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f380b09b0 con 0x7f5f380d02e0 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3e800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f5f380b0cb0 con 0x7f5f3815eff0 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f380cff00 0x7f5f380b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f380cff00 0x7f5f380b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42886/0 (socket says 172.21.15.44:42886) 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3d400640 1 -- 172.21.15.44:0/742405572 learned_addr learned my addr 172.21.15.44:0/742405572 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3ca00640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f380d02e0 0x7f5f381fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3de00640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f3815eff0 0x7f5f380ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3d400640 1 -- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f3815eff0 msgr2=0x7f5f380ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3d400640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f5f3815eff0 0x7f5f380ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3d400640 1 -- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f380d02e0 msgr2=0x7f5f381fa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3d400640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f380d02e0 0x7f5f381fa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3d400640 1 -- 172.21.15.44:0/742405572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f380b7960 con 0x7f5f380cff00 2024-09-13T08:54:53.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3d400640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f380cff00 0x7f5f380b6180 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c000c10 tx=0x7f5f2c0a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:53.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f35600640 1 -- 172.21.15.44:0/742405572 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0bc070 con 0x7f5f380cff00 2024-09-13T08:54:53.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f35600640 1 -- 172.21.15.44:0/742405572 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f2c004430 con 0x7f5f380cff00 2024-09-13T08:54:53.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f35600640 1 -- 172.21.15.44:0/742405572 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0b2040 con 0x7f5f380cff00 2024-09-13T08:54:53.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f380b7c40 con 0x7f5f380cff00 2024-09-13T08:54:53.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f380b5980 con 0x7f5f380cff00 2024-09-13T08:54:53.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.622+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f380d28c0 con 0x7f5f380cff00 2024-09-13T08:54:53.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f35600640 1 -- 172.21.15.44:0/742405572 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f5f2c0bf070 con 0x7f5f380cff00 2024-09-13T08:54:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f35600640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f18075620 0x7f5f18077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f35600640 1 -- 172.21.15.44:0/742405572 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f5f2c12c920 con 0x7f5f380cff00 2024-09-13T08:54:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f35600640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] conn(0x7f5f1807b5a0 0x7f5f1807da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f35600640 1 -- 172.21.15.44:0/742405572 --> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f2c0b2e50 con 0x7f5f1807b5a0 2024-09-13T08:54:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f35600640 1 -- 172.21.15.44:0/742405572 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f5f2c138050 con 0x7f5f380cff00 2024-09-13T08:54:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f3ca00640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f18075620 0x7f5f18077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f3de00640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] conn(0x7f5f1807b5a0 0x7f5f1807da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f3ca00640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f18075620 0x7f5f18077ae0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5f380b6f80 tx=0x7f5f240091e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:53.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f3de00640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] conn(0x7f5f1807b5a0 0x7f5f1807da20 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:53.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.626+0000 7f5f35600640 1 -- 172.21.15.44:0/742405572 <== osd.4 v2:172.21.15.129:6826/3123318411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f2c0b2e50 con 0x7f5f1807b5a0 2024-09-13T08:54:53.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 --> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5f38071dd0 con 0x7f5f1807b5a0 2024-09-13T08:54:53.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f35600640 1 -- 172.21.15.44:0/742405572 <== osd.4 v2:172.21.15.129:6826/3123318411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5f38071dd0 con 0x7f5f1807b5a0 2024-09-13T08:54:53.641 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:53.641 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T08:54:53.641 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:53.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] conn(0x7f5f1807b5a0 msgr2=0x7f5f1807da20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:6826/3123318411,v1:172.21.15.129:6827/3123318411] conn(0x7f5f1807b5a0 0x7f5f1807da20 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f18075620 msgr2=0x7f5f18077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f5f18075620 0x7f5f18077ae0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5f380b6f80 tx=0x7f5f240091e0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f380cff00 msgr2=0x7f5f380b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 --2- 172.21.15.44:0/742405572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f380cff00 0x7f5f380b6180 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c000c10 tx=0x7f5f2c0a57a0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3de00640 1 -- 172.21.15.44:0/742405572 reap_dead start 2024-09-13T08:54:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 shutdown_connections 2024-09-13T08:54:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 >> 172.21.15.44:0/742405572 conn(0x7f5f380d1da0 msgr2=0x7f5f3806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 shutdown_connections 2024-09-13T08:54:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.642+0000 7f5f3e800640 1 -- 172.21.15.44:0/742405572 wait complete. 2024-09-13T08:54:53.659 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T08:54:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.806+0000 7f0a8c800640 1 -- 172.21.15.44:0/578970944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840cf4e0 msgr2=0x7f0a840cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.806+0000 7f0a8c800640 1 --2- 172.21.15.44:0/578970944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840cf4e0 0x7f0a840cd950 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f0a84073a20 tx=0x7f0a8001cc60 comp rx=0 tx=0).stop 2024-09-13T08:54:53.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.806+0000 7f0a8c800640 1 -- 172.21.15.44:0/578970944 shutdown_connections 2024-09-13T08:54:53.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.806+0000 7f0a8c800640 1 --2- 172.21.15.44:0/578970944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840cf4e0 0x7f0a840cd950 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.806+0000 7f0a8c800640 1 --2- 172.21.15.44:0/578970944 >> v2:172.21.15.129:3300/0 conn(0x7f0a8415bd50 0x7f0a840ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.806+0000 7f0a8c800640 1 -- 172.21.15.44:0/578970944 >> 172.21.15.44:0/578970944 conn(0x7f0a840d1ea0 msgr2=0x7f0a840d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:53.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.806+0000 7f0a8c800640 1 -- 172.21.15.44:0/578970944 shutdown_connections 2024-09-13T08:54:53.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.806+0000 7f0a8c800640 1 -- 172.21.15.44:0/578970944 wait complete. 2024-09-13T08:54:53.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8c800640 1 Processor -- start 2024-09-13T08:54:53.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8c800640 1 -- start start 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8c800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840cf4e0 0x7f0a840b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8c800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a8415bd50 0x7f0a84158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8c800640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a8415c130 0x7f0a8406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8c800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84168ea0 con 0x7f0a840cf4e0 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8c800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84168d20 con 0x7f0a8415bd50 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8c800640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f0a84169020 con 0x7f0a8415c130 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a89e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840cf4e0 0x7f0a840b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a89e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840cf4e0 0x7f0a840b7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42912/0 (socket says 172.21.15.44:42912) 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a89e00640 1 -- 172.21.15.44:0/368233092 learned_addr learned my addr 172.21.15.44:0/368233092 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a89400640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a8415bd50 0x7f0a84158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8a800640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a8415c130 0x7f0a8406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a89e00640 1 -- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a8415c130 msgr2=0x7f0a8406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a89e00640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a8415c130 0x7f0a8406ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a89e00640 1 -- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a8415bd50 msgr2=0x7f0a84158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a89e00640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a8415bd50 0x7f0a84158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a89e00640 1 -- 172.21.15.44:0/368233092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8406ddd0 con 0x7f0a840cf4e0 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a89e00640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840cf4e0 0x7f0a840b7670 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800e940 tx=0x7f0a7800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a7e000640 1 -- 172.21.15.44:0/368233092 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7800cd80 con 0x7f0a840cf4e0 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a7e000640 1 -- 172.21.15.44:0/368233092 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a78005020 con 0x7f0a840cf4e0 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8c800640 1 -- 172.21.15.44:0/368233092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8406e040 con 0x7f0a840cf4e0 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a7e000640 1 -- 172.21.15.44:0/368233092 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78005450 con 0x7f0a840cf4e0 2024-09-13T08:54:53.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.810+0000 7f0a8c800640 1 -- 172.21.15.44:0/368233092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8406e600 con 0x7f0a840cf4e0 2024-09-13T08:54:53.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.814+0000 7f0a8c800640 1 -- 172.21.15.44:0/368233092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a840ce5e0 con 0x7f0a840cf4e0 2024-09-13T08:54:53.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.814+0000 7f0a7e000640 1 -- 172.21.15.44:0/368233092 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f0a780047b0 con 0x7f0a840cf4e0 2024-09-13T08:54:53.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.814+0000 7f0a7e000640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a68075620 0x7f0a68077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:53.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.814+0000 7f0a7e000640 1 -- 172.21.15.44:0/368233092 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f0a78093c60 con 0x7f0a840cf4e0 2024-09-13T08:54:53.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.814+0000 7f0a89400640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a68075620 0x7f0a68077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:53.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.818+0000 7f0a89400640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a68075620 0x7f0a68077ae0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0a8406d180 tx=0x7f0a74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:53.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.818+0000 7f0a7e000640 1 -- 172.21.15.44:0/368233092 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f0a780609b0 con 0x7f0a840cf4e0 2024-09-13T08:54:53.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.994+0000 7f0a8c800640 1 -- 172.21.15.44:0/368233092 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f0a84166890 con 0x7f0a68075620 2024-09-13T08:54:53.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a7e000640 1 -- 172.21.15.44:0/368233092 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0a84166890 con 0x7f0a68075620 2024-09-13T08:54:53.996 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T08:54:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 -- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a68075620 msgr2=0x7f0a68077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a68075620 0x7f0a68077ae0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0a8406d180 tx=0x7f0a74009040 comp rx=0 tx=0).stop 2024-09-13T08:54:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 -- 172.21.15.44:0/368233092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840cf4e0 msgr2=0x7f0a840b7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840cf4e0 0x7f0a840b7670 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800e940 tx=0x7f0a7800ee00 comp rx=0 tx=0).stop 2024-09-13T08:54:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 -- 172.21.15.44:0/368233092 shutdown_connections 2024-09-13T08:54:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f0a68075620 0x7f0a68077ae0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f0a8415c130 0x7f0a8406ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a8415bd50 0x7f0a84158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 --2- 172.21.15.44:0/368233092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840cf4e0 0x7f0a840b7670 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:53.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 -- 172.21.15.44:0/368233092 >> 172.21.15.44:0/368233092 conn(0x7f0a840d1ea0 msgr2=0x7f0a840d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:53.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 -- 172.21.15.44:0/368233092 shutdown_connections 2024-09-13T08:54:53.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:53.998+0000 7f0a8c800640 1 -- 172.21.15.44:0/368233092 wait complete. 2024-09-13T08:54:54.018 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T08:54:54.018 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T08:54:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 -- 172.21.15.44:0/552395463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc35015c1f0 msgr2=0x7fc3500cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 --2- 172.21.15.44:0/552395463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc35015c1f0 0x7fc3500cee10 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fc3500b0fb0 tx=0x7fc33801f580 comp rx=0 tx=0).stop 2024-09-13T08:54:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 -- 172.21.15.44:0/552395463 shutdown_connections 2024-09-13T08:54:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 --2- 172.21.15.44:0/552395463 >> v2:172.21.15.129:3300/0 conn(0x7fc3500b0b60 0x7fc3500ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 --2- 172.21.15.44:0/552395463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc35015c1f0 0x7fc3500cee10 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 -- 172.21.15.44:0/552395463 >> 172.21.15.44:0/552395463 conn(0x7fc3500d1da0 msgr2=0x7fc3500d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 -- 172.21.15.44:0/552395463 shutdown_connections 2024-09-13T08:54:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 -- 172.21.15.44:0/552395463 wait complete. 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 Processor -- start 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 -- start start 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3500b0b60 0x7fc3500c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc35015be10 0x7fc3501569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc35015c1f0 0x7fc3500b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3500cd4a0 con 0x7fc35015c1f0 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3500cd320 con 0x7fc35015be10 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.170+0000 7fc355200640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc3500cd620 con 0x7fc3500b0b60 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc34e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc35015c1f0 0x7fc3500b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc34e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc35015c1f0 0x7fc3500b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42944/0 (socket says 172.21.15.44:42944) 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc34e800640 1 -- 172.21.15.44:0/3242597884 learned_addr learned my addr 172.21.15.44:0/3242597884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc345400640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc35015be10 0x7fc3501569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc34de00640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3500b0b60 0x7fc3500c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc34e800640 1 -- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3500b0b60 msgr2=0x7fc3500c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc34e800640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc3500b0b60 0x7fc3500c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc34e800640 1 -- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc35015be10 msgr2=0x7fc3501569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc34e800640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc35015be10 0x7fc3501569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc34e800640 1 -- 172.21.15.44:0/3242597884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3500b0990 con 0x7fc35015c1f0 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc34e800640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc35015c1f0 0x7fc3500b6840 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fc34800d750 tx=0x7fc34800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc346a00640 1 -- 172.21.15.44:0/3242597884 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348017070 con 0x7fc35015c1f0 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc346a00640 1 -- 172.21.15.44:0/3242597884 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc348009d20 con 0x7fc35015c1f0 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc346a00640 1 -- 172.21.15.44:0/3242597884 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34800de60 con 0x7fc35015c1f0 2024-09-13T08:54:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3500cbce0 con 0x7fc35015c1f0 2024-09-13T08:54:54.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3500cba00 con 0x7fc35015c1f0 2024-09-13T08:54:54.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc35006fd80 con 0x7fc35015c1f0 2024-09-13T08:54:54.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.174+0000 7fc346a00640 1 -- 172.21.15.44:0/3242597884 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fc348005280 con 0x7fc35015c1f0 2024-09-13T08:54:54.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.178+0000 7fc346a00640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc32c075620 0x7fc32c077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.178+0000 7fc346a00640 1 -- 172.21.15.44:0/3242597884 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fc348093810 con 0x7fc35015c1f0 2024-09-13T08:54:54.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.178+0000 7fc346a00640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] conn(0x7fc32c07b5a0 0x7fc32c07da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.178+0000 7fc346a00640 1 -- 172.21.15.44:0/3242597884 --> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc32c07e0f0 con 0x7fc32c07b5a0 2024-09-13T08:54:54.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.178+0000 7fc346a00640 1 -- 172.21.15.44:0/3242597884 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7fc348093bc0 con 0x7fc35015c1f0 2024-09-13T08:54:54.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.178+0000 7fc34de00640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc32c075620 0x7fc32c077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.178+0000 7fc345400640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] conn(0x7fc32c07b5a0 0x7fc32c07da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.178+0000 7fc34de00640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc32c075620 0x7fc32c077ae0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc340004e30 tx=0x7fc340009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:54.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.178+0000 7fc345400640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] conn(0x7fc32c07b5a0 0x7fc32c07da20 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:54.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.178+0000 7fc346a00640 1 -- 172.21.15.44:0/3242597884 <== osd.5 v2:172.21.15.129:6811/3446737198 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc32c07e0f0 con 0x7fc32c07b5a0 2024-09-13T08:54:54.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.190+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 --> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc3500cdb40 con 0x7fc32c07b5a0 2024-09-13T08:54:54.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc346a00640 1 -- 172.21.15.44:0/3242597884 <== osd.5 v2:172.21.15.129:6811/3446737198 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc3500cdb40 con 0x7fc32c07b5a0 2024-09-13T08:54:54.192 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:54.192 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T08:54:54.192 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:54.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] conn(0x7fc32c07b5a0 msgr2=0x7fc32c07da20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc355200640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:6811/3446737198,v1:172.21.15.129:6813/3446737198] conn(0x7fc32c07b5a0 0x7fc32c07da20 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc32c075620 msgr2=0x7fc32c077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc355200640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc32c075620 0x7fc32c077ae0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc340004e30 tx=0x7fc340009040 comp rx=0 tx=0).stop 2024-09-13T08:54:54.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc35015c1f0 msgr2=0x7fc3500b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc355200640 1 --2- 172.21.15.44:0/3242597884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc35015c1f0 0x7fc3500b6840 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fc34800d750 tx=0x7fc34800dc10 comp rx=0 tx=0).stop 2024-09-13T08:54:54.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc34e800640 1 -- 172.21.15.44:0/3242597884 reap_dead start 2024-09-13T08:54:54.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 shutdown_connections 2024-09-13T08:54:54.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 >> 172.21.15.44:0/3242597884 conn(0x7fc3500d1da0 msgr2=0x7fc350070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:54.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 shutdown_connections 2024-09-13T08:54:54.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.194+0000 7fc355200640 1 -- 172.21.15.44:0/3242597884 wait complete. 2024-09-13T08:54:54.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T08:54:54.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.358+0000 7f8a62e00640 1 -- 172.21.15.44:0/1926827023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a400051d0 msgr2=0x7f8a4000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.358+0000 7f8a62e00640 1 --2- 172.21.15.44:0/1926827023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a400051d0 0x7f8a4000b9d0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400c640 tx=0x7f8a5401f9c0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.358+0000 7f8a60e00640 1 -- 172.21.15.44:0/1926827023 reap_dead start 2024-09-13T08:54:54.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.358+0000 7f8a62e00640 1 -- 172.21.15.44:0/1926827023 shutdown_connections 2024-09-13T08:54:54.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.358+0000 7f8a62e00640 1 -- 172.21.15.44:0/1926827023 >> 172.21.15.44:0/1926827023 conn(0x7f8a5c0b18f0 msgr2=0x7f8a5c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:54.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.358+0000 7f8a62e00640 1 -- 172.21.15.44:0/1926827023 shutdown_connections 2024-09-13T08:54:54.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.358+0000 7f8a62e00640 1 -- 172.21.15.44:0/1926827023 wait complete. 2024-09-13T08:54:54.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 Processor -- start 2024-09-13T08:54:54.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 -- start start 2024-09-13T08:54:54.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a400051d0 0x7f8a5c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c158a90 0x7f8a5c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8a5c158f10 0x7f8a5c09d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c06eab0 con 0x7f8a5c158a90 2024-09-13T08:54:54.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c0cef60 con 0x7f8a400051d0 2024-09-13T08:54:54.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5c06ec30 con 0x7f8a5c158f10 2024-09-13T08:54:54.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a53400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c158a90 0x7f8a5c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a53400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c158a90 0x7f8a5c0cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42958/0 (socket says 172.21.15.44:42958) 2024-09-13T08:54:54.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a53400640 1 -- 172.21.15.44:0/512764222 learned_addr learned my addr 172.21.15.44:0/512764222 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:54.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a60e00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8a5c158f10 0x7f8a5c09d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a5be00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a400051d0 0x7f8a5c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a53400640 1 -- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8a5c158f10 msgr2=0x7f8a5c09d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a53400640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8a5c158f10 0x7f8a5c09d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a53400640 1 -- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a400051d0 msgr2=0x7f8a5c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a53400640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a400051d0 0x7f8a5c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a53400640 1 -- 172.21.15.44:0/512764222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c1f7f60 con 0x7f8a5c158a90 2024-09-13T08:54:54.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a53400640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c158a90 0x7f8a5c0cb7c0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f8a440117d0 tx=0x7f8a44011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:54.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a58c00640 1 -- 172.21.15.44:0/512764222 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400fd50 con 0x7f8a5c158a90 2024-09-13T08:54:54.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a58c00640 1 -- 172.21.15.44:0/512764222 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a44004030 con 0x7f8a5c158a90 2024-09-13T08:54:54.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a58c00640 1 -- 172.21.15.44:0/512764222 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44004460 con 0x7f8a5c158a90 2024-09-13T08:54:54.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 -- 172.21.15.44:0/512764222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c1f8240 con 0x7f8a5c158a90 2024-09-13T08:54:54.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 -- 172.21.15.44:0/512764222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c1f8800 con 0x7f8a5c158a90 2024-09-13T08:54:54.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.362+0000 7f8a62e00640 1 -- 172.21.15.44:0/512764222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5c069410 con 0x7f8a5c158a90 2024-09-13T08:54:54.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.366+0000 7f8a58c00640 1 -- 172.21.15.44:0/512764222 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f8a44003600 con 0x7f8a5c158a90 2024-09-13T08:54:54.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.366+0000 7f8a58c00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a38075650 0x7f8a38077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.366+0000 7f8a58c00640 1 -- 172.21.15.44:0/512764222 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f8a44093df0 con 0x7f8a5c158a90 2024-09-13T08:54:54.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.366+0000 7f8a5be00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a38075650 0x7f8a38077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.366+0000 7f8a5be00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a38075650 0x7f8a38077b10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c000c10 tx=0x7f8a4c006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:54.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.370+0000 7f8a58c00640 1 -- 172.21.15.44:0/512764222 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f8a44060aa0 con 0x7f8a5c158a90 2024-09-13T08:54:54.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.542+0000 7f8a62e00640 1 -- 172.21.15.44:0/512764222 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f8a5c06f490 con 0x7f8a38075650 2024-09-13T08:54:54.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a58c00640 1 -- 172.21.15.44:0/512764222 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8a5c06f490 con 0x7f8a38075650 2024-09-13T08:54:54.544 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T08:54:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 -- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a38075650 msgr2=0x7f8a38077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a38075650 0x7f8a38077b10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c000c10 tx=0x7f8a4c006b80 comp rx=0 tx=0).stop 2024-09-13T08:54:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 -- 172.21.15.44:0/512764222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c158a90 msgr2=0x7f8a5c0cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c158a90 0x7f8a5c0cb7c0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f8a440117d0 tx=0x7f8a44011c90 comp rx=0 tx=0).stop 2024-09-13T08:54:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 -- 172.21.15.44:0/512764222 shutdown_connections 2024-09-13T08:54:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8a38075650 0x7f8a38077b10 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8a5c158f10 0x7f8a5c09d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c158a90 0x7f8a5c0cb7c0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 --2- 172.21.15.44:0/512764222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a400051d0 0x7f8a5c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 -- 172.21.15.44:0/512764222 >> 172.21.15.44:0/512764222 conn(0x7f8a5c0b18f0 msgr2=0x7f8a5c0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 -- 172.21.15.44:0/512764222 shutdown_connections 2024-09-13T08:54:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.546+0000 7f8a62e00640 1 -- 172.21.15.44:0/512764222 wait complete. 2024-09-13T08:54:54.565 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T08:54:54.566 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T08:54:54.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.718+0000 7fec27a00640 1 -- 172.21.15.44:0/941637732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec08004160 msgr2=0x7fec08004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.718+0000 7fec27a00640 1 --2- 172.21.15.44:0/941637732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec08004160 0x7fec08004810 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009b30 tx=0x7fec0c01d190 comp rx=0 tx=0).stop 2024-09-13T08:54:54.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.718+0000 7fec25a00640 1 -- 172.21.15.44:0/941637732 reap_dead start 2024-09-13T08:54:54.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.718+0000 7fec27a00640 1 -- 172.21.15.44:0/941637732 shutdown_connections 2024-09-13T08:54:54.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.718+0000 7fec27a00640 1 -- 172.21.15.44:0/941637732 >> 172.21.15.44:0/941637732 conn(0x7fec2006dd30 msgr2=0x7fec2006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:54.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.718+0000 7fec27a00640 1 -- 172.21.15.44:0/941637732 shutdown_connections 2024-09-13T08:54:54.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.718+0000 7fec27a00640 1 -- 172.21.15.44:0/941637732 wait complete. 2024-09-13T08:54:54.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.718+0000 7fec27a00640 1 Processor -- start 2024-09-13T08:54:54.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.718+0000 7fec27a00640 1 -- start start 2024-09-13T08:54:54.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec27a00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec08004160 0x7fec20142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec27a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20156830 0x7fec20142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec27a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec20143fa0 0x7fec200b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec27a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fec200d0fd0 con 0x7fec20156830 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec27a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec200d0e50 con 0x7fec20143fa0 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec27a00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fec2015e690 con 0x7fec08004160 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20156830 0x7fec20142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20156830 0x7fec20142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42974/0 (socket says 172.21.15.44:42974) 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1fe00640 1 -- 172.21.15.44:0/789521736 learned_addr learned my addr 172.21.15.44:0/789521736 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec25000640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec08004160 0x7fec20142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec25a00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec20143fa0 0x7fec200b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1fe00640 1 -- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec08004160 msgr2=0x7fec20142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1fe00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fec08004160 0x7fec20142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1fe00640 1 -- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec20143fa0 msgr2=0x7fec200b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1fe00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec20143fa0 0x7fec200b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1fe00640 1 -- 172.21.15.44:0/789521736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec20142c60 con 0x7fec20156830 2024-09-13T08:54:54.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1fe00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20156830 0x7fec20142720 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fec0c01cc00 tx=0x7fec0c0acd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:54.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1cc00640 1 -- 172.21.15.44:0/789521736 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c0b1040 con 0x7fec20156830 2024-09-13T08:54:54.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1cc00640 1 -- 172.21.15.44:0/789521736 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec0c0ac400 con 0x7fec20156830 2024-09-13T08:54:54.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec20142f00 con 0x7fec20156830 2024-09-13T08:54:54.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec1cc00640 1 -- 172.21.15.44:0/789521736 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c009730 con 0x7fec20156830 2024-09-13T08:54:54.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec201434c0 con 0x7fec20156830 2024-09-13T08:54:54.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.722+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec200b48f0 con 0x7fec20156830 2024-09-13T08:54:54.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec1cc00640 1 -- 172.21.15.44:0/789521736 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fec0c0c0020 con 0x7fec20156830 2024-09-13T08:54:54.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec1cc00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec00075620 0x7fec00077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec1cc00640 1 -- 172.21.15.44:0/789521736 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fec0c12eb20 con 0x7fec20156830 2024-09-13T08:54:54.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec1cc00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] conn(0x7fec0007b540 0x7fec0007d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec25000640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec00075620 0x7fec00077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec1cc00640 1 -- 172.21.15.44:0/789521736 --> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec0007e090 con 0x7fec0007b540 2024-09-13T08:54:54.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec1cc00640 1 -- 172.21.15.44:0/789521736 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7fec0c12eed0 con 0x7fec20156830 2024-09-13T08:54:54.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec25a00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] conn(0x7fec0007b540 0x7fec0007d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec25000640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec00075620 0x7fec00077ae0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fec140066c0 tx=0x7fec14006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:54.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec25a00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] conn(0x7fec0007b540 0x7fec0007d9c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:54.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.726+0000 7fec1cc00640 1 -- 172.21.15.44:0/789521736 <== osd.6 v2:172.21.15.129:6802/2280849631 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec0007e090 con 0x7fec0007b540 2024-09-13T08:54:54.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 --> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fec2006acc0 con 0x7fec0007b540 2024-09-13T08:54:54.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec1cc00640 1 -- 172.21.15.44:0/789521736 <== osd.6 v2:172.21.15.129:6802/2280849631 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fec2006acc0 con 0x7fec0007b540 2024-09-13T08:54:54.741 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:54.741 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T08:54:54.741 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:54.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] conn(0x7fec0007b540 msgr2=0x7fec0007d9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:6802/2280849631,v1:172.21.15.129:6803/2280849631] conn(0x7fec0007b540 0x7fec0007d9c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec00075620 msgr2=0x7fec00077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fec00075620 0x7fec00077ae0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fec140066c0 tx=0x7fec14006b80 comp rx=0 tx=0).stop 2024-09-13T08:54:54.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20156830 msgr2=0x7fec20142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 --2- 172.21.15.44:0/789521736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20156830 0x7fec20142720 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fec0c01cc00 tx=0x7fec0c0acd20 comp rx=0 tx=0).stop 2024-09-13T08:54:54.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec25a00640 1 -- 172.21.15.44:0/789521736 reap_dead start 2024-09-13T08:54:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 shutdown_connections 2024-09-13T08:54:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 >> 172.21.15.44:0/789521736 conn(0x7fec2006dd30 msgr2=0x7fec2006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 shutdown_connections 2024-09-13T08:54:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.742+0000 7fec27a00640 1 -- 172.21.15.44:0/789521736 wait complete. 2024-09-13T08:54:54.761 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T08:54:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.910+0000 7f1f86e00640 1 -- 172.21.15.44:0/1113685141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f64003a60 msgr2=0x7f1f64003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.910+0000 7f1f86e00640 1 --2- 172.21.15.44:0/1113685141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f64003a60 0x7f1f64003e80 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009b30 tx=0x7f1f7001e390 comp rx=0 tx=0).stop 2024-09-13T08:54:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.910+0000 7f1f86e00640 1 -- 172.21.15.44:0/1113685141 shutdown_connections 2024-09-13T08:54:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.910+0000 7f1f86e00640 1 --2- 172.21.15.44:0/1113685141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f64003a60 0x7f1f64003e80 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.910+0000 7f1f86e00640 1 -- 172.21.15.44:0/1113685141 >> 172.21.15.44:0/1113685141 conn(0x7f1f800b18f0 msgr2=0x7f1f800b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.910+0000 7f1f86e00640 1 -- 172.21.15.44:0/1113685141 shutdown_connections 2024-09-13T08:54:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.910+0000 7f1f86e00640 1 -- 172.21.15.44:0/1113685141 wait complete. 2024-09-13T08:54:54.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 Processor -- start 2024-09-13T08:54:54.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 -- start start 2024-09-13T08:54:54.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f64003a60 0x7f1f801f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f801e8110 0x7f1f80156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8006f1a0 0x7f1f80153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800cedc0 con 0x7f1f8006f1a0 2024-09-13T08:54:54.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800cec40 con 0x7f1f801e8110 2024-09-13T08:54:54.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f1f800cef40 con 0x7f1f64003a60 2024-09-13T08:54:54.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f84e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8006f1a0 0x7f1f80153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f84e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8006f1a0 0x7f1f80153390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43000/0 (socket says 172.21.15.44:43000) 2024-09-13T08:54:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f84e00640 1 -- 172.21.15.44:0/2599854298 learned_addr learned my addr 172.21.15.44:0/2599854298 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f77e00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f801e8110 0x7f1f80156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f7fe00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f64003a60 0x7f1f801f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f84e00640 1 -- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f64003a60 msgr2=0x7f1f801f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f84e00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f64003a60 0x7f1f801f5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f84e00640 1 -- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f801e8110 msgr2=0x7f1f80156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f84e00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f801e8110 0x7f1f80156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f84e00640 1 -- 172.21.15.44:0/2599854298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f801f63a0 con 0x7f1f8006f1a0 2024-09-13T08:54:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f84e00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8006f1a0 0x7f1f80153390 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f1f800b2860 tx=0x7f1f78006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f7cc00640 1 -- 172.21.15.44:0/2599854298 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800fc90 con 0x7f1f8006f1a0 2024-09-13T08:54:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f7cc00640 1 -- 172.21.15.44:0/2599854298 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f78018070 con 0x7f1f8006f1a0 2024-09-13T08:54:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 -- 172.21.15.44:0/2599854298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f801f6680 con 0x7f1f8006f1a0 2024-09-13T08:54:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f7cc00640 1 -- 172.21.15.44:0/2599854298 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78011690 con 0x7f1f8006f1a0 2024-09-13T08:54:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 -- 172.21.15.44:0/2599854298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f801f6c40 con 0x7f1f8006f1a0 2024-09-13T08:54:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.914+0000 7f1f86e00640 1 -- 172.21.15.44:0/2599854298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f800cf4a0 con 0x7f1f8006f1a0 2024-09-13T08:54:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.918+0000 7f1f7cc00640 1 -- 172.21.15.44:0/2599854298 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f1f78011830 con 0x7f1f8006f1a0 2024-09-13T08:54:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.918+0000 7f1f7cc00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f5c075650 0x7f1f5c077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.918+0000 7f1f7cc00640 1 -- 172.21.15.44:0/2599854298 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f1f78094300 con 0x7f1f8006f1a0 2024-09-13T08:54:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.918+0000 7f1f7fe00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f5c075650 0x7f1f5c077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:54.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.918+0000 7f1f7fe00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f5c075650 0x7f1f5c077b10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1f700af000 tx=0x7f1f7001b130 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:54.922+0000 7f1f7cc00640 1 -- 172.21.15.44:0/2599854298 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f1f780609e0 con 0x7f1f8006f1a0 2024-09-13T08:54:55.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.094+0000 7f1f86e00640 1 -- 172.21.15.44:0/2599854298 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f1f800744c0 con 0x7f1f5c075650 2024-09-13T08:54:55.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.094+0000 7f1f7cc00640 1 -- 172.21.15.44:0/2599854298 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1f800744c0 con 0x7f1f5c075650 2024-09-13T08:54:55.095 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T08:54:55.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 -- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f5c075650 msgr2=0x7f1f5c077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f5c075650 0x7f1f5c077b10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1f700af000 tx=0x7f1f7001b130 comp rx=0 tx=0).stop 2024-09-13T08:54:55.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 -- 172.21.15.44:0/2599854298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8006f1a0 msgr2=0x7f1f80153390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8006f1a0 0x7f1f80153390 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f1f800b2860 tx=0x7f1f78006720 comp rx=0 tx=0).stop 2024-09-13T08:54:55.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 -- 172.21.15.44:0/2599854298 shutdown_connections 2024-09-13T08:54:55.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f1f5c075650 0x7f1f5c077b10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8006f1a0 0x7f1f80153390 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f801e8110 0x7f1f80156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 --2- 172.21.15.44:0/2599854298 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f1f64003a60 0x7f1f801f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 -- 172.21.15.44:0/2599854298 >> 172.21.15.44:0/2599854298 conn(0x7f1f800b18f0 msgr2=0x7f1f800adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:55.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 -- 172.21.15.44:0/2599854298 shutdown_connections 2024-09-13T08:54:55.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.098+0000 7f1f86e00640 1 -- 172.21.15.44:0/2599854298 wait complete. 2024-09-13T08:54:55.116 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T08:54:55.116 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T08:54:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 -- 172.21.15.44:0/1185954951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f7015be10 msgr2=0x7f8f700ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 --2- 172.21.15.44:0/1185954951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f7015be10 0x7f8f700ce700 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009840 tx=0x7f8f6001ef30 comp rx=0 tx=0).stop 2024-09-13T08:54:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 -- 172.21.15.44:0/1185954951 shutdown_connections 2024-09-13T08:54:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 --2- 172.21.15.44:0/1185954951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f7015be10 0x7f8f700ce700 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 -- 172.21.15.44:0/1185954951 >> 172.21.15.44:0/1185954951 conn(0x7f8f700d1da0 msgr2=0x7f8f700d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 -- 172.21.15.44:0/1185954951 shutdown_connections 2024-09-13T08:54:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 -- 172.21.15.44:0/1185954951 wait complete. 2024-09-13T08:54:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 Processor -- start 2024-09-13T08:54:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 -- start start 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f700b0b60 0x7f8f700b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f7015be10 0x7f8f701f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8f7015c1f0 0x7f8f700c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8f700cd670 con 0x7f8f700b0b60 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f700cd4f0 con 0x7f8f7015be10 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f76600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f8f700cd7f0 con 0x7f8f7015c1f0 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f6f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f700b0b60 0x7f8f700b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f6f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f700b0b60 0x7f8f700b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43010/0 (socket says 172.21.15.44:43010) 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f6f400640 1 -- 172.21.15.44:0/3371361612 learned_addr learned my addr 172.21.15.44:0/3371361612 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f6fe00640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8f7015c1f0 0x7f8f700c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.270+0000 7f8f67e00640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f7015be10 0x7f8f701f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f6f400640 1 -- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8f7015c1f0 msgr2=0x7f8f700c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f6f400640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f8f7015c1f0 0x7f8f700c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f6f400640 1 -- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f7015be10 msgr2=0x7f8f701f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f6f400640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f7015be10 0x7f8f701f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f6f400640 1 -- 172.21.15.44:0/3371361612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f700cb070 con 0x7f8f700b0b60 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f6f400640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f700b0b60 0x7f8f700b51e0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f8f6001cb60 tx=0x7f8f60003600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f67400640 1 -- 172.21.15.44:0/3371361612 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f600bf070 con 0x7f8f700b0b60 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f67400640 1 -- 172.21.15.44:0/3371361612 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f60003bc0 con 0x7f8f700b0b60 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f700cb350 con 0x7f8f700b0b60 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f67400640 1 -- 172.21.15.44:0/3371361612 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f600b5040 con 0x7f8f700b0b60 2024-09-13T08:54:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f700cad40 con 0x7f8f700b0b60 2024-09-13T08:54:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f700cba10 con 0x7f8f700b0b60 2024-09-13T08:54:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f67400640 1 -- 172.21.15.44:0/3371361612 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f8f600c2070 con 0x7f8f700b0b60 2024-09-13T08:54:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f67400640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8f4c075620 0x7f8f4c077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.274+0000 7f8f67400640 1 -- 172.21.15.44:0/3371361612 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f8f60130740 con 0x7f8f700b0b60 2024-09-13T08:54:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.278+0000 7f8f67400640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] conn(0x7f8f4c07b5a0 0x7f8f4c07da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.278+0000 7f8f67400640 1 -- 172.21.15.44:0/3371361612 --> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f600b3e70 con 0x7f8f4c07b5a0 2024-09-13T08:54:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.278+0000 7f8f67400640 1 -- 172.21.15.44:0/3371361612 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f8f60130af0 con 0x7f8f700b0b60 2024-09-13T08:54:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.278+0000 7f8f67e00640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8f4c075620 0x7f8f4c077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.278+0000 7f8f6fe00640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] conn(0x7f8f4c07b5a0 0x7f8f4c07da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.278+0000 7f8f67e00640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8f4c075620 0x7f8f4c077ae0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8f58000fc0 tx=0x7f8f58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:55.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.278+0000 7f8f6fe00640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] conn(0x7f8f4c07b5a0 0x7f8f4c07da20 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:55.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.278+0000 7f8f67400640 1 -- 172.21.15.44:0/3371361612 <== osd.7 v2:172.21.15.129:6810/1299717984 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f600b3e70 con 0x7f8f4c07b5a0 2024-09-13T08:54:55.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.290+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 --> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8f700cdca0 con 0x7f8f4c07b5a0 2024-09-13T08:54:55.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.290+0000 7f8f67400640 1 -- 172.21.15.44:0/3371361612 <== osd.7 v2:172.21.15.129:6810/1299717984 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8f700cdca0 con 0x7f8f4c07b5a0 2024-09-13T08:54:55.292 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T08:54:55.292 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T08:54:55.292 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T08:54:55.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] conn(0x7f8f4c07b5a0 msgr2=0x7f8f4c07da20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f76600640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:6810/1299717984,v1:172.21.15.129:6812/1299717984] conn(0x7f8f4c07b5a0 0x7f8f4c07da20 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8f4c075620 msgr2=0x7f8f4c077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f76600640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f8f4c075620 0x7f8f4c077ae0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8f58000fc0 tx=0x7f8f58009040 comp rx=0 tx=0).stop 2024-09-13T08:54:55.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f700b0b60 msgr2=0x7f8f700b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f76600640 1 --2- 172.21.15.44:0/3371361612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f700b0b60 0x7f8f700b51e0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f8f6001cb60 tx=0x7f8f60003600 comp rx=0 tx=0).stop 2024-09-13T08:54:55.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f6fe00640 1 -- 172.21.15.44:0/3371361612 reap_dead start 2024-09-13T08:54:55.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 shutdown_connections 2024-09-13T08:54:55.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 >> 172.21.15.44:0/3371361612 conn(0x7f8f700d1da0 msgr2=0x7f8f700cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:55.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 shutdown_connections 2024-09-13T08:54:55.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.294+0000 7f8f76600640 1 -- 172.21.15.44:0/3371361612 wait complete. 2024-09-13T08:54:55.309 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T08:54:55.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.458+0000 7fe732600640 1 -- 172.21.15.44:0/1561973655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c15bfa0 msgr2=0x7fe72c0cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.458+0000 7fe732600640 1 --2- 172.21.15.44:0/1561973655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c15bfa0 0x7fe72c0cd000 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0708b0 tx=0x7fe71801eff0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.458+0000 7fe732600640 1 -- 172.21.15.44:0/1561973655 shutdown_connections 2024-09-13T08:54:55.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.458+0000 7fe732600640 1 --2- 172.21.15.44:0/1561973655 >> v2:172.21.15.129:3300/0 conn(0x7fe72c0cd540 0x7fe72c06e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.458+0000 7fe732600640 1 --2- 172.21.15.44:0/1561973655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c15bfa0 0x7fe72c0cd000 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.458+0000 7fe732600640 1 -- 172.21.15.44:0/1561973655 >> 172.21.15.44:0/1561973655 conn(0x7fe72c0b18f0 msgr2=0x7fe72c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:55.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.458+0000 7fe732600640 1 -- 172.21.15.44:0/1561973655 shutdown_connections 2024-09-13T08:54:55.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.458+0000 7fe732600640 1 -- 172.21.15.44:0/1561973655 wait complete. 2024-09-13T08:54:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.458+0000 7fe732600640 1 Processor -- start 2024-09-13T08:54:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.458+0000 7fe732600640 1 -- start start 2024-09-13T08:54:55.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe732600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe72c0cd540 0x7fe72c0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe732600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c15bfa0 0x7fe72c1f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe732600640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe72c15c3e0 0x7fe72c1f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe732600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c06f070 con 0x7fe72c15bfa0 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe732600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c06eef0 con 0x7fe72c0cd540 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe732600640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe72c06f1f0 con 0x7fe72c15c3e0 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe72aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c15bfa0 0x7fe72c1f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe72aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c15bfa0 0x7fe72c1f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43030/0 (socket says 172.21.15.44:43030) 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe72aa00640 1 -- 172.21.15.44:0/1244708818 learned_addr learned my addr 172.21.15.44:0/1244708818 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe72b400640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe72c0cd540 0x7fe72c0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe72aa00640 1 -- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe72c15c3e0 msgr2=0x7fe72c1f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe72aa00640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe72c15c3e0 0x7fe72c1f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe72aa00640 1 -- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe72c0cd540 msgr2=0x7fe72c0adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe72aa00640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe72c0cd540 0x7fe72c0adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe72aa00640 1 -- 172.21.15.44:0/1244708818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c0cc9f0 con 0x7fe72c15bfa0 2024-09-13T08:54:55.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe72aa00640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c15bfa0 0x7fe72c1f5340 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe718000c10 tx=0x7fe718004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:55.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe723400640 1 -- 172.21.15.44:0/1244708818 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7180bf070 con 0x7fe72c15bfa0 2024-09-13T08:54:55.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe723400640 1 -- 172.21.15.44:0/1244708818 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7180b08e0 con 0x7fe72c15bfa0 2024-09-13T08:54:55.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe723400640 1 -- 172.21.15.44:0/1244708818 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7180b5040 con 0x7fe72c15bfa0 2024-09-13T08:54:55.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe732600640 1 -- 172.21.15.44:0/1244708818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72c1f48c0 con 0x7fe72c15bfa0 2024-09-13T08:54:55.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe732600640 1 -- 172.21.15.44:0/1244708818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72c0ae500 con 0x7fe72c15bfa0 2024-09-13T08:54:55.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.462+0000 7fe732600640 1 -- 172.21.15.44:0/1244708818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72c069410 con 0x7fe72c15bfa0 2024-09-13T08:54:55.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.466+0000 7fe723400640 1 -- 172.21.15.44:0/1244708818 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fe7180c2070 con 0x7fe72c15bfa0 2024-09-13T08:54:55.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.466+0000 7fe723400640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c075620 0x7fe70c077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.466+0000 7fe723400640 1 -- 172.21.15.44:0/1244708818 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fe71812fdd0 con 0x7fe72c15bfa0 2024-09-13T08:54:55.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.466+0000 7fe72b400640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c075620 0x7fe70c077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.466+0000 7fe72b400640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c075620 0x7fe70c077ae0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0adb30 tx=0x7fe71c0013a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:55.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.470+0000 7fe723400640 1 -- 172.21.15.44:0/1244708818 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe72c1f48c0 con 0x7fe72c15bfa0 2024-09-13T08:54:55.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.642+0000 7fe732600640 1 -- 172.21.15.44:0/1244708818 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fe72c1f8e20 con 0x7fe70c075620 2024-09-13T08:54:55.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe723400640 1 -- 172.21.15.44:0/1244708818 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe71c001530 con 0x7fe70c075620 2024-09-13T08:54:55.644 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T08:54:55.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 -- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c075620 msgr2=0x7fe70c077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c075620 0x7fe70c077ae0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0adb30 tx=0x7fe71c0013a0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 -- 172.21.15.44:0/1244708818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c15bfa0 msgr2=0x7fe72c1f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c15bfa0 0x7fe72c1f5340 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe718000c10 tx=0x7fe718004620 comp rx=0 tx=0).stop 2024-09-13T08:54:55.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 -- 172.21.15.44:0/1244708818 shutdown_connections 2024-09-13T08:54:55.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe70c075620 0x7fe70c077ae0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe72c15c3e0 0x7fe72c1f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c15bfa0 0x7fe72c1f5340 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 --2- 172.21.15.44:0/1244708818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe72c0cd540 0x7fe72c0adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 -- 172.21.15.44:0/1244708818 >> 172.21.15.44:0/1244708818 conn(0x7fe72c0b18f0 msgr2=0x7fe72c0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:55.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 -- 172.21.15.44:0/1244708818 shutdown_connections 2024-09-13T08:54:55.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.646+0000 7fe732600640 1 -- 172.21.15.44:0/1244708818 wait complete. 2024-09-13T08:54:55.666 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T08:54:55.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 -- 172.21.15.44:0/1997471578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8880acd40 msgr2=0x7fe8880b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 --2- 172.21.15.44:0/1997471578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8880acd40 0x7fe8880b42d0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fe880009bc0 tx=0x7fe88001ca80 comp rx=0 tx=0).stop 2024-09-13T08:54:55.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 -- 172.21.15.44:0/1997471578 shutdown_connections 2024-09-13T08:54:55.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 --2- 172.21.15.44:0/1997471578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8880acd40 0x7fe8880b42d0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 --2- 172.21.15.44:0/1997471578 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe88806f8b0 0x7fe88815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 -- 172.21.15.44:0/1997471578 >> 172.21.15.44:0/1997471578 conn(0x7fe8880b18f0 msgr2=0x7fe8880b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:55.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 -- 172.21.15.44:0/1997471578 shutdown_connections 2024-09-13T08:54:55.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 -- 172.21.15.44:0/1997471578 wait complete. 2024-09-13T08:54:55.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 Processor -- start 2024-09-13T08:54:55.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 -- start start 2024-09-13T08:54:55.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88806f8b0 0x7fe888156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.814+0000 7fe88f000640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8880acd40 0x7fe8881fba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe88f000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8880cd100 0x7fe88814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe88f000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8881682b0 con 0x7fe88806f8b0 2024-09-13T08:54:55.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe88f000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe888168130 con 0x7fe8880cd100 2024-09-13T08:54:55.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe88f000640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fe888168430 con 0x7fe8880acd40 2024-09-13T08:54:55.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe887e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88806f8b0 0x7fe888156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe887e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88806f8b0 0x7fe888156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43048/0 (socket says 172.21.15.44:43048) 2024-09-13T08:54:55.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe887e00640 1 -- 172.21.15.44:0/4253894943 learned_addr learned my addr 172.21.15.44:0/4253894943 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:55.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe87fe00640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8880acd40 0x7fe8881fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe88d000640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8880cd100 0x7fe88814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe887e00640 1 -- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8880acd40 msgr2=0x7fe8881fba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe887e00640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8880acd40 0x7fe8881fba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe887e00640 1 -- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8880cd100 msgr2=0x7fe88814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe887e00640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8880cd100 0x7fe88814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe887e00640 1 -- 172.21.15.44:0/4253894943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8881654b0 con 0x7fe88806f8b0 2024-09-13T08:54:55.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe87fe00640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8880acd40 0x7fe8881fba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T08:54:55.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe887e00640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88806f8b0 0x7fe888156f10 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe87800e9f0 tx=0x7fe87800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:55.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe884c00640 1 -- 172.21.15.44:0/4253894943 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87800cd80 con 0x7fe88806f8b0 2024-09-13T08:54:55.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe884c00640 1 -- 172.21.15.44:0/4253894943 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe878010430 con 0x7fe88806f8b0 2024-09-13T08:54:55.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe884c00640 1 -- 172.21.15.44:0/4253894943 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878010860 con 0x7fe88806f8b0 2024-09-13T08:54:55.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe88f000640 1 -- 172.21.15.44:0/4253894943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe888164c10 con 0x7fe88806f8b0 2024-09-13T08:54:55.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe88f000640 1 -- 172.21.15.44:0/4253894943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8881651d0 con 0x7fe88806f8b0 2024-09-13T08:54:55.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe88f000640 1 -- 172.21.15.44:0/4253894943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe888069410 con 0x7fe88806f8b0 2024-09-13T08:54:55.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.818+0000 7fe884c00640 1 -- 172.21.15.44:0/4253894943 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fe878010a00 con 0x7fe88806f8b0 2024-09-13T08:54:55.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.822+0000 7fe884c00640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe864075620 0x7fe864077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:55.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.822+0000 7fe884c00640 1 -- 172.21.15.44:0/4253894943 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fe878093c60 con 0x7fe88806f8b0 2024-09-13T08:54:55.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.822+0000 7fe87fe00640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe864075620 0x7fe864077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:55.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.822+0000 7fe87fe00640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe864075620 0x7fe864077ae0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe888164640 tx=0x7fe870009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:55.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.826+0000 7fe884c00640 1 -- 172.21.15.44:0/4253894943 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fe878060380 con 0x7fe88806f8b0 2024-09-13T08:54:55.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.994+0000 7fe88f000640 1 -- 172.21.15.44:0/4253894943 --> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe88815c7e0 con 0x7fe864075620 2024-09-13T08:54:55.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe884c00640 1 -- 172.21.15.44:0/4253894943 <== mgr.4103 v2:172.21.15.129:6800/3066969387 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13426 (secure 0 0 0) 0x7fe88815c7e0 con 0x7fe864075620 2024-09-13T08:54:55.996 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T08:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 -- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe864075620 msgr2=0x7fe864077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe864075620 0x7fe864077ae0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe888164640 tx=0x7fe870009040 comp rx=0 tx=0).stop 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 -- 172.21.15.44:0/4253894943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88806f8b0 msgr2=0x7fe888156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88806f8b0 0x7fe888156f10 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe87800e9f0 tx=0x7fe87800eeb0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 -- 172.21.15.44:0/4253894943 shutdown_connections 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fe864075620 0x7fe864077ae0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8880cd100 0x7fe88814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fe8880acd40 0x7fe8881fba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 --2- 172.21.15.44:0/4253894943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88806f8b0 0x7fe888156f10 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 -- 172.21.15.44:0/4253894943 >> 172.21.15.44:0/4253894943 conn(0x7fe8880b18f0 msgr2=0x7fe8880b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 -- 172.21.15.44:0/4253894943 shutdown_connections 2024-09-13T08:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:55.998+0000 7fe88f000640 1 -- 172.21.15.44:0/4253894943 wait complete. 2024-09-13T08:54:56.018 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":670,"stamp":"2024-09-13T08:54:54.958264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1153448,"kb_used_data":15912,"kb_used_omap":12,"kb_used_meta":1137331,"kb_avail":753821272,"statfs":{"total":773094113280,"available":771912982528,"internally_reserved":0,"allocated":16293888,"data_stored":7881080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1164627505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3559,"num_read_kb":-14327869,"num_write":-8552,"num_write_kb":-17334316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-95,"num_bytes_recovered":-394264576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002177"},"pg_stats":[{"pgid":"1.0","version":"151'74","reported_seq":370,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-13T08:54:52.699581+0000","last_change":"2024-09-13T08:54:52.699581+0000","last_active":"2024-09-13T08:54:52.699581+0000","last_peered":"2024-09-13T08:54:52.699581+0000","last_clean":"2024-09-13T08:54:52.699581+0000","last_became_active":"2024-09-13T08:37:25.827613+0000","last_became_peered":"2024-09-13T08:37:25.827613+0000","last_unstale":"2024-09-13T08:54:52.699581+0000","last_undegraded":"2024-09-13T08:54:52.699581+0000","last_fullsized":"2024-09-13T08:54:52.699581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'74","last_scrub_stamp":"2024-09-13T08:54:52.699499+0000","last_deep_scrub":"151'74","last_deep_scrub_stamp":"2024-09-13T08:54:52.699499+0000","last_clean_scrub_stamp":"2024-09-13T08:54:52.699499+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:48:57.924978+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":119052,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252788,"statfs":{"total":96636764160,"available":96514854912,"internally_reserved":0,"allocated":1888256,"data_stored":837543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509652,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145992,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225848,"statfs":{"total":96636764160,"available":96487268352,"internally_reserved":0,"allocated":2482176,"data_stored":1427911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163476,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208364,"statfs":{"total":96636764160,"available":96469364736,"internally_reserved":0,"allocated":1888256,"data_stored":837543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141260,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230580,"statfs":{"total":96636764160,"available":96492113920,"internally_reserved":0,"allocated":1888256,"data_stored":837543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141268,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230572,"statfs":{"total":96636764160,"available":96492105728,"internally_reserved":0,"allocated":1888256,"data_stored":837543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509654,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137672,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234168,"statfs":{"total":96636764160,"available":96495788032,"internally_reserved":0,"allocated":2482176,"data_stored":1427911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141260,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230580,"statfs":{"total":96636764160,"available":96492113920,"internally_reserved":0,"allocated":1888256,"data_stored":837543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163468,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208372,"statfs":{"total":96636764160,"available":96469372928,"internally_reserved":0,"allocated":1888256,"data_stored":837543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165476806},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T08:54:56.022 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T08:54:56.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 -- 172.21.15.44:0/2315393320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86c8160490 msgr2=0x7f86c806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:56.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 --2- 172.21.15.44:0/2315393320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86c8160490 0x7f86c806e7a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009bc0 tx=0x7f86c001ca90 comp rx=0 tx=0).stop 2024-09-13T08:54:56.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 -- 172.21.15.44:0/2315393320 shutdown_connections 2024-09-13T08:54:56.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 --2- 172.21.15.44:0/2315393320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86c8160490 0x7f86c806e7a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 -- 172.21.15.44:0/2315393320 >> 172.21.15.44:0/2315393320 conn(0x7f86c80b18f0 msgr2=0x7f86c80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:56.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 -- 172.21.15.44:0/2315393320 shutdown_connections 2024-09-13T08:54:56.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 -- 172.21.15.44:0/2315393320 wait complete. 2024-09-13T08:54:56.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 Processor -- start 2024-09-13T08:54:56.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 -- start start 2024-09-13T08:54:56.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86c80cf1d0 0x7f86c8142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:56.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86c815f9a0 0x7f86c80add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:56.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c8160490 0x7f86c80ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:56.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86c816a7c0 con 0x7f86c8160490 2024-09-13T08:54:56.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86c8159dd0 con 0x7f86c80cf1d0 2024-09-13T08:54:56.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86cca00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7f86c816a940 con 0x7f86c815f9a0 2024-09-13T08:54:56.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86c6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c8160490 0x7f86c80ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86c6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c8160490 0x7f86c80ae2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43060/0 (socket says 172.21.15.44:43060) 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86c5e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86c80cf1d0 0x7f86c8142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86c6800640 1 -- 172.21.15.44:0/4057928138 learned_addr learned my addr 172.21.15.44:0/4057928138 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86bfe00640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86c815f9a0 0x7f86c80add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86c6800640 1 -- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86c815f9a0 msgr2=0x7f86c80add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86c6800640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86c815f9a0 0x7f86c80add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86c6800640 1 -- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86c80cf1d0 msgr2=0x7f86c8142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86c6800640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86c80cf1d0 0x7f86c8142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.170+0000 7f86c6800640 1 -- 172.21.15.44:0/4057928138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c8155680 con 0x7f86c8160490 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.174+0000 7f86c6800640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c8160490 0x7f86c80ae2d0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009b10 tx=0x7f86c00092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.174+0000 7f86be000640 1 -- 172.21.15.44:0/4057928138 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c00b2040 con 0x7f86c8160490 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.174+0000 7f86be000640 1 -- 172.21.15.44:0/4057928138 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86c0003cb0 con 0x7f86c8160490 2024-09-13T08:54:56.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.174+0000 7f86be000640 1 -- 172.21.15.44:0/4057928138 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c00042a0 con 0x7f86c8160490 2024-09-13T08:54:56.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.174+0000 7f86cca00640 1 -- 172.21.15.44:0/4057928138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c81553a0 con 0x7f86c8160490 2024-09-13T08:54:56.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.174+0000 7f86cca00640 1 -- 172.21.15.44:0/4057928138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c81436a0 con 0x7f86c8160490 2024-09-13T08:54:56.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.174+0000 7f86cca00640 1 -- 172.21.15.44:0/4057928138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c806e7a0 con 0x7f86c8160490 2024-09-13T08:54:56.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.174+0000 7f86be000640 1 -- 172.21.15.44:0/4057928138 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7f86c00bd030 con 0x7f86c8160490 2024-09-13T08:54:56.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.178+0000 7f86be000640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86a4075650 0x7f86a4077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:56.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.178+0000 7f86be000640 1 -- 172.21.15.44:0/4057928138 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f86c012cca0 con 0x7f86c8160490 2024-09-13T08:54:56.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.178+0000 7f86c5e00640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86a4075650 0x7f86a4077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:56.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.178+0000 7f86c5e00640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86a4075650 0x7f86a4077b10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f86c8155350 tx=0x7f86b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:56.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.182+0000 7f86be000640 1 -- 172.21.15.44:0/4057928138 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7f86c00c4060 con 0x7f86c8160490 2024-09-13T08:54:56.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.358+0000 7f86cca00640 1 -- 172.21.15.44:0/4057928138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f86c80cc970 con 0x7f86c8160490 2024-09-13T08:54:56.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.362+0000 7f86be000640 1 -- 172.21.15.44:0/4057928138 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v58)=0 v58) ==== 123+0+0 (secure 0 0 0) 0x7f86c00166b0 con 0x7f86c8160490 2024-09-13T08:54:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.362+0000 7f86be000640 1 -- 172.21.15.44:0/4057928138 <== mon.0 v2:172.21.15.44:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f86c00f8e10 con 0x7f86c8160490 2024-09-13T08:54:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.362+0000 7f86cca00640 1 -- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86a4075650 msgr2=0x7f86a4077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.362+0000 7f86cca00640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86a4075650 0x7f86a4077b10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f86c8155350 tx=0x7f86b8009040 comp rx=0 tx=0).stop 2024-09-13T08:54:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.362+0000 7f86cca00640 1 -- 172.21.15.44:0/4057928138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c8160490 msgr2=0x7f86c80ae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.362+0000 7f86cca00640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c8160490 0x7f86c80ae2d0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009b10 tx=0x7f86c00092f0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.366+0000 7f86cca00640 1 -- 172.21.15.44:0/4057928138 shutdown_connections 2024-09-13T08:54:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.366+0000 7f86cca00640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7f86a4075650 0x7f86a4077b10 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.366+0000 7f86cca00640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c8160490 0x7f86c80ae2d0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.366+0000 7f86cca00640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f86c815f9a0 0x7f86c80add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.366+0000 7f86cca00640 1 --2- 172.21.15.44:0/4057928138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86c80cf1d0 0x7f86c8142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.366+0000 7f86cca00640 1 -- 172.21.15.44:0/4057928138 >> 172.21.15.44:0/4057928138 conn(0x7f86c80b18f0 msgr2=0x7f86c815fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.366+0000 7f86cca00640 1 -- 172.21.15.44:0/4057928138 shutdown_connections 2024-09-13T08:54:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.366+0000 7f86cca00640 1 -- 172.21.15.44:0/4057928138 wait complete. 2024-09-13T08:54:56.382 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T08:54:56.383 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T08:54:56.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.530+0000 7fc5cdc00640 1 -- 172.21.15.44:0/3492585385 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc5c815bd50 msgr2=0x7fc5c80ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:56.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.530+0000 7fc5cdc00640 1 --2- 172.21.15.44:0/3492585385 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc5c815bd50 0x7fc5c80ce5e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009900 tx=0x7fc5b801c7c0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.530+0000 7fc5cdc00640 1 -- 172.21.15.44:0/3492585385 shutdown_connections 2024-09-13T08:54:56.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.530+0000 7fc5cdc00640 1 --2- 172.21.15.44:0/3492585385 >> v2:172.21.15.129:3300/0 conn(0x7fc5c80cf4e0 0x7fc5c80cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.530+0000 7fc5cdc00640 1 --2- 172.21.15.44:0/3492585385 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc5c815bd50 0x7fc5c80ce5e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.530+0000 7fc5cdc00640 1 -- 172.21.15.44:0/3492585385 >> 172.21.15.44:0/3492585385 conn(0x7fc5c80d1ea0 msgr2=0x7fc5c80d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:56.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.530+0000 7fc5cdc00640 1 -- 172.21.15.44:0/3492585385 shutdown_connections 2024-09-13T08:54:56.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.530+0000 7fc5cdc00640 1 -- 172.21.15.44:0/3492585385 wait complete. 2024-09-13T08:54:56.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.530+0000 7fc5cdc00640 1 Processor -- start 2024-09-13T08:54:56.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.530+0000 7fc5cdc00640 1 -- start start 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5cdc00640 1 --2- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc5c80cf4e0 0x7fc5c80cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5cdc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5c815bd50 0x7fc5c80709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5cdc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5c815c130 0x7fc5c8070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5cdc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8168f00 con 0x7fc5c815bd50 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5cdc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8168d80 con 0x7fc5c815c130 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5cdc00640 1 -- --> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c8169080 con 0x7fc5c80cf4e0 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5c815bd50 0x7fc5c80709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5c815bd50 0x7fc5c80709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43076/0 (socket says 172.21.15.44:43076) 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bfe00640 1 -- 172.21.15.44:0/2431226008 learned_addr learned my addr 172.21.15.44:0/2431226008 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5c6a00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc5c80cf4e0 0x7fc5c80cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:56.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5c7400640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5c815c130 0x7fc5c8070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:56.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bfe00640 1 -- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc5c80cf4e0 msgr2=0x7fc5c80cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:56.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bfe00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc5c80cf4e0 0x7fc5c80cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bfe00640 1 -- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5c815c130 msgr2=0x7fc5c8070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:56.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bfe00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5c815c130 0x7fc5c8070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bfe00640 1 -- 172.21.15.44:0/2431226008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c81452c0 con 0x7fc5c815bd50 2024-09-13T08:54:56.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bfe00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5c815bd50 0x7fc5c80709c0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c3c0 tx=0x7fc5b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:56.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bea00640 1 -- 172.21.15.44:0/2431226008 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0017070 con 0x7fc5c815bd50 2024-09-13T08:54:56.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bea00640 1 -- 172.21.15.44:0/2431226008 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fc5b00052d0 con 0x7fc5c815bd50 2024-09-13T08:54:56.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5bea00640 1 -- 172.21.15.44:0/2431226008 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0003860 con 0x7fc5c815bd50 2024-09-13T08:54:56.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5cdc00640 1 -- 172.21.15.44:0/2431226008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c809c110 con 0x7fc5c815bd50 2024-09-13T08:54:56.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5cdc00640 1 -- 172.21.15.44:0/2431226008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c8145b20 con 0x7fc5c815bd50 2024-09-13T08:54:56.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.534+0000 7fc5cdc00640 1 -- 172.21.15.44:0/2431226008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c8145d20 con 0x7fc5c815bd50 2024-09-13T08:54:56.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.538+0000 7fc5bea00640 1 -- 172.21.15.44:0/2431226008 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 62) ==== 100650+0+0 (secure 0 0 0) 0x7fc5b001a070 con 0x7fc5c815bd50 2024-09-13T08:54:56.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.538+0000 7fc5bea00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5a4075620 0x7fc5a4077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T08:54:56.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.538+0000 7fc5bea00640 1 -- 172.21.15.44:0/2431226008 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fc5b0092490 con 0x7fc5c815bd50 2024-09-13T08:54:56.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.538+0000 7fc5c6a00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5a4075620 0x7fc5a4077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T08:54:56.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.538+0000 7fc5c6a00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5a4075620 0x7fc5a4077ae0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80af000 tx=0x7fc5b80a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T08:54:56.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.542+0000 7fc5bea00640 1 -- 172.21.15.44:0/2431226008 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188482 (secure 0 0 0) 0x7fc5c8145d20 con 0x7fc5c815bd50 2024-09-13T08:54:56.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 -- 172.21.15.44:0/2431226008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5c80ce5e0 con 0x7fc5c815bd50 2024-09-13T08:54:56.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5bea00640 1 -- 172.21.15.44:0/2431226008 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+3914 (secure 0 0 0) 0x7fc5b005ea70 con 0x7fc5c815bd50 2024-09-13T08:54:56.753 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T08:54:56.753 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":98,"btime":"2024-09-13T08:54:43:251819+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/766502733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":766502733},{"type":"v1","addr":"172.21.15.129:6837","nonce":766502733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2074722286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2074722286},{"type":"v1","addr":"172.21.15.129:6835","nonce":2074722286}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3065267295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3065267295},{"type":"v1","addr":"172.21.15.44:6834","nonce":3065267295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":9780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1524630606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1524630606},{"type":"v1","addr":"172.21.15.44:6835","nonce":1524630606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[]} 2024-09-13T08:54:56.753 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 98 2024-09-13T08:54:56.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 -- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5a4075620 msgr2=0x7fc5a4077ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:56.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5a4075620 0x7fc5a4077ae0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80af000 tx=0x7fc5b80a7040 comp rx=0 tx=0).stop 2024-09-13T08:54:56.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 -- 172.21.15.44:0/2431226008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5c815bd50 msgr2=0x7fc5c80709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T08:54:56.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5c815bd50 0x7fc5c80709c0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c3c0 tx=0x7fc5b000c880 comp rx=0 tx=0).stop 2024-09-13T08:54:56.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 -- 172.21.15.44:0/2431226008 shutdown_connections 2024-09-13T08:54:56.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:6800/3066969387,v1:172.21.15.129:6801/3066969387] conn(0x7fc5a4075620 0x7fc5a4077ae0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5c815c130 0x7fc5c8070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5c815bd50 0x7fc5c80709c0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 --2- 172.21.15.44:0/2431226008 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7fc5c80cf4e0 0x7fc5c80cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T08:54:56.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 -- 172.21.15.44:0/2431226008 >> 172.21.15.44:0/2431226008 conn(0x7fc5c80d1ea0 msgr2=0x7fc5c815b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T08:54:56.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 -- 172.21.15.44:0/2431226008 shutdown_connections 2024-09-13T08:54:56.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T08:54:56.754+0000 7fc5cdc00640 1 -- 172.21.15.44:0/2431226008 wait complete. 2024-09-13T08:54:56.776 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-13T08:54:56.776 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T08:54:56.776 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T08:54:56.776 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:54:56.789 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T08:54:56.789 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T08:54:56.789 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:54:56.843 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T08:54:56.843 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T08:54:56.843 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:54:56.859 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T08:54:56.859 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T08:54:56.859 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:54:56.921 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T08:54:56.921 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T08:54:56.921 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T08:54:56.921 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:54:57.010 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T08:54:57.010 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T08:54:57.010 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:54:58.732 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-13T08:55:03.111 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T08:55:03.112 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T08:55:03.112 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:03.201 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T08:55:03.202 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T08:55:03.202 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:03.441 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:55:03.444 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:55:04.640 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-13T08:55:04.640 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-13T08:55:09.303 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T08:55:09.304 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T08:55:09.304 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:09.403 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T08:55:09.403 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T08:55:09.404 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:09.503 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T08:55:09.503 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T08:55:09.504 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:10.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-13T08:55:10.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-13T08:55:15.605 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T08:55:15.606 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T08:55:15.606 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:15.701 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T08:55:15.701 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T08:55:15.702 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T08:55:15.702 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:15.765 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T08:55:15.765 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T08:55:15.765 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:15.851 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~6s 2024-09-13T08:55:21.155 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-09-13T08:55:21.155 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-13T08:55:21.868 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T08:55:21.868 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T08:55:21.868 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T08:55:21.868 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:21.892 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T08:55:21.892 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T08:55:21.892 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:21.911 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T08:55:21.911 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T08:55:21.911 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T08:55:21.977 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T08:55:21.978 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T08:55:21.978 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-13T08:55:22.022 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:55:22.022 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T08:55:22.405 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:55:22.405 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T08:55:22.592 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:55:22.592 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T08:55:22.745 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:55:22.745 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T08:55:22.910 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:55:22.910 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T08:55:23.184 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:55:23.185 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T08:55:23.396 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:55:23.397 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T08:55:23.561 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:55:23.561 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T08:55:23.724 INFO:tasks.ceph:Archiving mon data... 2024-09-13T08:55:23.725 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319/data/mon.a.tgz 2024-09-13T08:55:23.726 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-09-13T08:55:23.735 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.UgYImTyxE9 2024-09-13T08:55:23.735 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.UgYImTyxE9 2024-09-13T08:55:24.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.UgYImTyxE9 2024-09-13T08:55:24.183 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.UgYImTyxE9 is 2MB 2024-09-13T08:55:24.270 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.UgYImTyxE9 2024-09-13T08:55:24.279 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319/data/mon.b.tgz 2024-09-13T08:55:24.279 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-13T08:55:24.288 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.2tEnpfIJxO 2024-09-13T08:55:24.288 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.2tEnpfIJxO 2024-09-13T08:55:24.650 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.2tEnpfIJxO 2024-09-13T08:55:24.726 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.2tEnpfIJxO is 2MB 2024-09-13T08:55:24.810 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.2tEnpfIJxO 2024-09-13T08:55:24.819 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319/data/mon.c.tgz 2024-09-13T08:55:24.819 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-13T08:55:24.870 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.2jpu4Oc0y4 2024-09-13T08:55:24.870 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.2jpu4Oc0y4 2024-09-13T08:55:25.214 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.2jpu4Oc0y4 2024-09-13T08:55:25.286 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.2jpu4Oc0y4 is 2MB 2024-09-13T08:55:25.371 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.2jpu4Oc0y4 2024-09-13T08:55:25.379 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T08:55:25.380 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T08:55:25.383 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T08:55:25.451 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-13T08:55:25.489 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-13T08:55:25.489 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T08:55:25.491 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319/remote/smithi044/crash 2024-09-13T08:55:25.492 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T08:55:25.519 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319/remote/smithi129/crash 2024-09-13T08:55:25.520 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T08:55:25.547 INFO:tasks.ceph:Shutting down logrotate 2024-09-13T08:55:25.548 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:55:25.569 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T08:55:25.644 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T08:55:25.665 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T08:55:25.683 INFO:tasks.ceph:Compressing logs... 2024-09-13T08:55:25.683 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T08:55:25.713 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T08:55:25.744 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39153.log 2024-09-13T08:55:25.745 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42813.log 2024-09-13T08:55:25.746 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51904.log 2024-09-13T08:55:25.746 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39153.log.gz 2024-09-13T08:55:25.747 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40647.log 2024-09-13T08:55:25.747 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42813.log.gz 2024-09-13T08:55:25.748 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53635.log 2024-09-13T08:55:25.748 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51904.log.gz 2024-09-13T08:55:25.748 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43126.log 2024-09-13T08:55:25.749 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40647.log.gz 2024-09-13T08:55:25.749 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46229.log 2024-09-13T08:55:25.750 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53635.log.gz 2024-09-13T08:55:25.750 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45836.log 2024-09-13T08:55:25.751 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46229.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43126.log.gz 2024-09-13T08:55:25.751 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56456.log 2024-09-13T08:55:25.751 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46229.log.gz 2024-09-13T08:55:25.752 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42175.log 2024-09-13T08:55:25.752 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45836.log.gz 2024-09-13T08:55:25.753 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47298.log 2024-09-13T08:55:25.753 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56456.log.gz 2024-09-13T08:55:25.754 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36508.log 2024-09-13T08:55:25.754 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42175.log.gz 2024-09-13T08:55:25.755 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50690.log 2024-09-13T08:55:25.755 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47298.log.gz 2024-09-13T08:55:25.756 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34740.log 2024-09-13T08:55:25.756 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36508.log.gz 2024-09-13T08:55:25.756 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.50690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50690.log.gz 2024-09-13T08:55:25.756 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49717.log 2024-09-13T08:55:25.757 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42944.log 2024-09-13T08:55:25.757 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34740.log.gz 2024-09-13T08:55:25.758 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38190.log 2024-09-13T08:55:25.758 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49717.log.gz 2024-09-13T08:55:25.759 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55326.log 2024-09-13T08:55:25.760 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42944.log.gz 2024-09-13T08:55:25.760 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49310.log 2024-09-13T08:55:25.760 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38190.log.gz 2024-09-13T08:55:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41899.log 2024-09-13T08:55:25.761 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55326.log.gz 2024-09-13T08:55:25.762 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28991.log 2024-09-13T08:55:25.762 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-13T08:55:25.762 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54304.log 2024-09-13T08:55:25.762 INFO:teuthology.orchestra.run.smithi044.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.49310.log.gz 2024-09-13T08:55:25.763 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2024-09-13T08:55:25.763 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41899.log.gz 2024-09-13T08:55:25.765 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53609.log 2024-09-13T08:55:25.765 INFO:teuthology.orchestra.run.smithi044.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54304.log.gz 2024-09-13T08:55:25.765 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36658.log 2024-09-13T08:55:25.765 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2024-09-13T08:55:25.765 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28991.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-13T08:55:25.766 INFO:teuthology.orchestra.run.smithi129.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.28991.log.gz 2024-09-13T08:55:25.766 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34203.log 2024-09-13T08:55:25.766 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28967.log 2024-09-13T08:55:25.766 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.3.34203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.34203.log.gz 2024-09-13T08:55:25.766 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29070.log 2024-09-13T08:55:25.766 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28967.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-13T08:55:25.766 INFO:teuthology.orchestra.run.smithi129.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28967.log.gz 2024-09-13T08:55:25.768 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29039.log 2024-09-13T08:55:25.768 INFO:teuthology.orchestra.run.smithi129.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29070.log.gz 2024-09-13T08:55:25.768 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29094.log 2024-09-13T08:55:25.770 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40881.log 2024-09-13T08:55:25.770 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53609.log.gz 2024-09-13T08:55:25.770 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47142.log 2024-09-13T08:55:25.770 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36658.log.gz 2024-09-13T08:55:25.770 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41211.log 2024-09-13T08:55:25.770 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40881.log.gz 2024-09-13T08:55:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43805.log 2024-09-13T08:55:25.771 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47142.log.gz 2024-09-13T08:55:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43462.log 2024-09-13T08:55:25.771 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41211.log.gz 2024-09-13T08:55:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43805.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-13T08:55:25.771 INFO:teuthology.orchestra.run.smithi044.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.43805.log.gz 2024-09-13T08:55:25.771 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37068.log 2024-09-13T08:55:25.771 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43462.log.gz 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46492.log 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40012.log 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37068.log.gz 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46492.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46492.log.gz 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48511.log 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46281.log 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40012.log.gz 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36573.log 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48511.log.gz 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49336.log 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.46281.log.gz 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36573.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.36573.log.gz 2024-09-13T08:55:25.777 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56610.log 2024-09-13T08:55:25.778 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29039.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-13T08:55:25.778 INFO:teuthology.orchestra.run.smithi129.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29039.log.gz 2024-09-13T08:55:25.778 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29094.log: gzip -5 --verbose 86.7% -- -- replaced with /var/log/ceph/ceph-client.admin.29094.log.gz /var/log/ceph/ceph.log 2024-09-13T08:55:25.778 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-13T08:55:25.778 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28943.log 2024-09-13T08:55:25.778 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34114.log 2024-09-13T08:55:25.779 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28943.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-13T08:55:25.779 INFO:teuthology.orchestra.run.smithi129.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28943.log.gz 2024-09-13T08:55:25.779 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.1.34114.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.29118.log -- replaced with /var/log/ceph/ceph-client.1.34114.log.gz 2024-09-13T08:55:25.779 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-13T08:55:25.779 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-13T08:55:25.779 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29118.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29118.log.gz 2024-09-13T08:55:25.779 INFO:teuthology.orchestra.run.smithi129.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-13T08:55:25.779 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29015.log 2024-09-13T08:55:25.779 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-13T08:55:25.780 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29015.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29015.log.gz 2024-09-13T08:55:25.782 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54881.log 2024-09-13T08:55:25.782 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49336.log.gz 2024-09-13T08:55:25.782 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56610.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56610.log.gz 2024-09-13T08:55:25.782 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51599.log 2024-09-13T08:55:25.783 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28790.log 2024-09-13T08:55:25.783 INFO:teuthology.orchestra.run.smithi044.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.54881.log.gz 2024-09-13T08:55:25.783 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44773.log 2024-09-13T08:55:25.783 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51599.log.gz 2024-09-13T08:55:25.783 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.28790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47376.log 2024-09-13T08:55:25.783 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28790.log.gz 2024-09-13T08:55:25.783 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44773.log: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-13T08:55:25.783 INFO:teuthology.orchestra.run.smithi044.stderr:gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.44773.log.gz --verbose 2024-09-13T08:55:25.783 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.46435.log 2024-09-13T08:55:25.784 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34096.log 2024-09-13T08:55:25.784 INFO:teuthology.orchestra.run.smithi129.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-13T08:55:25.785 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34198.log 2024-09-13T08:55:25.786 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35962.log 2024-09-13T08:55:25.786 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47376.log.gz 2024-09-13T08:55:25.786 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35026.log 2024-09-13T08:55:25.786 INFO:teuthology.orchestra.run.smithi044.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.46435.log.gz 2024-09-13T08:55:25.787 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55090.log 2024-09-13T08:55:25.787 INFO:teuthology.orchestra.run.smithi044.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.35962.log.gz 2024-09-13T08:55:25.787 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53694.log 2024-09-13T08:55:25.787 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35026.log.gz 2024-09-13T08:55:25.787 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49612.log 2024-09-13T08:55:25.787 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55090.log.gz 2024-09-13T08:55:25.789 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.1.34096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34096.log.gz 2024-09-13T08:55:25.789 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.3.34198.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-13T08:55:25.789 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.34198.log.gz 2024-09-13T08:55:25.789 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-13T08:55:25.790 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52891.log 2024-09-13T08:55:25.791 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53694.log.gz 2024-09-13T08:55:25.791 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56584.log 2024-09-13T08:55:25.791 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49612.log.gz 2024-09-13T08:55:25.791 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36269.log 2024-09-13T08:55:25.791 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52891.log.gz 2024-09-13T08:55:25.791 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42455.log 2024-09-13T08:55:25.791 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56584.log.gz 2024-09-13T08:55:25.791 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.2.36269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51085.log 2024-09-13T08:55:25.791 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.36269.log.gz 2024-09-13T08:55:25.792 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42455.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-13T08:55:25.792 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42455.log.gz 2024-09-13T08:55:25.793 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50731.log 2024-09-13T08:55:25.793 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51085.log.gz 2024-09-13T08:55:25.793 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38665.log 2024-09-13T08:55:25.794 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49124.log 2024-09-13T08:55:25.794 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50731.log.gz 2024-09-13T08:55:25.794 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40784.log 2024-09-13T08:55:25.794 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38665.log.gz 2024-09-13T08:55:25.795 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-13T08:55:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54667.log 2024-09-13T08:55:25.796 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49124.log.gz 2024-09-13T08:55:25.796 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55838.log 2024-09-13T08:55:25.797 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40784.log.gz 2024-09-13T08:55:25.797 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50406.log 2024-09-13T08:55:25.797 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54667.log.gz 2024-09-13T08:55:25.798 INFO:teuthology.orchestra.run.smithi129.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-13T08:55:25.799 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46654.log 2024-09-13T08:55:25.799 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55838.log.gz 2024-09-13T08:55:25.799 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48115.log 2024-09-13T08:55:25.799 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50406.log.gz 2024-09-13T08:55:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45061.log 2024-09-13T08:55:25.800 INFO:teuthology.orchestra.run.smithi044.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.46654.log.gz 2024-09-13T08:55:25.801 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38932.log 2024-09-13T08:55:25.801 INFO:teuthology.orchestra.run.smithi044.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48115.log.gz 2024-09-13T08:55:25.802 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45061.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.45061.log.gz -5 2024-09-13T08:55:25.802 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50562.log 2024-09-13T08:55:25.802 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41407.log 2024-09-13T08:55:25.803 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38932.log.gz 2024-09-13T08:55:25.803 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39452.log 2024-09-13T08:55:25.803 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50562.log.gz 2024-09-13T08:55:25.804 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41633.log 2024-09-13T08:55:25.804 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41407.log.gz 2024-09-13T08:55:25.805 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48914.log 2024-09-13T08:55:25.805 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39452.log.gz 2024-09-13T08:55:25.806 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50510.log 2024-09-13T08:55:25.806 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41633.log.gz 2024-09-13T08:55:25.807 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.48914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44114.log 2024-09-13T08:55:25.807 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48914.log.gz 2024-09-13T08:55:25.808 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50510.log: /var/log/ceph/ceph-client.admin.50638.log 2024-09-13T08:55:25.808 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50510.log.gz 2024-09-13T08:55:25.809 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37398.log 2024-09-13T08:55:25.809 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44114.log.gz 2024-09-13T08:55:25.810 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50638.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50638.log.gz 2024-09-13T08:55:25.810 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54629.log 2024-09-13T08:55:25.810 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41133.log 2024-09-13T08:55:25.810 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37398.log.gz 2024-09-13T08:55:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40725.log 2024-09-13T08:55:25.811 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54629.log.gz 2024-09-13T08:55:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44520.log 2024-09-13T08:55:25.812 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41133.log.gz 2024-09-13T08:55:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45545.log 2024-09-13T08:55:25.813 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40725.log.gz 2024-09-13T08:55:25.814 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44322.log 2024-09-13T08:55:25.814 INFO:teuthology.orchestra.run.smithi044.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44520.log.gz 2024-09-13T08:55:25.814 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.45545.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.0.45545.log.gz /var/log/ceph/ceph-client.admin.41339.log 2024-09-13T08:55:25.815 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-09-13T08:55:25.815 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47638.log 2024-09-13T08:55:25.816 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44322.log.gz 2024-09-13T08:55:25.816 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36602.log 2024-09-13T08:55:25.816 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41339.log.gz 2024-09-13T08:55:25.817 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39205.log 2024-09-13T08:55:25.817 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.47638.log.gz 2024-09-13T08:55:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45690.log 2024-09-13T08:55:25.818 INFO:teuthology.orchestra.run.smithi044.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36602.log.gz 2024-09-13T08:55:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56274.log 2024-09-13T08:55:25.819 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39205.log.gz 2024-09-13T08:55:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51032.log 2024-09-13T08:55:25.820 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45690.log.gz 2024-09-13T08:55:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48193.log 2024-09-13T08:55:25.821 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56274.log.gz 2024-09-13T08:55:25.821 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49258.log 2024-09-13T08:55:25.821 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.51032.log.gz 2024-09-13T08:55:25.822 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47220.log 2024-09-13T08:55:25.822 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48193.log.gz 2024-09-13T08:55:25.823 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56144.log 2024-09-13T08:55:25.823 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.49258.log.gz 2024-09-13T08:55:25.824 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55272.log 2024-09-13T08:55:25.824 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47220.log.gz 2024-09-13T08:55:25.825 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54121.log 2024-09-13T08:55:25.825 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56144.log.gz 2024-09-13T08:55:25.831 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43623.log 2024-09-13T08:55:25.831 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55272.log.gz 2024-09-13T08:55:25.831 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35179.log 2024-09-13T08:55:25.831 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54121.log.gz 2024-09-13T08:55:25.831 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43623.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.43623.log.gz 2024-09-13T08:55:25.831 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36183.log 2024-09-13T08:55:25.831 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44624.log 2024-09-13T08:55:25.831 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35179.log.gz 2024-09-13T08:55:25.831 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55380.log 87.8% 2024-09-13T08:55:25.831 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36183.log.gz 2024-09-13T08:55:25.832 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39836.log 2024-09-13T08:55:25.832 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44624.log.gz 2024-09-13T08:55:25.832 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49388.log 2024-09-13T08:55:25.832 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.55380.log.gz 2024-09-13T08:55:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39283.log 2024-09-13T08:55:25.833 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39836.log.gz 2024-09-13T08:55:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39582.log 2024-09-13T08:55:25.833 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.49388.log.gz 2024-09-13T08:55:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48929.log 2024-09-13T08:55:25.834 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39283.log.gz 2024-09-13T08:55:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49007.log 2024-09-13T08:55:25.835 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39582.log.gz 2024-09-13T08:55:25.835 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55760.log 2024-09-13T08:55:25.836 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48929.log.gz 2024-09-13T08:55:25.837 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48572.log 2024-09-13T08:55:25.837 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49007.log.gz 2024-09-13T08:55:25.837 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43269.log 2024-09-13T08:55:25.837 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55760.log.gz 2024-09-13T08:55:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47586.log 87.2% 2024-09-13T08:55:25.838 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48572.log.gz 2024-09-13T08:55:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53900.log 2024-09-13T08:55:25.840 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43269.log.gz 2024-09-13T08:55:25.840 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51006.log 2024-09-13T08:55:25.840 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47586.log.gz 2024-09-13T08:55:25.840 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52639.log 2024-09-13T08:55:25.841 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53900.log.gz 2024-09-13T08:55:25.841 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37094.log 2024-09-13T08:55:25.841 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51006.log.gz 2024-09-13T08:55:25.842 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36159.log 2024-09-13T08:55:25.842 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52639.log.gz 2024-09-13T08:55:25.843 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.4.log: 94.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-13T08:55:25.844 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37226.log 2024-09-13T08:55:25.844 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37094.log.gz 2024-09-13T08:55:25.844 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.36159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36159.log.gz 2024-09-13T08:55:25.844 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49580.log 2024-09-13T08:55:25.844 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53159.log 2024-09-13T08:55:25.845 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37226.log.gz 2024-09-13T08:55:25.845 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46409.log 2024-09-13T08:55:25.846 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49580.log.gz 2024-09-13T08:55:25.846 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50261.log 2024-09-13T08:55:25.847 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53159.log.gz 2024-09-13T08:55:25.847 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41769.log 2024-09-13T08:55:25.847 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46409.log.gz 2024-09-13T08:55:25.848 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53843.log 2024-09-13T08:55:25.848 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50261.log.gz 2024-09-13T08:55:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42970.log 2024-09-13T08:55:25.849 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.41769.log.gz 2024-09-13T08:55:25.850 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45586.log 2024-09-13T08:55:25.850 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53843.log.gz 2024-09-13T08:55:25.851 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35366.log 2024-09-13T08:55:25.851 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.42970.log.gz 2024-09-13T08:55:25.851 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50324.log 2024-09-13T08:55:25.852 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45586.log.gz 2024-09-13T08:55:25.852 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47966.log 2024-09-13T08:55:25.852 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35366.log.gz 2024-09-13T08:55:25.853 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45116.log 2024-09-13T08:55:25.853 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50324.log.gz 2024-09-13T08:55:25.854 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37656.log 2024-09-13T08:55:25.854 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47966.log.gz 2024-09-13T08:55:25.855 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53505.log 2024-09-13T08:55:25.855 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45116.log.gz 2024-09-13T08:55:25.856 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.37656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54499.log 2024-09-13T08:55:25.856 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37656.log.gz 2024-09-13T08:55:25.856 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53505.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.45742.log 2024-09-13T08:55:25.857 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53505.log.gz 2024-09-13T08:55:25.857 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29057.log 2024-09-13T08:55:25.857 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54499.log.gz 2024-09-13T08:55:25.858 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42097.log 2024-09-13T08:55:25.858 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45742.log.gz 2024-09-13T08:55:25.859 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55168.log 2024-09-13T08:55:25.860 INFO:teuthology.orchestra.run.smithi044.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.29057.log.gz 2024-09-13T08:55:25.860 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51245.log 2024-09-13T08:55:25.860 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42097.log.gz 2024-09-13T08:55:25.861 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2024-09-13T08:55:25.861 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55168.log.gz 2024-09-13T08:55:25.861 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43843.log 2024-09-13T08:55:25.862 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51245.log.gz 2024-09-13T08:55:25.862 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52769.log 2024-09-13T08:55:25.862 INFO:teuthology.orchestra.run.smithi044.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2024-09-13T08:55:25.863 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42299.log 2024-09-13T08:55:25.863 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43843.log.gz 2024-09-13T08:55:25.864 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45716.log 2024-09-13T08:55:25.864 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52769.log.gz 2024-09-13T08:55:25.864 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39504.log 2024-09-13T08:55:25.865 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42299.log.gz 2024-09-13T08:55:25.865 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37146.log 2024-09-13T08:55:25.866 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45716.log.gz 2024-09-13T08:55:25.866 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44270.log 2024-09-13T08:55:25.866 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39504.log.gz 2024-09-13T08:55:25.867 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50022.log 2024-09-13T08:55:25.867 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37146.log.gz 2024-09-13T08:55:25.868 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47940.log 2024-09-13T08:55:25.868 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44270.log.gz 2024-09-13T08:55:25.869 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45179.log 2024-09-13T08:55:25.869 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50022.log.gz 2024-09-13T08:55:25.869 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51271.log 2024-09-13T08:55:25.870 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47940.log.gz 2024-09-13T08:55:25.870 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49150.log 2024-09-13T08:55:25.871 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45179.log.gz 2024-09-13T08:55:25.871 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42917.log 2024-09-13T08:55:25.871 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51271.log.gz 2024-09-13T08:55:25.873 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49150.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-13T08:55:25.873 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49150.log.gz 2024-09-13T08:55:25.873 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44166.log 2024-09-13T08:55:25.873 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42917.log.gz 2024-09-13T08:55:25.874 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53302.log 2024-09-13T08:55:25.875 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54447.log 2024-09-13T08:55:25.875 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44166.log.gz 2024-09-13T08:55:25.876 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46125.log 2024-09-13T08:55:25.876 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53302.log.gz 2024-09-13T08:55:25.877 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54447.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54447.log.gz 2024-09-13T08:55:25.877 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37921.log 2024-09-13T08:55:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40090.log 2024-09-13T08:55:25.878 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.46125.log.gz 2024-09-13T08:55:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34658.log 2024-09-13T08:55:25.879 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37921.log.gz 2024-09-13T08:55:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55650.log 2024-09-13T08:55:25.880 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40090.log.gz 2024-09-13T08:55:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56482.log 2024-09-13T08:55:25.881 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34658.log.gz 2024-09-13T08:55:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55011.log 2024-09-13T08:55:25.882 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55650.log.gz 2024-09-13T08:55:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37172.log 2024-09-13T08:55:25.883 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56482.log.gz 2024-09-13T08:55:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49691.log 2024-09-13T08:55:25.884 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.55011.log.gz 2024-09-13T08:55:25.885 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46629.log 2024-09-13T08:55:25.885 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37172.log.gz 2024-09-13T08:55:25.886 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49691.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49691.log.gz 2024-09-13T08:55:25.886 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39899.log 2024-09-13T08:55:25.886 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49665.log 2024-09-13T08:55:25.887 INFO:teuthology.orchestra.run.smithi044.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.46629.log.gz 2024-09-13T08:55:25.888 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56248.log 2024-09-13T08:55:25.888 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.39899.log.gz 2024-09-13T08:55:25.888 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.49665.log: --verbose -- /var/log/ceph/ceph-client.admin.46840.log 2024-09-13T08:55:25.889 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49665.log.gz 2024-09-13T08:55:25.889 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55064.log 2024-09-13T08:55:25.890 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56248.log.gz 2024-09-13T08:55:25.890 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46840.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46840.log.gz 2024-09-13T08:55:25.890 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39932.log 2024-09-13T08:55:25.891 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35173.log 2024-09-13T08:55:25.892 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55064.log.gz 2024-09-13T08:55:25.892 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39932.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.39932.log.gz 2024-09-13T08:55:25.893 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49996.log 2024-09-13T08:55:25.893 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40751.log 2024-09-13T08:55:25.893 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35173.log.gz 2024-09-13T08:55:25.894 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48018.log 2024-09-13T08:55:25.895 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49996.log.gz 2024-09-13T08:55:25.895 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40751.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40751.log.gz 2024-09-13T08:55:25.895 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45914.log 2024-09-13T08:55:25.896 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33494.log 2024-09-13T08:55:25.897 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48018.log.gz 2024-09-13T08:55:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35183.log 2024-09-13T08:55:25.897 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45914.log.gz 2024-09-13T08:55:25.898 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.33494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36788.log 2024-09-13T08:55:25.899 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33494.log.gz 2024-09-13T08:55:25.899 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52230.log 2024-09-13T08:55:25.899 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35183.log.gz 2024-09-13T08:55:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38824.log 2024-09-13T08:55:25.900 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36788.log.gz 2024-09-13T08:55:25.901 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49362.log 87.6% 2024-09-13T08:55:25.901 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52230.log.gz 2024-09-13T08:55:25.902 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46044.log 2024-09-13T08:55:25.902 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38824.log.gz 2024-09-13T08:55:25.903 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43488.log 2024-09-13T08:55:25.903 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.49362.log.gz 2024-09-13T08:55:25.904 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35587.log 2024-09-13T08:55:25.904 INFO:teuthology.orchestra.run.smithi044.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.46044.log.gz 2024-09-13T08:55:25.905 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46759.log 2024-09-13T08:55:25.905 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43488.log.gz 2024-09-13T08:55:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52496.log 2024-09-13T08:55:25.906 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35587.log.gz 2024-09-13T08:55:25.907 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39437.log 2024-09-13T08:55:25.907 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46759.log.gz 2024-09-13T08:55:25.908 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38876.log 2024-09-13T08:55:25.908 INFO:teuthology.orchestra.run.smithi044.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52496.log.gz 2024-09-13T08:55:25.909 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.39437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39437.log.gz 2024-09-13T08:55:25.909 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40420.log 2024-09-13T08:55:25.910 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50206.log 2024-09-13T08:55:25.910 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38876.log.gz 2024-09-13T08:55:25.910 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35384.log 2024-09-13T08:55:25.911 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40420.log.gz 2024-09-13T08:55:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50376.log 2024-09-13T08:55:25.912 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50206.log.gz 2024-09-13T08:55:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52943.log 2024-09-13T08:55:25.913 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35384.log.gz 2024-09-13T08:55:25.913 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38561.log 2024-09-13T08:55:25.914 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50376.log.gz 2024-09-13T08:55:25.914 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40116.log 2024-09-13T08:55:25.915 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52943.log.gz 2024-09-13T08:55:25.915 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54907.log 2024-09-13T08:55:25.916 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38561.log.gz 2024-09-13T08:55:25.916 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40116.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-13T08:55:25.916 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40116.log.gz 2024-09-13T08:55:25.917 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54907.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54907.log.gz 2024-09-13T08:55:25.917 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52795.log 2024-09-13T08:55:25.918 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29285.log 2024-09-13T08:55:25.919 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49518.log 2024-09-13T08:55:25.920 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52795.log.gz 2024-09-13T08:55:25.920 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29285.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29285.log.gz 2024-09-13T08:55:25.921 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44062.log 2024-09-13T08:55:25.921 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38438.log 2024-09-13T08:55:25.922 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49518.log.gz 2024-09-13T08:55:25.923 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42013.log 2024-09-13T08:55:25.923 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44062.log.gz 2024-09-13T08:55:25.924 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38438.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38438.log.gz 2024-09-13T08:55:25.924 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40472.log 2024-09-13T08:55:25.925 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42013.log: /var/log/ceph/ceph-client.admin.48546.log 2024-09-13T08:55:25.925 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42013.log.gz 2024-09-13T08:55:25.926 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39309.log 2024-09-13T08:55:25.926 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40472.log.gz 2024-09-13T08:55:25.927 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48546.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48546.log.gz 2024-09-13T08:55:25.927 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40907.log 2024-09-13T08:55:25.928 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44650.log 2024-09-13T08:55:25.928 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39309.log.gz 2024-09-13T08:55:25.929 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51625.log 2024-09-13T08:55:25.929 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40907.log.gz 2024-09-13T08:55:25.930 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49284.log 2024-09-13T08:55:25.930 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44650.log.gz 2024-09-13T08:55:25.931 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56170.log 2024-09-13T08:55:25.932 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51625.log.gz 2024-09-13T08:55:25.932 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50232.log 2024-09-13T08:55:25.932 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49284.log.gz 2024-09-13T08:55:25.933 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56170.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56170.log.gzgzip 2024-09-13T08:55:25.933 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44851.log 2024-09-13T08:55:25.934 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35580.log 2024-09-13T08:55:25.935 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50232.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50232.log.gz 2024-09-13T08:55:25.935 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50685.log 2024-09-13T08:55:25.936 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44851.log.gz 2024-09-13T08:55:25.937 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54803.log 2024-09-13T08:55:25.937 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35580.log.gz 2024-09-13T08:55:25.937 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.50685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36298.log 2024-09-13T08:55:25.938 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50685.log.gz 2024-09-13T08:55:25.939 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36924.log 2024-09-13T08:55:25.939 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.54803.log.gz 2024-09-13T08:55:25.940 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38724.log 2024-09-13T08:55:25.940 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36298.log.gz 2024-09-13T08:55:25.941 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35910.log 2024-09-13T08:55:25.941 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36924.log.gz 2024-09-13T08:55:25.942 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56534.log 2024-09-13T08:55:25.942 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38724.log.gz 2024-09-13T08:55:25.943 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48862.log 2024-09-13T08:55:25.943 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35910.log.gz 2024-09-13T08:55:25.944 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56534.log: 86.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.56534.log.gz 2024-09-13T08:55:25.944 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47246.log 2024-09-13T08:55:25.945 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.48862.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37827.log 2024-09-13T08:55:25.946 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48862.log.gz 2024-09-13T08:55:25.946 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45966.log 2024-09-13T08:55:25.946 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47246.log.gz 2024-09-13T08:55:25.947 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34974.log 2024-09-13T08:55:25.947 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37827.log.gz 2024-09-13T08:55:25.948 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.45966.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50048.log 2024-09-13T08:55:25.949 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45966.log.gz 2024-09-13T08:55:25.949 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34974.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34974.log.gz 2024-09-13T08:55:25.950 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40498.log 2024-09-13T08:55:25.951 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47402.log 2024-09-13T08:55:25.951 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50048.log.gz 2024-09-13T08:55:25.952 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40498.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40498.log.gz 2024-09-13T08:55:25.952 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37947.log 2024-09-13T08:55:25.953 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39810.log 2024-09-13T08:55:25.953 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47402.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47402.log.gz 2024-09-13T08:55:25.954 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41873.log 2024-09-13T08:55:25.954 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37947.log.gz 2024-09-13T08:55:25.955 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39810.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.39810.log.gz 2024-09-13T08:55:25.955 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48428.log 2024-09-13T08:55:25.956 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45638.log 2024-09-13T08:55:25.957 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41873.log.gz 2024-09-13T08:55:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50757.log 2024-09-13T08:55:25.957 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48428.log.gz 2024-09-13T08:55:25.958 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40168.log 2024-09-13T08:55:25.959 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45638.log.gz 2024-09-13T08:55:25.959 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48909.log 2024-09-13T08:55:25.960 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50757.log.gz 2024-09-13T08:55:25.961 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37554.log 2024-09-13T08:55:25.961 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40168.log.gz 2024-09-13T08:55:25.962 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.48909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48909.log.gz 2024-09-13T08:55:25.962 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45087.log 2024-09-13T08:55:25.963 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52665.log 2024-09-13T08:55:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37554.log: /var/log/ceph/ceph-client.admin.45087.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.37554.log.gz 2024-09-13T08:55:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51677.log 2024-09-13T08:55:25.964 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45087.log.gz 2024-09-13T08:55:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54226.log 87.4% 2024-09-13T08:55:25.965 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52665.log.gz 2024-09-13T08:55:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55987.log 2024-09-13T08:55:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51677.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51677.log.gz 2024-09-13T08:55:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49206.log 2024-09-13T08:55:25.967 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54226.log.gz 2024-09-13T08:55:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40249.log 2024-09-13T08:55:25.968 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55987.log.gz 2024-09-13T08:55:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35383.log 2024-09-13T08:55:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49206.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49206.log.gz 2024-09-13T08:55:25.970 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55194.log 2024-09-13T08:55:25.970 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40249.log.gz 2024-09-13T08:55:25.971 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41289.log 2024-09-13T08:55:25.971 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35383.log.gz 2024-09-13T08:55:25.972 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.55194.log: -- /var/log/ceph/ceph-client.admin.44825.log 2024-09-13T08:55:25.973 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55194.log.gz 2024-09-13T08:55:25.973 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41563.log 2024-09-13T08:55:25.974 INFO:teuthology.orchestra.run.smithi044.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41289.log.gz 2024-09-13T08:55:25.974 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48167.log 2024-09-13T08:55:25.975 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44825.log.gz 2024-09-13T08:55:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55484.log 2024-09-13T08:55:25.976 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.41563.log.gz 2024-09-13T08:55:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35381.log 2024-09-13T08:55:25.977 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48167.log.gz 2024-09-13T08:55:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36762.log 2024-09-13T08:55:25.978 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55484.log.gz 2024-09-13T08:55:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51401.log 2024-09-13T08:55:25.979 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35381.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35381.log.gz 2024-09-13T08:55:25.980 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36324.log 2024-09-13T08:55:25.980 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36762.log.gz 2024-09-13T08:55:25.981 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54693.log 2024-09-13T08:55:25.981 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51401.log.gz 2024-09-13T08:55:25.982 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44436.log 2024-09-13T08:55:25.982 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36324.log.gz 2024-09-13T08:55:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37254.log 2024-09-13T08:55:25.983 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54693.log.gz 2024-09-13T08:55:25.984 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44436.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.42865.log -- replaced with /var/log/ceph/ceph-client.admin.44436.log.gz 2024-09-13T08:55:25.984 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-09-13T08:55:25.985 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.37254.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35184.log 2024-09-13T08:55:25.986 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37254.log.gz 2024-09-13T08:55:25.986 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53354.log 2024-09-13T08:55:25.986 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42865.log.gz 2024-09-13T08:55:25.987 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35596.log 2024-09-13T08:55:25.987 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35184.log.gz 2024-09-13T08:55:25.988 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37651.log 2024-09-13T08:55:25.989 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53354.log.gz 2024-09-13T08:55:25.989 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46151.log 2024-09-13T08:55:25.989 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35596.log.gz 2024-09-13T08:55:25.990 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.37651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37651.log.gz 2024-09-13T08:55:25.990 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34896.log 2024-09-13T08:55:25.991 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39758.log 2024-09-13T08:55:25.992 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46151.log.gz 2024-09-13T08:55:25.993 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38613.log 2024-09-13T08:55:25.993 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34896.log.gz 2024-09-13T08:55:25.994 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39758.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.39758.log.gz 2024-09-13T08:55:25.994 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37697.log 2024-09-13T08:55:25.995 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44403.log 2024-09-13T08:55:25.995 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38613.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38613.log.gz 2024-09-13T08:55:25.996 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42149.log 2024-09-13T08:55:25.996 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37697.log.gz 2024-09-13T08:55:25.997 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53276.log 2024-09-13T08:55:25.997 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44403.log.gz 2024-09-13T08:55:25.998 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.42149.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28831.log 2024-09-13T08:55:25.998 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42149.log.gz 2024-09-13T08:55:25.999 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49414.log 2024-09-13T08:55:25.999 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz 2024-09-13T08:55:26.000 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.28831.log: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.28831.log.gz 2024-09-13T08:55:26.000 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37528.log 2024-09-13T08:55:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43729.log 2024-09-13T08:55:26.001 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49414.log.gz 2024-09-13T08:55:26.002 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36352.log 2024-09-13T08:55:26.002 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37528.log.gz 2024-09-13T08:55:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54777.log 2024-09-13T08:55:26.003 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43729.log.gz 2024-09-13T08:55:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53935.log 2024-09-13T08:55:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36352.log: /var/log/ceph/ceph-client.admin.54777.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36352.log.gz 2024-09-13T08:55:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37476.log 2024-09-13T08:55:26.005 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.54777.log.gz 2024-09-13T08:55:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47804.log 2024-09-13T08:55:26.006 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53935.log.gz 2024-09-13T08:55:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42757.log 2024-09-13T08:55:26.007 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37476.log.gz 2024-09-13T08:55:26.008 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29338.log 2024-09-13T08:55:26.008 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47804.log.gz 2024-09-13T08:55:26.010 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42757.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42757.log.gz 2024-09-13T08:55:26.010 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50083.log 2024-09-13T08:55:26.010 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28967.log 2024-09-13T08:55:26.011 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29338.log: /var/log/ceph/ceph-client.admin.50083.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29338.log.gz 2024-09-13T08:55:26.012 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39702.log 2024-09-13T08:55:26.012 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50083.log.gz 2024-09-13T08:55:26.013 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.28967.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28967.log.gz 2024-09-13T08:55:26.013 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48732.log 2024-09-13T08:55:26.014 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35078.log 2024-09-13T08:55:26.014 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39702.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39702.log.gz 2024-09-13T08:55:26.015 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56222.log 2024-09-13T08:55:26.015 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48732.log.gz 2024-09-13T08:55:26.016 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38509.log 2024-09-13T08:55:26.016 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35078.log.gz 2024-09-13T08:55:26.017 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43048.log 2024-09-13T08:55:26.017 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56222.log.gz 2024-09-13T08:55:26.018 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38967.log 2024-09-13T08:55:26.018 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38509.log.gz 2024-09-13T08:55:26.019 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34632.log 2024-09-13T08:55:26.019 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43048.log.gz 2024-09-13T08:55:26.020 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51375.log 2024-09-13T08:55:26.021 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38967.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38967.log.gz 2024-09-13T08:55:26.021 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44938.log 2024-09-13T08:55:26.021 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34632.log.gz 2024-09-13T08:55:26.022 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51375.log: gzip -5 --verbose -- 87.0% /var/log/ceph/ceph-client.admin.43518.log -- replaced with /var/log/ceph/ceph-client.admin.51375.log.gz 2024-09-13T08:55:26.022 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-09-13T08:55:26.023 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35000.log 2024-09-13T08:55:26.024 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44938.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44938.log.gz 2024-09-13T08:55:26.024 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56196.log 2024-09-13T08:55:26.025 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43518.log.gz 2024-09-13T08:55:26.026 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35000.log: gzip -5 --verbose -- 87.3% /var/log/ceph/ceph-client.admin.41691.log -- replaced with /var/log/ceph/ceph-client.admin.35000.log.gz 2024-09-13T08:55:26.026 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-09-13T08:55:26.026 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50887.log 2024-09-13T08:55:26.027 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56196.log.gz 2024-09-13T08:55:26.028 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48483.log 2024-09-13T08:55:26.028 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41691.log.gz 2024-09-13T08:55:26.029 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51878.log 2024-09-13T08:55:26.029 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50887.log.gz 2024-09-13T08:55:26.029 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48483.log: -- /var/log/ceph/ceph-client.0.43223.log 2024-09-13T08:55:26.030 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48483.log.gz 2024-09-13T08:55:26.031 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50950.log 2024-09-13T08:55:26.031 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51878.log.gz 2024-09-13T08:55:26.032 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.43223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43223.log.gz 2024-09-13T08:55:26.032 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46337.log 2024-09-13T08:55:26.033 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.50950.log: --verbose -- /var/log/ceph/ceph-client.admin.54473.log 2024-09-13T08:55:26.033 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50950.log.gz 2024-09-13T08:55:26.034 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50809.log 2024-09-13T08:55:26.034 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46337.log.gz 2024-09-13T08:55:26.035 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37801.log 2024-09-13T08:55:26.035 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54473.log.gz 2024-09-13T08:55:26.036 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46070.log 2024-09-13T08:55:26.036 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50809.log.gz 2024-09-13T08:55:26.037 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41025.log 2024-09-13T08:55:26.037 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37801.log.gz 2024-09-13T08:55:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38798.log 2024-09-13T08:55:26.038 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46070.log.gz 2024-09-13T08:55:26.039 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.41025.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35169.log 2024-09-13T08:55:26.039 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41025.log.gz 2024-09-13T08:55:26.040 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38412.log 2024-09-13T08:55:26.040 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38798.log.gz 2024-09-13T08:55:26.041 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35169.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-13T08:55:26.042 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35169.log.gz 2024-09-13T08:55:26.042 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.38412.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56326.log 2024-09-13T08:55:26.043 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38412.log.gz 2024-09-13T08:55:26.043 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29019.log 2024-09-13T08:55:26.044 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45560.log 2024-09-13T08:55:26.045 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56326.log.gz 2024-09-13T08:55:26.046 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48089.log 2024-09-13T08:55:26.046 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29019.log.gz 2024-09-13T08:55:26.047 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54017.log 2024-09-13T08:55:26.048 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45560.log.gz 2024-09-13T08:55:26.048 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38904.log 2024-09-13T08:55:26.048 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48089.log.gz 2024-09-13T08:55:26.049 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54043.log 2024-09-13T08:55:26.049 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54017.log.gz 2024-09-13T08:55:26.051 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41263.log 2024-09-13T08:55:26.051 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38904.log.gz 2024-09-13T08:55:26.052 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37979.log 2024-09-13T08:55:26.052 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54043.log.gz 2024-09-13T08:55:26.052 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41925.log 2024-09-13T08:55:26.053 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41263.log.gz 2024-09-13T08:55:26.054 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42071.log 2024-09-13T08:55:26.054 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37979.log.gz 2024-09-13T08:55:26.055 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29425.log 2024-09-13T08:55:26.056 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41925.log.gz 2024-09-13T08:55:26.056 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41185.log 2024-09-13T08:55:26.057 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42071.log.gz 2024-09-13T08:55:26.058 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45780.log 2024-09-13T08:55:26.058 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29425.log.gz 2024-09-13T08:55:26.059 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56040.log 2024-09-13T08:55:26.059 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.41185.log.gz 2024-09-13T08:55:26.060 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50980.log 2024-09-13T08:55:26.060 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45780.log.gz 2024-09-13T08:55:26.061 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52613.log 2024-09-13T08:55:26.061 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56040.log.gz 2024-09-13T08:55:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44192.log 2024-09-13T08:55:26.062 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50980.log.gz 2024-09-13T08:55:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49970.log 2024-09-13T08:55:26.063 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52613.log.gz 2024-09-13T08:55:26.064 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44192.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23965.log 2024-09-13T08:55:26.064 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44192.log.gz 2024-09-13T08:55:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41847.log 2024-09-13T08:55:26.065 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49970.log.gz 2024-09-13T08:55:26.066 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.tmp-client.admin.23965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44799.log 2024-09-13T08:55:26.066 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.23965.log.gz 2024-09-13T08:55:26.068 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40446.log 2024-09-13T08:55:26.068 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41847.log.gz 2024-09-13T08:55:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45205.log 2024-09-13T08:55:26.069 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44799.log.gz 2024-09-13T08:55:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29049.log 2024-09-13T08:55:26.070 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40446.log.gz 2024-09-13T08:55:26.071 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55510.log 2024-09-13T08:55:26.071 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45205.log.gz 2024-09-13T08:55:26.072 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36022.log 2024-09-13T08:55:26.072 INFO:teuthology.orchestra.run.smithi044.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.29049.log.gz 2024-09-13T08:55:26.073 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52195.log 2024-09-13T08:55:26.073 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55510.log.gz 2024-09-13T08:55:26.075 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48402.log 2024-09-13T08:55:26.075 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36022.log.gz 2024-09-13T08:55:26.076 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51489.log 2024-09-13T08:55:26.076 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52195.log.gz 2024-09-13T08:55:26.077 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49180.log 2024-09-13T08:55:26.077 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48402.log.gz 2024-09-13T08:55:26.078 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45888.log 2024-09-13T08:55:26.078 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51489.log.gz 2024-09-13T08:55:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45493.log 2024-09-13T08:55:26.079 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49180.log.gz 2024-09-13T08:55:26.080 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54985.log 2024-09-13T08:55:26.080 INFO:teuthology.orchestra.run.smithi044.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.45888.log.gz 2024-09-13T08:55:26.081 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47914.log 2024-09-13T08:55:26.081 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45493.log.gz 2024-09-13T08:55:26.082 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37371.log 2024-09-13T08:55:26.083 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.54985.log.gz 2024-09-13T08:55:26.083 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51774.log 2024-09-13T08:55:26.084 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47914.log.gz 2024-09-13T08:55:26.085 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44747.log 2024-09-13T08:55:26.085 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37371.log.gz 2024-09-13T08:55:26.086 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55142.log 2024-09-13T08:55:26.086 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51774.log.gz 2024-09-13T08:55:26.087 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46567.log 2024-09-13T08:55:26.087 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44747.log.gz 2024-09-13T08:55:26.088 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55562.log 2024-09-13T08:55:26.088 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55142.log.gz 2024-09-13T08:55:26.090 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36632.log 2024-09-13T08:55:26.090 INFO:teuthology.orchestra.run.smithi044.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.46567.log.gz 2024-09-13T08:55:26.090 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51137.log 2024-09-13T08:55:26.091 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55562.log.gz 2024-09-13T08:55:26.092 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55536.log 2024-09-13T08:55:26.092 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36632.log.gz 2024-09-13T08:55:26.093 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50115.log 2024-09-13T08:55:26.093 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51137.log.gz 2024-09-13T08:55:26.094 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38587.log 2024-09-13T08:55:26.094 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55536.log.gz 2024-09-13T08:55:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35391.log 2024-09-13T08:55:26.095 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50115.log.gz 2024-09-13T08:55:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55458.log 2024-09-13T08:55:26.096 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38587.log.gz 2024-09-13T08:55:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55432.log 2024-09-13T08:55:26.098 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35391.log.gz 2024-09-13T08:55:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44140.log 2024-09-13T08:55:26.098 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55458.log.gz 2024-09-13T08:55:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42227.log 2024-09-13T08:55:26.100 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55432.log.gz 2024-09-13T08:55:26.101 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47454.log 2024-09-13T08:55:26.101 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44140.log.gz 2024-09-13T08:55:26.102 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40194.log 2024-09-13T08:55:26.102 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42227.log.gz 2024-09-13T08:55:26.103 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47454.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-13T08:55:26.103 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47454.log.gz 2024-09-13T08:55:26.104 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53075.log 2024-09-13T08:55:26.104 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40194.log.gz 2024-09-13T08:55:26.105 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45992.log 2024-09-13T08:55:26.106 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53328.log 2024-09-13T08:55:26.106 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53075.log.gz 2024-09-13T08:55:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45992.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45992.log.gz 2024-09-13T08:55:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36684.log 2024-09-13T08:55:26.108 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53328.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53328.log.gz 2024-09-13T08:55:26.109 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41107.log 2024-09-13T08:55:26.110 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38307.log 2024-09-13T08:55:26.110 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36684.log.gz 2024-09-13T08:55:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41107.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41107.log.gz 2024-09-13T08:55:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38307.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38307.log.gz 2024-09-13T08:55:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40368.log 2024-09-13T08:55:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52340.log 2024-09-13T08:55:26.116 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39385.log 2024-09-13T08:55:26.116 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40368.log.gz 2024-09-13T08:55:26.117 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53192.log 2024-09-13T08:55:26.117 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52340.log.gz 2024-09-13T08:55:26.118 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39385.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39385.log.gz 2024-09-13T08:55:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56560.log 2024-09-13T08:55:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53192.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53192.log.gz 2024-09-13T08:55:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35936.log 2024-09-13T08:55:26.121 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41051.log 2024-09-13T08:55:26.121 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56560.log.gz 2024-09-13T08:55:26.122 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35936.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35936.log.gz 2024-09-13T08:55:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34922.log 2024-09-13T08:55:26.123 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41051.log.gz 2024-09-13T08:55:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28841.log 2024-09-13T08:55:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48337.log 2024-09-13T08:55:26.126 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34922.log.gz 2024-09-13T08:55:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.28841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38333.log 2024-09-13T08:55:26.126 INFO:teuthology.orchestra.run.smithi044.stderr: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.28841.log.gz 2024-09-13T08:55:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48337.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48337.log.gz 2024-09-13T08:55:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55786.log 2024-09-13T08:55:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38333.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38333.log.gz 2024-09-13T08:55:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49466.log 2024-09-13T08:55:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52598.log 2024-09-13T08:55:26.131 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55786.log.gz 2024-09-13T08:55:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49466.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49466.log.gz 2024-09-13T08:55:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.52598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52598.log.gz 2024-09-13T08:55:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38005.log 2024-09-13T08:55:26.134 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55617.log 2024-09-13T08:55:26.135 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36456.log 2024-09-13T08:55:26.135 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38005.log.gz 2024-09-13T08:55:26.136 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44903.log 2024-09-13T08:55:26.137 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55617.log.gz 2024-09-13T08:55:26.138 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36456.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36456.log.gz 2024-09-13T08:55:26.138 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41795.log 2024-09-13T08:55:26.139 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44903.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44903.log.gz 2024-09-13T08:55:26.140 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53224.log 2024-09-13T08:55:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43228.log 2024-09-13T08:55:26.141 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.41795.log.gz 2024-09-13T08:55:26.142 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53224.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53224.log.gz 2024-09-13T08:55:26.143 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.43228.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.43228.log.gz -5 2024-09-13T08:55:26.143 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55961.log 2024-09-13T08:55:26.144 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34686.log 2024-09-13T08:55:26.146 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45313.log 2024-09-13T08:55:26.146 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55961.log.gz 2024-09-13T08:55:26.146 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36840.log 2024-09-13T08:55:26.146 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34686.log.gz 2024-09-13T08:55:26.148 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45313.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45313.log.gz 2024-09-13T08:55:26.148 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45144.log 2024-09-13T08:55:26.149 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36840.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36840.log.gz 2024-09-13T08:55:26.149 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34714.log 2024-09-13T08:55:26.150 INFO:teuthology.orchestra.run.smithi129.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-13T08:55:26.151 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51956.log 2024-09-13T08:55:26.151 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45144.log.gz 2024-09-13T08:55:26.152 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46989.log 2024-09-13T08:55:26.152 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34714.log.gz 2024-09-13T08:55:26.153 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35052.log 2024-09-13T08:55:26.153 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51956.log.gz 2024-09-13T08:55:26.154 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46989.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46989.log.gz 2024-09-13T08:55:26.155 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56300.log 2024-09-13T08:55:26.155 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35052.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35052.log.gz 2024-09-13T08:55:26.156 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-09-13T08:55:26.156 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53557.log 2024-09-13T08:55:26.157 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47888.log 2024-09-13T08:55:26.158 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56300.log.gz 2024-09-13T08:55:26.158 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53557.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53557.log.gz 2024-09-13T08:55:26.159 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48812.log 2024-09-13T08:55:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47888.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47888.log.gz 2024-09-13T08:55:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2024-09-13T08:55:26.162 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47168.log 2024-09-13T08:55:26.162 INFO:teuthology.orchestra.run.smithi044.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48812.log.gz 2024-09-13T08:55:26.163 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42996.log 2024-09-13T08:55:26.163 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2024-09-13T08:55:26.164 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47168.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47168.log.gz 2024-09-13T08:55:26.164 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43321.log 2024-09-13T08:55:26.165 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42996.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42996.log.gz 2024-09-13T08:55:26.165 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51703.log 2024-09-13T08:55:26.166 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39784.log 2024-09-13T08:55:26.167 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43321.log.gz 2024-09-13T08:55:26.168 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51703.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51703.log.gz 2024-09-13T08:55:26.169 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39784.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.39784.log.gz 2024-09-13T08:55:26.169 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43425.log 2024-09-13T08:55:26.170 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53817.log 2024-09-13T08:55:26.172 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42481.log 2024-09-13T08:55:26.172 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43425.log.gz 2024-09-13T08:55:26.173 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53817.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53817.log.gz 2024-09-13T08:55:26.173 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54603.log 2024-09-13T08:55:26.174 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42481.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42481.log.gz 2024-09-13T08:55:26.175 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52041.log 2024-09-13T08:55:26.176 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44877.log 2024-09-13T08:55:26.176 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54603.log.gz 2024-09-13T08:55:26.177 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56066.log 2024-09-13T08:55:26.177 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52041.log.gz 2024-09-13T08:55:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44877.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44877.log.gz 2024-09-13T08:55:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52969.log 2024-09-13T08:55:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56066.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56066.log.gz 2024-09-13T08:55:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34844.log 2024-09-13T08:55:26.182 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42377.log 2024-09-13T08:55:26.182 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52969.log.gz 2024-09-13T08:55:26.183 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34844.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34844.log.gz 2024-09-13T08:55:26.183 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42839.log 2024-09-13T08:55:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50835.log 2024-09-13T08:55:26.184 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42377.log.gz 2024-09-13T08:55:26.185 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35130.log 2024-09-13T08:55:26.186 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42839.log.gz 2024-09-13T08:55:26.187 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50835.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50835.log.gz 2024-09-13T08:55:26.187 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40316.log 2024-09-13T08:55:26.188 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35130.log.gz 2024-09-13T08:55:26.188 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51349.log 2024-09-13T08:55:26.190 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36274.log 2024-09-13T08:55:26.190 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40316.log.gz 2024-09-13T08:55:26.191 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51349.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.51349.log.gz 2024-09-13T08:55:26.192 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.2.36274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.36274.log.gz 2024-09-13T08:55:26.192 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40220.log 2024-09-13T08:55:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35176.log 2024-09-13T08:55:26.195 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51573.log 2024-09-13T08:55:26.195 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40220.log.gz 2024-09-13T08:55:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51989.log 2024-09-13T08:55:26.196 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35176.log.gz 2024-09-13T08:55:26.197 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38639.log 2024-09-13T08:55:26.197 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51573.log.gz 2024-09-13T08:55:26.198 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51989.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51989.log.gz 2024-09-13T08:55:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45365.log 2024-09-13T08:55:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38639.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38639.log.gz 2024-09-13T08:55:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47690.log 2024-09-13T08:55:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43399.log 2024-09-13T08:55:26.201 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45365.log.gz 2024-09-13T08:55:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47690.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47690.log.gz 2024-09-13T08:55:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43399.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43399.log.gz 2024-09-13T08:55:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40595.log 2024-09-13T08:55:26.205 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55352.log 2024-09-13T08:55:26.207 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53406.log 2024-09-13T08:55:26.207 INFO:teuthology.orchestra.run.smithi044.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40595.log.gz 2024-09-13T08:55:26.208 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55352.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55352.log.gz 2024-09-13T08:55:26.208 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52418.log 2024-09-13T08:55:26.209 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53406.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53406.log.gz 2024-09-13T08:55:26.209 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46918.log 2024-09-13T08:55:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39257.log 2024-09-13T08:55:26.211 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52418.log.gz 2024-09-13T08:55:26.211 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39179.log 2024-09-13T08:55:26.212 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.46918.log.gz 2024-09-13T08:55:26.213 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39257.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.39257.log.gz 2024-09-13T08:55:26.213 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38993.log 2024-09-13T08:55:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39179.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39179.log.gz 2024-09-13T08:55:26.214 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53479.log 2024-09-13T08:55:26.216 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41743.log 2024-09-13T08:55:26.216 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38993.log.gz 2024-09-13T08:55:26.217 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51297.log 2024-09-13T08:55:26.217 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53479.log.gz 2024-09-13T08:55:26.218 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40569.log 2024-09-13T08:55:26.218 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41743.log.gz 2024-09-13T08:55:26.220 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51297.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51297.log.gz 2024-09-13T08:55:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40569.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40569.log.gz 2024-09-13T08:55:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35371.log 2024-09-13T08:55:26.222 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39676.log 2024-09-13T08:55:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52256.log 2024-09-13T08:55:26.224 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35371.log.gz 2024-09-13T08:55:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39676.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39676.log.gz 2024-09-13T08:55:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42123.log 2024-09-13T08:55:26.226 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52256.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52256.log.gz 2024-09-13T08:55:26.226 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48271.log 2024-09-13T08:55:26.227 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39049.log 2024-09-13T08:55:26.228 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42123.log.gz 2024-09-13T08:55:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48271.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48271.log.gz 2024-09-13T08:55:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39049.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39049.log.gz 2024-09-13T08:55:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34818.log 2024-09-13T08:55:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50783.log 2024-09-13T08:55:26.233 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55037.log 2024-09-13T08:55:26.233 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34818.log.gz 2024-09-13T08:55:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37120.log 2024-09-13T08:55:26.234 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50783.log.gz 2024-09-13T08:55:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55037.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.55037.log.gz 2024-09-13T08:55:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37853.log 2024-09-13T08:55:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37120.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37120.log.gz 2024-09-13T08:55:26.237 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-13T08:55:26.238 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51826.log 2024-09-13T08:55:26.238 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37853.log.gz 2024-09-13T08:55:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.51826.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51826.log.gz 2024-09-13T08:55:26.241 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37424.log 2024-09-13T08:55:26.242 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43570.log 2024-09-13T08:55:26.243 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37424.log: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-13T08:55:26.243 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37424.log.gz 2024-09-13T08:55:26.244 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44296.log 2024-09-13T08:55:26.245 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43570.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43570.log.gz 2024-09-13T08:55:26.245 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52314.log 2024-09-13T08:55:26.247 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53583.log 2024-09-13T08:55:26.247 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44296.log.gz 2024-09-13T08:55:26.247 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52314.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52314.log.gz 2024-09-13T08:55:26.248 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53583.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53583.log.gz 2024-09-13T08:55:26.249 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43755.log 2024-09-13T08:55:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43959.log 2024-09-13T08:55:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49033.log 2024-09-13T08:55:26.252 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43755.log.gz 2024-09-13T08:55:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43959.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.43959.log.gz 2024-09-13T08:55:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49033.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49033.log.gz 2024-09-13T08:55:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55246.log 2024-09-13T08:55:26.255 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51059.log 2024-09-13T08:55:26.257 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45417.log 2024-09-13T08:55:26.257 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55246.log.gz 2024-09-13T08:55:26.258 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41387.log 2024-09-13T08:55:26.258 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.51059.log.gz 2024-09-13T08:55:26.259 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40990.log 2024-09-13T08:55:26.259 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45417.log.gz 2024-09-13T08:55:26.260 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.41387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41387.log.gz 2024-09-13T08:55:26.260 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40699.log 2024-09-13T08:55:26.261 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45612.log 2024-09-13T08:55:26.261 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40990.log.gz 2024-09-13T08:55:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40699.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.40699.log.gz 2024-09-13T08:55:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53961.log 2024-09-13T08:55:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45612.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45612.log.gz 2024-09-13T08:55:26.264 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35579.log 2024-09-13T08:55:26.265 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51521.log 2024-09-13T08:55:26.265 INFO:teuthology.orchestra.run.smithi044.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53961.log.gz 2024-09-13T08:55:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35579.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35579.log.gz 2024-09-13T08:55:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52833.log 2024-09-13T08:55:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51521.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51521.log.gz 2024-09-13T08:55:26.268 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44695.log 2024-09-13T08:55:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49085.log 2024-09-13T08:55:26.270 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52833.log.gz 2024-09-13T08:55:26.270 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44695.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44695.log.gz 2024-09-13T08:55:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49085.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49085.log.gz 2024-09-13T08:55:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36378.log 2024-09-13T08:55:26.273 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54525.log 2024-09-13T08:55:26.275 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37040.log 2024-09-13T08:55:26.275 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36378.log.gz 2024-09-13T08:55:26.276 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46177.log 2024-09-13T08:55:26.276 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54525.log.gz 2024-09-13T08:55:26.277 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35180.log 2024-09-13T08:55:26.277 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37040.log.gz 2024-09-13T08:55:26.278 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51547.log 2024-09-13T08:55:26.278 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46177.log.gz 2024-09-13T08:55:26.280 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35180.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35180.log.gz 2024-09-13T08:55:26.280 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53987.log 2024-09-13T08:55:26.281 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51547.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51547.log.gz 2024-09-13T08:55:26.281 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54749.log 2024-09-13T08:55:26.282 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44348.log 2024-09-13T08:55:26.283 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53987.log.gz 2024-09-13T08:55:26.284 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54749.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54749.log.gz 2024-09-13T08:55:26.285 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44348.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44348.log.gz 2024-09-13T08:55:26.285 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50432.log 2024-09-13T08:55:26.287 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41951.log 2024-09-13T08:55:26.288 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41159.log 2024-09-13T08:55:26.288 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50432.log.gz 2024-09-13T08:55:26.289 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41951.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.41951.log.gz 2024-09-13T08:55:26.290 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41159.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41159.log.gz 2024-09-13T08:55:26.291 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52593.log 2024-09-13T08:55:26.292 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38281.log 2024-09-13T08:55:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.52593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35572.log 2024-09-13T08:55:26.293 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52593.log.gz 2024-09-13T08:55:26.294 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52113.log 2024-09-13T08:55:26.295 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38281.log.gz 2024-09-13T08:55:26.296 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40933.log 2024-09-13T08:55:26.296 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35572.log.gz 2024-09-13T08:55:26.297 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52113.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52113.log.gz 2024-09-13T08:55:26.298 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.40933.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43243.log 2024-09-13T08:55:26.298 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40933.log.gz 2024-09-13T08:55:26.298 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54551.log 2024-09-13T08:55:26.300 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53453.log 2024-09-13T08:55:26.300 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43243.log.gz 2024-09-13T08:55:26.301 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54551.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54551.log.gz 2024-09-13T08:55:26.301 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46098.log 2024-09-13T08:55:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56430.log 2024-09-13T08:55:26.303 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53453.log.gz 2024-09-13T08:55:26.304 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42546.log 2024-09-13T08:55:26.304 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46098.log.gz 2024-09-13T08:55:26.305 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46018.log 2024-09-13T08:55:26.305 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.56430.log.gz 2024-09-13T08:55:26.306 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42546.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42546.log.gz 2024-09-13T08:55:26.307 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54069.log 2024-09-13T08:55:26.307 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46018.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46018.log.gz 2024-09-13T08:55:26.308 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52139.log 2024-09-13T08:55:26.309 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52392.log 2024-09-13T08:55:26.310 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54069.log.gz 2024-09-13T08:55:26.311 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52139.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52139.log.gz 2024-09-13T08:55:26.312 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52392.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52392.log.gz 2024-09-13T08:55:26.312 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49059.log 2024-09-13T08:55:26.313 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39101.log 2024-09-13T08:55:26.315 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47862.log 2024-09-13T08:55:26.315 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49059.log.gz 2024-09-13T08:55:26.316 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39101.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39101.log.gz 2024-09-13T08:55:26.317 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47862.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47862.log.gz 2024-09-13T08:55:26.317 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52546.log 2024-09-13T08:55:26.319 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54095.log 2024-09-13T08:55:26.320 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47194.log 2024-09-13T08:55:26.320 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52546.log.gz 2024-09-13T08:55:26.321 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47324.log 2024-09-13T08:55:26.321 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54095.log.gz 2024-09-13T08:55:26.322 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47194.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47194.log.gz 2024-09-13T08:55:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50458.log 2024-09-13T08:55:26.323 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47324.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47324.log.gz 2024-09-13T08:55:26.324 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46707.log 2024-09-13T08:55:26.326 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49795.log 2024-09-13T08:55:26.326 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50458.log.gz 2024-09-13T08:55:26.327 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46707.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.46707.log.gz 2024-09-13T08:55:26.327 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49743.log 2024-09-13T08:55:26.328 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28877.log 2024-09-13T08:55:26.328 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49795.log.gz 2024-09-13T08:55:26.330 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34538.log 2024-09-13T08:55:26.330 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49743.log.gz 2024-09-13T08:55:26.331 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.28877.log: 79.6% -- replaced with /var/log/ceph/ceph-client.admin.28877.log.gz 2024-09-13T08:55:26.331 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45806.log 2024-09-13T08:55:26.332 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34538.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34538.log.gz 2024-09-13T08:55:26.332 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39986.log 2024-09-13T08:55:26.334 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41717.log 2024-09-13T08:55:26.334 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45806.log.gz 2024-09-13T08:55:26.334 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39986.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39986.log.gz 2024-09-13T08:55:26.336 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41717.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41717.log.gz 2024-09-13T08:55:26.336 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48219.log 2024-09-13T08:55:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52282.log 2024-09-13T08:55:26.339 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54855.log 2024-09-13T08:55:26.339 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48219.log.gz 2024-09-13T08:55:26.339 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42783.log 2024-09-13T08:55:26.340 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52282.log.gz 2024-09-13T08:55:26.341 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45664.log 2024-09-13T08:55:26.341 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.54855.log.gz 2024-09-13T08:55:26.342 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42783.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42783.log.gz 2024-09-13T08:55:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.45664.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49232.log 2024-09-13T08:55:26.343 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45664.log.gz 2024-09-13T08:55:26.344 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53791.log 2024-09-13T08:55:26.345 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50536.log 2024-09-13T08:55:26.346 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.49232.log.gz 2024-09-13T08:55:26.346 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53791.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53791.log.gz 2024-09-13T08:55:26.347 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55116.log 2024-09-13T08:55:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50536.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50536.log.gz 2024-09-13T08:55:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52366.log 2024-09-13T08:55:26.349 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47272.log 2024-09-13T08:55:26.349 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55116.log.gz 2024-09-13T08:55:26.350 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54421.log 2024-09-13T08:55:26.350 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52366.log.gz 2024-09-13T08:55:26.352 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53023.log 2024-09-13T08:55:26.352 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.47272.log.gz 2024-09-13T08:55:26.352 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54421.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54421.log.gz 2024-09-13T08:55:26.353 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43295.log 2024-09-13T08:55:26.354 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54959.log 2024-09-13T08:55:26.354 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53023.log.gz 2024-09-13T08:55:26.355 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34606.log 2024-09-13T08:55:26.356 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43295.log.gz 2024-09-13T08:55:26.356 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54959.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54959.log.gz 2024-09-13T08:55:26.358 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34606.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34606.log.gz 2024-09-13T08:55:26.358 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49547.log 2024-09-13T08:55:26.359 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52917.log 2024-09-13T08:55:26.361 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35104.log 2024-09-13T08:55:26.361 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49547.log.gz 2024-09-13T08:55:26.361 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35594.log 2024-09-13T08:55:26.361 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52917.log.gz 2024-09-13T08:55:26.362 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56118.log 2024-09-13T08:55:26.363 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35104.log.gz 2024-09-13T08:55:26.364 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51930.log 2024-09-13T08:55:26.364 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35594.log.gz 2024-09-13T08:55:26.365 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40282.log 2024-09-13T08:55:26.365 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56118.log.gz 2024-09-13T08:55:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51930.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51930.log.gz 2024-09-13T08:55:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43100.log 2024-09-13T08:55:26.367 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40282.log.gz 2024-09-13T08:55:26.368 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47992.log 2024-09-13T08:55:26.370 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52691.log 2024-09-13T08:55:26.370 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43100.log.gz 2024-09-13T08:55:26.371 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47992.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47992.log.gz 2024-09-13T08:55:26.371 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55406.log 2024-09-13T08:55:26.372 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52691.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52691.log.gz 2024-09-13T08:55:26.372 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54173.log 2024-09-13T08:55:26.374 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40342.log 2024-09-13T08:55:26.374 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55406.log.gz 2024-09-13T08:55:26.375 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55588.log 2024-09-13T08:55:26.375 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54173.log.gz 2024-09-13T08:55:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34792.log 2024-09-13T08:55:26.376 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40342.log.gz 2024-09-13T08:55:26.377 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47664.log 2024-09-13T08:55:26.377 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55588.log.gz 2024-09-13T08:55:26.379 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34792.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34792.log.gz 2024-09-13T08:55:26.379 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38083.log 2024-09-13T08:55:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47664.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47664.log.gz 2024-09-13T08:55:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54199.log 2024-09-13T08:55:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50289.log 2024-09-13T08:55:26.382 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38083.log.gz 2024-09-13T08:55:26.382 INFO:teuthology.orchestra.run.smithi129.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-13T08:55:26.383 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54199.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54199.log.gz 2024-09-13T08:55:26.384 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50289.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50289.log.gz 2024-09-13T08:55:26.385 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36736.log 2024-09-13T08:55:26.386 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43373.log 2024-09-13T08:55:26.387 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52444.log 2024-09-13T08:55:26.387 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36736.log.gz 2024-09-13T08:55:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43373.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43373.log.gz 2024-09-13T08:55:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52444.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52444.log.gz 2024-09-13T08:55:26.390 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43872.log 2024-09-13T08:55:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38850.log 2024-09-13T08:55:26.392 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37014.log 2024-09-13T08:55:26.392 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43872.log.gz 2024-09-13T08:55:26.393 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39231.log 2024-09-13T08:55:26.394 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38850.log.gz 2024-09-13T08:55:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37014.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37014.log.gz 2024-09-13T08:55:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41459.log 2024-09-13T08:55:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39231.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39231.log.gz 2024-09-13T08:55:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2024-09-13T08:55:26.398 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41459.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41459.log.gz 2024-09-13T08:55:26.398 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47534.log 2024-09-13T08:55:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53101.log 2024-09-13T08:55:26.399 INFO:teuthology.orchestra.run.smithi044.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2024-09-13T08:55:26.400 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55734.log 2024-09-13T08:55:26.400 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47534.log.gz 2024-09-13T08:55:26.402 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53101.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53101.log.gz 2024-09-13T08:55:26.402 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37198.log 2024-09-13T08:55:26.402 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55734.log.gz 2024-09-13T08:55:26.403 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51800.log 2024-09-13T08:55:26.404 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55220.log 2024-09-13T08:55:26.405 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37198.log.gz 2024-09-13T08:55:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51800.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51800.log.gz 2024-09-13T08:55:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55220.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55220.log.gz 2024-09-13T08:55:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49492.log 2024-09-13T08:55:26.408 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45231.log 2024-09-13T08:55:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47041.log 2024-09-13T08:55:26.410 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49492.log.gz 2024-09-13T08:55:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35780.log 2024-09-13T08:55:26.411 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45231.log.gz 2024-09-13T08:55:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36892.log 2024-09-13T08:55:26.412 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.47041.log.gz 2024-09-13T08:55:26.413 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35780.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35780.log.gz 2024-09-13T08:55:26.413 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48141.log 2024-09-13T08:55:26.414 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53127.log 2024-09-13T08:55:26.414 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36892.log.gz 2024-09-13T08:55:26.416 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48141.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48141.log.gz 2024-09-13T08:55:26.417 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53127.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53127.log.gz 2024-09-13T08:55:26.417 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51748.log 2024-09-13T08:55:26.418 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50588.log 2024-09-13T08:55:26.419 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40142.log 2024-09-13T08:55:26.420 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51748.log.gz 2024-09-13T08:55:26.421 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50588.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50588.log.gz 2024-09-13T08:55:26.422 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40142.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.40142.log.gz 2024-09-13T08:55:26.422 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38255.log 2024-09-13T08:55:26.423 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36482.log 2024-09-13T08:55:26.425 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47482.log 2024-09-13T08:55:26.425 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38255.log.gz 2024-09-13T08:55:26.426 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36482.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-13T08:55:26.426 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36482.log.gz 2024-09-13T08:55:26.427 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47482.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47482.log.gz 2024-09-13T08:55:26.428 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34505.log 2024-09-13T08:55:26.429 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49944.log 2024-09-13T08:55:26.430 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34505.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34505.log.gz 2024-09-13T08:55:26.431 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40543.log 2024-09-13T08:55:26.432 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49944.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49944.log.gz 2024-09-13T08:55:26.432 INFO:teuthology.orchestra.run.smithi044.stderr: 90.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-13T08:55:26.433 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40543.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40543.log.gz 2024-09-13T08:55:26.434 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48628.log 2024-09-13T08:55:26.435 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52470.log 2024-09-13T08:55:26.437 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48628.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-13T08:55:26.437 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48628.log.gz 2024-09-13T08:55:26.437 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54577.log 2024-09-13T08:55:26.437 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52470.log.gz 2024-09-13T08:55:26.438 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38109.log 2024-09-13T08:55:26.440 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54577.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54577.log.gz 2024-09-13T08:55:26.440 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39958.log 2024-09-13T08:55:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38109.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38109.log.gz 2024-09-13T08:55:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38057.log 2024-09-13T08:55:26.443 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39958.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.39958.log.gz 2024-09-13T08:55:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41392.log 2024-09-13T08:55:26.444 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38057.log.gz 2024-09-13T08:55:26.445 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50861.log 2024-09-13T08:55:26.446 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.41392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41392.log.gz 2024-09-13T08:55:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44468.log 2024-09-13T08:55:26.448 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50861.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50861.log.gz 2024-09-13T08:55:26.449 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44468.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44468.log.gz 2024-09-13T08:55:26.450 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-09-13T08:55:26.451 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41485.log 2024-09-13T08:55:26.453 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49892.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gz 2024-09-13T08:55:26.453 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41485.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41485.log.gz 2024-09-13T08:55:26.454 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2024-09-13T08:55:26.455 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-13T08:55:26.457 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47508.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2024-09-13T08:55:26.459 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53380.log 2024-09-13T08:55:26.460 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29042.log 2024-09-13T08:55:26.461 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53380.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53380.log.gz 2024-09-13T08:55:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36430.log 2024-09-13T08:55:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39478.log 2024-09-13T08:55:26.464 INFO:teuthology.orchestra.run.smithi044.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.29042.log.gz 2024-09-13T08:55:26.465 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52717.log 2024-09-13T08:55:26.465 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36430.log.gz 2024-09-13T08:55:26.467 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39478.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39478.log.gz 2024-09-13T08:55:26.470 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52717.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52717.log.gz 2024-09-13T08:55:26.470 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55864.log 2024-09-13T08:55:26.473 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55864.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55864.log.gz 2024-09-13T08:55:26.473 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48786.log 2024-09-13T08:55:26.478 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48786.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48786.log.gz 2024-09-13T08:55:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42045.log 2024-09-13T08:55:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42045.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42045.log.gz 2024-09-13T08:55:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47350.log 2024-09-13T08:55:26.482 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47350.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47350.log.gz 2024-09-13T08:55:26.482 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38535.log 2024-09-13T08:55:26.485 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38535.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38535.log.gz 2024-09-13T08:55:26.485 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47015.log 2024-09-13T08:55:26.488 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53726.log 2024-09-13T08:55:26.489 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47015.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47015.log.gz 2024-09-13T08:55:26.489 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43675.log 2024-09-13T08:55:26.489 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53726.log.gz 2024-09-13T08:55:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43675.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.43675.log.gz 2024-09-13T08:55:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46963.log 2024-09-13T08:55:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46963.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46963.log.gz 2024-09-13T08:55:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51323.log 2024-09-13T08:55:26.497 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46461.log 2024-09-13T08:55:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51323.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51323.log.gz 2024-09-13T08:55:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52997.log 2024-09-13T08:55:26.501 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46461.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.46461.log.gz 2024-09-13T08:55:26.504 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52997.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52997.log.gz 2024-09-13T08:55:26.504 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34766.log 2024-09-13T08:55:26.506 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34766.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34766.log.gz 2024-09-13T08:55:26.507 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55812.log 2024-09-13T08:55:26.509 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55812.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55812.log.gz 2024-09-13T08:55:26.509 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44572.log 2024-09-13T08:55:26.512 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44572.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44572.log.gz 2024-09-13T08:55:26.513 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54829.log 2024-09-13T08:55:26.515 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54829.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54829.log.gz 2024-09-13T08:55:26.515 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35858.log 2024-09-13T08:55:26.518 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56092.log 2024-09-13T08:55:26.518 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35858.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35858.log.gz 2024-09-13T08:55:26.521 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56092.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56092.log.gz 2024-09-13T08:55:26.521 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49769.log 2024-09-13T08:55:26.524 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44374.log 2024-09-13T08:55:26.524 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49769.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.49769.log.gz 2024-09-13T08:55:26.525 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43347.log 2024-09-13T08:55:26.529 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44374.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44374.log.gz 2024-09-13T08:55:26.529 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43347.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43347.log.gz 2024-09-13T08:55:26.529 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42611.log 2024-09-13T08:55:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42611.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42611.log.gz 2024-09-13T08:55:26.533 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45287.log 2024-09-13T08:55:26.535 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45287.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45287.log.gz 2024-09-13T08:55:26.536 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47428.log 2024-09-13T08:55:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41237.log 2024-09-13T08:55:26.539 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47428.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47428.log.gz 2024-09-13T08:55:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41237.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41237.log.gz 2024-09-13T08:55:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44088.log 2024-09-13T08:55:26.544 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44088.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44088.log.gz 2024-09-13T08:55:26.544 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49918.log 2024-09-13T08:55:26.547 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49440.log 2024-09-13T08:55:26.547 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49918.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49918.log.gz 2024-09-13T08:55:26.548 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48654.log 2024-09-13T08:55:26.550 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49440.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49440.log.gz 2024-09-13T08:55:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28983.log 2024-09-13T08:55:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48654.log: /var/log/ceph/ceph-client.admin.28983.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28983.log.gz 2024-09-13T08:55:26.555 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48654.log.gz 2024-09-13T08:55:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51111.log 2024-09-13T08:55:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51111.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51111.log.gz 2024-09-13T08:55:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55909.log 2024-09-13T08:55:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55909.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55909.log.gz 2024-09-13T08:55:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51163.log 2024-09-13T08:55:26.564 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51163.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51163.log.gz 2024-09-13T08:55:26.564 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47742.log 2024-09-13T08:55:26.565 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43022.log 2024-09-13T08:55:26.567 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47742.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47742.log.gz 2024-09-13T08:55:26.570 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43022.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43022.log.gz 2024-09-13T08:55:26.570 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50924.log 2024-09-13T08:55:26.571 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46592.log 2024-09-13T08:55:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50924.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50924.log.gz 2024-09-13T08:55:26.576 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46592.log: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.46592.log.gz 2024-09-13T08:55:26.576 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44721.log 2024-09-13T08:55:26.579 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44721.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44721.log.gz 2024-09-13T08:55:26.579 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39127.log 2024-09-13T08:55:26.582 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54406.log 2024-09-13T08:55:26.582 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39127.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39127.log.gz 2024-09-13T08:55:26.585 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.54406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54406.log.gz 2024-09-13T08:55:26.585 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52087.log 2024-09-13T08:55:26.588 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44598.log 2024-09-13T08:55:26.588 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52087.log.gz 2024-09-13T08:55:26.589 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50180.log 2024-09-13T08:55:26.591 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44598.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44598.log.gz 2024-09-13T08:55:26.593 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54933.log 2024-09-13T08:55:26.594 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50180.log.gz 2024-09-13T08:55:26.597 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.54933.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.54933.log.gz 2024-09-13T08:55:26.597 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39608.log 2024-09-13T08:55:26.600 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39608.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39608.log.gz 2024-09-13T08:55:26.600 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44970.log 2024-09-13T08:55:26.606 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44970.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44970.log.gz 2024-09-13T08:55:26.606 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55682.log 2024-09-13T08:55:26.606 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55682.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55682.log.gz 2024-09-13T08:55:26.606 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37315.log 2024-09-13T08:55:26.610 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37315.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37315.log.gz 2024-09-13T08:55:26.610 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39335.log 2024-09-13T08:55:26.614 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34948.log 2024-09-13T08:55:26.614 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39335.log: /var/log/ceph/ceph-client.admin.34948.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.39335.log.gz 2024-09-13T08:55:26.614 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34948.log.gz 2024-09-13T08:55:26.614 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42201.log 2024-09-13T08:55:26.617 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42201.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42201.log.gz 2024-09-13T08:55:26.617 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29056.log 2024-09-13T08:55:26.620 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29056.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.29056.log.gz 2024-09-13T08:55:26.620 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44494.log 2024-09-13T08:55:26.623 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44494.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44494.log.gz 2024-09-13T08:55:26.623 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42722.log 2024-09-13T08:55:26.627 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46518.log 2024-09-13T08:55:26.627 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42722.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42722.log.gz 2024-09-13T08:55:26.627 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41077.log 2024-09-13T08:55:26.627 INFO:teuthology.orchestra.run.smithi044.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.46518.log.gz 2024-09-13T08:55:26.630 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41077.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41077.log.gz 2024-09-13T08:55:26.631 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39432.log 2024-09-13T08:55:26.634 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.39432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47836.log 2024-09-13T08:55:26.634 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39432.log.gz 2024-09-13T08:55:26.635 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36814.log 2024-09-13T08:55:26.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47836.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47836.log.gz 2024-09-13T08:55:26.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36814.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36814.log.gz 2024-09-13T08:55:26.643 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37775.log 2024-09-13T08:55:26.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37775.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37775.log.gz 2024-09-13T08:55:26.643 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38483.log 2024-09-13T08:55:26.644 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38483.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38483.log.gz 2024-09-13T08:55:26.644 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40673.log 2024-09-13T08:55:26.647 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42325.log 2024-09-13T08:55:26.648 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40673.log.gz 2024-09-13T08:55:26.649 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46363.log 2024-09-13T08:55:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42325.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42325.log.gz 2024-09-13T08:55:26.652 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35595.log 2024-09-13T08:55:26.654 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46363.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.46363.log.gz 2024-09-13T08:55:26.656 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39556.log 2024-09-13T08:55:26.657 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35595.log.gz 2024-09-13T08:55:26.658 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51852.log 2024-09-13T08:55:26.660 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39556.log: -- /var/log/ceph/ceph-client.admin.40394.log 2024-09-13T08:55:26.661 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51852.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39556.log.gz 2024-09-13T08:55:26.663 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36404.log 2024-09-13T08:55:26.667 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36404.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36404.log.gz 2024-09-13T08:55:26.667 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40394.log: -- /var/log/ceph/ceph-client.admin.42891.log 2024-09-13T08:55:26.668 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51852.log.gz 2024-09-13T08:55:26.668 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40394.log.gz 2024-09-13T08:55:26.671 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42891.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47560.log 2024-09-13T08:55:26.671 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42891.log.gz 2024-09-13T08:55:26.673 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43597.log 2024-09-13T08:55:26.674 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47560.log.gz 2024-09-13T08:55:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56378.log 2024-09-13T08:55:26.678 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43597.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.43597.log.gz 2024-09-13T08:55:26.681 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56378.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56378.log.gz 2024-09-13T08:55:26.681 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37341.log 2024-09-13T08:55:26.684 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37341.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37341.log.gz 2024-09-13T08:55:26.684 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54147.log 2024-09-13T08:55:26.687 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54147.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54147.log.gz 2024-09-13T08:55:26.687 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45862.log 2024-09-13T08:55:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45862.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45862.log.gz 2024-09-13T08:55:26.690 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41433.log 2024-09-13T08:55:26.693 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41433.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41433.log.gz 2024-09-13T08:55:26.693 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56352.log 2024-09-13T08:55:26.696 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56352.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56352.log.gz 2024-09-13T08:55:26.696 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29004.log 2024-09-13T08:55:26.699 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29004.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29004.log.gz 2024-09-13T08:55:26.699 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37671.log 2024-09-13T08:55:26.700 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38161.log 2024-09-13T08:55:26.703 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37671.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37671.log.gz 2024-09-13T08:55:26.703 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39019.log 2024-09-13T08:55:26.704 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38161.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38161.log.gz 2024-09-13T08:55:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39019.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39019.log.gz 2024-09-13T08:55:26.707 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39732.log 2024-09-13T08:55:26.709 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36866.log 2024-09-13T08:55:26.710 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39732.log.gz 2024-09-13T08:55:26.713 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52167.log 2024-09-13T08:55:26.717 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52167.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52167.log.gz 2024-09-13T08:55:26.717 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36866.log.gz 2024-09-13T08:55:26.717 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48598.log 2024-09-13T08:55:26.720 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37749.log 2024-09-13T08:55:26.720 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48598.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48598.log.gz 2024-09-13T08:55:26.722 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45261.log 2024-09-13T08:55:26.726 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45261.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45261.log.gz 2024-09-13T08:55:26.726 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37749.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37749.log.gz 2024-09-13T08:55:26.726 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37502.log 2024-09-13T08:55:26.730 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37502.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37502.log.gz 2024-09-13T08:55:26.730 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50705.log 2024-09-13T08:55:26.733 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50705.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50705.log.gz 2024-09-13T08:55:26.733 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47612.log 2024-09-13T08:55:26.735 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37289.log 2024-09-13T08:55:26.736 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47612.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47612.log.gz 2024-09-13T08:55:26.738 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37289.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37289.log.gz 2024-09-13T08:55:26.738 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43701.log 2024-09-13T08:55:26.741 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43701.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43701.log.gz 2024-09-13T08:55:26.741 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35372.log 2024-09-13T08:55:26.744 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35372.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35372.log.gz 2024-09-13T08:55:26.745 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39075.log 2024-09-13T08:55:26.747 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53531.log 2024-09-13T08:55:26.747 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39075.log.gz 2024-09-13T08:55:26.748 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48454.log 2024-09-13T08:55:26.750 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53531.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53531.log.gz 2024-09-13T08:55:26.753 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48454.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48454.log.gz 2024-09-13T08:55:26.753 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41511.log 2024-09-13T08:55:26.757 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41511.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41511.log.gz 2024-09-13T08:55:26.757 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29053.log 2024-09-13T08:55:26.759 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35166.log 2024-09-13T08:55:26.760 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29053.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.29053.log.gz 2024-09-13T08:55:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35166.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35166.log.gz 2024-09-13T08:55:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48680.log 2024-09-13T08:55:26.765 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48680.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48680.log.gz 2024-09-13T08:55:26.765 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48706.log 2024-09-13T08:55:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28947.log 2024-09-13T08:55:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48706.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48706.log.gz 2024-09-13T08:55:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.28947.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28947.log.gz 2024-09-13T08:55:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38386.log 2024-09-13T08:55:26.778 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38386.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38386.log.gz 2024-09-13T08:55:26.778 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44546.log 2024-09-13T08:55:26.778 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44546.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44546.log.gz 2024-09-13T08:55:26.778 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46203.log 2024-09-13T08:55:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46203.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.46203.log.gz 2024-09-13T08:55:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42514.log 2024-09-13T08:55:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42351.log 2024-09-13T08:55:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42514.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42514.log.gz 2024-09-13T08:55:26.787 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42351.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42351.log.gz 2024-09-13T08:55:26.787 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35611.log 2024-09-13T08:55:26.790 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35611.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35611.log.gz 2024-09-13T08:55:26.790 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29013.log 2024-09-13T08:55:26.793 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.29013.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29013.log.gz 2024-09-13T08:55:26.793 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53250.log 2024-09-13T08:55:26.796 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53250.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53250.log.gz 2024-09-13T08:55:26.796 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39530.log 2024-09-13T08:55:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39530.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39530.log.gz 2024-09-13T08:55:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34870.log 2024-09-13T08:55:26.802 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45339.log 2024-09-13T08:55:26.802 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.34870.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34870.log.gz 2024-09-13T08:55:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45339.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45339.log.gz 2024-09-13T08:55:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36988.log 2024-09-13T08:55:26.808 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36988.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36988.log.gz 2024-09-13T08:55:26.808 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51217.log 2024-09-13T08:55:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46682.log 2024-09-13T08:55:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51217.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51217.log.gz 2024-09-13T08:55:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28853.log 84.1% -- replaced with /var/log/ceph/ceph-client.admin.46682.log.gz 2024-09-13T08:55:26.812 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-09-13T08:55:26.815 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.28853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28853.log.gz 2024-09-13T08:55:26.815 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45035.log 2024-09-13T08:55:26.822 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45035.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45035.log.gz 2024-09-13T08:55:26.822 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28889.log 2024-09-13T08:55:26.828 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.28889.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.28889.log.gz 2024-09-13T08:55:26.828 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41821.log 2024-09-13T08:55:26.828 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41821.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41821.log.gz 2024-09-13T08:55:26.828 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38031.log 2024-09-13T08:55:26.831 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38031.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38031.log.gz 2024-09-13T08:55:26.831 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52859.log 2024-09-13T08:55:26.834 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52859.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52859.log.gz 2024-09-13T08:55:26.834 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44014.log 2024-09-13T08:55:26.837 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55300.log 2024-09-13T08:55:26.837 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44014.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.44014.log.gz 2024-09-13T08:55:26.840 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55300.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55300.log.gz 2024-09-13T08:55:26.840 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42692.log 2024-09-13T08:55:26.843 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42692.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42692.log.gz 2024-09-13T08:55:26.843 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41980.log 2024-09-13T08:55:26.847 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54354.log 2024-09-13T08:55:26.847 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41980.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41980.log.gz 2024-09-13T08:55:26.847 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40038.log 2024-09-13T08:55:26.848 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54354.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54354.log.gz 2024-09-13T08:55:26.849 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35884.log 2024-09-13T08:55:26.850 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40038.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40038.log.gz 2024-09-13T08:55:26.852 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35884.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35884.log.gz 2024-09-13T08:55:26.852 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46866.log 2024-09-13T08:55:26.855 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46866.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46866.log.gz 2024-09-13T08:55:26.855 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48981.log 2024-09-13T08:55:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48981.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48981.log.gz 2024-09-13T08:55:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48760.log 2024-09-13T08:55:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53661.log 2024-09-13T08:55:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48760.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48760.log.gz 2024-09-13T08:55:26.864 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53661.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53661.log.gz 2024-09-13T08:55:26.864 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49639.log 2024-09-13T08:55:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49639.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49639.log.gz 2024-09-13T08:55:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53872.log 2024-09-13T08:55:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47771.log 2024-09-13T08:55:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53872.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53872.log.gz 2024-09-13T08:55:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47771.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47771.log.gz 2024-09-13T08:55:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51189.log 2024-09-13T08:55:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37604.log 2024-09-13T08:55:26.876 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.51189.log.gz 2024-09-13T08:55:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48955.log 2024-09-13T08:55:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37604.log: /var/log/ceph/ceph-client.admin.48955.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2024-09-13T08:55:26.881 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37604.log.gz 2024-09-13T08:55:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40064.log 2024-09-13T08:55:26.884 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40064.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40064.log.gz 2024-09-13T08:55:26.884 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48245.log 2024-09-13T08:55:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48245.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48245.log.gz 2024-09-13T08:55:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38223.log 2024-09-13T08:55:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38223.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38223.log.gz 2024-09-13T08:55:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28865.log 2024-09-13T08:55:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.28865.log: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.28865.log.gz 2024-09-13T08:55:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42403.log 2024-09-13T08:55:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44218.log 2024-09-13T08:55:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42403.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42403.log.gz 2024-09-13T08:55:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36542.log 2024-09-13T08:55:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44218.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44218.log.gz 2024-09-13T08:55:26.900 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50350.log 2024-09-13T08:55:26.901 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36542.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36542.log.gz 2024-09-13T08:55:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50350.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50350.log.gz 2024-09-13T08:55:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43176.log 2024-09-13T08:55:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43176.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43176.log.gz 2024-09-13T08:55:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35832.log 2024-09-13T08:55:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35832.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35832.log.gz 2024-09-13T08:55:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47090.log 2024-09-13T08:55:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47090.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47090.log.gz 2024-09-13T08:55:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48304.log 2024-09-13T08:55:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37450.log 2024-09-13T08:55:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48304.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48304.log.gz 2024-09-13T08:55:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36710.log 2024-09-13T08:55:26.920 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37450.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37450.log.gz 2024-09-13T08:55:26.920 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54401.log 2024-09-13T08:55:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.36710.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36710.log.gz 2024-09-13T08:55:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.54401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54401.log.gz 2024-09-13T08:55:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43649.log 2024-09-13T08:55:26.927 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43649.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.43649.log.gz 2024-09-13T08:55:26.927 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55935.log 2024-09-13T08:55:26.930 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55935.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55935.log.gz 2024-09-13T08:55:26.930 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55708.log 2024-09-13T08:55:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46734.log 2024-09-13T08:55:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55708.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55708.log.gz 2024-09-13T08:55:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46734.log: 83.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38691.log 2024-09-13T08:55:26.935 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46734.log.gz 2024-09-13T08:55:26.936 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36141.log 2024-09-13T08:55:26.938 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38691.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38691.log.gz 2024-09-13T08:55:26.940 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.36141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36141.log.gz 2024-09-13T08:55:26.941 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54252.log 2024-09-13T08:55:26.944 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42637.log 2024-09-13T08:55:26.945 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54252.log.gz 2024-09-13T08:55:26.946 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45443.log 2024-09-13T08:55:26.948 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42637.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42637.log.gz 2024-09-13T08:55:26.951 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45540.log/var/log/ceph/ceph-client.admin.45443.log: 2024-09-13T08:55:26.952 INFO:teuthology.orchestra.run.smithi044.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45443.log.gz 2024-09-13T08:55:26.954 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52743.log 2024-09-13T08:55:26.958 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52743.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52743.log.gz 2024-09-13T08:55:26.959 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.45540.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45540.log.gz 2024-09-13T08:55:26.959 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56508.log 2024-09-13T08:55:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56508.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56508.log.gz 2024-09-13T08:55:26.962 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46814.log 2024-09-13T08:55:26.964 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45391.log 2024-09-13T08:55:26.965 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46814.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.46814.log.gz 2024-09-13T08:55:26.967 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45391.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45391.log.gz 2024-09-13T08:55:26.967 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35392.log 2024-09-13T08:55:26.970 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.35392.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35392.log.gz 2024-09-13T08:55:26.970 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42663.log 2024-09-13T08:55:26.973 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51456.log 2024-09-13T08:55:26.973 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42663.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42663.log.gz 2024-09-13T08:55:26.976 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51456.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51456.log.gz 2024-09-13T08:55:26.976 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54723.log 2024-09-13T08:55:26.979 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54723.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54723.log.gz 2024-09-13T08:55:26.979 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42272.log 2024-09-13T08:55:26.982 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42272.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42272.log.gz 2024-09-13T08:55:26.982 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50484.log 2024-09-13T08:55:26.984 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49840.log 2024-09-13T08:55:26.985 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50484.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50484.log.gz 2024-09-13T08:55:26.988 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49840.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49840.log.gz 2024-09-13T08:55:26.988 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51651.log 2024-09-13T08:55:26.991 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51651.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51651.log.gz 2024-09-13T08:55:26.991 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37723.log 2024-09-13T08:55:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37723.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37723.log.gz 2024-09-13T08:55:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53049.log 2024-09-13T08:55:26.997 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53049.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53049.log.gz 2024-09-13T08:55:26.997 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54278.log 2024-09-13T08:55:27.000 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54278.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54278.log.gz 2024-09-13T08:55:27.000 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56404.log 2024-09-13T08:55:27.003 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56404.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56404.log.gz 2024-09-13T08:55:27.003 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51427.log 2024-09-13T08:55:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51427.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51427.log.gz 2024-09-13T08:55:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48063.log 2024-09-13T08:55:27.007 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42429.log 2024-09-13T08:55:27.009 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48063.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48063.log.gz 2024-09-13T08:55:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42429.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42429.log.gz 2024-09-13T08:55:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2024-09-13T08:55:27.015 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40962.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gz 2024-09-13T08:55:27.015 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41537.log 2024-09-13T08:55:27.017 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41537.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41537.log.gz 2024-09-13T08:55:27.018 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38360.log 2024-09-13T08:55:27.021 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38360.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38360.log.gz 2024-09-13T08:55:27.021 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46892.log 2024-09-13T08:55:27.024 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46892.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46892.log.gz 2024-09-13T08:55:27.024 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40621.log 2024-09-13T08:55:27.027 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40816.log 2024-09-13T08:55:27.027 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40621.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40621.log.gz 2024-09-13T08:55:27.030 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40816.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40816.log.gz 2024-09-13T08:55:27.030 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47716.log 2024-09-13T08:55:27.035 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43074.log 2024-09-13T08:55:27.035 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47716.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47716.log.gz 2024-09-13T08:55:27.036 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43074.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35806.log 2024-09-13T08:55:27.036 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43074.log.gz 2024-09-13T08:55:27.038 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38135.log 2024-09-13T08:55:27.039 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41659.log 2024-09-13T08:55:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38135.log: /var/log/ceph/ceph-client.admin.35806.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35806.log.gz 2024-09-13T08:55:27.043 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44244.log 2024-09-13T08:55:27.044 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38135.log.gz 2024-09-13T08:55:27.044 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41659.log.gz 2024-09-13T08:55:27.045 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47116.log 2024-09-13T08:55:27.048 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44244.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44244.log.gz 2024-09-13T08:55:27.049 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46255.log 2024-09-13T08:55:27.050 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47116.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47116.log.gz 2024-09-13T08:55:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46255.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46255.log.gz 2024-09-13T08:55:27.229 INFO:teuthology.orchestra.run.smithi044.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-13T08:55:29.279 INFO:teuthology.orchestra.run.smithi129.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-13T08:55:29.480 INFO:teuthology.orchestra.run.smithi129.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-13T08:55:29.529 INFO:teuthology.orchestra.run.smithi129.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-13T08:55:29.592 INFO:teuthology.orchestra.run.smithi044.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-13T08:55:29.971 INFO:teuthology.orchestra.run.smithi044.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-13T08:55:30.121 INFO:teuthology.orchestra.run.smithi044.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-13T08:55:30.666 INFO:teuthology.orchestra.run.smithi129.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-13T08:55:31.067 INFO:teuthology.orchestra.run.smithi044.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-13T08:55:31.132 INFO:teuthology.orchestra.run.smithi044.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-13T08:55:32.057 INFO:teuthology.orchestra.run.smithi129.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-13T08:55:32.059 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-13T08:55:32.059 INFO:teuthology.orchestra.run.smithi129.stderr:real 0m6.323s 2024-09-13T08:55:32.059 INFO:teuthology.orchestra.run.smithi129.stderr:user 0m21.411s 2024-09-13T08:55:32.059 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m1.857s 2024-09-13T08:55:33.100 INFO:teuthology.orchestra.run.smithi044.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-13T08:55:33.102 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-09-13T08:55:33.102 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m7.385s 2024-09-13T08:55:33.102 INFO:teuthology.orchestra.run.smithi044.stderr:user 0m28.167s 2024-09-13T08:55:33.102 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m3.613s 2024-09-13T08:55:33.103 INFO:tasks.ceph:Archiving logs... 2024-09-13T08:55:33.103 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319/remote/smithi044/log 2024-09-13T08:55:33.104 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T08:55:36.606 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319/remote/smithi129/log 2024-09-13T08:55:36.606 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T08:55:39.041 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T08:55:39.049 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T08:55:39.050 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T08:55:39.053 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T08:55:39.138 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T08:55:39.146 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T08:55:39.147 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T08:55:39.150 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T08:55:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T08:55:39.163 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================== 2024-09-13T08:55:39.163 INFO:teuthology.orchestra.run.smithi044.stdout:+hv01.front.sepi 67.205.162.81 3 u 65 64 377 0.069 -4.217 0.769 2024-09-13T08:55:39.163 INFO:teuthology.orchestra.run.smithi044.stdout:+hv02.front.sepi 63.231.80.2 3 u 4 64 377 0.090 -2.897 0.426 2024-09-13T08:55:39.163 INFO:teuthology.orchestra.run.smithi044.stdout:*hv03.front.sepi 74.6.168.72 3 u 62 64 377 0.079 -2.222 1.047 2024-09-13T08:55:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-13T08:55:39.165 INFO:teuthology.orchestra.run.smithi129.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T08:55:39.165 INFO:teuthology.orchestra.run.smithi129.stdout:============================================================================== 2024-09-13T08:55:39.165 INFO:teuthology.orchestra.run.smithi129.stdout:+hv01.front.sepi 67.205.162.81 3 u 3 64 377 0.074 -5.390 0.422 2024-09-13T08:55:39.165 INFO:teuthology.orchestra.run.smithi129.stdout:*hv02.front.sepi 63.231.80.2 3 u 2 64 377 0.092 -4.668 0.595 2024-09-13T08:55:39.165 INFO:teuthology.orchestra.run.smithi129.stdout:+hv03.front.sepi 74.6.168.72 3 u 59 64 377 0.084 -4.825 0.711 2024-09-13T08:55:39.165 INFO:teuthology.orchestra.run.smithi129.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-13T08:55:39.166 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T08:55:39.262 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T08:55:39.263 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T08:55:39.299 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T08:55:39.340 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T08:55:39.382 INFO:teuthology.task.internal:Duration was 1825.413513 seconds 2024-09-13T08:55:39.383 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T08:55:39.451 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T08:55:39.452 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T08:55:39.455 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T08:55:39.557 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T08:55:39.557 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T08:55:39.558 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T08:55:39.579 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T08:55:39.580 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T08:55:39.600 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T08:55:39.600 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T08:55:39.626 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T08:55:39.685 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T08:55:39.685 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T08:55:39.717 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T08:55:39.807 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T08:55:39.818 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T08:55:39.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T08:55:39.839 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T08:55:39.858 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T08:55:39.942 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T08:55:39.946 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T08:55:39.960 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-09-13T08:55:39.963 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-09-13T08:55:39.980 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T08:55:40.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:55:40.033 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T08:55:40.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:55:40.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T08:55:40.051 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T08:55:40.052 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319/remote/smithi044 2024-09-13T08:55:40.052 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T08:55:40.102 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2024-09-13_08:07:46-fs:volumes-wip-athakkar-testing-2024-09-12-1849-distro-default-smithi/7903319/remote/smithi129 2024-09-13T08:55:40.102 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T08:55:40.128 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T08:55:40.128 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T08:55:40.141 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T08:55:40.178 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T08:55:40.191 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T08:55:40.191 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T08:55:40.320 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T08:55:40.321 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T08:55:40.324 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T08:55:40.329 INFO:teuthology.orchestra.run.smithi044.stdout: 658143 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 08:55 /home/ubuntu/cephtest 2024-09-13T08:55:40.333 INFO:teuthology.orchestra.run.smithi129.stdout: 658143 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 08:55 /home/ubuntu/cephtest 2024-09-13T08:55:40.335 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T08:55:40.373 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T08:55:40.458 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} duration: 1825.4135129451752 flavor: default owner: scheduled_khiremat@teuthology success: true 2024-09-13T08:55:40.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T08:55:40.541 INFO:teuthology.run:pass